PhD Defense by Yen-Chang Hsu

Event Details
  • Date/Time:
    • Monday March 23, 2020 - Tuesday March 24, 2020
      2:00 pm - 3:59 pm
  • Location: Coda C1115 Druid Hills
  • Phone:
  • URL: BlueJeans Link
  • Email:
  • Fee(s):
  • Extras:
No contact information submitted.

Summary Sentence: Learning with Pairwise Similarity for Visual Categorization

Full Summary: No summary paragraph submitted.

Title: Learning with Pairwise Similarity for Visual Categorization

Yen-Chang Hsu

Robotics Ph.D. Candidate
School of Electrical and Computer Engineering
Georgia Institute of Technology


Date: Monday, March 23th, 2020
Time: 2:00 pm to 4:00 pm (ET)
Location: Coda C1115 Druid Hills



Dr. Zsolt Kira, CoC, Advisor
Dr. Patricio Vela, ECE
Dr. Dhruv Batra, CoC
Dr. Judy Hoffman, CoC
Dr. Phillip Odom, GTRI



Learning high-capacity machine learning models for perception, especially for high-dimensional inputs such as in computer vision, requires a large amount of human-annotated data. Many efforts have been made to construct such large-scale annotated datasets. However, there are not many options for transferring knowledge from those datasets to another task with different categories, limiting the value of these efforts. While one common option for transfer is reusing a learned feature representation, other options for reusing supervision across tasks are generally not considered due to the tight association between labels and tasks. This thesis proposes to use an intermediate form of supervision, pairwise similarity, for enabling the transferability of supervision across different categorization tasks that have different sets of classes. We show that pairwise similarity, defined as whether two pieces of data have the same semantic meaning or not, is sufficient as the primary supervision for learning categorization problems such as clustering and classification.


We investigate this idea by answering two transfer learning questions, which are how and when to transfer. We develop two loss functions for answering how to transfer and show the same framework can support supervised, unsupervised, and semi-supervised learning paradigms, demonstrating better performance over previous methods. This result makes discovering unseen categories in unlabeled data possible by transferring a learned pairwise similarity prediction function. Additionally, we provide a decomposed confidence strategy for answering when to transfer, achieving state-of-the-art results on out-of-distribution data detection. Lastly, we apply our loss function to the application of instance segmentation, demonstrating the scalability of our method in utilizing pairwise similarity within a real-world problem.

Additional Information

In Campus Calendar

Graduate Studies

Invited Audience
Faculty/Staff, Public, Graduate students, Undergraduate students
Phd Defense
  • Created By: Tatianna Richardson
  • Workflow Status: Published
  • Created On: Mar 10, 2020 - 1:08pm
  • Last Updated: Mar 17, 2020 - 8:39am