Title: Harnessing Synthetic Data for Robust and Reliable Vision

Date: July 2nd, 2024

Time: 2:30pm - 4:30pm ET

Location: CODA C1115 Druid Hills

Zoom Link: https://gatech.zoom.us/j/96316092795

Prithvijit Chattopadhyay

Computer Science PhD Student

Interactive Computing

Georgia Institute of Technology

Committee

1. Judy Hoffman (Advisor, IC, GT)
2. Dhruv Batra (IC, GT)
3. James Hays (IC, GT)
4. Animesh Garg (IC, GT)
5. Roozbeh Mottaghi (Meta AI)

Abstract

Progress in computer vision has been driven by models trained on large amounts of exemplar data for different tasks. These exemplar data sources intend to capture task-specific information and instance-level variations that a trained model will likely encounter in the wild. However, for conditions where curating lots of labeled real-world data is prohibitively expensive, synthetic data can serve as a cost-effective alternative. Synthetic data sources offer a few key benefits: fast access to labeled task-specific data at scale, labels across varying task complexities, and curation of labeled data across diverse conditions in a controlled manner.

In this thesis defense, I will focus on how “controlled variations” in synthetic data can be used to develop robust and reliable vision models. Controlled variations refer to intentional, systematic modifications to synthetic data, designed to either explore specific aspects of model behavior or improve model transfer across distributions. First, I will discuss RobustNav and SkyScenes, which apply controlled variations internally at the data-engine (simulator) stage to create diverse instances to systematically investigate the robustness of trained vision models. Next, I will discuss PASTA and AUGCAL, which apply controlled variations externally as data augmentations, guided by observed distributional discrepancies between synthetic and real data, to develop robust and reliable models.