Location: CODA C1215 Midtown
Zoom: https://gatech.zoom.us/j/93076373347?pwd=mLt44wxlZxwbAFpQANiNgRnfnWLdfx.1

Karmesh Yadav
Ph.D. Student
School of Interactive Computing
Georgia Institute of Technology

Committee members
Dr. Zsolt Kira (advisor): School of Interactive Computing, Georgia Institute of Technology
Dr. Dhruv Batra (advisor): School of Interactive Computing, Georgia Institute of Technology
Dr. James Hays: School of Interactive Computing, Georgia Institute of Technology
Dr. Animesh Garg: School of Interactive Computing, Georgia Institute of Technology

Abstract
Vision is one of the primary interfaces between intelligence and the real world. While modern foundation models achieve strong results on many vision-language tasks, reliable multimodal intelligence requires visual systems that go beyond recognizing or describing isolated inputs: they must support interaction, memory, and grounded reasoning.

In this thesis, I study visual foundation models across perception, memory, and reasoning. First, I analyze how pretrained visual representations transfer to embodied tasks such as navigation and manipulation, identifying how architecture, data, scale, and adaptation affect policy learning. Next, I introduce FindingDory, a benchmark for evaluating whether embodied agents can remember spatial, semantic, and temporal information from prior experience and use it for future navigation decisions. Finally, I propose to study faithful visual grounding in multimodal reasoning by developing reward signals that encourage models to make claims and decisions supported by visual input. Together, these works argue that robust visual foundation models require stronger visual signals across perception, memory, and reasoning, laying groundwork for systems that can act, adapt, and reason reliably in realistic environments.