Shanmukha Ramakrishna Vedantam

Ph.D. Student

School of Interactive Computing

College of Computing

Georgia Institute of Technology

Date: Wednesday, October 24, 2018

Time: 5:30PM - 7:30PM (EDT)

Location: College of Computing Building (CCB) Room 312A

Committee:

Dr. Devi Parikh (Advisor, School of Interactive Computing,
Georgia Institute of Technology)

Dr. Dhruv Batra (School of Interactive Computing, Georgia
Institute of Technology)

Dr. Jacob Eisenstein (School of Interactive Computing, Georgia
Institute of Technology)

Dr. Kevin P. Murphy (Research Scientist, Google Research)

Dr. C. Lawrence Zitnick (Research Manager, Facebook AI Research)

Abstract:

Understanding how to model computer vision and natural language
jointly is a long-standing challenge in artificial intelligence. In this
thesis, I study how modeling vision and language using semantic and pragmatic
considerations can help derive more human-like inferences from machine learning
models. Specifically, I consider three related problems: interpretation,
grounding, and imagination.

In interpretation, the goal is to get machine learning models to
understand an image and describe its contents using natural language in a
contextually relevant manner. In grounding, I study how to connect natural
language to referents in the physical world, and understand if this can help
learn common sense. Finally, in imagination, I study how to ‘imagine’ visual
concepts completely and accurately across the full range and (potentially
unseen) compositions of their visual attributes. I will analyze these problems
from computational as well as algorithmic perspectives and suggest exciting
directions for future work.