Phd Defense by Jianwei Yang

Event Details
  • Date/Time:
    • Thursday January 2, 2020 - Friday January 3, 2020
      4:00 pm - 5:59 pm
  • Location: Coda C1003 Adair
  • Phone:
  • URL:
  • Email:
  • Fee(s):
  • Extras:
No contact information submitted.

Summary Sentence: Structured Visual Understanding, Generation and Reasoning

Full Summary: No summary paragraph submitted.

Title: Structured Visual Understanding, Generation and Reasoning


Jianwei Yang

Ph.D. Candidate in Computer Science

School of Interactive Computing

Georgia Institute of Technology


Date: Thursday, January 2nd, 2020

Time: 4:00-6:00 PM (EST)

Location: Coda C1003 Adair





Dr. Devi Parikh (Advisor), School of Interactive Computing, Georgia Institute of Technology

Dr. Dhruv Batra, School of Interactive Computing, Georgia Institute of Technology

Dr. David Crandall, School of Informatics, Computing and Engineering, Indiana University

Dr. Stefan Lee, School of Electrical  Engineering and Computer Science, Oregon State University

Dr. Judy Hoffman, School of Interactive Computing, Georgia Institute of Technology




The world around us is highly structured. In the real world, multiple objects usually exist in a scene and interact with each other in predictable ways (e.g., mug on table, keyboard below computer monitor); for a single object, it usually consists of multiple components under some structured configurations (e.g., a person has different body parts). These structures manifest themselves in the visual data that captures the world around us, and thus can potentially provide a strong inductive bias to various vision tasks. In this talk, I will discuss how to integrate such structure priors into different tasks including visual understanding, generation and reasoning. Specifically,


  1. I will talk about how to integrate the structure prior into visual system to improve its visual understanding ability at different levels. Particularly, I will talk about how we leverage the relational sparsity and context in images for a better scene graph generation;
  2. I will show how we can exploit the structure in images to address the dual problem, visual generation. Specifically, I will explain how we can generate images in a compositional manner by generating background and foreground objects separately and compose them together;
  3. I will discuss how to use the structured semantic visual representations as the interface to bridge visual perception and reasoning to address vision and language tasks. Specifically, I will present a meta-active-learning framework in which the agent reasonings on symbolic scene graph to generate informative questions to ask oracle to improve its visual system incrementally.


On these different levels of tasks, we demonstrate that modeling the structures in visual data and the associated text can not only improve the model performance but also increase the model transparency. To the end, I will briefly discuss the challenges in this domain and the extensions of recent works.

Additional Information

In Campus Calendar

Graduate Studies

Invited Audience
Faculty/Staff, Public, Graduate students, Undergraduate students
Phd Defense
  • Created By: Tatianna Richardson
  • Workflow Status: Published
  • Created On: Jan 2, 2020 - 1:54pm
  • Last Updated: Jan 2, 2020 - 1:54pm