{"614606":{"#nid":"614606","#data":{"type":"event","title":"PhD Proposal by Jianwei Yang","body":[{"value":"\u003Cp\u003E\u003Cstrong\u003ETitle: Modeling Structure for Visual Understanding and Generation\u003C\/strong\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003EDate\u003C\/strong\u003E: Thursday, November 29, 2018\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ETime\u003C\/strong\u003E: 11:30AM - 1:00PM (ET)\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ELocation\u003C\/strong\u003E: TSRB 223\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EJianwei Yang\u003C\/p\u003E\r\n\r\n\u003Cp\u003EPh.D. Student in Computer Science\u003C\/p\u003E\r\n\r\n\u003Cp\u003ESchool of Interactive Computing\u003C\/p\u003E\r\n\r\n\u003Cp\u003EGeorgia Institute of Technology\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Ca href=\u0022https:\/\/www.cc.gatech.edu\/~jyang375\/\u0022\u003Ehttps:\/\/www.cc.gatech.edu\/~jyang375\/\u003C\/a\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ECommittee:\u003C\/strong\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Devi Parikh (Advisor, School of Interactive Computing, Georgia Institute of Technology)\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Dhruv Batra (School of Interactive Computing, Georgia Institute of Technology)\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. David J. Crandall (School of Informatics, Computing, and Engineering, Indiana University)\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Stefan Lee (School of Interactive Computing, Georgia Institute of Technology)\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003EAbstract\u003C\/strong\u003E:\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThe world around us is highly structured. Objects interact with each other in predictable ways (e.g., mugs are often on tables, keyboards are often below computer monitors, the sky is in the background, grass is often green). This structure manifests itself in the visual data that captures the world around us, and in text that describes it. In this thesis, the goal is to leverage this structure in our visual world for visual understanding and its dual problem visual generation, both with and without interactions with language. Specifically, this thesis work makes the following contributions.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EOn visual understanding side:\u003C\/p\u003E\r\n\r\n\u003Cp\u003E1)\u0026nbsp; Proposed an effective approach for scene graph generation, that learns to compute the relationship-ness between objects and prune the dense graph accordingly before performing graph labeling.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E2)\u0026nbsp; Proposed a language-based meta-active-learning framework for an agent, that can learn to ask informative questions to the human\/oracle based on a structured representation of the scene, and then learn its visual recognition models incrementally.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EOn visual generation side:\u003C\/p\u003E\r\n\r\n\u003Cp\u003E1)\u0026nbsp; Proposed a new model for generating images by considering the layer-by-layer structure in images, that generates image background and foreground step-by-step and compose them into a single image with proper spatial configuration.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E2)\u0026nbsp; In the proposed work, I will further leverage the layer-by-layer structure in images and text for visual generation conditioned on language. Specifically, given a description of images, the model learns to extract the structure in the sentence, and then generate the image layer-by-layer accordingly so that the generated images are consistent with the given description.\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":"","field_summary_sentence":[{"value":"Modeling Structure for Visual Understanding and Generation"}],"uid":"27707","created_gmt":"2018-11-26 20:14:21","changed_gmt":"2018-11-26 20:14:21","author":"Tatianna Richardson","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2018-11-29T11:30:00-05:00","event_time_end":"2018-11-29T13:30:00-05:00","event_time_end_last":"2018-11-29T13:30:00-05:00","gmt_time_start":"2018-11-29 16:30:00","gmt_time_end":"2018-11-29 18:30:00","gmt_time_end_last":"2018-11-29 18:30:00","rrule":null,"timezone":"America\/New_York"},"extras":[],"groups":[{"id":"221981","name":"Graduate Studies"}],"categories":[],"keywords":[{"id":"102851","name":"Phd proposal"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1788","name":"Other\/Miscellaneous"}],"invited_audience":[{"id":"78761","name":"Faculty\/Staff"},{"id":"78771","name":"Public"},{"id":"174045","name":"Graduate students"},{"id":"78751","name":"Undergraduate students"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}