{"615231":{"#nid":"615231","#data":{"type":"event","title":"PhD Proposal by Abhishek Das","body":[{"value":"\u003Cp\u003E\u003Cstrong\u003ETitle\u003C\/strong\u003E: Building Agents That Can See, Talk, and Act\u003C\/p\u003E\r\n\r\n\u003Cp\u003E----------------\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003EDate\u003C\/strong\u003E: Monday, December 17th, 2018\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ETime\u003C\/strong\u003E: 11:00am to 12:30pm (ET)\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ELocation\u003C\/strong\u003E: CCB 247\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003EBluejeans\u003C\/strong\u003E:\u0026nbsp;\u003Ca href=\u0022https:\/\/bluejeans.com\/6506900315\u0022\u003Ehttps:\/\/bluejeans.com\/6506900315\u003C\/a\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EAbhishek Das\u003C\/p\u003E\r\n\r\n\u003Cp\u003EComputer Science Ph.D. Student\u003C\/p\u003E\r\n\r\n\u003Cp\u003ESchool of Interactive Computing\u003C\/p\u003E\r\n\r\n\u003Cp\u003EGeorgia Institute of Technology\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Ca href=\u0022https:\/\/abhishekdas.com\/\u0022\u003Eabhishekdas.com\u003C\/a\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003ECommittee:\u003C\/p\u003E\r\n\r\n\u003Cp\u003E----------------\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Dhruv Batra (Advisor; School of Interactive Computing, Georgia Institute of Technology \u0026amp; Facebook AI Research)\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Devi Parikh (School of Interactive Computing, Georgia Institute of Technology \u0026amp; Facebook AI Research)\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Joelle Pineau (McGill University \u0026amp; Facebook AI Research, Montr\u0026eacute;al)\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. James Hays (School of Interactive Computing, Georgia Institute of Technology \u0026amp; Argo AI)\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Jitendra Malik (University of California, Berkeley \u0026amp; Facebook AI Research)\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EAbstract:\u003C\/p\u003E\r\n\r\n\u003Cp\u003E----------------\u003C\/p\u003E\r\n\r\n\u003Cp\u003EMy research goal is to build intelligent agents (the next generation of Cortana, Alexa, Siri, etc.) that possess the ability to perceive the rich visual environment around us, communicate this understanding in natural language to humans and other agents, and execute actions in a physical environment. Even a small advance towards such agents can fundamentally change our lives \u0026ndash; from assistive chatbots for the visually impaired, to natural language interaction with self-driving cars and in-home mobile robots!\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003ETowards this grand goal, in this thesis, I will present my work on\u003C\/p\u003E\r\n\r\n\u003Cp\u003E1) visual dialog (see+talk) \u0026mdash; agents capable of holding free-form conversations about images and reinforcement learning-based algorithms to train these visual dialog agents without exhaustively collecting human-annotated datasets (via self-play),\u003C\/p\u003E\r\n\r\n\u003Cp\u003E2) embodied question answering (see+talk+act) \u0026mdash; agents with hierarchical and modular navigation architectures that can move around, actively perceive, and answer questions in simulated environments,\u003C\/p\u003E\r\n\r\n\u003Cp\u003E3) targeted multi-agent communication (multi-agent see+talk+act) \u0026mdash; agents that can communicate with each other in a targeted manner for cooperative tasks, such that they learn both what messages to send and who to communicate with, solely from downstream task-specific reward without any communication supervision.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EIn proposed work, I will develop architectures for multi-agent embodied question answering, where the goal is to answer complex 3D visual reasoning questions (such as \u0026quot;What size is the cylinder that is left of the brown metal thing that is left of the big sphere?\u0026quot;) by appropriately combining first-person active perception with navigation actions and inter-agent communication.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E----------------\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":"","field_summary_sentence":[{"value":"Building Agents That Can See, Talk, and Act"}],"uid":"27707","created_gmt":"2018-12-11 20:59:35","changed_gmt":"2018-12-11 20:59:35","author":"Tatianna Richardson","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2018-12-17T11:00:00-05:00","event_time_end":"2018-12-17T12:30:00-05:00","event_time_end_last":"2018-12-17T12:30:00-05:00","gmt_time_start":"2018-12-17 16:00:00","gmt_time_end":"2018-12-17 17:30:00","gmt_time_end_last":"2018-12-17 17:30:00","rrule":null,"timezone":"America\/New_York"},"extras":[],"groups":[{"id":"221981","name":"Graduate Studies"}],"categories":[],"keywords":[{"id":"102851","name":"Phd proposal"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1788","name":"Other\/Miscellaneous"}],"invited_audience":[{"id":"78761","name":"Faculty\/Staff"},{"id":"78771","name":"Public"},{"id":"174045","name":"Graduate students"},{"id":"78751","name":"Undergraduate students"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}