{"677469":{"#nid":"677469","#data":{"type":"event","title":"PhD Proposal by  Naoki Yokoyama","body":[{"value":"\u003Cp\u003EDate: Thursday, October 24th 2024\u003C\/p\u003E\u003Cp\u003ETime: 5:00 PM \u2013 7:00 PM EST\u0026nbsp; \/ 2:00 PM \u2013 4:00 PM PST\u003C\/p\u003E\u003Cp\u003ELocation: Zoom (\u003Ca href=\u0022https:\/\/gatech.zoom.us\/j\/5825218212?pwd=NnBMcmNDTlFoNVcxTC91dndacFRadz09\u0022 title=\u0022https:\/\/gatech.zoom.us\/j\/5825218212?pwd=NnBMcmNDTlFoNVcxTC91dndacFRadz09\u0022\u003Ehttps:\/\/gatech.zoom.us\/j\/5825218212?pwd=NnBMcmNDTlFoNVcxTC91dndacFRadz09\u003C\/a\u003E)\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ECommittee:\u003C\/p\u003E\u003Cp\u003EDr. Sehoon Ha (Advisor) \u2013 School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Dhruv Batra (Advisor) \u2013 School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Jie Tan \u2013 School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Vladlen Koltun \u2013 Distinguished Scientist, Apple\u003C\/p\u003E\u003Cp\u003EDr. Mrinal Kalakrishnan \u2013 Research Lead, Meta\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ETitle:\u003C\/p\u003E\u003Cp\u003EFrom Web to World: Harnessing Foundation Models for Intelligent Robotic Assistants in Real-World Environments\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EAbstract:\u003C\/p\u003E\u003Cp\u003EIn this thesis, we explore how simulated embodied experience and spatial grounding can be leveraged to \u0027embody\u0027 foundation models for robotics, bridging the gap between their abstract reasoning capabilities and the physical realities of robotic interaction. We present three key contributions: (1) Adaptive Skill Coordination (ASC) and Language-guided Skill Coordination (LSC), approaches for open-vocabulary long-horizon mobile manipulation tasks that demonstrate how simulators can be used to develop fundamental sensorimotor skills, creating a robust \u0027body\u0027 of capabilities that foundation models can employ to interact with the real world. (2) Vision-Language Frontier Maps (VLFM), an approach that combines pre-trained vision-language models with low-level navigation policies trained in simulation. By grounding pre-trained vision-language models with explicit spatial maps of the environment, VLFM enhances their ability to reason about and navigate in the real world. (3) A proposed approach to fine-tune vision-language models using simulated data to enhance their spatial-temporal reasoning for navigation tasks. By exposing these models to diverse simulated scenarios, we hypothesize they will develop a more nuanced understanding of physical interactions, causality, and temporal dynamics. This research aims to create embodied AI systems that can leverage the strengths of foundation models while effectively operating in real-world environments.\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EFrom Web to World: Harnessing Foundation Models for Intelligent Robotic Assistants in Real-World Environments\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"From Web to World: Harnessing Foundation Models for Intelligent Robotic Assistants in Real-World Environments"}],"uid":"27707","created_gmt":"2024-10-10 21:01:28","changed_gmt":"2024-10-10 21:02:15","author":"Tatianna Richardson","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2024-10-24T17:00:00-04:00","event_time_end":"2024-10-24T19:00:12-04:00","event_time_end_last":"2024-10-24T19:00:12-04:00","gmt_time_start":"2024-10-24 21:00:00","gmt_time_end":"2024-10-24 23:00:12","gmt_time_end_last":"2024-10-24 23:00:12","rrule":null,"timezone":"America\/New_York"},"location":"Zoom","extras":[],"groups":[{"id":"221981","name":"Graduate Studies"}],"categories":[],"keywords":[{"id":"102851","name":"Phd proposal"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1788","name":"Other\/Miscellaneous"}],"invited_audience":[{"id":"78771","name":"Public"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}