{"681612":{"#nid":"681612","#data":{"type":"event","title":"PhD Defense by Naoki H Yokoyama","body":[{"value":"\u003Cp\u003EDate: Friday, April 18th 2025\u003C\/p\u003E\u003Cp\u003ETime: 1:00 PM \u2013 3:00 PM EST\u003C\/p\u003E\u003Cp\u003ELocation: Zoom (\u003Ca href=\u0022https:\/\/gatech.zoom.us\/j\/5825218212?pwd=NnBMcmNDTlFoNVcxTC91dndacFRadz09\u0022 title=\u0022https:\/\/gatech.zoom.us\/j\/5825218212?pwd=NnBMcmNDTlFoNVcxTC91dndacFRadz09\u0022\u003Ehttps:\/\/gatech.zoom.us\/j\/5825218212?pwd=NnBMcmNDTlFoNVcxTC91dndacFRadz09\u003C\/a\u003E)\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ECommittee:\u003C\/p\u003E\u003Cp\u003EDr. Sehoon Ha (Advisor) \u2013 School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Dhruv Batra (Advisor) \u2013 School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Jie Tan \u2013 School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Vladlen Koltun \u2013 Distinguished Scientist, Apple\u003C\/p\u003E\u003Cp\u003EDr. Mrinal Kalakrishnan \u2013 Research Lead, Meta\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ETitle:\u003C\/p\u003E\u003Cp\u003EFrom Web to World: Harnessing Foundation Models for Intelligent Robotic Assistants in Real-World Environments\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EAbstract:\u003C\/p\u003E\u003Cp\u003EIn this dissertation, we explore how simulated embodied experience and spatial grounding can enhance foundation models for robotics, bridging the gap between abstract reasoning capabilities and physical robotic interaction. We present three key contributions:\u003C\/p\u003E\u003Cp\u003E(1) Adaptive Skill Coordination (ASC) and Language-guided Skill Coordination (LSC): These approaches address open-vocabulary long-horizon mobile manipulation tasks, demonstrating how simulators can develop fundamental sensorimotor skills. This creates a robust repertoire of capabilities that foundation models can employ for real-world interaction.\u003C\/p\u003E\u003Cp\u003E(2) Vision-Language Frontier Maps (VLFM): This approach combines pre-trained vision-language models with low-level navigation policies trained in simulation. By grounding these models with explicit spatial maps of the environment, VLFM enhances their ability to reason about and navigate in the real world.\u003C\/p\u003E\u003Cp\u003E(3) A novel method for fine-tuning multi-modal large language models using simulated data: This approach enables models to develop reasoning capabilities beyond semantic understanding for navigation tasks. By fine-tuning with diverse simulated scenarios, we demonstrate that models leverage knowledge from both their pre-training on web-scale data and navigation training to achieve superior navigation performance.\u003C\/p\u003E\u003Cp\u003EThis research aims to create embodied AI systems that leverage the strengths of foundation models while effectively operating in real-world environments.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EFrom Web to World: Harnessing Foundation Models for Intelligent Robotic Assistants in Real-World Environments\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"From Web to World: Harnessing Foundation Models for Intelligent Robotic Assistants in Real-World Environments"}],"uid":"27707","created_gmt":"2025-04-04 19:41:16","changed_gmt":"2025-04-04 19:41:50","author":"Tatianna Richardson","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2025-04-18T13:00:00-04:00","event_time_end":"2025-04-18T15:00:00-04:00","event_time_end_last":"2025-04-18T15:00:00-04:00","gmt_time_start":"2025-04-18 17:00:00","gmt_time_end":"2025-04-18 19:00:00","gmt_time_end_last":"2025-04-18 19:00:00","rrule":null,"timezone":"America\/New_York"},"location":"ZOOM","extras":[],"groups":[{"id":"221981","name":"Graduate Studies"}],"categories":[],"keywords":[{"id":"100811","name":"Phd Defense"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1788","name":"Other\/Miscellaneous"}],"invited_audience":[{"id":"78771","name":"Public"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}