{"693484":{"#nid":"693484","#data":{"type":"event","title":"PhD Defense by Selim Furkan Tekin","body":[{"value":"\u003Cp\u003EDear faculty members and fellow students,\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EYou are cordially invited to my PhD thesis defense on Monday, November 23rd.\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ETitle:\u0026nbsp;\u003C\/strong\u003EAdaptive and Robust Model Fusion for Multi-Modal Learning Systems\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EDate:\u0026nbsp;\u003C\/strong\u003EMonday,\u003Cstrong\u003E\u0026nbsp;\u003C\/strong\u003ENovember 23, 2026\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ETime:\u0026nbsp;\u003C\/strong\u003E1:00 PM - 3:00 PM Eastern Time (US)\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ELocation:\u0026nbsp;\u003C\/strong\u003EKACB 3402\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EVirtual (Zoom):\u0026nbsp;\u003C\/strong\u003E\u003Ca href=\u0022https:\/\/nam12.safelinks.protection.outlook.com\/?url=https%3A%2F%2Fgatech.zoom.us%2Fj%2F94537626884%3Fpwd%3Df2aDUdkbH2u8p6w9oLP0zMBdv6CJCN.1\u0026amp;data=05%7C02%7Cannouncements%40grad.gatech.edu%7Cd2b568c3ba464970d91008df330be8ae%7C482198bbae7b4b258b7a6d7f32faa083%7C1%7C0%7C639285798403191466%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C\u0026amp;sdata=W3V%2Bt%2BoccpOE1%2Fld8p07z75M4XqgVQb3329fvEkxfv0%3D\u0026amp;reserved=0\u0022\u003Ehttps:\/\/gatech.zoom.us\/j\/94537626884?pwd=f2aDUdkbH2u8p6w9oLP0zMBdv6CJCN.1\u003C\/a\u003E\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ESelim\u0026nbsp;Furkan\u0026nbsp;Tekin\u003C\/strong\u003E\u003Cbr\u003EPh.D. Candidate in Computer Science\u003Cbr\u003ESchool of Computer Science\u003Cbr\u003EGeorgia Institute of Technology\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ECommittee members:\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003E1. Dr. Ling Liu (Advisor) \u0026nbsp;- School of Computer Science, Georgia Institute of Technology\u003Cbr\u003E2. Dr. Francisco Romero - School of Computer Science, Georgia Institute of Technology\u003Cbr\u003E3. Dr. Anand Padmanabha Iyer - School of Computer Science, Georgia Institute of Technology\u003Cbr\u003E4. Dr. Calton Pu - School of Computer Science, Georgia Institute of Technology\u003Cbr\u003E5. Dr. Margaret Loper - Information and Communications Laboratory, Georgia Tech Research Institute\u003Cbr\u003E6. Dr. Lakshmish Ramaswamy - School of Computing, University of Georgia\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EAbstract\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EThe advancement of large-scale AI models in recent years has shown their remarkable performance across a broad spectrum of zero- and one-shot tasks across language and vision modalities. However, based on the benchmark studies and model release announcements revealed that even the newest released or largest models tend to carry some weaknesses despite the unique strengths, indicating no one model can be the persistent winner in generalization performance across all tasks. Complementing this, user-generated queries tend to have varying complexity and involve simple common knowledge tasks or complex reasoning or specific domain knowledge. For simple queries involving superficial understanding, smaller models can do a perfect job at a minimal cost. Hence, there is a challenge in adequately choosing a model or group of models based on the client-user constraints and the downstream learning tasks at hand. This dissertation is dedicated to tackling the above grand challenges by designing and developing innovative methodologies, algorithms, and optimizations in selecting, refining, and integrating multiple gen-AI models.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EOur research began with a comprehensive empirical study, which led to three observations: First, in classical few-shot learning methods, ensemble gains come largely from two channels: structural (model architecture) and geometric (distance metric) diversity\u2013motivating heterogeneous model selection. We later found that the same holds for modern few-shot learners such as LLMs and VLMs. Second, larger heterogeneous ensemble teams may not necessarily offer better generalization performance, because some members make redundant predictions that add inference cost without adding new information. Third, the language models and vision models require different multi-stage representation learning to effectively capture duplicated learning capabilities. Motivated by these observations, we have made four novel and fundamental contributions:\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cem\u003EFirst\u003C\/em\u003E, we have developed a novel diversity computation framework that can be tailored for both language and vision representation learning to capture the disagreement correlation among multiple gen-AI models (LLMs, LVMs, and VLMs).\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cem\u003ESecond\u003C\/em\u003E, we have designed the first of its kind diversity-optimized ensemble pruning framework, which supports LLM and VLM ensemble pruning. The pruning utilizes genetic search optimization on introduced metrics: the focal-diversity metric and focal Central Kernel Alignment (CKA) to identify informative complementary representations for vision-language ensemble pruning.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cem\u003EThird\u003C\/em\u003E, we study learning-to-combine architectures and develop two diversity-optimized ensemble fusion methods. The first is a supervised approach that uses diversity-driven pruning to select an ensemble team offline and then learns to fuse its outputs, achieving stronger generalization than state-of-the-art methods. The second selects the team and fuses outputs dynamically at inference time, using a two-stage reinforcement learning framework.\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cem\u003EFinally\u003C\/em\u003E, we address model routing and fusion through joint optimization, learning how to \u003Cem\u003Eselect\u003C\/em\u003E, \u003Cem\u003Ereason\u003C\/em\u003E, and \u003Cem\u003Erefine\u003C\/em\u003E\u0026nbsp;across multiple VLMs. We develop two routing systems trained with reinforcement learning. The first starts from prior beliefs about each VLM\u2019s performance: a routing policy chooses which model to consult, and a fusion policy decides whether to keep consulting or answer. The second targets physical reasoning in real-world scenes: it matches the skills a query requires to each VLM\u2019s perception strengths, routes to models in sequence, and refines a lightweight policy using critiques from a tool-augmented critic that extracts visual clues. In both lines of work, we compare our approaches against widely used AI chatbots and show that our approaches succeed in real-world scenarios where existing models often fail.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ECollectively, these contributions offer new ways to build teams of diverse AI models that reason together, combine their outputs, and verify their answers. Such methods become increasingly important as the number and variety of large-scale models continue to grow.\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EAdaptive and Robust Model Fusion for Multi-Modal Learning Systems\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"Adaptive and Robust Model Fusion for Multi-Modal Learning Systems"}],"uid":"27707","created_gmt":"2026-10-26 16:55:35","changed_gmt":"2026-10-26 16:56:02","author":"Tatianna Richardson","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2026-11-23T13:00:53-05:00","event_time_end":"2026-11-23T15:00:00-05:00","event_time_end_last":"2026-11-23T15:00:00-05:00","gmt_time_start":"2026-11-23 18:00:53","gmt_time_end":"2026-11-23 20:00:00","gmt_time_end_last":"2026-11-23 20:00:00","rrule":null,"timezone":"America\/New_York"},"location":"KACB 3402","extras":[],"groups":[{"id":"221981","name":"Graduate Studies"}],"categories":[],"keywords":[{"id":"100811","name":"Phd Defense"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1788","name":"Other\/Miscellaneous"}],"invited_audience":[{"id":"78771","name":"Public"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}