{"691037":{"#nid":"691037","#data":{"type":"event","title":"PhD Defense by Yao Dou","body":[{"value":"\u003Cp\u003E\u003Cstrong\u003ETitle:\u0026nbsp;\u003C\/strong\u003EScalable and Structured Evaluation of Large Language Models\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EDate:\u0026nbsp;\u003C\/strong\u003EFriday, July 17th 2026\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ETime:\u0026nbsp;\u003C\/strong\u003E11:00 - 12:30 PM EST\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ELocation:\u003C\/strong\u003E\u0026nbsp;Coda C1115 Druid Hills\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EZoom:\u0026nbsp;\u003C\/strong\u003E\u003Ca href=\u0022https:\/\/nam12.safelinks.protection.outlook.com\/?url=https%3A%2F%2Fgatech.zoom.us%2Fj%2F2123115504%3Fpwd%3DeTJFZWR1dXN6ZlZ3WGtwRlMzQmZNQT09\u0026amp;data=05%7C02%7Ctm186%40gtvault.onmicrosoft.com%7Ce1dd71bfbcf64586f66d08ded982fd13%7C482198bbae7b4b258b7a6d7f32faa083%7C1%7C0%7C639187353799818774%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C\u0026amp;sdata=FfEurrGHbGSxbX7iLBG8js1Jqeh%2B9afGDIOIPTdksDE%3D\u0026amp;reserved=0\u0022 target=\u0022_blank\u0022\u003Ehttps:\/\/gatech.zoom.us\/j\/2123115504?pwd=eTJFZWR1dXN6ZlZ3WGtwRlMzQmZNQT09\u003C\/a\u003E\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EYao Dou\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EPh.D. Student\u003C\/p\u003E\u003Cp\u003ESchool of Interactive Computing\u003C\/p\u003E\u003Cp\u003EGeorgia Institute of Technology\u003C\/p\u003E\u003Cp\u003E\u003Ca href=\u0022https:\/\/nam12.safelinks.protection.outlook.com\/?url=https%3A%2F%2Fyao-dou.github.io%2F\u0026amp;data=05%7C02%7Ctm186%40gtvault.onmicrosoft.com%7Ce1dd71bfbcf64586f66d08ded982fd13%7C482198bbae7b4b258b7a6d7f32faa083%7C1%7C0%7C639187353800020623%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C\u0026amp;sdata=%2BCc7UU7FpoDuiXxPgjRVbm5JSQ2Wl2d%2BMImqWmvoSoE%3D\u0026amp;reserved=0\u0022\u003Ehttps:\/\/yao-dou.github.io\/\u003C\/a\u003E\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ECommittee members\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EDr. Wei Xu (advisor): School of Interactive\u0026nbsp;Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Alan Ritter: School of Interactive\u0026nbsp;Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Polo Chau: School of CSE, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Michel Galley:\u0026nbsp;Senior Principal\u0026nbsp;Research Manager at Microsoft Research\u003C\/p\u003E\u003Cp\u003EDr. Dipanjan Das: Senior Director of Research at Google Deepmind\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EAbstract:\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EAs large language models (LLMs) move into real-world, open-ended applications, evaluating them becomes both more important and more difficult. My thesis develops evaluation methods that go beyond multiple-choice accuracy to handle multi-turn interaction, long-context tasks, and fine-grained text quality.\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EIn this thesis defense, I will first present our work on evaluating multi-turn assistants with user simulators. I introduce SimulatorArena, a benchmark of annotated human--LLM conversations that assesses how closely simulators match real user behavior and how well their assistant evaluations align with human judgments, and show that simulators conditioned on structured user profiles align closely with human ratings, making scalable multi-turn evaluation practical. I will next introduce GAVEL, an evaluation framework for long-context legal summarization, where case documents often exceed 100K tokens and expert summaries are long and information-dense. GAVEL-Ref evaluates model summaries against expert-written references through checklist, residual-fact, and writing-style evaluation, and GAVEL-Agent navigates the case documents with tools to extract checklist items directly, achieving competitive performance with substantially lower token usage and generalizing to the medical domain.\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EScalable and Structured Evaluation of Large Language Models\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"Scalable and Structured Evaluation of Large Language Models"}],"uid":"27707","created_gmt":"2026-07-06 13:39:49","changed_gmt":"2026-07-06 13:40:21","author":"Tatianna Richardson","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2026-07-17T11:00:00-04:00","event_time_end":"2026-07-17T12:30:00-04:00","event_time_end_last":"2026-07-17T12:30:00-04:00","gmt_time_start":"2026-07-17 15:00:00","gmt_time_end":"2026-07-17 16:30:00","gmt_time_end_last":"2026-07-17 16:30:00","rrule":null,"timezone":"America\/New_York"},"location":"Coda C1115 Druid Hills","extras":[],"groups":[{"id":"221981","name":"Graduate Studies"}],"categories":[],"keywords":[{"id":"100811","name":"Phd Defense"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1788","name":"Other\/Miscellaneous"}],"invited_audience":[{"id":"78771","name":"Public"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}