{"675843":{"#nid":"675843","#data":{"type":"event","title":"PhD Defense by Cheng Wan","body":[{"value":"\u003Cp\u003E\u003Cstrong\u003ETitle: Optimizing Sparsity in Distributed Machine Learning Training\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003E\u0026nbsp;\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EDate:\u0026nbsp;\u003C\/strong\u003EWednesday, August 14th, 2024\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ETime:\u0026nbsp;\u003C\/strong\u003E10:00 AM - 11:30 AM EST\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ELocation:\u0026nbsp;\u003C\/strong\u003EKACB 3126\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EVirtual:\u003C\/strong\u003E \u003Ca href=\u0022https:\/\/gatech.zoom.us\/j\/95651978956?pwd=cNa98ds2wGysW5SlajJDBAkouSiAvH.1\u0022\u003Ehttps:\/\/gatech.zoom.us\/j\/95651978956?pwd=cNa98ds2wGysW5SlajJDBAkouSiAvH.1\u003C\/a\u003E\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ECheng Wan\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003ESchool of Computer Science\u003C\/p\u003E\u003Cp\u003ECollege of Computing\u003C\/p\u003E\u003Cp\u003EGeorgia Institute of Technology\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003E\u0026nbsp;\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ECommittee Members:\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EDr. Yingyan (Celine) Lin (Advisor) \u2013 School of Computer Science, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Pan Li\u0026nbsp;\u2013 School of Computational Science and Engineering, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr.\u0026nbsp;Alexey Tumanov \u2013 School of Computer Science, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Anand Iyer\u0026nbsp;\u2013 School of Computer Science, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003E\u0026nbsp;\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EAbstract:\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EAs machine learning models and datasets continue to grow, distributed machine learning has become essential for meeting the computational demands of large-scale training. While recent frameworks have improved scalability and throughput, the optimization of sparsity within distributed deep neural networks remains under-explored. Sparsity, however, poses a significant challenge in advanced models such as graph neural networks and mixture-of-experts models.\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EThis thesis proposal systematically investigates three types of sparsity in distributed machine learning training: sparse data access, sparse operations, and sparse workflows. We introduce both system-level and algorithm-level innovations to address inefficiencies caused by these types of sparsity. Through theoretical insights and practical implementations, we demonstrate how these optimizations can significantly reduce communication overhead and enhance scalability, thereby improving the efficiency and performance of distributed machine learning systems. Our work advances the understanding of sparsity optimization and lays the groundwork for developing more efficient distributed training architectures in the future.\u003C\/p\u003E","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003E\u003Cstrong\u003E\u0026nbsp;Optimizing Sparsity in Distributed Machine Learning Training\u003C\/strong\u003E\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"Optimizing Sparsity in Distributed Machine Learning Training"}],"uid":"27707","created_gmt":"2024-08-07 20:55:12","changed_gmt":"2024-08-07 20:55:12","author":"Tatianna Richardson","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2024-08-14T10:00:00-04:00","event_time_end":"2024-08-14T11:30:00-04:00","event_time_end_last":"2024-08-14T11:30:00-04:00","gmt_time_start":"2024-08-14 14:00:00","gmt_time_end":"2024-08-14 15:30:00","gmt_time_end_last":"2024-08-14 15:30:00","rrule":null,"timezone":"America\/New_York"},"location":"KACB 3126","extras":[],"groups":[{"id":"221981","name":"Graduate Studies"}],"categories":[],"keywords":[{"id":"100811","name":"Phd Defense"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1788","name":"Other\/Miscellaneous"}],"invited_audience":[{"id":"78771","name":"Public"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}