{"676023":{"#nid":"676023","#data":{"type":"event","title":"PhD Proposal by Adam Coscia","body":[{"value":"\u003Cp\u003E\u003Cstrong\u003ETitle:\u003C\/strong\u003E\u0026nbsp;Visual Analytics for Trustworthy Large Language Models in Education\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EDate:\u003C\/strong\u003E\u0026nbsp;Thursday, August 22nd\u0026nbsp;2024\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ETime:\u003C\/strong\u003E\u0026nbsp;9 a.m. - 11 a.m. ET (US)\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ELocation:\u003C\/strong\u003E\u0026nbsp;Technology Square Research Building (TSRB) 334 (\u003Cem\u003Ethird floor conference room; just walk in, no special access needed\u003C\/em\u003E)\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EVirtual meeting:\u003C\/strong\u003E\u0026nbsp;\u003Ca href=\u0022https:\/\/gatech.zoom.us\/j\/3100254613?pwd=QWlKajNkOWlPbWkxR3N5MkZsTE9FZz09\u0022 title=\u0022https:\/\/gatech.zoom.us\/j\/3100254613?pwd=QWlKajNkOWlPbWkxR3N5MkZsTE9FZz09\u0022\u003EClick here to join Zoom meeting\u003C\/a\u003E\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EAdam Coscia\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EPh.D. Student in Human-Centered Computing\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ESchool of Interactive Computing\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EGeorgia Institute of Technology\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Ca href=\u0022https:\/\/adamcoscia.com\/\u0022\u003Ehttps:\/\/adamcoscia.com\/\u003C\/a\u003E\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ECommittee\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EDr. Alex Endert (Advisor) - School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Duen Horng (Polo) Chau - School of Computational Science \u0026amp; Engineering, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Cindy Xiong Bearfield - School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Yalong Yang - School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\u003Cp\u003EDr. Scott Crossley - Department of Special Education, Vanderbilt University\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EAbstract\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EDevelopers in education technology and the learning sciences are rapidly integrating transformer-based large language models (LLMs) such as ChatGPT into novel adaptive learning tools for improving online education. Both scalable and generalizable, LLMs enable adaptive learning in a variety of ways -- from user-facing interfaces such as conversational chatbots for enhanced learning and feedback, to time-saving behind-the-scenes grading and content moderation. However, the often unpredictable behaviors of LLMs have also introduced several pedagogical risks and harms, such as responding with misinformation and discriminatory language in conversation, as well as biasing scores against individuals when used for grading. As a result, multiple stakeholders in education, including developers, instructors, and learners, are distrustful of LLMs being used in educational technologies, inhibiting the adoption of positive and transformational advances in adaptive learning enabled by LLMs.\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EOne of the greatest barriers to building trust in safely using LLMs for education is a lack of tools that help stakeholders understand what LLMs are capable of and how they might impact learning outcomes. Thus, to address stakeholders\u0027 concerns around using LLMs in education, we propose to investigate how to build tools that establish trustworthy LLMs in education. \u003Cstrong\u003EThe goal of this dissertation is to enable developers, instructors, and learners to calibrate their trust in LLMs by building novel visual analytics tools that help developers first evaluate the trustworthiness of LLMs in education, and then communicate the results of evaluation to non-technical stakeholders.\u003C\/strong\u003E\u0026nbsp;We believe the use cases, study findings, and lessons learned from this work will inspire new techniques and advances in developing novel visualizations that help establish trustworthy LLMs in education.\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EThe approach of this work is two-fold:\u003C\/p\u003E\u003Col type=\u00221\u0022\u003E\u003Cli\u003EFirst, we developed \u003Cstrong\u003Evisual analytics tools\u003C\/strong\u003E\u0026nbsp;for developers (\u003Cem\u003E\u003Cstrong\u003EKnowledgeVIS\u003C\/strong\u003E\u0026nbsp;\u003C\/em\u003Eand \u003Cem\u003E\u003Cstrong\u003EiScore\u003C\/strong\u003E\u003C\/em\u003E) that helped them evaluate the trustworthiness of their LLMs embedded in educational technology. We identified the challenges and tasks of developers when evaluating LLMs in the context of their technical workflows building LLM-powered educational technologies, then evaluated the effectiveness of our designs for helping developers understand, evaluate and build trust in how LLMs work.\u003C\/li\u003E\u003Cli\u003ESecond, we propose to create a \u003Cstrong\u003Evisual analytics toolkit\u003C\/strong\u003E\u0026nbsp;that helps developers communicate the trust they calibrated with instructors and learners. We will curate a set of useful metrics for measuring the trustworthiness of LLMs in the context of education, implement our metrics in a visual analytics toolkit for building solutions such as dashboards, and study the usability of our toolkit with developers, as well as the effectiveness of our visualizations for communicating trust with stakeholders.\u003C\/li\u003E\u003C\/ol\u003E","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EVisual Analytics for Trustworthy Large Language Models in Education\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"Visual Analytics for Trustworthy Large Language Models in Education"}],"uid":"27707","created_gmt":"2024-08-15 14:23:55","changed_gmt":"2024-08-15 14:23:55","author":"Tatianna Richardson","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2024-08-22T09:00:00-04:00","event_time_end":"2024-08-22T11:00:00-04:00","event_time_end_last":"2024-08-22T11:00:00-04:00","gmt_time_start":"2024-08-22 13:00:00","gmt_time_end":"2024-08-22 15:00:00","gmt_time_end_last":"2024-08-22 15:00:00","rrule":null,"timezone":"America\/New_York"},"location":"Technology Square Research Building (TSRB) 334 ","extras":[],"groups":[{"id":"221981","name":"Graduate Studies"}],"categories":[],"keywords":[{"id":"102851","name":"Phd proposal"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1788","name":"Other\/Miscellaneous"}],"invited_audience":[{"id":"78771","name":"Public"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}