678282 event 1731054411 1731054538 <![CDATA[Ph.D. Proposal Oral Exam - Anirudh Sundar]]> Title:  Augmenting Large Language Models with the Ability to Hold Situated Dialogue Over Structured Knowledge Contained in Scientific Documents

Committee: 

Dr. Heck, Advisor 

Dr. Muthukumar, Chair

Dr. Davenport

]]> The objective of the proposed research is to develop language-model-based methods to address the problem of situated and multimodal interactive conversations in scientific papers. While significant progress has been made in large multimodal language models capable of multi-turn dialogue, creating a conversational assistant that can effectively navigate and discuss structured knowledge sources remains an open challenge, particularly within the domain of scientific literature. Scientific documents present a unique set of challenges due to their inherently multimodal nature. Researchers utilize various formats to convey complex information: figures illustrate key concepts, equations express mathematical principles, and tables summarize experimental setups and results. Moreover, the interconnected nature of these elements—with researchers frequently cross-referencing figures, tables, and equations throughout the text—adds another layer of complexity. This research seeks to address these challenges by developing methods that can seamlessly integrate and interpret these diverse modalities, enabling more sophisticated and context-aware interactions with scientific content.

]]> <![CDATA[]]> 434371 1788 102851 1808