689795 event 1776363962 1776364031 <![CDATA[Ph.D. Proposal Oral Exam - Yavuz Yarici]]> Title:  Understanding and Utilizing Domain-Dependent Information Structure in Multimodal Systems

Committee:

Dr. AlRegib, Advisor

Dr. Vela, Chair

Dr. Calhoun

]]> The objective of the proposed research is to characterize the domain-dependent structure of task-relevant information in multimodal systems and to develop methods that maintain robust performance under distribution shift. One of the main promises of multimodal learning is to combine data from multiple sensing modalities via the use of their complementary information for a targeted downstream task. However, the benefits of multimodal fusion are typically demonstrated when training and evaluation data share the same distribution. Domain generalization methods address distribution shift for single-modality systems, but do not account for the interactions that arise when multiple encoders are trained jointly through a shared objective. While information-theoretic frameworks provide a principled decomposition of task-relevant information into redundant, unique, and synergistic components, they assume stationary distributions. How this information structure changes under domain shift, and how such changes interact with joint optimization, remain unexplored. This proposal addresses this gap by analyzing how each component of multimodal information varies across domains and by developing training procedures that preserve robust representations under domain shift. Preliminary research establishes that multimodal fusion improves accuracy under matched conditions, but this advantage diminishes under distribution shift. The proposed research characterizes how domain shift affects multimodal information structure and develops methods to maintain robust performance.

]]> <![CDATA[]]> https://teams.microsoft.com/meet/2365305733776?p=RUG0VJyE5W3Th8UEYL 434371 1788 102851 1808