690278 event 1778708078 1778708185 <![CDATA[Ph.D. Dissertation Defense - Kuo-Wei Lai]]> Title:  Learning implicitly biased overparameterized models

Committee:

Dr. Vidya Muthukumar, ECE, Chair, Advisor

Dr. Molei Tao, Math

Dr. Mark Davenport, ECE

Dr. Justin Romberg, ECE

Dr. Christos Thrampoulidis, UBC

]]> The success of modern machine learning is built upon large-scale, highly parameterized models. In the overparameterized regime, these models routinely achieve near-zero training loss, yet generalize remarkably well to unseen data. A central mechanism underlying this phenomenon is implicit bias: gradient-based optimization does not converge to an arbitrary minimizer, but rather systematically favors solutions with particular structures or properties, determined by the interplay of the loss function, model architecture, and data distribution. This thesis studies the implicit bias of gradient descent (GD) in the overparameterized regime from two complementary perspectives: characterizing the mathematical and geometric structure of the solutions selected by the optimizer, and understanding the downstream consequences of this structure for generalization across diverse scenarios, including in-distribution (ID) and out-of-distribution (OOD) settings. Taken together, the results in this thesis reveal a unifying picture: gradient descent in overparameterized models inherently tends toward minimum-norm solutions across a wide range of settings. Understanding this bias precisely is key to explaining both the benign generalization behavior observed in-distribution and the relative fragility or robustness of models under distribution shift.

]]> <![CDATA[]]> 434381 1788 100811 1808