681406 event 1743075655 1743075744 <![CDATA[Ph.D. Proposal Oral Exam - Kuo-Wei Lai]]> Title:  Learning implicitly biased overparameterized models

Committee:

Dr. Muthukumar, Advisor

Dr. Davenport, Chair

Dr. Tao

]]> The objective of the proposed research is to investigate the nature of implicit biases in overparameterized machine learning models, with an emphasis on extending findings from linear to nonlinear settings. In modern machine learning, the classical bias-variance trade-off framework fails to explain why overparameterized models often exhibit excellent generalization performance. This surprising phenomenon, known as ``double descent," describes how generalization error can decrease again in the overparameterized regime. Recent studies suggest that specific optimization processes may ``implicitly bias" these models toward solutions with desirable properties, such as minimum norm, which can contribute to their effectiveness. Understanding this implicit bias is crucial to grasping the mechanisms underlying the success of overparameterized models. Moreover, while deep learning models have demonstrated remarkable performance across diverse applications, their nonlinearity presents challenges for theoretical analysis. This research aims not only to explore the more tractable linear settings but also to extend the investigation to nonlinear models, seeking to clarify the factors driving the success of overparameterized deep networks. This proposal will first review the background and previous work of implicit bias and generalization in overparameterized models. Following this, I will present my preliminary research on linear models, covering implicit bias characterization for general loss functions, error bounds in out-of-distribution generalization, and task shifts from classification to regression. Finally, I will outline the proposed work on characterizing implicit bias in ReLU-based models, focusing on expanding insights from linear to nonlinear frameworks.

]]> <![CDATA[]]> 434371 1788 102851 1808