Toward Theoretical Foundations of Training and Generalization Dynamics in Deep Learning

Abstract: Understanding training and generalization dynamics remains a central challenge in machine learning, particularly as neural networks, datasets, and learning tasks become increasingly complex in practice. However, existing theoretical characterizations remain largely restricted to simplified models or limiting regimes. In this talk, I will present a general convergence framework for gradient descent on a broad family of neural networks and data beyond the neural tangent kernel regime, together with a characterization of their generalization dynamics. This framework also helps explain the role of residual connections, differences in layerwise generalization behavior, and the grokking phenomenon.

 

Bio: Yuqing Wang is a Data Science Postdoctoral Fellow in the Department of Applied Mathematics and Statistics at Johns Hopkins University. Before that, she was a Research Fellow at the Simons Institute at the University of California, Berkeley. She received her Ph.D. in Mathematics from the Georgia Institute of Technology in 2024. Her research focuses on the theoretical foundations of deep learning theory, with an emphasis on characterizing the training dynamics. She is the recipient of multiple rising star awards and the Sigma Xi Best PhD thesis award.