Featuring Xiuyuan Cheng, Duke University

Abstract: Normalizing flow has become a popular class of deep generative models for efficient sampling and density estimation. Recently, the remarkable success of score-based diffusion models has inspired flow-based models closely related to the diffusion process. In particular, the celebrated Jordan-Kinderleherer-Otto (JKO) scheme captures the variational nature of the Fokker-Planck equation of a diffusion process as a Wasserstein gradient flow, and in the context of normalizing flow, this naturally suggests a progressive way of training a flow model that implements a proximal gradient descent in the Wasserstein-2 space. In this talk, we introduce such a JKO flow model that achieves competitive performance compared to existing diffusion and flow models on generating high dimensional real data. The proposed flow network stacks residual blocks one after another, where each block corresponds to a JKO step. Thanks to the invertibility of the neural ODE model, both the forward (data-to-noise) and reverse (noise-to-data) processes are deterministic. On the theoretical side, the connection to Wasserstein proximal gradient descent allows us to prove the exponentially fast convergence of the discrete-time flow in both directions. Joint work with Yao Xie, Chen Xu (Georgia Tech), and Jianfeng Lu, Yixin Tan (Duke).

Bio: Dr. Xiuyuan Cheng is an Associate Professor of Mathematics at Duke University. She develops theoretical and computational techniques to solve problems in high-dimensional data analysis and machine learning. Cheng received her Ph.D. in Applied and Computational Mathematics from Princeton University in 2013.  The work of Dr. Cheng was recognized by a Sloan Fellowship in 2019 and a National Science Foundation CAREER award in 2023.