Center for Signals and Information Processing (CSIP) Seminar

Date: Friday, February 16, 2024

Time: 3:00 p.m. - 4:00 p.m.

Location: Centergy Building 5126. The associated zoom link is: https://gatech.zoom.us/j/92180566322

Speaker: Chen Xu

Speakers' Title: Operations Research Ph.D. in ISyE

Seminar Title: Normalizing flow neural networks by JKO scheme

Abstract: Chen Xu is currently a fourth-year Operations Research PhD in ISyE, where he is supervised by Prof. Yao Xie. His current research interests are two-fold. (1) Uncertainty quantification for machine learning models. Specifically, advance conformal prediction as a distribution-free method for arbitrarily complex deep models, especially in the context of time-series modeling. (2) Generative models through flow-based neural networks. Specifically, develop scalable computational tools for problems at the intersection of statistics and optimization, including extensions to high-dimensional optimal transport, distributionally robust optimization, and differential privacy. His works have appeared in top machine learning conferences (e.g., ICML 2021 oral, NeurIPS 2023 spotlight) and journals (e.g., IEEE TPAMI 2023, IEEE JSAIT).

Bio: Normalizing flow is a class of deep generative models for efficient sampling and likelihood estimation, which achieves attractive performance, particularly in high dimensions. The flow is often implemented using a sequence of invertible residual blocks. Existing works adopt special network architectures and regularization of flow trajectories. In this paper, we develop a neural ODE flow network called JKO-iFlow, inspired by the Jordan-Kinderleherer-Otto (JKO) scheme, which unfolds the discrete-time dynamic of the Wasserstein gradient flow. The proposed method stacks residual blocks one after another, allowing efficient block-wise training of the residual blocks, avoiding sampling SDE trajectories and score matching or variational learning, thus reducing the memory load and difficulty in end-to-end training. We also develop adaptive time reparameterization of the flow network with a progressive refinement of the induced trajectory in probability space to improve the model accuracy further. Experiments with synthetic and real data show that the proposed JKO-iFlow network achieves competitive performance compared with existing flow and diffusion models at a significantly reduced computational and memory cost.