Title: Statistical and Computational Analysis of Adversarial Training

Date: May 7th, 2025

Time: 10:00 AM – 11:30 AM EST

Location: Groseclose 303 Conference Room 

Meeting Link: https://gatech.zoom.us/j/97220632408?pwd=VtHPYZNxjKN2Ug4jQMXIXmRXtJQiLJ.1

Yiling Xie

Ph.D. Candidate in Industrial Engineering (Specialization in Statistics)

School of Industrial and Systems Engineering

Georgia Institute of Technology

Committee:

Dr. Xiaoming Huo (Advisor)

School of Industrial and Systems Engineering, Georgia Institute of Technology

Dr. Arkadi Nemirovski

School of Industrial and Systems Engineering, Georgia Institute of Technology

Dr. Roshan Joseph

School of Industrial and Systems Engineering, Georgia Institute of Technology

Dr. Ashwin Pananjady

School of Industrial and Systems Engineering, Georgia Institute of Technology

Dr. Cheng Mao

School of Mathematics, Georgia Institute of Technology

Abstract:

Adversarial training is proposed to hedge against adversarial perturbations and has attracted much research interest in recent years. In this thesis, we study adversarial training and its related concepts — Wasserstein distributionally robust optimization and Wasserstein distance — from statistical and computational perspectives.

In Chapter 1, we focus on the Wasserstein distance. It can be shown that computing the empirical Wasserstein distance in the Wasserstein-distance-based independence test is an optimal transport (OT) problem with a special structure. This observation inspires us to study a special type of OT problem and propose a modified Hungarian algorithm to solve it exactly. For the OT problem involving two marginals with $m$ and $n$ atoms ($m\geq n$), respectively, the computational complexity of the proposed algorithm is $\mathcal{O}(m^2n)$. The experiment results demonstrate that the proposed modified Hungarian algorithm compares favorably with the Hungarian algorithm, the well-known Sinkhorn algorithm,  and the network simplex algorithm.

In Chapter 2, we focus on the Wasserstein distributionally robust optimization. We propose an adjusted Wasserstein distributionally robust estimator—based on a nonlinear transformation of the Wasserstein distributionally robust (WDRO) estimator in statistical learning. The classic WDRO estimator is asymptotically biased, while our adjusted WDRO estimator is asymptotically unbiased, resulting in a smaller asymptotic mean squared error. Further, under certain conditions, our proposed adjustment technique provides a general principle to de-bias asymptotically biased estimators. Specifically, we will investigate how the adjusted WDRO estimator is developed in the generalized linear model, including logistic regression, linear regression, and Poisson regression. 

In Chapter 3 and Chapter 4, we focus on the statistical adversarial training. In Chapter 3, we focus on adversarial training under $\ell_\infty$-perturbation. The asymptotic behavior of the adversarial training estimator is investigated in the generalized linear model. The results imply that the asymptotic distribution of the adversarial training estimator under $\ell_\infty$-perturbation could put a positive probability mass at $0$ when the true parameter is $0$, providing a theoretical guarantee of the associated sparsity-recovery ability. Alternatively, a two-step procedure is proposed---adaptive adversarial training, which could further improve the performance of adversarial training under $\ell_\infty$-perturbation.  Specifically, the proposed procedure could achieve asymptotic variable-selection consistency and unbiasedness.  In Chapter 4, we deliver a non-asymptotic consistency analysis of the adversarial training procedure under $\ell_\infty$-perturbation in high-dimensional linear regression. It will be shown that, under the restricted eigenvalue condition,  the associated convergence rate of prediction error can achieve the minimax rate up to a logarithmic factor in the high-dimensional linear regression on the class of sparse parameters. Additionally, the group adversarial training procedure is analyzed. Compared with classic adversarial training, it will be proved that the group adversarial training procedure enjoys a better prediction error upper bound under certain group-sparsity patterns.