TITLE: "Sparse" computation of gradients for optimization with large data sets

SPEAKER: Dr. Guanghui Lan

ABSTRACT:

The last few years have seen an increasing interest in utilizing optimization for large-scale data analysis. However, optimization problems arising from these applications often involve, in addition to expensive smooth components for data fitting, nonsmooth and nonseparable regularization terms/constraints to enforce certain structural properties for the generated solutions (e.g, low rank or group sparsity). It is well-known that such nonsmooth components can significantly slow down the convergence of existing first-order optimization algorithms, leading to a large number of traverses through the data sets. To address this issue, we present a new class of optimization techniques, referred as to gradient sliding and conditional gradient sliding methods, which can skip the computation of gradients from time to time while still maintaining the overall optimal rate of convergence. In particular, the number of gradient evaluations required for these algorithms will be the same as if the aforementioned nonsmooth and nonseparable components do not exist. When applied to data analysis problems, these algorithms can reduce the number of scans through the data sets by orders of magnitude. Numerical experiments have been conducted to illustrate the effectiveness of these techniques.

 Short-bio: Guanghui (George) Lan obtained his Ph.D. degree in Industrial and Systems Engineering from Georgia Institute of Technology in August, 2009. He then joined the Department of Industrial and Systems Engineering at the University of Florida as an assistant professor thereafter. His main research interests lie in stochastic optimization, nonlinear programming, simulation-based optimization, and their applications in data sciences. His research has been supported by the National Science Foundation and Office of Naval Research. The academic honors that he received include the INFORMS Computing Society Student Paper Competition First Place (2008), INFORMS George Nicholson Paper Competition Second Place (2008), Mathematical Optimization Society Tucker Prize Finalist (2012), INFORMS Junior Faculty Interest Group (JFIG) Paper Competition First Place (2012) and the National Science Foundation CAREER Award (2013).