Title: Robust Efficient Edge AI: New Principles and Frameworks for Empowering AI on Edge Devices 

Rahul Duggal

School of Computational Science & Engineering

College of Computing

Georgia Institute of Technology

www.rahulduggal.com

Date: Thursday, July 14, 2022

Time: 1-3pm (ET)

Location (virtual): zoom

Committee:

Dr. Polo Chau (advisor), School of Computational Science & Engineering, Georgia Institute of Technology

Dr. Richard Vuduc, School of Computational Science & Engineering, Georgia Institute of Technology

Dr. Ada Gavrilovska, School of Computer Science, Georgia Institute of Technology

Dr. Callie Hao, School of Electrical and Computer Engineering, Georgia Institute of Technology

Dr. Jimeng Sun, Department of Computer Science, University of Illinois at Urbana Champaign

Abstract:

Deep learning has revolutionized a breadth of industries by automating critical tasks while achieving superhuman accuracy. However, many of these benefits are driven by huge neural networks deployed on cloud servers that consume enormous energy. This thesis contributes two classes of novel frameworks and algorithms that extend the deployment frontier of deep learning models to tiny edge devices, which commonly operate in noisy environments with limited compute footprints: (1) New frameworks for efficient edge AI: We introduce methods that reduce inference cost through filter pruning and efficient network design. CUP presents a new method for compressing and accelerating models, by clustering and pruning similar filters in each layer. CMP-NAS presents a new visual search framework that optimizes a small and efficient edge model to work in tandem with a large server model to achieve high accuracy, achieving up to 80x compute cost reduction. (2) New methods for robust edge AI: We Introduce new methods that enable robustness to real-world noise while reducing inference cost. REST, extends the scope of pruning to obtain networks that are 9x more efficient, run 6x faster and are robust to adversarial and gaussian noise. HAR generalizes the idea of early exiting in multi-branch neural networks to the training phase leading to networks that obtain state-of-the-art accuracy under class imbalance while saving up to 20% inference compute. IMB-NAS optimizes neural architectures on imbalanced datasets through super-network adaptation strategies that lead to 5x compute savings compared to searching from scratch. This dissertation makes significant impact to industry and society: CMP-NAS enables the edge deployment use-case for image retrieval services, and was highlighted at Amazon company-wide to thousands of researchers and developers. REST enables at-home sleep monitoring through a mobile phone and was highlighted by several news media.