691075 event 1783621450 1783621537 <![CDATA[Ph.D. Dissertation Defense - Riyasat Ohib]]> Title:  The Hidden Structure of Deep Models: Discovering and Exploiting Sparse Subnetworks

Committee:

Dr. Vince Calhoun, ECE, Chair, Advisor

Dr. Sergey Plis, GSU, Co-Advisor

Dr. David Anderson, ECE

Dr. Li Xiong, Emory

Dr. Many Malek, Google DeepMind

Dr. Irfan Essa, CoC

]]> Deep learning’s growing scale makes computational, memory, and communication efficiency increasingly important. This dissertation treats sparsity as an integral component of learning in three settings: controllable sparse projection, sparse reinforcement learning, and communication-efficient federated learning. First, Group Sparse Projection (GSP) directly controls the average Hoyer sparsity of grouped vectors through one interpretable parameter while allowing adaptive individual sparsity levels. The dissertation establishes the projection’s theoretical properties and algorithm, then demonstrates it in sparse nonnegative matrix factorization and network pruning. Second, Data Adaptive Pathway Discovery (DAPD) learns sparse reinforcement-learning pathways under changing data distributions. It adapts masks during a warm-up phase and freezes them for stable optimization. Across single-task, multi-task, online, and offline settings, DAPD preserves strong policy performance with a small fraction of a shared network’s parameters. Third, Salient Sparse Federated Learning (SSFL) addresses communication-constrained, non-IID federated learning by aggregating clients’ private-data saliency scores into a shared mask before training. Experiments across CIFAR-10, CIFAR-100, Tiny-ImageNet, and deeper ResNets show competitive accuracy with less communication. A geographically distributed deployment yields wall-clock savings, and one-shot mask rediscovery adapts to out-of-distribution classes. Together, these methods show that matching sparsity discovery to the learning problem can reduce parameter, computation and performance.

]]> <![CDATA[]]> https://gatech.zoom.us/j/93976878784?pwd=0KMQtIFrOf5M1jw8IlzvAQZ4Ozrmzp.1&from=addon 434381 1788 100811 1808