Deep learning’s growing scale makes computational, memory, and communication efficiency increasingly important. This dissertation treats sparsity as an integral component of learning in three settings: controllable sparse projection, sparse reinforcement learning, and communication-efficient federated learning. First, Group Sparse Projection (GSP) directly controls the average Hoyer sparsity of grouped vectors through one interpretable parameter while allowing adaptive individual sparsity levels. The dissertation establishes the projection’s theoretical properties and algorithm, then demonstrates it in sparse nonnegative matrix factorization and network pruning. Second, Data Adaptive Pathway Discovery (DAPD) learns sparse reinforcement-learning pathways under changing data distributions. It adapts masks during a warm-up phase and freezes them for stable optimization. Across single-task, multi-task, online, and offline settings, DAPD preserves strong policy performance with a small fraction of a shared network’s parameters. Third, Salient Sparse Federated Learning (SSFL) addresses communication-constrained, non-IID federated learning by aggregating clients’ private-data saliency scores into a shared mask before training. Experiments across CIFAR-10, CIFAR-100, Tiny-ImageNet, and deeper ResNets show competitive accuracy with less communication. A geographically distributed deployment yields wall-clock savings, and one-shot mask rediscovery adapts to out-of-distribution classes. Together, these methods show that matching sparsity discovery to the learning problem can reduce parameter, computation and performance.