Modern machine learning workloads place growing demands on memory capacity, storage space, and data communication. Although specialized hardware provides substantial computational capability, its benefits are often limited by redundant data movement, inefficient memory utilization, and the increasing size of models and datasets. Addressing these challenges requires coordinated optimization of how data is represented, stored, transferred, and processed. This research aims to enable memory- and storage-efficient machine learning through architecture, system, and tool co-design—developing efficient hardware architectures, extending these principles to resource management across multiple devices, and creating tools that simplify the development of reusable hardware.