The objective of the proposed research is to claim that the coordinated model–system–hardware tri-design is the right lever to achieve Efficient ML across CNNs, GNNs, and LLMs. This dissertation substantiates this claim with advances at three granularities, (i) single-layer methods that compress, quantize, and distill models to achieve lower compute and memory footprint, (ii) two-layer co-designs that translate algorithmic structure into runtime scheduling and hardware primitives to reduce latency and data movement, and raise on-chip utilization (model–system, model–hardware, system-hardware), and (iii) full model–system–hardware integration that combines innovations in all design abstraction layers in end-to-end pipelines, thereby reducing the dominant costs of data movement, numeric precision limits, sparsity, and real-time serving into measurable gains, which eventually results in improving the accuracy–latency–energy Pareto frontier.