The primary goal of this thesis is to present comprehensive design methodologies and electronic design automation (EDA) techniques aimed at enhancing the efficiency of 3D machine learning accelerator designs. This research extensively investigates the challenges associated with scaling near-memory and in-memory compute accelerators in the X/Y dimensions and proposes optimized scaling solutions by leveraging heterogeneous 3D integration. While numerous 3D physical design EDA methodologies exist, they often involve a multitude of parameters that require careful tuning based on the specific design in question. To address this, the thesis examines the impact of these parameters on various design types using cutting-edge machine learning-based parameter autotuning tools. It subsequently recommends the most effective parameter tuning techniques depending on the particular 3D EDA approach being employed. Using the EDA tools optimized through machine learning, the thesis delves into the advantages of heterogeneous 3D implementation for near-memory and in-memory compute ML accelerators at various technology nodes. Heterogeneous 3D integration delivers substantial improvements in terms of Power-Performance-Area (PPA), throughput, energy efficiency, and area efficiency when compared to their 2D counterparts. Finally, the thesis introduces a straightforward yet highly efficient machine-learning framework for exploring the design space of heterogeneous 3D accelerators. This framework assists designers in selecting the optimal accelerator architecture tailored to the requirements of their target applications.