Balancing Generality and Specialization for Machine Learning in the Post ISA Era

Divya Mahajan

PhD Student

School of Computer Science

College of Computing

Georgia Institute of Technology

Date: Friday, April 27 2018

Time: 12:00 Noon - 2:00 PM (EDT)

Location: Klaus 1202 

Committee:

Dr. Hadi Esmaeilzadeh, (Advisor), School of Computer Science, Georgia Institute of Technology & Department of Computer Science and Engineering, University of California, San Diego

Dr. Milos Prvulovic, School of Computer Science, Georgia Institute of Technology

Dr. Hyesoon Kim, School of Computer Science, Georgia Institute of Technology

Dr. Doug Burger, Distinguished Engineer, Microsoft Research

Dr. Dean Tullsen, Department of Computer Science and Engineering, University of California, San Diego

------------------------

Abstract:

Advances in Machine Learning (ML) are set to revolutionize medicine, robotics, commerce, transportation, and many other aspects of our daily lives. However, such transformative effects are predicated on providing both high-performance compute capabilities that enable these learning algorithms and the simultaneous advancement in algorithms that can adapt to the continuously changing landscape of the data revolution. The compute needs of emerging applications, especially Machine Learning, demands hardware acceleration. Although, the trend in the industry has moved towards embedding alternative hardware platforms in data-centers, people still rely on the traditional compute stack. The focus of my thesis is to devise a comprehensive full-stack solution that exposes a high-level mathematical programming interface to users that have limited knowledge about hardware design, but nevertheless, can benefit from large performance and efficiency gains through acceleration. Therefore, this thesis strikes a balance between generality and specialization by breaking the long-held traditional abstraction of Instruction Set Architecture (ISA) and delving into the algorithmic foundations of Machine Learning. My dissertation work: (1) rethinks the hardware-software abstraction from an algorithmic perspective to enable the developed accelerated platforms to support fast changing machine learning algorithms and models; (2) co-designs languages, compilers, runtime systems, and hardware to provide maximal performance and efficiency from accelerated systems while providing flexibility and programmability; (3) segregates algorithmic specification from implementation to allow continuous revision of the specialized hardware/software solution and support the emerging heterogeneity in computer platforms without requiring programmers to reimplement their algorithms for every revision or every accelerator type; (4) develops real cross-stack prototypes to examine the innovations in a real-world setting and making them open-source to maximize community engagement and industry impact.