Many of today's most important applications rely on high-performance manipulation of matrices, tensors, and general arrays of data. The neural networks at the heart of AI language models are a prime example. The AI software stack relies on kernels (e.g., for matrix-matrix product) that are specifically designed to deliver throughput close to the hardware's theoretical peak. However, writing kernels that can deliver maximum performance is extremely challenging. In this talk, I will discuss some of the work we have done to simplify the task of authoring such kernels. This includes new abstractions to allow for algebraic manipulation of array layouts and new programming models for more easily expressing the kernels themselves.
Michael Garland has been a researcher at NVIDIA since 2006. He joined the company as one of the founding members of NVIDIA Research and is currently Senior Director of Programming Systems Research. He leads a research group focused on developing technologies that will help programmers take advantage of modern high-performance machines. Their work spans the software stack with a particular focus on parallel algorithms, programming languages, compilers and runtime systems, and low-level hardware/software interfaces. He and his team have both published numerous academic articles and contributed to many software packages in broad use by CUDA developers.