Access to high performance computing (HPC) has become essential for many scientific fields. However, domain experts in chemistry or physics can’t obtain second PhDs in computer science solely to obtain the coding expertise needed to execute parallel computing processes in their research.
University of Illinois Urbana-Champaign CS professor Billy Moses has received the 2024 SIGHPC Doctoral Dissertation Award for his dissertation “Supercharging Programming through Compiler Technology.” Moses received his PhD in electrical engineering and computer science at MIT. SIGHPC Chair called the dissertation “an impressive achievement and a definitive statement on the ability of general-purpose compiler technologies to deliver important HPC capabilities, enabling HPC applications to exploit domain-specific abstractions efficiently.” In addition to his position at The Grainger College of Engineering Siebel School of Computing and Data Science, Moses is a researcher at Google.
Moses describes his research by saying, “A key goal of our work is making tools that let that scientists leverage computation without becoming experts in everything else. As an example, I work with some physicists who simulate black holes. A few lines of math equations in a paper quickly turn into thousands of lines of code involving parallel cores, machine learning models, memory transfers, and more. We’re building tools which let users focus on writing the code that matches their expertise and endow the compiler with the expertise in HPC and machine learning to automatically generate the code that they will actually run.”
His research work overturns the conventional wisdom that domain-specific abstractions must be implemented by source-code tools. For instance, Moses’ Tapir compiler optimizes parallel code by embedding parallelism directly into the compiler’s internal representation. Tapir now forms the basis of the open-source OpenCilk compiler. As another example, the Enzyme compiler showed that derivatives of arbitrary functions expressed as program code can be generated automatically, yielding significant performance improvement over source-based tools. A key advantage of Moses’ compiler-based approach is composability, allowing seamless combination of compiler tools.
“Compilers can democratize access to computing. Instead of requiring each user to bear this burden, endowing the compiler with this expertise lets researchers write the code they want and still leverage the latest developments in the field like parallelism, security, and machine learning.”
Moses says, “Existing approaches to maintaining that expertise required you to write an entirely new tool just for, say, understanding graphics, climate science, GPUs, or whatever else. The key advantage of my approach is that instead of making all these separate silos of individual components, creating representations of arbitrary domain expertise in a common compiler enables them all to compose. As an example, our black hole colleagues want to not only compute their physics equations, but also have it run in parallel on a distributed cluster of machines and act as part of an ML model. Prior to our work they were stuck. You might have a domain-specific language (DSL) which can understands astrophysics, or a different DSL which understands GPUs. If you want to do something that combines both the astrophysics and GPUs, however, let alone the machine learning, you’re stuck. At a high level the question my thesis answered was: can we build tools that allow arbitrary domain experts, be it in high performance computing, be it in machine learning, be it in security, or even your black hole physicist, to be able to take their domain expertise, and put it into a common framework that can optimize across all these boundaries.”
Moses and a team wrote a 2022 paper demonstrating combining differentiation-specific optimizations in the Enzyme automatic differentiation (AD) framework with parallel-specific optimizations he worked on earlier in Tapir and Polygeist to give improvements significantly greater than the sum of its parts. The team, which included staff members from Argonne National Laboratory, where Moses did a practicum, a research aide from the Technical University of Munich, and another MIT doctoral candidate, won a Best Student Paper Award and Best Paper Finalist at SC22.
Moses says the dissertation is “just the beginning” of his work. “There are many other critical domains like communication that need to be represented, and also the more general question of how we can enable experts in less common domains to automate their expertise without becoming compiler experts themselves.” Moses leads the PROgram Optimization geNeration and Transformation (PRONTO) lab at Illinois and has a team of three graduate students, currently working on distributed computing, numerics, and tensor optimization, as well as several undergraduates and collaborators across academic, industry, and national labs.
Billy Moses is an Illinois Grainger Engineering professor of computer science and is affiliated with electrical and computer engineering and the Coordinated Science Laboratory.
