Machine learning models, particularly deep neural networks, require substantial computational resources. While frameworks like PyTorch and TensorFlow provide high-level abstractions for model design, executing these models efficiently on specific hardware requires optimization at the compiler level. This course covers the architecture and mechanics of machine learning compilers, focusing on how high-level computation graphs are transformed into efficient machine code.
You will study the lifecycle of an ML model from graph capture to code generation. The curriculum examines intermediate representations (IR), graph-level transformations, and low-level loop optimizations. You will learn how compilers perform operator fusion, memory layout rewriting, and hardware-specific instruction mapping. The course also addresses auto-tuning strategies used to find optimal execution schedules without manual intervention. By working through the material, you will gain the technical skills to inspect, debug, and enhance model performance using modern compiler stacks.
Prerequisites Basic ML & programming
Level:
Compiler Architecture
Understand the components of an ML compiler stack, including frontend, intermediate representation, and backend code generation.
Graph Optimization
Apply high-level transformations such as operator fusion, constant folding, and dead code elimination to computation graphs.
Loop Scheduling
Implement low-level optimizations including tiling, vectorization, and loop reordering to maximize hardware utilization.
Auto-Tuning
Configure and run automated search processes to identify optimal parameters for specific hardware targets.