Previous chapters focused on transforming and optimizing machine learning models within intermediate representations. This chapter transitions to the final stage: translating this optimized IR into efficient machine code for a variety of hardware targets, from standard CPUs and GPUs to specialized AI accelerators.

The performance of an ML model heavily depends on how well the generated code utilizes the specific capabilities of the underlying hardware. Generating optimal code requires understanding target-specific instruction sets, memory architectures, and specialized execution units like GPU tensor cores or dedicated matrix multiplication units. Simply mapping operations is insufficient; the compiler backend must make intelligent choices about instruction selection, register allocation, and scheduling tailored to each unique architecture.

In this chapter, you will examine the techniques used in compiler backends to produce high-performance code for heterogeneous systems. We will cover:

• Strategies for selecting appropriate hardware instructions for ML computations.
• Techniques for managing registers, particularly for vector ($SIMD$) and matrix operations.
• Methods for generating code for GPUs using frameworks like CUDA and ROCm, including targeting specific hardware features like NVIDIA Tensor Cores or AMD Matrix Cores.
• Approaches for compiling to custom AI hardware (e.g., TPUs, NPUs).
• The role of target-agnostic intermediate formats like SPIR-V in managing hardware diversity.
• How compilers integrate with and utilize vendor-specific libraries (e.g., cuDNN, MIOpen, oneDNN) for maximum performance.

This chapter provides the methods needed to bridge the gap between a high-level optimized representation and high-performance executable code running on contemporary, diverse hardware platforms.