After optimizing ML models through various compilation stages, the resulting code requires an efficient execution environment. This is the role of the runtime system. This chapter examines the design and implementation of advanced runtime systems tailored for demanding ML tasks.

We will cover critical runtime responsibilities, including:

• Managing tensors with dynamic shapes (e.g., sizes like $N \times C \times ? \times ?$ determined during execution).
• Implementing effective memory management techniques for large data structures.
• Orchestrating asynchronous execution to overlap computation and data movement.
• Scheduling operations efficiently across heterogeneous hardware like CPUs and GPUs.
• Integrating custom-built operators and kernels.
• Ensuring interoperability with high-level ML frameworks.

Studying these components will provide insight into building and analyzing the systems that bring compiled ML models to life on target hardware.