Having addressed optimizations at the graph level, we now concentrate on the computationally intensive kernels within machine learning models: the tensor operations. Operations such as matrix multiplication, often expressed as $C = A \times B$, or convolutions frequently manifest as complex, nested loop structures in the implementation. Efficient execution of these loops is fundamental to maximizing performance.

This chapter presents methods for optimizing these specific tensor computations. You will learn about:

• Representing tensor operations like matrix multiplication and convolution as nests of loops.
• The principles of polyhedral modeling, a formal technique used to analyze and restructure loop nests based on their iteration spaces and data dependencies.
• Applying polyhedral transformations, including loop tiling and scheduling, to enhance data locality and expose parallelism.
• Generating optimized loop code from polyhedral schedule representations.
• Complementary optimization strategies like auto-vectorization for SIMD execution and techniques for managing the memory hierarchy, such as cache tiling and data prefetching.