Once a high-level framework lowers a model into an intermediate representation (IR), the resulting computational graph represents the logical flow of operations. However, a direct translation of this graph into machine code often yields suboptimal performance. Executing each node as a separate kernel introduces significant scheduling overhead and puts unnecessary pressure on memory bandwidth. This chapter focuses on optimizing the structure of this graph before the compiler generates specific hardware instructions.

The primary objective here is to reduce the number of memory accesses and kernel launches. For instance, executing an operation like

$C = \text{ReLU}(A + B)$

as two distinct kernels requires writing the result of $A + B$ to global memory, only to read it back immediately for the activation function. By applying graph-level transformations, a compiler can fuse these operations into a single kernel, keeping intermediate data in registers or cache.

You will learn how to implement specific optimization passes such as operator fusion, where compatible nodes are merged to improve locality. The text also covers algebraic simplification, using mathematical properties to replace expensive operations with cheaper alternatives. We will examine layout transformations, ensuring tensors are stored in memory formats like $NCHW$ or $NHWC$ that align with the target hardware's access patterns. Finally, the chapter addresses general clean-up passes like Dead Code Elimination (DCE) and Common Subexpression Elimination (CSE) to ensure the graph contains only essential computations.