Chapter 1: Foundations of AI Compute Infrastructure

An AI model's performance is a direct function of the hardware it runs on. To build effective systems, you must first understand the computational requirements of machine learning workloads and the hardware components that satisfy them. This chapter provides that initial analysis.

You will learn to differentiate between training and inference workloads and their distinct hardware demands. We will analyze the roles of CPUs for sequential tasks and GPUs for the parallel computations common in deep learning. We will compare their architectures and see why a GPU excels at executing thousands of operations simultaneously, such as the matrix multiplications ($C = A \cdot B$) at the core of neural networks. The discussion also covers specialized accelerators like TPUs and the critical supporting roles of memory, storage, and networking.

The chapter ends with a practical exercise where you will benchmark a task on both a CPU and a GPU to observe these performance differences firsthand.