Chapter 1: LLM Hardware and Bottleneck Basics

Operating Large Language Models in production requires precise hardware capacity planning to balance compute resources and memory allocations against latency targets. An accurate calculation of GPU memory and compute requirements prevents out-of-memory errors, avoids underutilized hardware, and ensures predictable serving performance.

This chapter establishes the core mechanics of transformer execution and hardware constraints across memory capacity, memory bandwidth, compute throughput, and interconnect speeds. It evaluates how prompt processing and autoregressive token generation interact with physical hardware, distinguishing fixed resident model parameters from dynamic runtime memory allocations.