Chapter 6: Capacity Planning and Hardware Sizing

Capacity planning for production LLM deployments translates workload demands and Service Level Agreements into concrete hardware specifications. Meeting targets for Time-to-First-Token, Time-Per-Output-Token, and aggregate Queries-Per-Second requires balancing memory capacity, memory bandwidth, compute throughput, and interconnect bandwidth.

This chapter synthesizes memory and compute calculations into an end-to-end sizing framework. It applies roofline performance modeling, dynamic batching curves, and concurrency scaling formulas to size GPU clusters accurately for both single-stream latency SLAs and high-throughput production workloads.