Having introduced Large Language Models, their measurement in parameters, and the essential hardware components like GPUs and VRAM, we now connect these pieces. This chapter focuses on how the number of parameters in an LLM directly translates into hardware resource demands.

You will learn about:

• Memory Consumption: How model parameters are loaded into memory, primarily VRAM.
• Data Types and Precision: The impact of different numerical formats (like $FP16$, $INT8$) on memory usage.
• Quantization: A technique to reduce the model's memory footprint.
• Compute Requirements: Understanding the computational load ($FLOPS$) associated with model size.
• Memory Bandwidth: Why the speed of memory access is significant for LLM performance.