Having examined Large Language Models, their size in parameters, and the associated hardware components like GPUs and VRAM, we now turn to how these models are actually utilized. There are two fundamental modes of operation: using a pre-existing model to generate outputs (inference) and the process of creating or adapting a model (training).

These distinct activities impose substantially different loads on the hardware resources discussed earlier. Recognizing the difference between inference and training is key to understanding why certain hardware configurations are suitable for simply running an LLM, while others are necessary for development or fine-tuning.

This chapter covers the following points:

• Defining model inference: using a trained model for prediction or generation.
• Outlining the hardware resources typically required for inference.
• Defining model training: the process of building or adjusting model parameters from data.
• Contrasting the significantly greater hardware demands associated with training.
• Establishing why inference requirements are the primary consideration for most end-user interactions with LLMs.

By comparing the resource needs of these two processes, we can better contextualize the hardware estimation techniques discussed later.