Having prepared your system in the previous chapter, you now need a model to run. This chapter focuses on navigating the options for Large Language Models designed for local use. You will learn where to find models, primarily focusing on resources like the Hugging Face Hub.

We will examine key factors influencing your choice, including model size (often denoted by parameters, e.g., $7B$ or $13B$), file formats like GGUF which are optimized for local inference, and the concept of quantization used to reduce resource requirements. Furthermore, understanding model cards and software licenses is essential for appropriate selection and usage. The goal is to equip you with the criteria needed to select a suitable initial model that aligns with your hardware capabilities and objectives.