Home Blog Learn

All Courses

Getting Started with Local LLMs

Chapter 1: Introduction to Large Language Models

What Is a Large Language Model (LLM)?

A Simple View of How LLMs Work

Understanding Tokens and Text Generation

Why Run LLMs Locally?

Local vs. Cloud-Based LLMs

Chapter 2: Preparing Your Local Environment

Hardware Considerations: CPU

Hardware Considerations: RAM

Hardware Considerations: GPU and VRAM

Checking Your System Specifications

Operating System Compatibility

Installing Python (Optional but Recommended)

Introduction to the Command Line / Terminal

Chapter 3: Finding and Selecting Local LLMs

Where to Find LLM Models: Hugging Face Hub

Understanding Model Sizes and Parameters

Model Formats: GGUF and Others

Quantization: Making Models Smaller

Reading Model Cards for Information

Model Licenses and Usage Restrictions

Choosing Your First Model

Chapter 4: Running Your First Local LLM

Introduction to Local LLM Runners

Setting up Ollama

Downloading a Model with Ollama

Running a Model with Ollama (Command Line)

Setting up LM Studio

Finding and Downloading Models in LM Studio

Loading and Chatting with a Model in LM Studio

Introduction to llama.cpp (Concept)

Hands-on Practical: Running a Model

Chapter 5: Basic Interaction and Prompting

What is a Prompt?

Your First Prompt: Simple Questions

Giving Instructions

Understanding Context Window

Basic Prompt Formatting Tips

Temperature and Creativity

Common Interaction Patterns

Practice: Simple Prompting Techniques

Finding and Selecting Local LLMs

Having prepared your system in the previous chapter, you now need a model to run. This chapter focuses on navigating the options for Large Language Models designed for local use. You will learn where to find models, primarily focusing on resources like the Hugging Face Hub.

We will examine key factors influencing your choice, including model size (often denoted by parameters, e.g., $7B$ or $13B$ ), file formats like GGUF which are optimized for local inference, and the concept of quantization used to reduce resource requirements. Furthermore, understanding model cards and software licenses is essential for appropriate selection and usage. The goal is to equip you with the criteria needed to select a suitable initial model that aligns with your hardware capabilities and objectives.

Sections

3.1 Where to Find LLM Models: Hugging Face Hub
3.2 Understanding Model Sizes and Parameters
3.3 Model Formats: GGUF and Others
3.4 Quantization: Making Models Smaller
3.5 Reading Model Cards for Information
3.6 Model Licenses and Usage Restrictions
3.7 Choosing Your First Model

© 2025 ApX Machine Learning

;