Getting Started with Local LLMs
Chapter 1: Introduction to Large Language Models
What Is a Large Language Model (LLM)?
A Simple View of How LLMs Work
Understanding Tokens and Text Generation
Why Run LLMs Locally?
Local vs. Cloud-Based LLMs
Quiz for Chapter 1
Chapter 2: Preparing Your Local Environment
Hardware Considerations: CPU
Hardware Considerations: RAM
Hardware Considerations: GPU and VRAM
Checking Your System Specifications
Operating System Compatibility
Installing Python (Optional but Recommended)
Introduction to the Command Line / Terminal
Quiz for Chapter 2
Chapter 3: Finding and Selecting Local LLMs
Where to Find LLM Models: Hugging Face Hub
Understanding Model Sizes and Parameters
Model Formats: GGUF and Others
Quantization: Making Models Smaller
Reading Model Cards for Information
Model Licenses and Usage Restrictions
Choosing Your First Model
Quiz for Chapter 3
Chapter 4: Running Your First Local LLM
Introduction to Local LLM Runners
Setting up Ollama
Downloading a Model with Ollama
Running a Model with Ollama (Command Line)
Setting up LM Studio
Finding and Downloading Models in LM Studio
Loading and Chatting with a Model in LM Studio
Introduction to llama.cpp (Concept)
Hands-on Practical: Running a Model
Quiz for Chapter 4
Chapter 5: Basic Interaction and Prompting
What is a Prompt?
Your First Prompt: Simple Questions
Giving Instructions
Understanding Context Window
Basic Prompt Formatting Tips
Temperature and Creativity
Common Interaction Patterns
Practice: Simple Prompting Techniques
Quiz for Chapter 5
Choosing Your First Model
Was this section helpful?
- llama.cpp, Georgi Gerganov and the llama.cpp Contributors, 2023 - Source project for the GGUF format, detailing quantization techniques for efficient LLM inference.
- Mistral 7B, Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed, 2023 arXiv DOI: 10.48550/arXiv.2310.06825 - Introduces the Mistral 7B model, recognized for its strong performance at a smaller size, suitable for local use.
- Textbooks Are All You Need II: phi-1.5 Technical Report, Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, Yin Tat Lee, 2023 arXiv preprint arXiv:2309.05463 DOI: 10.48550/arXiv.2309.05463 - Explains the data-centric training approach for small, capable language models like Phi, emphasizing efficiency.