Getting Started with Local LLMs
Chapter 1: Introduction to Large Language Models
What Is a Large Language Model (LLM)?
A Simple View of How LLMs Work
Understanding Tokens and Text Generation
Why Run LLMs Locally?
Local vs. Cloud-Based LLMs
Quiz for Chapter 1
Chapter 2: Preparing Your Local Environment
Hardware Considerations: CPU
Hardware Considerations: RAM
Hardware Considerations: GPU and VRAM
Checking Your System Specifications
Operating System Compatibility
Installing Python (Optional but Recommended)
Introduction to the Command Line / Terminal
Quiz for Chapter 2
Chapter 3: Finding and Selecting Local LLMs
Where to Find LLM Models: Hugging Face Hub
Understanding Model Sizes and Parameters
Model Formats: GGUF and Others
Quantization: Making Models Smaller
Reading Model Cards for Information
Model Licenses and Usage Restrictions
Choosing Your First Model
Quiz for Chapter 3
Chapter 4: Running Your First Local LLM
Introduction to Local LLM Runners
Setting up Ollama
Downloading a Model with Ollama
Running a Model with Ollama (Command Line)
Setting up LM Studio
Finding and Downloading Models in LM Studio
Loading and Chatting with a Model in LM Studio
Introduction to llama.cpp (Concept)
Hands-on Practical: Running a Model
Quiz for Chapter 4
Chapter 5: Basic Interaction and Prompting
What is a Prompt?
Your First Prompt: Simple Questions
Giving Instructions
Understanding Context Window
Basic Prompt Formatting Tips
Temperature and Creativity
Common Interaction Patterns
Practice: Simple Prompting Techniques
Quiz for Chapter 5
Loading and Chatting with a Model in LM Studio
Was this section helpful?
- LM Studio Official Website, LM Studio Team, 2024 - Provides the official software, guides, and community resources for LM Studio, essential for downloading, managing, and interacting with local large language models.
- llama.cpp GitHub Repository, Georgi Gerganov and the llama.cpp contributors, 2024 - The foundational project for efficient CPU/GPU inference of large language models, including the development of the GGUF file format used by LM Studio.
- A Survey of Large Language Models, Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, Ji-Rong Wen, 2023 arXiv preprint arXiv:2303.18223 DOI: 10.48550/arXiv.2303.18223 - Provides a comprehensive overview of large language models, covering their architectures, training, capabilities, and applications, useful for understanding the underlying technology.
- QLoRA: Efficient Finetuning of Quantized LLMs, Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer, 2023 arXiv preprint arXiv:2305.14314 DOI: 10.48550/arXiv.2305.14314 - Introduces techniques for efficient finetuning of quantized large language models, providing insights into how quantization reduces memory footprint and impacts performance for local deployment.