Downloading a Model with Ollama

Downloading a Large Language Model (LLM) for interaction is a primary task when working with local LLMs. Ollama facilitates this process, allowing you to download models using a command in your terminal or command prompt.

The primary command for fetching models is ollama pull. You use this command followed by the name of the model you wish to download. Models on the Ollama Hub are typically identified by a name and often a tag, similar to how Docker images are tagged. The tag usually specifies the version, size, or variant of the model. The format looks like this:

ollama pull <model_name>:<tag>

Let's break this down:

• <model_name>: This is the identifier for the model family, such as llama3, mistral, phi3, etc. You can find available models on the Ollama website's model library.
• <tag>: This specifies a particular version or configuration of the model. Common tags indicate:
  • Size: Like 7b (7 billion parameters), 8b (8 billion parameters), 13b (13 billion parameters). Smaller models require less RAM and disk space but might be less capable than larger ones.
  • Variant: Such as instruct (tuned for following instructions) or chat (tuned for conversation).
  • Quantization Level: Sometimes tags indicate specific quantization levels (like q4_0, q5_K_M), although Ollama often selects a suitable default quantization. We touched on quantization in Chapter 3.
  • Version: If a model gets updated, tags might differentiate versions (e.g., 0.1).
  • latest: If you omit the tag, Ollama usually defaults to downloading the latest tag, which generally points to a recent, recommended version. However, it's often better to be specific about the model size you want, matching it to your hardware capabilities.

Downloading Your First Model

Let's try downloading a popular and capable model suitable for a range of systems: Llama 3 with 8 billion parameters. Open your terminal or command prompt and enter the following command:

ollama pull llama3:8b

Press Enter. Ollama will now connect to its library and begin the download process. You should see output similar to this (the exact details like layer IDs and sizes will vary):

pulling manifest
pulling 174b62a2b47d... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 4.7 GB                         
pulling 8c35f094345b... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏  11 KB                         
pulling 294572a8cc96... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏  485 B                         
pulling 58cd7654e763... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏   59 B                         
pulling e446f111b7e4... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏  483 B                         
verifying sha256 digest
writing manifest
removing any unused layers
success

Here's what's happening:

1. pulling manifest: Ollama first fetches a small "manifest" file. This file contains information about the model, including the different files (layers) that make it up.
2. pulling <layer_id>...: The model itself is downloaded in parts, often called layers. You'll see progress bars for each layer. The size (e.g., 4.7 GB in the example) indicates the amount of data being downloaded. This step can take some time depending on your internet speed and the model's size.
3. verifying sha256 digest: After downloading, Ollama checks the integrity of the files to ensure they weren't corrupted during the download.
4. writing manifest: The manifest file is saved locally.
5. removing any unused layers: Ollama might clean up temporary files.
6. success: This message confirms that the model has been successfully downloaded and is ready to use.

Ollama manages the storage of these models for you, typically placing them in a hidden directory within your user profile (like ~/.ollama/models on Linux/macOS or C:\Users\<username>\.ollama\models on Windows). You generally don't need to interact with these files directly.

With the model successfully downloaded, you're now ready to run it and start interacting with your first local LLM using the Ollama command line, which we'll cover in the next section.