Crafting prompts helps tell your local Large Language Model (LLM) what you want it to do. Asking clear questions and giving specific instructions are primary skills for this. However, sometimes you might notice the LLM's responses vary in style. They might be very straightforward and predictable, or perhaps more imaginative and unexpected. A parameter often called "temperature" is one factor influencing this.
Think of temperature as a control knob for the randomness or "creativity" of the LLM's output. When generating text, the model constantly calculates probabilities for what the next word (or token, more accurately) should be. The temperature setting adjusts how the model uses these probabilities.
Low Temperature (e.g., values closer to 0, like 0.1 to 0.5): At low temperatures, the model becomes more deterministic and focused. It strongly prefers the words with the highest calculated probability. This makes the output highly predictable, consistent, and often repetitive. If you ask the same question multiple times with a very low temperature, you're likely to get nearly identical answers. This setting is useful for tasks where precision and sticking to the facts are important, like extracting specific information or generating code based on strict rules.
High Temperature (e.g., values greater than 1, like 1.1 to 2.0): At high temperatures, the model becomes more adventurous. It increases the chances of selecting less likely words, effectively flattening the probability distribution. This leads to more randomness, surprise, and creativity in the output. However, if the temperature is set too high, the generated text can become incoherent, nonsensical, or lose track of the original prompt's context. High temperatures can be interesting for brainstorming, writing fiction, or generating diverse ideas, but require careful handling.
Medium Temperature (e.g., values around 0.7 to 1.0): This range often provides a balance between predictability and creativity. The model follows the most likely paths but allows for some variation. Many chat applications use a default temperature in this range for general-purpose conversation.
Let's illustrate this with a simplified example. Imagine the model has processed the input "The cat sat on the" and needs to choose the next word. It calculates probabilities for potential words:
| Word | Probability (Original, T=1.0) | Probability (Low T, e.g., 0.2) | Probability (High T, e.g., 1.5) |
|---|---|---|---|
| mat | 0.80 | ~0.99 | ~0.60 |
| chair | 0.15 | ~0.01 | ~0.25 |
| windowsill | 0.04 | <0.01 | ~0.10 |
| moon | 0.01 | <0.01 | ~0.05 |
At low temperature, the model almost certainly picks "mat". At high temperature, "chair", "windowsill", or even the nonsensical "moon" become more plausible options for the model to choose, leading to more varied (and potentially strange) outputs.
Probabilities for the next word after "The cat sat on the" at different temperature settings. Low temperature strongly favors the most likely word ("mat"), while high temperature makes less common words more probable.
You won't always see a temperature setting directly, especially in the simplest command-line interactions. However, graphical tools like LM Studio typically expose it prominently. Look for a slider or input field labeled "Temperature" in the chat or model settings panel. It usually allows values between 0 and 2. Some command-line tools or libraries (like Ollama when used via its API) also allow you to set temperature, but it might require specific flags or configuration options.
Choosing the right temperature depends on your goal:
Use Low Temperature (0.1 - 0.5) for:
Use Medium Temperature (0.6 - 1.0) for:
Use High Temperature (1.1 - 2.0) for:
The best way to understand temperature is to experiment. Try the same prompt with different temperature settings in a tool like LM Studio:
Temperature is one of the primary ways you can influence the style and variability of your LLM's output, complementing the content control you exercise through prompting. Understanding and adjusting it gives you another layer of control over your interactions with local LLMs.
Was this section helpful?
transformers library.© 2026 ApX Machine LearningEngineered with