Utilizing Few-Shot Examples for Agent Guidance

General instructions and role assignments provide a strong foundation for agent control. To guide an agent's behavior with more concrete illustrations, utilizing few-shot examples directly within prompts becomes an invaluable technique. This approach shows an agent how to perform a task through demonstration, rather than solely telling it what to do.

Few-shot learning, in the context of Large Language Models (LLMs) and agentic systems, refers to providing the model with a small number of examples (the "shots") of the desired input-output behavior. This approach helps the agent understand patterns, formats, and even implicit reasoning steps, often more effectively than lengthy descriptive instructions alone. It's a middle ground between zero-shot prompting (where the agent relies solely on the instruction) and extensive fine-tuning (which involves retraining the model on a large dataset). For agentic workflows, few-shot examples offer a pragmatic way to achieve specific behaviors without the overhead of model retraining.

Why Few-Shot Examples Enhance Agent Guidance

Agents powered by LLMs are adept at pattern recognition. By providing a few well-crafted examples, you tap into this capability, allowing the agent to infer the desired course of action. There are several reasons why this method is effective for agent control:

Clarifying Ambiguity: Instructions can sometimes be open to interpretation. Examples provide concrete instances of what you expect, reducing ambiguity. If an agent needs to call a tool with specific parameter formatting, showing an example of that format is far clearer than describing it abstractly.
Demonstrating Complex Logic: For tasks requiring a particular sequence of thought or a specific way of breaking down a problem, examples can illustrate this logic implicitly. This is particularly useful when the reasoning is subtle and hard to capture in explicit instructions.
Specifying Output Formats: If your agent needs to produce output in a structured format (e.g., JSON, XML, or a custom schema), few-shot examples are an excellent way to define that structure. The agent learns the format by seeing it.
Adapting to Details: Agents can be guided to adopt specific styles, tones, or to handle edge cases by including examples that demonstrate these details. For instance, if an agent should respond empathetically to certain user queries, an example can show this empathetic response style.
Improving Reliability: By providing a template for behavior, few-shot examples can lead to more consistent and reliable agent performance, especially for repetitive aspects of a task.

Designing Effective Few-Shot Examples

The success of few-shot prompting relies on the quality and relevance of the examples provided. Here are some guidelines for designing effective examples for agent guidance:

Quality over Quantity: A small number of high-quality, diverse, and representative examples (typically 2 to 5) is often more effective than a large number of redundant or poorly constructed ones. Each example consumes token space in your prompt, so be economical.
Relevance is Important: Ensure your examples directly pertain to the task the agent is expected to perform. Irrelevant examples can confuse the agent or lead it astray.
Consistency in Structure: Maintain a consistent structure across your examples. If an example shows Input -> Thought -> Action -> Output, all examples should follow this pattern. This helps the agent learn the expected sequence.
Clarity of Intent: Each part of the example should clearly demonstrate the desired behavior. If you're showing how to use a tool, the tool call and its parameters should be unambiguous.
Cover Necessary Variations: If the task has common variations or requires different handling based on input, include examples that cover these different scenarios. This helps the agent generalize better. For instance, show how to handle missing information or different types of user requests.
Positive and Negative Examples (Optional but Powerful): While less common in basic few-shot setups, for complex tasks, you might consider showing an example of what not to do, alongside corrections, though this requires careful framing. More typically, you'd focus on positive exemplars.

Structuring Few-Shot Examples in Prompts

When incorporating few-shot examples, you typically present them as a preamble before the actual task or query the agent needs to address. The structure of each example should be clear and mimic the process you want the agent to follow.

A common structure for an example within a prompt might look like this:

[Optional Preamble: "Here are some examples of how to handle X:"]

Example 1:
User Query: [Example of a user's request or input]
Thought: [Optional: A brief description of the agent's reasoning process or plan. This can guide the agent's internal "monologue" if you're aiming for specific reasoning patterns like Chain-of-Thought.]
Tool Call: [Example of a tool invocation, e.g., search("query") or api_call(endpoint="...", params={...})]
Tool Observation: [Example of the result/data returned by the tool]
Agent Response: [Example of the final response to the user, or an internal summary]

Example 2:
User Query: [...]
Thought: [...]
Tool Call: [...]
Tool Observation: [...]
Agent Response: [...]

[End of examples]

Current Task:
User Query: [The actual, current user query for the agent to process]
Thought: [Agent fills this in]
Tool Call: [Agent fills this in]
Agent Response: [Agent fills this in]

The agent is expected to follow the pattern established by the examples when it processes the "Current Task."

Let's consider an agent designed to extract contact information from text and format it as JSON.

Zero-Shot Attempt (Potentially Ambiguous):

Extract contact information (name, email, phone) from the following text and return it as JSON.
Text: "Contact Jane Doe at [email protected] or (555) 123-4567 for details."

The agent might get this right, but the exact JSON structure, field names, and handling of missing information are not explicitly defined.

Few-Shot Approach (Clearer Guidance):

You are an expert contact information extractor. Given a text, extract the name, email, and phone number. Format the output as a JSON object. If a piece of information is not found, use null for its value.

Example 1:
Text: "Reach out to John Smith via [email protected]. His number is 123-456-7890."
Output:
{
  "name": "John Smith",
  "email": "[email protected]",
  "phone": "123-456-7890"
}

Example 2:
Text: "For support, email [email protected]."
Output:
{
  "name": null,
  "email": "[email protected]",
  "phone": null
}

Current Task:
Text: "Contact Jane Doe at [email protected] or (555) 123-4567 for details."
Output:

By providing these examples, the agent receives clear guidance on:

The desired JSON keys (name, email, phone).
How to represent missing information (null).
The overall structure of the output.

This significantly increases the likelihood of the agent producing the correct output consistently.

When to Employ Few-Shot Examples

Few-shot examples are particularly beneficial in scenarios such as:

Tool Usage Specification: When an agent needs to call a specific tool with precise argument names and formats.
Output Formatting: To enforce a strict output structure, like JSON, XML, or a specific textual template.
Demonstrating Reasoning: When you want the agent to follow a particular line of reasoning or problem decomposition (e.g., showing a simplified Chain-of-Thought).
Style and Tone Adaptation: To guide the agent's communication style (e.g., formal, informal, empathetic).
Handling Ambiguity or Edge Cases: When instructions alone are insufficient to cover all necessary variations or tricky situations.
Improving Task Success Rate: If zero-shot prompting yields inconsistent or incorrect results, few-shot examples can often stabilize and improve performance.

Interaction with Other Prompting Techniques

Few-shot examples do not exist in isolation; they can be powerfully combined with other advanced prompting strategies discussed in this chapter.

Roles and Personas: Examples can demonstrate how an assigned role or persona should behave in specific situations. If an agent is a "helpful customer service assistant," examples can show polite and solution-oriented interactions.
Chain-of-Thought (CoT) / Tree-of-Thought (ToT): You can explicitly include a "Thought:" or "Reasoning:" step in your few-shot examples to demonstrate the desired thought process. This guides the agent not just on the final action or output, but also on the intermediate reasoning steps.

For example, combining role-play, CoT, and few-shot for a troubleshooting agent:

You are Dr. Network, an expert network troubleshooter. When a user describes a problem, first think step-by-step to diagnose, then suggest an action.

Example 1:
User: "My internet is down."
Dr. Network's Thought:
1. Check physical connection: Is the modem/router powered on? Lights okay?
2. Check local network: Can other devices connect?
3. Check external service: Is there a known outage?
Dr. Network's Action: "First, please check if your modem and router have power and if the status lights look normal. Are other devices in your home also unable to connect?"

Current Problem:
User: "I can't access my email."
Dr. Network's Thought:
Dr. Network's Action:

Limitations and Challenges

While effective, few-shot prompting has some considerations:

Token Consumption: Examples add to the length of your prompt, consuming tokens from the LLM's context window. This is an important factor for models with smaller context limits or when trying to optimize for cost and latency.
Risk of Overfitting: If your examples are too narrow or too specific, the agent might "overfit" to them and struggle with inputs that deviate even slightly. Aim for examples that cover a reasonable range of expected inputs.
Example Quality is Crucial: The performance of few-shot prompting is highly dependent on the quality of the examples. Poorly chosen or misleading examples can degrade performance.
Maintenance: If the task, tools, or desired outputs evolve, your few-shot examples will also need to be updated to remain relevant. This adds a maintenance aspect to your prompt engineering efforts.

By thoughtfully incorporating few-shot examples, you gain a finer degree of control over your agent's behavior, enabling it to perform complex tasks with greater accuracy and consistency. This technique is a practical step towards building more sophisticated and reliable agentic workflows. As you move forward, remember that the art of prompt engineering often involves iterating on these examples, observing agent behavior, and refining your demonstrations until the desired performance is achieved.

Build LLM apps faster with Kerb

Cleaner syntax. Built-in debugging. Production-ready from day one.

Built for the AI systems behind ApX Machine Learning

Was this section helpful?

References

Language Models are Few-Shot Learners, Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei, 2020 Advances in Neural Information Processing Systems (NeurIPS) DOI: 10.48550/arXiv.2005.14165 - This paper introduced the concept of few-shot learning for large language models, demonstrating their ability to perform new tasks with only a few examples without fine-tuning.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, 2022 arXiv DOI: 10.48550/arXiv.2201.11903 - This paper introduced Chain-of-Thought prompting, a technique that improves LLM reasoning by explicitly showing intermediate reasoning steps, which can be combined with few-shot examples for agent guidance.
ReAct: Synergizing Reasoning and Acting in Language Models, Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao, 2023 International Conference on Learning Representations (ICLR) DOI: 10.48550/arXiv.2210.03629 - This paper presents the ReAct framework, which combines reasoning and acting for LLM agents. Few-shot examples are often used within ReAct to guide tool use and decision-making for complex tasks.
Prompt Engineering Guide, OpenAI, 2023 (OpenAI) - Official guide from OpenAI offering practical techniques and best practices for prompt engineering, including the effective use of few-shot examples for model control.