Managing Tool Inputs and Outputs through Prompts

Effectively wielding external tools involves more than just instructing an AI agent to call a specific function; it requires careful management of the data that flows into and out of these tools. Your prompts are the primary interface for this control, guiding the agent on how to structure inputs for precise tool operation and how to interpret the outputs to continue its task or inform the user. Since LLMs operate on text and tools often expect structured data like JSON or return complex responses, your prompts must bridge this gap.

Structuring Tool Inputs via Prompts

For an agent to use a tool correctly, it needs to provide inputs in the exact format the tool expects. Your prompts must guide the agent in preparing these inputs.

Explicit Instructions and Parameter Definition

The most direct way to guide input formatting is by explicitly detailing the required parameters, their data types, and any specific formatting rules within the prompt. This reduces ambiguity and increases the likelihood of the tool receiving valid data.

For instance, if you have a tool calculate_loan_payment that requires principal (number), annual_interest_rate (number, e.g., 0.05 for 5%), and loan_term_years (integer), your prompt might include:

To use the `calculate_loan_payment` tool, you must provide the following parameters:
- `principal`: The total loan amount (e.g., 20000).
- `annual_interest_rate`: The annual interest rate as a decimal (e.g., for 6.5% interest, provide 0.065).
- `loan_term_years`: The duration of the loan in years (e.g., 5).

The user wants to calculate payments for a $30,000 loan over 10 years at a 4.2% annual rate. Prepare the input for the `calculate_loan_payment` tool.

The agent should then be able to determine the correct values and structure for the tool call.

Generating Structured Inputs (JSON/XML)

Many tools, especially those interacting with APIs, expect inputs in structured formats like JSON or XML. You can instruct the agent to generate these structures directly. This is particularly useful when the tool input itself is complex or has nested elements.

Consider an agent needing to add an item to an e-commerce cart using a cart_add_item tool.

Prompt Snippet:

The `cart_add_item` tool expects a JSON object with `product_id` (string) and `quantity` (integer).
The user wants to add 2 units of product "A-B-123" to their cart. Generate the JSON input for the `cart_add_item` tool.

The agent's internal reasoning (or actual output if designed this way) should lead to something like:

{
  "tool_name": "cart_add_item",
  "tool_input": {
    "product_id": "A-B-123",
    "quantity": 2
  }
}

Your agent framework would then parse this JSON and execute the tool call.

Dynamic Input Generation from Context

Agents often need to extract input values from the ongoing conversation, previous tool outputs, or their internal memory. Prompts can guide this extraction process.

Example:

User: I'd like to book a flight from London to New York for next Friday.
Agent's Prompt: The user wants to book a flight. Use the `flight_search` tool.
It requires `origin_city`, `destination_city`, and `departure_date`.
Extract these details from the user's request and today's date (assume today is 2023-10-20) to determine "next Friday". Then, formulate the input for `flight_search`.

The agent would need to parse "London", "New York", calculate "next Friday" (e.g., 2023-10-27), and then construct the input.

Handling Optional Parameters and Defaults

Tools often have optional parameters with default values. Your prompts should make the agent aware of these, so it can decide whether to provide them or let the defaults apply.

Prompt Snippet for an image_resize tool:

The `image_resize` tool takes `image_url` (string, required), `width` (integer, optional), and `height` (integer, optional).
If only `width` is provided, maintain aspect ratio for `height`. If only `height` is provided, maintain aspect ratio for `width`.
If neither `width` nor `height` is provided, resize to a default of 512 pixels for the larger dimension, maintaining aspect ratio.
Resize the image at "http://example.com/image.jpg" to a width of 800 pixels.

This guides the agent to provide image_url and width, understanding that height will be handled appropriately by the tool or a subsequent processing step guided by the prompt.

Processing Tool Outputs with Prompts

Once a tool executes, it returns an output. This output might be a simple value, a complex JSON object, a text blob, or an error message. The agent needs guidance, again via prompts, on how to interpret and utilize this information.

Summarization and Information Extraction

Tool outputs, such as long API responses or web page content, can be verbose. You'll often need the agent to summarize this information or extract specific details relevant to its current task.

Example: A web_search tool returns a lengthy text snippet about a company. Prompt Snippet:

The `web_search` tool returned the following information about 'Innovatech Corp.':
"[Long text containing company history, products, and recent news...]"
Extract the CEO's name and summarize Innovatech's main product lines.

The agent should then process the text and provide a concise answer.

Format Conversion and Presentation

Tool outputs might not be in a user-friendly format. Prompts can instruct the agent to convert the output into natural language or a more suitable structured format for subsequent steps or user presentation.

Example: A database_query tool returns a list of product records. Tool Output:

[
  {"id": "P101", "name": "Wireless Mouse", "price": 25.99, "stock": 150},
  {"id": "P102", "name": "USB Keyboard", "price": 45.00, "stock": 80}
]

Prompt Snippet:

The `database_query` tool returned the above list of products. Present this information to the user as a readable summary, highlighting products with stock below 100.

The agent might then generate: "We found these products: The Wireless Mouse costs $25.99 and we have 150 in stock. The USB Keyboard is$45.00, and we have 80 in stock, which is below our threshold of 100."

Conditional Logic Based on Output

Agents often need to make decisions based on tool outputs. Prompts can define conditional logic for the agent to follow.

Example: A payment_process tool. Tool Output (Scenario 1 - Success): {"status": "success", "transaction_id": "txn_123abc"} Tool Output (Scenario 2 - Failure): {"status": "failed", "error_message": "Insufficient funds"}

Prompt Snippet:

The `payment_process` tool has completed.
If the `status` is "success", confirm the transaction ID to the user and proceed to the `shipping_安排` tool.
If the `status` is "failed", inform the user about the `error_message` and ask if they want to try a different payment method.
The tool returned: [Insert tool output here]
What is your next step?

This guides the agent's branching logic.

Interpreting Error Messages

Tools can fail, returning error messages. Prompts must equip the agent to understand common errors and suggest appropriate actions, such as retrying, asking the user for clarification, or trying an alternative approach.

Example: An api_call tool returns an error. Tool Output: {"error_code": 401, "message": "Unauthorized: API key invalid or expired."}

Prompt Snippet:

The `api_call` tool returned the following error:
`{"error_code": 401, "message": "Unauthorized: API key invalid or expired."}`
This typically means the provided API key is no longer valid. Inform the user about this specific issue and request them to provide an updated API key.

This allows the agent to handle errors gracefully instead of halting or providing unhelpful responses.

Managing Data Flow in Multi-Tool Interactions

In many agentic workflows, the output of one tool becomes the input for another, or influences the agent's subsequent decisions. Prompts are essential for managing this data flow and maintaining context across multiple steps.

Consider an agent tasked with planning a trip:

1. User Request: "Plan a weekend trip to a sunny beach destination within a $500 budget."
2. Prompt 1 (Goal Decomposition & Tool Selection): Agent identifies sub-tasks: find destinations, check flights, check hotels, verify budget.
3. Tool 1 (find_beach_destinations): Input derived from "sunny beach".
   • Prompt for Tool 1 Input: "Use find_beach_destinations with criteria: 'sunny'. Output should be a list of city names."
   • Output 1: ["Miami", "Cancun", "Malibu"]
4. Prompt 2 (Iterate & Tool 2 Selection): "For each destination from the list ["Miami", "Cancun", "Malibu"], use flight_search and hotel_search tools to find options for next weekend. Keep track of costs."
5. Tool 2 & 3 (flight_search, hotel_search): Inputs include destinations from Output 1.
   • Prompt for Tool 2/3 Inputs: "For 'Miami', find flights for [dates] and hotels for [dates]. Extract prices."
   • Outputs 2 & 3: Flight and hotel options with prices for Miami.
6. Prompt 3 (Aggregation & Decision): "Aggregate costs for Miami. Is it within the $500 budget? If yes, present it. If no, or if more options needed, repeat for 'Cancun'."

This sequence shows how prompts continuously guide the agent, ensuring that data (like destination names and prices) is passed and processed correctly through the workflow.

This diagram illustrates a multi-tool data flow where prompts guide the agent in using the output of one tool as input for subsequent tools, progressively building towards a solution.

Best Practices for Prompting Tool Input/Output

Mastering the management of tool inputs and outputs through prompts is an ongoing process of refinement. Here are some best practices to guide you:

• Be Explicit and Unambiguous: Clearly define data formats, required fields, optional fields, and potential values. The more specific your instructions, the less room for misinterpretation by the agent.
• Provide In-Prompt Examples (Few-Shot Learning): Show the agent exactly what you expect. For input generation, include a sample of the desired JSON or parameter structure. For output processing, show an example of raw output and the desired processed form.

  // Example for input formatting
  Tool: `add_event_to_calendar`
  Input format expected: {"event_title": "string", "date": "YYYY-MM-DD", "time": "HH:MM"}
  User says: "Add 'Team Meeting' to my calendar for tomorrow at 2 PM."
  (Assuming today is 2023-11-15)
  Generate the tool input.
  

• Instruct on Error Interpretation and Recovery: Don't assume tools will always succeed. Provide guidance on how the agent should interpret common error codes or messages and what fallback actions it can take (e.g., "If you receive a 404 error from the get_document tool, inform the user the document was not found and ask if they have an alternative ID.").
• Iterate and Test Rigorously: The first version of your prompt might not be perfect. Test your agent with various scenarios, observe how it handles tool inputs and outputs, and refine your prompts accordingly. Logging the agent's reasoning and the data exchanged with tools is invaluable here.
• Encourage Structured Output from the LLM for Internal Use: When the LLM processes a tool's output for further internal use (e.g., to feed into another tool or a decision-making process), prompt it to summarize or transform that output into a consistent, structured format (like JSON). This makes it easier for your agent's code to consume this information reliably.
• Consider the Context Window: If tool outputs are very large, be mindful of the LLM's context window. You may need to prompt the agent to summarize or extract only the most pertinent information to avoid exceeding limits or losing older context. Techniques for information condensation, covered in the chapter on memory, are relevant here.

By carefully designing prompts that manage the intricacies of tool inputs and outputs, you empower your AI agents to interact with the external world more effectively and reliably, significantly expanding their capabilities to perform complex tasks.