Overview of Agent Architectures

To effectively manage complex, multi-step tasks, AI agents often rely on structured operational frameworks known as agent architectures. Think of an architecture as a blueprint that defines how an agent's core components. such as its Large Language Model (LLM) brain, memory, and tools. interact to achieve a goal. These architectures provide a systematic way for agents to plan, execute actions, and adapt to new information. While various architectures exist, understanding one or two common patterns provides a solid foundation for prompt engineering in agentic systems.

The ReAct Framework: Reasoning and Acting

One influential and widely discussed agent architecture is ReAct, which stands for "Reasoning and Acting." Developed by researchers at Google Brain and Princeton University, ReAct enables LLMs to solve complex tasks by explicitly interleaving reasoning traces with task-specific actions. This approach allows the agent to generate internal thoughts about what to do next, take an action, observe the outcome, and then use that observation to inform its next thought and subsequent action. This iterative loop continues until the task is completed.

The core cycle of a ReAct agent typically involves three main components:

Thought: This is where the agent's LLM component performs internal reasoning. It analyzes the current task, the history of previous actions and observations, and the overall goal. Based on this, it formulates a plan or a rationale for the next action. Your prompts play a significant role in guiding this thought process, instructing the LLM on how to break down problems, what strategies to consider, and how to evaluate progress. For example, a thought might be: "I need to find the capital of France. I should use the search tool for this."
Action: Based on the "thought," the agent selects and executes an action. Actions are usually calls to predefined tools or functions available to the agent. Examples include querying a search engine, accessing a database, performing a calculation, or even asking the user for clarification. The prompt defines the available tools and the format the LLM should use to specify its chosen action and parameters. For instance: Action: search("capital of France").
Observation: After an action is performed, the agent receives an observation. This is the result or feedback from the executed action, such as search results, data retrieved from an API, the output of a computation, or an error message if the action failed. This observation is then fed back into the agent's context for the next "thought" step. For example: Observation: "Paris".

This Thought-Action-Observation loop allows the agent to dynamically plan and adjust its strategy. If an action doesn't yield the desired result, the agent can reason about the failure in the next thought step and try a different approach.

Let's illustrate with a simplified example. Suppose we want an agent to answer the question: "What is the population of the city where the Eiffel Tower is located?"

Initial Task: Answer "What is the population of the city where the Eiffel Tower is located?"
Loop 1:
- Thought: I need to first identify the city where the Eiffel Tower is located. I will use a search tool.
- Action: search("city of Eiffel Tower")
- Observation: "Paris, France"
Loop 2:
- Thought: Okay, the Eiffel Tower is in Paris. Now I need to find the population of Paris. I will use the search tool again.
- Action: search("population of Paris")
- Observation: "Approximately 2.1 million (as of 2023)"
Loop 3:
- Thought: I have found the city (Paris) and its population (2.1 million). I have all the information needed to answer the original question.
- Action: finish("The population of Paris, where the Eiffel Tower is located, is approximately 2.1 million.")

The ReAct framework provides a clear structure for agents to tackle problems that require multiple steps and interaction with external information sources. Prompt engineering is essential at each stage, from defining the initial task and available tools to guiding the LLM's reasoning process and helping it interpret observations.

Below is a diagram illustrating the general flow of the ReAct architecture:

The ReAct cycle: an iterative process of thought, action, and observation that guides the agent towards task completion.

Other Architectural Approaches

While ReAct is a prominent example, it's one of many approaches to designing agent architectures. Other architectures might emphasize different aspects, such as:

Hierarchical Planning: Agents that break down large goals into a hierarchy of sub-goals and sub-tasks, addressing each level systematically.
Multi-Agent Systems: Architectures designed for collaboration between multiple specialized agents to solve a larger problem.
State Machines: Agents whose behavior is modeled as a series of states and transitions, where prompts might define the conditions for transitions or the actions to take within a state.

The choice of architecture often depends on the complexity of the tasks, the types of tools available, and the desired level of autonomy and adaptability. However, many foundational principles, like the need for clear goal definition, action selection, and observation processing, are common across different agent designs.

Regardless of the specific architecture, prompt engineering remains a central mechanism for steering the agent. Your prompts will define the agent's "SOP" (Standard Operating Procedure) within its architectural framework, guiding its reasoning, decision-making, and interaction with its environment and tools. As you progress through this course, you'll learn how to craft prompts that effectively implement these elements within agentic systems. Understanding these underlying structures is important for designing prompts that lead to reliable and effective agent behavior.

Build LLM apps faster with Kerb

Cleaner syntax. Built-in debugging. Production-ready from day one.

Built for the AI systems behind ApX Machine Learning

Was this section helpful?

References

ReAct: Synergizing Reasoning and Acting in Language Models, Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao, 2022 arXiv preprint arXiv:2210.03629 DOI: 10.48550/arXiv.2210.03629 - The original research paper that introduces the ReAct framework, detailing its design for combining reasoning and acting in LLM-based agents.
A Survey of Large Language Model Based Agents, Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, Ji-Rong Wen, 2023 arXiv preprint arXiv:2308.11432 (cs) DOI: 10.48550/arXiv.2308.11432 - A broad survey providing an overview of different architectures and methodologies employed in large language model-based agents.
Toolformer: Language Models Can Teach Themselves to Use Tools, Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, Thomas Scialom, 2023 arXiv preprint arXiv:2302.04761 DOI: 10.48550/arXiv.2302.04761 - This paper demonstrates how LLMs can learn to interact with and use external tools, which is a fundamental aspect of the 'Action' component in agent architectures like ReAct.