Building upon the foundational concepts of agent components and basic interaction loops, this chapter focuses on the architectures that enable sophisticated reasoning and decision-making within LLM agents. Simple prompt-action cycles often struggle with complex tasks requiring detailed planning or exploration of different possibilities. Here, we examine specific frameworks designed to structure the agent's thought process, leading to more effective problem-solving.

You will learn to analyze and implement several key agent architectures:

• ReAct (Reason + Act): Understand how this framework integrates reasoning steps with actions ($a_i$) based on observations ($o_i$) to dynamically create and adjust plans. We'll study its characteristic thought-action-observation loop.
• Self-Ask: See how agents can break down complex queries into simpler, dependent questions, aiming for improved factual grounding before synthesizing a final answer.
• Tree of Thoughts (ToT): Study how this method allows agents to explore multiple reasoning paths concurrently, evaluate intermediate "thoughts" as nodes in a tree, and use search strategies (like BFS or DFS) over the thought space, often represented as a graph $G=(V, E)$.
• Graph-Based Reasoning: Investigate the application of explicit graph structures for managing intricate reasoning states, plans, and dependencies within an agent.

This chapter provides implementation details, comparative analysis of these reasoning frameworks, and practical exercises, such as building a custom ReAct agent. The goal is to equip you with the knowledge to select and construct appropriate reasoning mechanisms for advanced agentic systems.