Agent Mental States: Beliefs, Desires, Intentions

As we explore sophisticated reasoning in agent collectives, understanding the internal states that drive agent decisions becomes increasingly important. While Large Language Models (LLMs) don't possess consciousness or genuine mental states in a human sense, we can employ established agent models to structure their behavior in a way that emulates rational decision-making. The Belief-Desire-Intention (BDI) model, originating from philosophical theories of human practical reasoning and later adopted in artificial intelligence, offers a powerful framework for designing agents that act with apparent purpose and deliberation.

In the BDI architecture, an agent's behavior is primarily driven by three components:

• Beliefs: These represent the agent's information about the current state of the world, including itself and other agents. For an LLM agent, beliefs can be an amalgamation of its pre-trained knowledge, information provided in its current context window, data retrieved from external knowledge bases (e.g., vector databases through RAG), and facts derived from previous interactions or tool use.
• Desires: These are the agent's goals or objectives, representing states of the world it wishes to bring about. In LLM systems, desires are often explicitly assigned by developers through initial prompts, system objectives, or user requests. They can range from simple tasks like "summarize this document" to complex, multi-step goals.
• Intentions: These are desires that the agent has committed to achieving. An intention signifies a chosen course of action or a plan the agent is actively pursuing. Intentions are more robust than mere desires; an agent typically maintains an intention until it's fulfilled, becomes unachievable, or is superseded by a more pressing intention.

These three components interact within a reasoning cycle that allows the agent to perceive its environment, update its beliefs, deliberate on its desires, commit to intentions, and then execute actions to fulfill those intentions.

The BDI model provides a structure for an agent's decision-making process, where beliefs inform desires, leading to intentions and subsequent actions. The practical reasoner, often leveraging the LLM, orchestrates this flow.

Beliefs in LLM Agents

For an LLM-based agent, "beliefs" are not stored in a discrete symbolic database as in traditional BDI systems. Instead, they are a composite of:

1. Parametric Knowledge: The vast information encoded within the LLM's weights during pre-training. This forms a general, often implicit, world model.
2. Contextual Information: Data provided in the current prompt, including task instructions, recent conversation history, and specific facts relevant to the ongoing interaction. This acts as a short-term, working memory for beliefs.
3. Retrieved Knowledge: Information dynamically fetched from external sources like vector databases, structured databases, or APIs. This allows beliefs to be grounded in up-to-date or domain-specific information.
4. Inferred Knowledge: New facts or understanding derived by the LLM itself through reasoning over the available information (e.g., using chain-of-thought prompting).

Managing beliefs effectively requires mechanisms to update them based on new perceptions (e.g., API call results, messages from other agents) and to resolve inconsistencies. For instance, an LLM might be prompted to "update its understanding" based on new data, or a supervising process might curate and filter information before it solidifies as a working belief. The reliability of these beliefs is directly tied to the LLM's grounding capabilities and its susceptibility to generating inaccurate information.

Desires as Agent Motivators

Desires represent the motivational aspect of the agent. In most current multi-agent LLM systems, desires are not emergent but are typically:

• Programmatically Defined: Hardcoded high-level goals or objectives for the agent.
• User-Instigated: Derived from direct user requests or instructions.
• System-Driven: Arising from the agent's role within a larger workflow (e.g., an "analyst" agent desires to process data provided to it).

A critical aspect is managing multiple, potentially conflicting desires. An agent might need to prioritize desires based on urgency, importance, or dependencies. This prioritization logic can be explicitly coded or, more flexibly, an LLM itself can be prompted to perform this prioritization given a set of desires and the current beliefs. For example, an agent might have a standing desire to minimize cost, which could conflict with a desire to achieve a task quickly. Deliberation is needed to reconcile such conflicts.

Intentions: Committing to Action

Intentions bridge the gap between deliberation and execution. When an agent forms an intention, it commits to a course of action to achieve a specific desire. This commitment is significant because it implies persistence:

• The agent should pursue its intentions until they are achieved.
• If an action fails, the agent (or its orchestrator) should ideally re-plan or attempt alternative actions to satisfy the intention.
• Intentions are only dropped if they are fulfilled, become impossible to achieve, or are explicitly overridden by a new, higher-priority intention.

For LLM agents, an intention often manifests as a generated plan or a sequence of steps (e.g., a ReAct trajectory). The LLM is prompted to not just identify what to do, but to commit to doing it. Maintaining intentions across multiple turns or LLM calls requires external state management, where the current set of active intentions is stored and referenced to guide subsequent LLM prompts.

For example, if an agent intends to "book a flight to New York for next Tuesday," its practical reasoner (the LLM guided by specific prompts) might generate a plan:

1. Check flight availability for the specified date.
2. If flights are available, present options to the user.
3. If user confirms, proceed to book the selected flight. This sequence represents the agent's active intention.

The BDI Reasoning Cycle with LLMs

The BDI model operates on a continuous reasoning cycle. While not a literal software loop inside the LLM, this cycle can be emulated through careful orchestration of LLM calls and state management:

1. Perception & Belief Update: The agent receives inputs (user queries, sensor data via tools, messages from other agents). The LLM processes this new information, potentially updating its understanding of the world (beliefs). This might involve RAG calls or summarizing new data into its working context.
   • Example: Prompt: "You are a research assistant. Your current understanding is X. You've just received new report Y. How does Y change your understanding?"
2. Option Generation (Desire Deliberation): Based on current beliefs and active desires, the agent considers possible courses of action or sub-goals. The LLM can be prompted to generate these options.
   • Example: Prompt: "Given your goal to Z and current situation B, what are the possible next steps you could take?"
3. Filtering & Intention Formation: The agent evaluates the generated options against its desires and beliefs, selecting one or more options to commit to as intentions. This is a crucial step where the LLM's reasoning capabilities are leveraged for decision-making.
   • Example: Prompt: "Considering options O1, O2, O3, and your primary desire D, which option is the most effective to pursue now? Explain your reasoning and state your chosen intention."
4. Planning & Execution: Once an intention is formed, the agent (or the LLM) devises a plan to achieve it. This plan is then executed, often involving tool use, API calls, or generating messages. The outcomes of actions feed back into the perception phase.
   • Example (leading to ReAct-style output): Prompt: "Your intention is I. What is the first action you need to take? Specify the tool and parameters."

This cycle highlights that a BDI-inspired LLM agent is more than just a single LLM call. It involves a structured process, often managed by an overarching control loop or framework (like LangChain or AutoGen) that maintains state and directs the LLM's focus at each stage.

Implementing BDI Concepts with LLM Agents: Practical Considerations

Applying BDI principles to LLM agents is less about direct implementation of classical BDI data structures and more about using the model as an architectural guide:

• Belief Management: Combine the LLM's internal knowledge with external, dynamically updated belief stores (e.g., a vector database for facts, a simple key-value store for agent state). The context window serves as the immediate "attentional focus" for beliefs. Ensure mechanisms for updating these beliefs based on new information or LLM inferences.
• Desire Representation: Clearly articulate agent goals in prompts. For complex tasks, decompose high-level desires into hierarchies of sub-goals. Use prompt engineering to allow agents to reason about goal priorities if necessary.
• Intention Handling: Explicitly track committed goals (intentions). An LLM can generate a plan (a sequence of intended actions) based on an intention. Orchestration logic must monitor plan execution, handle failures (triggering replanning by the LLM), and recognize intention fulfillment.
• Deliberation via LLM: Use the LLM itself for the deliberation process. Prompts can be designed to make the LLM "think through" options, weigh pros and cons based on its beliefs and desires, and then decide on an intention. Structured output formats (e.g., JSON) can help in extracting the chosen intention and plan.

Advantages and Challenges

Adopting a BDI-like structure for LLM agents offers several advantages:

• Structured Rationality: It provides a clear model for how an agent should make decisions, leading to more predictable and understandable behavior.
• Goal-Directedness: Agents are driven by explicit goals (desires and intentions), making them more focused and effective in complex, long-running tasks.
• Improved Explainability: By making beliefs, desires, and intentions more explicit (even if managed externally to the LLM core), it can be easier to trace why an agent took a particular action.
• Foundation for Advanced Behaviors: BDI supports planning, replanning, and adaptation, which are essential for more autonomous agents.

However, there are challenges:

• Implicit Nature in LLMs: LLMs do not inherently possess BDI components. These must be architected around the LLM using prompting, external memory, and control logic. The "mental state" is an interpretation we impose.
• State Management Overhead: Persisting and updating BDI components across multiple, often stateless, LLM API calls requires robust external state management. This can add complexity to the system.
• Computational Cost of Deliberation: Using LLMs for every step of deliberation (option generation, filtering, planning) can be slow and expensive in terms of token usage and latency. Simpler heuristics or more lightweight models might be needed for certain deliberation steps.
• Grounding and Consistency: Ensuring that LLM-generated beliefs are accurate and that its reasoning from beliefs to intentions is sound remains a significant challenge, directly tied to issues like hallucination and logical fallibility.
• Scalability for Numerous Agents: Managing distinct BDI states for a large collective of agents can be demanding.

BDI in Multi-Agent Systems

In a multi-agent context, the BDI framework can extend to inter-agent communication and coordination. Agents might communicate their:

• Beliefs: Sharing information about the environment or task.
• Desires: Announcing goals to find collaborators.
• Intentions: Declaring commitments to avoid conflicts or redundant work.

For example, an agent intending to access a shared resource could announce its intention, allowing other agents to adjust their own plans. Negotiation protocols can be designed where agents exchange proposals based on their individual BDI states to reach mutually agreeable outcomes.

While the BDI model offers a structured approach to agent design, its application with LLMs is an abstraction that helps organize the flow of information and decision-making. It guides how we prompt LLMs and how we build the surrounding infrastructure to support more reasoned and autonomous behavior. As LLMs continue to evolve, their capacity to internally manage states akin to beliefs, or to perform more sophisticated deliberation, may increase, potentially simplifying the explicit BDI scaffolding required today. For now, BDI remains a valuable guiding framework for engineers building the next generation of intelligent LLM-powered agent systems.