Remembering Recent Events

Imagine trying to have a conversation with someone who forgets everything you said a moment ago. Each sentence would be a fresh start, making any meaningful exchange impossible. LLM agents, without a way to remember recent interactions, would face a similar challenge. This ability to "remember recent events" is what we refer to as short-term memory in the context of an LLM agent. It's a fundamental component that allows an agent to maintain context during an ongoing operation or conversation.

Why Agents Need to Keep Track

Unlike the LLM's pre-trained knowledge which is static, short-term memory is dynamic. It pertains to the specific task at hand or the current interactive session. Think of it as the agent's scratchpad, where it jots down notes about what has just happened. This is not about learning new facts permanently, like the capital of France, which the LLM already knows. Instead, it's about remembering that you just asked about flights to Paris in this conversation, so when you follow up with "What about hotels there?", the agent understands "there" refers to Paris.

Without this memory, every interaction with the agent would be isolated. The agent wouldn't be able to:

• Understand follow-up questions.
• Refer to information provided earlier in the conversation.
• Perform tasks that require multiple steps where context from previous steps is important.
• Maintain a coherent "personality" or state throughout an interaction.

Essentially, short-term memory is what gives an agent a sense of continuity. It allows the agent to build upon previous exchanges, making the interaction smoother and more effective.

How Short-Term Memory Works (A Simple View)

At a basic level, short-term memory for an LLM agent often involves keeping a log or history of recent interactions. This history typically includes:

• The user's inputs.
• The agent's own responses or actions.

When the agent needs to process a new input from the user, it doesn't just look at that single piece of information in isolation. Instead, it (or the system controlling it) provides the LLM not only with the new input but also with some or all of this recent history. This combined information forms a richer prompt, giving the LLM the necessary context to generate a relevant and informed response.

For example, if you're interacting with an agent:

1. You: "Add 'buy milk' to my to-do list."
2. Agent (records in memory): User said "Add 'buy milk' to my to-do list". Agent responded "Okay, 'buy milk' added."
3. You: "Also add 'call John'."
4. Agent (when processing "Also add 'call John'"): The LLM receives "Also add 'call John'" and the memory of the previous interaction about milk. This helps it understand that "also add" refers to the to-do list.

The following diagram illustrates how short-term memory fits into an agent's operational flow:

This diagram shows the cycle where user input is combined with existing short-term memory to form a complete prompt for the LLM. The LLM's output is then used to respond to the user and also to update the short-term memory for future turns.

Keeping it "Short-Term"

The term "short-term" is significant. LLMs have limitations on how much text they can process at once (often called the "context window"). If the conversation history becomes too long, it might exceed this limit. Therefore, practical implementations of short-term memory often involve strategies like:

• Keeping only the last few turns of a conversation.
• Summarizing older parts of the conversation.
• Using more sophisticated techniques to retrieve relevant pieces of past information (which we'll touch upon in later chapters).

For now, the important idea is that an agent needs some mechanism to remember recent events to function intelligently. This capability, even in its simplest form, transforms an LLM from a stateless text generator into a more useful and interactive assistant. As we build our first agents, we'll see how even basic memory makes a big difference. In Chapter 6, "LLM Agent Memory: Remembering Information," we will explore these mechanisms in much greater detail.