How AgentCore Episodic Memory Makes AI Agents Smarter Over Time

Problem

Most AI agents rely only on information visible in the current interaction. They can retrieve facts but cannot remember how similar problems were solved previously, nor understand why certain methods succeeded or failed. This limits learning, knowledge iteration, and error avoidance.

Core Design Challenges

Temporal and causal coherence : scenario fragments must preserve the order and causality of reasoning steps.

Multi‑goal identification : the system must distinguish overlapping or switching objectives within a single conversation.

Learning from past episodes : each fragment is evaluated for success or failure, compared with similar past fragments, and distilled into reusable principles.

Architecture Overview

Scenario‑Fragment Extraction Module

The module converts raw user‑agent interactions into structured scenario fragments through a two‑stage process.

Turn‑level extraction : each dialogue turn is broken down into turn context , turn intent , turn action , turn thought , turn evaluation , and goal evaluation .

Episode extraction : after a goal is achieved or a session ends, the collected turns are aggregated into a coherent episode that records the full workflow from problem definition to resolution.

The resulting episode contains:

Scenario background

Explicit user goal

Effectiveness assessment

Evidence for assessment

Insights (successful patterns and pitfalls)

Reflection Module

The reflection module synthesizes experience across episodes. For a new task it retrieves historically successful episodes with similar intents, performs cross‑episode analysis, and generates generalized insights. Each insight is stored with a confidence score (0.1–1.0) and can be used to guide future decisions.

Customization Options

Developers can override built‑in strategies with custom prompts, custom models, and namespace hierarchies.

Custom prompts : define extraction criteria, integration rules, conflict‑resolution mechanisms, and insight‑generation logic.

Custom models : specify a model via the memory_resource parameter in the console or API.

Namespace hierarchy : example path for a travel‑booking app – travel_booking/users/userABC/episodes for scenario fragments and travel_booking/users/userABC for reflection memory (must be a sub‑path of the fragment namespace).

Performance Evaluation

Benchmarks used real‑world retail and airline customer‑service scenarios from the τ2‑bench dataset. Three configurations were compared:

Baseline: Claude 3.7 agent without memory.

Context‑learning (exemplar) method: retrieve relevant episodes as examples.

Reflection‑guided method: retrieve synthesized strategic insights.

Each query was attempted four times; success rates were measured as Pass^k (at least k successes out of 4).

Reflection improved Pass^1 by 11.4 % and Pass^3 by 13.6 % over the baseline.

In airline tasks, the exemplar method achieved Pass^3 of 43.0 % , slightly higher than reflection’s 41.0 % , indicating that detailed step‑by‑step examples benefit highly procedural workflows.

Code Samples

def retrieve_exemplars(task: str) -> str:
    """Retrieve example processes to help solve the given task.

    Args:
        task: The task to solve that requires example processes.

    Returns:
        str: The example processes to help solve the given task.
    """

def retrieve_reflections(task: str, k: int = 5) -> str:
    """Retrieve synthesized reflection knowledge from past agent experiences.

    Args:
        task: The current task or goal you are trying to accomplish.
        k: Number of reflection knowledge entries to retrieve. Default is 5.

    Returns:
        str: The synthesized reflection knowledge from past agent experiences.
    """

Example Scenario Fragment

** Context **
A customer(Jane Doe) contacted customer service expressing frustration about a recent flight delay that disrupted their travel plans and wanted to discuss compensation or resolution options for the inconvenience they experienced.

** Goal **
The user's primary goal was to obtain compensation or some form of resolution for a flight delay they experienced, seeking acknowledgment of the disruption and appropriate remediation from the airline.
---
### Step 1:
**Thought:**
The assistant chose to gather information systematically rather than making assumptions, as flight delay investigations require specific reservation and flight details. This approach facilitates accurate assistance and demonstrates professionalism by acknowledging the customer's frustration while taking concrete steps to help resolve the issue.
**Action:**
The assistant responded conversationally without using any tools, asking the user to provide their user ID to access reservation details.
--- End of Step 1 ---
...
** Episode Reflection **:
The conversation demonstrates an excellent systematic approach to flight modifications: starting with reservation verification, then identifying confirmation, followed by comprehensive flight searches, and finally processing changes with proper authorization. The assistant effectively used appropriate tools in a logical sequence – get_reservation_details for verification, get_user_details for identity/payment info, search_direct_flight for options, and update tools for processing changes. Key strengths included transparent pricing calculations, proactive mention of insurance benefits, clear presentation of options, and proper handling of policy constraints (explaining why mixed cabin classes aren't allowed). The assistant successfully leveraged user benefits (Gold status for free bags) and maintained security protocols throughout. This methodical approach ensured user needs were addressed while following proper procedures for reservation modifications.

Example Reflection Memory

**Title:** Proactive Alternative Search Despite Policy Restrictions

**Use Cases:**
This applies when customers request flight modifications or changes that are blocked by airline policies (e.g., basic economy no‑change rules, fare class restrictions, or booking timing limitations). Rather than simply declining the request, the pattern involves immediately searching for alternative solutions to help customers achieve their underlying goals. It is valuable for emergency situations, budget‑conscious travelers, or when customers have specific timing needs that their current reservations don't accommodate.

**Hints:**
When policy restrictions prevent the requested modification, immediately pivot to solution‑finding rather than just explaining limitations. Use <code>search_direct_flight</code> to find alternative options that could meet the customer's needs, even if it requires separate bookings or different approaches. Present both the policy constraint explanation AND viable alternatives in the same response to maintain momentum toward resolution. Consider the customer's underlying goal (getting home earlier, changing dates, etc.) and search for flights that accomplish this objective. When presenting alternatives, organize options clearly by date and price, highlight budget‑friendly choices, and explain the trade‑offs between keeping existing reservations versus canceling and rebooking. This approach turns policy limitations into problem‑solving opportunities and maintains customer satisfaction even when the original request cannot be fulfilled.

References

https://aws.amazon.com/blogs/machine-learning/amazon-bedrock-agentcore-memory-building-context-aware-agents/

https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/episodic-memory-strategy.html

https://github.com/awslabs/amazon-bedrock-agentcore-samples/tree/main/01-tutorials/04-AgentCore-memory