'Observational memory' cuts AI agent costs 10x and outscores RAG on long-context benchmarks

RAG isn't always fast enough or intelligent enough for modern agentic AI workflows. As teams move from short-lived chatbots to long-running, tool-heavy agents embedded in production systems, those limitations are becoming harder to work around.

In response, teams are experimenting with alternative memory architectures — sometimes called contextual memory or agentic memory — that prioritize persistence and stability over dynamic retrieval.

One of the more recent implementations of this approach is "observational memory," an open-source technology developed by Mastra, which was founded by the engineers who previously built and sold the Gatsby framework to Netlify.

Unlike RAG systems that retrieve context dynamically, observational memory uses two background agents (Observer and Reflector) to compress conversation history into a dated observation log. The compressed observations stay in context, eliminating retrieval entirely. For text content, the system achieves 3-6x compression. For tool-heavy agent workloads generating large outputs, compression ratios hit 5-40x.

The tradeoff is that observational memory prioritizes what the agent has already seen and decided over searching a broader external corpus, making it less suitable for open-ended knowledge discovery or compliance-heavy recall use cases.

The system scored 94.87% on LongMemEval using GPT-5-mini, while maintaining a completely stable, cacheable context window. On the standard GPT-4o model, observational memory scored 84.23% compared to Mastra's own RAG implementation at 80.05%.

"It has this great characteristic of being both simpler and it is more powerful, like it scores better on the benchmarks," Sam Bhagwat, co-founder and CEO of Mastra, told VentureBeat.

How it works: Two agents compress history into observations

The architecture is simpler than traditional memory systems but delivers better results. 

Observational memory divides the context window into two blocks. The first contains observations — compressed, dated notes extracted from previous conversations. The second holds raw message history from the current session.

Two background agents manage the compression process. When unobserved messages hit 30,000 tokens (configurable), the Observer agent compresses them into new observations and appends them to the first block. The original messages get dropped. When observations reach 40,000 tokens (also configurable), the Reflector agent restructures and condenses the observation log, combining related items and removing superseded information.

"The way that you're sort of compressing these messages over time is you're actually just sort of getting messages, and then you have an agent sort of say, 'OK, so what are the key things to remember from this set of messages?'" Bhagwat said. "You kind of compress it, and then you get in another 30,000 tokens, and you compress that."

The format is text-based, not structured objects. No vector databases or graph databases required.

Stable context windows cut token costs up to 10x

The economics of observational memory come from prompt caching. Anthropic, OpenAI, and other providers reduce token costs by 4-10x for cached prompts versus those that are uncached. Most memory systems can't take advantage of this because they change the prompt every turn by injecting dynamically retrieved context, which invalidates the cache. For production teams, that instability translates directly into unpredictable cost curves and harder-to-budget agent workloads.

Observational memory keeps the context stable. The observation block is append-only until reflection runs, which means the system prompt and existing observations form a consistent prefix that can be cached across many turns. Messages keep getting appended to the raw history block until the 30,000 token threshold hits. Every turn before that is a full cache hit.

When observation runs, messages are replaced with new observations appended to the existing observation block. The observation prefix stays consistent, so the system still gets a partial cache hit. Only during reflection (which runs infrequently) is the entire cache invalidated.

The average context window size for Mastra's LongMemEval benchmark run was around 30,000 tokens, far smaller than the full conversation history would require.

Why this differs from traditional compaction

Most coding agents use compaction to manage long context. Compaction lets the context window fill all the way up, then compresses the entire history into a summary when it's about to overflow. The agent continues, the window fills again, and the process repeats.

Compaction produces documentation-style summaries. It captures the gist of what happened but loses specific events, decisions and details. The compression happens in large batches, which makes each pass computationally expensive. That works for human readability, but it often strips out the specific decisions and tool interactions agents need to act consistently over time.

The Observer, on the other hand, runs more frequently, processing smaller chunks. Instead of summarizing the conversation, it produces an event-based decision log — a structured list of dated, prioritized observations about what specifically happened. Each observation cycle handles less context and compresses it more efficiently.

The log never gets summarized into a blob. Even during reflection, the Reflector reorganizes and condenses the observations to find connections and drop redundant data. But the event-based structure persists. The result reads like a log of decisions and actions, not documentation.

Enterprise use cases: Long-running agent conversations

Mastra's customers span several categories. Some build in-app chatbots for CMS platforms like Sanity or Contentful. Others create AI SRE systems that help engineering teams triage alerts. Document processing agents handle paperwork for traditional businesses moving toward automation.

What these use cases share is the need for long-running conversations that maintain context across weeks or months. An agent embedded in a content management system needs to remember that three weeks ago the user asked for a specific report format. An SRE agent needs to track which alerts were investigated and what decisions were made.

"One of the big goals for 2025 and 2026 has been building an agent inside their web app," Bhagwat said about B2B SaaS companies. "That agent needs to be able to remember that, like, three weeks ago, you asked me about this thing, or you said you wanted a report on this kind of content type, or views segmented by this metric."

In those scenarios, memory stops being an optimization and becomes a product requirement — users notice immediately when agents forget prior decisions or preferences.

Observational memory keeps months of conversation history present and accessible. The agent can respond while remembering the full context, without requiring the user to re-explain preferences or previous decisions.

The system shipped as part of Mastra 1.0 and is available now. The team released plug-ins this week for LangChain, Vercel's AI SDK, and other frameworks, enabling developers to use observational memory outside the Mastra ecosystem.

What it means for production AI systems

Observational memory offers a different architectural approach than the vector database and RAG pipelines that dominate current implementations. The simpler architecture (text-based, no specialized databases) makes it easier to debug and maintain. The stable context window enables aggressive caching that cuts costs. The benchmark performance suggests that the approach can work at scale.

For enterprise teams evaluating memory approaches, the key questions are:

• How much context do your agents need to maintain across sessions?

• What's your tolerance for lossy compression versus full-corpus search?

• Do you need the dynamic retrieval that RAG provides, or would stable context work better?

• Are your agents tool-heavy, generating large amounts of output that needs compression?

The answers determine whether observational memory fits your use case. Bhagwat positions memory as one of the top primitives needed for high-performing agents, alongside tool use, workflow orchestration, observability, and guardrails. For enterprise agents embedded in products, forgetting context between sessions is unacceptable. Users expect agents to remember their preferences, previous decisions and ongoing work.

"The hardest thing for teams building agents is the production, which can take time," Bhagwat said. "Memory is a really important bit in that, because it's just jarring if you use any sort of agentic tool and you sort of told it something and then it just kind of forgot it."

As agents move from experiments to embedded systems of record, how teams design memory may matter as much as which model they choose.