6 min read1,251 words
The Interface Resolution Problem in Agent Memory
Why the same stored facts produce different answers depending on the retrieval surface.
2 agents query the same memory store with the same underlying claims. One gets the right answer with a usable epistemic status. The other gets a different answer, or a worse one, or no answer at all. The data is the same; only the interface differs.
This is the second piece in a 4-part series. The
first piece introduced the benchmark. Each
scenario is graded on the answer, the citation chain, the epistemic status (such as
POTENTIALLY_STALE: the answer may be right, but a premise behind it has been superseded), and
whether the system finishes ingest in bounded time. A scenario passes only when all 4 pass. The headline corpus is 395 scenarios; numbers like
354/395 below are full-rubric pass counts on it.
This article shows that the same stored claims pass or fail depending on the API the agent uses. On Graphiti and Mem0, the 2 systems that offer both a structured API and a natural-language search, the same store produced very different scores depending on which surface the agent used.
The Interface Levels#
Direct Corpus Interaction (DCI), from recent agentic-search work[1], argues that retrieval quality depends not only on the agent's reasoning ability but on the resolution of the interface through which it touches the corpus: agents constrained to a top-k similarity call cannot verify evidence that a lower-resolution surface has hidden. The same applies to persistent memory, except that what the interface hides is not a passage or document but the chain of earlier claims an answer depends on, and whether each was still current at the time asked about. The ladder below organizes memory interfaces by what they expose of that structure.
Memory interfaces fall across 6 resolution levels. Each level adds a capability over the previous: from a bare answer string, to timestamps, to the source of each claim, and finally to the full chain of claims an answer depends on. The benchmark measures what each level can and cannot resolve.
Scroll explainer
Six levels of what an interface can expose.
Resolution levels
What the interface exposes
query return at L5
// Tools the agent can call: walk_premises(claim, depth=3) current_value(e, p) range(e, p, t_from, t_to)
Level L0
Answer only
Exposes answer string. Factual recall. No stale-premise detection.
The agent learns the answer is yes, but cannot say where it came from or whether the source is still current.
Level L1
Answer + citation
Exposes source document or text. Grounding. No dependency propagation.
The agent can verify the answer against a specific source, but it cannot tell whether that source itself rests on an earlier claim.
Level L2
Timestamped citation
Exposes valid time + transaction time. Bitemporal tests. No dependency tests.
The agent can audit what the system believed at a past transaction time, but a claim's premises are still invisible.
Level L3
Direct provenance
Exposes direct derived_from premises. Shallow stale-premise checks.
The agent sees the one-hop premise behind a derived claim, but a supersession three hops back is still hidden.
Level L4
Dependency closure
Exposes transitive premises + current comparison. Cascading-staleness detection.
The agent sees the full chain a claim rests on and compares each link to the current value, so a supersession anywhere upstream becomes visible.
Level L5
Direct memory interaction
Exposes bounded traversal tools. Tests whether agents can discover closure.
The agent gets traversal primitives and has to compose the walk itself; the bound is whether it can.
L0: Answer only
Factual recall. No stale-premise detection.
{
answer: "yes"
}L1: Answer + citation
Grounding. No dependency propagation.
{
answer: "yes",
citation: "doc_3.txt:L4"
}L2: Timestamped citation
Bitemporal tests. No dependency tests.
{
answer: "yes",
citation: {
doc: "doc_3.txt:L4",
valid_from: "2027-01-15",
txn_time: "T03"
}
}L3: Direct provenance
Shallow stale-premise checks.
{
answer: "yes",
citation: { ... },
derived_from: ["doc_2.txt"]
}L4: Dependency closure
Cascading-staleness detection.
{
answer: "yes",
citation: { ... },
derived_from: ["doc_2"],
closure: ["doc_2", "doc_1"],
current: {
"(Apple, ceo)":
"Sarah Chen"
},
status: "POTENTIALLY_STALE"
}L5: Direct memory interaction
Tests whether agents can discover closure.
// Tools the agent can call: walk_premises(claim, depth=3) current_value(e, p) range(e, p, t_from, t_to)
Higher levels are not always better; some agents only need L0. But the level the interface exposes is the upper bound on what the agent can reason about.
L5, the top of the ladder, sits closest to DCI's own setting in agentic search: the agent is given traversal primitives and asked to compose the walk itself, instead of going through a constrained retrieval API.
Same Store, 2 Surfaces#
Many production memory systems give you 2 surfaces against the same store: a structured API and a natural-language retrieval call. The structured API looks claims up directly and filters them by when each was true and when the memory learned it. The natural-language path runs embedding similarity and reranking. Both read from the same persisted claims.
Same store
Different interface, different view.
I call these the structured path and the NL path (Mode A and Mode B in the paper and in the charts below). A third, the dispatcher path, is a control I introduce in a moment.
The 308-Scenario Drop#
When I scored Graphiti and Mem0 under both surfaces, both scored far lower on the NL path.
Empirical drop
Mode A vs Mode B on the same store.
Graphiti goes from 354 on the structured path to 46 on the NL path, out of 395, by changing nothing but the API call. Mem0 goes from 359 to 158. The drop is broad, not confined to 1 part of the test set: questions that should return multiple values, point-in-time audits and date-range lookups all fail.
The Privileged Dispatch Objection#
One objection is that the structured path wins because its regex template parser knows the predicate names and entity IDs ahead of time, while the NL path has to discover them. So I built the dispatcher path: the same indexed-scan retrieval, but with the regex parser replaced by a gpt-5-mini JSON dispatcher. The model decides what to look up; the storage path is identical.
Privileged dispatch control
The regex parser is not the cause.
Mode A
Mode A: regex parser
- 1NL question
- 2regex template parser
- 3indexed scan
- 4bitemporal filter
full-rubric
359/ 395
Mode C
Mode C: LLM dispatcher
- 1NL question
- 2gpt-5-mini JSON dispatcher
- 3indexed scan
- 4bitemporal filter
full-rubric
357/ 395
Mode B
Mode B: NL retrieval
- 1NL question
- 2hybrid retrieval
full-rubric
158/ 395
On Mem0 the dispatcher scores 357 against the structured path's 359: a difference of 2 scenarios out of 395. On Graphiti it scores 345 against 354, a difference of 9. The parser's advantage is small on both libraries, and it does not explain the NL path's drop.
Where the Gap Lives#
If the parser is not the cause, the next candidate is the LLM-driven ingest. The NL path uses
Mem0's infer=True ingest, where an LLM rewrites the incoming document into structured
claims. Maybe extraction quality matters more than the retrieval surface. I tested the cross
(structured retrieval against the same infer=True ingest), and that run scored 356 out of 395.
The 201-scenario gap splits like this:
Where the gap lives
Of the Δ201 drop on Mem0:
98% of the drop comes from the retrieval surface, not the LLM extraction. The same representation passes or fails depending on which API the agent calls.
The Limit of Structured Access#
Structured access wins on this comparison, but it has its own ceiling. It scores in the high 350s out of 395, not 395. It catches answer-correctness and direct provenance, but it does not walk the chain of premises behind a derived claim. A claim can sit in the store, indexed, retrievable, and answer-correct, while a premise 3 hops back has been superseded.
This boundary is close to the one drawn by MRAgent, which argues that graph memory should be read through active reconstruction rather than one-shot retrieval or fixed neighborhood expansion.[4] That is the same interface-resolution lesson from the other side: a graph representation helps only if the read interface can adapt as intermediate evidence changes what should be inspected next. For this benchmark, adaptive graph traversal is still not the whole contract; the answer also needs a bitemporal status derived from the support chain.
That is the failure mode the next article is
about: cascading staleness. A small query-time walk over declared derived_from links closes the
gap on every storage backend I tested.
The Same Effect on a Shipping Product#
The structured/NL split is not an artifact of the open-source libraries or my own wrapper. Zep Cloud, a managed product with native date filters, retrieves the right claim on ~309 of the 395 scenarios through its own interface but passes the full audit on only 160. The fourth article covers the hosted products.
References#
- Li, Z., Zhang, H., Wei, C., Lu, P., Nie, P., Lu, Y., Bai, Y., Feng, S., Zhu, H., Zhong, M., Zhang, Y., Xie, J., Choi, Y., Zou, J., Han, J., Chen, W., Lin, J., Jiang, D., & Zhang, Y. (2026). Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction. arXiv:2605.05242.
- Rasmussen, P., Paliychuk, P., Beauvais, T., Ryan, J., & Chalef, D. (2025). Zep: A Temporal Knowledge Graph Architecture for Agent Memory. arXiv:2501.13956. Repo: github.com/getzep/graphiti.
- Chhikara, P., Khant, D., Aryan, S., Singh, T., & Yadav, D. (2025). Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413. Site: mem0.ai.
- Ji, S., Li, Y., & Hooi, B. (2026). Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents. arXiv:2606.06036.
Limitations#
- Requires a query schema. If entities and predicates are not canonicalized, the dispatcher can send a query to the wrong coordinate.
- Benchmark scores are controlled evaluations; production distributions and corpus noise can widen or shrink the gap.
- Structured access still does not solve cascading staleness by itself; it needs the premise walk in the next article.
- Mem0's retrieval algorithm continued to evolve through April–May 2026 (blog 1, blog 2). Both predate this article, and I had not seen them when writing it. The numbers here reflect the version evaluated in the paper, not the later state-key +
event_endreranking layer. The fourth article discusses how that update relates to the benchmark.