Kai Hirota

6 min read1,251 words

The Interface Resolution Problem in Agent Memory

Why the same stored facts produce different answers depending on the retrieval surface.

2 agents query the same memory store with the same underlying claims. One gets the right answer with a usable epistemic status. The other gets a different answer, or a worse one, or no answer at all. The data is the same; only the interface differs.

This is the second piece in a 4-part series. The first piece introduced the benchmark. Each scenario is graded on the answer, the citation chain, the epistemic status (such as POTENTIALLY_STALE: the answer may be right, but a premise behind it has been superseded), and whether the system finishes ingest in bounded time. A scenario passes only when all 4 pass. The headline corpus is 395 scenarios; numbers like 354/395 below are full-rubric pass counts on it.

This article shows that the same stored claims pass or fail depending on the API the agent uses. On Graphiti and Mem0, the 2 systems that offer both a structured API and a natural-language search, the same store produced very different scores depending on which surface the agent used.

The Interface Levels#

Direct Corpus Interaction (DCI), from recent agentic-search work[1], argues that retrieval quality depends not only on the agent's reasoning ability but on the resolution of the interface through which it touches the corpus: agents constrained to a top-k similarity call cannot verify evidence that a lower-resolution surface has hidden. The same applies to persistent memory, except that what the interface hides is not a passage or document but the chain of earlier claims an answer depends on, and whether each was still current at the time asked about. The ladder below organizes memory interfaces by what they expose of that structure.

Memory interfaces fall across 6 resolution levels. Each level adds a capability over the previous: from a bare answer string, to timestamps, to the source of each claim, and finally to the full chain of claims an answer depends on. The benchmark measures what each level can and cannot resolve.

Six levels of what an interface can expose.

L0: Answer only

Factual recall. No stale-premise detection.

{
  answer: "yes"
}

L1: Answer + citation

Grounding. No dependency propagation.

{
  answer: "yes",
  citation: "doc_3.txt:L4"
}

L2: Timestamped citation

Bitemporal tests. No dependency tests.

{
  answer: "yes",
  citation: {
    doc: "doc_3.txt:L4",
    valid_from: "2027-01-15",
    txn_time: "T03"
  }
}

L3: Direct provenance

Shallow stale-premise checks.

{
  answer: "yes",
  citation: { ... },
  derived_from: ["doc_2.txt"]
}

L4: Dependency closure

Cascading-staleness detection.

{
  answer: "yes",
  citation: { ... },
  derived_from: ["doc_2"],
  closure: ["doc_2", "doc_1"],
  current: {
    "(Apple, ceo)":
      "Sarah Chen"
  },
  status: "POTENTIALLY_STALE"
}

L5: Direct memory interaction

Tests whether agents can discover closure.

// Tools the agent can call:
walk_premises(claim, depth=3)
current_value(e, p)
range(e, p, t_from, t_to)

Higher levels are not always better; some agents only need L0. But the level the interface exposes is the upper bound on what the agent can reason about.

L5, the top of the ladder, sits closest to DCI's own setting in agentic search: the agent is given traversal primitives and asked to compose the walk itself, instead of going through a constrained retrieval API.

Same Store, 2 Surfaces#

Many production memory systems give you 2 surfaces against the same store: a structured API and a natural-language retrieval call. The structured API looks claims up directly and filters them by when each was true and when the memory learned it. The natural-language path runs embedding similarity and reranking. Both read from the same persisted claims.

Same store

Different interface, different view.

querysame storeNL retrievalStructured APIMode B agentMode A agent
The store and its claims are the same. The retrieval interface decides what the agent sees and what kinds of question it can answer.

I call these the structured path and the NL path (Mode A and Mode B in the paper and in the charts below). A third, the dispatcher path, is a control I introduce in a moment.

The 308-Scenario Drop#

When I scored and under both surfaces, both scored far lower on the NL path.

Empirical drop

Mode A vs Mode B on the same store.

395 (full)Mode AMode BGraphiti (OSS)35446Δ308Mem0 (OSS)359158Δ201
Scores out of 395 on the headline corpus. Mode A is a structured API; Mode B is the same store accessed via natural-language retrieval. The drop is the cost of the lower-resolution interface.

Graphiti goes from 354 on the structured path to 46 on the NL path, out of 395, by changing nothing but the API call. Mem0 goes from 359 to 158. The drop is broad, not confined to 1 part of the test set: questions that should return multiple values, point-in-time audits and date-range lookups all fail.

The Privileged Dispatch Objection#

One objection is that the structured path wins because its regex template parser knows the predicate names and entity IDs ahead of time, while the NL path has to discover them. So I built the dispatcher path: the same indexed-scan retrieval, but with the regex parser replaced by a JSON dispatcher. The model decides what to look up; the storage path is identical.

Privileged dispatch control

The regex parser is not the cause.

A vs C: Δ2 / Δ9A vs B: Δ201 / Δ308

Mode A

Mode A: regex parser

  1. 1NL question
  2. 2regex template parser
  3. 3indexed scan
  4. 4bitemporal filter

full-rubric

359/ 395

Mode C

Mode C: LLM dispatcher

  1. 1NL question
  2. 2gpt-5-mini JSON dispatcher
  3. 3indexed scan
  4. 4bitemporal filter

full-rubric

357/ 395

Mode B

Mode B: NL retrieval

  1. 1NL question
  2. 2hybrid retrieval

full-rubric

158/ 395

Mode C replaces Mode A's regex template parser with a gpt-5-mini JSON dispatcher and keeps the same indexed-scan retrieval. On Mem0, the score moves by 2 out of 395; the same control on Graphiti moves by 9. So the parser does not explain the Mode B drop.

On Mem0 the dispatcher scores 357 against the structured path's 359: a difference of 2 scenarios out of 395. On Graphiti it scores 345 against 354, a difference of 9. The parser's advantage is small on both libraries, and it does not explain the NL path's drop.

Where the Gap Lives#

If the parser is not the cause, the next candidate is the LLM-driven ingest. The NL path uses Mem0's infer=True ingest, where an LLM rewrites the incoming document into structured claims. Maybe extraction quality matters more than the retrieval surface. I tested the cross (structured retrieval against the same infer=True ingest), and that run scored 356 out of 395. The 201-scenario gap splits like this:

Where the gap lives

Of the Δ201 drop on Mem0:

Holding Mem0's LLM-extraction ingest constant and varying only the retrieval surface, 98% of the Mode A → Mode B drop comes from the retrieval surface, not from extraction.

98% of the drop comes from the retrieval surface, not the LLM extraction. The same representation passes or fails depending on which API the agent calls.

The Limit of Structured Access#

Structured access wins on this comparison, but it has its own ceiling. It scores in the high 350s out of 395, not 395. It catches answer-correctness and direct provenance, but it does not walk the chain of premises behind a derived claim. A claim can sit in the store, indexed, retrievable, and answer-correct, while a premise 3 hops back has been superseded.

This boundary is close to the one drawn by MRAgent, which argues that graph memory should be read through active reconstruction rather than one-shot retrieval or fixed neighborhood expansion.[4] That is the same interface-resolution lesson from the other side: a graph representation helps only if the read interface can adapt as intermediate evidence changes what should be inspected next. For this benchmark, adaptive graph traversal is still not the whole contract; the answer also needs a bitemporal status derived from the support chain.

That is the failure mode the next article is about: cascading staleness. A small query-time walk over declared derived_from links closes the gap on every storage backend I tested.

The Same Effect on a Shipping Product#

The structured/NL split is not an artifact of the open-source libraries or my own wrapper. Zep Cloud, a managed product with native date filters, retrieves the right claim on ~309 of the 395 scenarios through its own interface but passes the full audit on only 160. The fourth article covers the hosted products.

References#

  1. Li, Z., Zhang, H., Wei, C., Lu, P., Nie, P., Lu, Y., Bai, Y., Feng, S., Zhu, H., Zhong, M., Zhang, Y., Xie, J., Choi, Y., Zou, J., Han, J., Chen, W., Lin, J., Jiang, D., & Zhang, Y. (2026). Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction. arXiv:2605.05242.
  2. Rasmussen, P., Paliychuk, P., Beauvais, T., Ryan, J., & Chalef, D. (2025). Zep: A Temporal Knowledge Graph Architecture for Agent Memory. arXiv:2501.13956. Repo: github.com/getzep/graphiti.
  3. Chhikara, P., Khant, D., Aryan, S., Singh, T., & Yadav, D. (2025). Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413. Site: mem0.ai.
  4. Ji, S., Li, Y., & Hooi, B. (2026). Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents. arXiv:2606.06036.

Limitations#

  • Requires a query schema. If entities and predicates are not canonicalized, the dispatcher can send a query to the wrong coordinate.
  • Benchmark scores are controlled evaluations; production distributions and corpus noise can widen or shrink the gap.
  • Structured access still does not solve cascading staleness by itself; it needs the premise walk in the next article.
  • Mem0's retrieval algorithm continued to evolve through April–May 2026 (blog 1, blog 2). Both predate this article, and I had not seen them when writing it. The numbers here reflect the version evaluated in the paper, not the later state-key + event_end reranking layer. The fourth article discusses how that update relates to the benchmark.