8 min read1,802 words
More Reasoning Doesn't Fix Stale Memory
Across every reasoning-budget setting I tested, the benchmark's hardest failure family stayed at zero. What closes it is 40 lines of query-time lookup.
The hardest scenarios in the benchmark are not the ones where the memory has the wrong answer. They are the ones where the memory has the right answer, with the right citation, and still cannot tell that the answer is no longer safe to act on. The benchmark calls this family cascading staleness: the stored answer and its citation are both right, but a premise 1 or more steps back has been superseded. Every system I tested, before adding the fix below, scored 0 on it.
This is the third piece in a 4-part series. The second piece showed that structured access to the store reaches the high 350s out of 395 but misses answers whose premise 3 hops back has changed. This article is about those answers.
Here is the example. The store has an answer claim: Maya Patel has full signing authority at Apple. That claim derives from Maya Patel is chief of staff, which derives from Tim Cook is CEO. Later, the memory learns that Priya Raman became CEO on November 1, 2027. The answer claim has not disappeared, but one upstream premise is no longer current.
Benchmark row E.5
One answer, two different memory states.
Question
What is the signing_authority of Maya Patel?
- Expected answer
- full
- Expected citation
- ap_d3
- Expected status
- POTENTIALLY_STALE
Prompt rule used by the LLM walk
- For every doc_id p in c.derived_from, transitively, look up the premise document p.
- For each premise, find the current claim for the same entity and predicate at the queried (T_v, T_t).
- If that current claim's value differs from p.value, the premise is superseded.
- If c is valid and any transitive premise is superseded, set epistemic_status = POTENTIALLY_STALE while still returning c.value.
Evidence chain
Returned claim
ap_d3 / valid nowMaya Patel holds full signing authority for routine corporate matters at Apple.
Direct premise
ap_d2 / depends on ap_01Maya Patel serves as chief of staff to Apple's CEO Tim Cook.
Transitive premise
ap_01 / supersededTim Cook continues as CEO of Apple through most of 2027.
Current successor
ap_03 / currentApple appoints Priya Raman as Chief Executive Officer effective November 1, 2027.
Row store
No premise walk
- answer
- full
- primary_document_id
- ap_d3
- citation_level
- 3 / 3
- epistemic_status
- UNVERIFIED
The value and citation are right, but the interface treats the claim as a clean single-source fact.
Sonnet 4.6
Prompted premise walk
- answer
- full
- primary_document_id
- ap_d3
- citation_level
- 3 / 3
- epistemic_status
- POTENTIALLY_STALE
When the prompt surfaces `derived_from` and asks for the transitive walk, the same answer carries the warning.
Row store
Recursive CTE walk
- answer
- full
- primary_document_id
- ap_d3
- citation_level
- 3 / 3
- epistemic_status
- POTENTIALLY_STALE
The storage walk produces the same status without spending tokens or asking a model to reason through the chain.
The same pattern appears outside the executive-delegation chain. Here is the NVIDIA acquisition-freeze row from the corpus, shown with the prior documents the system sees, the prompt contract asking for epistemic status, and the structured answer that passes or fails the audit.
Benchmark rows E.5 / E.6
Asking for status in the prompt is not enough; the read path has to compute it.
Prior ingested documents
Premise
nv_06
The European Commission opens a formal antitrust review of NVIDIA's GPU licensing practices effective August 1, 2027.
predicate: regulatory_status
value: under_antitrust_review
Derived answer
nv_d2
Pursuant to NVIDIA's antitrust review status, an internal acquisition freeze takes effect 2027-08-01 and remains in place pending regulatory resolution.
predicate: acquisition_freeze
value: active
derived_from: nv_06
Successor
nv_07
The European Commission concludes its NVIDIA antitrust review with no enforcement action; status returns to cleared as of February 1, 2028.
predicate: regulatory_status
value: cleared
supersedes: nv_06
Representative prompt contract
You are answering from an auditable memory store. Use the ingested claim records and their document_id, valid-time interval, transaction time, and derived_from links. Return answer, primary_document_id, citation_level, and epistemic_status. epistemic_status must be one of UNVERIFIED, CORROBORATED, CONTESTED, ENRICHED, or POTENTIALLY_STALE. If the answer claim derives from another claim, transitively inspect each premise at the query valid time and transaction time. If any transitive premise has a different current value at the query point, keep the answer but set epistemic_status to POTENTIALLY_STALE.
Baseline failure
What is the acquisition_freeze of NVIDIA?
answer: active primary_document_id: nv_d2 citation_level: 3 epistemic_status: UNVERIFIED
The answer and citation are right, but the memory state is wrong. The read path returned nv_d2 without walking back to nv_06 and comparing that premise with its current successor nv_07.
Giving the models more reasoning did not fix it.
I ran 3 tests that kept the model and prompt fixed and changed only the reasoning setting. Total scores move, and by a lot: Sonnet 4.6 picks up 83 scenarios when extended thinking is turned on. GPT-5-mini scores identically across its 3 effort levels; Honcho 3 wanders within a 4-scenario band. Cascading staleness stays at 0/29 in every condition.
Within-subject reasoning controls
Scores move with reasoning budget. Cascading staleness doesn't.
However much extra computation the model spends, it does not flag these answers as stale. What closes the family is 40 lines of code that follow links between claims back from the answer at query time. Each link is recorded when a claim is stored. The code is small enough to write in an afternoon, and the score lift is the same on every store that records the links.
Score lift
Before / after the premise walk.
The Walk#
A derived_from link is just an authored field on each claim: when the ingest step records a new
claim that was inferred from an earlier one, it writes down which earlier claim it depended on.
The procedure is: retrieve the candidate claim. Walk back along those declared derived_from
edges. For each premise on the chain, ask the store what the current value is for that
(entity, predicate) pair under the query's valid-time and transaction-time bounds. If any premise has been superseded,
flip the answer's epistemic status to POTENTIALLY_STALE.
On the SQLite row store, the walk is 2 queries:
-- 1. Every premise behind the answer claim, however many hops back.
WITH RECURSIVE premises(doc_id) AS (
SELECT premise_doc FROM claim_derivations WHERE derived_doc = :answer_doc
UNION
SELECT cd.premise_doc FROM claim_derivations cd
JOIN premises p ON cd.derived_doc = p.doc_id
)
SELECT c.entity, c.predicate, c.value
FROM premises p JOIN claims c ON c.document_id = p.doc_id;
-- 2. For each premise, the value current at the query's 2 time points.
SELECT value FROM claims
WHERE entity = :entity AND predicate = :predicate
AND t_vs <= :t_v -- true in the world at the queried time
AND (t_ve IS NULL OR t_ve > :t_v)
AND t_t_ord <= :t_t -- known to the memory at the queried time
ORDER BY t_vs DESC, t_t_ord DESC
LIMIT 1;If any premise's current value differs from the value recorded on the chain, the answer is
POTENTIALLY_STALE. This is simplified from the benchmark's reference store, which also
handles queries without a time bound and predicates that hold several values at once.
Scroll explainer
Walking the chain, one premise at a time.
Premise walk
POTENTIALLY_STALEJan 15: answer claim
Maya Patel has signing authority
Derived from the chief-of-staff claim.
Jan 8: direct premise
Maya Patel is chief of staff
Derived from the CEO claim.
Jan 1: root premise
Tim Cook is CEO of Apple
The premise the chain ultimately rests on.
current claim for (Apple, ceo, query time) →
Nov 1: current premise
Priya Raman is CEO of Apple
Supersedes Tim Cook. The chain's root premise has changed.
Step 1
Start at the answer claim.
The query returns Maya Patel's signing authority. A safe interface starts a walk here instead of stopping.
Step 2
Hop back along derived_from.
The signing-authority claim was derived from a chief-of-staff claim, which was derived from a CEO claim. The walk follows the declared dependency edges.
Step 3
Compare each premise to the current value.
For every (entity, predicate) on the chain, ask the store: who is current under the query time bounds? Tim Cook was the premise. The current premise is Priya Raman.
Step 4
If any premise was superseded, flip the status.
The answer claim is still returned, but its epistemic status changes from UNVERIFIED to POTENTIALLY_STALE, because its support chain now passes through a superseded premise.
Step 1
Start at the answer claim.
The query returns Maya Patel's signing authority. A safe interface starts a walk here instead of stopping.
Step 2
Hop back along derived_from.
The signing-authority claim was derived from a chief-of-staff claim, which was derived from a CEO claim. The walk follows the declared dependency edges.
Step 3
Compare each premise to the current value.
For every (entity, predicate) on the chain, ask the store: who is current under the query time bounds? Tim Cook was the premise. The current premise is Priya Raman.
Step 4
If any premise was superseded, flip the status.
The answer claim is still returned, but its epistemic status changes from UNVERIFIED to POTENTIALLY_STALE, because its support chain now passes through a superseded premise.
It uses no machine learning, embedding, reranker or calibration: it is a recursive lookup with a deterministic time filter.
4 Implementations, 1 Algorithm#
The procedure transfers cleanly. I implemented it on a SQLite row store
as a recursive SQL query, on Mem0 by walking the metadata.derived_from field in Python, on
Graphiti by walking the same kind of field on its edge objects, and on the Claude Sonnet 4.6 model
as a prompt rule with extended thinking turned on (a setting that lets the model spend more tokens
on internal reasoning before answering). The same lift appears in all 4.
Same algorithm
4 implementations, 1 walk, the same lift.
The walk
Substrate
Row-store (SQLite)
recursive SQL query over claim_derivations
Substrate
Mem0
walk metadata.derived_from
Substrate
Graphiti
walk EntityEdge.attributes
Substrate
Sonnet 4.6 (in-context)
prompt rule + extended thinking
So the fix for cascading staleness is a read-path algorithm, not a storage feature. As long as the write path has authored the dependency edges and the read path is willing to walk them, it does not matter what runs it.
False Alarms#
The naive fix is to flag any claim whose support chain touches a superseded premise. That catches every cascading-staleness case, but it also catches a lot of cases that aren't actually stale.
Concretely: suppose the agent asks "did Maya have signing authority in January?" The support
chain still passes through "Tim Cook is CEO", a premise that was later superseded, when Priya
Raman became CEO on November 1, 2027. A walk that flags every supersession in the chain would mark
the January claim POTENTIALLY_STALE. But for the period being asked about, the chain was sound. The
supersession sits in the future relative to the query.
This is the E.6 specificity test: 28 scenarios where a real supersession exists, but it lies outside the bitemporal scope the agent is asking about. In the January example, it is outside the valid-time window; in a late-correction case, it may be outside the transaction-time knowledge cutoff. A correct walk leaves these alone.
The denominator differs from the headline stale-case count because the near-misses are a matched real-entity control: 28 real-entity stale cases get 28 near-misses. The headline count is 29 because it also includes the Acme worked-example case.
Valid-time scope
Drag the query bound across the supersession.
If query bound < supersession (E.6)
The supersession is outside the query's valid-time scope. Status: UNVERIFIED.
If query bound ≥ supersession (E.5)
A premise was superseded inside scope. Status: POTENTIALLY_STALE.
So the walk has to be scoped to the query's bitemporal bounds. For each premise, compare
against the value that is current for the same (entity, predicate) at the query's valid time
and known by the query's transaction time . A supersession that starts after the
period being audited, or that the memory had not learned by the transaction-time cutoff, should not
make the January answer stale.
Why Traversal, Not Reasoning#
The sweeps above show that cascading staleness is not a reasoning problem the model could solve with a larger budget. What the family needs is a procedure that walks the chain, and that is a property of the read path rather than of the policy running on top of it.
MRAgent's active-reconstruction result points in the same direction: memory access should be an iterative read procedure whose next step depends on evidence already found, not a passive retrieval call fixed by the original query.[4] The result here is a narrower audit version of that claim. Active traversal can help find the right support, but the benchmark still requires one extra output: whether any premise on the declared derivation path has been superseded under the query's valid-time and transaction-time bounds.
When the LLM Walks the Chain Itself#
If the storage path will not walk the chain, the model has to. I tested this by giving capable long-context LLMs the source documents plus a prompt rule that described the walk, and asked them to execute it themselves. 2 pass/fail tests separate the results: did the system catch the genuinely stale case (sensitivity), and did it leave the near-miss alone (specificity)?
Sensitivity vs specificity
Step 1 of 3: All systems
13 systems queued for 2 tests.
Each system in its default setup, plus six variants that walk the dependency chain at query time.
GPT-5-mini and Sonnet 4.6 with extended thinking on match the storage walk on both tests. Sonnet 4.6 with extended thinking off catches the stale cases but also marks every near-miss stale, raising false alarms on cases that aren't actually stale. Smaller models miss both axes. Storage walks always end up in the top-right cell: every stale case caught, every near-miss left alone.
What the Walk Changes#
A concrete failure, using the Apple example above: after the November 1 CEO update, ask
“what is the signing_authority of Maya Patel?” The signing-authority claim is in the store with
its derived_from link pointing at the chief-of-staff claim, which points back at the Jan 1
“Tim Cook is CEO” premise. The system returns full, with the right citation. The
November 1 update where Priya Raman becomes CEO never enters the answer. A reader of the answer cannot
tell that the support chain has changed; only the status on the answer should have flipped
from UNVERIFIED to POTENTIALLY_STALE.
The lift is about 29 scenarios, and it is the same on every implementation. SQLite row-store 359 → 388, Mem0 A 359 → 388, Graphiti A 354 → 383, Sonnet 4.6 (with extended thinking) 359 → 388. Same procedure, same delta, on 3 storage backends and an in-context LLM. The lift depends on the corpus, not on the backend.
The Sonnet path has 2 effects, not 1. Sonnet 4.6 in-context starts at 276/395. Turning on extended thinking lifts that to 359/395 (Δ83) without touching the stale cases at all; the model gets better at the rest of the corpus. Adding the premise-walk prompt on top contributes the same Δ29 the storage walks deliver elsewhere, taking it to 388. The walk and the reasoning budget do different things, and Sonnet needs both to reach 388.
The walk adds status, not accuracy. For systems whose base answer+citation score was already high (Mem0 A: 389/395 on answer+citation), the +walk answer+citation barely moves (389 → 389), while the full-rubric jumps (359 → 388). The walk is closing the gap between “I have the right answer” and “I can also say whether the answer is still safe to use”. It fixes epistemic status, not retrieval.
The next article puts every system on 1 chart and sends the same questions to the managed products you can buy, not only to libraries run locally.
What the Walk Depends On#
The walk works conditional on a write path that authors the derivation edges. The benchmark
assumes that input. If your ingest step does not preserve derived_from, the read-path procedure
has nothing to walk.
Scope
What the benchmark measures, and what it doesn't.
Write-path extraction quality is a separate problem, with its own literature. The benchmark shows that, given the edges, the read path is a small piece of code that works on any store. It is not yet standard in agent-memory APIs.
The failure is invisible from the answer alone: the answer and the citation are right, only the status is wrong, and most memory APIs have no place to put a status. The benchmark measures that status directly. The framing piece draws the parallel to the metacognition argument for parametric models: in both settings, reliability hinges on a per-answer faithful signal, not on aggregate accuracy.
Limitations#
- Depends on write-path quality: missing provenance links reduce recall of stale-chain detection.
- Needs stable claim IDs across ingestion, compaction, and deduplication.
- Deep or noisy provenance graphs need cycle handling, fan-out caps, and observability around skipped links.
- Benchmark scores are controlled evaluations; production distributions and noise can differ.
References#
- Rasmussen, P., Paliychuk, P., Beauvais, T., Ryan, J., & Chalef, D. (2025). Zep: A Temporal Knowledge Graph Architecture for Agent Memory. arXiv:2501.13956.
- Chhikara, P., Khant, D., Aryan, S., Singh, T., & Yadav, D. (2025). Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413.
- Marković, V., Obradović, L., Hajdu, L., & Pavlović, J. (2025). Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning. arXiv:2505.24478.
- Ji, S., Li, Y., & Hooi, B. (2026). Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents. arXiv:2606.06036.