What Did the Agent Know at Time T? Bi-Temporal Memory and Its Missing Benchmark

Most memory benchmarks ask whether an agent remembers the right thing. Almost none ask the question an auditor actually cares about: what did the agent know at time T — and was that true then?

The anachronistic agent

An agent's memory can fail in a subtler way than hallucination: it can be anachronistic. It acts on a fact that was true last month and is false now. A user's address changes, a price is superseded, a policy is revoked. A memory system that simply overwrites the old value loses the ability to explain why the agent did what it did last week — and the moment something goes wrong, that explanation is exactly what you need.

Two questions pull in opposite directions. "What is true now?" wants latest-wins semantics: the newest fact replaces the old. "What did we believe at time T?" wants as-of semantics: reconstruct the world as the agent saw it at that instant. Plain vector stores answer the first by overwriting; they cannot answer the second at all, because the old embedding was destroyed when the new one was written.

Two timelines, four timestamps

The fix is to stop treating time as one axis. Bi-temporal systems record two independent timelines: valid time, the window during which a fact was true in the world, and transaction time, the window during which the system itself held that record. The first answers "when was this so?"; the second answers "when did we learn it?".

Zep's Graphiti engine makes this concrete by storing four timestamps on every edge:

Field	Timeline	Meaning
t_valid	valid time	when the fact became true in the world
t_invalid	valid time	when it stopped being true
t_created	transaction time	when the system ingested it
t_expired	transaction time	when the system superseded it

When new information contradicts an existing edge whose valid interval overlaps, the system does not delete the old edge — it invalidates it by setting the old edge's t_invalid to the new edge's t_valid. The new fact wins going forward; the old fact remains, bounded in time. Nothing is lost, so the graph can still answer historical questions.

Point-in-time queries

With those edges, "what did the agent believe at time T?" becomes an ordinary interval query rather than a reconstruction project.

// What did the agent believe about Alice's employer on 1 September 2026?
MATCH (a:Entity {name: 'Alice'})-[r:WORKS_FOR]->(o:Entity)
WHERE r.t_valid <= datetime($asOf)
  AND (r.t_invalid IS NULL OR datetime($asOf) < r.t_invalid)
RETURN o.name AS employer, r.t_valid AS since

The same shape reconstructs an entire belief state — every edge valid at $asOf — which is what you replay to answer "why did the agent do that?" six weeks later. A latest-only store has no equivalent; the answer was overwritten.

The evidence it works

Zep and Graphiti report measurable gains over a stateless baseline, and the temporal model is the reason: retrieval can filter by validity window before ranking, so stale facts never enter the context.

System	DMR (accuracy)	LongMemEval
MemGPT	93.4%	baseline
Zep / Graphiti	94.8%	up to +18.5% accuracy, −90% latency

The retrieval itself is a hybrid — vector similarity, BM25, and graph traversal — deliberately without an LLM in the reranking loop, so latency stays predictable. This is not an isolated design: Engram models agent memory as a bi-temporal engine, DYNA pairs episodic memory nets with temporal graphs, and TVA and PETGraphDB attack version-aware storage directly. The direction is converging.

The missing benchmark

Here is the uncomfortable part. The agent-memory literature has a documented gap: there is no benchmark that evaluates temporal reasoning — "what did the agent know at time T?" — alongside the usual recall and accuracy scores. Neither is there a standard for multimodal agent memory or for multi-agent memory sharing. The tooling has outrun the measurement.

A benchmark that closed this gap would need to score at least five behaviours:

1. Staleness handling. Does the agent stop relying on a superseded fact at the right moment — not before, not long after?
2. Supersession attribution. Can it explain a past decision using the facts it actually held at that time, rather than today's facts?
3. Retraction timing. When told to forget or correct something, does it take effect exactly then, and is the change attributable?
4. Contradiction resolution. With two conflicting sources, which wins, and is the loser time-bounded rather than silently deleted?
5. Multi-hop temporal consistency. Does a chain of reasoning remain valid-time-consistent end to end, or does it quietly mix facts from incompatible epochs?

Existing benchmarks measure whether the agent recalls the current answer. They do not penalise anachronism — acting confidently on a fact that expired last week — which is the failure mode that actually breaks audits.

Caveats worth stating plainly

Bi-temporal memory is not free. Storage grows because every edge carries four timestamps and its superseded history. Queries get harder: interval predicates defeat naive indexes built for point lookups, so you need temporal-aware storage. Ingestion ordering matters: transaction time requires a monotonic clock, and if events arrive out of order, valid time has to be reconstructed rather than assigned.

And the invalidation step is only as good as its contradiction detector, which in Zep's design is an LLM. A false invalidation is expensive — it retires a fact that was still true — and hard to notice, because the graph looks healthy afterwards. Treat invalidation like any other model-driven mutation: log it, make it reversible, and sample it for review.

The takeaway

Add the second timeline before you need the audit. A bi-temporal graph turns "what does the agent know?" into "what did the agent know, when, and why?" — the version of the question that debugging, incident review, and regulators all ask. The engine exists; what is missing is a benchmark that refuses to reward an agent for answering yesterday's question with today's answer.