Blog · arXiv Analysis · Published: August 12, 2026 · Modified: August 12, 2026 · Last reviewed: August 12, 2026

The Memory Summary Loses the Clock

A compressed memory can retain the people, events, and plans in a conversation while deleting the dates and durations that place them in order.

A recent agent-memory experiment makes that omission measurable. It also shows why a better prompt is only a narrow repair: institutions need preservation contracts, category-level tests, and an auditable path back to the original record.

The Paper

The source is Nicholas E. Kyrkewood’s The Sleeping Agent: What Gist-Based Context Compression Loses and Why, arXiv:2608.11775v1 [cs.AI], submitted August 12, 2026. The arXiv record describes a seven-page paper with five tables and appendices. Its Salience-Weighted Consolidation system is presented as a diagnostic probe for gist compression, not as proof that an agent sleeps, dreams, remembers like a person, or possesses consciousness.

Compression Is a Rewrite

When a long-running agent replaces old conversation turns with a compact summary, it does more than save tokens. It decides which distinctions remain available to future answers. The paper’s SWC design scores sessions for salience, keeps high-priority material verbatim, summarizes a middle tier, and discards a low tier under a 4,000-token budget. That makes every instruction to “preserve” or “discard” a small constitution for the agent’s future memory.

What Was Tested

The experiment compares recent-token truncation, sliding-window summarization, ordinary SWC, and a temporal-protecting SWC variant on all ten LoCoMo conversations. After a keyword filter removed 51 image or video questions, the matched text-only set contained 1,935 questions. The primary aggregate used 1,501 questions in categories one through four; 434 adversarial questions were excluded because their scoring rule can reward removal of information. A full-context reference was evaluated only on conversations zero and one.

The answer model was Claude Sonnet 4.6, while Claude Haiku compressed context and judged whether answers matched the gold answers. The paper reports 95 percent agreement on a blind human check of 60 judge decisions, including 17 of 17 temporal decisions. That is useful validation, but it remains a small sample and does not turn model-judged accuracy into a direct measure of production reliability.

The Clock Drops Out

The central result is unusually specific. In the paper’s verbatim-preservation analysis, ordinary SWC retained 3.05 percent of 952 temporal expressions. The temporal variant retained 62.39 percent. Named-entity preservation moved only from 8.03 to 8.22 percent, while event-mention preservation moved from 5.03 to 5.61 percent. On the paper’s measure, the summary was not forgetting everything evenly. It was often preserving who and what while losing when.

A Narrow Repair

The intervention was one added instruction telling the summarizer to preserve dates, times, durations, ages, and relative temporal expressions verbatim. In the 315 matched temporal questions, judge accuracy rose from 0.156 for ordinary SWC to 0.470 for temporal SWC, a paired increase of 0.314 with a reported 95 percent bootstrap interval from 0.254 to 0.375. The direction was positive in all ten conversations.

This does not establish a universal fix for memory. The paper tested the change inside SWC, not inside every summarizer or retrieval architecture. Even after the change, 37.61 percent of detected temporal expressions did not survive verbatim; on the two conversations where both were evaluated, the temporal condition also remained below the full-context reference. The experiment supports explicit temporal protection in this setup; it does not show that one sentence makes compressed memory complete.

The Aggregate Trap

The category table is more informative than one headline score. Ordinary SWC performed much better than truncation on multi-hop and single-hop questions while remaining weak on temporal questions. For open-domain questions, confidence intervals overlapped across all conditions and the authors draw no reliable conclusion. An aggregate can therefore improve while one class of memory remains systematically damaged. Evaluation should ask which faculty was preserved, not only whether the mean rose.

The Benchmark Boundary

LoCoMo is a real published benchmark, but it is not a log of deployed assistants serving real users. Its ACL 2024 paper describes a machine-human pipeline that generated long dialogues from personas and temporal event graphs, followed by human verification and editing. The ten-conversation release averages about 600 turns and 16,000 tokens over as many as 32 sessions. This makes it useful for controlled long-range questions. It does not establish how often timestamp loss occurs in workplace agents, medical records, legal files, or personal companions.

The Artifact Boundary

The public companion repository pinned at commit 2a8e2c0 contains MIT-licensed code and aggregate JSON files for the matched comparison, bootstrap intervals, preservation analysis, and judge validation. It does not include the LoCoMo data, raw logs, or session-summary caches. I inspected those tracked artifacts and recomputed selected ratios from the JSON, but did not rerun the paid model calls.

There is also an unresolved methods conflict. The paper says all reported results used temperature zero. The repository’s pinned assumptions file says the historical runs currently reported used Anthropic’s default temperature, and that the code sets temperature zero only from that commit forward. The same file refers to a variance-analysis artifact that is absent from the tracked results tree at that commit. Until paper and artifact agree, the sampling claim should be treated as unverified and the exact estimates as reported results rather than an independently reproduced fact.

The Preservation Contract

A production compressor should declare what must survive before it is allowed to rewrite memory. The contract should name protected fields such as dates, deadlines, durations, sequence, quantities, identities, negations, permissions, provenance, and unresolved commitments. Each field needs a test, a failure threshold, and a route back to the source. A semantic paraphrase may be acceptable for an anecdote; it is not enough when “before approval,” “after revocation,” or “within 30 days” controls an action.

The corresponding receipt should record the source span, summary version, compressor model and prompt, token budget, retained and discarded tiers, field-level preservation scores, question categories, judge and human-validation protocol, raw-record retrieval path, artifact commit, and correction history. That receipt turns compression from invisible housekeeping into a reviewable transformation.

The Governance Standard

The institutional lesson is this site’s inference from the experiment: never let a gist summary become the only authoritative copy of consequential history. Keep the raw record under an appropriate retention and access policy; attach timestamps as structured data rather than hoping prose will preserve them; test memory by question type; and route high-stakes temporal queries back to source evidence. A summary that remembers the event but loses the clock has not merely forgotten a detail. It has changed what the event can authorize.

Sources

Return to Blog