AI Memory Insights: Does Writing Notes Really Make Claude Smarter?
Anthropic's 6 November post scores agent memory on CL-Bench 1.0: Opus 5.5 rises from 0.655 to 0.838, yet a rule to record only failures cuts the gain by 0.234. Storage size is not the point.
Kierkegaard's line about living forward and understanding backward gets quoted at every second software conference, usually to no effect. Anthropic's engineering team used it on 6 November anyway, and this time it holds weight. The subject is what an AI agent should carry from one session to the next, and the uncomfortable part is that the obvious answer — save the transcript, reload it later — is close to the worst one available.
The analogy the team reaches for is biological. Human memory encodes almost everything in the moment, but only a fraction survives into long-term storage, and much of that consolidation happens offline, during sleep and rest, when the hippocampus replays the day. Agents have an equivalent window: the gap between sessions. Anthropic calls the work done there dreaming, borrowing a term already circulating in the literature as sleep-time compute — the researchers behind it at Letta and UC Berkeley published the idea as arXiv 2504.13171.
The article is worth reading closely because it is one of the few public write-ups where a lab shows its own agent failing, then shows the measurement that made the failure legible. What follows is the argument, the numbers, and the parts Anthropic left unresolved.
Four Layers Between One Session and the Next
Anthropic's framework stacks memory into four layers, bottom to top.
At the base sit transcripts — a searchable event log of what the agent actually did. Above them are memory notes, short observations and rules the model writes for itself. Above those sit memory representations: a wiki is the example, a set of interlinked pages the agent composes and maintains the way a person builds a mental map. On top is injected context, the material preloaded into every session before the first token of a task.
Dreaming moves material up and down this stack. It reads transcripts to check notes, composes representations, promotes the important notes into injected context, and demotes details into links. The definition is deliberately narrow: memory here means context Claude acquired from experience and keeps across sessions. Weight updates do not count. Neither does in-context learning, nor compaction — the trick of replacing old turns with a summary.
That boundary matters more than it looks. Much of what vendors call "memory" in 2026 is compaction with better branding, and compaction cannot answer a question that was never asked in the current window.
Memory Is a Bet on What the Next Session Will Need
If memory is a prediction, it can be scored like one. Anthropic used the database task in Continual Learning Bench (CL-Bench) 1.0, where every question opens a fresh session and the only thing carried over is a shared memory directory. The metric is mean reward, and the results separate the model generations more sharply than most public leaderboards do.
| Model | No memory | With memory |
|---|---|---|
| Claude Opus 5.5 | 0.655 | 0.838 |
| Claude Sonnet 5.5 | 0.644 | 0.833 |
| Claude Opus 5 | 0.564 | 0.828 |
| Claude Opus 4.7 | 0.383 | 0.700 |
| Claude Sonnet 5 | 0.306 | 0.610 |
| Claude Sonnet 4.6 | 0.330 | 0.364 |
The top three rows are the interesting ones. Opus 5.5 and Sonnet 5.5 lose only about 0.18 to the stateless condition — the memory directory is doing real work, but the gap is modest. Sonnet 5 roughly doubles, from 0.306 to 0.610. Sonnet 4.6 barely moves, gaining 0.034.
Anthropic's explanation is that memory only pays when the model runs a full loop: hit a failure, write it down, form a hypothesis, test it, then promote the survivor to a rule. Opus 5 was the first model in the series observed doing all five steps unprompted. Older models wrote notes but never revisited them, which is roughly as useful as a diary nobody opens.
What Deserves to Be Written Down
Knowing what to record is harder than recording it. Anthropic tested this on Pokémon Red, an environment where the game state changes constantly and a note written at step 4,000 is stale by step 6,000.
At around the 10,000-step mark, Claude Sonnet 3.7 had produced 413 memory files across 225 folders, 977,930 characters in total, and won one badge. Claude Fable 5 had 9 files plus a single top-level `global.md` — 10 files, two folders, 43,406 characters — and four badges.
The failure mode is visible in the log. Sonnet 3.7 opened a fourth "master" folder near step 10,000, then created six files and deleted nine within 17 steps without taking a single action in the game. It was maintaining the memory instead of playing. Fable 5 kept what the game would not say out loud: strategies, plans, rules it had verified. After seven failed Poké Balls on a Magikarp it wrote a rule to change rhythm after three consecutive misses, then filed it where the next session would find it.
The lesson generalises past games. A memory system that captures everything becomes a second job. One that captures only what transfers becomes leverage.
Telling Claude What Memory Is For, Not How to Use It
Anthropic's steering guidance is short: describe the purpose of the memory directory, not the procedure. Filling the prompt with prescribed steps measurably hurt.
The clearest experiment is the negative one. Instructing Claude to "record only failures" cut the memory gain by 0.234 for Opus 5 and 0.150 for Opus 4.7; Sonnet 4.6 lost 0.020. A rule that sounds like good practice — learn from mistakes — threw away most of the benefit, because the salient thing in a session is often a success worth repeating, not a failure worth avoiding.
Prompting helped most where the model was weakest. Sonnet 4.6 sat at 0.330 without memory, 0.364 with it, and 0.641 once it was told to read memory before answering and to explicitly test its own hypotheses. That is a jump of 0.311 from two sentences of guidance, past Opus 4.7's 0.700 ceiling. Behavioural difference underneath: Sonnet 5 reads its memory files before answering in 9 of 10 cases unaided, Sonnet 4.6 in fewer than half.
Dreaming: Five Jobs the Agent Does Between Sessions
The middle of the article is a catalogue of what offline processing actually buys. Five jobs, each with a measurement attached.
Correcting bad notes. On Mt. Moon in Pokémon Red, Claude misrecorded the direction of a PokéCenter, concluded the dungeon was finished, fell down a one-way ledge, and misdiagnosed its own position as "far west." Dreaming checked the note against the transcript and fixed it. With memory plus dreaming the run reached Cerulean City at action 1,935. With raw memory it fell three times and ended back in Pewter City; with no memory it never arrived at all, inside a 3,000-action budget.
Building representations. Anthropic ran a wiki over the internal documentation of Stainless, a company it acquired in May 2026 — roughly 350,000 files, about 420 million tokens, and a 25-question evaluation set. Growing the wiki from the output of 2 dreaming sessions to 128 sessions moved the score from 13% to 48%. The network went from 103 pages and 766 links to 222 pages and 12,893 links; dreaming volume went from 2 million to 128 million tokens.
Promoting notes upward. Anything injected into every session costs tokens on every turn, and a wrong note promoted to the top becomes a permanent tax. Anthropic's own Claude Code auto memory loads only the first 200 lines or 25 KB of `MEMORY.md` as an index — a hard ceiling that exists because the alternative degrades instruction-following.
Focusing memory on a goal. In a contract-review task, multiple agents writing to one shared `provisions.md` kept only 41 of 57 graded facts, or 72%, mostly by overwriting each other. After Claude reviewed its own failures and rewrote the instruction — one page per contract, verbatim clause quoting — the run captured 57 of 57, 100%.
Filling gaps between sessions. One wiki carried nine links to a page that did not exist, spread across six pages. A second dreaming pass redirected all nine to the correct target and added three pages, moving the wiki from 38 entries and 225 links to 41 entries and 278 links.
When the Wiki Outgrows the Corpus
Not every corpus rewards a wiki, and Anthropic says so with a table.
On 24 M&A questions from the Harvey LAB benchmark, where more than half required stitching documents together and nine needed judgement or arithmetic, wiki plus search scored 87% against 73% for search alone. On a second lab task, 57 graded criteria that mostly sit inside a single document, the wiki added nothing measurable.
The failure case is the memorable one. For a 10-document, 115 KB corpus, the wiki grew to 68 pages, 255 links and 203 KB — 1.77 times the size of the source material. Anthropic reaches for Borges and the map drawn at the scale of the empire. An agent that maintains a representation larger than the thing it represents has invented a new full-time job for itself, and the tokens are billed either way.
What This Means for Anyone Paying Per Token
Strip the benchmarks down and the finding is economic. Memory is not free intelligence; it is a purchase, and the price is paid twice — once in the tokens spent writing and maintaining the notes, again in the tokens injected into every session.
The numbers support that reading. Wiki plus search cut roughly 20,000 tokens per question on the Stainless corpus while scoring 61% against 54%, but only after 128 sessions of dreaming at 128 million tokens of processing. Search alone held flat at 53% for no maintenance cost at all. Whether the wiki is worth it depends entirely on how many questions the corpus will answer before it drifts.
That is the same mistake buyers make when they pick an AI tool from a leaderboard screenshot: ranking position is a property of a benchmark run, not of your workload. What actually transfers is cost per completed task — tokens, retries, maintenance, and the share of work that silently fails. This is the case for comparing tools the way Anthropic compares memory designs, on measured cost per task rather than on a headline score, and it is why aibookmarks.net organises tools by category and starting price instead of by ranking alone.
The Questions Anthropic Left Open
Two problems stay unsolved, and the article is honest about both.
The first is timing. Writing memory during a session pulls attention away from the task; writing it only during dreaming means the notes arrive late, and a long task may end before consolidation runs. Anthropic has no rule for the split, only examples of it going wrong in both directions.
The second is drift. Notes age, and the correction mechanism costs compute proportional to how much memory exists. One option Anthropic raises is focusing dreaming on fast-changing facts and letting stable ones rot in place; another is scheduling dreaming by access frequency, so cold pages get reviewed rarely. Neither is validated at scale.
There is also an unresolved design question underneath both: whether transcripts should be the substrate at all. If a searchable log plus a good query is enough, most of the representation layer is optional overhead. The Stainless results suggest the wiki earns its keep on cross-document questions and loses money on lookup questions — which is a conditional answer, not a verdict.
What the article does establish is narrower and more useful. Memory helps in proportion to whether the model closes the loop on its own mistakes, judges salience instead of hoarding, and keeps the injected layer small. Those three habits are worth more than any amount of storage.
---
Written by Lance Martin, Anthropic Engineering. First published as The components of agent memory on 6 November 2026; all figures are as reported by the authors.
Published by AI Bookmarks · More AI news →