A memory you can walk, not scan.
Attention reads the whole past for every token. Tree-Memory organises the past as a hierarchy and reads only what can change the answer — then proves it.
Write. Read. Consolidate.
Write
Each new fact becomes an exact leaf. Ancestors keep a summary — mean key, radius, value sum and count — updated along the path. Only surprising facts are stored, so predictable repeats cost nothing.
Read
Start at the root. Open a branch only if its certified upper bound could still change the answer. Everything else is skipped, with a guarantee rather than a guess.
Consolidate
During “sleep”, important leaves are replayed into small adapters, turning short-term traces into permanent knowledge without overwriting what the model already knows.
Opening only what matters.
The query starts at the root summary. At each level the model compares the bound of the two children, opens the one that can win, and proves that the sibling cannot overtake it. Four levels, five opens — instead of sixteen reads.
Skip rule
A node summarises its keys with a mean μ and radius r. For query q, its attention mass lies between exp((q·μ − ‖q‖r)/√d) and exp((q·μ + ‖q‖r)/√d), times its count. If the range cannot change the result, the node stays closed.
Fresh radii
Radii are recomputed from live keys after write bursts. In our tests this alone cut reads at 32,768 keys from 2,364 to 343 nodes.
Trained keys
Bounds are only tight when keys cluster. We train keys with a hierarchical clustering loss so that related facts sit together in the tree.
Five mechanisms, one memory.
Surprise-gated writes
The model stores what it failed to predict. In a stream of 6,000 tokens drawn from 300 facts, it kept 300 leaves instead of 6,000 — with 100% recall.
Fast memory
New rules are picked up immediately with no backpropagation. On chess, a model that had never seen en passant reached 89–93% on it through episodic memory alone while keeping every old skill.
Supersession & trust
When a fact changes, the newest version wins and the old one is retired but kept. Write-trust rules — reputation, quorum and overwrite budgets — blocked every attacker type in simulation.
Sleep consolidation
Replaying surprising leaves into last-layer adapters taught en passant to 100% while castling, promotion and ordinary moves stayed at 100% — no forgetting.
Multi-token reads
When several tokens are generated together, one shared read covers all of them. In prototypes, grouped reads cost 4–7× less per token. Draft heads that exploit this are on the roadmap.
What is proven, and what is next.
We publish the gaps as well as the wins.
Chess world, 4K → 1M addresses
Reads ≈ 9.4 × log₂ n, recall 1.000, identical to full attention, 100% exact boards. Long-context test: 100% at every fact age; 256- to 4,096-move windows fell to ~57–61%.
Pretrained language models
Converting a real LLM needs trained routing keys. Our first attempt on a 0.5B model taught us what to change; run 2 is next.
GPU kernel & tiered storage
Our reads are about 5,500× fewer than full attention, but the Python reference is not yet faster on the clock. A fused GPU kernel and hot/cold tiers are next.
Noise-robust keys
At 1M addresses the key margin is small under heavy noise. Larger-margin keys are part of the text-model design.