Research in the open, measured honestly.
We work on the architecture of AI itself. Every experiment is logged with what worked, what failed and what we changed — and our research repository is public.
What we are working on
Tree-Memory architecture
Log-time memory with certified exact reads, no context window, learning while running.
In researchTernary-100M
A small language model with weights in {−1, 0, +1}, built for efficient hardware.
PrototypeSHVQ-LR compression
Codebook quantization plus a low-rank residual, applied to existing models.
Coming soonGraph-Compressed Embeddings
Anchor vectors and quantized residuals for compact vector storage.
How Tree-Memory got here
Failures are part of the record. The sequence below is why the design looks the way it does.
Prototype and fixes (v1, v2)
Four candidate architectures built. Stale radii explained most of the early cost growth; refreshing them cut reads 7×. Trained keys cut certified reads 5.6–22× on a synthetic task.
Real-model conversion, root cause
On Qwen2.5-0.5B the tree structure was fine, but computable bounds were 4–17× wider than the true score spread, so nothing could be pruned. Fix: train the keys.
Trained routing keys — run 1 missed
A 16-dimensional routing head fell short: recall dropped from 0.59 to 0.28 once the clustering loss switched on. We traced it to a self-reinforcing loss and planned run 2.
Chess testbed: rules, learning, recall
100% rule accuracy on unseen games; the memory scaled to 262,144 addresses at 347× fewer reads; sleep consolidation first failed, then was fixed.
Final chess validation — passed
A factored tree read held at about 9.4 × log₂ n from 4K to 1M addresses: 189 reads of 1,048,576, identical to full attention.
Our standards
Same answer as full attention
An efficiency gain only counts if the answer is identical. We report recall, KL divergence and top-1 agreement against full attention.
Scale to a million
Every memory result is measured from thousands to a million-plus addresses, with the growth exponent fitted — so “log n” is a measurement, not a slogan.
Report the failures
Negative results go in the log with their diagnosis. They are usually the most useful part.