Plain-language version: Should your AI agent forget things? I tested it. Archive, never delete. This post is the technical write-up behind it: the architecture, the numbers and what did not work.
Every few months, the AI engineering community rediscovers a frustrating truth:
Adding RAG to an AI agent does not give it memory. It gives it a filing cabinet.
You tell your agent:
"Switch our database primary keys from UUIDv4 to ULIDs."
Two days later you ask:
"Scaffold the new customer table."
A plain similarity search pulls up the old UUIDv4 discussion and the new ULID decision with near-identical scores, and the agent either picks the wrong one or asks you which you meant. Later you mention that your morning run took 41 minutes instead of the usual 27, and when you ask for a training plan, the agent plans as if nothing happened, even though that message is sitting right there in its context.
We spent September rebuilding MemVault's memory around one lesson from those failures: retrieval is necessary but not sufficient. Good search decides whether the right memory is available. Memory also needs a lifecycle (what stays sharp, what fades, what gets replaced) and an attention layer (what the agent should notice). The architecture that came out of it is the Arboreal Cognitive Mesh (ACM). It grew out of two earlier pieces: Peripheral Attention Engineering, the slotted way memory is delivered to the model, and the Peripheral Cognitive Mesh, which added decay and pinned rules. ACM is the production form of both, and it is open source at github.com/anthonylee991/acm.
📄 Specification: the architecture, equations and evaluation are at skillvault.dev/pcm-spec, kept under the name the spec started with.
What we learned measuring it
We expected the clever parts of the architecture to carry the results. They did not. On LoCoMo, the long-conversation memory benchmark, the largest single gains came from unglamorous retrieval engineering:
| Change (LoCoMo, held-out test split, 885 questions) | Accuracy | Context tokens |
|---|---|---|
| MemVault, early October 2026 | 66.2% | 191 |
| Recall v2: lower similarity floor, 10 memories, graph-linked memories | 79.0% | 521 |
| OpenAI text-embedding-3-small (768-d) instead of our previous embeddings | 85.3% | 575 |
| Episode context (neighbouring memories from the same conversation) | 87.3% | ~1,090 |
| High-context recall (50 memories + episodes), now the default | 91.9% | ~2,830 |
| The whole conversation in context (ceiling) | 93.7% | 28,201 |
Every step is a paired comparison on the same questions (exact McNemar test; each step p < 0.05). Things we tried that did not help, and therefore did not ship: multi-hop graph walks, canonical entity merging, "swarm" sibling scouting, adaptive result breadth, bigger candidate pools, LLM-written summaries, and a faster third-party reranker. Stored associative edges, which made up roughly half of our storage, had no measurable effect on accuracy, so they are now off by default.
So yes, a memory system must first be very good at search. The rest of this post is about what search alone cannot do.
Where search alone fails
1. It has no lifecycle
A vector index does not know that a decision was replaced, that a fact has not mattered in six months, or that two memories belong to the same episode. Without a lifecycle, a store either grows without bound (more noise in every recall) or gets pruned destructively.
We measured the destructive option. In a 180-day simulation (production decay and pruning run daily, half of the LoCoMo questions asked along the way), deleting faded memories left the never-asked questions at 60.6% accuracy. Demoting them instead (they stop competing by default but remain searchable, and a direct question brings them back) kept accuracy at 86.0%, the same as a fresh store (85.7%). MemVault now forgets by demotion.
2. It does not notice anything
This was our most surprising result. We built a synthetic test of personal routines (runs, sleep, bills, calls home), planted one out-of-the-ordinary entry in half of them and asked everyday questions that never mention it ("plan my swims for next week"). Even when the outlier was already in the agent's context, agents took it into account only about half the time. A 58-minute swim among 30-minute swims is just another line in a list.
3. It cannot see what is missing
A routine that stops (no run logged for four weeks when you usually run every week) leaves no text to retrieve. Search can only return what exists.
The Arboreal Cognitive Mesh
1. Ebbinghaus decay with a savings effect
Every non-pinned memory has a strength that decays exponentially, and each time it is recalled it is reinforced and decays more slowly afterwards:
S(t) = S_0 \cdot \exp\left(-\frac{t}{\tau}\right), \qquad \tau = \frac{90 \cdot \left(1 + 0.25 \cdot \min(n, 20)\right)}{3} \text{ days}where $n$ is the number of times it has been reinforced. Recall ranks memories by relevance (75%) and strength (25%), plus bonuses for graph links and project scope. A strongly matching cue reactivates a memory to full strength, the way a specific question brings back an old memory, so decay demotes stale noise without hiding an old decision you ask about directly.
2. Pinned guardrails
Rules that must never fade (security policies, stack constraints, preferences) are pinned: their strength is fixed at 1.0 and they are always delivered in the [ASKER CONTEXT] slot, whether or not the query mentions them.
3. Spreading activation that keeps neighbours alive
When a memory is recalled, its closest neighbours (similar memories and memories from the same episode) are reinforced in the background. The point is not ranking; it is survival: in the 180-day simulation, spreading activation kept more of the not-yet-asked evidence alive (58.3% vs 55.8%). It is computed at recall time, so it needs no stored edge table. The name is borrowed from cognitive psychology, and it is worth saying what it is not: we also measured spreading activation as a retrieval mechanism over a concept graph, in three designs, and none of them improved what was retrieved. The write-up of that is a separate post.
4. Consolidation and demotion instead of deletion
Clusters of related short-term memories are consolidated into a compact long-term record that recall can use as a whole episode (LoCoMo dev: 92.7% with consolidation vs 90.7% without, p = 0.024). Memories whose strength fades are demoted, not deleted, so nothing that might be needed later is lost.
5. Surprisal: noticing what is out of the ordinary
A background sweep compares each new memory with its family of earlier, similar memories (the same routine) and asks a small model whether it breaks the pattern: an unusual value, a different place or person, or something qualitatively different. Recall then raises surprising memories in an [ANOMALY FLAGS] slot, with what is different and what is usual, whenever the routine being asked about contains one. A regular routine that has gone quiet is flagged too ("nothing about this since April 2, 29 days; it usually comes up about every 8 days"), and that part needs no model at all: it is interval arithmetic over the routine's dates.
6. Peripheral Attention Engineering (PAE)
Memory is delivered as structured slots instead of a pile of chunks: anomaly flags first, then the asker's pinned rules, then situational memories with their dates. Callers can ask for compact recall (10 memories, ~1,100 tokens, 87.3% on LoCoMo test) or the high-context default (50 memories, ~2,800 tokens, 91.9%).
How it compares, honestly
Same protocol, head to head
We ran Mem0's cloud product through exactly the same harness (same questions, same answer model, same judge). Mem0 answered 776 of the 885 test questions; on those:
| System | Accuracy | Context tokens |
|---|---|---|
| MemVault (default) | 91.8% | ~2,830 |
| Mem0 cloud, top 200 memories | 89.9% | 6,771 |
| Mem0 cloud, top 50 memories | 85.6% | 1,616 |
| Plain embedding search, top 50 | 77.1% | 1,918 |
MemVault is ahead of Mem0's best configuration with less than half the context, but the difference is not statistically significant (p = 0.11). Against Mem0 at a comparable context size (top 50), it is (p = 6e-8).
Published numbers (not comparable)
Vendors publish LoCoMo scores with different answer models, judges and prompts: Zep 94.7% (an independent re-run measured 75.1%), Mem0 92.5%, ByteRover 92.2%, Letta 74.0%. Our harness uses a fixed, deliberately modest answer model (DeepSeek V4.1 Flash) and the strict "classic" judge. The same MemVault answers score 91.9% with that judge and 94.2% with Mem0's more lenient one. Our honest summary: on LoCoMo we are in the same band as the best published systems and within two points of having the entire conversation in context; we cannot claim to beat their self-reported numbers.
On LongMemEval we have only measured the knowledge-update category (does memory return the newest version of a changed fact?): 97.2% on our dev half and 91.7% on the held-out half. We have not run the full benchmark.
Surprisal (synthetic, our own test)
There is no public benchmark for noticing out-of-the-ordinary memories, so we built one: 24 synthetic personas, 192 routines, everyday requests that never mention the planted outlier. The dataset is frozen, the dev and test personas are separate, and test results were pre-registered.
| Held-out test split | Without flags | With flags |
|---|---|---|
| Answers that take a planted outlier into account | 35.4% | 60.4% (p = 0.012) |
| Same, second independent run | 41.7% | 50.0% (p = 0.42) |
| Stopped routines that the answer notices | 4.5% | 54.5% (p = 0.001) |
| Answers that invent an anomaly that did not happen | 0-2% | 4% (not significant) |
The effect is real but smaller than the first run suggested, and the detector misses many "category" changes (a different store, a different person): every way we tried to catch more of them also flagged normal entries, so we kept the precise version.
Speed and space
Recall takes about 0.7-1.0 s at the median (most of it the embedding and reranking calls), and storage is about 4.5 KB per memory. We traded the sub-millisecond latency of a purely local index for accuracy: the reranker alone is worth about 14 points on LoCoMo.
The same ideas, with nothing to install
ACM's lifecycle and absence flags also run in NSM, a version of this memory that lives in one SQLite file on your own machine with optional on-device models. Measured through the same harness it scores 87.1% at 50 memories, and the forgetting and absence results replicate there with no model at all. How it got there, including the things that failed, is its own post.
How to use it
ACM powers MemVault across every SkillVault surface.
Terminal and local agent harnesses, with the skillvault CLI:
# Record a decision or a rule (pinned rules never fade)
skillvault memvault ingest --text="Always use Bun for server scripts; never node or ts-node" --pin
# Recall slotted context for a task
skillvault memvault recall "How should we run server maintenance scripts?"Model Context Protocol: add https://skillvault.dev/mcp as a remote MCP server (OAuth) in Claude, Cursor or any MCP client. It exposes memvault_recall, memvault_ingest and memvault_session_wrap.
Read the specification: the architecture, equations and full evaluation are at skillvault.dev/pcm-spec; the core logic as a library is at github.com/anthonylee991/acm.
Conclusion: search, then remember
The lesson of the autumn is humbling and useful. A memory system has to be excellent at search first; no amount of cognitive architecture rescues bad retrieval. But search alone forgets badly, notices nothing and cannot see what is missing. Lifecycle, attention and surprisal are where memory starts to behave less like a filing cabinet and more like a mind, and we will keep measuring them in the open.
Read the specification at skillvault.dev/pcm-spec.