Give an AI agent a memory and it starts filling up. Every conversation adds more. After a few months you have thousands of entries, most of them stale, and every time the agent looks something up it is wading through all of it.
So the standard advice is to prune. Score each memory by how recently it was used, and delete the ones nobody has touched in a while. It sounds like hygiene. Most memory tools either do it or tell you to.
I wanted to know what pruning actually costs, so I measured it. Then I measured the thing the pruning advice never mentions: whether the agent notices when something in your life stops happening.
How I tested it
I used the same exam I run every memory system through: long conversations between two people, 885 questions about what was said, one AI answering from whatever the memory hands it, a separate AI grading. Then I added time.
I loaded a memory store as if everything had been saved on day one, then simulated 180 days of use. Every day, the memory's forgetting process ran, the way it would in real life. Along the way the agent was asked half of the questions, so the memories those questions touched got "used" and the rest did not. At the end of the six months, I asked the other half, the questions nobody had asked during the simulation. Those are the memories that look stale to a pruning rule, and the questions you cannot answer if the rule got it wrong.
Three versions of the store went through this:
- Never forget. Everything stays exactly as it was. The baseline.
- Delete. Memories that fade from disuse are removed.
- Archive. Memories that fade are moved out of the way. They stop showing up by default, but they still exist, and a direct question about them brings them back.
I ran it on MemVault, my hosted product, and again on NSM, my free local one that runs with no AI model at all, to make sure the answer was not an accident of one system.
What I found
Deleting cost a quarter of the answers. On MemVault, the never-asked questions were answered correctly 85.7% of the time on the untouched store. On the store that deleted faded memories, 60.6%. The information was simply gone. On NSM the untouched store scored 63.8% and the deleting store 47.6%, a 16-point drop with the same cause.
Archiving cost nothing. Literally nothing: 86.0% on MemVault against 85.7% untouched, 63.8% against 63.8% on NSM. Not a single question changed its answer. The archived memories did not get in the way of anything, and when a question needed one, the question itself was enough to pull it back.
That is the whole result for pruning, and it is not close. The feeling that a tidy store answers better is wrong. A tidy store answers the same. A deleted store answers worse. The fix is one setting: archive instead of delete. Both of my systems now do that by default.
The part nobody talks about
While I was testing forgetting, I ran into the opposite problem. Memory systems are built to find what you said. They have no way to notice what you stopped saying.
Say you log a run every week for three months, and then you stop for a month. There is no entry that says "stopped running." There is nothing to search for. If you then ask the agent to plan your training, it plans as if nothing happened. The gap is invisible to a system that can only retrieve things that exist.
I built a test for this, since no public one exists: 24 made-up people with everyday routines, runs, sleep, bills, calls home, some of which quietly stop. Then everyday requests that never mention the gap.
Without any help, the agent noticed a stopped routine in about one answer out of twenty. With a small check added to the memory system that looks at the timing of each routine and flags a gap when one is well overdue, it noticed in about one out of two. On MemVault that went from 4.5% to 54.5%; on NSM, with no AI model involved in the check, from 9.1% to 50.0%. The flagged version did not start inventing problems that were not there.
Half is not great. But it is ten times better than the default, and the default is what every memory product ships with.
What to do about it
- Never let your agent delete memories. If the tool you use has a pruning or cleanup setting, look for an archive option instead. If it only deletes, turn it off. The store growing is not the problem you think it is.
- If you keep routines in your agent's memory, ask it directly about gaps. "Is there anything I usually do that I have not done lately?" The agent cannot volunteer that on its own unless the memory is built to flag it, but it can usually answer when asked.
- If you want the flags built in, both of my systems have them. NSM runs on your own computer with no account and does the check with no AI model at all.
What I did not test
The simulation is six months of a two-person conversation, not six months of a real working life with thousands of varied entries. The stopped-routine test is one I built myself, on invented people, because nobody has published one; the dataset and the results are frozen so it can be checked, but it is my test and I say so. And the absence check catches gaps in timing. It does not catch a routine that changed in kind, like a different store or a different person, because every version that tried also flagged things that were normal.
Every number here traces to a run I can show you, including the ones that went down. The technical write-up, with the architecture, the statistics and the parts that did not work, is here: Why RAG is the wrong architecture for AI agent memory.