Memory Engineering: How to Build an Agent That Gets Smarter Every Day
文章目录
Stanford and Google gave 25 AI agents a memory and dropped them into a small simulated town
One of them decided to throw a Valentine's Day party, the invitations spread from agent to agent on their own, and on the day five of them showed up at the right place and time, nobody scripted any of it
Your agent, meanwhile, makes the same mistake on Monday that you corrected on Friday
Same kind of model, the difference is what gets written down, what gets read back and what gets thrown away
Most people turn memory on, let it save everything, and wonder why the agent gets worse instead of better
This article is how to set up memory so your agent is a little smarter every day than it was the day before
The 30-second version
Memory engineering has six parts, and an agent only gets smarter when all six run:
-
Write - save corrections, decisions and results, never whole transcripts
-
Sort - rules, facts and events live in different places, because they're read differently
-
Recall - load only what this task needs, picked by how recent, how important and how relevant it is
-
Reflect - on a schedule, turn the raw notes into a few rules
-
Forget - new facts overwrite old ones, one-off notes expire
-
Review - a person reads the memory once a week, because one wrong line gets repeated forever
Skip write and nothing is learned, skip forget and it learns the wrong thing, skip review and it learns the wrong thing with confidence
What memory actually buys you
Before building anything, it helps to see what the research says memory is worth, and when it backfires
A pile of memories isn't enough
In the Stanford town, every agent kept a memory stream: everything it saw, with a timestamp
The stream alone wasn't what made it work, two other things did: retrieval that scored each memory on recency, importance and relevance, and reflection, where the agent stopped now and then, asked what its memories added up to, and saved the answer as a new memory
In the paper's evaluation the full setup came out the most believable, and taking away reflection, planning or memory made the agents worse each time
Writing down what went wrong beats trying again
Reflexion had the agent write a short note to itself after every failed attempt and read it before the next one
On the HumanEval coding benchmark it reached 91% on the first try, against 80% for GPT-4 on its own
Saving what worked compounds
Voyager, an NVIDIA-led Minecraft agent, saved every piece of code that worked as a skill in a library and reused it later
It collected 3.3x more unique items and reached key tech tree milestones up to 15.3x faster than agents without the library
Memory works best next to forgetting
When Anthropic shipped a memory tool for Claude, it shipped context editing in the same release, which clears stale tool results out of the context
In Anthropic's own agent tests, the memory tool plus context editing improved performance by 39% over the baseline, and on a 100-turn web search task context editing cut token use by 84%
The pattern across all four: memory pays off when it's small, sorted and rewritten, and it backfires when it's a growing transcript
Write: lessons, not transcripts
The default is to save everything, and that's the first mistake
A transcript of yesterday is thousands of tokens the agent has to reread to find the one line that mattered
What's worth writing is short and reusable:
-
a correction: client A wants prices without VAT
-
a decision: we stopped using the old CRM on November 3
-
a preference: reports in bullet points, never tables
-
a result: the subject line with a number in it got twice the replies
Write it the moment it happens, one line, in the agent's own file, not as a summary at the end of the week
The rule: if it won't change what the agent does next time, it doesn't get written
Sort: rules, facts and events live apart
Researchers at Princeton, in a paper called CoALA, split agent memory into the same kinds human memory has:
-
Working - what's in the context right now
-
Episodic - what happened, when, with whom
-
Semantic - facts about the world, the user, the clients
-
Procedural - how to do things, the rules and the skills
Each kind gets read differently: rules get loaded on every task, facts get looked up, events get searched only when something needs a date
Mixed into one file, the agent rereads 300 lines of old events to find the 5 rules it needs every single time
MemGPT, from UC Berkeley, took the idea further: the context is RAM, everything else is disk, and the agent itself decides what to page in and out
The rule: rules on top and always loaded, facts in a lookup, events in a log the agent searches only when it has to
Recall: load only what this task needs
A memory the agent can't find at the right moment doesn't exist, and a memory it reads at the wrong moment is noise
The Stanford agents scored every memory on three things before reading it, and you don't need the math to use the idea:
-
Recent - this week's notes outrank last quarter's
-
Important - a client complaint outranks a typo
-
Relevant - notes are tagged by client, project or task, and only the matching tag gets loaded
The third one does most of the work, it's what stops the agent from writing to one client with another client's details
The rule: memory is loaded by tag, not by default, the task decides what the agent remembers
Reflect: turn notes into rules
Twenty notes that say the same thing are worse than one rule
Once a day, or every few tasks, the agent rereads its new notes and answers one question: what should I do differently from now on?
The answer goes into the rules, and the notes that produced it get archived
This is the step that actually makes the agent smarter, it's where "the client complained about tables three times" turns into "reports go in bullets"
The rule: notes are raw material, rules are the memory
Forget: the newest fact wins
An agent that never forgets will quote last month's price with full confidence
Three kinds of forgetting cover almost everything:
-
Overwrite - a new fact replaces the old one in place, it never goes in next to it
-
Expire - notes about a one-off task get a date and disappear after it
-
Archive - rules that haven't fired in 30 days move out of the always-loaded file
The rule: every fact has exactly one line, and the newest version wins
Review: a person reads the memory weekly
Memory makes the agent's mistakes permanent, and that's the risk nobody mentions when they switch it on
One wrong line, a misread instruction, a joke taken literally, a line copied in from a document somebody else wrote, and the agent repeats it on every task from then on
That's why ChatGPT lets you see and delete what it remembers, and why a memory file you can read beats one you can't
Ten minutes a week is enough: read the rules top to bottom, delete what's wrong, merge what repeats
The rule: if nobody reads the memory, nobody knows what the agent learned
A worked example: one email agent, four weeks
Here's the same idea on a real-looking setup
Say you run one agent that writes follow-up emails to your leads
The default version has memory switched on, every conversation saved, nothing sorted, nothing ever deleted
By week four it holds about 300 notes, loads all of them for every email, and you're still correcting the same things: the wrong sign-off, prices with VAT, emails that run too long, follow-ups to deals that already closed, the old booking link
The memory version:
-
Write: only corrections and results, one line each, 300 notes become 41
-
Sort: 12 rules on top, facts filed by client, everything else in a dated log
-
Recall: each email loads the rules plus the facts tagged with that lead's company, nothing else
-
Reflect: every Friday the week's notes get boiled down, "too long" three times becomes "under 90 words"
-
Forget: the new booking link overwrites the old one, closed deals expire
-
Review: ten minutes on Monday morning reading the rules
Corrections per week go 14, then 9, then 4, then 1, same agent, same model, it just remembers less and better
The file: MEMORY.md
Give it to any agent that can read and write a file, the brackets are the only part you change:
RULES (always loaded, max 20 lines): [how I work, what I never do] FACTS (one line each, newest wins): [client or project | fact | date] LOG (search only when needed): [date | task | what happened | what I'd change] WRITE: after every correction, decision or result, one line, right away NEVER WRITE: transcripts, small talk, anything that won't change the next task RECALL: RULES + the FACTS tagged with this task's [client / project], LOG only when asked REFLECT: every [Friday], turn new LOG lines into at most [3] rules, archive those lines FORGET: overwrite facts in place, expire one-off notes after [7] days, archive rules unused for [30] days REVIEW: every [Monday] a person reads RULES top to bottom and deletes what's wrong
The 20-line cap on RULES is the line that matters most, a rules file that grows forever turns back into a transcript
Which part your memory is missing
When an agent isn't getting smarter, the symptom tells you which part is broken:
-
it repeats the mistake you corrected on Friday → write, the correction was never saved
-
it loads 300 notes to send one email → sort, rules and events are in one pile
-
it brings up the wrong client's details → recall, notes aren't tagged
-
the same note shows up 20 times and never becomes a rule → reflect
-
it quotes last month's price with full confidence → forget, the old fact was never overwritten
-
a weird habit appeared out of nowhere and won't go away → review, nobody reads the memory
Six symptoms, six parts, and a bigger context window fixes none of them
The mistakes that make memory useless
Thinking "memory on" means memory engineering
The switch decides that things get saved, it says nothing about what, where, or for how long
Saving the whole chat
The one line that matters is buried in thousands of tokens the agent pays to reread every time
Rules the agent writes and nobody reads
Unreviewed rules drift, and a drifted rule is followed more faithfully than any instruction you type
One memory for every client
Without tags, the agent's memory of one client leaks into the emails of the next one
Never deleting anything
Anthropic shipped context editing in the same release as its memory tool for a reason, forgetting is half the system
Treating the context window as memory
A million-token window is the desk, not the filing cabinet, and the desk is empty again tomorrow
Where to start tonight
-
Create MEMORY.md with three headings: RULES, FACTS, LOG
-
Go through last week's chats and write down every correction you made more than once, those are your first rules
-
Tell the agent to add one line after every correction, never a summary
-
Set a Friday reflect step: new log lines become at most 3 rules
-
On Monday morning, read the rules and delete at least one
Then count how many times you correct it next week
P.S: An agent doesn't get smarter by remembering more, it gets smarter by remembering less, and better
If you read this far
→ Bookmark this, so the 6 parts, the MEMORY.md template and the six broken memories are right there the next time your agent forgets what you told it on Friday
→ Join my Telegram channel
→ Follow @distortgeekin for a new AI agent breakdown every day







