Technical Notes

Technical Notes

The agent remembered - it just remembered wrong

The agent remembered - it just remembered wrong

Thrindex

A customer asked her assistant to send the quarterly report to her work address. It did — pulled the address from memory, formatted the email, sent it before she'd finished her coffee. Fast and tidy and wrong. She'd changed offices in March and mentioned it at the time. The report went to a floor she no longer worked on.

Nothing broke. There was no error, no failed call, no red line in a log somewhere. The memory system was asked for an address and it gave one back. It gave back the address it had the most evidence for — the one mentioned across a dozen past conversations, embedded again and again, sitting comfortably near the top of every search. The newer fact, said once and in passing, never stood a chance against it.

This is the failure that doesn't look like a failure. The agent didn't forget. It remembered — confidently, instantly — and what it remembered was out of date.

Most memory systems work by similarity. You ask a question, the system looks for the stored text closest in meaning to your question, and hands it back. As an instinct, that's reasonable. As a judge of what's true, it falls apart quickly. "I work out of the Berlin office" and "I've just moved to Munich" look almost the same to a similarity model — same subject, same shape, nearly the same words. The model can see they're about the same thing. It has no way to see that one replaced the other.

So it returns both, and ranks them by confidence. And confidence, quietly, tends to favor the older memory — because the older memory has had more time to be repeated, reinforced, and embedded. The unsettling part is the consequence: a similarity-based system is most certain about the facts that have been true the longest. Which is precisely backwards. The facts most likely to have gone stale are the ones that have had the most time to pile up evidence.

This isn't a rare edge case you can patch around. Almost anything worth remembering about a person tends to change — where they work, what they prefer, what they've decided, who they report to, what something costs. Agents get deployed into exactly those situations: ongoing relationships, long horizons, facts in motion. A memory that can't handle change isn't handling the common case. It's failing it politely.

The mistake underneath all of this is treating memory as a search problem. Searching memory is the easy part. Any system can find the text that matches. The hard part — the part that actually decides whether an agent can be trusted — is knowing which of the matching memories still holds. That isn't a question of similarity. It's a question of reasoning: has this fact been replaced? By what? When? Does the new version contradict the old one outright, or just narrow it?

A system that can't ask those questions isn't really remembering. It's storing. And storage with good search on top is not the same thing as memory. Human memory isn't a transcript you scan top to bottom. It's a living model of what's currently true, quietly corrected every time the world moves.

That's the standard we think agent memory should be held to. Not "can you find what the user said" — that's table stakes. The real question is "do you know what's still true." The two sound almost identical. The distance between them is the distance between an agent that helps and an agent that sends the report to the wrong floor.

The agent remembered. That was never the problem. The problem is that remembering, on its own, was never enough.

A customer asked her assistant to send the quarterly report to her work address. It did — pulled the address from memory, formatted the email, sent it before she'd finished her coffee. Fast and tidy and wrong. She'd changed offices in March and mentioned it at the time. The report went to a floor she no longer worked on.

Nothing broke. There was no error, no failed call, no red line in a log somewhere. The memory system was asked for an address and it gave one back. It gave back the address it had the most evidence for — the one mentioned across a dozen past conversations, embedded again and again, sitting comfortably near the top of every search. The newer fact, said once and in passing, never stood a chance against it.

This is the failure that doesn't look like a failure. The agent didn't forget. It remembered — confidently, instantly — and what it remembered was out of date.

Most memory systems work by similarity. You ask a question, the system looks for the stored text closest in meaning to your question, and hands it back. As an instinct, that's reasonable. As a judge of what's true, it falls apart quickly. "I work out of the Berlin office" and "I've just moved to Munich" look almost the same to a similarity model — same subject, same shape, nearly the same words. The model can see they're about the same thing. It has no way to see that one replaced the other.

So it returns both, and ranks them by confidence. And confidence, quietly, tends to favor the older memory — because the older memory has had more time to be repeated, reinforced, and embedded. The unsettling part is the consequence: a similarity-based system is most certain about the facts that have been true the longest. Which is precisely backwards. The facts most likely to have gone stale are the ones that have had the most time to pile up evidence.

This isn't a rare edge case you can patch around. Almost anything worth remembering about a person tends to change — where they work, what they prefer, what they've decided, who they report to, what something costs. Agents get deployed into exactly those situations: ongoing relationships, long horizons, facts in motion. A memory that can't handle change isn't handling the common case. It's failing it politely.

The mistake underneath all of this is treating memory as a search problem. Searching memory is the easy part. Any system can find the text that matches. The hard part — the part that actually decides whether an agent can be trusted — is knowing which of the matching memories still holds. That isn't a question of similarity. It's a question of reasoning: has this fact been replaced? By what? When? Does the new version contradict the old one outright, or just narrow it?

A system that can't ask those questions isn't really remembering. It's storing. And storage with good search on top is not the same thing as memory. Human memory isn't a transcript you scan top to bottom. It's a living model of what's currently true, quietly corrected every time the world moves.

That's the standard we think agent memory should be held to. Not "can you find what the user said" — that's table stakes. The real question is "do you know what's still true." The two sound almost identical. The distance between them is the distance between an agent that helps and an agent that sends the report to the wrong floor.

The agent remembered. That was never the problem. The problem is that remembering, on its own, was never enough.

GET STARTED

Let's find your perfect coverage

Tell us about your property and we'll create a custom insurance plan just for you in less than 5 minutes.

GET STARTED

Let's find your perfect coverage

Tell us about your property and we'll create a custom insurance plan just for you in less than 5 minutes.

GET STARTED

Let's find your perfect coverage

Tell us about your property and we'll create a custom insurance plan just for you in less than 5 minutes.