RAG vs Memory
Why retrieving the most similar chunk is not the same as remembering the right state, and the three gaps that turn a vector search into a memory system.
A retrieval system can hand back exactly the passage you asked for and still be wrong. It returns the text that looks most like your question, ranked by similarity. That is not the same as the text that is currently true, that you have already acted on, or that actually happened. The whole field keeps rediscovering this distinction, usually the hard way, after shipping something that retrieves beautifully and remembers terribly.
The cleanest way to feel the gap is a small story that supermemory uses in its docs. On day 1 you tell the assistant you love Adidas. On day 30 you mention your Adidas pair fell apart. On day 31 you say you switched to Puma and they are great. On day 45 you ask: what sneakers should I buy? A pure retrieval-augmented-generation (RAG) system embeds that question, finds the chunk with the highest cosine similarity, and that chunk is "I love Adidas." It confidently recommends Adidas. Every later fact that overrides the first one scores slightly lower against the literal query, so the system surfaces a preference you no longer hold.
#What each one is actually answering
RAG retrieves stateless document chunks. The corpus is the same for everyone, the chunks do not change because you learned something new, and the ranking is "how close is this text to the query." It answers a knowledge question: what do I know about this topic? That is exactly right for a product manual, a legal corpus, or a set of research papers, where the facts are fixed and shared.
Memory answers a different question: what do I remember about you? It tracks facts about a specific person over time, including which ones have been superseded, which were resolved, and which were never reliably established in the first place. Supermemory frames the split well: RAG answers "what do I know," memory answers "what do I remember about you."
It helps to be precise about the relationship, because "RAG vs memory" overstates the opposition. A memory system almost always contains retrieval. It embeds, it stores vectors, it ranks by similarity. The difference is everything wrapped around that step: deciding what is allowed to become a durable fact, deciding when an old fact stops being true, and deciding whether a true fact is even worth surfacing right now. Retrieval is a component. Memory is retrieval plus state management. (RAG covers the component in depth.)
#Three gaps that separate the two
A community thread on agent memory put it sharply: retrieving the right-looking chunk is not the same as remembering the right state. Underneath that line are three concrete failure modes that show up in production, and each has a fix that has nothing to do with choosing a better vector database.
#Storage vs admission
The subtle failure here is unverified model output hardening into "memory." An agent should be able to propose a memory without being able to bless it. Durable writes want a real trigger behind them: a tool-grounded event, an explicit user confirmation, or a deterministic validator. One builder described a useful layering, from most to least trusted: an append-only event log of tool and user facts, then derived memory (summaries that carry source IDs), then working task state, then user-approved preferences, and only at the bottom candidate memory that the model suggested and nobody has checked.
In MemoryPlugin this is a deliberate product stance rather than a background process. The system proposes operations over your memories (merge these near-duplicates, reconcile this contradiction) and nothing changes until you accept, and you can edit the merged text before you do. That is the storage-versus-admission split turned into a UI: the model curates, the human admits. See memory suggestions for the full pattern.
#Confabulation
Self-writing agents do not just go stale. They invent. A builder reported watching a run confidently log three actions it never took, then use those logs as authoritative context in the next session. This is worse than forgetting, because once a confabulated entry is in the store it passes every retrieval check. Similarity search scores how close text is to a query; it has no opinion on whether the text describes something that happened. The defence is to tie writes to a verifiable ground-truth signal with a real timestamp: a tool result, a system event, an external record. If a "memory" cannot be traced to something that actually occurred, it should not be trusted as one.
#Resolved vs relevant
A closed support ticket and an open one look identical to a vector database. Both are semantically close to the query, so both get retrieved, and the agent cheerfully re-answers the question that was settled last week. The fix one builder found was not a smarter embedding but a field: mark the resolution state alongside the chunk, and once something is resolved, stop letting it compete for injection even when similarity would pull it in. Relevance is necessary but not sufficient. A fact can be both highly relevant and completely done with.
#When RAG is the right answer
None of this makes retrieval obsolete, and reaching for a full memory system by default is its own mistake. If your facts do not change and are not specific to one user, the bookkeeping that memory adds (versioning, supersession, admission control, resolution flags) is pure cost with no payoff. A vector index over a stable corpus is the correct tool, and the same flat embeddings that fail the sneaker example are exactly right for "find the clause about refunds."
Memory earns its complexity in the opposite case: when facts evolve, contradict each other, get resolved, and belong to a particular person. That is when "return the most similar chunk" stops being good enough and you need the machinery to ask not just is this relevant but is this still true, did this really happen, and is it done. The next pages take those one at a time: turning conversation into durable facts in smart extraction, and reconciling facts that disagree in updates and conflicts and temporal memory.
#References
- Supermemory, "Memory vs RAG" and "User Profiles" concept docs, github.com/supermemoryai/supermemory (the Adidas-to-Puma illustration and the "what do I know vs what do I remember about you" framing).
- r/LLMDevs, "RAG has not felt like enough for agent memory" (retrieving the right-looking chunk vs remembering the right state; storage-vs-admission, confabulation, and resolved-vs-relevant, reported as practitioner experience).
- r/LocalLLaMA, "Are vector databases fundamentally insufficient for long-term LLM memory?" (flat text with no update mechanism; you can append new chunks but not update an old fact).