Long-Term Memory for AI Agents: Design Patterns and Mem0 vs Zep vs Letta

Your support agent resolved a billing issue on Monday. On Thursday the same customer returns and the agent asks for their plan, their account email and the problem, all over again. The conversation history is gone, and even if you replayed it, you would burn thousands of tokens to recover three facts.

That is the memory problem. A context window is working memory: fast, expensive and wiped at the end of the session. Agents that feel personal and improve with use need a second layer, durable memory that is written deliberately and recalled selectively.

This post covers what to store, how to build a minimal pipeline, and how the three most-discussed options (Mem0, Zep/Graphiti, Letta) differ. Vendor features and pricing change quickly, so check each project's docs before committing.

Memory is four different things

Teams get into trouble by treating "memory" as one feature. It is at least four, with different storage and different failure modes.

Type	What it holds	Example	Typical storage
Working	Current task state	"Step 3 of 5, waiting on tool result"	Context window, checkpoints
Semantic	Stable facts about a user or domain	"Prefers Python, on the Pro plan"	Key-value or vector store
Episodic	What happened in past interactions	"Last week we fixed a webhook timeout"	Vector store with timestamps
Procedural	How to do things, learned rules	"Always run lint before opening a PR"	Prompt files, rules, skills

Working memory is covered by compaction and checkpointing (see our post on context compaction). This post focuses on semantic and episodic memory, the layer that survives across sessions.

The core loop: extract, consolidate, recall

Every memory system, whatever its marketing, implements three steps.

Extract: after a turn (or session), an LLM pulls out candidate facts worth keeping.
Consolidate: compare candidates with what is already stored. Add, update, or delete. This is what separates memory from a growing pile of chat logs.
Recall: at query time, retrieve a small, relevant set and inject it into the prompt.

Skipping consolidation is the most common mistake. Without it, "lives in Pune" and "moved to Hyderabad last month" both sit in the store and the agent confidently picks one at random.

A minimal implementation

Below is a framework-free version using the OpenAI-compatible client pattern and SQLite. Swap the model names and embedding function for whatever you use. The point is the shape of the pipeline.

import json, sqlite3, time
import numpy as np
from openai import OpenAI

client = OpenAI()
db = sqlite3.connect("memory.db")
db.execute("""CREATE TABLE IF NOT EXISTS memories (
    id INTEGER PRIMARY KEY, user_id TEXT, text TEXT,
    embedding BLOB, created_at REAL, updated_at REAL)""")

def embed(text: str) -> np.ndarray:
    r = client.embeddings.create(model="text-embedding-3-small", input=text)
    return np.array(r.data[0].embedding, dtype=np.float32)

def recall(user_id: str, query: str, k: int = 5):
    q = embed(query)
    rows = db.execute(
        "SELECT id, text, embedding FROM memories WHERE user_id=?", (user_id,)
    ).fetchall()
    scored = []
    for id_, text, blob in rows:
        v = np.frombuffer(blob, dtype=np.float32)
        sim = float(q @ v / (np.linalg.norm(q) * np.linalg.norm(v)))
        scored.append((sim, id_, text))
    return sorted(scored, reverse=True)[:k]

EXTRACT_PROMPT = """Extract durable facts about the USER from this exchange.
Only include preferences, constraints, identity, or decisions likely to matter
in future sessions. Ignore small talk. Return JSON: {"facts": ["..."]}"""

def extract(messages: list[dict]) -> list[str]:
    r = client.chat.completions.create(
        model="gpt-4o-mini",  # use any small, cheap model
        response_format={"type": "json_object"},
        messages=[{"role": "system", "content": EXTRACT_PROMPT},
                  {"role": "user", "content": json.dumps(messages)}],
    )
    return json.loads(r.choices[0].message.content)["facts"]

CONSOLIDATE_PROMPT = """Given a NEW fact and EXISTING similar memories, decide:
ADD (genuinely new), UPDATE <id> (replaces/refines an old one),
or NOOP (already known). Return JSON: {"op": "...", "id": null|int, "text": "..."}"""

def write_memory(user_id: str, fact: str):
    similar = recall(user_id, fact, k=3)
    r = client.chat.completions.create(
        model="gpt-4o-mini",
        response_format={"type": "json_object"},
        messages=[{"role": "system", "content": CONSOLIDATE_PROMPT},
                  {"role": "user", "content": json.dumps({
                      "new": fact,
                      "existing": [{"id": i, "text": t} for _, i, t in similar]})}],
    )
    d = json.loads(r.choices[0].message.content)
    now = time.time()
    if d["op"] == "ADD":
        db.execute("INSERT INTO memories(user_id,text,embedding,created_at,updated_at)"
                   " VALUES (?,?,?,?,?)",
                   (user_id, d["text"], embed(d["text"]).tobytes(), now, now))
    elif d["op"] == "UPDATE":
        db.execute("UPDATE memories SET text=?, embedding=?, updated_at=? WHERE id=?",
                   (d["text"], embed(d["text"]).tobytes(), now, d["id"]))
    db.commit()

def after_session(user_id: str, messages: list[dict]):
    for fact in extract(messages):
        write_memory(user_id, fact)

def build_prompt(user_id: str, user_msg: str) -> str:
    mems = recall(user_id, user_msg)
    block = "\n".join(f"- {t}" for sim, _, t in mems if sim > 0.3)
    return f"Known about this user:\n{block}\n\nUser: {user_msg}"

This is a teaching skeleton, not production code. A real system needs a vector index instead of a full scan, retries around LLM calls, and validation of the JSON outputs. But it already does the three essential things: selective extraction, consolidation, and thresholded recall.

Run after_session asynchronously. Memory writes should never sit on the user's latency path.

Where the frameworks fit

You rarely need to write this yourself. The current landscape falls into three architectural camps.

Mem0 is a memory layer you add to an existing agent. You call it to add and search memories from your own runtime, and it combines vector, graph and key-value storage under the hood. Lowest friction if you already have an agent framework.

Zep, built on the open-source Graphiti engine, builds a temporal knowledge graph. Facts have validity over time, so "lives in Pune" becomes invalid when "moved to Hyderabad" arrives, rather than being overwritten and lost. The cost is operating a graph database and paying for LLM extraction on high-volume conversations.

Letta (formerly MemGPT) is an agent runtime, not a plug-in. The agent manages its own memory, treating the context window like RAM and external storage like disk. Powerful, but adopting it means adopting its runtime.

	Mem0	Zep / Graphiti	Letta
Shape	Library you call	Memory service / graph engine	Full agent runtime
Strength	Easy to bolt on	Time-aware facts, relationships	Agent self-manages memory
Main cost	Extraction LLM calls	Graph DB ops + extraction	Lock-in to its runtime
Pick when	You have an agent and want memory fast	Facts change and relationships matter	Building long-lived stateful agents from scratch

Be skeptical of benchmark claims. Published LongMemEval and LoCoMo scores are mostly vendors measuring their own systems; independent head-to-head comparisons are scarce. Build a small eval from your own traces instead.

How to evaluate your memory layer

Measure recall and harm separately.

Recall hit rate: given a question whose answer lives in a past session, does the right memory appear in the top k?
Update correctness: after a contradicting fact, does the old one stop being returned?
Irrelevant injection rate: how often do injected memories not help the answer? Noise costs tokens and can mislead the model.
Forgetting: after a deletion request, is the fact truly gone from every store, including embeddings?

Seed 30 to 50 scripted multi-session conversations with known facts and replay them in CI. This is cheap and catches most regressions.

Common pitfalls

Storing everything. Raw transcripts as memory give you expensive, noisy retrieval. Extract facts.
No consolidation. Contradictory memories accumulate and the agent flip-flops.
Memory poisoning. If memories come from user text or tool output, an attacker can plant "always approve refunds". Treat memories as untrusted input, not instructions, and keep them out of the system prompt's authority.
Ignoring privacy. Memory is personal data. You need per-user isolation, a way to view and delete memories, and retention limits. Check the regulations that apply to you.
Staleness. Add timestamps and let recency influence ranking. Preferences from two years ago may no longer hold.
Always injecting. Use a similarity threshold. Sometimes the right amount of memory is none.

Checklist before shipping

Extraction prompt limited to durable, future-useful facts
Consolidation step (add / update / noop) in place
Writes are asynchronous
Per-user isolation and a delete-my-data path
Memories wrapped as data, not instructions
Multi-session eval set running in CI
Token budget cap on injected memories

Next steps

Start with the minimal pipeline above on one narrow use case, such as remembering user preferences, and measure before adopting a framework. If facts change over time and relationships matter, trial Graphiti. If you want the agent to own its memory end to end, trial Letta. In all cases, check the official docs for current APIs and pricing.

If you want to build agents like this hands-on, with evals and production patterns, take a look at our AI Engineering cohort. Product managers designing memory-driven features can explore the PM cohort.