AI Agent Memory Systems: A Practical Guide for Engineers (Mem0, LangMem, Letta, Zep)

Most agents you build today are amnesiac. A user spends 20 minutes teaching the agent their preferences, context, and goals — and the next session starts from zero. That's not a limitation of LLMs; it's a plumbing problem you can fix.

This post is a practical engineering guide to agent memory. You'll understand the four memory types that matter, when each framework fits, and see enough code to get unstuck.

Why Agents Forget (and Why It's Not the LLM's Fault)

LLMs don't "remember" anything between API calls. Each call is stateless — you pass a context window, get a response, and the model discards everything. The context window is your working memory, and it's finite.

Two problems emerge as you scale:

1. Context bloat: Stuffing every prior message into the prompt is cheap for demos, expensive in production. At 100k tokens per call, a 10,000-conversation history burns your budget instantly.

2. No selective retention: Not all information is equally important. A user's name and role matter for months. A clarification about one task matters for one turn. You need a system that distinguishes between them.

Agent memory is the infrastructure layer that solves this. It sits between your agent and your LLM, deciding what goes into the context window and what gets persisted to external stores.

The Four Memory Types

Every serious agent memory system organises storage into these four layers. Understanding them will help you choose the right architecture.

Type	What it stores	Lifespan	Example
Working	Current conversation turn(s)	Single session	"The user just asked about billing"
Episodic	Specific past interactions	Multi-session	"Last Tuesday the user complained about latency"
Semantic	Extracted facts and preferences	Long-term	"User is a backend engineer who prefers Go"
Procedural	Agent instructions, learned behaviors	Semi-permanent	"Always use metric units with this user"

Most simple chatbots only implement working memory. Production agents that feel genuinely useful implement at least semantic and episodic memory.

The Four Frameworks in 2026

Mem0 — Best General-Purpose Choice

Mem0 (48k+ GitHub stars, Series A funded) is the most widely adopted framework. It uses a hybrid store: vector search for similarity retrieval, a knowledge graph for relational facts, and key-value for structured attributes.

The key advantage: automatic memory extraction. You don't annotate what to remember — Mem0 figures it out by analysing conversation turns.

from mem0 import Memory

memory = Memory.from_config({
    "llm": {"provider": "anthropic", "config": {"model": "claude-sonnet-4-6"}},
    "embedder": {"provider": "openai", "config": {"model": "text-embedding-3-small"}},
    "vector_store": {"provider": "qdrant", "config": {"host": "localhost", "port": 6333}},
})

user_id = "user_123"

# After a conversation turn, let Mem0 extract and store what matters
messages = [
    {"role": "user", "content": "I'm a backend engineer at a fintech startup. We use Go and PostgreSQL."},
    {"role": "assistant", "content": "Got it. I'll keep that context in mind."},
]
memory.add(messages, user_id=user_id)

# On the next session, retrieve relevant context before calling the LLM
context = memory.search("what stack does this user use?", user_id=user_id)
for item in context["results"]:
    print(item["memory"])
    # "User is a backend engineer at a fintech startup"
    # "Uses Go and PostgreSQL"

For production, use the managed API instead of self-hosting. The local setup requires running Qdrant + a graph DB + Redis simultaneously — operational overhead most teams don't need early on.

from mem0 import MemoryClient

client = MemoryClient(api_key="m0-xxx")
client.add(messages, user_id=user_id)
results = client.search("stack preference", user_id=user_id)

---

LangMem — Best for LangGraph Shops

If you're already using LangGraph, LangMem is the natural choice. It integrates directly into the LangGraph state model without any additional infrastructure.

LangMem gives you explicit control over what gets stored. You define memory schemas as Pydantic models, and the agent decides when to write to them via tool calls.

from langmem import create_memory_store_manager
from langgraph.store.memory import InMemoryStore
from langchain_anthropic import ChatAnthropic

# For production, swap InMemoryStore for a persistent Postgres-backed store
store = InMemoryStore()
llm = ChatAnthropic(model="claude-sonnet-4-6")

manager = create_memory_store_manager(
    llm,
    namespace=("user_profile", "{user_id}"),
    store=store,
)

# In your LangGraph node, after getting a response:
async def update_memory(state, config):
    user_id = config["configurable"]["user_id"]
    await manager.ainvoke(
        {"messages": state["messages"]},
        config={"configurable": {"user_id": user_id}},
    )
    return {}

# At session start, inject stored memory into the system prompt
def inject_memory(state, config):
    user_id = config["configurable"]["user_id"]
    memories = store.search(("user_profile", user_id))
    memory_text = "\n".join(m.value["content"] for m in memories)
    return {"system_prompt": f"User context:\n{memory_text}"}

LangMem is free and self-hosted — you own your data. The tradeoff: you manage the persistence layer yourself. For most LangGraph users already on LangGraph Platform with Postgres, that's already solved.

---

Zep — Best When Temporal Reasoning Matters

Zep is built around Graphiti, a temporal knowledge graph engine. Instead of storing flat facts, Zep stores facts with time metadata and automatically handles contradictions.

If a user says "I work at Stripe" in March and "I moved to a new company" in July, Zep knows the old fact is superseded. Other stores will return both. That's a critical difference for long-running personal assistants or CRM-style agents.

from zep_cloud.client import AsyncZep
from zep_cloud.types import Message

client = AsyncZep(api_key="z-xxx")

# Create a user and session
await client.user.add(user_id="user_123", email="user@company.com")
session_id = "session_abc"
await client.memory.add_session(session_id=session_id, user_id="user_123")

# Add messages after a conversation turn
await client.memory.add(
    session_id=session_id,
    messages=[
        Message(role_type="user", content="I just joined Razorpay as a senior engineer."),
        Message(role_type="assistant", content="Congratulations! I've noted that."),
    ],
)

# When starting the next session, retrieve memory as a formatted context string
memory = await client.memory.get(session_id=session_id)
print(memory.context)  # Pre-formatted string ready to inject into your system prompt

The memory.context string is one of Zep's nicest features — it handles formatting for you. The tradeoff: graph construction runs asynchronously, so very recent facts (written minutes ago) may not be retrievable immediately.

---

Letta — Best for OS-Style Memory Management

Letta (formerly MemGPT) treats memory as a first-class agent concern. The agent itself has explicit tools to read and write its own memory banks, similar to how a process manages its own heap.

This is the most powerful option for agents that need to self-manage their memory: research agents, long-horizon task executors, or digital twins.

from letta import create_client

client = create_client()

# Create an agent with defined memory blocks
agent = client.create_agent(
    name="research_assistant",
    memory_blocks=[
        {"label": "human", "value": "Name: User. Role: Senior Engineer. Prefers Go."},
        {"label": "persona", "value": "I am a precise, code-first research assistant."},
    ],
    model="claude-sonnet-4-6",
    embedding="text-embedding-3-small",
)

# The agent can modify its own memory mid-conversation via tool calls
response = client.send_message(
    agent_id=agent.id,
    role="user",
    message="I just switched from Go to Rust for my new project.",
)
# Internally, Letta calls core_memory_replace to update the human block
# No manual extraction step needed

# Inspect what the agent chose to remember
agent_state = client.get_agent(agent_id=agent.id)
print(agent_state.memory.get_block("human").value)

Letta has the steepest learning curve of the four but the most predictable memory behaviour — you can audit exactly what the agent chose to remember and why.

---

Head-to-Head Comparison

	Mem0	LangMem	Zep	Letta
Setup complexity	Low (managed) / High (self-hosted)	Low (if on LangGraph)	Low (managed)	Medium
Memory extraction	Automatic	Semi-automatic	Automatic	Agent-driven
Temporal fact handling	Basic	None	Excellent	Good
Self-hosting option	Yes	Yes (no extra deps)	Yes	Yes
Cost	Free tier + paid API	Free (OSS)	Free tier + paid API	Free (OSS)
Best for	Most teams	LangGraph users	Personal assistants, CRM agents	Long-horizon autonomous agents
GitHub stars	48k+	4k+	8k+	16k+

---

Common Pitfalls

Writing everything to long-term memory

Not every message deserves to be remembered. Storing noise degrades retrieval quality — semantic search will surface irrelevant old facts alongside useful ones. Set a relevance threshold, or use a summarisation step before writing.

Ignoring the retrieval latency

A memory query before every LLM call adds latency. Budget for it. Mem0 and Zep's managed APIs typically add 100–300ms. For latency-sensitive flows, retrieve asynchronously or batch at session start rather than per-turn.

Not handling contradictions

If your store doesn't handle contradictions (most vector stores don't), a user who changes their preferences will have conflicting facts in memory. Either use Zep, or implement a deduplication and update step when writing new facts.

Mixing scopes in one namespace

User-level facts ("who am I talking to?") and session-level facts ("what are we doing right now?") need separate namespaces. Mem0's user_id / agent_id / run_id tagging pattern is the right mental model. Scope your writes carefully from day one — it's painful to retrofit.

Skipping memory evals

You write evals for your agent's outputs — do the same for memory. Test that important facts get stored, irrelevant facts don't, and contradictions get resolved correctly. Memory bugs are silent and compound over time.

---

Choosing Your Path

• Building on LangGraph? → Start with LangMem. Zero new infrastructure.
• Need the simplest possible integration? → Mem0 managed API. One package, one API key.
• Building a long-running personal assistant or CRM agent? → Evaluate Zep for its temporal graph.
• Need the agent to self-manage its own memory with full auditability? → Letta.
• Not sure yet? → Mem0. It's the easiest to swap out later.

Whatever you choose, add memory early. Retrofitting memory into an agent designed without it is painful. The architecture decisions — session IDs, user scoping, context injection — are much easier to bake in from the start than to add later.

---

If you want to go deeper on building production agents — memory, evals, orchestration, and deployment — the Joinloop AI Engineering Cohort covers all of it in a structured, hands-on programme with engineers who are actively shipping AI products.

---

Verify specific API methods and version compatibility against each framework's official docs before production use — these ecosystems move fast.