[ Skip to main content ](https://www.dailydoseofds.com/p/hands-on-agent-memory-is-only-as-good-as-its-schema/#sx-main) [ ![Daily Dose of Data Science](https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2025/12/dailyds-logo-transparent-1.png) ](https://www.dailydoseofds.com) * [ Newsletter ](https://dailydoseofds.github.io/ai-engg-book/) * [ Archive ](https://www.dailydoseofds.com/archive/) * Guidebooks * [ MCP Guidebook ](https://www.dailydoseofds.com/what-is-mcp/) [ Agents Guidebook ](https://drive.google.com/file/d/1Th8mN_IF7Ttc8bI_OLtUuQ7Mjx3aJ6Hi/view) [ AI Engineering Guidebook ](https://drive.google.com/file/d/1_s2UaD7tEPECjgC5YMOa1LIOB9HUWFO-/view) [ DS Guidebook ](https://drive.google.com/file/d/1ZunUHwkD2Ut0_LrgDTDmlQRQkfT2lkVf/view) [ Courses ](https://www.dailydoseofds.com/) [ MLOps/LLMOps Course ](https://www.dailydoseofds.com/mlops-crash-course-part-1/) [ Reinforcement Learning Course ](https://www.dailydoseofds.com/rl-course-part-1/) [ AI Agents Course ](https://www.dailydoseofds.com/ai-agents-crash-course-part-1-with-implementation/) [ MCP Course ](https://www.dailydoseofds.com/model-context-protocol-crash-course-part-1/) [ RAG Systems Course ](https://www.dailydoseofds.com/a-crash-course-on-building-rag-systems-part-1-with-implementations/) * Courses * [ MLOps/LLMOps Course ](https://www.dailydoseofds.com/mlops-crash-course-part-1/) [ Reinforcement Learning Course ](https://www.dailydoseofds.com/rl-course-part-1/) [ AI Agents Course ](https://www.dailydoseofds.com/ai-agents-crash-course-part-1-with-implementation/) [ MCP Course ](https://www.dailydoseofds.com/model-context-protocol-crash-course-part-1/) [ RAG Systems Course ](https://www.dailydoseofds.com/a-crash-course-on-building-rag-systems-part-1-with-implementations/) [ LLM and Fine-tuning ](https://www.dailydoseofds.com/bi-encoders-and-cross-encoders-for-sentence-pair-similarity-scoring-part-1/) [ Classical ML and Deep Learning ](https://www.dailydoseofds.com/where-did-the-assumptions-of-linear-regression-originate-from/) [ Engineering Best Practices ](https://www.dailydoseofds.com/8-fatal-yet-non-obvious-pitfalls-and-cautionary-measures-in-data-science/) * * [ Newsletter ](https://dailydoseofds.github.io/ai-engg-book/) [ Archive ](https://www.dailydoseofds.com/archive/) Guidebooks * [ MCP Guidebook ](https://www.dailydoseofds.com/what-is-mcp/) * [ Agents Guidebook ](https://drive.google.com/file/d/1Th8mN_IF7Ttc8bI_OLtUuQ7Mjx3aJ6Hi/view) * [ AI Engineering Guidebook ](https://drive.google.com/file/d/1_s2UaD7tEPECjgC5YMOa1LIOB9HUWFO-/view) * [ DS Guidebook ](https://drive.google.com/file/d/1ZunUHwkD2Ut0_LrgDTDmlQRQkfT2lkVf/view) * [ Courses ](https://www.dailydoseofds.com/) * [ MLOps/LLMOps Course ](https://www.dailydoseofds.com/mlops-crash-course-part-1/) * [ Reinforcement Learning Course ](https://www.dailydoseofds.com/rl-course-part-1/) * [ AI Agents Course ](https://www.dailydoseofds.com/ai-agents-crash-course-part-1-with-implementation/) * [ MCP Course ](https://www.dailydoseofds.com/model-context-protocol-crash-course-part-1/) * [ RAG Systems Course ](https://www.dailydoseofds.com/a-crash-course-on-building-rag-systems-part-1-with-implementations/) Courses * [ MLOps/LLMOps Course ](https://www.dailydoseofds.com/mlops-crash-course-part-1/) * [ Reinforcement Learning Course ](https://www.dailydoseofds.com/rl-course-part-1/) * [ AI Agents Course ](https://www.dailydoseofds.com/ai-agents-crash-course-part-1-with-implementation/) * [ MCP Course ](https://www.dailydoseofds.com/model-context-protocol-crash-course-part-1/) * [ RAG Systems Course ](https://www.dailydoseofds.com/a-crash-course-on-building-rag-systems-part-1-with-implementations/) * [ LLM and Fine-tuning ](https://www.dailydoseofds.com/bi-encoders-and-cross-encoders-for-sentence-pair-similarity-scoring-part-1/) * [ Classical ML and Deep Learning ](https://www.dailydoseofds.com/where-did-the-assumptions-of-linear-regression-originate-from/) * [ Engineering Best Practices ](https://www.dailydoseofds.com/8-fatal-yet-non-obvious-pitfalls-and-cautionary-measures-in-data-science/) Sign In [ Get Started ](https://www.dailydoseofds.com/membership) / * [ Newsletter ](https://dailydoseofds.github.io/ai-engg-book/) * [ Archive ](https://www.dailydoseofds.com/archive/) * Guidebooks * [ MCP Guidebook ](https://www.dailydoseofds.com/what-is-mcp/) * [ Agents Guidebook ](https://drive.google.com/file/d/1Th8mN_IF7Ttc8bI_OLtUuQ7Mjx3aJ6Hi/view) * [ AI Engineering Guidebook ](https://drive.google.com/file/d/1_s2UaD7tEPECjgC5YMOa1LIOB9HUWFO-/view) * [ DS Guidebook ](https://drive.google.com/file/d/1ZunUHwkD2Ut0_LrgDTDmlQRQkfT2lkVf/view) * [ Courses ](https://www.dailydoseofds.com/) * [ MLOps/LLMOps Course ](https://www.dailydoseofds.com/mlops-crash-course-part-1/) * [ Reinforcement Learning Course ](https://www.dailydoseofds.com/rl-course-part-1/) * [ AI Agents Course ](https://www.dailydoseofds.com/ai-agents-crash-course-part-1-with-implementation/) * [ MCP Course ](https://www.dailydoseofds.com/model-context-protocol-crash-course-part-1/) * [ RAG Systems Course ](https://www.dailydoseofds.com/a-crash-course-on-building-rag-systems-part-1-with-implementations/) * Courses * [ MLOps/LLMOps Course ](https://www.dailydoseofds.com/mlops-crash-course-part-1/) * [ Reinforcement Learning Course ](https://www.dailydoseofds.com/rl-course-part-1/) * [ AI Agents Course ](https://www.dailydoseofds.com/ai-agents-crash-course-part-1-with-implementation/) * [ MCP Course ](https://www.dailydoseofds.com/model-context-protocol-crash-course-part-1/) * [ RAG Systems Course ](https://www.dailydoseofds.com/a-crash-course-on-building-rag-systems-part-1-with-implementations/) * [ LLM and Fine-tuning ](https://www.dailydoseofds.com/bi-encoders-and-cross-encoders-for-sentence-pair-similarity-scoring-part-1/) * [ Classical ML and Deep Learning ](https://www.dailydoseofds.com/where-did-the-assumptions-of-linear-regression-originate-from/) * [ Engineering Best Practices ](https://www.dailydoseofds.com/8-fatal-yet-non-obvious-pitfalls-and-cautionary-measures-in-data-science/) * Sign In May 23, 2026 Memory # [Hands-on] Agent memory is only as good as its schema A deep dive on building production-grade memory for Agents. ![Avi Chawla](../images/000.jpg) Avi Chawla 👉 * * * When you give an agent a knowledge graph for memory, the default behavior is that the LLM handling extraction decides the structure on its own. It picks the entity types, the relationship labels, and the attributes. The results are generic. Everything becomes a `Topic` or `Object` and every connection is labeled `RELATES_TO`, resulting in a graph that can’t be queried meaningfully. [​**Zep’s Graphiti (open-source with 26k+ GitHub stars)** ​](https://github.com/getzep/graphiti?ref=dailydoseofds.com) fixes this with a prescribed ontology. ![](../images/001.png) You can define entity types and edge types as Pydantic models, and the extraction model classifies against these definitions instead of guessing. Let’s look at why this matters, how the extraction pipeline works, with code! * * * #### Why flat retrieval breaks on multi-hop reasoning Vector-based memory stores facts as text chunks and retrieves them by semantic similarity. That works until a query requires connecting facts that don’t appear in the same chunk. Consider three facts stored about a project. * Alice manages Project Atlas * Project Atlas runs on PostgreSQL * The PostgreSQL cluster went down Tuesday A query like “`was Alice’s project affected by Tuesday’s outage`” needs all three. ![](../images/002.png) Vector search will retrieve just facts 1 and 3 because both mention relevant terms. Fact 2 is the bridge connecting Alice to PostgreSQL through Project Atlas, but it mentions neither Alice nor Tuesday. Similarity search misses it. A knowledge graph stores entities as nodes and relationships as edges. Instead of matching text, it traverses connections. That chain (Alice → manages → Project Atlas → runs on → PostgreSQL) is what makes multi-hop reasoning work, and it is invisible to flat vector retrieval. * * * #### Where extraction fits in the pipeline Every graph-based memory system follows the same pipeline. LLM accepts the raw data and extracts entities and relationships. These get stored as nodes and edges, and at query time, the system searches the graph and injects relevant facts into the agent’s prompt. The extraction step determines everything downstream. It decides what the graph contains, how it’s structured, and what can be queried. When extraction is unstructured, the LLM picks entity types and relationship labels on its own. ![](../images/003.png) A developer conversation about building a web app called Nexus with Python, TypeScript, React, and Docker produces nodes labeled `Topic` and `Object` with `RELATES_TO` edges across the board. Two things break without proper structure: * Retrieval collapses to semantic similarity because it can’t filter by type when everything shares the same label. * Domain constraints disappear because there are no structured attributes like `status` or `category` to distinguish “active vs. completed” or “frontend framework vs. database.” * * * #### Defining the schema with Pydantic The fix is the same pattern used everywhere in the AI stack. * FastAPI endpoints get Pydantic response models. * Function calling tools get Pydantic schemas. * Agent memory works the same way in Zep. Define custom entity types using `EntityModel` (a subclass of Pydantic’s `BaseModel`) with `EntityText` fields and descriptions that guide the extraction model. ![](../images/004.jpg) The docstrings and field descriptions are important here because good descriptions with concrete examples give the extractor enough signal to classify accurately. The Pydantic descriptions above aren’t just classification instructions. They teach the extractor vocabulary it doesn’t know. A Technology entity follows the same pattern. ![](../images/005.jpg) Edge types use `EdgeModel` and carry their own attributes. ![](../images/006.jpg) Finally, wire these into the graph with source/target constraints using `EntityEdgeSourceTarget`, which defines which entity types can connect through which edge types: ![](../images/007.jpg) The code enforces that: * `WORKS_ON` can only connect a `User` to a `Project` * `USES_TECHNOLOGY` can only connect a `User` to a `Technology`. * Any relationship that doesn't match these constraints won't produce a typed edge. #### Under the hood When a conversation is ingested with a schema, Zep’s extraction pipeline runs through five steps: ![](../images/008.png) 1. Entity extraction identifies named entities in the text and classifies them against the defined types 2. Entity resolution merges duplicates (”Nexus” and “the Nexus project” become one node) 3. Fact extraction identifies relationships and outputs them as typed edges using the source/target constraints 4. Fact resolution detects contradictions and invalidates outdated facts, preserving history through Graphiti’s bi-temporal model, which tracks both event time and ingestion time 5. Temporal extraction parses time references and maps them to validity windows on each edge The Pydantic schema defined above guides steps 1 and 3. Entity types tell the extractor what to look for. Edge types with their constraints tell it what relationships to classify. Resolution and temporal processing happen automatically. Every edge also carries explicit validity intervals (`t_valid`, `t_invalid`). When information changes (”Alex moved from Project Atlas to Project Nexus”), the old fact is invalidated, not deleted. This way, you can query what’s true now as well as reconstruct what the state would have been at any point in time. #### The output in practice We ingest a conversation where a developer named Alex discusses their work (an active web app called Nexus, their tech stack, proficiency levels): ![](../images/009.png) Querying for `Project` nodes returns Nexus with populated `project_status` and `project_type` attributes. ![](https://substackcdn.com/image/fetch/$s_!F1gH!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6a1f879-cc56-4d2e-8010-e04fedcf9871_1456x134.png) The node isn’t a generic “Topic” or “Object.” It’s a `Project` with structured fields as defined in the schema. The edges are typed too. * `WORKS_ON` carries `role: lead developer` ![](../images/010.png) * `USES_TECHNOLOGY` carries `proficiency: advanced` for Python and Docker, `proficiency: intermediate` for TypeScript. ![](../images/011.png) This can now filter projects by status, technologies by category, and query “which active projects use PostgreSQL” with a precise answer. #### Context templates The final piece is context templates, which assemble typed facts into a prompt-ready block. You can define which edge types and entity types to include, and Zep formats them with temporal annotations into a single string injected into the agent’s prompt. ![](https://embed.filekitcdn.com/e/k7YHPN24SoxyM8nGKZnDxa/ruQ3fXbQn2isXezR1Qeo26/email) It looks like this: ![](../images/012.png) Every entry in the resulting context block is typed, temporally annotated, and carries the attributes defined. Save the template once, reference it by ID in agent calls. * * * #### The 10/10/10 constraint and schema as a reasoning boundary Zep enforces a hard limit of 10 custom entity types, 10 custom edge types, and 10 fields per type. ![](../images/013.png) That’s intentional to force a dev to think about what matters in a domain rather than modeling everything. The source/target constraints also act as guardrails on what an agent is allowed to remember. If a schema doesn’t include an edge type connecting `Project` to `Competitor`, the extraction model won’t create that relationship, even if a conversation mentions both. The schema defines the space of valid memories. This is the same principle behind typed function calling, where we constrain the LLM’s output space so that it can’t produce invalid arguments. Memory schemas apply that same constraint to what the agent stores. Start with 3-4 entity types and 3-4 edge types that capture 80% of your domain logic, and add complexity incrementally. * * * Agent memory without schema discipline is a graph that behaves like a vector store. In a way, you pay the cost of graph construction without getting the benefit of structured retrieval. The schema is how you get that benefit back, and the fact that it’s Pydantic means there’s nothing new to learn. [​**You can find Zep’s GitHub repo here →** ​](https://github.com/getzep/graphiti?ref=dailydoseofds.com) (don’t forget to star 🌟) Thanks for reading! Published on May 23, 2026 Comments Share Copy link [ Share to X ](https://x.com/intent/tweet?url=https%3A%2F%2Fwww.dailydoseofds.com%2Fp%2Fhands-on-agent-memory-is-only-as-good-as-its-schema%2F&text=%5BHands-on%5D%20Agent%20memory%20is%20only%20as%20good%20as%20its%20schema) [ Share to Facebook ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fwww.dailydoseofds.com%2Fp%2Fhands-on-agent-memory-is-only-as-good-as-its-schema%2F) [ Share to Linkedin ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fwww.dailydoseofds.com%2Fp%2Fhands-on-agent-memory-is-only-as-good-as-its-schema%2F) Copied [ Previous Model-Free Learning ](https://www.dailydoseofds.com/rl-course-part-4/) [ Next Function Approximation ](https://www.dailydoseofds.com/rl-course-part-5/) [ ![Daily Dose of Data Science](https://storage.ghost.io/c/3f/df/3fdf6ed2-17ac-4b12-a693-8078bd13e748/content/images/2025/12/dailyds-logo-transparent-1.png) ](https://www.dailydoseofds.com) A daily column with insights, observations, tutorials and best practices on python and data science. Read by industry professionals at big tech, startups, and engineering students. * Menu * [ Contact ](https://www.dailydoseofds.com/contact/) * [ FAQ ](https://www.dailydoseofds.com/faq/) [Daily Dose of Data Science](https://www.dailydoseofds.com) © 2026 [ ](https://x.com/_avichawla "X \(Formerly Twitter\)")