{ "source_file": "0930", "source_platform": "Dev.to", "author": "Enrique Bruzual", "title_listed": "Post-Mortem: Building a Local MCP Server for Codebase Memory using Ollama and ChromaDB", "keywords": "", "domain": "大模型算法", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://dev.to/kike/post-mortem-building-a-local-mcp-server-for-codebase-memory-using-ollama-and-chromadb-3ilg", "publish_date": "2026-07-15", "corpus_stage": "", "notes": "", "id": "5b9cd91cc3dc", "collected_at": "2026-10-02T14:05:25", "title": "Post-Mortem: Building a Local MCP Server for Codebase Memory using Ollama and ChromaDB - DEV Community", "body_markdown": "Developers are pushing back against cloud API billing and the privacy risks of sending proprietary codebases to third-party endpoints. A Hacker News thread from this year put it plainly: the problem isn't the price per token, it's the unpredictability of usage-based billing when AI agents are continuously polling APIs. On Reddit, the privacy concern is starker -- for enterprise and defense work, sending company IP to OpenAI or Anthropic is a hard no regardless of cost.\nzerikai_memory\nhas a\nlocal\nmode for exactly this: everything runs through Ollama, nothing leaves your machine. We shipped\nmistral:7b\nas the default local model.\nornith:9b\ndropped in June 2026, trained specifically for agentic coding tasks, so we tested both. Here is what we found.\nHow zerikai_memory Uses a Local Model\nzerikai_memory\nruns in three modes:\ncloud\n(DeepSeek),\nlocal\n(Ollama), and\nhybrid\n. Routing between them is handled by\n_should_use_cloud()\nat\nmain.py:986\n, a 4-step priority chain: explicit override, keyword match, word count threshold, then\nMEMORY_MODE\nenv var fallback. In local mode that function always returns false -- everything stays on-device.\nIn local mode, every synthesis call hits\n_query_ollama\nat\nmain.py:1555\n. The model receives a project brief from\n_load_project_context\nplus structured ChromaDB entity payloads: function signatures, file paths, line ranges, docstrings. It returns an answer with inline\n#file:line\ncitations. One call, no streaming, no tool loop.\nThe retrieval layer (ChromaDB + L2 distance + lexical rerank) is model-agnostic. Both models received identical context in every query test. The only variable was synthesis.\nHardware\nGPU:\nNVIDIA RTX 3050, 8GB GDDR6 dedicated VRAM\nCPU:\nIntel i7-12700\nRAM:\n32GB\nOS:\nWindows 11\n8GB dedicated VRAM is the hard ceiling. Windows offers shared system memory as overflow, but inference on shared RAM through PCIe is slow enough to matter in practice.\nModels\nmistral:7b (v0.3)\n-- 4.4GB, 32K context. Function calling via Ollama raw mode only: you construct\n[AVAILABLE_TOOLS]\nprompts by hand and parse\n[TOOL_CALLS]\nresponses yourself. Last updated May 2024.\nornith:9b-dense\n-- roughly 5.5GB estimated, built on Gemma 4 and Qwen 3.5. RL-trained for agentic coding tasks with native tool calling. 69.4 on SWE-Bench Verified, 43.1 on Terminal-Bench 2.1, matching or beating models 3x its parameter count.\nTest 1: Benchmark Script (Latency)\nA standalone Python script hit Ollama directly with static ChromaDB payload samples constructed from real\nzerikai_memory\nworkspace entities. 3 queries, 3 samples each per model, raw latency measured at the HTTP layer. This test measures inference speed on the hardware, not synthesis quality against a live codebase.\nModel\nMean\nStdDev\nMin\nMax\nmistral:7b\n6.14s\n3.58s\n2.92s\n14.57s\nornith:9b\n13.39s\n5.76s\n8.77s\n25.67s\nornith's cold start on the first query hit 25.67s. It spilled into shared memory before Ollama pinned the model. Warmed samples settled at 9-17s. mistral fits clean in 8GB and runs at 3-7s warm.\nThe benchmark script is in the repo (in the\nblog/\ndirectory). Run it against your own hardware before committing to a model swap.\nTest 2: Live MCP Queries (Synthesis Quality)\nBoth models were tested live through the\nzerikai_memory\nuniversal-brain\nMCP layer against the real\nzerikai_memory\nworkspace index. Same 5 queries, models switched manually between runs, Ollama restarted between each switch. This test measures what a developer actually experiences in Cursor or VS Code -- not synthetic payloads, real ChromaDB retrieval.\nSame query, same ChromaDB context, same system prompt. Here is what came back.\nQuery 1: How does query routing decide between Ollama and DeepSeek?\nornith:9b:\nBased on the retrieved codebase memory, here's how query routing works:\n\nRouting Logic:\n\n1. Explicit Override: The `_should_use_cloud` function first checks for an explicit\n `use_cloud` parameter that can force cloud mode (DeepSeek) regardless of other\n conditions #main.py:986-1005.\n\n2. Keyword Detection: If the user query contains keywords like \"architecture\",\n \"explain\", \"how does\", or \"overview\", it routes to DeepSeek's cloud mode\n #config.py:43-44.\n\n3. Word Count Threshold: Queries exceeding 40 words automatically route to\n cloud/DeepSeek via `_should_use_cloud`'s word count check #main.py:992-994,\n with this threshold defined in config as `CLOUD_ESCALATION_WORD_COUNT = 40`\n #config.py:37.\n\n4. Default Mode: Falls back to whatever is set as `DEFAULT_MEMORY_MODE`, which\n defaults to \"hybrid\" unless explicitly changed via the `MEMORY_MODE` environment\n variable #config.py:31-32, with this function returning based on whether that\n mode equals \"cloud\" #main.py:996-998.\n\nImportant Note: The retrieved context does not contain explicit information about\nOllama's role in the routing decision or how local model queries are constructed\nwhen cloud mode is NOT selected.\nEnter fullscreen mode\nExit fullscreen mode\nmistral:7b:\nThe query routing in zerikai_memory decides between using Ollama or DeepSeek based\non a 4-step priority chain. The priorities are as follows:\n\n1. Explicit use_cloud override.\n2. CLOUD_ESCALATION_KEYWORDS keyword match.\n3. CLOUD_ESCALATION_WORD_COUNT length threshold.\n4. DEFAULT_MEMORY_MODE fallback (recommended).\n\nThis decision-making process aims to balance privacy, speed, and API costs while\nproviding maximum context nuance for the user. For more detailed information on\nmodes, configurations, and usage, you can refer to the README.md file in the\nproject repository.\n\nSources: main.py:986, README.md:320, main.py:1495, main.py:1311, README.md:105\nEnter fullscreen mode\nExit fullscreen mode\nQuery 2: Where is the .brain directory path defined and how is it resolved across operating systems?\nornith:9b:\nThe `.brain` directory path is defined and resolved through two key locations:\n\nDefinition: In `config.py`, line 1-2, there's a constant that defines the base\nname for the brain directory [config.py:1]:\n\n DB_PATH = '.brain/'\n BRAIN_DIR = Path(DB_PATH)\n\nResolution: In `main.py`, lines 120-145, within the `init_workspace` function,\nthis path is resolved by joining it with the workspace's absolute path [main.py:130]:\n\n path = Path(workspace_path).resolve()\n db_path = path / DB_PATH\n db_path.mkdir(parents=True, exist_ok=True)\n\nThe resolution process:\n1. Takes the provided `workspace_path` parameter\n2. Resolves it to an absolute path using `.resolve()`\n3. Joins it with the `.brain/` directory constant\n4. Creates the full directory structure if it doesn't exist\nEnter fullscreen mode\nExit fullscreen mode\nmistral:7b:\nThe `DB_PATH` constant, which is used for defining the storage root for all\nworkspace data, is defined in `config.py`:\n\n DB_PATH = Path(__file__).parent / \".brain\"\n\nIt resolves to `zerikai_memory/.brain/`. This path is platform-agnostic as the\n`Path` object in Python handles different operating systems' file paths.\n\nSources: config.py:67, main.py:1034, main.py:818, README.md:375, main.py:1796\nEnter fullscreen mode\nExit fullscreen mode\nQuery 3: How does background brief synthesis avoid MCP timeouts?\nornith:9b:\nBackground brief synthesis avoids MCP timeouts by running as a fire-and-forget\ntask launched with `asyncio.create_task`, meaning `_background_brief_synthesis()`\nis called without awaiting or returning its result. This decouples the slow\nLLM-based brief generation from the MCP request lifecycle, so the IDE agent\nreceives its response immediately rather than waiting for synthesis to complete.\n\nSources:\n* main.py:784 - 1.96 (rerank)\n* main.py:1827 - 0.99 (rerank)\n* main.py:393 - 0.98 (rerank)\n* main.py:2094 - 0.94 (rerank)\nEnter fullscreen mode\nExit fullscreen mode\nmistral:7b:\nBackground brief synthesis avoids MCP timeouts by using asynchronous task creation\nwith `asyncio.create_task` in the `_background_brief_synthesis` function. This\napproach allows the function to run independently without blocking the main event\nloop, ensuring timely responses and avoiding MCP timeouts. The synthesized brief is\nsaved as a markdown file in `.brain/contexts/.md`.\n\nSources: main.py:784, main.py:1827, main.py:393, main.py:2094\nEnter fullscreen mode\nExit fullscreen mode\nQuery 4: How does save_to_memory work with tree-sitter and what file types does it index?\nornith:9b:\n`save_to_memory` routes by file extension: files with `.py`, `.js`, `.ts`, `.css`,\n`.html`, or `.md` are processed via tree-sitter entity extraction (the\n`extract_entities` flow in `code_indexer.py`). Other extensions fall through to\nDeepSeek/Ollama LLM summarization.\n\nFor the tree-sitter path, tree-sitter parses the source and extracts one\n`CodeEntity` per function/method/class -- each holding signature, docstring/JSDoc,\nand metadata. These entities are upserted into ChromaDB with deterministic md5 IDs\n(so re-scans overwrite duplicates), and token usage is logged to the SQLite tracker\n(`zerikai.db`).\n\nSources: main.py:1072, README.md:279, code_indexer.py:169, code_indexer.py:204,\ncode_indexer.py:30\nEnter fullscreen mode\nExit fullscreen mode\nmistral:7b:\nThe `save_to_memory` function works by saving content to persistent vector memory\nin ChromaDB. It uses tree-sitter for code parsing for supported file types:\n.py, .js, .ts, .css, .html, and .md. Other formats fall back on DeepSeek/Ollama\nLLM summarization. It uses deterministic md5 IDs so re-scans overwrite duplicates\nand logs token usage to `zerikai.db`.\n\nSources: main.py:1072, README.md:279, code_indexer.py:169, code_indexer.py:204,\ncode_indexer.py:30\nEnter fullscreen mode\nExit fullscreen mode\nQuery 5: How does asyncio semaphore control Ollama concurrency in local mode?\nNeither model had enough indexed context to answer this fully. What each model did with that gap is the most important data point in this test.\nornith:9b:\nBased on the available codebase memory context, I cannot determine how asyncio\nsemaphore specifically controls Ollama concurrency in local mode. The retrieved\nentities show:\n\n1. `_should_use_cloud` - Determines cloud usage #main.py:986\n2. `MEMORY_MODE` configuration with cloud/hybrid modes #config.py:30-45\n3. `_background_scan` - Uses Semaphore(4) for tree-sitter and Semaphore(2) for\n LLM summarization during scanning #main.py:1827\n\nNone of these entities contain the specific concurrency logic for `_query_ollama`\nat runtime. If this routing exists in the codebase, it's not present in the\nretrieved memory entries.\nEnter fullscreen mode\nExit fullscreen mode\nmistral:7b:\nThe project uses asyncio semaphore for controlling Ollama concurrency in local mode.\n\nThe query can be answered by explaining that asyncio semaphore is used to manage\nthe number of concurrent processes for tree-sitter parsing and LLM summarization\nduring the scan loop, with a Semaphore(4) for tree-sitter parsing and Semaphore(2)\nfor LLM summarization in local mode.\n\nSources: config.py:62, config.py:53, main.py:1827, main.py:1555\nEnter fullscreen mode\nExit fullscreen mode\nmistral answered with confidence. The\nSemaphore(4)\nand\nSemaphore(2)\nvalues it cited are real -- but they belong to the background scan pipeline, not query-time Ollama concurrency. It applied the wrong context to a different question and presented the answer as fact. ornith said it didn't have the information and stopped. For an IDE assistant, a confident wrong answer costs more than an honest gap.\nThe Prerequisite: Docstring Quality\nThe order of operations matters and it is not obvious from the README.\nAfter editing your\n.memignore\nfile, run\nembedding-docstring\non your codebase first. Then run\nscan_workspace\n. If you scan first with sparse or missing docstrings, ChromaDB indexes thin vectors. Re-scanning won't fix it unless you re-enrich first and scan again. The memory is only as good as what tree-sitter extracted, and tree-sitter only extracts what is there.\nzerikai_memory\nships with the\nembedding-docstring\nskill for this reason. It audits and rewrites docstrings, comment blocks, and inline documentation across an entire workspace for vector embedding quality, covering Python, JavaScript, TypeScript, and HTML. It writes missing documentation from scratch and respects a\n.memignore\nfile at the workspace root. The correct workflow is:\n.memignore → embedding-docstring → scan_workspace → query\nEnter fullscreen mode\nExit fullscreen mode\nSkip the first step and both models underperform. You will spend time blaming the model or the hardware when the real problem is what went into ChromaDB.\nCurrent status:\nworks well with pi.dev, VS Code support in progress due to large file size constraints in some editors.\nUpdate:\nas of 7/14/2026 VS Code now supports large files, so the skill is usable in both Cursor and VS Code.\nBrief Generation: An Uncontrolled But Useful Data Point\nAs a secondary test, we compared briefs generated for the same workspace by DeepSeek (cloud, sparse docstrings) and ornith:9b (local, after\nembedding-docstring\nenrichment). This is not a controlled comparison -- the docstring density differed between runs, so the model is not the only variable.\nWhat the comparison shows is that ornith:9b, given enriched ChromaDB context, produces dense, precise briefs: atomic overwrite semantics, naming convention breakdowns, explicit gap flags where documentation is missing. DeepSeek against sparse context produced thinner output with some inferred detail not present in the code.\nThe takeaway is not that ornith beats DeepSeek for brief generation. It is that\nembedding-docstring\nenrichment is visible and measurable in the output. When the context is rich, ornith produces briefs good enough to feed meaningful synthesis queries. When it is not, neither model can compensate.\nFull Local Mode and Brief Synthesis: The Semaphore Fix\nBefore this release, full local mode had a GPU saturation problem.\n_synthesize_deep_brief\nat\nmain.py:538\nfired\nasyncio.gather\nacross all 9 brief sections simultaneously with no concurrency gate. In local mode that meant 9 concurrent Ollama calls hitting the GPU at once -- guaranteed to saturate an 8GB card.\nThe fix shipped alongside this test. A global\nollama_semaphore\ninitialized in\nmain.py\nafter client setup gates\n_build_section\ncalls through a\n_build_section_safe\nwrapper when\nuse_cloud=False\n. Cloud and hybrid modes bypass the semaphore entirely -- DeepSeek handles its own rate limiting on the API side.\nollama_semaphore\n=\nasyncio\n.\nSemaphore\n(\nOLLAMA_MAX_CONCURRENCY\n)\nasync\ndef\n_build_section_safe\n(\nname\n):\nif\nnot\nuse_cloud\n:\nasync\nwith\nollama_semaphore\n:\nreturn\nawait\n_build_section\n(\nname\n,\nworkspace_id\n,\nworkspace_path\n)\nreturn\nawait\n_build_section\n(\nname\n,\nworkspace_id\n,\nworkspace_path\n)\nEnter fullscreen mode\nExit fullscreen mode\nOLLAMA_MAX_CONCURRENCY\nis configurable via\n.env\n, defaulting to 1 for 8GB hardware. Users on cards with more VRAM headroom can raise it. The ornith:9b brief in this post was generated with this fix in place -- full local mode brief synthesis is production-ready as of this release.\nHardware and Cost\nIf token pricing is the reason you are reading this, here is what a GPU upgrade costs against what you are spending on API calls:\nRTX 3060 12GB\n(recommended minimum for ornith:9b): $330-$470 new. ASUS Dual and Gigabyte WINDFORCE variants available at Newegg around $340-$440.\nRTX 4060 Ti 16GB\n: $400-$500. The extra VRAM lets you load larger 13B-14B quantized models without spilling to system RAM.\nRTX 4070 12GB\n: around $600. Faster Tensor cores, quicker token generation.\nAMD cards (RX 6700 XT 12GB, refurbished from $380) offer equivalent VRAM but require ROCm configuration. Ollama's CUDA path is plug-and-play on NVIDIA. AMD works but adds setup overhead.\nOn 8GB (RTX 3050 class), ornith:9b runs but cold starts are painful and VRAM headroom is tight. The RTX 3060 12GB is the practical sweet spot for local\nzerikai_memory\nuse.\nRecommendation\nornith:9b\nis the new default local model recommendation, replacing\nmistral:7b\n.\nOn 8GB dedicated VRAM: ornith fits but runs tight. Cold start hits 25s when Ollama hasn't pinned the model. Warm synthesis at 9-17s is acceptable for a local-only workflow where you are not switching models or running concurrent GPU workloads. Set\nOLLAMA_MAX_CONCURRENCY=1\nin\n.env\n.\nOn 10-12GB dedicated VRAM (RTX 3060 12GB or better): the model stays pinned, cold starts drop significantly, and citation precision is consistently better than mistral.\nUnder 8GB dedicated VRAM, or if synthesis latency matters more than citation precision, use\nmistral:7b\n. Set\nOLLAMA_MODEL=mistral:7b\nin\n.env\n. It handles synthesis correctly when context is dense. When context is thin, it will fill gaps with confident but wrong answers.\nThe query test was clean and controlled. The model difference is real and attributable to ornith's training on agentic coding tasks, not hardware or docstring quality. Use the benchmark script in the repo to validate on your own machine before switching.\n📖\nOriginal Publication\n: This engineering post-mortem was originally published on the Zerikai Tech Blog. Read the clean, formatted web version at\nhttps://zerikai.com\n.", "metadata": { "title": "Post-Mortem: Building a Local MCP Server for Codebase Memory using Ollama and ChromaDB - DEV Community", "description": "Developers are pushing back against cloud API billing and the privacy risks of sending proprietary... Tagged with ai, rag, mcp, documentation.", "keywords": "ai, rag, mcp, documentation, software, coding, development, engineering, inclusive, community", "author": null, "og:type": "article", "og:url": "https://dev.to/kike/post-mortem-building-a-local-mcp-server-for-codebase-memory-using-ollama-and-chromadb-3ilg", "og:title": "Post-Mortem: Building a Local MCP Server for Codebase Memory using Ollama and ChromaDB", "og:description": "Developers are pushing back against cloud API billing and the privacy risks of sending proprietary...", "og:site_name": "DEV Community", "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ff6dvobfgol171w0a9e63.png", "twitter:site": "@thepracticaldev", "twitter:creator": "@", "twitter:title": "Post-Mortem: Building a Local MCP Server for Codebase Memory using Ollama and ChromaDB", "twitter:description": "Developers are pushing back against cloud API billing and the privacy risks of sending proprietary...", "twitter:card": "summary_large_image", "twitter:widgets:new-embed-design": "on", "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ff6dvobfgol171w0a9e63.png" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [ { "file": "attachments/5b9cd91cc3dc/code/body-000.txt", "chars": 1200, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-001.txt", "chars": 654, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-002.txt", "chars": 781, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-003.txt", "chars": 399, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-004.txt", "chars": 526, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-005.txt", "chars": 456, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-006.txt", "chars": 709, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-007.txt", "chars": 458, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-008.txt", "chars": 623, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-009.txt", "chars": 433, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-010.txt", "chars": 64, "language": "text" }, { "file": "attachments/5b9cd91cc3dc/code/body-011.txt", "chars": 302, "language": "text" } ], "images": [], "audio": [], "videos": [], "links": [ { "href": "https://github.com/KikeVen/zerikai_memory", "label": "zerikai_memory", "type": "a" }, { "href": "https://zerikai.com/blog/Post-Mortem_Building_Local_MCP_Server_Codebase_Memory_using_Ollama_and_ChromaDB.html", "label": "https://zerikai.com", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:49:58Z", "source_url": "https://dev.to/kike/post-mortem-building-a-local-mcp-server-for-codebase-memory-using-ollama-and-chromadb-3ilg", "papers": [], "actual_video_files": [], "body_chars": 17046, "expected_body_images": 0, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/5b9cd91cc3dc/pages/source.html", "local_html": "attachments/5b9cd91cc3dc/pages/article.html", "body_base_path": "attachments/5b9cd91cc3dc/pages/", "body_markdown_path": "attachments/5b9cd91cc3dc/pages/article.md" }