{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Tae Kim",
  "title_listed": "Building an investing knowledge graph, part 5: what LIVE actually means",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-5-what-live-actually-means-18g3",
  "publish_date": "2026-09-02",
  "corpus_stage": "",
  "notes": "",
  "id": "cbe9be5550c3",
  "collected_at": "2026-10-02T14:03:49",
  "title": "Building an investing knowledge graph, part 5: what LIVE actually means - DEV Community",
  "body_markdown": "Building an investing knowledge graph, part 5: what LIVE actually means\nPart 4 ended with the resolver working locally and me calling it live on Railway. That framing glossed over a gap. Something running locally and something running in production are different in ways that are obvious in retrospect and invisible until they bite you.\nHere's what changed once I had a caller that wasn't me.\nThe first thing that breaks when someone else calls your API\nThe investing knowledge graph pipeline was the only caller for a long time. I knew its call patterns. I'd written both sides. The error budget was implicit — if something failed, I fixed it and kept going.\nThe first external caller changed that. When the API started returning errors for them, there was no \"I'll fix it and retry.\" There was a broken integration on their end, and they had no context for diagnosing it.\nThe specific failure was unhelpful: a 500 with a generic error message on a request that had worked in my own testing. It took debugging from both ends to isolate the cause. The request was valid. The issue was that it hit the API during a window when the registry backend was mid-write, and the response came back internally inconsistent — some aliases resolved correctly, others returned stale data.\nThis wasn't a bug in the resolver logic. It was a backend durability problem I hadn't needed to care about when I was the only caller.\nWhat the backend switch actually fixed\nLocally, I used a file-based registry backend. The registry serializes to a JSON file, loaded into memory at startup and read during inference. Writes append and periodically flush to disk. Simple, fast, no dependencies.\nThe failure mode is obvious once you see it: during a write, if a read hits a partially flushed state, you get inconsistency. For a single-user pipeline running sequentially, this almost never surfaces. You'd have to be unlucky for a read to interleave with a write in a way that causes problems. Once there are concurrent callers, that probability stops being unlucky and becomes predictable.\nThe production backend is now PostgreSQL via Supabase. Writes go through transactions. Reads get a consistent snapshot. The alias table and the entity registry update atomically — either both change or neither does. A caller mid-request gets a consistent registry view regardless of what writes are in flight.\nThe switch is controlled by an environment variable (\nER_REGISTRY_BACKEND\n). The application logic doesn't change, just the persistence layer. This separation made the transition straightforward: I could test both backends against the same code without restructuring anything.\nThe healthcheck\nRailway handles deployments by routing traffic only after a service passes its healthcheck. The\n/health\nendpoint returns 200 when the API is ready to serve requests and the registry backend is reachable.\nThis matters in practice because startup isn't instantaneous. The service loads the model, initializes the Splink configuration, and verifies the backend connection before reporting healthy. Without an accurate healthcheck, Railway might route traffic to an instance that's running but not ready, which produces the kind of half-initialized state that generates confusing errors.\nDuring a deploy, Railway keeps the old version serving traffic until the new one passes its healthcheck. No request reaches the new version before it's ready. This zero-downtime behavior is Railway's default — I didn't configure it specially — but it only works correctly if the healthcheck accurately reflects whether the service is ready.\nThe healthcheck has failed in two ways across different deployments: backend unreachable, and model initialization error. Both are things I'd rather catch at startup than discover mid-request. The healthcheck turned each into a failed deploy rather than a silent production problem.\nRegistry integrity at 47,853 entities\nThe registry currently holds 47,853 resolved entities and 47,883 aliases. Most entries are stable. Some are probably wrong.\nEarly in the project, when I was still calibrating the Splink threshold, the model was more permissive than it should have been. Some merges happened that were likely incorrect — entity strings that shared enough surface similarity to pass the threshold but were different companies. A few of those early-period decisions are still in the registry.\nI know this in general. I don't know specifically which ones without re-running the full corpus against the current model. That re-run is on the list but hasn't happened yet.\nWhat I have now is a review layer before new merges commit to the stable registry. Pairs that the model scores as probable matches but below a high-confidence threshold go into a pending state rather than being committed immediately. They get reviewed before joining the stable set. This doesn't fix the early errors already in the registry, but it stops new errors from accumulating at the same rate.\nThe production API serves only the stable registry. Pending pairs aren't exposed to external callers. If the model is uncertain, the caller gets a\nsplit\ndecision. They can re-query later once additional evidence has accumulated and the pair has cleared review.\nWhat \"beta\" actually means here\nThe last few posts in this series have mentioned \"beta\" without being specific. Here's what it actually implies:\nThe API is running. External callers are using it. I'm expanding access carefully rather than opening it wide.\nIt's not a prototype — the infrastructure decisions described above are real, the registry has nearly 48k entities built from actual article processing, and the resolver has been running reliably for months. It's also not a mature product with SLAs, a dedicated support tier, or uptime commitments beyond what Railway's infrastructure provides.\nIf you have an entity resolution problem — corporate name disambiguation, customer record deduplication, knowledge graph construction from heterogeneous sources — and want to try the service, there's a form at\nhannune.ai\n. I'll respond directly.\nNext: the final part of this series wraps up the full arc and looks at where the same five-step pattern — DB limits, knowledge graph, ER bottleneck, direct implementation, cloud deployment — shows up outside investing graphs.\nBuilt on\nSplink\nfor probabilistic record linkage. Part 1 is\nhere\n. Part 2 is\nhere\n. Part 3 is\nhere\n. Part 4 is\nhere\n.",
  "metadata": {
    "title": "Building an investing knowledge graph, part 5: what LIVE actually means - DEV Community",
    "description": "Building an investing knowledge graph, part 5: what LIVE actually means   Part 4 ended with... Tagged with graphdb, knowledgegraph, dataengineering, api.",
    "keywords": "graphdb, knowledgegraph, dataengineering, api, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-5-what-live-actually-means-18g3",
    "og:title": "Building an investing knowledge graph, part 5: what LIVE actually means",
    "og:description": "Building an investing knowledge graph, part 5: what LIVE actually means   Part 4 ended with...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fimages.unsplash.com%2Fphoto-1726250873166-814c1f2cb74b%3Fixid%3DM3w5NjA5Njd8MHwxfHJhbmRvbXx8fHx8fHx8fDE3ODgzMTY0NTd8%26ixlib%3Drb-4.1.0%26w%3D1200%26h%3D630%26fit%3Dcrop",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@HannuneTech",
    "twitter:title": "Building an investing knowledge graph, part 5: what LIVE actually means",
    "twitter:description": "Building an investing knowledge graph, part 5: what LIVE actually means   Part 4 ended with...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fimages.unsplash.com%2Fphoto-1726250873166-814c1f2cb74b%3Fixid%3DM3w5NjA5Njd8MHwxfHJhbmRvbXx8fHx8fHx8fDE3ODgzMTY0NTd8%26ixlib%3Drb-4.1.0%26w%3D1200%26h%3D630%26fit%3Dcrop"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://hannune.ai/",
      "label": "hannune.ai",
      "type": "a"
    },
    {
      "href": "https://github.com/moj-analytical-services/splink",
      "label": "Splink",
      "type": "a"
    },
    {
      "href": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-1-from-does-this-news-matter-to-a-graph-i-can-query-11bi",
      "label": "here",
      "type": "a"
    },
    {
      "href": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-2-the-query-sql-cant-answer-l9f",
      "label": "here",
      "type": "a"
    },
    {
      "href": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-3-same-company-seven-names-b7d",
      "label": "here",
      "type": "a"
    },
    {
      "href": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-4-building-the-resolver-3dn7",
      "label": "here",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:49:37Z",
  "source_url": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-5-what-live-actually-means-18g3",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 6415,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/cbe9be5550c3/pages/source.html",
  "local_html": "attachments/cbe9be5550c3/pages/article.html",
  "body_base_path": "attachments/cbe9be5550c3/pages/",
  "body_markdown_path": "attachments/cbe9be5550c3/pages/article.md"
}