{ "source_file": "0930", "source_platform": "Dev.to", "author": "Tae Kim", "title_listed": "Building an investing knowledge graph, part 4: building the resolver", "keywords": "", "domain": "文本处理", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-4-building-the-resolver-3dn7", "publish_date": "2026-09-01", "corpus_stage": "", "notes": "", "id": "03b50b7b1a7f", "collected_at": "2026-10-02T14:03:49", "title": "Building an investing knowledge graph, part 4: building the resolver - DEV Community", "body_markdown": "Building an investing knowledge graph, part 4: building the resolver\nPart 3 ended with a question I left open: how does the threshold actually get set, and what happens to the registry when new articles keep coming in? That's what this one covers.\nThe cost of getting it wrong in both directions\nBefore I touch thresholds, it's worth being specific about what failing in each direction costs.\nA false merge combines two distinct entities into one node. In the Samsung SDI case from part 3, that would mean battery-supply disruptions show up when you traverse from the semiconductor business. The graph gives you an answer. The answer is wrong. You don't necessarily know it's wrong without already knowing the answer, which defeats the point.\nA false split keeps the same entity as two separate nodes. \"Samsung Electronics\" and \"the largest memory chipmaker in the world\" stay disconnected. Edges pile up on each independently. A traversal from one doesn't reach the other. The graph gives you a partial answer. You lose coverage.\nThese aren't symmetric. A false merge injects noise that's hard to detect. A false split just means a disconnected node — you miss some connections, but you don't get fabricated ones.\nSo I set the threshold conservatively. Pairs below a high confidence mark stay split. Some real aliases end up never linking. That's the tradeoff I chose: lower recall, higher precision. For a graph I'm using to make judgments about which news matters, a confidently wrong edge is worse than a missing one.\nWhat the endpoint does\nThe resolution service exposes a\n/v1/splink-pairs\nendpoint. It takes a list of candidate entity mention strings and returns a score for each pair, along with whether that pair crosses the registry threshold.\nA minimal request looks like this:\nPOST\n/v\n1\n/splink-pairs\n{\n\"candidates\"\n:\n[\n\"Samsung Electronics Co.\"\n,\n\"the largest memory chipmaker in the world\"\n,\n\"Samsung SDI\"\n,\n\"TSMC\"\n]\n}\nEnter fullscreen mode\nExit fullscreen mode\nThe response returns a score matrix for pairs that the model considers worth evaluating — not every combination, just the ones above a blocking threshold that filters out obviously unrelated pairs before the full model runs.\n{\n\"pairs\"\n:\n[\n{\n\"left\"\n:\n\"Samsung Electronics Co.\"\n,\n\"right\"\n:\n\"the largest memory chipmaker in the world\"\n,\n\"match_probability\"\n:\n0.96\n,\n\"decision\"\n:\n\"match\"\n,\n\"canonical_id\"\n:\n\"company:samsung_electronics\"\n},\n{\n\"left\"\n:\n\"Samsung Electronics Co.\"\n,\n\"right\"\n:\n\"Samsung SDI\"\n,\n\"match_probability\"\n:\n0.12\n,\n\"decision\"\n:\n\"split\"\n,\n\"canonical_id\"\n:\nnull\n}\n]\n}\nEnter fullscreen mode\nExit fullscreen mode\nThe caller gets a decision (\nmatch\nor\nsplit\n) and the canonical entity ID when there's a match. If the match is against a known registry entry, the alias gets recorded. If it's a new pair the model hadn't seen before, that pair gets added to the evidence set for future model updates.\nHow the registry grows\nWhen the investing knowledge graph pipeline processes a new article, it extracts entity mentions and runs them through the resolver. Three things can happen:\nKnown alias.\nThe string matches an existing entry in the alias table. Lookup is fast, no model call needed. The mention gets written to the graph under the existing canonical ID.\nUnknown mention, matches existing entity.\nThe string isn't in the alias table, but the model scores it as a probable match against an existing entity. The alias gets added to the registry. Future articles that use the same string skip the model call.\nGenuinely new entity.\nThe model doesn't find a confident match against anything in the registry. A new canonical entry gets created. It starts small — one mention, no resolved aliases — and accumulates evidence as future articles mention the same company.\nThe registry currently has 47,853 resolved entities and 47,883 aliases. A lot of those started as single-mention nodes. Some have since merged as more articles confirmed the connection.\nA few early decisions are probably still wrong. In the first batches I was more aggressive with merging, before I tightened the threshold. There are likely some nodes that should be split. I know this in general; I don't know specifically which ones without re-running the full corpus against the current model, which I haven't done.\nWhat Splink is doing under the hood\nSplink\nis the open source probabilistic record linkage library the model is built on. It uses DuckDB as the computation backend for the pair scoring, which handles the candidate generation and blocking step — filtering the space of possible pairs before running the full model.\nFor corporate entity mentions, the features that ended up mattering:\nString similarity\non the canonical name: catches abbreviations and common shorthand\nAlias coverage\n: once a surface form is in the registry, future instances match by lookup rather than model score — this is how \"the largest memory chipmaker in the world\" eventually becomes reliable without its string similarity to \"Samsung Electronics\" ever improving\nToken overlap\non co-occurring named entities within the same article: an article about DRAM yields tends to mention different companies than an article about battery chemistry\nBase rate weighting\n: a rare entity (one mention, no confirmed aliases) shouldn't get absorbed into a high-frequency entity just because they share a token\nSplink learns the feature weights from labeled pairs. The training set was several hundred manually verified matches and non-matches from the first two thousand articles. Not large. But I had enough confirmed Samsung/Samsung SDI pairs to get the context features weighted correctly.\nThe point where this became a service\nSomewhere around the third project where I had the same entity resolution problem — merging customer records from two systems, with the same fragmentation-versus-false-merge tradeoff — I stopped treating this as a per-project function and wrapped it into a standalone FastAPI service.\nThe knowledge graph pipeline now calls it over HTTP. Other things call it the same way. The registry is shared across callers. A match that one pipeline discovers helps every other caller that sees the same entity later.\nThat service is what I'm calling\nER API\n. It runs on Railway, it's live, and the registry it's backing is the same one that's kept the investing knowledge graph from turning into a tangle of phantom nodes.\nNext: what it takes to keep this running in production — the parts that only showed up after the first real caller wasn't me.\nBuilt on\nSplink\nfor probabilistic record linkage. Part 1 is\nhere\n. Part 2 is\nhere\n. Part 3 is\nhere\n.", "metadata": { "title": "Building an investing knowledge graph, part 4: building the resolver - DEV Community", "description": "Building an investing knowledge graph, part 4: building the resolver Part 3 ended with a... Tagged with graphdb, knowledgegraph, entityresolution, python.", "keywords": "graphdb, knowledgegraph, entityresolution, python, software, coding, development, engineering, inclusive, community", "author": null, "og:type": "article", "og:url": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-4-building-the-resolver-3dn7", "og:title": "Building an investing knowledge graph, part 4: building the resolver", "og:description": "Building an investing knowledge graph, part 4: building the resolver Part 3 ended with a...", "og:site_name": "DEV Community", "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fimages.unsplash.com%2Fphoto-1607799279861-4dd421887fb3%3Fw%3D1200%26h%3D630%26fit%3Dcrop", "twitter:site": "@thepracticaldev", "twitter:creator": "@HannuneTech", "twitter:title": "Building an investing knowledge graph, part 4: building the resolver", "twitter:description": "Building an investing knowledge graph, part 4: building the resolver Part 3 ended with a...", "twitter:card": "summary_large_image", "twitter:widgets:new-embed-design": "on", "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fimages.unsplash.com%2Fphoto-1607799279861-4dd421887fb3%3Fw%3D1200%26h%3D630%26fit%3Dcrop" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [ { "file": "attachments/03b50b7b1a7f/code/body-000.txt", "chars": 158, "language": "text" }, { "file": "attachments/03b50b7b1a7f/code/body-001.txt", "chars": 417, "language": "text" } ], "images": [], "audio": [], "videos": [], "links": [ { "href": "https://github.com/moj-analytical-services/splink", "label": "Splink", "type": "a" }, { "href": "https://hannune.ai/", "label": "ER API", "type": "a" }, { "href": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-1-from-does-this-news-matter-to-a-graph-i-can-query-11bi", "label": "here", "type": "a" }, { "href": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-2-the-query-sql-cant-answer-l9f", "label": "here", "type": "a" }, { "href": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-3-same-company-seven-names-b7d", "label": "here", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:49:38Z", "source_url": "https://dev.to/hannune/building-an-investing-knowledge-graph-part-4-building-the-resolver-3dn7", "papers": [], "actual_video_files": [], "body_chars": 6599, "expected_body_images": 0, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/03b50b7b1a7f/pages/source.html", "local_html": "attachments/03b50b7b1a7f/pages/article.html", "body_base_path": "attachments/03b50b7b1a7f/pages/", "body_markdown_path": "attachments/03b50b7b1a7f/pages/article.md" }