{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Jaival Suthar",
  "title_listed": "Knowledge Vault: Finding Where RAG Retrieval Actually Fails",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://jaivalsuthar.hashnode.dev/knowledge-vault-finding-where-rag-retrieval-actually-fails",
  "publish_date": "2026-08-17",
  "corpus_stage": "",
  "notes": "",
  "id": "ec151a95a200",
  "collected_at": "2026-10-02T14:12:15",
  "title": "Debugging RAG Retrieval: Reranking, Evidence & Ranking Noise",
  "body_markdown": "Knowledge Vault: Finding Where RAG Retrieval Actually Fails\nUpdated\nAugust 17, 2026\n•\n21\nmin read\n•\nView as Markdown\nJ\nJaival Suthar\nI write technical articles that analyze real world engineering problems through the lens of production systems.\nMy work focuses on architectural tradeoffs, UI layer system design, performance optimization, and reliability patterns derived from shipping and maintaining scalable applications.\nPart of series\nBuilding AI Systems from Scratch\nBuilding a RAG system is relatively easy to demonstrate.\nBuilding one where you can\nmeasure why it fails, reproduce the failure, and identify which stage caused it\nis a very different engineering problem.\nIn the first part of my Knowledge Vault journey, I focused on building and benchmarking the retrieval pipeline. I compared dense retrieval against BGE reranking and found something interesting: the reranker changed\n27 out of 35 Top-1 results — 77.1% of them\n.\nIf you haven't read Part 1, start with\nBuilding Knowledge Vault: From RAG Implementation to Retrieval Engineering\n. That article covers the initial retrieval architecture and benchmark that led to this investigation.\nThat number looked significant.\nBut there was a problem.\nA ranking change is not automatically an improvement.\nThe reranker could be correcting bad retrieval, or it could simply be replacing one passage with another passage that looks more relevant to the model but contains worse evidence.\nAt the end of that investigation, I had a hypothesis.\nDocument structure might be part of the problem.\nTable-of-contents pages, indexes, glossaries, copyright pages, and similar structural content can be highly similar to a query while being poor sources of answer-bearing evidence.\nBut a hypothesis is not an engineering conclusion.\nI needed to prove it.\nThat changed the direction of M1 completely.\nInstead of asking whether the RAG system produced good answers, I started asking a much more useful question:\nWhen the correct evidence exists, where does the system lose it?\nAnd answering that question required building the measurement infrastructure first.\nThe problem with evaluating only the final answer\nA RAG system has several independent failure boundaries.\nA question can fail because the document parser extracted the content incorrectly.\nIt can fail because chunking split the relevant passage badly.\nIt can fail because dense retrieval never found the correct passage.\nIt can fail because the correct passage was retrieved but ranked too low.\nOr the retrieval pipeline can work perfectly and the generation model can still produce a poor answer.\nIf I only evaluate the final response, all of those failures collapse into one number.\nThat makes debugging extremely difficult.\nSo I changed the evaluation boundary.\nInstead of treating the generated answer as the only source of truth, I created\npassage-level gold evidence\nfor the benchmark.\nKnowledge Vault's evaluation set contains 35 controlled questions: 30 answerable questions used for strict retrieval metrics and 5 deliberately unanswerable questions.\nFor the answerable questions, the evaluator knows which passage contains the expected evidence.\nThat allows the pipeline to answer questions that are much more useful than \"Was the final answer correct?\"\nFor example:\nDid dense retrieval find the correct evidence?\nAnd if it did:\nDid the reranker preserve that evidence?\nThose are completely different failure modes.\nThe benchmark therefore tracks candidate Recall@k, Recall@1, Recall@5, Recall@20, MRR, nDCG@5, per-question evidence ranks, dense and reranker scores, and stage-level latency.\nThe important change wasn't the number of metrics.\nIt was the visibility into the pipeline.\nI could finally observe where evidence entered the system and where it disappeared.\nEstablishing the baseline\nThe dense retrieval baseline gave me a useful starting point.\nAcross the benchmark, dense retrieval achieved:\nMetric\nDense Baseline\nQuestions\n35\nStrict-metric questions\n30\nRecall@1\n36.67%\nRecall@5\n66.67%\nMRR\n0.4944\nnDCG@5\n0.4274\nThese numbers weren't particularly interesting by themselves.\nWhat mattered was what happened when I changed the candidate pool.\nThe retrieval pipeline exposes a parameter called\ncandidate_k\n.\nThis controls how many passages dense retrieval sends to the reranker.\nThe final\ntop_k\ncontrols how many passages eventually become evidence for the model.\nThat distinction sounds small.\nIt turned out to be one of the most important details in the investigation.\nMore candidates did not mean better final retrieval\nI ran a controlled comparison across dense retrieval and reranking:\nConfiguration\nCandidate k\nReranking\nRecall@1\nRecall@5\nMRR\nnDCG@5\nDense\n5\nOff\n36.7%\n66.7%\n0.4944\n0.4274\nDense\n20\nOff\n36.7%\n66.7%\n0.4944\n0.4274\nRerank\n5\nOn\n40.0%\n66.7%\n0.5067\n0.4293\nRerank\n20\nOn\n20.0%\n66.7%\n0.3911\n0.3479\nSomething immediately stood out.\nWith\ncandidate_k=20\n, dense retrieval was able to find the gold evidence in the candidate pool at\n83.3% Recall@20\n.\nThat sounds promising.\nBut after reranking, the final ranking quality actually got worse.\nRecall@1 dropped from\n40.0% to 20.0%\nwhen the reranker received 20 candidates instead of 5.\nMRR and nDCG@5 dropped as well.\nThis changed the debugging question.\nI no longer needed to ask:\n\"Can retrieval find the passage?\"\nI needed to ask:\n\"When retrieval already found the passage, what is causing the ranking stage to remove it?\"\nThat is a much narrower problem.\nAnd narrower problems are much easier to investigate.\nI stopped treating the reranker as a black box\nThe next step wasn't to change the reranker.\nThat would have made the experiment harder to interpret.\nInstead, I instrumented the transition between candidate retrieval and final ranking.\nFor every evaluated question, I started preserving information such as:\ncandidate rank\nfinal rank\ndense score\nreranker score\nstructural category\ngold-evidence presence\nwhether the gold passage survived into the final top-k\nThis gave me something I didn't have before.\nA forensic view of the ranking pipeline.\nI could take a question where the final answer was wrong and trace the evidence backward:\nQuestion\n   ↓\nDense candidates\n   ↓\nWas gold evidence present?\n   ↓\nWhat rank did it have?\n   ↓\nWhat did the reranker score?\n   ↓\nWhat replaced it?\n   ↓\nWhy did that replacement look attractive?\nThat changed the debugging process from speculation to evidence.\nAnd a pattern started appearing.\nSemantic relevance was not the same as evidentiary usefulness\nSome of the passages being promoted by the reranker were genuinely related to the query.\nThey just weren't the passages that actually answered it.\nThis distinction is easy to miss.\nImagine a question asking:\nWhat does the author mean by the fourth degree of action?\nA table-of-contents passage might contain:\nThe Fourth Degree: Massive Action\nA chapter passage might contain several paragraphs explaining what that concept actually means.\nBoth passages are semantically related to the query.\nOnly one is useful as evidence.\nThe first provides\nnavigation information\n.\nThe second provides\nanswer-bearing evidence\n.\nA semantic retrieval system doesn't automatically know that difference.\nAnd this is where document structure became interesting.\nThe book contained structural chunks such as:\nCONTENTS\nINDEX\nCOPYRIGHT\nGLOSSARY\nThese chunks aren't bad data.\nIn fact, they can be extremely useful.\nIf someone asks:\nWhere does the book discuss massive action?\nan index entry may be exactly what we want.\nBut if someone asks:\nWhat does massive action mean according to the author?\nthe same index entry is insufficient.\nThe problem wasn't simply \"structural chunks are bad.\"\nThe problem was more subtle:\nDifferent document structures provide different kinds of relevance.\nA table of contents is relevant for navigation.\nAn index is relevant for locating concepts.\nA glossary is relevant for terminology.\nA substantive passage is relevant for evidence.\nTreating all of them as interchangeable evidence creates a ranking problem.\nThen the benchmark itself needed debugging\nWhile investigating individual retrieval failures, I discovered something else that was important enough to stop the experiment and audit the benchmark.\nAn earlier evaluation artifact contained a stale gold-evidence identifier.\nThe underlying question/evidence mapping had subsequently been corrected in the evaluation data, but not every previously generated result artifact had been regenerated from the updated annotations.\nThat distinction matters.\nA benchmark result isn't just a number.\nIt is the output of:\nQuestion set\n    +\nEvidence specification\n    +\nRetriever configuration\n    +\nEvaluation code\n    ↓\nResult artifact\nChange one of those inputs and the result has a different provenance.\nThis led to a broader engineering lesson:\nEvaluation artifacts need provenance.\nIf the benchmark changes, I need to know exactly which question set and evidence specification produced a particular result.\nOtherwise, two JSON files can appear comparable while actually representing different versions of the experiment.\nI also added an important validation requirement: evidence identifiers should not merely be syntactically valid; they should resolve to actual indexed passages.\nThe evaluator itself needs guardrails.\nOtherwise, the system can end up confidently measuring an invalid experiment.\nA failure I could finally explain\nOne of the clearest examples was Q022:\nWhat is the relationship between 10X goals and 10X actions according to the author?\nThe important part wasn't that the answer failed.\nIt was\nwhere\nit failed.\nThe correct evidence was already inside the dense candidate pool.\nDense retrieval had done its job.\nBut after reranking, that evidence was displaced by a glossary chunk.\nThat meant the failure wasn't:\nDense retrieval\n      ↓\n      ✗ Could not find evidence\nIt was:\nDense retrieval\n      ↓\n      ✓ Found evidence\n      ↓\nReranking\n      ↓\n      ✗ Discarded evidence\nThat distinction changes what you should fix.\nIncreasing the candidate pool won't solve a failure that happens after candidate retrieval.\nChanging the embedding model isn't necessarily the right answer either.\nThe evidence was already there.\nThe failure boundary had moved downstream.\nThe numbers showed that these weren't always small mistakes\nAcross the forensic analysis, I found:\n40 cases where gold evidence was present in candidate-k=20\n24 cases where it survived into final top-5\n16 cases where gold evidence was lost after reranking\nFor 12 of those losses, I could compare the reranker score of the gold passage against the replacement passage.\nThe results were surprisingly large:\nMeasurement\nValue\nMean gold reranker score\n0.1827\nMean replacement score\n0.7915\nMean score margin\n0.6088\nMedian score margin\n0.6713\nThis was important.\nThe reranker wasn't always making a tiny mistake where two passages had almost identical scores.\nIn these cases, the replacement often received a substantially stronger score.\nThat made the problem more interesting.\nThe question became:\nWhy does a semantically plausible but less evidentiary passage look so much better to the reranker?\nAnd that is a much harder question than \"the reranker ranked the wrong thing.\"\nThe hypothesis became narrow enough to test\nAt this point I had enough evidence to formulate a controlled hypothesis:\nStructural candidates can interfere with reranking when they are treated as ordinary substantive evidence.\nNotice what I wasn't claiming.\nI wasn't claiming that structural chunks caused every reranking failure.\nI wasn't claiming that the BGE reranker was fundamentally bad.\nI wasn't claiming that filtering would solve RAG retrieval.\nI was testing one specific contributor to the observed failure.\nThat distinction matters in engineering experiments.\nIf the hypothesis is too broad, almost any result can appear to support it.\nA narrow hypothesis gives you something falsifiable.\nThe intervention was deliberately small\nI didn't remove structural chunks from the vector database.\nThat would have changed the corpus and potentially affected other query types.\nInstead, I added an eligibility filter\nbetween dense retrieval and reranking\n.\nThe pipeline became:\nDense Retrieval\n       ↓\nStructural Eligibility Filter\n       ↓\nBGE Reranker\n       ↓\nFinal Evidence\nThe filter excluded navigation-oriented structural categories such as:\nCONTENTS\nINDEX\nCOPYRIGHT\nGLOSSARY\nfrom the evidence-ranking path.\nEverything else stayed the same.\nSame corpus.\nSame embeddings.\nSame dense retrieval.\nSame candidate depth.\nSame reranker.\nSame final top-k.\nOnly the eligibility decision changed.\nThat made the experiment much easier to reason about.\nThe result was positive — but deliberately not dramatic\nThe controlled before/after experiment produced:\nMetric\nBaseline\nStructural Filter\nCandidate Recall@20\n83.3%\n83.3%\nRecall@1\n20.0%\n20.0%\nRecall@5\n66.7%\n70.0%\nMRR\n0.3911\n0.4022\nnDCG@5\n0.3561\n0.3722\nThe most important thing about these results is what\ndidn't\nchange.\nCandidate Recall@20 remained exactly the same.\nThat is expected.\nThe filter didn't improve dense retrieval.\nIt acted downstream of dense retrieval.\nRecall@1 also remained at 20%.\nSo the intervention didn't suddenly make the reranker capable of consistently selecting the best passage.\nWhat changed was the quality of the final top-5.\nRecall@5 improved:\n66.7% → 70.0%\nMRR improved:\n0.3911 → 0.4022\nnDCG@5 improved:\n0.3561 → 0.3722\nThat is exactly the kind of result I wanted from the experiment.\nNot a dramatic \"we solved RAG\" number.\nA measurable improvement at the stage where the hypothesis predicted an improvement.\nLooking at the individual failures made the result easier to understand\nQ022 was one example.\nBefore filtering, the correct evidence was inside the candidate pool but was displaced by a glossary chunk.\nAfter filtering, the substantive evidence survived into the final context.\nAnother useful example was Q004:\nWhat does the fourth degree of action mean?\nThe gold passage was already present in the candidate pool.\nBefore filtering, it appeared at rank 5.\nAfter filtering, it moved to rank 4.\nThe corresponding metrics improved:\nMRR\n0.20 → 0.25\n\nnDCG@5\n0.3869 → 0.4307\nAgain, this wasn't a retrieval breakthrough.\nNo new evidence was discovered.\nThe system simply ranked existing evidence slightly better.\nAnd that is exactly why passage-level metrics are useful.\nThey tell you what kind of improvement actually occurred.\nWhat the experiment really taught me about candidate pools\nOne of my early assumptions was straightforward:\nMore candidates should give the reranker more opportunities to find the right answer.\nThat's true.\nBut it is only half the equation.\nMore candidates also give the reranker more opportunities to encounter distractors.\nA larger candidate pool provides:\nMore recall opportunity\n        +\nMore candidate diversity\n        +\nMore potential evidence\n        -\nMore candidate noise\nSo increasing\ncandidate_k\nis not automatically an optimization.\nIt changes the ranking problem.\nIn this benchmark, candidate Recall@20 reached\n83.3%\n, but final Recall@5 was still only\n66.7%\nbefore structural filtering.\nThe system had the evidence.\nThe challenge was selecting it.\nThat is a very different engineering problem from simply \"improve retrieval.\"\nThe biggest improvement wasn't the 3.3 percentage points\nThe structural filter improved Recall@5 by 3.3 percentage points.\nThat's useful.\nBut the more important improvement was observability.\nBefore instrumenting the pipeline, a failure looked like:\n\"The answer is wrong.\"\nAfter instrumentation, I could say:\n\"The gold passage entered the top-20 dense candidate set but was displaced before the final top-5.\"\nThat is an entirely different debugging state.\nThe debugging loop became:\nWas gold evidence retrieved?\n        │\n        ├── No\n        │     ↓\n        │   Investigate dense retrieval\n        │\n        └── Yes\n              ↓\n        What displaced it?\n              ↓\n        What type of passage was it?\n              ↓\n        What were the scores?\n              ↓\n        What intervention could address it?\n              ↓\n        Re-run the benchmark\nThis is what good evaluation infrastructure gives you.\nNot just metrics.\nA mechanism for turning an aggregate failure into a testable hypothesis.\nThe remaining gap is more interesting than the improvement\nAt the beginning of this investigation, I was looking for a fix.\nBy the end, I was more interested in the remaining failure.\nThat is usually a good sign in systems engineering.\nAfter structural filtering, the benchmark still had:\nCandidate Recall@20    83.3%\nFinal Recall@5         70.0%\nRecall@1               20.0%\nThe system can frequently retrieve the right evidence.\nBut it still struggles to put the best evidence at the top.\nAnd structural filtering only explains part of that gap.\nThe remaining failures are heterogeneous.\nSome are likely related to semantic similarity.\nSome may be caused by the limitations of the reranking signal.\nSome may be related to chunking.\nSome may require complementary lexical retrieval.\nThat means the next investigation shouldn't simply be another isolated tweak to the reranker.\nI need another retrieval signal.\nWhat I am deliberately not claiming\nThis experiment does\nnot\nestablish that BGE is a poor reranker.\nIt does not establish that structural chunks cause all reranking failures.\nIt does not establish that the improvement will generalize to every document.\nIt does not establish that Recall@5 is now sufficient.\nAnd it certainly doesn't mean the RAG system is production-grade.\nThe current evaluation is based on one 214-page text-layer PDF.\nThere are still major areas that require separate investigation: OCR and scanned documents, complex tables, multi-column layouts, reading-order corruption, larger and more diverse corpora, and broader evaluation sets.\nThe benchmark itself also contains related questions, so the raw count of questions should not automatically be interpreted as the same number of independent failure trials.\nThese limitations aren't footnotes to hide.\nThey define the boundary of what the experiment actually proves.\nWhat M1 changed\nThe most valuable outcome of M1 wasn't one metric.\nIt was the decomposition of the retrieval problem.\nInstead of treating RAG as:\nQuestion\n   ↓\nMagic retrieval\n   ↓\nAnswer\nI now have observable boundaries:\nDocument extraction\n       ↓\nChunking\n       ↓\nCandidate retrieval\n       ↓\nCandidate eligibility\n       ↓\nReranking\n       ↓\nFinal evidence selection\n       ↓\nGeneration\nAnd each boundary can be evaluated independently.\nThat changes how I think about RAG engineering.\nA low answer accuracy number is not a diagnosis.\nIt is a symptom.\nThe engineering work begins when you can determine\nwhich stage produced the symptom\n.\nThe conclusion I was looking for was not \"reranking works\"\nM1 started with a relatively simple question:\nCan Knowledge Vault retrieve useful evidence from a PDF?\nIt ended with a much better one:\nWhen the correct evidence exists, what causes the system to lose it before generation?\nDense retrieval at candidate-k=20 achieved\n83.3% candidate Recall@20\n.\nBefore the structural intervention, final Recall@5 was\n66.7%\n.\nForensic analysis showed that some of the missing evidence was displaced by structurally related but less useful passages.\nA controlled structural eligibility filter improved Recall@5 from\n66.7% to 70.0%\n, MRR from\n0.3911 to 0.4022\n, and nDCG@5 from\n0.3561 to 0.3722\n.\nBut Recall@1 remained at\n20%\n.\nThat last number is important.\nIt tells me the problem isn't solved.\nThe structural filter improved one part of the ranking behavior, but it didn't close the fundamental gap between:\nEvidence retrieved\n        ↓\nEvidence selected\nAnd that's a much more useful conclusion than saying:\n\"Our RAG pipeline improved by 3.3%.\"\nThe real result is that the\nfailure boundary is now measurable\n.\nCode and experiments\nThe implementation, evaluation harness, benchmark artifacts, and retrieval experiments are available in the\nKnowledge Vault GitHub repository\nWhere the investigation goes next\nThe next question is now much clearer:\nIf dense retrieval can find the correct evidence 83.3% of the time at k=20, why does final evidence ranking remain substantially worse?\nThe next experiment will investigate whether complementary retrieval signals can close some of that gap.\nSpecifically:\nQuery\n                  │\n          ┌───────┴───────┐\n          ↓               ↓\n       Dense             BM25\n          │               │\n          └───────┬───────┘\n                  ↓\n                 RRF\n                  ↓\n              Reranking\n                  ↓\n               Evidence\nThe goal isn't to assume that hybrid retrieval will solve the problem.\nThe goal is to measure whether lexical and semantic retrieval provide\ncomplementary evidence coverage\n, and whether that improves candidate recall and final ranking independently.\nThat's where M2 begins.\nFinal takeaway\nThe interesting part of building a RAG system isn't getting the first demo to work.\nIt's getting to the point where a bad answer stops being mysterious.\nA production-minded retrieval system should let you ask:\nDid we fail to find the evidence?\nDid we find it but rank it incorrectly?\nDid document structure confuse the ranking model?\nDid the benchmark itself accurately describe the expected evidence?\nDid the intervention actually improve the stage we intended to change?\nThat's the difference between\nbuilding a RAG pipeline\nand\nengineering a retrieval system\n.\nAnd for me, that was the real outcome of Knowledge Vault M1.\nI didn't solve retrieval.\nI made its failures\nobservable enough to keep solving them\n.\nA small glossary for the metrics and terminology\nIf you're reading this for the first time, some of the terminology can feel unnecessarily dense. Here are the terms that matter most in this investigation.\nKeyword\nMeaning\nRAG\nRetrieval-Augmented Generation. A system retrieves relevant information first and then gives that information to an LLM to generate an answer.\nDense Retrieval\nRetrieval based on semantic embeddings. The query and document chunks are represented as vectors, and similar vectors are retrieved.\nBM25\nA traditional lexical search algorithm that scores documents based largely on how well their words match the query.\nEmbedding\nA numerical vector representation of text that captures semantic relationships between pieces of text.\nVector Database\nA database optimized for storing and searching vector embeddings. Knowledge Vault uses Qdrant.\nCandidate Pool\nThe set of passages retrieved before the final ranking stage. With\ncandidate_k=20\n, the reranker receives up to 20 candidates.\ncandidate_k\nThe number of candidates retrieved for downstream processing. Increasing it can improve recall but can also introduce more distractors.\ntop_k\nThe number of passages ultimately selected as final context for the LLM.\nReranker\nA model that takes an initial set of retrieved passages and reorders them based on query-passage relevance.\nCross-Encoder\nA model that processes the query and passage together to produce a relevance score. BGE reranking uses this approach.\nGold Evidence\nThe passage identified by the benchmark as containing the evidence required to answer a question.\nCandidate Recall@20\nThe percentage of questions where the gold evidence appears somewhere within the first 20 retrieved candidates.\nRecall@1\nThe percentage of questions where the correct evidence is ranked first.\nRecall@5\nThe percentage of questions where the correct evidence appears within the top 5 results.\nMRR\nMean Reciprocal Rank.\nMeasures how high the first relevant result appears. Rank 1 contributes 1.0, rank 2 contributes 0.5, rank 3 contributes 0.33, and so on. Higher is better.\nnDCG@5\nNormalized Discounted Cumulative Gain.\nMeasures ranking quality while giving more importance to relevant results appearing near the top. Higher is better.\nRanking Failure\nA case where the correct evidence was retrieved but ranked too low or displaced by another passage.\nRetrieval Failure\nA case where the correct evidence never entered the candidate pool in the first place.\nStructural Chunk\nA chunk representing document organization or navigation rather than substantive content, such as a TOC, index, glossary, or copyright page.\nStructural Filtering\nPreventing certain structural chunks from participating in the evidence-ranking stage while keeping them available in the underlying corpus.\nSemantic Relevance\nHow conceptually related a passage is to a query. A passage can be semantically relevant without containing the answer.\nEvidentiary Relevance\nHow useful a passage actually is as evidence for answering the question. This is the distinction that became central to the investigation.\nRRF\nReciprocal Rank Fusion.\nA technique for combining rankings from different retrieval systems, such as dense retrieval and BM25.\nHybrid Retrieval\nCombining multiple retrieval strategies, typically semantic and lexical retrieval, to improve coverage.\nLatency\nThe time taken by a stage or the entire retrieval pipeline to produce a result.\nProvenance\nInformation describing exactly which data, benchmark version, configuration, and evaluation specification produced a result.\nFailure Boundary\nThe specific stage where a previously correct piece of information becomes lost, corrupted, or incorrectly ranked.\n#\nrag\n#\nretrieval-augmented-generation\n#\nllm\n#\nai-engineering",
  "metadata": {
    "title": "Debugging RAG Retrieval: Reranking, Evidence & Ranking Noise",
    "description": "An engineering investigation into RAG retrieval, reranking, structural noise, and how evidence reaches the final context.",
    "keywords": null,
    "author": null,
    "og:title": "Debugging RAG Retrieval: Reranking, Evidence & Ranking Noise",
    "og:description": "An engineering investigation into RAG retrieval, reranking, structural noise, and how evidence reaches the final context.",
    "og:url": "https://jaivalsuthar.hashnode.dev/knowledge-vault-finding-where-rag-retrieval-actually-fails",
    "og:site_name": "Jaival Suthar | Engineering Portfolio",
    "og:image": "https://cdn.hashnode.com/uploads/og-images/6972fc71207454ddbf7903dd/f59152b2-f6c8-43d0-93b8-e219e31a6197.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:creator": "https://x.com/Jaival_Suthar",
    "twitter:title": "Debugging RAG Retrieval: Reranking, Evidence & Ranking Noise",
    "twitter:description": "An engineering investigation into RAG retrieval, reranking, structural noise, and how evidence reaches the final context.",
    "twitter:image": "https://cdn.hashnode.com/uploads/og-images/6972fc71207454ddbf7903dd/f59152b2-f6c8-43d0-93b8-e219e31a6197.png",
    "article:published_time": "2026-08-17T18:56:47.426Z",
    "article:modified_time": "2026-08-17T18:56:47.426Z",
    "article:author": "https://jaivalsuthar.hashnode.dev/@JaivalSuthar"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/ec151a95a200/code/body-000.txt",
      "chars": 195,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-001.txt",
      "chars": 34,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-002.txt",
      "chars": 116,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-003.txt",
      "chars": 56,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-004.txt",
      "chars": 100,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-005.txt",
      "chars": 101,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-006.txt",
      "chars": 34,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-007.txt",
      "chars": 14,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-008.txt",
      "chars": 16,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-009.txt",
      "chars": 16,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-010.txt",
      "chars": 40,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-011.txt",
      "chars": 124,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-012.txt",
      "chars": 383,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-013.txt",
      "chars": 87,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-014.txt",
      "chars": 42,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-015.txt",
      "chars": 171,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-016.txt",
      "chars": 47,
      "language": "text"
    },
    {
      "file": "attachments/ec151a95a200/code/body-017.txt",
      "chars": 313,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/uploads/covers/6972fc71207454ddbf7903dd/d2099980-fa36-41aa-9707-b8850148937f.png",
      "bytes": 1582600,
      "reused": true,
      "local_path": "attachments/ec151a95a200/images/000.png",
      "src": "https://cdn.hashnode.com/uploads/covers/6972fc71207454ddbf7903dd/d2099980-fa36-41aa-9707-b8850148937f.png",
      "alt": "Knowledge Vault: Finding Where RAG Retrieval Actually Fails",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://jaivalsuthar.hashnode.dev/knowledge-vault-finding-where-rag-retrieval-actually-fails.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@JaivalSuthar",
      "label": "J",
      "type": "a"
    },
    {
      "href": "https://x.com/Jaival_Suthar",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/in/jaival-suthar-57195b257/",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://jaivalsuthar.hashnode.dev/series/building-ai-systems-from-scratch",
      "label": "Part of series Building AI Systems from Scratch",
      "type": "a"
    },
    {
      "href": "https://jaivalsuthar.hashnode.dev/building-knowledge-vault-rag-retrieval-engineering",
      "label": "Building Knowledge Vault: From RAG Implementation to Retrieval Engineering",
      "type": "a"
    },
    {
      "href": "https://github.com/Jaival-Suthar/pdf-rag-system",
      "label": "Knowledge Vault GitHub repository",
      "type": "a"
    },
    {
      "href": "https://jaivalsuthar.hashnode.dev/tag/rag",
      "label": "# rag",
      "type": "a"
    },
    {
      "href": "https://jaivalsuthar.hashnode.dev/tag/retrieval-augmented-generation",
      "label": "# retrieval-augmented-generation",
      "type": "a"
    },
    {
      "href": "https://jaivalsuthar.hashnode.dev/tag/llm",
      "label": "# llm",
      "type": "a"
    },
    {
      "href": "https://jaivalsuthar.hashnode.dev/tag/ai-engineering",
      "label": "# ai-engineering",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:10:30",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:51:03Z",
  "source_url": "https://jaivalsuthar.hashnode.dev/knowledge-vault-finding-where-rag-retrieval-actually-fails",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 25397,
  "expected_body_images": 1,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/ec151a95a200/pages/source.html",
  "local_html": "attachments/ec151a95a200/pages/article.html",
  "body_base_path": "attachments/ec151a95a200/pages/",
  "body_markdown_path": "attachments/ec151a95a200/pages/article.md"
}