保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Large Language Model

Three Kinds of RAG Corpus, and What It Costs to Build for the Wrong One

Enterprise Document Intelligence [Vol.1 #14A] - Three questions tell you which shape a document collection has, and each shape wants a different architecture Angela and Kezhan Shi August 20, 202617 min read Photo by Peter Holmboe, via Pexels. Ask an AI engineer how to take a RAG system past one document and the answer has been the same for three years. Put every page of every file into a vector store, embed the question, and let similarity search pick the passages. On a demo folder that works. On an enterprise shelf it stops working, and not because a parameter is set wrong. Similarity search returns passages, so it never picks a document, and the answer comes back assembled out of three files that were never meant to be read together. The step that gets skipped is the one before that. A collection of documents is not one kind of thing. A shared drive of unrelated reports, five thousand copies of the same contract, and a stack of claim folders are three different problems, and an architecture that suits any one of them suits the other two badly. This article is the map of that decision: Learn this step by step with the interactive AI Engineer roadmap.

This article opens Part IV of Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks. Part IV is where the input stops being a file and becomes a folder, and the first thing to settle is which kind of folder is in front of you. 🧭 New to the series? Start with the map : Prompt, Context, Loop sets out the three engineering layers every RAG system is built on, the prompt (the call itself), the context (what fills the model’s window), the loop (when the next call fires and when it stops), and walks the whole series through that lens, article by article. It is the shortest way to see what is covered and where this one sits. where this article sits: the opening of Part IV - Image by author 📓 Run the five-PDF baseline in the companion notebook : one question, five model calls, and the per-document table that shows one real answer and four blanks. Then point the same loop at a folder of your own and watch the same ratio appear. Repo → doc-intel/notebooks-vol1. The public companion-code repo at doc-intel/notebooks-vol1 - Image by author The worked example in section 4 is five public NIST publications: the Cybersecurity Framework v1.1 (NIST CSWP 04162018), FIPS 199, SP 800-207, AI 100-1 and CSWP 29. All five are US Government work, public domain in the US (see the NIST copyright statement).

1. A quarter of a million documents and one question

Everything in the series so far could assume the system knew which document to look at, because there was only one. This section is what happens when that assumption goes.

1.1. What the expert does, and what the pipeline does

A mid-size insurance broker keeps about a quarter of a million documents. Fifteen years of policies, certificates, contracts, claims and correspondence, some scanned and some native, most of them never opened twice. A user asks: “what are the seller’s obligations in our distribution agreements with retailer X?” A senior claims handler answers in under a minute, and she does it without reading anything she does not need. Obligations sit in distribution agreements, so invoices and certificates are out. For retailer X there is one master from 2019 and two amendments. Obligations are usually section 4 of the master, and an amendment wins where it says so. She pulls three folders, reads three sections, comes back. Point the textbook pipeline at the same question and it does something else entirely. Every passage of every document is already embedded in one store. The question gets embedded, the store returns the thirty nearest passages by cosine similarity, and those thirty go into one prompt. Sometimes the answer is right. Often the thirty hold two paragraphs from a contract with a different retailer, a certificate that belongs to no contract at all, and a 2017 invoice whose only qualification is the word obligation. Nothing in the input tells the model that those three should never be read together, so it writes a paragraph that reads well and describes no document that exists. The gap between the two is not accuracy. They solve different problems. The handler decides which documents count and then reads. The pipeline never decides anything about documents at all. one question, two ways to answer it, and only one of them picks documents - Image by author

1.2. Five ways a flat pile fails at scale

At small scale the pile works often enough to demo. On a few hundred documents the right one usually lands in the top-k by luck, and the demo gets read as a validation of the approach rather than of the corpus size. Five things break as the shelf grows, and they break independently of each other. five failures, and why none of them is a tuning problem - Image by author Read the last column first. A wider top-k pulls in noise at the same rate as signal. A different embedding model moves the failures around instead of removing them. Chunk size has nothing to say about a document boundary that was never recorded in the first place.

1.3. Why a re-ranker does not rescue it

The usual next move is a cross-encoder: retrieve a wide top-100, re-score every candidate against the question one at a time, keep the best three. A re-ranker is a real tool and it is worth having, but not here and not yet. It re-orders the list the embedding step returned. If the right document never entered that list, and vocabulary drift is precisely the condition under which it does not, there is nothing to re-order. Article 2bis (what a cross-encoder adds over bi-encoder embeddings, measured) works through where the layer does pay, and the answer is that it pays after the candidate set is right, not instead of getting it right. There is a second cost that gets less attention. The re-ranker adds a number. Asked why the system answered this way, you can now offer a cosine score, a cross-encoder score and a top-3 cut. Three numbers, none of which a claims handler can argue with. The fix sits upstream of all of it. Record the shape of the collection before any question arrives, then use that shape to cut the candidate set. A clause like doc_type = 'distribution agreement' AND client = 'retailer X' is exact, fast, and readable by the person who will be held responsible for the answer. It takes a quarter of a million documents down to three before a single embedding is computed, and with three candidates the single-document pipeline is already enough.

2. Which shape is your collection

The useful part is that you can tell which shape you have without writing anything. Three questions do it, and they are questions for a person rather than for a script.

2.1. Three questions, asked before any code is written

The test costs one conversation with one business user, and it settles the architecture. Three questions, asked of the collection rather than of any one document. Question one: do two documents in the collection ever point at each other? An amendment names its master. A renewal names last year’s policy. A certificate belongs to a contract. A yes here does not pick a shape. It adds a requirement to whichever shape you land on, and section 3.2 says what that requirement is. Question two: can a business user name a field that every document carries, meaning the same thing in each one? Client, effective date, premium, policy number. What tells you the answer is the speed, not the words. If four examples come back in two seconds, the collection is a database that nobody has typed yet, and typing it is the job. If the answer arrives after a pause and with qualifications, treat it as a no. Question three: do the documents arrive in bundles, several of them about one case? A claim, a credit application, a medical record. Not a folder someone made to stay tidy: a bundle where a fixed set of pieces is expected, and where the interesting questions are about the bundle rather than about any piece in it. three questions, and the shape each answer points to - Image by author Questions two and three pick the shape. When both come back yes, and in insurance they often both do, the row of the table is the case rather than the document. That is the third shape, not a fourth one. One practical note. Run the test on a subtree, not on a drive. A shared drive that has grown for ten years usually holds two of these next to each other, and the right answer for the whole drive is the right answer for none of it.

2.2. Three shapes, three architectures

Here they are side by side, with what each one asks you to build once and what the wrong call costs. three shapes, three preparations, three different bills for guessing wrong - Image by author A pile of unrelated files. There is no field to filter on, so there is nothing to build a filter with. The preparation collapses to two artefacts, and the parser already returns one of them: a summary line per file saying what that file is about, and each file’s own table of contents. Retrieval routes down those two levels, which is the hierarchical retrieval of Part III applied one floor higher. One document type, many copies. Five thousand policies of the same product, ten years of invoices, every claim form the company has ever received. The documents look alike because the same process produced them, which is exactly why the columns exist and why someone can name them. This is the shape that gets an index, and the index is a table rather than a vector store. Bundles about one case. A claim folder holds a declaration, a police report, three invoices, a photograph and two letters. Heterogeneous inside, repetitive across: every claim folder holds roughly the same kinds of piece. The unit of work is the bundle, so retrieval assembles it first and reads across it second.

2.3. What it costs to build for the wrong shape

Three shapes means three ways to be wrong, and the bills are not the same size. Treating a typed corpus as a pile. This is the default, because it requires no decision at all. Everything goes into the vector store and the five failure modes of section 1 arrive on schedule. The measurement in section 4 is the small version of it. The real cost is not the compute: it is that the system is wrong in a way nobody can trace, on a corpus where a business user could have written the correct filter by hand in ten seconds. Treating a pile as a typed corpus. This one is expensive in a different way, because it looks like diligence. A team runs a classification pass and an extraction pass over every file to fill columns that no business user could name. Months later there is a table where most cells are empty and the filled ones mean different things per row: a date that is a publication date here, an effective date there, a review deadline somewhere else. Then a filter runs on that column and quietly drops the right document. A column that means three things is worse than no column, because a filter trusts it. Treating case files as independent documents. The quiet one. Every piece answers on its own, so nothing looks broken. What never happens is the comparison. The contract says the cover started on 1 March, the certificate says 1 April, and no question is ever asked that puts the two side by side. The other half is the missing piece: when the medical report was never filed, a document-by-document system returns an empty result, and an empty result reads as “no information” rather than “a required piece is not here”.

3. The same four bricks, one floor up

Whichever shape came out of the test, the pipeline underneath is the one from Part III. What moves is the input each of its four bricks sees, so laying the two side by side is the last thing to do before building anything.

3.1. Nothing new, everything widened

Each of the four bricks does the job it already did on one document, over a wider input. Part IV adds no fifth one. the same four bricks, widened one floor up - Image by author The right-hand column is where the work goes. Parsing gains a second pass that writes one row per document into a table of the collection. Question parsing gains a handful of typed fields whose only job is to narrow that table. Retrieval gains a filtering step in front of the filtering it already does. Generation gains a second source, because sometimes the answer is a count of rows and no document is opened at all. The names do not change and neither do the contracts between the bricks. What changes is how many documents reach the loop body.

3.2. Versions cut across all three shapes

A collection where documents supersede each other is not a fourth shape. It is a dimension that any of the three can carry, and question one is what detects it. It adds two things to whatever you were going to build. A relations table, saying which document replaces or amends which. And a pair of dates on each row, saying from when and until when that document was the one in force. Skip them and the system answers correctly from a document that stopped applying in 2023. That failure is worse than a missing answer because it looks right. The 2019 master says the seller gives 30 days of notice, the 2024 amendment says 90, and a pipeline with no time columns has no reason to prefer one over the other. Which version answers is a retrieval question, and Article 17 (querying a corpus, with the time window as part of the scope) works it out. What matters at this point is only that the two columns exist, because retrieval cannot filter on a field nobody wrote.

4. The baseline, and the waste it makes visible

Before any of the three architectures, it helps to see what the collection problem costs when you ignore it. The simplest possible version wraps the single-document pipeline in a loop, and running it on five real PDFs puts the whole issue in one table.

4.1. A for-loop around the pipeline you already have

The single-document pipeline of Article 1 (the four-brick baseline, a PDF in and a highlighted answer out) has the signature pdf_qa_baseline(pdf_path, question). The collection version has the signature corpus_qa_baseline(question, pdf_paths). Everything between the two is a for-loop and a DataFrame. python

def corpus_qa_baseline(question, pdf_paths, *, client=None, top_k=3):
    rows = []
    for pdf in pdf_paths:
        answer = pdf_qa_baseline(pdf, question, top_k=top_k, client=client)
        text = answer.answer.strip()
        rows.append({
            "doc_id": Path(pdf).stem,
            "answer": answer.answer,
            "start_page": answer.start_page_num,
            "confidence": answer.confidence,
            "has_answer": bool(text) and text.upper() != "NA",
        })
    per_doc = pd.DataFrame(rows).sort_values(
        ["has_answer", "confidence"], ascending=[False, False]
    ).reset_index(drop=True)
    return CorpusAnswer(question=question, per_doc=per_doc)

No classification here, no index, no vocabulary, no filter. The function opens every document, asks every document the same question, and sorts what comes back. That is deliberate. It makes the mirror from section 3 visible with none of the machinery on top, and it is the honest place to measure from.

4.2. Five PDFs, one question, four calls that returned nothing

The question is “What is a Profile in the Cybersecurity Framework?”. Four of the five NIST PDFs cover cybersecurity from a different angle and the fifth covers AI risk. The Profile concept is defined in the framework document and nowhere else in the set. one question, five documents, one answer, four wasted calls - Image by author One document answered: page 5 of the framework, the definition quoted, confidence 0.95. The other four returned nothing usable. Two came back with an empty string, one with the literal NA, and one with a bare slash. That last one is worth a second look. The baseline decides whether a document answered by testing the output string: non-empty, and not the literal NA. A single slash passes both tests, so the sort promoted a document that had said nothing. A heuristic on the output is not a substitute for knowing, before the call, which documents could hold the answer. The run took 14.3 seconds for five documents, so about 2.9 seconds per document for a parse plus one model call. The total is linear in the number of documents, because nothing in the loop is shared between them. Five documents at that rate is a demo. The broker’s shelf at that rate is roughly eight days of wall time for one question. The pipeline is not broken. The right answer came from the right document with a page behind it. What is broken is the ratio. Four calls out of five did no work, and that share only rises as the shelf grows.

5. What the follow-ups build, and what stays out of scope

Three articles follow this one, one per shape. They can be read in any order, because the shapes do not depend on each other. The pile. Article 14B (RAG on a folder of unrelated PDFs: one summary line per file is the whole index) takes the collection with no shared field. It needs the least new machinery of the three: the summary lines and the outlines are the whole index, and retrieval is the hierarchical routing of Part III run one level higher. The typed corpus. Article 14C (the homogeneous corpus, where the business already knows the columns) takes the collection where question two came back yes. Its claim is the one that saves the most time in practice: on a homogeneous corpus you do not discover the schema, the business already holds it, and the interview that produces the columns takes an afternoon. The case file. Article 14D (case files, one bundle about a single entity) takes the third shape. It has the least prior art of the three in the RAG literature, because its two hardest questions are not retrieval questions. Does piece A contradict piece B, and which expected piece is not here. Two things stay outside this group and go to follow-up work. Formats other than PDF. Word, Excel, PowerPoint and mail each need their own parser before any of this applies. The three shapes carry over unchanged; the parsing does not. Document comparison. Asking what changed between version 1 and version 2 of a policy is a different operation from asking a question of a collection. The version chain is in scope here, because retrieval has to know which version answers. The diff itself is not. Case files are often kept out of a corpus pipeline as an oddity. They are common enough in insurance, lending and healthcare that leaving them out means leaving out most of the work, which is why they get an article rather than a caveat.

6. Conclusion

A folder is not a corpus, and a corpus is not one thing. Three questions decide which of three shapes you have, they cost one conversation with one business user, and they are worth asking before a line of ingestion code is written. What they buy is small and specific. Question one tells you whether you need relations and validity dates. Question two tells you whether there is a table to build. Question three tells you whether the row of that table is a document or a case. The for-loop from section 4 survives all three answers. Every article in this group is about the same thing: shrinking what that loop iterates over, from every document on the shelf down to the handful that could plausibly answer.

7. Further reading and sources

Earlier in the series, on the pieces this article stands on:

Also referenced above, listed here without links: Article 1 (the four-brick baseline, a PDF in and a highlighted answer out), Article 2bis (what a cross-encoder adds over bi-encoder embeddings, measured), Article 14B (the folder of unrelated PDFs), Article 14C (the homogeneous corpus), Article 14D (case files), and Article 17 (querying a corpus, with the time window as part of the scope). External sources:

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

Angela and Kezhan Shi

See all from Angela and Kezhan Shi Document ProcessingLlmRagRag ArchitectureRetrieval Augmented Generation Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Using Local LLMs to Discover High-Performance Algorithms

Large Language Models How I used open-source models to explore new frontiers in efficient code generation, using my MacBook and local LLMs Stefano Bosisio January 19, 20268 min read

Vector RAG Isn’t Enough — I Built a Context Graph Layer for Multi-Agent Memory

Large Language Model A structured context graph beat a flat chat dump and a vector-only RAG pipeline on the same multi-agent conversations — full working code, real benchmark numbers, zero API calls Emmimal P Alexander June 26, 202617 min read

How to Apply Powerful AI Audio Models to Real-World Applications

Machine Learning Learn about different types of AI audio models and the application areas they can be used in. Eivind Kjosbakken October 27, 20257 min read

The Next Frontier in LLM Accuracy

Artificial Intelligence Exploring the Power of Lamini Memory Tuning Mariya Mansurova January 4, 202519 min read

The Future of AI Agent Communication with ACP

Artificial Intelligence A practical guide to connecting and coordinating multiple AI agents Mariya Mansurova July 15, 202516 min read

Artificial Intelligence, Machine Learning, Deep Learning, and Generative AI — Clearly Explained

Artificial Intelligence Understanding AI in 2026 — from machine learning to generative models Sabrine Bendimerad December 7, 202510 min read

Improve Your RAG Context Recall by 95% with an Adapted Embedding Model.

Large Language Models Step by Step Model Adaptation Code and Results Attached. Vignesh Baskaran October 12, 202410 min read

How to Build an Efficient Knowledge Base for AI Models

Large Language Model Building a knowledge base for AI models isn’t a one-time task but an iterative process of refinement. Nidhin Karunakaran Ponon May 4, 202615 min read

Cut an Enterprise RAG Pipeline’s Latency and Cost by Calling the LLM Less, Not by Buying a Faster Model

Large Language Model Enterprise Document Intelligence [Vol.1 #9ter] - The pipeline from Article 9 calls a model at several steps to be sure it is right. On easy questions that is needless latency. A per-question signal routes them past the model, about two seconds saved for a keyword match. Angela and Kezhan Shi August 13, 202611 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network

Recommended Articles

Close