保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Large Language Models

A Production RAG Pipeline for PDFs: Relational Parsing, TOC Retrieval, Typed Answers

Enterprise Document Intelligence [Vol.1 #9A] - Same paper, same question as Article 1. One upgraded contract per brick: document parsing, question parsing, retrieval, generation Angela and Kezhan Shi July 7, 202624 min read Photo by Job Vermeulen, via Unsplash. This article opens Part III of Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks: document parsing, question parsing, retrieval, and generation. It is the first of two parts on the upgraded pipeline: this part upgrades each brick, one contract at a time, on the same paper and the same question as Article 1 (minimal RAG). The second part, Composing the four RAG bricks into one pipeline, tested on real documents (link to come), wires them into one call and runs it on several real documents. where this article sits in the series: Article 9 (the upgraded pipeline), opening Part III - Image by author 📓 Runnable companion notebooks are on GitHub : doc-intel/notebooks-vol1. Learn this step by step with the interactive AI Engineer roadmap. The public companion-code repo at doc-intel/notebooks-vol1 - Image by author A hundred lines of Python wire four functions together: parse the PDF, parse the question, retrieve a few pages, ask a model. That pipeline returns the right answer on a clean question against a paper with a built-in table of contents. On a real corpus it breaks the first time the user types “positonal encodig” with two typos, the first time the document is a 200-page contract with no PDF outline, the first time the question asks for every exclusion instead of one, the first time downstream code wants a typed object instead of a string. Four bricks need an upgrade each before the pipeline ships.

The running paper is a public 15-page arXiv submission, Attention Is All You Need. The question, “What are the options for positional encoding?” , comes in with two typos so the question-parsing brick has something to do. The output is what an enterprise user needs: a typed answer, with verbatim quotes tied to line ranges, and a full audit trail from question to citation.

1. Where the baseline RAG breaks

Pick a clean question on a clean PDF, run it through the simplest RAG pipeline: parse the PDF, extract keywords from the question, retrieve a few pages, ask an LLM. On the Attention paper with the question “What are the options for positional encoding?” , that pipeline returns “sinusoidal positional encoding and learned positional embeddings” with one contiguous span of pages cited. It worked, on a clean question, on a clean paper. The Article 1 baseline: the weak point each brick exposes on enterprise input, and its §2 fix. - Image by author Drop the same pipeline into an enterprise setting and four weak points appear quickly, one per brick:

The four bricks below address those four weak points. Same paper, same question, real LLM calls. The output is a typed list with verbatim quotes, line ranges, and the full chain of decisions that produced it.

2. Four bricks, upgraded

The shape that survives the upgrade is the same four bricks Article 1 (minimal RAG) introduced. What changes is the contract per brick: what each one consumes, what it produces, and how the next one consumes it. The diagram below is the full contract: every brick, the outputs it produces, and which downstream brick consumes each one (including the parsing_summary side channel into both question parsing and generation). The per-brick subsections then zoom into each box. The full contract: every brick, its outputs, and which brick consumes each. The per-brick schemas below zoom in on one box at a time. - Image by author Each of the four bricks is upgraded in its own articles, worth reading for the full contract:

2.1 Document parsing: a small relational set

Parsing runs once and turns the PDF into the small set of tables every later brick reuses.

In: pdf_path, the PDF on disk. Out: line_df, page_df, toc_df, parsing_summary. Document parsing:_parse_pdf_reads the PDF once and returns the small relational set every downstream brick reuses. - Image by author What comes out, one row per unit:

Article 1 (minimal RAG) parsed the PDF into one DataFrame called line_df, one row per visible line of text. Enough for keyword retrieval, not enough for anything else. Article 5 (document parsing) reframes parsing as building a small relational set : line_df stays, page_df aggregates lines to pages with their text, and toc_df carries the document’s native table of contents. The bootstrap chunk at the top of the article already produced the three DataFrames on the Attention paper. A look at each: One row per visible line, with _page_num_and _line_num_for citations - Image by author The page_num and line_num on each row are what turn an answer into a citation. Drawn back onto the page they came from, the rows are literal: every recognized line is one box, and its line_num sits in the gutter. The same rows, drawn back onto the page: each blue box is one _line_df_row, the gutter number is its _line_num_- Image by author Aggregating the lines page by page gives page_df, the natural page unit that carries whole-page text and page-level context: One row per page: the natural page unit, whole-page text plus context - Image by author toc_df carries the native outline; on the Attention paper it has three levels and twenty-two entries, one row per section with its title, level, and start page: The native TOC, three levels deep, read straight from the document: one row per section, with its level and start page - Image by author Three tables, same shape contract, same numeric primary keys. Downstream bricks read what they need without re-parsing the PDF; Retrieval (Section 2.3) scans keyword hits and reads toc_df to anchor on the right section, then sizes the context around it (the whole section, or a line window) at the granularity the question implies. page_df is the page-level scan unit, toc_df the map, line_df the atomic lines the anchor and window are cut from. parse_pdf actually returns more in the same dict (image regions, internal references, named objects) plus a parsing_summary carrying document-level metadata (doc type, language, page count, layout, typical fields, a short summary); this section focuses on the three tables retrieval reads, and parsing_summary returns in the second part as the side channel that travels into the LLM bricks.

2.2 Question parsing: from noise to brief

Question parsing turns the raw user string into a typed brief the next two bricks can act on.

In: the raw question, the expert’s concept_keywords_df, and parsing_summary for document context. Out: a ParsedQuestion carrying intent, keywords, a RetrievalQuery brief, and a GenerationBrief. Question parsing: one LLM call corrects the typos and extracts the keywords, then concept_keywords_df expands them. - Image by author What comes out:

Article 1 (minimal RAG) called get_keywords_from_question on the clean question and got a corrected_question plus a short list of keywords back, in one LLM call. That single call does two jobs at once: it fixes the surface typos and pulls the content keywords. The corrected keywords come straight out of the parse, with no separate spell-checking pass bolted on afterward. If a keyword genuinely does not exist in the document, the recovery is retrieval’s feedback loop (Article 13, the workflow pipeline), which re-searches with the terms the document actually uses, not a blind snap onto the nearest corpus token. What this article adds on top of the parse is the expert vocabulary layer. The corrected keywords are expanded with the domain terms a practitioner would also search for, pulled from the expert’s concept_keywords_df. Asked about positional encoding , the expansion adds the two concrete methods, sinusoidal and learned , so retrieval matches them even though the question never named them. Article 6 (question parsing) bundles both layers into the full parse_question brick. The output is a typed ParsedQuestion Pydantic carrying the user’s intent, the extracted keywords, a RetrievalQuery sub-brief that retrieval consumes (main_query, rewrites, anchor_keywords, section_hint, layout_hint), and structural_hints for page / sheet / slide pinning when the question carries them. The question’s intent also fixes how much to keep around the anchor: the whole section when it maps to one, or a line window for a pinpoint fact, so retrieval knows the granularity, not just where to look. Article 7A (retrieval as filtering) develops this two-level anchor / context model in full. Section 1.2 of Article 6 (question parsing) develops the two derived briefs pattern: the same ParsedQuestion yields one brief for retrieval and a GenerationBrief for generation, each carrying only the fields its consumer can act on. Each step is explicit, so retrieval downstream can use all of it and an audit log can be reconstructed. The noisy question is the same one a frustrated user would type: The one call already corrected the typos: optoins and posiitional came back as clean, content keywords, with no second spell-checking pass bolted on. Now expand them with the expert’s vocabulary, the concept_keywords_df table that maps each topic to the terms a practitioner would also search for: python

{
  "original_question": "What are the optoins for posiitional encoding?",
  "keywords": ["positional encoding"],
  "expanded_keywords": ["positional encoding", "sinusoidal", "learned"]
}

Two things happened in one pass. The keywords came back corrected: posiitional and optoins were typos, and the single parse_question call fixed them while pulling the content noun phrase, dropping framing words like options. If a keyword still did not exist in the document, retrieval’s feedback loop (Article 13, the workflow pipeline) would recover it, not a blind snap onto the nearest corpus token. The expanded keywords come from concept_keywords_df. Asked about positional encoding , the expansion adds the two concrete methods the paper uses, sinusoidal and learned , so retrieval matches them even though the question never named them. The table is small on purpose: each entry narrows on the topic, not a generic word like position that would pollute the search. Article 6 (question parsing) develops how it is built and maintained. For the question-parsing code in detail, see the three articles that develop the brick:

2.3 Retrieval: structured tables

Retrieval narrows the document down to the lines generation will read. It filters on the structured tables, it does not search a vector index.

In: the RetrievalQuery brief (from question parsing), plus line_df and toc_df (from document parsing). Out: a RetrievalResult: the kept pages and filtered_line_df, the section or line window generation reads. Retrieval filters, it does not search: keyword hits per section and the TOC router anchor on the section, then the context is sized (the whole section, or a line window) into filtered_line_df. - Image by author How it works, in two phases:

Article 1 (minimal RAG) ran one retrieval method, keyword matching on page_df, and kept the top three pages by match count. Article 7 (retrieval) reframes retrieval as a filter on the small relational set built in Section 2.1: narrow the candidate scope using structured tables before scoring keywords. The keyword method still runs, but toc_df opens a second signal. The natural way to use it is not substring matching on titles. The author of the document already grouped lines into sections and wrote a title for each. A small LLM call can read the whole TOC, reason about which section answers the question, and return its picks with a one-sentence rationale. Retrieval works in two phases (Article 7A, retrieval as filtering). First it finds the anchor : keyword hits, counted per TOC section, tell the router which sections carry the question’s terms; reason_on_toc reads the TOC plus those counts and picks the section. Then it sizes the context around that anchor, following the granularity the question implies (Section 2.2): the whole section for a listing or section question, or a line window around the match for a pinpoint fact. The section is the natural context unit; page_df is the coarse scan surface, line_df the atomic lines the window is cut from. Here, to stay incremental on Article 1 (minimal RAG), this article runs the two detectors and merges their pages; the production hybrid routes them into sections instead (Article 7 (retrieval), the section-first arbiter above). The keyword method runs first (cheap, no LLM, deterministic). The LLM TOC router runs next on the same toc_df. The union of their pages goes to generation. Article 7B (anchor detection) develops the LLM TOC router in detail (the prompt, the Pydantic output, why it beats substring matching on real documents). Article 7C (the LLM arbiter) completes the picture, ranking every candidate page from every detector in a single call. We use the standalone reason_on_toc here because it carries its weight on its own: the upgrade from substring is the single most impactful change a team running Article 1 (minimal RAG)’s pipeline can make to retrieval. Article 7 (retrieval) introduces the unified retrieve_context(question, line_df, *, method, top_k, ...) → RetrievalResult dispatcher that routes to keyword / embedding / TOC / hybrid behind a single signature, and returns a typed RetrievalResult instead of two tuples. We use the raw retrieve_pages and reason_on_toc here to keep this article incremental on top of Article 1 (minimal RAG); production code calls retrieve_context or the shared dispatch_page_retrieval helper. Keyword matching at the page level: a few high-count pages, zeros dropped - Image by author A useful check before trusting that table. The first matching line column scans line by line, one line at a time. That works for single-token keywords. A multi-word keyword like positional encoding can be broken across a line break in the PDF, with positional at the end of one line and encoding at the start of the next. Each line on its own contains neither phrase. The line-by-line scan finds nothing, even when the keyword is right there on the page. Count the misses across the document: Line-unit vs passage-unit detection: multi-word keywords lose hits across line breaks - Image by author The line is a PDF rendering artifact. The text the parser sees as one line is whatever fits between two visual line-break decisions made by the PDF generator. There is no semantic reason the unit of detection should map to that arbitrary boundary. The fix is to detect on a passage instead, where a passage is a small window of adjacent lines joined with a space. Any keyword that exists in the passage will be found, no matter where the line breaks fall. Same pages, same counts; snippets now built on three-line passages - Image by author The page-level match_count was already correct, because retrieve_pages joins all lines on a page before scanning. The fix targets the line-grained helpers that the snippet column, the highlighting, and any later chunking step need. From here on, every helper that scans below the page level uses a passage window, not a single line. The LLM TOC router fills the other gap exposed above. Same toc_df from parsing, but the LLM reads it whole and reasons about which section answers the question, returning the picked section ids plus a one-sentence rationale. The function itself is short (format each TOC row, drop it in a prompt with the question, parse a typed SectionSelection back); Article 7B (anchor detection) shows it in full. Run on the noisy question against the paper’s 22 TOC entries: The LLM read 22 TOC entries and picked the one that answers the question - Image by author One LLM call, the whole TOC inside the prompt, a typed list of section_ids back with a sentence of reasoning. The substring matcher this article used to ship would have caught 3.5 Positional Encoding on this specific question because encoding appears in the title. It would have missed every question phrased differently from the author. “What happens if we exit early?” against a contract whose section is titled “Termination” : substring zero, LLM one. The cost is a small LLM call (a few thousand tokens for a typical TOC, a few hundred milliseconds), and it is cached forever on identical inputs. TOC as backbone with per-section keyword hits; empty sections stay visible - Image by author This view is what an expert reads. Not a flat page list with cosine scores, but the document’s own outline, with the parsed-question keywords marked where they land. Article 7C (the LLM arbiter) consumes exactly this shape as a structured brief, one row per candidate, and ranks them with per-candidate roles + reasons. That arbiter is an upgrade on top of the LLM TOC router walked above, kept out of scope here to stay incremental on Article 1 (minimal RAG). For the retrieval code in detail, see the three articles that develop the brick:

2.4 Generation: a typed answer

Generation fills a typed schema from the retrieved lines. It is controlled execution against a contract, not free-form prose.

In: the GenerationBrief (question and answer shape, from question parsing), filtered_line_df (from retrieval), and the answer schema. Out: a typed AnswerWithEvidence, or a ListAnswer when the question asks for a list. Generation as controlled execution: the LLM fills a typed schema whose every field is checkable against the retrieved lines. - Image by author What comes out:

Article 1 (minimal RAG)’s AnswerWithEvidence returns one answer: str, one contiguous evidence span, a few quotes. Good for a question with a single answer (“What dataset was used?”). The running question asks for options , plural. The baseline schema can list them inside the string, but the caller has no clean way to iterate over the items or to attribute one citation per option. Article 8 makes the schema match the shape of the expected answer. When the question parser flagged expected_answer_shape = "listing" (Article 12 (listing) develops these in depth), the generation brick returns a ListAnswer with one entry per item, each carrying its own evidence span and its own verbatim quote. The shape is what matters: one AnswerItem per option (its text, a start/end line-range span, and a verbatim quote), wrapped in a ListAnswer that also carries the answer-quality flags (answer_found, complete_answer_found, context_structured, confidence, caveats). Article 8 (generation) develops the schema in full. Filled in on the noisy positional-encoding question, the schema returns two items and the full set of quality indicators: python

{
  "items": [
    {
      "text": "Sinusoidal positional encodings (sine and cosine functions of different frequencies).",
      "start_page_num": 6, "start_line_num": 33, "end_page_num": 6, "end_line_num": 37,
      "quote": "we use sine and cosine functions of different frequencies"
    },
    {
      "text": "Learned positional embeddings.",
      "start_page_num": 6, "start_line_num": 39, "end_page_num": 6, "end_line_num": 41,
      "quote": "we also experimented with using learned positional embeddings"
    }
  ],
  "answer_found": true,
  "complete_answer_found": true,
  "context_completeness": 1.0,
  "context_structured": true,
  "confidence": 0.98,
  "caveats": []
}

The four indicators carry the answer’s trust profile. complete_answer_found says whether the answer covers every option the document mentions or only a subset. context_completeness says how well the retrieved lines covered the question, separate from whether the answer itself is right. context_structured flips to false when the LLM cannot follow the reading order, the canonical OCR-failure signal. A downstream router reads them: high confidence + complete + structured = ship the answer; low completeness or unstructured context = retry retrieval, or fall back to a deeper parse (the path Article 10 develops). context_structured is the indicator the article does not get to see fire on a clean paper. To prove the LLM uses it, the same lines fed in random order should flip it to false: python

{
  "clean context": {
    "answer_found": true,
    "complete_answer_found": true,
    "context_completeness": 1.0,
    "context_structured": true,
    "confidence": 1.0,
    "n_items": 2,
    "caveats": []
  },
  "shuffled context (same lines, random order)": {
    "answer_found": true,
    "complete_answer_found": true,
    "context_completeness": 1.0,
    "context_structured": true,
    "confidence": 0.95,
    "n_items": 2,
    "caveats": []
  }
}

In production, scrambled context comes from a different source than df.sample. PDFs with two-column layouts that the parser read column by column instead of row by row, scanned documents OCR’d by a model that lost the layout, exports from Word that broke long tables across hidden anchors. The pipeline cannot self-correct on those, but the answer schema tells the caller something went wrong, and a separate code path takes over. Compare with the baseline RAG output. Same question, same paper, but the caller now gets a structured object: number of items known up front, each item independently citable, four quality indicators that route the answer to the right next step. A UI renders the items as a clickable list. A SQL pipeline writes one row per item. An annotated PDF highlights each quote on its source page. None of those consumers had to parse prose, and none of them ships a confidently-wrong answer because the schema makes the failure modes visible.

2.5 Before and after, per brick

Four bricks, four upgrades. Side by side, the same pipeline before and after, one brick per row: Same four bricks, two contracts: the baseline outputs on the left, the upgraded typed outputs on the right - Image by author The four upgrades are independent. A team running Article 1 (minimal RAG)’s pipeline can adopt them one at a time: a TOC at parsing, a corpus-vocab spell-check at question parsing, an LLM TOC router + keyword fallback at retrieval, a list-shaped schema at generation. Each one is a small drop-in, none of them rewrites the surrounding code.

3. Conclusion

The four bricks now speak in typed contracts:

Each upgrade earned its place against a concrete failure of the baseline: the typo that missed the page, the TOC the keyword search ignored, the list flattened into prose. What the bricks do not have yet is a single entry point and a feedback path between them. The second part wires them into one call and runs it on real documents, including one whose TOC is broken, in Composing the four RAG bricks into one pipeline, tested on real documents (link to come). It helps to see where this pass sits on a longer climb. The same four bricks stay fixed; what changes from one rung to the next is where the control lives. This article is rung two: pdf_qa, a single richer pass that already emits the feedback fields (retrieval confidence, missing keywords) but does nothing with them yet. Article 13 turns that pass into pdf_qa_flow, the Volume 1 composite that dispatches on question patterns and re-runs the pass in a bounded loop, acting on exactly those fields. Volume 2’s pdf_chat puts a multi-intent entry in front. Only the last rung hands the control loop to the LLM itself. Same four bricks, five levels of control: this article is rung two, the richer pass whose feedback fields become the loop one rung up - Image by author

4. Sources and further reading

This part upgrades the four bricks (from Articles 5-8) one contract at a time, on the Attention Is All You Need paper. The second part composes them end to end and runs the assembled pipeline on several documents. The responses.parse(text_format=Schema) pattern at the question-parsing and generation boundaries uses OpenAI’s Structured Outputs (Aug 2024). The closest published production-grade write-up of this kind of pipeline is Anthropic’s Contextual Retrieval (Sept 2024). The agentic upgrade path on top of the same four bricks is follow-up work; the per-brick provenance keeps the agent’s choices auditable. Earlier in the series:

What works, what breaks

Document parsing

Question parsing

Retrieval

Generation

Same direction as the article:

Runnable code paths call OpenAI services governed by OpenAI’s Terms of Use. Different angle, different context:

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

Angela and Kezhan Shi

See all from Angela and Kezhan Shi Document ProcessingLlmRagRag Architectureretrieval Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Recursive Language Models: An All-in-One Deep Dive

Large Language Models Exactly how does it differ from ReAct, CodeAct, Self-Loops, and Subagents? Avishek Biswas May 16, 202619 min read

Automate Video Chaptering with LLMs and TF-IDF

Large Language Models Transform raw transcripts into well-structured documents Yann-Aël Le Borgne September 9, 202412 min read

Using Self-Organizing Map To Bolster Retrieval-Augmented Generation In Large Language Models

Large Language Models SOM is proposed to bolster efficient retrieval of LLM context for RAG... Murali Kashaboina March 16, 202417 min read

Understanding LLMs from Scratch Using Middle School Math

Artificial Intelligence In this article, we talk about how LLMs work, from scratch - assuming only that you know how to add and multiply two numbers. The article... Rohit Patel October 19, 202443 min read

Is RAG Dead? The Rise of Context Engineering and Semantic Layers for Agentic AI

Large Language Models How retrieval-augmented generation is evolving from static pipelines to governed, context-aware systems that make AI more explainable, trustworthy, and scalable Steve Hedden October 21, 202515 min read

Advanced Retriever Techniques to Improve Your RAGs

Master Advanced Information Retrieval: Cutting-edge Techniques to Optimize the Selection of Relevant Documents with Langchain to Create... Damian Gil April 17, 202420 min read

How to Design My First AI Agent

Artificial Intelligence The foundations of designing an AI agent Fabiana Clemente June 3, 20258 min read

MultiChoice Question Answering In HuggingFace

Deep Learning Unveiling the power of question answering Mina Ghashami February 7, 202416 min read

Augmenting LLMs with RAG

Machine Learning An End to End Example Of Seeing How Well An LLM Model Can Answer Amazon SageMaker Related Questions Ram Vegiraju October 10, 20239 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network

Recommended Articles

Close