![Revisit consent button](../images/body-df29beacd7496220cc94.svg) We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg) We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data. * Cookie BCTempID * Duration 10 minutes * Description No description available. * Cookie __cf_bm * Duration 1 hour * Description This cookie, set by Cloudflare, is used to support Cloudflare Bot Management. * Cookie AWSALBCORS * Duration 7 days * Description Amazon Web Services set this cookie for load balancing. * Cookie _cfuvid * Duration session * Description Cloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services * Cookie li_gc * Duration 6 months * Description Linkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes. * Cookie __hssrc * Duration session * Description This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session. * Cookie __hssc * Duration 1 hour * Description HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie. * Cookie wpEmojiSettingsSupports * Duration session * Description WordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly. * Cookie BCSessionID * Duration 1 year 1 month 4 days * Description Blueconic sets this cookie as a unique identifier for the BlueConic profile. * Cookie _octo * Duration 1 year * Description No description available. * Cookie logged_in * Duration 1 year * Description No description available. * Cookie __Secure-YEC * Duration past * Description YouTube sets this cookie to stores the user's video player preferences using embedded YouTube video * Cookie __eoi * Duration 6 months * Description Description is currently not available. * Cookie AWSALBTGCORS * Duration 7 days * Description No description available. * Cookie login-status-p * Duration past * Description Description is currently not available. * Cookie AWSALBTG * Duration 7 days * Description No description available. * Cookie csrf_token * Duration session * Description No description available. * Cookie token_v2 * Duration 1 day * Description Description is currently not available. * Cookie D * Duration 1 year * Description Description is currently not available. * Cookie PHPSESSID * Duration session * Description This cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed. * Cookie VISITOR_PRIVACY_METADATA * Duration 6 months * Description YouTube sets this cookie to store the user's cookie consent state for the current domain. * Cookie cookietest * Duration session * Description The cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience. * Cookie __Host-airtable-session * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie __Host-airtable-session.sig * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie m * Duration 1 year 1 month 4 days * Description Stripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly. * Cookie BIGipServer* * Duration session * Description Marketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. * Cookie __cfruid * Duration session * Description Cloudflare sets this cookie to identify trusted web traffic. * Cookie _GRECAPTCHA * Duration 6 months * Description Google Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks. * Cookie __Secure-YNID * Duration 6 months * Description Google cookie used to protect user security and prevent fraud, especially during the login process. * Cookie cookieyes-consent * Duration 1 year * Description CookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors. Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features. * Cookie lidc * Duration 1 day * Description LinkedIn sets the lidc cookie to facilitate data center selection. * Cookie brw * Duration 1 year * Description No description available. * Cookie brwConsent * Duration 5 minutes * Description Description is currently not available. * Cookie WMF-Uniq * Duration 1 year * Description Description is currently not available. * Cookie loom_anon_comment * Duration 1 year * Description No description available. * Cookie loom_referral_video * Duration session * Description Description is currently not available. * Cookie VISITOR_INFO1_LIVE * Duration 6 months * Description A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface. * Cookie yt-remote-connected-devices * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie ytidb::LAST_RESULT_ENTRY_KEY * Duration Never Expires * Description The cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future. * Cookie yt-remote-device-id * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie yt-remote-session-name * Duration session * Description The yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video. * Cookie yt-remote-fast-check-period * Duration session * Description The yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos. * Cookie yt-remote-session-app * Duration session * Description The yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player. * Cookie yt-remote-cast-available * Duration session * Description The yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player. * Cookie yt-remote-cast-installed * Duration session * Description The yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video. * Cookie cp_session * Duration 3 months * Description Codepen sets this cookie for Help systems found in the website. * Cookie loid * Duration 1 year 1 month 4 days * Description This cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform. Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc. * Cookie __hstc * Duration 6 months * Description Hubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session). * Cookie hubspotutk * Duration 6 months * Description HubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts. * Cookie _ga * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors. * Cookie _ga_* * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to store and count page views. * Cookie __Host-psifi.analyticsTrace * Duration 6 hours * Description Description is currently not available. * Cookie __Host-psifi.analyticsTraceV2 * Duration 6 hours * Description Description is currently not available. * Cookie _gh_sess * Duration session * Description GitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form. * Cookie YSC * Duration session * Description YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages. * Cookie ajs_anonymous_id * Duration 1 year * Description This cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before. * Cookie vuid * Duration 1 year 1 month 4 days * Description Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors. * Cookie AWSALB * Duration 7 days * Description AWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target. * Cookie acq * Duration past * Description Description is currently not available. * Cookie acq.sig * Duration past * Description Description is currently not available. * Cookie ptc * Duration 2 years * Description No description available. Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns. * Cookie muc_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to collect user behaviour and interaction data to optimize the website. * Cookie guest_id_marketing * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie guest_id_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie personalization_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting. * Cookie guest_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences. * Cookie bcookie * Duration 1 year * Description LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs. * Cookie __Secure-ROLLOUT_TOKEN * Duration 6 months * Description YouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment. * Cookie yt.innertube::nextId * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie yt.innertube::requests * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie session_tracker * Duration session * Description This cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website. * Cookie edgebucket * Duration session * Description Reddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content. * Cookie did * Duration 1 year * Description Arbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year. Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation [](https://towardsdatascience.com/) Publish AI, ML & data-science insights to a global community of data professionals. Sign in[Submit an Article](https://write.insightmediagroup.io) [Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/) * * * [Write For TDS](https://towardsdatascience.com/submissions/) [LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search [Large Language Model](https://towardsdatascience.com/category/large-language-model/) # RAG Workflow and Loop Engineering: The Dispatcher That Decides When to Loop and When to Stop Enterprise Document Intelligence [Vol.1 #13] - Putting the patterns together, and why this is what “agentic RAG” should look like [Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/) August 14, 202627 min read ![](../images/body-4e9bca8a9f9ba81206b0.jpg) [Photo](https://www.pexels.com/photo/32529354/) by Fernando Narvaez, via Pexels. Over the last few articles, we built loop patterns one at a time: a re-parse when a page check fails, a second retrieval when the answer points to another section, an aggregation sweep when the question asks for a complete list. Each pattern has its own article, its own trigger, and its own test. Each one works fine on its own. Real questions do not arrive one pattern at a time. Take one a compliance officer would actually ask on the [NIST Cybersecurity Framework](https://www.nist.gov/cyberframework) (US Government work, public domain in the US, see [NIST copyright statement](https://www.nist.gov/director/copyright-fair-use-and-licensing-statements-srd-data-software-and-technical-series)): _“What are all the Categories under GOVERN, and which one covers supply chain risk?”_ The question sounds ordinary. Inside the pipeline, it fires three patterns at once: * TOC retrieval, to land on the right section; * listing aggregation, to enumerate every Category, not just the most-cited ones; * a synthesis step, to pick the one that covers supply chain risk. Each of the three carries its own iteration mechanic: a re-retrieval here, a re-generation there, an LLM flag that triggers a second pass. Run them on the same question and a practical problem shows up: which one decides when to stop? Leave that undecided and every new kind of question becomes another special case bolted on the side, and nobody can say what the pipeline will do next. The fashionable answer today is to hand that decision to an agent and let the model orchestrate. For an enterprise pipeline, we prefer a piece of code we can read: a dispatcher that turns the parsed question and the document profile into an explicit plan, and bounded loops that say, in code, how far each pattern may iterate. That is what this article builds: the feedback loops, the bounded iteration, and the dispatcher that composes them into one workflow. Learn this step by step with the interactive [AI Engineer](https://roadmap.sh/ai-engineer) roadmap. This article closes Part III of [Enterprise Document Intelligence](https://towardsdatascience.com/prompt-context-loop-the-three-engineering-layers-every-rag-system-is-built-on/), a series that builds an enterprise RAG system from four bricks: document parsing, question parsing, retrieval, and generation. 🧭 **New to the series? Start with the map** : [Prompt, Context, Loop](https://towardsdatascience.com/prompt-context-loop-the-three-engineering-layers-every-rag-system-is-built-on/) sets out the three engineering layers every RAG system is built on, the prompt (the call itself), the context (what fills the model’s window), the loop (when the next call fires and when it stops), and walks the whole series through that lens, article by article. It is the shortest way to see what is covered and where this one sits. ![](../images/000.png) _where this article sits in the series: Article 13 (the workflow pipeline), closing Part III - Image by author_ 📓 The runnable companion drives the loop machinery yourself: you run `pdf_qa_loop` on a question that fails its first pass, print the `IterationRecord` history showing what each retry changed, and watch `should_continue` cut the loop when the candidates stop moving. On GitHub: [doc-intel/notebooks-vol1](https://github.com/doc-intel/notebooks-vol1). ![](../images/001.png) _The public companion-code repo at doc-intel/notebooks-vol1 - Image by author_ In production, real questions stack the patterns: _“List the obligations of the seller, including any referenced standards”_ on a contract adds two-hop reference resolution on top of listing. The iteration mechanics ride alongside them. It helps to place this article on a five-rung climb, each rung a more capable version of the same PDF question-answering function. The **baseline** ([Article 1](https://towardsdatascience.com/baseline-enterprise-rag-from-pdf-to-highlighted-answer-enterprise-document-intelligence-vol-1-1/)) chains the four bricks once with keyword retrieval: one pass, return the answer. The **upgraded** version (Article 9, the four bricks upgraded contract by contract) keeps the single pass but makes it richer, with a full relational parse and TOC routing, and it returns a typed answer carrying feedback fields (is the context complete? did the parse hold?), produced but not yet acted on. The **workflow** rung, this article, turns that pass into one step inside a bounded loop: a dispatcher picks which patterns fire, and the loop reads the feedback fields to decide whether to retry, with the control staying in code. The **multi-intent** rung (follow-up work) widens the entrance: a chat entry classifies what the user wants (a question, a translation, a summary, or just _“hello”_ , which deserves a direct reply and no pipeline at all) and routes to the right pipeline, the decision still in code. The **agentic** rung (further out) moves the control loop into the LLM, which picks the next step itself; this article stops two rungs below, where the pipeline is still reproducible and auditable. Rungs one to four keep control in code; only the last jump moves who holds the loop from code to model. ![](../images/body-b1c4967f37f3de3278dc.png) _Five levels of control, control in code until the last rung; this article is the workflow rung - Image by author_ This article is about the **composite pipeline** : a single orchestrator function that takes a question, decides which patterns to activate, runs them, manages the feedback loops, and produces an answer. The function is `**pdf_qa_loop**`: the single pass from the upgraded pipeline, grown one rung, dispatching across Part III’s patterns within`(pdf, qa)`. Follow-up work adds the entry points above it (other document formats, other intents); that scope sits outside this article. The patterns become a toolkit. `pdf_qa_loop` becomes their orchestrator. ![](../images/body-b804bdf9a65d43d274af.png) _Four patterns ON, three OFF; only ON patterns reach the Answer - Image by author_ The dispatcher is where **“amplify the expert”** lands at the composition layer. The team’s routing wisdom (which patterns fire for which question types, in which order) gets written down once, in deterministic Python, and runs across every future question. The LLM appears at every brick. It never chooses the next call. Pattern selection is the dispatcher’s job; the iterate-or-stop decision is the loop machinery’s. Both stay in code, reviewable by the people whose judgment they codify. ## 1. One layer composes, the bricks stay apart A workflow like this needs exactly one new home in the codebase: the composition layer. It holds the pieces that exist only because bricks are being combined: the dispatcher that picks which patterns fire, the feedback machinery that reads the typed flags, and one flow per intent. Nothing else moves. The four bricks keep living in their own modules, and the patterns of Part III stay where they belong as sub-functions of their brick: TOC retrieval in the retrieval module, escalation in parsing, two-hop resolution across retrieval and generation. The two layers talk through typed objects (a `ParsedQuestion`, a `DocumentProfile`, an `AnswerWithEvidence`), never through each other’s internals. One way to lay that out on disk, the one this series uses: a `pipeline/` folder sitting next to the brick modules: ![](../images/body-eb24335e5806898d128d.png) `_pipeline/_`_holds the composition layer; the bricks themselves live next door - Image by author_ Inside the composition layer, code is organised by intent (one subpackage per intent, one file per format inside), with the cross-intent infrastructure flat at the root; Article 18 (code architecture) develops the file layout in detail. This split makes the system maintainable. When you improve TOC retrieval, you change the retrieval module only. When you fix a bug in two-hop resolution, the orchestrator stays untouched. The contract between orchestrator and bricks is the interface (a `ParsedQuestion`, a `DocumentProfile`, an `Activations` dict, an `AnswerWithEvidence`), not the implementation. A senior engineer can trace any request from input to highlighted answer in a few minutes: `pdf_qa_loop` (`pipeline/qa/pdf_flow.py`) reads the question, calls `decide_pipeline_patterns` for activations, runs the four-brick pass (the same parse and retrieval entry points as Article 9’s `pdf_qa`) with those activations, reads the generation feedback fields, and either returns the answer or queues a loop. No hidden state, no agent loop. The file layout has a runtime twin: how the handlers nest when a request actually runs. Each layer calls the one inside it and adds exactly one concern, and the control loop stays in code at every level. ![](../images/body-a3f55234a12a3334354b.png) _Each ring calls the one inside and adds one concern, with the control loop always in code - Image by author_ ## 2. The orchestrator The orchestrator does three things, in order: **1. It reads the parsed question** (`ParsedQuestion` from Article 6) and decides which patterns to activate. Some questions need all of them. Most don’t. **2. It runs the active patterns** , passing the right inputs to each. The order matters: TOC retrieval before keyword retrieval (so structural anchors come first), primary retrieval before two-hop (so the two-hop step has something to follow from), and so on. **3. It manages feedback loops.** When the generation step signals an incomplete answer, the orchestrator decides whether to iterate and how many times. Multiple loops can fire on one question (parsing escalation, vocabulary expansion, reference expansion, listing iteration). The orchestrator picks the order across iterations (parsing first, vocabulary next, references next, listing last) and shares one budget across all of them. Two to three iterations cover the vast majority of cases; the composite’s hard cap is four, because several loops can fire on one question and each needs its own pass. Order matters because re-retrieving on bad parsing wastes the iteration; better keywords improve every later step. Concretely: python ``` # pdf_qa_loop, condensed # decide_pipeline_patterns and iterate_with_bound come from the companion notebook. def pdf_qa_loop(pdf_path, question, *, registry=None, max_iterations=4): registry = registry or PatternRegistry.default() # 1. Question understanding parsed = registry.parse_question(question) # 2. Cheap layer-1 parse of the document line_df, page_df, toc_df = registry.parse_layer1(pdf_path) # 3. Document profile doc_profile = registry.detect_document_type(pdf_path) # 4. Dispatcher : which patterns fire ? activations = decide_pipeline_patterns(parsed, doc_profile) # 5-7. Retrieval + generation inside a bounded feedback loop state = _State(parsed=parsed, line_df=line_df, page_df=page_df, toc_df=toc_df) outcome = iterate_with_bound( initial_state=state, run_pass=lambda s: _one_pass(s, registry, activations, pdf_path), is_satisfactory=lambda a: not _needs_iteration(a), adjust=lambda s, a: _adjust_state(s, a, registry, activations, pdf_path), record=_build_record, max_iterations=max_iterations, ) return CompositeOutput(answer=outcome.result, activations=activations, history=outcome.history, exhausted=outcome.exhausted) ``` That’s the whole orchestrator. The `pdf_qa_loop` body is a page of glue code sitting on top of the per-pattern modules. The third bullet (feedback loops) has its own dedicated machinery; section 3 walks through it. The first two bullets (pattern selection, run order) are deterministic once `decide_pipeline_patterns` returns its column of ON/OFF flags; section 4 walks through the dispatcher itself. ## 3. The feedback loop pattern The composite pipeline’s distinguishing feature, compared to a naive RAG, is that the answer is treated as a **provisional output that the system itself critiques**. The loop machinery sits between generation and the final return. ### 3.1 From naive RAG to feedback-driven RAG A naive RAG pipeline runs the four bricks in sequence and returns whatever comes out: parse the document, parse the question, retrieve, generate. ![](../images/body-9a83ee90e850c1b675f5.png) _The four bricks chained once; parsed_q reaches Retrieval and Generation, no critique step - Image by author_ If retrieval missed the right passage, the answer is wrong. If parsing was insufficient, the LLM hallucinates around the gap. The user never knows. The composite pipeline adds a critique step at the end of the row, with two feedback rails that loop back whenever the LLM signals a problem: back into retrieval when the answer is incomplete (`complete_answer_found=false`, expand the keywords and re-retrieve), and back into parsing when the context came out unstructured (`context_structured=false`, re-parse the flagged pages): ![](../images/body-1fbac035845b88b24fee.png) _Same four bricks plus Critique; two rails re-inject on a trigger, the green arrow exits on convergence - Image by author_ The critique step reads the structured feedback fields the LLM produced (Article 8): is the context complete? did the LLM discover new keywords during reading? was the parsing sufficient? If anything is off, the pipeline takes targeted action and re-generates. These rails are the pipeline’s _big_ loops: the ones that cross bricks, triggered by generation reading its own input (adaptive parsing in Article 10, reference resolution in Article 11, scope feedback here). Each brick also runs its own _small_ bounded loops inside (the image cascade in parsing, the TOC descent in retrieval, the schema retry in generation); those never reach the orchestrator. The dispatcher owns the big ones only. This is what most vendors now market as **agentic RAG**. The term covers three quite different things; section 6 sorts them out. ### 3.2 Signal, trigger, action Each loop has three dimensions: the **signal** (what the LLM or a programmatic check reports), the **trigger** (which state of the output starts a retry), and the **action** (what the pipeline changes for the next pass). Together with the bounds of section 3.3, these are the three control surfaces every loop in the series wears: signal plus trigger decide when a new pass fires, the action is the recovery (the one thing that changes before the retry), and the bounds are the termination. **Signals come from three sources** : * **LLM self-assessment:** Fields like `complete_answer_found`, `confidence`, `caveats`. Soft signal, broadest coverage. * **Programmatic checks:** Deterministic code validates citations, format constraints, cardinality. Stronger signal because the check itself does not depend on the LLM’s judgment, even though it runs on the LLM’s output. * **External validation:** A separate model, a downstream tool, or a human. Strongest, most expensive. Reserved for high-stakes outputs. **Triggers map to specific actions** (each was introduced once in Part II or III; here is the consolidated catalogue): * **Incomplete answer** (`complete_answer_found=False` from Article 9’s `AnswerWithEvidence`, or `is_likely_complete=False` from Article 12’s `ListAnswer`) → expand retrieval scope or activate a new pattern (listing, two-hop). * **Insufficient context** (`context_structured=False`, Article 10) → trigger adaptive parsing on the failing pages. * **Discovered vocabulary** (`llm_discovered_keywords` populated, Article 8) → re-retrieve with the expanded vocabulary. * **Pending references** (`pending_references` non-empty, Article 11) → run two-hop retrieval on each reference. * **Cardinality mismatch** (Article 12) → iterate with expanded keywords or a broader section restriction until the count matches or a bound is hit. * **Failed validation** (any programmatic check failed) → re-retrieve or ask the LLM to correct. Each trigger maps to a named, targeted action, which keeps the loop debuggable. ### 3.3 Bounding the loops The main risk of iterative pipelines is unbounded iteration. Three controls. **Maximum iterations:** A ceiling on `pdf_qa_loop`’s `max_iterations` parameter. Two or three iterations cover the vast majority of cases. Beyond that, returns diminish sharply. python ``` def iterate_with_bound( *, initial_state: TState, run_pass: Callable[[TState], TResult], is_satisfactory: Callable[[TResult], bool], adjust: Callable[[TState, TResult], TState], record: Callable[[int, TResult, TState, TResult | None], IterationRecord], max_iterations: int = 3, ) -> IterationOutcome[TState, TResult]: """Run a bounded pipeline-with-feedback loop.""" state = initial_state history: list[IterationRecord] = [] result = run_pass(state) previous_result: TResult | None = None if is_satisfactory(result): history.append(record(0, result, state, previous_result)) return IterationOutcome(result=result, history=history, exhausted=False) for i in range(1, max_iterations): next_state = adjust(state, result) history.append(record(i, result, next_state, previous_result)) previous_result = result state = next_state result = run_pass(state) if is_satisfactory(result): return IterationOutcome(result=result, history=history, exhausted=False) return IterationOutcome(result=result, history=history, exhausted=True) ``` A loop that hits `max_iterations` doesn’t crash. It returns the best output produced so far, flagged so the user and the audit trail know iteration was exhausted. **Termination conditions:** Beyond max iterations, the loop stops when: * **No new signal:** The same trigger fires twice with the same suggested action. * **No new candidates:** Retrieval returned the same set of passages as last iteration. * **Decreasing confidence:** The loop is making things worse, not better. Stop iterating and return the highest-confidence answer seen so far. python ``` def should_continue(history: list, current: _ResultProto, confidence_drop_threshold: float = 0.1) -> bool: """Decide whether the iteration loop should run another pass. Three reasons to stop, in order of priority: 1. Candidates are stable (same set as last pass): nothing new will be found. 2. Suggested keywords are stable: the LLM is repeating itself. 3. Confidence is decreasing past the threshold: the loop is making things worse. """ if history: prev = history[-1] if getattr(prev, "candidates", None) == current.candidates: return False if getattr(prev, "suggested_keywords", None) == current.suggested_keywords: return False prev_conf = getattr(prev, "confidence", None) if prev_conf is not None and current.confidence < prev_conf - confidence_drop_threshold: return False return current.needs_iteration() ``` Without these, even with a max-iterations bound, the pipeline burns three full iterations on cases where it had clearly stopped progressing after the first. **Drift detection:** A subtler failure: the loop iterates _successfully_ (each pass finds new candidates) but the answer drifts away from the original question. Keyword expansion picked up a tangent. The user asked about _“premium”_ , the LLM saw _“insurance”_ , expanded to _“policy”_ , then to _“coverage”_ , then to _“reinsurance”_ , and the answer ends up about reinsurance economics. The fix is to keep the original question in scope at every iteration. Expanded keywords add to the retrieval, they don’t replace the original anchors. Generation always sees the original question. python ``` def expand_query_safely(parsed, new_keywords: list[str], max_keywords: int = 15): try: expanded = parsed.model_copy(deep=True) except AttributeError: import copy as _copy expanded = _copy.deepcopy(parsed) originals = list(expanded.retrieval.anchor_keywords or []) additions = [k for k in new_keywords if k not in originals] combined = originals + additions if len(combined) > max_keywords: kept_originals = originals[:max_keywords] room = max_keywords - len(kept_originals) combined = kept_originals + additions[-room:] if room > 0 else kept_originals expanded.retrieval.anchor_keywords = combined return expanded ``` A drift check at each iteration: if the new candidates have low overlap with the _original_ anchor keywords, that’s drift, and the loop should stop. ### 3.4 Anti-patterns and audit trail Five anti-patterns to keep out of the orchestrator: * **Iterating on confidence alone:** Confidence is noisy. An LLM can be highly confident on a wrong answer or low confidence on a correct one. Use confidence as one signal among several, never alone. * **Unbounded keyword expansion:** Cap the keyword set size (10 to 15) and prune the least informative when the cap is hit. Otherwise after three iterations the retrieval query is fifty terms long and signal-to-noise collapses. * **Re-parsing more than necessary:** When `context_structured=False` is signaled, re-parse only the pages the LLM flagged, not the whole document. * **Hiding iteration from the audit trail.** Every iteration must produce an `IterationRecord`. Compliance contexts require this; debugging contexts benefit from it. * **Deciding to iterate purely with the LLM.** The LLM can contribute _signal_ to the iteration decision but the decision to iterate or stop must be in code, with explicit rules. `should_continue` (above) is where those rules live. The dispatcher decides only which patterns to fire upfront, not whether to loop. json ``` { "iteration_number": 2, "trigger": "vocabulary_gap", "action_taken": "re-retrieve with `excess`", "result_summary": "page 7 joined the candidate set; confidence rose by 0.22", "confidence_before": 0.62, "confidence_after": 0.84 } ``` The full iteration history travels with the final answer. For audit and compliance, this is the difference between _“the system gave this answer”_ and _“the system gave this answer through these specific steps, each justified by an explicit trigger.”_ The UI presents a one-line narration per iteration so the user sees the system making informed choices rather than running blind retries. ## 4. The dispatcher The dispatcher (`decide.py`) is where most of the explicit decisions happen. It takes the parsed question and the document profile, and returns a dictionary of pattern activations. python ``` def decide_pipeline_patterns( parsed: ParsedQuestion, doc_profile: DocumentProfile, ) -> Activations: activations = dict(DEFAULT_ACTIVATIONS) # TOC retrieval: enable whenever the document has a usable TOC. if doc_profile.has_usable_toc: activations["toc_retrieval"] = True # Dense retrieval (embeddings): only as a fallback for questions whose # vocabulary may not match the document's wording. if parsed.intent in ("open_scoped", "open_corpus_wide"): activations["dense_retrieval"] = True # Two-hop references: enable when the question carries a structural # hint that points outside its primary passage. if parsed.retrieval.section_hint or parsed.retrieval.layout_hint: activations["two_hop_references"] = True if parsed.intent in ("section_retrieval", "open_scoped"): activations["two_hop_references"] = True # Listing aggregation: enable when the question intent is listing. if parsed.intent == "listing": activations["listing_aggregation"] = True return activations ``` This function encodes the team’s understanding of _which questions need what_. As the system runs in production, it evolves: new question types, new heuristics, edge cases observed in evaluation feed back into the dispatcher. The dispatcher is also the file the team should iterate on with care. A bad activation rule can either over-engineer simple questions (running listing aggregation when not needed) or under-engineer complex ones (skipping two-hop on questions that need it). Each rule should have a test case attached. ## 5. Worked example: a listing-plus-references question on the Transformer paper The previous articles have looked at NIST CSF listing questions. To make the dispatcher do something more, we pick a question on the [_Attention Is All You Need_](https://arxiv.org/abs/1706.03762) paper (Vaswani et al. 2017; arXiv non-exclusive distribution license, declared on the [arXiv abstract page](https://arxiv.org/abs/1706.03762)) that fires two retrieval patterns at once. Runnable code paths call OpenAI services governed by [OpenAI’s Terms of Use](https://openai.com/policies/terms-of-use). **Question:** _“What regularization techniques does the Transformer paper use, and where do they show the impact on BLEU?”_ The first half is a listing question (Article 12 already covered it: three techniques, Section 5.4). The second half is a cross-reference question (Article 11: the impact numbers live in Table 3, not in Section 5.4). Both have to fire on the same call. ### 5.1 The inputs: parsed question and document profile The orchestrator’s two inputs are a `ParsedQuestion` (from Article 6) and a `DocumentProfile` (a cheap probe on the PDF). Both are constructed below from real code. `detect_document_type` opens the Transformer PDF; the `ParsedQuestion` is what the LLM-backed `parse_question` would produce for the question above. json ``` { "original_question": "What regularization techniques does the Transformer paper use, and where do they show the impact on BLEU?", "keywords": ["regularization", "dropout", "label smoothing", "BLEU"], "intent": "listing", "retrieval": { "main_query": "regularization techniques and their impact on BLEU", "rewrites": ["Residual Dropout", "Attention Dropout", "Label Smoothing", "Pdrop", "label smoothing epsilon", "Table 3"], "anchor_keywords": ["regularization", "dropout", "label smoothing", "Table 3"], "section_hint": "5.4", "layout_hint": "table" } } ``` The three fields the dispatcher reads: `intent = "listing"`, `section_hint = "5.4"`, `layout_hint = "table"`. The first activates listing aggregation; the other two activate two-hop references. json ``` { "total_pages": 15, "has_usable_toc": true, "is_likely_scanned": false, "suggested_strategy": "native_text_only" } ``` The profile is cheap: one `fitz.open` call, no body parsing. Page count, TOC presence (`has_usable_toc = True` because the Transformer paper carries 22 bookmarks), and a scan flag (False here, since the PDF is native text). The dispatcher reads `has_usable_toc` to enable TOC retrieval. ### 5.2 Dispatcher decision `decide_pipeline_patterns(parsed_q, doc_profile)` reads the two objects above and returns an activation column. The output below is the real result from running the dispatcher on this question and this document: json ``` { "toc_retrieval": true, "keyword_retrieval": true, "dense_retrieval": false, "two_hop_references": true, "listing_aggregation": true, "adaptive_parsing": false, "iterative_feedback": true } ``` Four out of seven patterns activated (plus iterative feedback as the always-on safety net). The two-hop pattern makes this question different from a plain listing. ### 5.3 First pass: retrieval, listing, and the gap signal `pdf_qa_loop` runs its pass (the same parse and retrieval entry points as Article 9’s `pdf_qa`) with the activation flags above. Inside that pass: * **TOC retrieval** (Article 9) jumps to Section 5.4 _“Regularization”_ (page 7). The keyword retrieval, anchored on `regularization`, `dropout`, `label smoothing`, reinforces pages 7-8 (where the bold headers `Residual Dropout` and `Label Smoothing` sit). RRF fusion ranks pages 7-8 first. * **Listing aggregation** (Article 12) restricts to Section 5.4 and picks up three techniques: sub-layer dropout (Pdrop = 0.1, on the output of each sub-layer before add+layernorm, page 8), embedding dropout (Pdrop = 0.1, on the sum of token and positional embeddings, page 8), and label smoothing (εls = 0.1, page 8). The first two share the _Residual Dropout_ paragraph in the paper; label smoothing is its own paragraph. The cardinality cue _“We employ three types of regularization during training”_ at the bottom of page 7 confirms the count. * **First generation pass** produces an answer with the three techniques but no validation numbers. The LLM sets `complete_answer_found=False` and `pending_references=["Table 3"]` because the prose says results are reported in Table 3 but Table 3 wasn’t in the candidate set. The pipeline doesn’t return an answer yet. The `pending_references` trigger fires. ### 5.4 Second pass: two-hop fetches Table 3, generation closes the loop The orchestrator’s feedback machinery sees `pending_references = ["Table 3"]` and runs the two-hop pattern (Article 11). Page 9 carries _Table 3: Variations on the Transformer architecture_. The relevant rows for this question are: * **Row (D)** : varies `Pdrop` at 0.0 / 0.1 / 0.2. * **Row (E)** : varies `εls` at 0.0 / 0.1 / 0.2, plus a positional-embedding variant. Rows (A), (B), (C) vary attention heads, key dimension, and model size and are not relevant to the regularization question. The two-hop retriever pulls page 9 and joins it to the candidate set, then `pdf_qa` re-runs generation with Section 5.4 plus Table 3 in the context: `The Transformer paper uses three regularization techniques during training: 1. Residual Dropout -- Pdrop = 0.1, page 8 (Section 5.4, bold header) Validated in Table 3 row (D), page 9: varying Pdrop changes BLEU on newstest2013 (4.92 PPL / 25.8 BLEU at the base 0.1 value). 2. Attention Dropout -- Pdrop = 0.1, page 8 (same paragraph as Residual Dropout) Same Table 3 row (D) covers it: Pdrop is applied uniformly. 3. Label Smoothing -- epsilon_ls = 0.1, page 8 (Section 5.4, bold header) Validated in Table 3 row (E), page 9: varying epsilon_ls changes BLEU. The text notes this "hurts perplexity but improves accuracy and BLEU". Cardinality: Section 5.4 opens with "We employ three types of regularization during training" (page 7). All three are present in this list.` `complete_answer_found=True`, `pending_references=[]`. The orchestrator stops. The answer is structured (3 enumerated techniques), exhaustive (cardinality cue confirmed), and complete (each technique has both its definition page and the Table 3 row that varies it). ### 5.5 Cost breakdown Indicative cost shape for a 15-page native PDF on a typical OpenAI-class model. The numbers are rough; what matters is the **shape** : the LLM calls dominate, everything else is sub-second. * Cheap parsing (PyMuPDF on 15 pages): well under one second. * Document profile (one `fitz.open` + TOC read): ~50 ms. * Combined retrieval (TOC + keyword + RRF fusion): a few hundred ms. * Listing aggregation + cardinality check: a few hundred ms. * Two-hop reference resolution (one regex sweep + a re-retrieve on page 9): under one second. * Two generation calls (gpt-4-class): ~2 to 4 seconds each. * **Total: typically 5 to 9 seconds** for a question that activates four patterns and loops once on a reference. The two generation calls drive almost all of it. Without the dispatcher, you would either pay for every pattern on every call, or hand-code the routing for every question shape. Dense retrieval and adaptive parsing stayed OFF here because the profile said they were not needed; that decision sits in `decide.py`, not in the question itself. ## 6. Dispatched RAG vs autonomous agents Section 3 noted that the feedback-loop shape is what most vendors call **agentic RAG**. The term is used three different ways, and choosing between them shapes the whole architecture. **Usage 1: agentic as marketing:** Any pipeline more sophisticated than `embed → retrieve → generate` gets called agentic. The label is empty. We can ignore this usage. **Usage 2: feedback-driven control:** The LLM produces signal (`complete_answer_found`, `llm_discovered_keywords`, `context_structured`), and deterministic Python code reacts to that signal. This is what we’ve built. The LLM is _in the system_ but not _in control of the system_. Control stays in the dispatcher. **Usage 3: autonomous agents:** The LLM has a set of tools and decides at each step which one to call next. The control loop is in the LLM itself. The Python code executes whatever the LLM requests. The distinction between Usage 2 and Usage 3 is the architectural decision that matters most for enterprise RAG. ### 6.1 What an autonomous agent looks like In Usage 3, the orchestrator above would be replaced by something like: python ``` # Autonomous agent - what we are NOT building tools = [retrieve_by_toc, retrieve_by_keywords, retrieve_dense, re_parse_with_camelot, follow_reference, ask_clarification] state = {"question": question, "history": []} while not is_satisfied(state): next_action = llm.choose_next_tool(state, tools) result = execute(next_action) state["history"].append((next_action, result)) ``` The LLM decides which tool to call, with which arguments, in which order. The pipeline is generated at runtime by the LLM, not authored by the engineer. This is appealing in theory: the LLM can combine tools in ways the engineer didn’t anticipate. In practice, for enterprise RAG, three things go wrong. **Reproducibility breaks:** The same question on the same document can take different paths on different runs. For audit, compliance, legal review, this is disqualifying. _“Why did the system retrieve this passage?”_ has no single answer when the LLM chose the path freely. **Costs explode:** Each tool decision is an LLM call. Five to ten tool calls per question. At realistic scale, the cost is one to two orders of magnitude higher than the dispatcher pattern. For a system answering thousands of questions per day, this is the difference between affordable and impossible. **Debugging becomes guesswork:** When an autonomous agent produces a bad answer, you have to read its reasoning trace to understand what went wrong. The trace is long, branchy, and not always coherent. Compared to the dispatcher pattern, where you can pinpoint exactly which activation rule fired or didn’t, autonomy is a step backward in maintainability. ### 6.2 What we keep from “agentic” The valuable part of the agentic idea isn’t autonomy. It’s that **the pipeline thinks about the question before searching, and reacts to its own outputs**. Both of those are present in our composite pipeline: * The pipeline thinks about the question (Article 6, question understanding). * The pipeline reacts to its own outputs through feedback fields (Article 8, generation feedback). * The pipeline iterates when needed (section 3 above, bounded by the orchestrator). What we don’t keep is the autonomy of the LLM in choosing tool sequences. That decision belongs to the dispatcher, in code, where it’s testable, auditable, and reproducible. A more honest name is **structured RAG** or **dispatched RAG**. “Agentic” survives because the field has settled on it, but the architecture stays in code, not in the LLM. ### 6.3 When autonomy is the right choice Autonomous agents have their place. They’re the right pattern when: * The set of tools is open and changes frequently. A research assistant that needs to combine retrieval, web search, code execution, calculator, etc., across very different question types. * The question space is too varied to dispatch deterministically. _“Help me plan this trip”_ legitimately needs different combinations of tools each time. * Reproducibility isn’t a hard requirement. Consumer applications, exploratory tooling, internal R&D. For enterprise RAG on documents, which is what this series is about, the structure of the problem is well-understood. We know what a document looks like, what kinds of questions get asked, what kinds of patterns help. The dispatcher pattern captures that knowledge. Autonomy adds cost and unpredictability without adding capability. ### 6.4 The team-level implication Dispatched RAG has a property that autonomous RAG doesn’t: **the team’s understanding of the problem is encoded in the dispatcher**. When a new engineer joins, they read `decide.py` and learn how the team thinks about question types and pattern activations. When a question fails in production, the team adds a test case and updates the dispatcher. When a new pattern is added (say, table-comparison retrieval for cross-document questions), it gets a new activation rule. An unaudited autonomous agent has no such artifact: its behavior emerges from the LLM’s reasoning at runtime, which doesn’t accumulate, doesn’t get reviewed, and doesn’t transfer between team members. This is part of why the dispatcher pattern has held up in production: the dispatcher is something a team can own, review, and hand off. The layer that sits on top, where the agent picks among patterns like the ones dispatched here, is follow-up work. ## 7. Conclusion The architectural commitment that holds the four bricks together is one sentence: decisions live in `decide.py`, not in an LLM prompt at runtime. The orchestrator runs the bricks in order, the feedback-loop machinery turns generation’s structured output into a bounded retry decision, and the dispatcher picks the patterns from the parsed question and the document profile. The remaining articles address what surrounds the pipeline rather than what is inside: scaling from one document to a corpus (Part IV), and evaluating, running, storing, and securing the system in production (Part V). ## 8. Sources and further reading The workflow-versus-agent debate was named by Anthropic in [Building Effective Agents](https://www.anthropic.com/research/building-effective-agents) (Dec 2024). The dispatcher in this article is a workflow in that sense. The reflection-token idea from Asai et al. ([Self-RAG](https://arxiv.org/abs/2310.11511), ICLR 2024) shows up directly in the structured feedback fields the orchestrator reads. The agentic side, with the patterns dispatched here as the agent’s audited toolkit, is follow-up work. **Earlier in the series:** * [Document Intelligence: series intro](https://towardsdatascience.com/document-intelligence-a-series-on-building-rag-brick-by-brick-from-minimal-to-corpus-scale/). What the series builds, brick by brick, and in what order. _What works, what breaks_ * [Baseline Enterprise RAG, from PDF to highlighted answer](https://towardsdatascience.com/baseline-enterprise-rag-from-pdf-to-highlighted-answer-enterprise-document-intelligence-vol-1-1/). The four-brick pipeline end to end: PDF in, highlighted answer out. * [Embeddings Aren’t Magic: The Predictable Failure Modes of RAG Retrieval](https://towardsdatascience.com/embeddings-arent-magic-the-predictable-failure-modes-of-rag-retrieval-enterprise-document-intelligence-vol-1-2/). Where embedding similarity wins (synonyms, typos, paraphrase), where it predictably breaks (unknown terms, negation, term-vs-answer relevance), and how to use it anyway. * Rerankers Aren’t Magic Either: When the Cross-Encoder Layer Is Worth the Cost. What a cross-encoder adds over bi-encoder embeddings, measured, and when it is worth the latency. * [RAG is not machine learning, and the ML toolkit solves the wrong problem](https://towardsdatascience.com/rag-is-not-machine-learning-and-the-ml-toolkit-solves-the-wrong-problem/). Why chunk-size sweeps and finetuning optimize the wrong thing; route by question type instead. * From regex to vision models: which RAG technique fits which problem. Two axes, document complexity and question control, that pick the technique for each case. * [10 common RAG mistakes we keep seeing in production](https://towardsdatascience.com/10-common-rag-mistakes-we-keep-seeing-in-production/). Ten production mistakes, organized brick by brick, with the fix for each. _Document parsing_ * [Beyond extract_text: the two layers of a PDF that drive RAG quality](https://towardsdatascience.com/beyond-extract_text-the-two-layers-of-a-pdf-that-drive-rag-quality/). The first half of the parsing brick: the document’s nature, signals, and summary. * [Stop returning flat text from a PDF: the relational tables RAG needs](https://towardsdatascience.com/stop-returning-flat-text-from-a-pdf-the-relational-shape-rag-needs/). The second half of the parsing brick: the relational tables every downstream brick reads. * [When PyMuPDF can’t see the table: parse PDFs for RAG with Azure Layout](https://towardsdatascience.com/when-pymupdf-cant-see-the-table-parse-pdfs-for-rag-with-azure-layout/). The same tables from Azure Layout: native table cells, OCR, paragraph roles. * [Parse PDFs for RAG locally with Docling: rich tables, no cloud upload](https://towardsdatascience.com/parse-pdfs-for-rag-locally-with-docling-rich-tables-no-cloud-upload/). The same tables computed locally with Docling: TableFormer cells, nothing leaves the machine. * [Vision LLMs are PDF parsers too: reading charts and diagrams for RAG](https://towardsdatascience.com/vision-llms-are-pdf-parsers-too-reading-charts-and-diagrams-for-rag/). Vision as a parser: the pictures become searchable text. * Parse scanned PDFs for RAG with EasyOCR: free OCR gives you words, not a document. Where traditional OCR stops: text recovered, structure lost. * [Making a PDF’s images searchable for RAG, without paying to read them all](https://towardsdatascience.com/making-a-pdfs-images-searchable-for-rag-without-paying-to-read-them-all/). The image cascade: filter cheap, classify, describe only what is worth reading. * Reconstructing the table of contents a PDF forgot to ship, so RAG can scope by section. Rebuilding toc_df when the PDF prints a contents page but has no outline. _Question parsing_ * RAG questions need parsing too: turn the user’s string into briefs for retrieval and generation. The thesis of question parsing: why a user string needs the same parsing as a document, and how it splits into a retrieval brief and a generation brief. * What the question parser extracts from a user string: keywords, scope, shape, decomposition, clarification. The five families of columns the parser reads straight from the user’s question, with the code that fills each one. * Dispatching the parsed RAG question: chunk strategy, model tier, activations, audit. The decisions the parser makes on top of the user string, using the document’s profile: dispatch, activations, full schema, the audit trail (pipeline_trace.json), and a broker-corpus walkthrough. * The Clarification Loop and Learned Defaults: When the Question Is Not Precise Enough. One focused clarification when the question is too vague, and the default learned from the answer. * [Context engineering for RAG question parsing: from a raw question to typed fields that steer retrieval and generation](https://towardsdatascience.com/context-engineering-for-rag-question-parsing-from-a-raw-question-to-typed-fields-that-steer-retrieval-and-generation/). What enters the question-parse call and the typed fields that come out, steering retrieval and generation. _Retrieval_ * Retrieval is filtering, not search: a mental model for enterprise RAG. Retrieval reframed as filtering on line_df and toc_df: anchors small, context large. * Anchor detection for RAG: parallel detectors, then one LLM call at the end. Parallel anchor detectors: keyword always, embeddings alongside, one LLM call at the end. * Letting an LLM pick the right RAG page: the arbiter pattern at the end of retrieval. The LLM arbiter: candidates ranked with reasons, one typed JSON out. * [Context Engineering: The Four Typed Inputs Behind Every Answer](https://towardsdatascience.com/context-engineering-for-rag-the-four-typed-inputs-behind-every-rag-answer/). Context engineering given a structure: the four typed pieces (fixed system prompt, retrieved lines, doc-context block, PromptContext wrapper) that fill one single-document RAG LLM call. _Generation_ * [Stop returning text from RAG: the typed answer contract that prevents hallucination](https://towardsdatascience.com/stop-returning-text-from-rag-the-typed-answer-contract-that-prevents-hallucination/). The answer schema as the contract: typed values, items with evidence spans, self-assessment fields, and the completeness signal the pipeline computes itself. * Assemble each RAG generation prompt from a base prompt plus the rules each question needs. The dispatcher: a fixed BASE prompt plus the rules each question needs, the schema picked from the registry, and the full trace kept on every call. * [Validating the RAG answer before the user sees it: spans, quotes, and the feedback loop](https://towardsdatascience.com/validating-the-rag-answer-before-the-user-sees-it-spans-quotes-and-the-feedback-loop/). The post-generation validator (spans, verbatim quotes, formats), not-found as a first-class answer, and the feedback loops that close the pipeline. _One-document pipelines_ * [A production RAG pipeline for PDFs: relational parsing, TOC retrieval, typed answers](https://towardsdatascience.com/a-production-rag-pipeline-for-pdfs-relational-parsing-toc-retrieval-typed-answers/). Each of the four bricks upgraded one contract at a time: relational parsing, corpus-aware questions, TOC-routed retrieval, typed answers. * One RAG pipeline, four very different PDFs: same four bricks, every answer typed and cited. The four upgraded bricks wired into one call, run end to end on a paper, a compliance doc, and a broken-TOC document. * [Loop engineering with adaptive PDF parsing: start cheap, pay for a heavier parser only when the page needs it](https://towardsdatascience.com/loop-engineering-with-adaptive-pdf-parsing-start-cheap-pay-for-a-heavier-parser-only-when-the-page-needs-it/). The escalation cascade and the free deterministic checks that flag a failed parse before you pay for a deeper one. * [Loop engineering with adaptive parsing in action: flattened tables to Azure, figures to a vision LLM](https://towardsdatascience.com/loop-engineering-with-adaptive-parsing-in-action-parsing-flat-tables-with-azure-and-figures-with-a-vision-llm/). The LLM as last line of defence, then two real escalations: a flat table to Azure, a figure to a vision model. * _Loop engineering for cross-references: when RAG answers ‘see Section 7.2’ instead of the actual answer_ (link to come). When the answer says “see Section X”, the pipeline loops back and fetches it. * _Loop engineering for listing questions: when the answer is every passage, not the top one_ (link to come). Listing questions: the answer is all the passages, not one, and the aggregation that finds them. ## Register Now to Continue Reading ### Unlock Towards Data Science with a Free Account All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account * * * ![](../images/body-249d024535392fb76067.jpeg) Written By ## [Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/) [See all from Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/) [Deep Dives](https://towardsdatascience.com/tag/deep-dives/)[Llm](https://towardsdatascience.com/tag/llm/)[Rag](https://towardsdatascience.com/tag/rag/)[Rag Architecture](https://towardsdatascience.com/tag/rag-architecture/)[loop engineering](https://towardsdatascience.com/tag/loop-engineering/) Share This Article * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Frag-workflow-and-loop-engineering-the-dispatcher-that-decides-when-to-loop-and-when-to-stop%2F&title=RAG%20Workflow%20and%20Loop%20Engineering%3A%20The%20Dispatcher%20That%20Decides%20When%20to%20Loop%20and%20When%20to%20Stop) * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Frag-workflow-and-loop-engineering-the-dispatcher-that-decides-when-to-loop-and-when-to-stop%2F&title=RAG%20Workflow%20and%20Loop%20Engineering%3A%20The%20Dispatcher%20That%20Decides%20When%20to%20Loop%20and%20When%20to%20Stop) * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Frag-workflow-and-loop-engineering-the-dispatcher-that-decides-when-to-loop-and-when-to-stop%2F&text=RAG%20Workflow%20and%20Loop%20Engineering%3A%20The%20Dispatcher%20That%20Decides%20When%20to%20Loop%20and%20When%20to%20Stop) Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. [Write for TDS](https://towardsdatascience.com/submissions/) ## Related Articles * [![](../images/002.jpg)](https://towardsdatascience.com/six-degrees-of-sql/) ## [Recursive CTEs: SQL’s Hidden Graph Traversal Engine](https://towardsdatascience.com/six-degrees-of-sql/) [Data Science](https://towardsdatascience.com/category/data-science/) A practical guide to navigate hierarchies, find routes, detect cycles and calculate degrees of separation [Thomas Reid](https://towardsdatascience.com/author/thomas_reid/) August 25, 202615 min read * [![](../images/003.jpg)](https://towardsdatascience.com/the-untaught-lessons-of-rag-retrieval-cosine-is-not-the-foundation/) ## [6 common RAG retrieval mistakes we keep seeing in production](https://towardsdatascience.com/the-untaught-lessons-of-rag-retrieval-cosine-is-not-the-foundation/) [Large Language Model](https://towardsdatascience.com/category/large-language-model/) Enterprise Document Intelligence [Vol.1 #7ter] - Six positions on the retrieval brick that contradict the cosine-first reflex of mainstream RAG [Kezhan Shi](https://towardsdatascience.com/author/kezhanshi/) July 3, 20268 min read * [![](../images/004.png)](https://towardsdatascience.com/chat-with-your-images-using-multimodal-llms-60af003e8bfa/) ## [Chat with Your Images using Multimodal LLMs](https://towardsdatascience.com/chat-with-your-images-using-multimodal-llms-60af003e8bfa/) [](https://towardsdatascience.com/rag-workflow-and-loop-engineering-the-dispatcher-that-decides-when-to-loop-and-when-to-stop/) Learn how to build Llama 3.2-Vision locally in a chat-like mode, and explore its Multimodal skills on a Colab notebook [Lihi Gur Arie, PhD](https://towardsdatascience.com/author/lihigurarie/) December 5, 20247 min read * [![](../images/005.png)](https://towardsdatascience.com/using-vector-steering-to-improve-model-guidance-9cca64635510/) ## [Using Vector Steering to Improve Model Guidance](https://towardsdatascience.com/using-vector-steering-to-improve-model-guidance-9cca64635510/) [](https://towardsdatascience.com/rag-workflow-and-loop-engineering-the-dispatcher-that-decides-when-to-loop-and-when-to-stop/) Exploring the Research on Vector Steering and Coding Up an Implementation [Matthew Gunton](https://towardsdatascience.com/author/mgunton7/) October 21, 20249 min read * [![](../images/006.jpg)](https://towardsdatascience.com/speak-to-me-how-many-words-a-model-is-reading-331e3af86d27/) ## [Speak to me: How many words a model is reading](https://towardsdatascience.com/speak-to-me-how-many-words-a-model-is-reading-331e3af86d27/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) Why and how to overcome the inner limit of a Large Language Model [Salvatore Raieli](https://towardsdatascience.com/author/salvatore-raieli/) July 14, 202319 min read * [![](../images/007.jpg)](https://towardsdatascience.com/proxy-pointer-rag-structure-meets-scale-100-accuracy-with-smarter-retrieval/) ## [Proxy-Pointer RAG: Structure Meets Scale at 100% Accuracy with Smarter Retrieval](https://towardsdatascience.com/proxy-pointer-rag-structure-meets-scale-100-accuracy-with-smarter-retrieval/) [Large Language Model](https://towardsdatascience.com/category/large-language-model/) Open source. 5-minute setup. Vector RAG done right—try it yourself. [Partha Sarkar](https://towardsdatascience.com/author/partha_sarkar/) April 19, 202613 min read * [![](../images/008.jpg)](https://towardsdatascience.com/loop-engineering-for-cross-references-when-rag-answers-see-section-7-2-instead-of-the-actual-answer/) ## [Loop Engineering for Cross-References: When RAG Answers ‘see Section 7.2’ Instead of the Actual Answer](https://towardsdatascience.com/loop-engineering-for-cross-references-when-rag-answers-see-section-7-2-instead-of-the-actual-answer/) [Large Language Model](https://towardsdatascience.com/category/large-language-model/) Enterprise Document Intelligence [Vol.1 #11] - When the first answer points elsewhere in the document, the pipeline loops back to fetch the linked context [Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/) August 6, 202620 min read * [![](../images/009.png)](https://towardsdatascience.com/bit-lora-as-an-application-of-bitnet-and-1-58-bit-neural-network-technologies-17ee80bf79f9/) ## [Bit-LoRA as an application of BitNet and 1.58 bit neural network technologies](https://towardsdatascience.com/bit-lora-as-an-application-of-bitnet-and-1-58-bit-neural-network-technologies-17ee80bf79f9/) [](https://towardsdatascience.com/rag-workflow-and-loop-engineering-the-dispatcher-that-decides-when-to-loop-and-when-to-stop/) Abstract: applying ~1bit transformer technology to LoRA adapters allows us to reach comparable performance with full-precision LoRA... [Roman Smirnov](https://towardsdatascience.com/author/r.smirnov.mailbox/) June 2, 202414 min read * [![](../images/010.jpg)](https://towardsdatascience.com/a-comprehensive-study-of-mixed-integer-programming-with-jump-on-julia-part-1-8d47418324d4/) ## [A comprehensive study of Mixed Integer Programming with JuMP on Julia (Part 1)](https://towardsdatascience.com/a-comprehensive-study-of-mixed-integer-programming-with-jump-on-julia-part-1-8d47418324d4/) [](https://towardsdatascience.com/rag-workflow-and-loop-engineering-the-dispatcher-that-decides-when-to-loop-and-when-to-stop/) Some basics of Linear/Mixed Integer Programming & How to use a heuristic callback inside a MIP solver. [Ouaguenouni Mohamed](https://towardsdatascience.com/author/ouaguenouni-hachemi/) March 29, 202114 min read * [YouTube](https://www.youtube.com/c/TowardsDataScience) * [X](https://x.com/TDataScience) * [LinkedIn](https://www.linkedin.com/company/towards-data-science/) * [Threads](https://www.threads.net/@towardsdatascience) * [Bluesky](https://bsky.app/profile/towardsdatascience.com) [](https://towardsdatascience.com/) Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/) © Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network) #### Recommended Articles Close * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607631/image?etag=1781543525510&width=500&height=500) Critical Mistakes Companies Make When Integrating AI/ML into Their Processes ](https://towardsdatascience.com/critical-mistakes-companies-make-when-integrating-ai-ml-into-their-processes/ "Critical Mistakes Companies Make When Integrating AI/ML into Their Processes") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ "How Relevance Models Foreshadowed Transformers for NLP") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ "Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ "The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs")