保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Large Language Model

RAG Workflow and Loop Engineering: The Dispatcher That Decides When to Loop and When to Stop

Enterprise Document Intelligence [Vol.1 #13] - Putting the patterns together, and why this is what “agentic RAG” should look like Angela and Kezhan Shi August 14, 202627 min read Photo by Fernando Narvaez, via Pexels. Over the last few articles, we built loop patterns one at a time: a re-parse when a page check fails, a second retrieval when the answer points to another section, an aggregation sweep when the question asks for a complete list. Each pattern has its own article, its own trigger, and its own test. Each one works fine on its own. Real questions do not arrive one pattern at a time. Take one a compliance officer would actually ask on the NIST Cybersecurity Framework (US Government work, public domain in the US, see NIST copyright statement): “What are all the Categories under GOVERN, and which one covers supply chain risk?” The question sounds ordinary. Inside the pipeline, it fires three patterns at once:

Each of the three carries its own iteration mechanic: a re-retrieval here, a re-generation there, an LLM flag that triggers a second pass. Run them on the same question and a practical problem shows up: which one decides when to stop? Leave that undecided and every new kind of question becomes another special case bolted on the side, and nobody can say what the pipeline will do next. The fashionable answer today is to hand that decision to an agent and let the model orchestrate. For an enterprise pipeline, we prefer a piece of code we can read: a dispatcher that turns the parsed question and the document profile into an explicit plan, and bounded loops that say, in code, how far each pattern may iterate. That is what this article builds: the feedback loops, the bounded iteration, and the dispatcher that composes them into one workflow. Learn this step by step with the interactive AI Engineer roadmap. This article closes Part III of Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks: document parsing, question parsing, retrieval, and generation. 🧭 New to the series? Start with the map : Prompt, Context, Loop sets out the three engineering layers every RAG system is built on, the prompt (the call itself), the context (what fills the model’s window), the loop (when the next call fires and when it stops), and walks the whole series through that lens, article by article. It is the shortest way to see what is covered and where this one sits. where this article sits in the series: Article 13 (the workflow pipeline), closing Part III - Image by author 📓 The runnable companion drives the loop machinery yourself: you run pdf_qa_loop on a question that fails its first pass, print the IterationRecord history showing what each retry changed, and watch should_continue cut the loop when the candidates stop moving. On GitHub: doc-intel/notebooks-vol1. The public companion-code repo at doc-intel/notebooks-vol1 - Image by author In production, real questions stack the patterns: “List the obligations of the seller, including any referenced standards” on a contract adds two-hop reference resolution on top of listing. The iteration mechanics ride alongside them. It helps to place this article on a five-rung climb, each rung a more capable version of the same PDF question-answering function. The baseline (Article 1) chains the four bricks once with keyword retrieval: one pass, return the answer. The upgraded version (Article 9, the four bricks upgraded contract by contract) keeps the single pass but makes it richer, with a full relational parse and TOC routing, and it returns a typed answer carrying feedback fields (is the context complete? did the parse hold?), produced but not yet acted on. The workflow rung, this article, turns that pass into one step inside a bounded loop: a dispatcher picks which patterns fire, and the loop reads the feedback fields to decide whether to retry, with the control staying in code. The multi-intent rung (follow-up work) widens the entrance: a chat entry classifies what the user wants (a question, a translation, a summary, or just “hello” , which deserves a direct reply and no pipeline at all) and routes to the right pipeline, the decision still in code. The agentic rung (further out) moves the control loop into the LLM, which picks the next step itself; this article stops two rungs below, where the pipeline is still reproducible and auditable. Rungs one to four keep control in code; only the last jump moves who holds the loop from code to model. Five levels of control, control in code until the last rung; this article is the workflow rung - Image by author This article is about the composite pipeline : a single orchestrator function that takes a question, decides which patterns to activate, runs them, manages the feedback loops, and produces an answer. The function is **pdf_qa_loop**: the single pass from the upgraded pipeline, grown one rung, dispatching across Part III’s patterns within(pdf, qa). Follow-up work adds the entry points above it (other document formats, other intents); that scope sits outside this article. The patterns become a toolkit. pdf_qa_loop becomes their orchestrator. Four patterns ON, three OFF; only ON patterns reach the Answer - Image by author The dispatcher is where “amplify the expert” lands at the composition layer. The team’s routing wisdom (which patterns fire for which question types, in which order) gets written down once, in deterministic Python, and runs across every future question. The LLM appears at every brick. It never chooses the next call. Pattern selection is the dispatcher’s job; the iterate-or-stop decision is the loop machinery’s. Both stay in code, reviewable by the people whose judgment they codify.

1. One layer composes, the bricks stay apart

A workflow like this needs exactly one new home in the codebase: the composition layer. It holds the pieces that exist only because bricks are being combined: the dispatcher that picks which patterns fire, the feedback machinery that reads the typed flags, and one flow per intent. Nothing else moves. The four bricks keep living in their own modules, and the patterns of Part III stay where they belong as sub-functions of their brick: TOC retrieval in the retrieval module, escalation in parsing, two-hop resolution across retrieval and generation. The two layers talk through typed objects (a ParsedQuestion, a DocumentProfile, an AnswerWithEvidence), never through each other’s internals. One way to lay that out on disk, the one this series uses: a pipeline/ folder sitting next to the brick modules: _pipeline/_holds the composition layer; the bricks themselves live next door - Image by author Inside the composition layer, code is organised by intent (one subpackage per intent, one file per format inside), with the cross-intent infrastructure flat at the root; Article 18 (code architecture) develops the file layout in detail. This split makes the system maintainable. When you improve TOC retrieval, you change the retrieval module only. When you fix a bug in two-hop resolution, the orchestrator stays untouched. The contract between orchestrator and bricks is the interface (a ParsedQuestion, a DocumentProfile, an Activations dict, an AnswerWithEvidence), not the implementation. A senior engineer can trace any request from input to highlighted answer in a few minutes: pdf_qa_loop (pipeline/qa/pdf_flow.py) reads the question, calls decide_pipeline_patterns for activations, runs the four-brick pass (the same parse and retrieval entry points as Article 9’s pdf_qa) with those activations, reads the generation feedback fields, and either returns the answer or queues a loop. No hidden state, no agent loop. The file layout has a runtime twin: how the handlers nest when a request actually runs. Each layer calls the one inside it and adds exactly one concern, and the control loop stays in code at every level. Each ring calls the one inside and adds one concern, with the control loop always in code - Image by author

2. The orchestrator

The orchestrator does three things, in order: 1. It reads the parsed question (ParsedQuestion from Article 6) and decides which patterns to activate. Some questions need all of them. Most don’t. 2. It runs the active patterns , passing the right inputs to each. The order matters: TOC retrieval before keyword retrieval (so structural anchors come first), primary retrieval before two-hop (so the two-hop step has something to follow from), and so on. 3. It manages feedback loops. When the generation step signals an incomplete answer, the orchestrator decides whether to iterate and how many times. Multiple loops can fire on one question (parsing escalation, vocabulary expansion, reference expansion, listing iteration). The orchestrator picks the order across iterations (parsing first, vocabulary next, references next, listing last) and shares one budget across all of them. Two to three iterations cover the vast majority of cases; the composite’s hard cap is four, because several loops can fire on one question and each needs its own pass. Order matters because re-retrieving on bad parsing wastes the iteration; better keywords improve every later step. Concretely: python

# pdf_qa_loop, condensed
# decide_pipeline_patterns and iterate_with_bound come from the companion notebook.

def pdf_qa_loop(pdf_path, question, *, registry=None, max_iterations=4):
    registry = registry or PatternRegistry.default()
    # 1. Question understanding
    parsed = registry.parse_question(question)
    # 2. Cheap layer-1 parse of the document
    line_df, page_df, toc_df = registry.parse_layer1(pdf_path)
    # 3. Document profile
    doc_profile = registry.detect_document_type(pdf_path)
    # 4. Dispatcher : which patterns fire ?
    activations = decide_pipeline_patterns(parsed, doc_profile)
    # 5-7. Retrieval + generation inside a bounded feedback loop
    state = _State(parsed=parsed, line_df=line_df,
                   page_df=page_df, toc_df=toc_df)
    outcome = iterate_with_bound(
        initial_state=state,
        run_pass=lambda s: _one_pass(s, registry, activations, pdf_path),
        is_satisfactory=lambda a: not _needs_iteration(a),
        adjust=lambda s, a: _adjust_state(s, a, registry,
                                          activations, pdf_path),
        record=_build_record,
        max_iterations=max_iterations,
    )
    return CompositeOutput(answer=outcome.result, activations=activations,
                           history=outcome.history,
                           exhausted=outcome.exhausted)

That’s the whole orchestrator. The pdf_qa_loop body is a page of glue code sitting on top of the per-pattern modules. The third bullet (feedback loops) has its own dedicated machinery; section 3 walks through it. The first two bullets (pattern selection, run order) are deterministic once decide_pipeline_patterns returns its column of ON/OFF flags; section 4 walks through the dispatcher itself.

3. The feedback loop pattern

The composite pipeline’s distinguishing feature, compared to a naive RAG, is that the answer is treated as a provisional output that the system itself critiques. The loop machinery sits between generation and the final return.

3.1 From naive RAG to feedback-driven RAG

A naive RAG pipeline runs the four bricks in sequence and returns whatever comes out: parse the document, parse the question, retrieve, generate. The four bricks chained once; parsed_q reaches Retrieval and Generation, no critique step - Image by author If retrieval missed the right passage, the answer is wrong. If parsing was insufficient, the LLM hallucinates around the gap. The user never knows. The composite pipeline adds a critique step at the end of the row, with two feedback rails that loop back whenever the LLM signals a problem: back into retrieval when the answer is incomplete (complete_answer_found=false, expand the keywords and re-retrieve), and back into parsing when the context came out unstructured (context_structured=false, re-parse the flagged pages): Same four bricks plus Critique; two rails re-inject on a trigger, the green arrow exits on convergence - Image by author The critique step reads the structured feedback fields the LLM produced (Article 8): is the context complete? did the LLM discover new keywords during reading? was the parsing sufficient? If anything is off, the pipeline takes targeted action and re-generates. These rails are the pipeline’s big loops: the ones that cross bricks, triggered by generation reading its own input (adaptive parsing in Article 10, reference resolution in Article 11, scope feedback here). Each brick also runs its own small bounded loops inside (the image cascade in parsing, the TOC descent in retrieval, the schema retry in generation); those never reach the orchestrator. The dispatcher owns the big ones only. This is what most vendors now market as agentic RAG. The term covers three quite different things; section 6 sorts them out.

3.2 Signal, trigger, action

Each loop has three dimensions: the signal (what the LLM or a programmatic check reports), the trigger (which state of the output starts a retry), and the action (what the pipeline changes for the next pass). Together with the bounds of section 3.3, these are the three control surfaces every loop in the series wears: signal plus trigger decide when a new pass fires, the action is the recovery (the one thing that changes before the retry), and the bounds are the termination. Signals come from three sources :

Triggers map to specific actions (each was introduced once in Part II or III; here is the consolidated catalogue):

Each trigger maps to a named, targeted action, which keeps the loop debuggable.

3.3 Bounding the loops

The main risk of iterative pipelines is unbounded iteration. Three controls. Maximum iterations: A ceiling on pdf_qa_loop’s max_iterations parameter. Two or three iterations cover the vast majority of cases. Beyond that, returns diminish sharply. python

def iterate_with_bound(
    *,
    initial_state: TState,
    run_pass: Callable[[TState], TResult],
    is_satisfactory: Callable[[TResult], bool],
    adjust: Callable[[TState, TResult], TState],
    record: Callable[[int, TResult, TState, TResult | None], IterationRecord],
    max_iterations: int = 3,
) -> IterationOutcome[TState, TResult]:
    """Run a bounded pipeline-with-feedback loop."""
    state = initial_state
    history: list[IterationRecord] = []
    result = run_pass(state)
    previous_result: TResult | None = None
    if is_satisfactory(result):
        history.append(record(0, result, state, previous_result))
        return IterationOutcome(result=result, history=history, exhausted=False)
    for i in range(1, max_iterations):
        next_state = adjust(state, result)
        history.append(record(i, result, next_state, previous_result))
        previous_result = result
        state = next_state
        result = run_pass(state)
        if is_satisfactory(result):
            return IterationOutcome(result=result, history=history, exhausted=False)
    return IterationOutcome(result=result, history=history, exhausted=True)

A loop that hits max_iterations doesn’t crash. It returns the best output produced so far, flagged so the user and the audit trail know iteration was exhausted. Termination conditions: Beyond max iterations, the loop stops when:

python

def should_continue(history: list, current: _ResultProto,
                    confidence_drop_threshold: float = 0.1) -> bool:
    """Decide whether the iteration loop should run another pass.

    Three reasons to stop, in order of priority:
      1. Candidates are stable (same set as last pass): nothing new will be found.
      2. Suggested keywords are stable: the LLM is repeating itself.
      3. Confidence is decreasing past the threshold: the loop is making things worse.
    """
    if history:
        prev = history[-1]
        if getattr(prev, "candidates", None) == current.candidates:
            return False
        if getattr(prev, "suggested_keywords", None) == current.suggested_keywords:
            return False
        prev_conf = getattr(prev, "confidence", None)
        if prev_conf is not None and current.confidence < prev_conf - confidence_drop_threshold:
            return False
    return current.needs_iteration()

Without these, even with a max-iterations bound, the pipeline burns three full iterations on cases where it had clearly stopped progressing after the first. Drift detection: A subtler failure: the loop iterates successfully (each pass finds new candidates) but the answer drifts away from the original question. Keyword expansion picked up a tangent. The user asked about “premium” , the LLM saw “insurance” , expanded to “policy” , then to “coverage” , then to “reinsurance” , and the answer ends up about reinsurance economics. The fix is to keep the original question in scope at every iteration. Expanded keywords add to the retrieval, they don’t replace the original anchors. Generation always sees the original question. python

def expand_query_safely(parsed, new_keywords: list[str],
                        max_keywords: int = 15):
    try:
        expanded = parsed.model_copy(deep=True)
    except AttributeError:
        import copy as _copy
        expanded = _copy.deepcopy(parsed)

    originals = list(expanded.retrieval.anchor_keywords or [])
    additions = [k for k in new_keywords if k not in originals]
    combined = originals + additions
    if len(combined) > max_keywords:
        kept_originals = originals[:max_keywords]
        room = max_keywords - len(kept_originals)
        combined = kept_originals + additions[-room:] if room > 0 else kept_originals
    expanded.retrieval.anchor_keywords = combined
    return expanded

A drift check at each iteration: if the new candidates have low overlap with the original anchor keywords, that’s drift, and the loop should stop.

3.4 Anti-patterns and audit trail

Five anti-patterns to keep out of the orchestrator:

json

{
  "iteration_number": 2,
  "trigger": "vocabulary_gap",
  "action_taken": "re-retrieve with `excess`",
  "result_summary": "page 7 joined the candidate set; confidence rose by 0.22",
  "confidence_before": 0.62,
  "confidence_after": 0.84
}

The full iteration history travels with the final answer. For audit and compliance, this is the difference between “the system gave this answer” and “the system gave this answer through these specific steps, each justified by an explicit trigger.” The UI presents a one-line narration per iteration so the user sees the system making informed choices rather than running blind retries.

4. The dispatcher

The dispatcher (decide.py) is where most of the explicit decisions happen. It takes the parsed question and the document profile, and returns a dictionary of pattern activations. python

def decide_pipeline_patterns(
    parsed: ParsedQuestion,
    doc_profile: DocumentProfile,
) -> Activations:
    activations = dict(DEFAULT_ACTIVATIONS)

    # TOC retrieval: enable whenever the document has a usable TOC.
    if doc_profile.has_usable_toc:
        activations["toc_retrieval"] = True

    # Dense retrieval (embeddings): only as a fallback for questions whose
    # vocabulary may not match the document's wording.
    if parsed.intent in ("open_scoped", "open_corpus_wide"):
        activations["dense_retrieval"] = True

    # Two-hop references: enable when the question carries a structural
    # hint that points outside its primary passage.
    if parsed.retrieval.section_hint or parsed.retrieval.layout_hint:
        activations["two_hop_references"] = True
    if parsed.intent in ("section_retrieval", "open_scoped"):
        activations["two_hop_references"] = True

    # Listing aggregation: enable when the question intent is listing.
    if parsed.intent == "listing":
        activations["listing_aggregation"] = True

    return activations

This function encodes the team’s understanding of which questions need what. As the system runs in production, it evolves: new question types, new heuristics, edge cases observed in evaluation feed back into the dispatcher. The dispatcher is also the file the team should iterate on with care. A bad activation rule can either over-engineer simple questions (running listing aggregation when not needed) or under-engineer complex ones (skipping two-hop on questions that need it). Each rule should have a test case attached.

5. Worked example: a listing-plus-references question on the Transformer paper

The previous articles have looked at NIST CSF listing questions. To make the dispatcher do something more, we pick a question on the Attention Is All You Need paper (Vaswani et al. 2017; arXiv non-exclusive distribution license, declared on the arXiv abstract page) that fires two retrieval patterns at once. Runnable code paths call OpenAI services governed by OpenAI’s Terms of Use. Question: “What regularization techniques does the Transformer paper use, and where do they show the impact on BLEU?” The first half is a listing question (Article 12 already covered it: three techniques, Section 5.4). The second half is a cross-reference question (Article 11: the impact numbers live in Table 3, not in Section 5.4). Both have to fire on the same call.

5.1 The inputs: parsed question and document profile

The orchestrator’s two inputs are a ParsedQuestion (from Article 6) and a DocumentProfile (a cheap probe on the PDF). Both are constructed below from real code. detect_document_type opens the Transformer PDF; the ParsedQuestion is what the LLM-backed parse_question would produce for the question above. json

{
  "original_question": "What regularization techniques does the Transformer paper use, and where do they show the impact on BLEU?",
  "keywords": ["regularization", "dropout", "label smoothing", "BLEU"],
  "intent": "listing",
  "retrieval": {
    "main_query": "regularization techniques and their impact on BLEU",
    "rewrites": ["Residual Dropout", "Attention Dropout", "Label Smoothing", "Pdrop", "label smoothing epsilon", "Table 3"],
    "anchor_keywords": ["regularization", "dropout", "label smoothing", "Table 3"],
    "section_hint": "5.4",
    "layout_hint": "table"
  }
}

The three fields the dispatcher reads: intent = "listing", section_hint = "5.4", layout_hint = "table". The first activates listing aggregation; the other two activate two-hop references. json

{
  "total_pages": 15,
  "has_usable_toc": true,
  "is_likely_scanned": false,
  "suggested_strategy": "native_text_only"
}

The profile is cheap: one fitz.open call, no body parsing. Page count, TOC presence (has_usable_toc = True because the Transformer paper carries 22 bookmarks), and a scan flag (False here, since the PDF is native text). The dispatcher reads has_usable_toc to enable TOC retrieval.

5.2 Dispatcher decision

decide_pipeline_patterns(parsed_q, doc_profile) reads the two objects above and returns an activation column. The output below is the real result from running the dispatcher on this question and this document: json

{
  "toc_retrieval": true,
  "keyword_retrieval": true,
  "dense_retrieval": false,
  "two_hop_references": true,
  "listing_aggregation": true,
  "adaptive_parsing": false,
  "iterative_feedback": true
}

Four out of seven patterns activated (plus iterative feedback as the always-on safety net). The two-hop pattern makes this question different from a plain listing.

5.3 First pass: retrieval, listing, and the gap signal

pdf_qa_loop runs its pass (the same parse and retrieval entry points as Article 9’s pdf_qa) with the activation flags above. Inside that pass:

The pipeline doesn’t return an answer yet. The pending_references trigger fires.

5.4 Second pass: two-hop fetches Table 3, generation closes the loop

The orchestrator’s feedback machinery sees pending_references = ["Table 3"] and runs the two-hop pattern (Article 11). Page 9 carries Table 3: Variations on the Transformer architecture. The relevant rows for this question are:

Rows (A), (B), (C) vary attention heads, key dimension, and model size and are not relevant to the regularization question. The two-hop retriever pulls page 9 and joins it to the candidate set, then pdf_qa re-runs generation with Section 5.4 plus Table 3 in the context: The Transformer paper uses three regularization techniques during training: 1. Residual Dropout -- Pdrop = 0.1, page 8 (Section 5.4, bold header) Validated in Table 3 row (D), page 9: varying Pdrop changes BLEU on newstest2013 (4.92 PPL / 25.8 BLEU at the base 0.1 value). 2. Attention Dropout -- Pdrop = 0.1, page 8 (same paragraph as Residual Dropout) Same Table 3 row (D) covers it: Pdrop is applied uniformly. 3. Label Smoothing -- epsilon_ls = 0.1, page 8 (Section 5.4, bold header) Validated in Table 3 row (E), page 9: varying epsilon_ls changes BLEU. The text notes this "hurts perplexity but improves accuracy and BLEU". Cardinality: Section 5.4 opens with "We employ three types of regularization during training" (page 7). All three are present in this list. complete_answer_found=True, pending_references=[]. The orchestrator stops. The answer is structured (3 enumerated techniques), exhaustive (cardinality cue confirmed), and complete (each technique has both its definition page and the Table 3 row that varies it).

5.5 Cost breakdown

Indicative cost shape for a 15-page native PDF on a typical OpenAI-class model. The numbers are rough; what matters is the shape : the LLM calls dominate, everything else is sub-second.

Without the dispatcher, you would either pay for every pattern on every call, or hand-code the routing for every question shape. Dense retrieval and adaptive parsing stayed OFF here because the profile said they were not needed; that decision sits in decide.py, not in the question itself.

6. Dispatched RAG vs autonomous agents

Section 3 noted that the feedback-loop shape is what most vendors call agentic RAG. The term is used three different ways, and choosing between them shapes the whole architecture. Usage 1: agentic as marketing: Any pipeline more sophisticated than embed → retrieve → generate gets called agentic. The label is empty. We can ignore this usage. Usage 2: feedback-driven control: The LLM produces signal (complete_answer_found, llm_discovered_keywords, context_structured), and deterministic Python code reacts to that signal. This is what we’ve built. The LLM is in the system but not in control of the system. Control stays in the dispatcher. Usage 3: autonomous agents: The LLM has a set of tools and decides at each step which one to call next. The control loop is in the LLM itself. The Python code executes whatever the LLM requests. The distinction between Usage 2 and Usage 3 is the architectural decision that matters most for enterprise RAG.

6.1 What an autonomous agent looks like

In Usage 3, the orchestrator above would be replaced by something like: python

# Autonomous agent - what we are NOT building
tools = [retrieve_by_toc, retrieve_by_keywords, retrieve_dense,
         re_parse_with_camelot, follow_reference, ask_clarification]

state = {"question": question, "history": []}
while not is_satisfied(state):
    next_action = llm.choose_next_tool(state, tools)
    result = execute(next_action)
    state["history"].append((next_action, result))

The LLM decides which tool to call, with which arguments, in which order. The pipeline is generated at runtime by the LLM, not authored by the engineer. This is appealing in theory: the LLM can combine tools in ways the engineer didn’t anticipate. In practice, for enterprise RAG, three things go wrong. Reproducibility breaks: The same question on the same document can take different paths on different runs. For audit, compliance, legal review, this is disqualifying. “Why did the system retrieve this passage?” has no single answer when the LLM chose the path freely. Costs explode: Each tool decision is an LLM call. Five to ten tool calls per question. At realistic scale, the cost is one to two orders of magnitude higher than the dispatcher pattern. For a system answering thousands of questions per day, this is the difference between affordable and impossible. Debugging becomes guesswork: When an autonomous agent produces a bad answer, you have to read its reasoning trace to understand what went wrong. The trace is long, branchy, and not always coherent. Compared to the dispatcher pattern, where you can pinpoint exactly which activation rule fired or didn’t, autonomy is a step backward in maintainability.

6.2 What we keep from “agentic”

The valuable part of the agentic idea isn’t autonomy. It’s that the pipeline thinks about the question before searching, and reacts to its own outputs. Both of those are present in our composite pipeline:

What we don’t keep is the autonomy of the LLM in choosing tool sequences. That decision belongs to the dispatcher, in code, where it’s testable, auditable, and reproducible. A more honest name is structured RAG or dispatched RAG. “Agentic” survives because the field has settled on it, but the architecture stays in code, not in the LLM.

6.3 When autonomy is the right choice

Autonomous agents have their place. They’re the right pattern when:

For enterprise RAG on documents, which is what this series is about, the structure of the problem is well-understood. We know what a document looks like, what kinds of questions get asked, what kinds of patterns help. The dispatcher pattern captures that knowledge. Autonomy adds cost and unpredictability without adding capability.

6.4 The team-level implication

Dispatched RAG has a property that autonomous RAG doesn’t: the team’s understanding of the problem is encoded in the dispatcher. When a new engineer joins, they read decide.py and learn how the team thinks about question types and pattern activations. When a question fails in production, the team adds a test case and updates the dispatcher. When a new pattern is added (say, table-comparison retrieval for cross-document questions), it gets a new activation rule. An unaudited autonomous agent has no such artifact: its behavior emerges from the LLM’s reasoning at runtime, which doesn’t accumulate, doesn’t get reviewed, and doesn’t transfer between team members. This is part of why the dispatcher pattern has held up in production: the dispatcher is something a team can own, review, and hand off. The layer that sits on top, where the agent picks among patterns like the ones dispatched here, is follow-up work.

7. Conclusion

The architectural commitment that holds the four bricks together is one sentence: decisions live in decide.py, not in an LLM prompt at runtime. The orchestrator runs the bricks in order, the feedback-loop machinery turns generation’s structured output into a bounded retry decision, and the dispatcher picks the patterns from the parsed question and the document profile. The remaining articles address what surrounds the pipeline rather than what is inside: scaling from one document to a corpus (Part IV), and evaluating, running, storing, and securing the system in production (Part V).

8. Sources and further reading

The workflow-versus-agent debate was named by Anthropic in Building Effective Agents (Dec 2024). The dispatcher in this article is a workflow in that sense. The reflection-token idea from Asai et al. (Self-RAG, ICLR 2024) shows up directly in the structured feedback fields the orchestrator reads. The agentic side, with the patterns dispatched here as the agent’s audited toolkit, is follow-up work. Earlier in the series:

What works, what breaks

Document parsing

Question parsing

Retrieval

Generation

One-document pipelines

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

Angela and Kezhan Shi

See all from Angela and Kezhan Shi Deep DivesLlmRagRag Architectureloop engineering Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Recursive CTEs: SQL’s Hidden Graph Traversal Engine

Data Science A practical guide to navigate hierarchies, find routes, detect cycles and calculate degrees of separation Thomas Reid August 25, 202615 min read

6 common RAG retrieval mistakes we keep seeing in production

Large Language Model Enterprise Document Intelligence [Vol.1 #7ter] - Six positions on the retrieval brick that contradict the cosine-first reflex of mainstream RAG Kezhan Shi July 3, 20268 min read

Chat with Your Images using Multimodal LLMs

Learn how to build Llama 3.2-Vision locally in a chat-like mode, and explore its Multimodal skills on a Colab notebook Lihi Gur Arie, PhD December 5, 20247 min read

Using Vector Steering to Improve Model Guidance

Exploring the Research on Vector Steering and Coding Up an Implementation Matthew Gunton October 21, 20249 min read

Speak to me: How many words a model is reading

Artificial Intelligence Why and how to overcome the inner limit of a Large Language Model Salvatore Raieli July 14, 202319 min read

Proxy-Pointer RAG: Structure Meets Scale at 100% Accuracy with Smarter Retrieval

Large Language Model Open source. 5-minute setup. Vector RAG done right—try it yourself. Partha Sarkar April 19, 202613 min read

Loop Engineering for Cross-References: When RAG Answers ‘see Section 7.2’ Instead of the Actual Answer

Large Language Model Enterprise Document Intelligence [Vol.1 #11] - When the first answer points elsewhere in the document, the pipeline loops back to fetch the linked context Angela and Kezhan Shi August 6, 202620 min read

Bit-LoRA as an application of BitNet and 1.58 bit neural network technologies

Abstract: applying ~1bit transformer technology to LoRA adapters allows us to reach comparable performance with full-precision LoRA... Roman Smirnov June 2, 202414 min read

A comprehensive study of Mixed Integer Programming with JuMP on Julia (Part 1)

Some basics of Linear/Mixed Integer Programming & How to use a heuristic callback inside a MIP solver. Ouaguenouni Mohamed March 29, 202114 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network

Recommended Articles

Close