![Revisit consent button](../images/body-df29beacd7496220cc94.svg) We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg) We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data. * Cookie BCTempID * Duration 10 minutes * Description No description available. * Cookie __cf_bm * Duration 1 hour * Description This cookie, set by Cloudflare, is used to support Cloudflare Bot Management. * Cookie AWSALBCORS * Duration 7 days * Description Amazon Web Services set this cookie for load balancing. * Cookie _cfuvid * Duration session * Description Cloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services * Cookie li_gc * Duration 6 months * Description Linkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes. * Cookie __hssrc * Duration session * Description This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session. * Cookie __hssc * Duration 1 hour * Description HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie. * Cookie wpEmojiSettingsSupports * Duration session * Description WordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly. * Cookie BCSessionID * Duration 1 year 1 month 4 days * Description Blueconic sets this cookie as a unique identifier for the BlueConic profile. * Cookie _octo * Duration 1 year * Description No description available. * Cookie logged_in * Duration 1 year * Description No description available. * Cookie __Secure-YEC * Duration past * Description YouTube sets this cookie to stores the user's video player preferences using embedded YouTube video * Cookie __eoi * Duration 6 months * Description Description is currently not available. * Cookie AWSALBTGCORS * Duration 7 days * Description No description available. * Cookie login-status-p * Duration past * Description Description is currently not available. * Cookie AWSALBTG * Duration 7 days * Description No description available. * Cookie csrf_token * Duration session * Description No description available. * Cookie token_v2 * Duration 1 day * Description Description is currently not available. * Cookie D * Duration 1 year * Description Description is currently not available. * Cookie PHPSESSID * Duration session * Description This cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed. * Cookie VISITOR_PRIVACY_METADATA * Duration 6 months * Description YouTube sets this cookie to store the user's cookie consent state for the current domain. * Cookie cookietest * Duration session * Description The cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience. * Cookie __Host-airtable-session * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie __Host-airtable-session.sig * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie m * Duration 1 year 1 month 4 days * Description Stripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly. * Cookie BIGipServer* * Duration session * Description Marketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. * Cookie __cfruid * Duration session * Description Cloudflare sets this cookie to identify trusted web traffic. * Cookie _GRECAPTCHA * Duration 6 months * Description Google Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks. * Cookie __Secure-YNID * Duration 6 months * Description Google cookie used to protect user security and prevent fraud, especially during the login process. * Cookie cookieyes-consent * Duration 1 year * Description CookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors. Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features. * Cookie lidc * Duration 1 day * Description LinkedIn sets the lidc cookie to facilitate data center selection. * Cookie brw * Duration 1 year * Description No description available. * Cookie brwConsent * Duration 5 minutes * Description Description is currently not available. * Cookie WMF-Uniq * Duration 1 year * Description Description is currently not available. * Cookie loom_anon_comment * Duration 1 year * Description No description available. * Cookie loom_referral_video * Duration session * Description Description is currently not available. * Cookie VISITOR_INFO1_LIVE * Duration 6 months * Description A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface. * Cookie yt-remote-connected-devices * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie ytidb::LAST_RESULT_ENTRY_KEY * Duration Never Expires * Description The cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future. * Cookie yt-remote-device-id * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie yt-remote-session-name * Duration session * Description The yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video. * Cookie yt-remote-fast-check-period * Duration session * Description The yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos. * Cookie yt-remote-session-app * Duration session * Description The yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player. * Cookie yt-remote-cast-available * Duration session * Description The yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player. * Cookie yt-remote-cast-installed * Duration session * Description The yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video. * Cookie cp_session * Duration 3 months * Description Codepen sets this cookie for Help systems found in the website. * Cookie loid * Duration 1 year 1 month 4 days * Description This cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform. Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc. * Cookie __hstc * Duration 6 months * Description Hubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session). * Cookie hubspotutk * Duration 6 months * Description HubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts. * Cookie _ga * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors. * Cookie _ga_* * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to store and count page views. * Cookie __Host-psifi.analyticsTrace * Duration 6 hours * Description Description is currently not available. * Cookie __Host-psifi.analyticsTraceV2 * Duration 6 hours * Description Description is currently not available. * Cookie _gh_sess * Duration session * Description GitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form. * Cookie YSC * Duration session * Description YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages. * Cookie ajs_anonymous_id * Duration 1 year * Description This cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before. * Cookie vuid * Duration 1 year 1 month 4 days * Description Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors. * Cookie AWSALB * Duration 7 days * Description AWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target. * Cookie acq * Duration past * Description Description is currently not available. * Cookie acq.sig * Duration past * Description Description is currently not available. * Cookie ptc * Duration 2 years * Description No description available. Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns. * Cookie muc_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to collect user behaviour and interaction data to optimize the website. * Cookie guest_id_marketing * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie guest_id_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie personalization_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting. * Cookie guest_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences. * Cookie bcookie * Duration 1 year * Description LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs. * Cookie __Secure-ROLLOUT_TOKEN * Duration 6 months * Description YouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment. * Cookie yt.innertube::nextId * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie yt.innertube::requests * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie session_tracker * Duration session * Description This cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website. * Cookie edgebucket * Duration session * Description Reddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content. * Cookie did * Duration 1 year * Description Arbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year. Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation [](https://towardsdatascience.com/) Publish AI, ML & data-science insights to a global community of data professionals. Sign in[Submit an Article](https://write.insightmediagroup.io) [Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/) * * * [Write For TDS](https://towardsdatascience.com/submissions/) [LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search [Large Language Model](https://towardsdatascience.com/category/large-language-model/) # Loop Engineering for RAG: The Small Loops Inside Each Step, the Big Loops Across the Pipeline Enterprise Document Intelligence [Vol.1 #13bis] - The four bricks return useful results most of the time. Loop engineering is what the system does the rest of the time: when retrieval misses, when generation fails the schema, when the listing comes back incomplete, when an API call times out. Three control surfaces (trigger, termination, recovery) and one rule that separates a useful loop from a spinning one [Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/) August 17, 202614 min read ![](../images/000.jpg) [Photo](https://www.pexels.com/photo/11598090/) by Nina Simková, via Pexels. AI engineers today talk a lot about loop engineering. The phrase took off after Boris Cherny, the Anthropic engineer behind Claude Code, said he had mostly stopped prompting: he writes loops that prompt the model until a condition is met. For a coding agent with a test suite to check against, that works. For an enterprise RAG pipeline, the loop itself is the part that needs engineering, because the failures it has to absorb are ordinary: the parser flattens the one table the answer lived in, retrieval returns the page next to the right one, the model sends back JSON that fails the schema, the API times out in the middle of a batch. The four bricks handle the clean path; loop engineering is what the system does on every other path, and doing it well is what lets the pipeline recover instead of spin. A one-shot pipeline commits to its first try: parse once, retrieve once, generate once, return whatever comes out. When retrieval comes back empty or the answer is half-formed, there is no second chance. A loop gives the pipeline one: notice the miss, adjust, and run the weak step again before the user ever sees it. This article is a standalone companion to [Enterprise Document Intelligence](https://towardsdatascience.com/prompt-context-loop-the-three-engineering-layers-every-rag-system-is-built-on/), a series that builds an enterprise RAG system from four bricks. It sits next to [Article 7bis (context engineering for single-document RAG)](https://towardsdatascience.com/context-engineering-for-rag-the-four-typed-inputs-behind-every-rag-answer/) and treats the layer just above it. Article 13 (the composite pipeline) glued the bricks together and named the dispatcher. This article zooms into what the dispatcher does when a single call is not enough. Volume 4 develops the agentic version of the same discipline. This piece stays at single-document scope. The rest is the map: what loop engineering is, the anatomy of one loop (trigger, termination, recovery), the two scales it runs at, the failure modes to guard against, and the outer loop the human holds. Learn this step by step with the interactive [AI Engineer](https://roadmap.sh/ai-engineer) roadmap. 🧭 **New to the series? Start with the map** : [Prompt, Context, Loop](https://towardsdatascience.com/prompt-context-loop-the-three-engineering-layers-every-rag-system-is-built-on/) sets out the three engineering layers every RAG system is built on, the prompt (the call itself), the context (what fills the model’s window), the loop (when the next call fires and when it stops), and walks the whole series through that lens, article by article. It is the shortest way to see what is covered and where this one sits. ![](../images/001.png) _where this article sits in the series: Article 13bis (loop engineering), a companion to the composite pipeline - Image by author_ 📓 The runnable companion shows the loops firing: you trip `llm_parse` on a simulated timeout and watch the backoff schedule count down, then flip `complete_answer_found` to false on a listing answer and watch the dispatcher widen the retrieval scope and regenerate. On GitHub: [doc-intel/notebooks-vol1](https://github.com/doc-intel/notebooks-vol1). ![](../images/002.png) _The public companion-code repo at doc-intel/notebooks-vol1 - Image by author_ ## 1. What loop engineering is ### 1.1 The third layer of the agent stack Three disciplines stack on top of an LLM call. Prompt engineering writes the call: system message, user message, schema. Context engineering chooses what enters and exits the model’s context window between calls (see Article 7bis). Loop engineering decides _when the next call happens_ , _what triggers it_ , _when the loop stops_ , and _how the system recovers when something goes wrong_. A polished prompt with a clean retrieval upstream produces a correct answer most of the time. The rest of the time the call fails in one of the ways the opening listed: an invalid JSON, a self-flagged incomplete listing, a 429 at the deployment’s rate cap. Loop engineering is the discipline of designing what happens next. The shortest critique of bad loop engineering, paraphrasing the practitioner literature, is this: a loop that retries the same action on the same error is not learning, it is spinning. The difference between a loop that helps and a loop that wastes tokens is whether each iteration changes something the previous iteration did not address. ### 1.2 The lineage in one paragraph Bounded retry is the oldest pattern in the catalogue: Erlang’s _let it crash_ model has carried some version of it since the 1980s. ReAct (Princeton and Google, October 2022) put it inside an LLM call for the first time: reason about the result, decide to act again or stop. AutoGPT (March 2023) made the autonomous version public. Reflexion (NeurIPS 2023) added self-evaluation. Plan-and-Execute separated planning from execution. Geoffrey Huntley’s Ralph Loop (July 2025) put the goal on disk so a context reset never lost track. Anthropic released the `/goal` command in Claude Code (May 2026) and packaged the whole stack two weeks later as Dynamic Workflows. The single-document case in this article uses a much smaller subset: retry-with-backoff, schema-fail retry, completeness check, dispatcher branching. None of the heavier orchestration primitives bite at single-document scope. They will, in V1 Part IV (corpus) and V4 (agentic). ## 2. The anatomy of a loop Before cataloguing the loops in a RAG pipeline, it helps to fix the parts of a single one. Every loop, small or big, has the same three control surfaces and answers to the same one rule. ### 2.1 Three control surfaces: trigger, termination, recovery A loop has three control surfaces that are easier to design separately than together. **Trigger.** The condition under which a _new_ call fires after the first one returned. Three triggers show up in the single-document case. _Schema validation failed_ : the model produced JSON that the Pydantic schema rejected. _The answer carries a self-flag_ : the listing schema’s `complete_answer_found` field is `false`, the answer schema’s `confidence` is below 0.6, the model wrote `needs_clarification`. _Transient API failure_ : an `APITimeoutError`, an `APIConnectionError`, a 429, a 5xx. Each trigger is a single named predicate. The trace store logs which one fired; the dispatcher (Article 13) decides which one to handle and how. **Termination.** The condition under which the loop stops. Three classes show up in the single-document case. _Loop-until-done_ terminates when the predicate is finally satisfied (the schema validates, `complete_answer_found` is true, the confidence is above threshold). _Loop-until-budget_ terminates when a quota is exhausted (six retries reached, sixty seconds of wall-clock burned). _Hard cap_ terminates unconditionally after N attempts (the `llm_parse` wrapper hard-stops at six retries with `RuntimeError`). A loop without an explicit termination is the canonical failure mode the Anthropic Cookbook flags first. **Recovery.** What happens when a call fails. The library this series provides does the simplest answer in its LLM wrapper: retry-with-backoff. Six attempts, delays of 2 / 4 / 8 / 16 / 32 / 60 seconds plus jitter, on a fixed list of retriable exceptions (`APITimeoutError`, `APIConnectionError`, `RateLimitError`, 429, 500, 502, 503, 504). Three richer answers fit different cases: fallback to a different model (the 3B failed schema, the 7B might), escalate to a human reviewer (any answer below 0.6 confidence on a high-stakes question), or skip the item and return what was computed (the batch result carries the list of successes and the list of failures, the consumer decides whether to retry the failures later). These three controls describe most of what a loop _does_ between LLM calls. ![](../images/body-c47e68c98ade5aace361.png) _One LLM call, three control surfaces around it: trigger, termination, recovery. - Image by author_ The one control surface the library provides as a ready-made primitive is recovery, in the shape of retry-with-backoff. It lives in `llm_parse` and wraps every parse call: bounded attempts, an exponential delay with jitter, a hard cap that raises rather than spins. python ``` # the library's LLM wrapper - retry-with-backoff, built into every parse call def llm_parse(*, input, text_format, max_retries=6, cache=True, **opts): for attempt in range(max_retries): try: return client.responses.parse(input=input, text_format=text_format) except Exception as err: if not is_retriable(err): # timeout/429/5xx retry ; raise # context-length never does if attempt == max_retries - 1: raise # hard cap -> propagate delay = _compute_delay(attempt, base=2, cap=60, jitter=0.3, retry_after=extract_retry_after(err)) # 2,4,..,60s time.sleep(delay) # honour Retry-After floor ``` ### 2.2 The anti-spinning rule A loop that fires its trigger, retries with the same payload, gets the same failure, and retries again is not a loop. It is a denial-of-service against the company’s API budget. The rule that prevents this is the simplest one in the discipline. A loop should only retry when _something has changed_ between the previous attempt and the next. Three legitimate things to change. The _payload_ : the dispatcher widened the retrieval scope (50 lines became 200 lines), the system prompt got an extra constraint (_“the previous answer did not validate; produce ONLY a JSON matching this schema with no prose around it”_), the question was rephrased (the user clarified what they meant by “in Q3”). The _model_ : the small model returned garbage, the large model gets the second shot. [Article 8quater (adaptive model selection)](https://towardsdatascience.com/loop-engineering-for-rag-generation-an-llm-cascade-from-a-cheap-local-model-up-to-a-hosted-flagship/) builds this loop: start with the cheapest model the job allows, escalate only when validation rejects. The _strategy_ : the keyword retrieval missed, the second attempt uses dense retrieval. This is the dispatcher’s job (Article 13) and the V4 catalogue calls it _classify-and-act_. Retry that does not change one of the three is spinning. The implementation rule is operational: every retry should carry a trace block that names what changed. If the trace block is empty, the retry should not have fired. The fix is to bound retries at a small N (the library uses six) and to log the _changed-since-last_ field on every iteration. The day a user complains that _“the same question now costs four times as much”_ , the trace tells you whether the loop was working or spinning. ## 3. Two scales of loop A loop is not one magic mechanism bolted onto the pipeline. Each one is a named, bounded response to a specific way a brick can fall short, and they sort into two scales: small loops that run inside a single brick, and big loops that cross bricks when generation distrusts its own input. ### 3.1 Small loops: one inside each brick **Small loops live inside a single brick.** Document parsing runs the image cascade: filter the page’s images cheaply, classify what each one is, and pay a vision model to describe only the ones worth reading (Article 5). It also rebuilds a missing outline from body typography, looping until the outline stabilises (Article 5octies), and dispatches each page to the parser its signals call for (Article 5nonies). Retrieval walks the table of contents down to the right subsection (Article 7quater). Question parsing asks one clarifying question and learns the default (Article 6bis). Generation retries when the JSON fails the schema, and reruns retrieval and generation when the listing comes back incomplete (Articles 8 and 12). Each of these has the same three control surfaces as any loop, a trigger, a termination, and a bounded depth; it just iterates over its own brick’s material rather than reacting to a downstream result. Retry-with-backoff wraps every API call underneath all of them, the one loop that is pure plumbing rather than a brick’s own reasoning. ![](../images/body-e4dae080c43e196c671e.png) _Small loops sit inside a brick, big loops cross bricks when generation distrusts its input. - Image by author_ ### 3.2 Big loops: across bricks **Big loops cross bricks.** Generation reads its own input, decides it was not good enough to answer from, and sends the pipeline back to an earlier brick. The single-document case has three: adaptive parsing (generation’s `context_structured=false` re-parses the offending page with a deeper parser, Articles 10A and 10B), reference resolution (generation reports `pending_references`, the orchestrator resolves the pointer against the parsing tables, re-retrieves the target region, regenerates, Article 11), and scope feedback (generation’s `complete_answer_found=false` widens retrieval, Article 13). Same shape each time: a typed flag on the answer, a bounded trip back upstream, a second generation pass that re-checks itself. The check is a typed flag, the escalation is bounded to the flagged page, and the loop stops the moment the re-checked answer trusts its context. ![](../images/body-a7ea2b086f4ebf6c3125.png) _The adaptive-parsing big loop: verify the parse, escalate the flagged page, regenerate, re-verify. - Image by author_ ### 3.3 The named patterns the series already has Looking at the existing V1 code with the 2026 loop-engineering vocabulary on hand: most of the named primitives have a counterpart already in the code, under a less catchy name. ![](../images/body-e4dae080c43e196c671e.png) _The five primitives the single-document case uses already exist in the series under quieter names. - Image by author_ **Loop-until-done** : the V1 Article 12 listing completeness loop. The generator answers, the schema carries the `complete_answer_found` flag, the loop reruns retrieval and generation if the flag is false. The pattern came with the listing schema and predates the name. **Retry-with-backoff** : the library’s `llm_parse` wrapper. Six attempts with the exponential schedule. The cost analysis in Article 21 Section 4.6 details when this triggers and what the operational impact looks like. **Generate-and-filter** : V1 Article 8’s structured-output schema. The LLM returns JSON, the Pydantic validator rejects anything that does not match, the loop retries if validation fails. The 2026 vocabulary makes the loop layer explicit; the discipline has been in the series since the schema-first generation article. **Classify-and-act** : V1 Article 13’s composite dispatcher. The parsed question carries a shape (single-value, listing, aggregation, comparison), the dispatcher routes to the matching handler. Anthropic’s classify-and-act applies the same shape to free-form classification rather than to a parsed-question taxonomy; the underlying machinery is the same. **Feedback-loop-between-bricks** : V1 Article 13’s between-bricks feedback. Generation flags the retrieval scope as insufficient, the dispatcher widens the scope, generation reruns. The 2026 vocabulary calls this _evaluator-optimiser_ ; the series called it the dispatcher’s feedback loop and built it in 2024. What is _not_ yet in V1: adversarial verification (a separate agent that tries to refute the answer), the tournament pattern (N candidate answers, judges pick the winner), explicit _loop-until-dry_ termination (K consecutive empty rounds). Those are V4-scope; the single-document case rarely needs them. ### 3.4 The loops at a glance Every loop in the single-document pipeline, small or big, wears the same three control surfaces from Section 2. Read down the trigger column and the shape is the same each time: the loop fires on one specific, named signal, terminates on a bound, and changes exactly one thing before it retries. ![](../images/body-9abc75b626988dd8d317.png) _Every loop as one row against the three control surfaces of Section 2. - Image by author_ ## 4. Failure modes the discipline guards against Three failure modes are routine in production single-document RAG and each one maps to a specific loop-engineering rule. **Infinite-loop on transient failure.** The retry layer has no hard cap. The deployment hits a 429, the loop retries, the next 429 arrives within the rate-limit window, the loop retries again, the API budget burns through the night. _Fix_ : a hard cap on retries (the library uses six), an explicit `RuntimeError` after the cap, and an alert on the trace store when the cap is hit. **Spinning on a deterministic failure.** The Pydantic schema rejects the output because the model wrote a field name with a typo. The loop retries with the same prompt. The model produces the same typo. Six retries, six rejections, the user sees a `RuntimeError`. _Fix_ : on schema-fail retry, modify the prompt to include the validator error message verbatim (“ _your previous answer failed validation with:_`_field_name_`_should be_` _field_name_actual_`_. Produce a JSON matching the schema exactly._ ”). The next attempt has new information. **Confidence-flag ignored.** The answer comes back with `confidence = 0.3` and the dispatcher returns it to the user without flagging anything. _Fix_ : the dispatcher applies a confidence threshold, low-confidence answers either escalate to a richer retrieval pass (widen scope, switch from keyword to dense) or carry an explicit `low_confidence` warning to the consumer. ## 5. Loops within loops: the human holds the outer one Loop engineering as covered so far is one loop: the dispatcher fires a call, reads the result, decides to retry, terminates when satisfied. Zoom out and there are three loops running at different cadences around the same system, borrowing the frame Andrew Ng laid out in [his “Three Key Loops for Building Great Software” letter](https://www.deeplearning.ai/the-batch/three-key-loops-for-building-great-software): * **The inner loop, seconds to a minute.** The dispatcher itself. Retry-with-backoff on a 502, regenerate on a schema fail, expand the anchor set on `complete_answer_found: false`. No human, bounded budget, code decides. Sections 2 to 4 of this article. * **The dev loop, minutes to hours.** An engineer reads the audit trail, spots a class of failures, edits `concept_keywords_df`, adds a synonym pair, tightens a Pydantic schema, deploys. The dispatcher then runs the corrected version on the next call. Article 6’s expert dictionary and Article 20’s evaluation dashboard are the surfaces this loop reads from. * **The outer loop, hours to weeks.** A user edits an extracted field the system got wrong. The correction lands in the storage layer with a citation, becomes ground truth for the next evaluation pass, feeds into the concept catalog through a review queue. Or a claims handler pushes back on a question template the system was answering literally, and the template gets rephrased for everyone downstream. The refresh-field pattern from Article 19 and the question-template review from Article 20 sit here. Ng’s point is that as long as the human knows something the AI does not, that outer loop cannot be closed by more automation, and treating it as noise to be optimised away is the mistake. _“As long as the human knows something the AI does not, human-in-the-loop is needed to inject that knowledge into the system.”_ This is the series’ amplify-the-expert thesis in 2026 vocabulary. The dispatcher owns the inner loop, the engineer owns the dev loop, the expert owns the outer loop, and none of them can substitute for the others. The corollary for design: keep the outer loop cheap for the expert. If correcting a wrong field takes ten clicks, the outer loop breaks and the system’s error rate stops improving. The refresh-field UX, the review queue, and the audit trail keep the outer loop fast enough to compound. ## 6. Out of scope, with pointers Three siblings stay open here. * **The corpus case.** Single-document loops are bounded by the size of one document. Corpus loops add cascade-with-partial-recovery (twelve documents in a batch, three fail, keep the nine successes and let the consumer retry the failures later) and refresh-field on user edit. V1 Article 17quater treats the corpus loop engineering. * **The agentic case.** Single-document loops are bounded in turns (the dispatcher fires at most a handful of retries before terminating). Agentic loops keep the budget open across dozens of turns, with sub-agents, memory tools, and the `/compact` lifecycle. V4 Article 9bis covers context engineering for the agentic loop; V4 Article 9ter covers loop engineering for the same. * **Adversarial verification.** This article focuses on bounded retry and self-flagged completeness checks. The richer discipline of _independent agents whose job is to refute the answer_ is V4-scope. V1 single-document RAG rarely needs it; the V4 article develops it. ## 7. Conclusion Loop engineering is the third discipline after prompt engineering and context engineering. At single-document scope, it owns four primitives: retry-with-backoff for transient API failures, generate-and-filter for schema-fail retries, loop-until-done for completeness checks, classify-and-act for dispatcher branching. The single rule that separates a useful loop from a spinning one is _every retry changes something_. The library this series provides implements the first primitive directly; the other three are in the dispatcher (Article 13) and the listing schema (Article 12). The 2026 vocabulary puts a name on a discipline the series has been practising since 2024. ## Register Now to Continue Reading ### Unlock Towards Data Science with a Free Account All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account * * * ![](../images/body-249d024535392fb76067.jpeg) Written By ## [Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/) [See all from Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/) [Document Processing](https://towardsdatascience.com/tag/document-processing/)[Llm](https://towardsdatascience.com/tag/llm/)[Rag](https://towardsdatascience.com/tag/rag/)[Rag Architecture](https://towardsdatascience.com/tag/rag-architecture/)[loop engineering](https://towardsdatascience.com/tag/loop-engineering/) Share This Article * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Floop-engineering-for-rag-the-small-loops-inside-each-step-the-big-loops-across-the-pipeline%2F&title=Loop%20Engineering%20for%20RAG%3A%20The%20Small%20Loops%20Inside%20Each%20Step%2C%20the%20Big%20Loops%20Across%20the%20Pipeline) * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Floop-engineering-for-rag-the-small-loops-inside-each-step-the-big-loops-across-the-pipeline%2F&title=Loop%20Engineering%20for%20RAG%3A%20The%20Small%20Loops%20Inside%20Each%20Step%2C%20the%20Big%20Loops%20Across%20the%20Pipeline) * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Floop-engineering-for-rag-the-small-loops-inside-each-step-the-big-loops-across-the-pipeline%2F&text=Loop%20Engineering%20for%20RAG%3A%20The%20Small%20Loops%20Inside%20Each%20Step%2C%20the%20Big%20Loops%20Across%20the%20Pipeline) Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. [Write for TDS](https://towardsdatascience.com/submissions/) ## Related Articles * [![](../images/003.jpg)](https://towardsdatascience.com/code-understanding-on-your-own-hardware-dd38c4f266d6/) ## [Code understanding on your own hardware](https://towardsdatascience.com/code-understanding-on-your-own-hardware-dd38c4f266d6/) [](https://towardsdatascience.com/loop-engineering-for-rag-the-small-loops-inside-each-step-the-big-loops-across-the-pipeline/) Setting up an LLM to talk about your code - with LangChain and local hardware [Dorian Drost](https://towardsdatascience.com/author/doriandrost/) July 5, 20237 min read * [![](../images/004.jpg)](https://towardsdatascience.com/how-to-increase-coding-iteration-speed/) ## [How to Increase Coding Iteration Speed](https://towardsdatascience.com/how-to-increase-coding-iteration-speed/) [LLM Applications](https://towardsdatascience.com/category/artificial-intelligence/llm-applications/) Learn how to become a more efficient programmer with local testing [Eivind Kjosbakken](https://towardsdatascience.com/author/oieivind/) December 13, 20257 min read * [![](../images/005.jpg)](https://towardsdatascience.com/prompt-engineering-isnt-enough-i-built-a-control-layer-that-works-in-production/) ## [Prompt Engineering Isn’t Enough — I Built a Control Layer That Works in Production](https://towardsdatascience.com/prompt-engineering-isnt-enough-i-built-a-control-layer-that-works-in-production/) [Large Language Model](https://towardsdatascience.com/category/large-language-model/) Most production LLM integrations treat prompts like the final layer. That gets you 0% reliability on structured output. I built an 8-component system that brought it to 100% — without changing a single prompt. [Emmimal P Alexander](https://towardsdatascience.com/author/emmimalp.alexander/) May 21, 202620 min read * [![](../images/006.jpg)](https://towardsdatascience.com/llm-fallbacks-break-agent-pipelines-i-built-the-missing-recovery-layer/) ## [LLM Fallbacks Break Agent Pipelines — I Built the Missing Recovery Layer](https://towardsdatascience.com/llm-fallbacks-break-agent-pipelines-i-built-the-missing-recovery-layer/) [Large Language Model](https://towardsdatascience.com/category/large-language-model/) Standard model swapping forces your production AI to accept corrupted data. Here is the zero-dependency, state-aware recovery layer I built to preserve schema integrity during failovers. [Emmimal P Alexander](https://towardsdatascience.com/author/emmimalp.alexander/) June 16, 202614 min read * [![](../images/007.jpg)](https://towardsdatascience.com/stop-returning-text-from-rag-the-typed-answer-contract-that-prevents-hallucination/) ## [Stop Returning Text from RAG: The Typed Answer Contract That Prevents Hallucination](https://towardsdatascience.com/stop-returning-text-from-rag-the-typed-answer-contract-that-prevents-hallucination/) [Large Language Model](https://towardsdatascience.com/category/large-language-model/) Enterprise Document Intelligence [Vol.1 #8A] - The schema is the contract: every field is a question the pipeline asks the model, and every answer is checkable [Kezhan Shi](https://towardsdatascience.com/author/kezhanshi/) July 4, 202626 min read * [![](../images/008.png)](https://towardsdatascience.com/building-a-knowledge-graph-from-scratch-using-llms-f6f677a17f07/) ## [Building a Knowledge Graph From Scratch Using LLMs](https://towardsdatascience.com/building-a-knowledge-graph-from-scratch-using-llms-f6f677a17f07/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) Turn your Pandas Data Frame into a Knowledge Graph using LLMs. Build your own LLM graph-builder and QA your KG. [Cristian Leo](https://towardsdatascience.com/author/cristianleo120/) November 25, 202439 min read * [![](../images/009.jpg)](https://towardsdatascience.com/llms-pandas-how-i-use-generative-ai-to-generate-pandas-dataframe-summaries-2/) ## [LLMs + Pandas: How I Use Generative AI to Generate Pandas DataFrame Summaries](https://towardsdatascience.com/llms-pandas-how-i-use-generative-ai-to-generate-pandas-dataframe-summaries-2/) [Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/) Local Large Language Models can convert massive DataFrames to presentable Markdown reports — here's how. [Dario Radečić](https://towardsdatascience.com/author/radecicdario/) June 2, 202510 min read * [![](../images/010.jpg)](https://towardsdatascience.com/one-rag-pipeline-four-very-different-pdfs-same-four-bricks-every-answer-typed-and-cited/) ## [A Production RAG Pipeline in Action: Every Answer Typed and Cited](https://towardsdatascience.com/one-rag-pipeline-four-very-different-pdfs-same-four-bricks-every-answer-typed-and-cited/) [Large Language Model](https://towardsdatascience.com/category/large-language-model/) Enterprise Document Intelligence [Vol.1 #9B] - One call wires the four upgraded bricks together, run on a paper, a NIST standard, and a report with a broken TOC [Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/) July 17, 202625 min read * [![](../images/011.jpg)](https://towardsdatascience.com/cut-an-enterprise-rag-pipelines-latency-and-cost-by-calling-the-llm-less-not-by-buying-a-faster-model/) ## [Cut an Enterprise RAG Pipeline’s Latency and Cost by Calling the LLM Less, Not by Buying a Faster Model](https://towardsdatascience.com/cut-an-enterprise-rag-pipelines-latency-and-cost-by-calling-the-llm-less-not-by-buying-a-faster-model/) [Large Language Model](https://towardsdatascience.com/category/large-language-model/) Enterprise Document Intelligence [Vol.1 #9ter] - The pipeline from Article 9 calls a model at several steps to be sure it is right. On easy questions that is needless latency. A per-question signal routes them past the model, about two seconds saved for a keyword match. [Angela and Kezhan Shi](https://towardsdatascience.com/author/angela.shi/) August 13, 202611 min read * [YouTube](https://www.youtube.com/c/TowardsDataScience) * [X](https://x.com/TDataScience) * [LinkedIn](https://www.linkedin.com/company/towards-data-science/) * [Threads](https://www.threads.net/@towardsdatascience) * [Bluesky](https://bsky.app/profile/towardsdatascience.com) [](https://towardsdatascience.com/) Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/) © Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network) #### Recommended Articles Close * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607631/image?etag=1781543525510&width=500&height=500) Critical Mistakes Companies Make When Integrating AI/ML into Their Processes ](https://towardsdatascience.com/critical-mistakes-companies-make-when-integrating-ai-ml-into-their-processes/ "Critical Mistakes Companies Make When Integrating AI/ML into Their Processes") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ "How Relevance Models Foreshadowed Transformers for NLP") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ "Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ "The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs")