{
  "source_file": "0930",
  "source_platform": "Medium",
  "author": "Benjamin Nweke",
  "title_listed": "I Replaced GPT-4 with a Local SLM and My CI/CD Pipeline Stopped Failing",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://towardsdatascience.com/i-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing/",
  "publish_date": "2026-04-21",
  "corpus_stage": "",
  "notes": "",
  "id": "ca4739e88720",
  "collected_at": "2026-10-02T07:43:22",
  "title": "I Replaced GPT-4 with a Local SLM and My CI/CD Pipeline Stopped Failing | Towards Data Science",
  "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n  * Cookie\nBCTempID\n  * Duration\n10 minutes\n  * Description\nNo description available.\n\n\n  * Cookie\n__cf_bm\n  * Duration\n1 hour\n  * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n  * Cookie\nAWSALBCORS\n  * Duration\n7 days\n  * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n  * Cookie\n_cfuvid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n  * Cookie\nli_gc\n  * Duration\n6 months\n  * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n  * Cookie\n__hssrc\n  * Duration\nsession\n  * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n  * Cookie\n__hssc\n  * Duration\n1 hour\n  * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n  * Cookie\nwpEmojiSettingsSupports\n  * Duration\nsession\n  * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n  * Cookie\nBCSessionID\n  * Duration\n1 year 1 month 4 days\n  * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n  * Cookie\n_octo\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nlogged_in\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\n__Secure-YEC\n  * Duration\npast\n  * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n  * Cookie\n__eoi\n  * Duration\n6 months\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTGCORS\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\nlogin-status-p\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTG\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\ncsrf_token\n  * Duration\nsession\n  * Description\nNo description available.\n\n\n  * Cookie\ntoken_v2\n  * Duration\n1 day\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nD\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nPHPSESSID\n  * Duration\nsession\n  * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n  * Cookie\nVISITOR_PRIVACY_METADATA\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n  * Cookie\ncookietest\n  * Duration\nsession\n  * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n  * Cookie\n__Host-airtable-session\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\n__Host-airtable-session.sig\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\nm\n  * Duration\n1 year 1 month 4 days\n  * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n  * Cookie\nBIGipServer*\n  * Duration\nsession\n  * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n  * Cookie\n__cfruid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n  * Cookie\n_GRECAPTCHA\n  * Duration\n6 months\n  * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n  * Cookie\n__Secure-YNID\n  * Duration\n6 months\n  * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n  * Cookie\ncookieyes-consent\n  * Duration\n1 year\n  * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n  * Cookie\nlidc\n  * Duration\n1 day\n  * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n  * Cookie\nbrw\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nbrwConsent\n  * Duration\n5 minutes\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nWMF-Uniq\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nloom_anon_comment\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nloom_referral_video\n  * Duration\nsession\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nVISITOR_INFO1_LIVE\n  * Duration\n6 months\n  * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n  * Cookie\nyt-remote-connected-devices\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n  * Duration\nNever Expires\n  * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n  * Cookie\nyt-remote-device-id\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-name\n  * Duration\nsession\n  * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\nyt-remote-fast-check-period\n  * Duration\nsession\n  * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-app\n  * Duration\nsession\n  * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-available\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-installed\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\ncp_session\n  * Duration\n3 months\n  * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n  * Cookie\nloid\n  * Duration\n1 year 1 month 4 days\n  * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n  * Cookie\n__hstc\n  * Duration\n6 months\n  * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n  * Cookie\nhubspotutk\n  * Duration\n6 months\n  * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n  * Cookie\n_ga\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n  * Cookie\n_ga_*\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n  * Cookie\n__Host-psifi.analyticsTrace\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n__Host-psifi.analyticsTraceV2\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n_gh_sess\n  * Duration\nsession\n  * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n  * Cookie\nYSC\n  * Duration\nsession\n  * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n  * Cookie\najs_anonymous_id\n  * Duration\n1 year\n  * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n  * Cookie\nvuid\n  * Duration\n1 year 1 month 4 days\n  * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n  * Cookie\nAWSALB\n  * Duration\n7 days\n  * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n  * Cookie\nacq\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nacq.sig\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nptc\n  * Duration\n2 years\n  * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n  * Cookie\nmuc_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n  * Cookie\nguest_id_marketing\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\nguest_id_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\npersonalization_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n  * Cookie\nguest_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n  * Cookie\nbcookie\n  * Duration\n1 year\n  * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n  * Cookie\n__Secure-ROLLOUT_TOKEN\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n  * Cookie\nyt.innertube::nextId\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nyt.innertube::requests\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nsession_tracker\n  * Duration\nsession\n  * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n  * Cookie\nedgebucket\n  * Duration\nsession\n  * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n  * Cookie\ndid\n  * Duration\n1 year\n  * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\n# I Replaced GPT-4 with a Local SLM and My CI/CD Pipeline Stopped Failing\nThe hidden cost of probabilistic outputs in systems that demand reliability\n[Benjamin Nweke](https://towardsdatascience.com/author/benjaminnweke/)\nApril 21, 202611 min read\n![](../images/000.jpg)\nImage by author (Generated with Nano Banana Pro)\nI was on my seventh rewrite of the same system prompt.\n_You MUST return ONLY valid JSON. No markdown. No code fences. No explanation. JUST the JSON object._\nI had written MUST in all-caps. To a language model. As if emphasis would work on something that doesn't have feelings or, apparently, a consistent definition of \"valid JSON.\"\nIt didn't work. Here's what did.\n## How GPT-4 Ended Up in a Nightly Batch Job\nOur team consumes research documents, such as PDFs and plain text, and occasionally those pesky semi-structured reports that some vendor clearly exported from a spreadsheet they were very proud of. And part of that pipeline classifies them and extracts structured fields before anything touches the data warehouse. Methodology type, dataset source, key metrics.\nThis sounds like a solved problem. It usually is, until there are about forty types of methods listed and the documents stop looking anything like the ones you trained on.\nFor a while, we handled this using regex, rule-based extractors, and a fine-tuned BERT model. The good thing is, it worked, but maintaining it felt like fixing a CSS file from 2015, where you touch one rule and something unrelated breaks on a page you haven't visited in months.\nSo when GPT-4 came along, we tried it.\nNot going to lie, it was kind of incredible. Edge cases that were driving BERT crazy for months, formats we'd never seen before, and documents with inconsistent sections were all taken care of cleanly by GPT-4.\nThe team demo went well. I mean, someone even said \"wow\" out loud.\nI sent my manager a text that evening: _\"think we've cracked the extraction problem.\"_ Sent with confidence.\nTwo weeks after we deployed it, the failures started.\n## The Problem With \"Mostly Consistent\"\nFrom my experience, GPT-4 is capable and non-deterministic.\nFor a lot of use cases, the non-determinism doesn't matter. For a nightly batch pipeline feeding a data warehouse, it matters a lot.\nAt `temperature=0` you get _mostly_ consistent outputs. In a CI/CD context, \"mostly\" means \"it'll break on a Friday.\"\nThe failures weren't dramatic either; if not, that would've been easier to debug. GPT-4 was not hallucinating fields or returning garbage.\nIt was doing subtle things. `\"dataset_source\"` one night, `\"datasetSource\"` the next, `\"source_dataset\"` the night after. Markdown code fences around the JSON even though we'd told it not to, across every version of the prompt. Numbers returned as strings. JSON null returned as the Python string `\"None\"`; I spent longer than I'd like to admit staring at that one.\nEach failure followed the same ritual. \nPydantic catches it downstream, pipeline fails, I confirm it's another formatting quirk, then I get to work immediately to re-run manually. It passes, then about three days later, a different quirk all over again.\nSo I started keeping a log. Six weeks of it:\n  * **23 pipeline failures** from GPT-4 output inconsistency\n  * **~18 minutes** average to diagnose and re-run\n  * **0 actual bugs** in the pipeline code\n\n\nZero real bugs. Every single failure was the model being subtly different from what it had been the night before. That's the number that made me stop defending the setup.\n## What I Tried Before Admitting the Real Problem\n### Prompting \nSeven rewrites over two days.\nI didn't even realize it had gotten that much. I just wanted a good fix. And it didn't even stop there. \nI tried all-caps instructions, few-shot examples, and counter-examples with a \"Do NOT do this\" header. I even tried adding a reminder at the very end of the user message as a last-ditch nudge, as if the model would get to that line and think _oh right, JSON only, nearly forgot._\nAt one point, the same instruction appeared in three different places in the same prompt, and I genuinely thought that might help. Failures continued, unchanged.\n### A cleanup parser\nWhen prompting failed, I wrote code to clean up any mess that GPT-4 threw back at me. \nStrip markdown fences, find the JSON object if it was buried in text, and log the raw output when nothing worked.\nThis method actually worked for about a week, which was enough time for me to feel good about it.\nAnd then GPT-4 began returning structurally valid JSON with wrong key names, camelCase instead of snake_case. The parser passed it through fine, and the error surfaced three steps later.\nI was playing whack-a-mole with a model that had infinite moles.\n`**response_format**`**+**`**temperature=0**`**.** OpenAI's `response_format={\"type\": \"json_object\"}` combined with `temperature=0` brought failures from 23 down to about 9. Meaningful progress. Still not zero, and \"nine random failures per six weeks\" is not a pipeline property I could defend.\n### Function calling\nThis was the one that almost worked. Forcing output through a typed schema contract genuinely tightened things.\nI stopped checking logs every morning, stopped bracing for the Slack notification. Told someone on the team it was stable. Which, if you work in software, you know is the fastest way to jinx something.\npython\n\n```\nfunctions = [\n    {\n        \"name\": \"extract_document_metadata\",\n        \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n                \"methodology_type\": {\n                    \"type\": \"string\",\n                    \"enum\": [\"experimental\", \"observational\", \"review\", \"simulation\", \"mixed\"]\n                },\n                \"dataset_source\": {\"type\": \"string\"},\n                \"primary_metric\": {\"type\": \"string\"},\n                \"year\": {\"type\": \"integer\"},\n                \"confidence_score\": {\"type\": \"number\", \"minimum\": 0, \"maximum\": 1}\n            },\n            \"required\": [\"methodology_type\", \"dataset_source\", \"year\"]\n        }\n    }\n]\n\nresponse = openai_client.chat.completions.create(\n    model=\"gpt-4-turbo\",\n    messages=messages,\n    functions=functions,\n    function_call={\"name\": \"extract_document_metadata\"},\n    temperature=0\n)\n```\n\nThen one Tuesday, the OpenAI API had a 20-minute disruption. Pipeline failed hard; it wasn't a model issue, just a network dependency we'd never properly reckoned with.\nWe couldn't version-lock the model, couldn't run offline, couldn't answer \"what changed between the run that worked and the run that didn't?\" because the model on the other end wasn't ours.\nSitting there waiting for someone else's API to recover, I finally asked the question I should have asked months earlier: Does this specific job actually need a frontier model?\n## The Local Models Are Better Than I Expected\nI went in fully expecting to spend a day confirming they weren't good enough. I had a conclusion ready before I'd run a single test: _tried it, quality wasn't there, staying on GPT-4 with better retry logic._ A story where the first decision was still defensible.\nIt took maybe three hours to realize I'd been thinking about this wrong.\nDocument extraction into a fixed schema isn't actually a hard task for a language model. Not in the way that makes a frontier model necessary. There's no reasoning involved, no synthesis, no need for the breadth of world knowledge that makes GPT-4 worth what it costs. \nWhat it actually is: structured reading comprehension. The model reads a document and fills in fields. A well-trained 7B model does this fine. And with the right setup, specifically, seeded inference, it does it _identically_ every single run.\nI ran four models against 50 documents I'd manually annotated:\n  * **Phi-3-mini (3.8B)** : Better instruction following than I expected for a 3.8B model, but it fell apart on anything over about 3,000 tokens. I nearly picked this before I looked at the longer doc results.\n  * **Mistral 7B Instruct** : Solid everywhere, no real surprises in either direction. The Toyota Camry of local models. You'd be fine with this.\n  * **Qwen2.5-7B-Instruct** : This one won clearly. Best structured output consistency of the four by a margin wide enough that it wasn't a close call.\n  * **Llama 3.2 3B Instruct** : This is fast, but the extraction quality dropped off enough on edge cases that I wouldn't run it on production data without a lot more validation work first.\n\n\nQwen2.5 and Mistral both hit 90–95% accuracy on the annotated set, lower than GPT-4 on genuinely ambiguous documents, yes. \nBut I ran Qwen2.5 on the same 20 documents three times and went back to diff the results. No diffs. Same JSON, same values, same field order every run. \nAfter six weeks of failures I couldn't predict, that felt almost too clean to be real.\n## Before and After\nThe GPT-4 extractor at its most stable, not the original prototype, the version after months of defensive code had accumulated around it:\npython\n\n```\n# extractor_gpt4.py\nimport os\nimport json\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=os.environ[\"OPENAI_API_KEY\"])\n\nSYSTEM_PROMPT = \"\"\"You are a research document metadata extractor.\nGiven the text of a research document, extract the specified metadata fields.\nBe precise. If you are unsure about a field, use your best judgment based on context.\"\"\"\n\nEXTRACTION_SCHEMA = {\n    \"name\": \"extract_document_metadata\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"methodology_type\": {\n                \"type\": \"string\",\n                \"enum\": [\"experimental\", \"observational\", \"review\", \"simulation\", \"mixed\"]\n            },\n            \"dataset_source\": {\"type\": \"string\"},\n            \"primary_metric\": {\"type\": \"string\"},\n            \"year\": {\"type\": \"integer\"},\n            \"confidence_score\": {\"type\": \"number\"}\n        },\n        \"required\": [\"methodology_type\", \"dataset_source\", \"year\"]\n    }\n}\n\ndef extract_metadata_gpt4(document_text: str) -> dict:\n    response = client.chat.completions.create(\n        model=\"gpt-4-turbo\",\n        messages=[\n            {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n            {\"role\": \"user\", \"content\": f\"Extract metadata from this document:\\n\\n{document_text[:8000]}\"}\n        ],\n        functions=[EXTRACTION_SCHEMA],\n        function_call={\"name\": \"extract_document_metadata\"},\n        temperature=0,\n        timeout=30\n    )\n    try:\n        args = response.choices[0].message.function_call.arguments\n        return json.loads(args)\n    except (AttributeError, json.JSONDecodeError) as e:\n        raise ValueError(f\"Failed to parse GPT-4 response: {e}\")\n```\n\nLatency: 3.5–5.8s per document. Cost: ~$0.04 per call. Failure rate: ~6% after all the fixes.\nFor the replacement, I chose Ollama. A colleague mentioned it offhandedly, and the docs looked reasonable at 11 pm, which is honestly how a lot of infrastructure decisions get made. The REST API is close enough to OpenAI's that the swap took about an hour:\npython\n\n```\n# extractor_slm.py\nimport json\nimport logging\nimport os\nimport re\n\nimport requests\n\nlogger = logging.getLogger(__name__)\n\nOLLAMA_URL = os.getenv(\"OLLAMA_URL\", \"http://localhost:11434\")\nMODEL = os.getenv(\"MODEL_NAME\", \"qwen2.5:7b-instruct-q4_K_M\")\n\n# Tested empirically: 7B q4 on a T4 takes ~8s on cold first token,\n# then ~1.5s per doc chunk. 45s covers bad days.\n_TIMEOUT = 45\n\n_SYSTEM_PROMPT = \"\"\"\\\nYou are a metadata extractor for research documents.\nReturn ONLY a JSON object — no explanation, no markdown, no surrounding text.\n\nFields to extract:\n- methodology_type (required): one of experimental | observational | review | simulation | mixed\n- dataset_source (required): where the data came from\n- year (required): integer\n- primary_metric: main eval metric if present\n- confidence_score: your confidence 0.0–1.0\n\nOutput example:\n{\"methodology_type\": \"experimental\", \"dataset_source\": \"ImageNet\", \"year\": 2022, \"primary_metric\": \"top-1 accuracy\", \"confidence_score\": 0.95}\n\"\"\"\n\ndef call_ollama(doc_text: str) -> dict:\n    payload = {\n        \"model\": MODEL,\n        \"messages\": [\n            {\"role\": \"system\", \"content\": _SYSTEM_PROMPT},\n            {\"role\": \"user\", \"content\": doc_text[:6000]},\n        ],\n        \"stream\": False,\n        \"options\": {\n            \"temperature\": 0,\n            \"seed\": 42,  # determinism — this is the whole point\n        },\n    }\n\n    try:\n        resp = requests.post(f\"{OLLAMA_URL}/api/chat\", json=payload, timeout=_TIMEOUT)\n        resp.raise_for_status()\n    except requests.exceptions.Timeout:\n        raise RuntimeError(f\"Ollama timed out after {_TIMEOUT}s — is the model loaded?\")\n    except requests.exceptions.ConnectionError:\n        raise RuntimeError(f\"Can't reach Ollama at {OLLAMA_URL} — is the container running?\")\n\n    raw = resp.json()[\"message\"][\"content\"].strip()\n    cleaned = re.sub(r\"^```(?:json)?\\s*|\\s*```$\", \"\", raw).strip()\n\n    try:\n        return json.loads(cleaned)\n    except json.JSONDecodeError as e:\n        logger.error(\"Failed to parse model output: %s\\nRaw was: %s\", e, raw[:400])\n        raise\n```\n\nThe `seed: 42` in options is what actually delivers the determinism. Ollama supports seeded generation; same input, same seed, same output, every time, not approximately. `temperature=0` with a hosted API implied this but never guaranteed it because you're not controlling the runtime. Locally, you are.\n### Getting It Into GitHub Actions\nTwo things that aren't obvious until they bite you. First: GitHub Actions service containers are [network-isolated from the runner](https://docs.github.com/en/actions/using-containerized-services/about-service-containers). You can't `docker exec` into them; the model pull has to go through the REST API. Second: cache the model. A cold pull is 4.7GB and adds 3–4 minutes to every job.\nyaml\n\n```\n# The non-obvious parts — the rest is standard Actions boilerplate\n- name: Cache Ollama model\n  uses: actions/cache@v4\n  with:\n    path: ~/.ollama/models\n    key: ollama-qwen2.5-7b-q4\n\n- name: Pull SLM model\n  # Can't docker exec into service containers — use the API\n  run: |\n    curl -s http://localhost:11434/api/pull \\\n      -d '{\"name\": \"qwen2.5:7b-instruct-q4_K_M\"}' \\\n      --max-time 300\n\n- name: Warm up model\n  run: |\n    curl -s http://localhost:11434/api/generate \\\n      -d '{\"model\": \"qwen2.5:7b-instruct-q4_K_M\", \"prompt\": \"hello\", \"stream\": false}' \\\n      > /dev/null\n\n- name: Run ingestion pipeline\n  run: python pipeline/run_ingestion.py\n  env:\n    OLLAMA_URL: \"http://localhost:11434\"\n    MODEL_NAME: \"qwen2.5:7b-instruct-q4_K_M\"\n```\n\n### The Capitalization Bug I Spent Too Long On\nQwen 2.5 occasionally returns `\"Experimental\"` with a capital E despite explicit instructions not to. The `Literal` type check rejects it, and you get a validation failure on a document the model actually extracted correctly.\nI spent an embarrassingly long time on this because the error message just said \"invalid value,\" and my first instinct was to look at the document, then the extractor, before I finally looked at the validator and saw it.\nA `normalize_methodology` validator with `mode=\"before\"` fixes it before the type check runs:\npython\n\n```\n# pipeline/validation.py\nfrom typing import Literal, Optional\nfrom pydantic import BaseModel, Field, field_validator\n\nclass DocMetadata(BaseModel):\n    methodology_type: Literal[\"experimental\", \"observational\", \"review\", \"simulation\", \"mixed\"]\n    dataset_source: str = Field(min_length=3)\n    year: int = Field(ge=1950, le=2030)\n    primary_metric: Optional[str] = None\n    confidence_score: Optional[float] = Field(default=None, ge=0.0, le=1.0)\n\n    @field_validator(\"dataset_source\")\n    @classmethod\n    def strip_whitespace(cls, v: str) -> str:\n        return v.strip()\n\n    @field_validator(\"methodology_type\", mode=\"before\")\n    @classmethod\n    def normalize_methodology(cls, v: str) -> str:\n        # Qwen occasionally returns \"Experimental\" despite instructions\n        return v.lower().strip() if isinstance(v, str) else v\n```\n\n···\nI want to be straight about the tradeoffs because I hate articles that don't. \nGPT-4 is genuinely better on ambiguous documents, non-English text, anything requiring real inference. There's a class of document in our corpus, older PDFs, reports mixing multiple languages, non-standard formats, where the SLM stumbles and GPT-4 wouldn't. \nWe flag those separately and route them to a review queue. That's not seamless, but it's honest. The SLM does the 90% it's good at and doesn't get asked to do the rest.\nThe setup is also more work than `pip install openai`. First-time configuration takes a real afternoon. And Ollama doesn't auto-update models, so version management is my problem now. I've genuinely made peace with that, knowing exactly what model version ran last night and the night before is the whole point.\n···\n## What I Think Now\nThe morning the pipeline first ran clean, no notification, no re-run, just a log file showing 312 documents processed in 8.4 minutes, I checked it twice, then ran it manually again. \nI'd spent six weeks half-expecting an alert before I'd even finished my coffee. Watching it pass quietly felt genuinely strange.\nThe failure wasn't using GPT-4. The failure was treating a probabilistic system like a deterministic function. `temperature=0` reduces variance. It doesn't eliminate it. \nI understood this in theory the whole time. It took 23 failures to understand it in a way that changed how I build things.\nIf you're running an LLM in an automated pipeline and things keep breaking in ways you can't reproduce, it might not be your code or your prompts. It might be the nature of what you're depending on. \nA local model on hardware you control, seeded for determinism, is a genuinely different class of tool for this kind of job. Not better at everything but better at being the same thing twice. And for a pipeline that runs unattended every night, that's most of what actually matters.\n···\n## Before you go!\nIf this was useful, I write more about the messy reality behind building with AI, what breaks in production, what actually works, and the tradeoffs behind the fixes.\nYou can **subscribe to my**[**newsletter**](https://thatcsguy.substack.com/?r=79lldp&utm_campaign=pub-share-checklist) if you'd like more of that.\n### Connect With Me\n  * [LinkedIn](http://www.linkedin.com/in/nwekebenjamin)\n  * [Substack](https://thatcsguy.substack.com/)\n\n\n···\n### References\n  * OpenAI. _Reproducible outputs_. Retrieved from <https://platform.openai.com/docs/guides/text-generation/reproducible-outputs>\n  * Ollama. _API documentation_. Retrieved from <https://github.com/ollama/ollama/blob/main/docs/api.md>\n\n\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-4ad89a2fc5e69dde007d.jpeg)\nWritten By\n## [Benjamin Nweke](https://towardsdatascience.com/author/benjaminnweke/)\nBenjamin Nweke is a Computer Science graduate and technical writer exploring AI, machine learning, and data science. He writes about building reliable AI systems, model evaluation, agentic architectures, and the practical challenges that appear when machine learning moves beyond experiments. His work focuses on making technically complex ideas clear, useful, and grounded in real implementation.\n[See all from Benjamin Nweke](https://towardsdatascience.com/author/benjaminnweke/)\n[Deep Dives](https://towardsdatascience.com/tag/deep-dives/)[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Document Processing](https://towardsdatascience.com/tag/document-processing/)[Llm](https://towardsdatascience.com/tag/llm/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fi-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing%2F&title=I%20Replaced%20GPT-4%20with%20a%20Local%20SLM%20and%20My%20CI%2FCD%20Pipeline%20Stopped%20Failing)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fi-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing%2F&title=I%20Replaced%20GPT-4%20with%20a%20Local%20SLM%20and%20My%20CI%2FCD%20Pipeline%20Stopped%20Failing)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fi-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing%2F&text=I%20Replaced%20GPT-4%20with%20a%20Local%20SLM%20and%20My%20CI%2FCD%20Pipeline%20Stopped%20Failing)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](../images/001.png)](https://towardsdatascience.com/how-to-build-a-super-easy-handwritten-numbers-identifier-463850ab7d68/)\n## [How to build a super easy handwritten numbers identifier](https://towardsdatascience.com/how-to-build-a-super-easy-handwritten-numbers-identifier-463850ab7d68/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nA beginner friendly use of cosine similarity\n[Jose Manuel Napoles-Duarte](https://towardsdatascience.com/author/jnapoles/)\nJuly 8, 20213 min read\n  * [![](../images/002.jpeg)](https://towardsdatascience.com/say-goodbye-to-sql-headaches-with-this-python-tool-75099f5ff33d/)\n## [ChatGPT Doesn't Understand SQL All the Time But This Python Tool Does](https://towardsdatascience.com/say-goodbye-to-sql-headaches-with-this-python-tool-75099f5ff33d/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nSQLGlot - A one-stop SQL parsing and manipulation Python Library\n[Christopher Tao](https://towardsdatascience.com/author/christophertao/)\nJanuary 15, 202410 min read\n  * [![](../images/003.png)](https://towardsdatascience.com/word-embeddings-and-embedding-projector-of-tensorflow-c946b98c9b3f/)\n## [Word Embeddings and Embedding Projector of TensorFlow](https://towardsdatascience.com/word-embeddings-and-embedding-projector-of-tensorflow-c946b98c9b3f/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nTheoretical explanation and a practical example.\n[Soner Yıldırım](https://towardsdatascience.com/author/sonery/)\nMay 24, 20204 min read\n  * [![](../images/004.jpg)](https://towardsdatascience.com/essential-things-you-need-to-know-about-f1-score-dbd973bf1a3/)\n## [Essential Things You Need to Know About F1-Score](https://towardsdatascience.com/essential-things-you-need-to-know-about-f1-score-dbd973bf1a3/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nLearn about the key fundamentals of F1-score, one of the most important evaluation metrics in machine learning.\n[Zeya LT](https://towardsdatascience.com/author/zeyalt/)\nNovember 23, 20219 min read\n  * [![](../images/005.jpeg)](https://towardsdatascience.com/ai-agent-workflows-a-complete-guide-on-whether-to-build-with-langgraph-or-langchain-117025509fa0/)\n## [AI Agent Workflows: A Complete Guide on Whether to Build With LangGraph or LangChain](https://towardsdatascience.com/ai-agent-workflows-a-complete-guide-on-whether-to-build-with-langgraph-or-langchain-117025509fa0/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nLangChain and LangGraph - A deep dive on key building blocks, how they handle core pieces of their functionality, and deciding between...\n[Sandi Besen](https://towardsdatascience.com/author/sandibesen/)\nOctober 25, 202411 min read\n  * [![](../images/006.jpg)](https://towardsdatascience.com/sampling-isnt-enough-profile-your-ml-data-instead-6a28fcfb2bd4/)\n## [Sampling isn't enough, profile your ML data instead](https://towardsdatascience.com/sampling-isnt-enough-profile-your-ml-data-instead-6a28fcfb2bd4/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nProduction logging approaches for AI and data pipelines\n[Isaac Backus](https://towardsdatascience.com/author/ibackus8/)\nSeptember 22, 20208 min read\n  * [![](../images/007.jpg)](https://towardsdatascience.com/graph-engineering-isnt-about-more-connections-its-about-which-ones-get-used/)\n## [Graph Engineering Isn’t About More Connections — It’s About Which Ones Get Used](https://towardsdatascience.com/graph-engineering-isnt-about-more-connections-its-about-which-ones-get-used/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nI expected adding more communication pathways between agents to dramatically change network performance. It didn't. Figuring out why turned into a more useful article than the one I set out to write.\n[Emmimal P Alexander](https://towardsdatascience.com/author/emmimalp.alexander/)\nAugust 18, 202614 min read\n  * [![](../images/008.jpg)](https://towardsdatascience.com/significance-or-hypothesis-tests-with-python-7ed35e9ac9b6/)\n## [Significance or Hypothesis Tests with Python](https://towardsdatascience.com/significance-or-hypothesis-tests-with-python-7ed35e9ac9b6/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nCollege Statistics with Python\n[Luís Roque](https://towardsdatascience.com/author/luisroque/)\nJuly 1, 202113 min read\n  * [![](../images/009.jpeg)](https://towardsdatascience.com/using-ai-you-may-want-to-start-conducting-human-rights-impact-assessments-hria-89ec0fdbce52/)\n## [Using AI? You may want to start conducting Human Rights Impact Assessments (HRIA)](https://towardsdatascience.com/using-ai-you-may-want-to-start-conducting-human-rights-impact-assessments-hria-89ec0fdbce52/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nIn a recent post I offered a few predictions on how the Biden Administration may change the AI playing field. The main takeaway is that AI...\n[Natalia Modjeska](https://towardsdatascience.com/author/nmodjeska/)\nMarch 4, 20219 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n",
  "metadata": {
    "title": "I Replaced GPT-4 with a Local SLM and My CI/CD Pipeline Stopped Failing | Towards Data Science",
    "description": "The hidden cost of probabilistic outputs in systems that demand reliability",
    "keywords": null,
    "author": "Benjamin Nweke",
    "og:locale": "en_US",
    "og:type": "article",
    "og:title": "I Replaced GPT-4 with a Local SLM and My CI/CD Pipeline Stopped Failing",
    "og:description": "The hidden cost of probabilistic outputs in systems that demand reliability",
    "og:url": "https://towardsdatascience.com/i-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing/",
    "og:site_name": "Towards Data Science",
    "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/04/Gemini_Generated_Image_j19tm3j19tm3j19t-scaled-1.jpg",
    "og:image:width": "2560",
    "og:image:height": "1396",
    "og:image:type": "image/jpeg",
    "twitter:card": "summary_large_image",
    "twitter:site": "@TDataScience",
    "twitter:creator": "@TDataScience",
    "twitter:label1": "Written by",
    "twitter:data1": "Benjamin Nweke",
    "twitter:label2": "Est. reading time",
    "twitter:data2": "11 min read",
    "article:published_time": "2026-04-21T07:30:06.000Z",
    "article:modified_time": "2026-04-21T07:30:06.000Z",
    "article:author": "https://towardsdatascience.com/author/benjaminnweke/"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 202,
    "error": ""
  },
  "code_blocks": [
    {
      "language": "txt",
      "file": "attachments/ca4739e88720/code/block_000.txt",
      "chars": 878
    },
    {
      "language": "txt",
      "file": "attachments/ca4739e88720/code/block_001.txt",
      "chars": 1648
    },
    {
      "language": "txt",
      "file": "attachments/ca4739e88720/code/block_002.txt",
      "chars": 1850
    },
    {
      "language": "$\",",
      "file": "attachments/ca4739e88720/code/block_003.txt",
      "chars": 181
    },
    {
      "language": "txt",
      "file": "attachments/ca4739e88720/code/block_004.txt",
      "chars": 750
    },
    {
      "language": "txt",
      "file": "attachments/ca4739e88720/code/block_005.txt",
      "chars": 834
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn-cookieyes.com/assets/images/revisit.svg",
      "bytes": 2339,
      "reused": false,
      "content_type": "image/svg+xml",
      "local_path": "attachments/ca4739e88720/images/body-df29beacd7496220cc94.svg",
      "src": "https://cdn-cookieyes.com/assets/images/revisit.svg",
      "alt": "Revisit consent button",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://cdn-cookieyes.com/assets/images/close.svg",
      "bytes": 1345,
      "reused": false,
      "content_type": "image/svg+xml",
      "local_path": "attachments/ca4739e88720/images/body-2159bb67f3de8ad870eb.svg",
      "src": "https://cdn-cookieyes.com/assets/images/close.svg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/04/Gemini_Generated_Image_j19tm3j19tm3j19t-scaled-1.jpg",
      "bytes": 477579,
      "reused": true,
      "local_path": "attachments/ca4739e88720/images/000.jpg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/04/Gemini_Generated_Image_j19tm3j19tm3j19t-scaled-1.jpg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/1787657271062_x78p7k.jpeg",
      "bytes": 194410,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/ca4739e88720/images/body-4ad89a2fc5e69dde007d.jpeg",
      "src": "https://assets.insightmediagroup.io/media/1787657271062_x78p7k.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/07/18VLgJDlgx9RRIDVNkXwVDQ.png",
      "bytes": 45110,
      "reused": true,
      "local_path": "attachments/ca4739e88720/images/001.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/07/18VLgJDlgx9RRIDVNkXwVDQ.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/01/16wUhgvKwtpZC_UOsW9lWIQ.jpeg",
      "bytes": 417062,
      "reused": true,
      "local_path": "attachments/ca4739e88720/images/002.jpeg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/01/16wUhgvKwtpZC_UOsW9lWIQ.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/05/1WTt3WXi7tiwjjfo0Scdvgg.png",
      "bytes": 528058,
      "reused": true,
      "local_path": "attachments/ca4739e88720/images/003.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/05/1WTt3WXi7tiwjjfo0Scdvgg.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/11/0cRwr5KUl3ZxtdGNQ-scaled.jpg",
      "bytes": 378995,
      "reused": true,
      "local_path": "attachments/ca4739e88720/images/004.jpg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/11/0cRwr5KUl3ZxtdGNQ-scaled.jpg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/10/1htbJaFZFAmzhPqcMh0sQQQ.jpeg",
      "bytes": 127500,
      "reused": true,
      "local_path": "attachments/ca4739e88720/images/005.jpeg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/10/1htbJaFZFAmzhPqcMh0sQQQ.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/09/0YzXXZUoS6BaWK4Uo-scaled.jpg",
      "bytes": 404873,
      "reused": true,
      "local_path": "attachments/ca4739e88720/images/006.jpg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/09/0YzXXZUoS6BaWK4Uo-scaled.jpg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/08/Graph-Engineering.jpg",
      "bytes": 321864,
      "reused": true,
      "local_path": "attachments/ca4739e88720/images/007.jpg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/08/Graph-Engineering.jpg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/07/0sNu-59Hzn1eA8P8K.jpg",
      "bytes": 88985,
      "reused": true,
      "local_path": "attachments/ca4739e88720/images/008.jpg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/07/0sNu-59Hzn1eA8P8K.jpg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1nFW3OvWQqSx-DFVzPQ_Blw-scaled.jpeg",
      "bytes": 907586,
      "reused": true,
      "local_path": "attachments/ca4739e88720/images/009.jpeg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1nFW3OvWQqSx-DFVzPQ_Blw-scaled.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/c/TowardsDataScience",
      "kind": "channel_reference",
      "status": "reference_only"
    }
  ],
  "links": [
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/latest/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/editors-pick/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/the-variable/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/shipai/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/submissions/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/artificial-intelligence/large-language-models/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/benjaminnweke/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/deep-dives/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/artificial-intelligence/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/document-processing/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/llm/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/machine-learning/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/how-to-build-a-super-easy-handwritten-numbers-identifier-463850ab7d68/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/artificial-intelligence/machine-learning/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/jnapoles/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/say-goodbye-to-sql-headaches-with-this-python-tool-75099f5ff33d/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/artificial-intelligence/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/christophertao/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/word-embeddings-and-embedding-projector-of-tensorflow-c946b98c9b3f/"
    },
    {
      "type": "external",
      "href": "https://write.insightmediagroup.io"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/company/towards-data-science/"
    },
    {
      "type": "external",
      "href": "https://x.com/TDataScience"
    },
    {
      "type": "external",
      "href": "https://docs.github.com/en/actions/using-containerized-services/about-service-containers"
    },
    {
      "type": "external",
      "href": "https://thatcsguy.substack.com/?r=79lldp"
    },
    {
      "type": "external",
      "href": "http://www.linkedin.com/in/nwekebenjamin"
    },
    {
      "type": "external",
      "href": "https://thatcsguy.substack.com/"
    },
    {
      "type": "external",
      "href": "https://platform.openai.com/docs/guides/text-generation/reproducible-outputs"
    },
    {
      "type": "external",
      "href": "https://github.com/ollama/ollama/blob/main/docs/api.md"
    },
    {
      "type": "external",
      "href": "https://www.facebook.com/sharer/sharer.php?title=I+Replaced+GPT-4+with+a+Local+SLM+and+My+CI%2FCD+Pipeline+Stopped+Failing&u=https%3A%2F%2Ftowardsdatascience.com%2Fi-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing%2F"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/shareArticle?mini=true&title=I+Replaced+GPT-4+with+a+Local+SLM+and+My+CI%2FCD+Pipeline+Stopped+Failing&url=https%3A%2F%2Ftowardsdatascience.com%2Fi-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing%2F"
    },
    {
      "type": "external",
      "href": "https://x.com/share?text=I+Replaced+GPT-4+with+a+Local+SLM+and+My+CI%2FCD+Pipeline+Stopped+Failing&url=https%3A%2F%2Ftowardsdatascience.com%2Fi-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing%2F"
    },
    {
      "type": "external",
      "href": "https://www.youtube.com/c/TowardsDataScience"
    },
    {
      "type": "external",
      "href": "https://www.threads.net/@towardsdatascience"
    },
    {
      "type": "external",
      "href": "https://bsky.app/profile/towardsdatascience.com"
    },
    {
      "type": "external",
      "href": "https://thenewstack.io"
    },
    {
      "type": "external",
      "href": "https://roadmap.sh"
    },
    {
      "type": "external",
      "href": "https://insightmediagroup.io/creator-network"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-20T17:24:44Z",
  "source_url": "https://towardsdatascience.com/i-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing/",
  "delivery_status": "unverified",
  "discovery_status": "http_403",
  "missing_reason": "http_403",
  "repair_error": "",
  "actual_video_files": [],
  "expected_saved_body_images": 13,
  "image_discovery": "saved_markdown",
  "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n  * Cookie\nBCTempID\n  * Duration\n10 minutes\n  * Description\nNo description available.\n\n\n  * Cookie\n__cf_bm\n  * Duration\n1 hour\n  * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n  * Cookie\nAWSALBCORS\n  * Duration\n7 days\n  * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n  * Cookie\n_cfuvid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n  * Cookie\nli_gc\n  * Duration\n6 months\n  * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n  * Cookie\n__hssrc\n  * Duration\nsession\n  * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n  * Cookie\n__hssc\n  * Duration\n1 hour\n  * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n  * Cookie\nwpEmojiSettingsSupports\n  * Duration\nsession\n  * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n  * Cookie\nBCSessionID\n  * Duration\n1 year 1 month 4 days\n  * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n  * Cookie\n_octo\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nlogged_in\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\n__Secure-YEC\n  * Duration\npast\n  * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n  * Cookie\n__eoi\n  * Duration\n6 months\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTGCORS\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\nlogin-status-p\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTG\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\ncsrf_token\n  * Duration\nsession\n  * Description\nNo description available.\n\n\n  * Cookie\ntoken_v2\n  * Duration\n1 day\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nD\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nPHPSESSID\n  * Duration\nsession\n  * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n  * Cookie\nVISITOR_PRIVACY_METADATA\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n  * Cookie\ncookietest\n  * Duration\nsession\n  * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n  * Cookie\n__Host-airtable-session\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\n__Host-airtable-session.sig\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\nm\n  * Duration\n1 year 1 month 4 days\n  * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n  * Cookie\nBIGipServer*\n  * Duration\nsession\n  * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n  * Cookie\n__cfruid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n  * Cookie\n_GRECAPTCHA\n  * Duration\n6 months\n  * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n  * Cookie\n__Secure-YNID\n  * Duration\n6 months\n  * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n  * Cookie\ncookieyes-consent\n  * Duration\n1 year\n  * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n  * Cookie\nlidc\n  * Duration\n1 day\n  * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n  * Cookie\nbrw\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nbrwConsent\n  * Duration\n5 minutes\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nWMF-Uniq\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nloom_anon_comment\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nloom_referral_video\n  * Duration\nsession\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nVISITOR_INFO1_LIVE\n  * Duration\n6 months\n  * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n  * Cookie\nyt-remote-connected-devices\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n  * Duration\nNever Expires\n  * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n  * Cookie\nyt-remote-device-id\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-name\n  * Duration\nsession\n  * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\nyt-remote-fast-check-period\n  * Duration\nsession\n  * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-app\n  * Duration\nsession\n  * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-available\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-installed\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\ncp_session\n  * Duration\n3 months\n  * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n  * Cookie\nloid\n  * Duration\n1 year 1 month 4 days\n  * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n  * Cookie\n__hstc\n  * Duration\n6 months\n  * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n  * Cookie\nhubspotutk\n  * Duration\n6 months\n  * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n  * Cookie\n_ga\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n  * Cookie\n_ga_*\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n  * Cookie\n__Host-psifi.analyticsTrace\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n__Host-psifi.analyticsTraceV2\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n_gh_sess\n  * Duration\nsession\n  * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n  * Cookie\nYSC\n  * Duration\nsession\n  * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n  * Cookie\najs_anonymous_id\n  * Duration\n1 year\n  * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n  * Cookie\nvuid\n  * Duration\n1 year 1 month 4 days\n  * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n  * Cookie\nAWSALB\n  * Duration\n7 days\n  * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n  * Cookie\nacq\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nacq.sig\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nptc\n  * Duration\n2 years\n  * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n  * Cookie\nmuc_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n  * Cookie\nguest_id_marketing\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\nguest_id_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\npersonalization_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n  * Cookie\nguest_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n  * Cookie\nbcookie\n  * Duration\n1 year\n  * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n  * Cookie\n__Secure-ROLLOUT_TOKEN\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n  * Cookie\nyt.innertube::nextId\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nyt.innertube::requests\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nsession_tracker\n  * Duration\nsession\n  * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n  * Cookie\nedgebucket\n  * Duration\nsession\n  * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n  * Cookie\ndid\n  * Duration\n1 year\n  * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/)\n# I Replaced GPT-4 with a Local SLM and My CI/CD Pipeline Stopped Failing\nThe hidden cost of probabilistic outputs in systems that demand reliability\n[Benjamin Nweke](https://towardsdatascience.com/author/benjaminnweke/)\nApril 21, 202611 min read\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/04/Gemini_Generated_Image_j19tm3j19tm3j19t-scaled-1.jpg)\nImage by author (Generated with Nano Banana Pro)\nI was on my seventh rewrite of the same system prompt.\n_You MUST return ONLY valid JSON. No markdown. No code fences. No explanation. JUST the JSON object._\nI had written MUST in all-caps. To a language model. As if emphasis would work on something that doesn't have feelings or, apparently, a consistent definition of \"valid JSON.\"\nIt didn't work. Here's what did.\n## How GPT-4 Ended Up in a Nightly Batch Job\nOur team consumes research documents, such as PDFs and plain text, and occasionally those pesky semi-structured reports that some vendor clearly exported from a spreadsheet they were very proud of. And part of that pipeline classifies them and extracts structured fields before anything touches the data warehouse. Methodology type, dataset source, key metrics.\nThis sounds like a solved problem. It usually is, until there are about forty types of methods listed and the documents stop looking anything like the ones you trained on.\nFor a while, we handled this using regex, rule-based extractors, and a fine-tuned BERT model. The good thing is, it worked, but maintaining it felt like fixing a CSS file from 2015, where you touch one rule and something unrelated breaks on a page you haven't visited in months.\nSo when GPT-4 came along, we tried it.\nNot going to lie, it was kind of incredible. Edge cases that were driving BERT crazy for months, formats we'd never seen before, and documents with inconsistent sections were all taken care of cleanly by GPT-4.\nThe team demo went well. I mean, someone even said \"wow\" out loud.\nI sent my manager a text that evening: _\"think we've cracked the extraction problem.\"_ Sent with confidence.\nTwo weeks after we deployed it, the failures started.\n## The Problem With \"Mostly Consistent\"\nFrom my experience, GPT-4 is capable and non-deterministic.\nFor a lot of use cases, the non-determinism doesn't matter. For a nightly batch pipeline feeding a data warehouse, it matters a lot.\nAt `temperature=0` you get _mostly_ consistent outputs. In a CI/CD context, \"mostly\" means \"it'll break on a Friday.\"\nThe failures weren't dramatic either; if not, that would've been easier to debug. GPT-4 was not hallucinating fields or returning garbage.\nIt was doing subtle things. `\"dataset_source\"` one night, `\"datasetSource\"` the next, `\"source_dataset\"` the night after. Markdown code fences around the JSON even though we'd told it not to, across every version of the prompt. Numbers returned as strings. JSON null returned as the Python string `\"None\"`; I spent longer than I'd like to admit staring at that one.\nEach failure followed the same ritual. \nPydantic catches it downstream, pipeline fails, I confirm it's another formatting quirk, then I get to work immediately to re-run manually. It passes, then about three days later, a different quirk all over again.\nSo I started keeping a log. Six weeks of it:\n  * **23 pipeline failures** from GPT-4 output inconsistency\n  * **~18 minutes** average to diagnose and re-run\n  * **0 actual bugs** in the pipeline code\n\n\nZero real bugs. Every single failure was the model being subtly different from what it had been the night before. That's the number that made me stop defending the setup.\n## What I Tried Before Admitting the Real Problem\n### Prompting \nSeven rewrites over two days.\nI didn't even realize it had gotten that much. I just wanted a good fix. And it didn't even stop there. \nI tried all-caps instructions, few-shot examples, and counter-examples with a \"Do NOT do this\" header. I even tried adding a reminder at the very end of the user message as a last-ditch nudge, as if the model would get to that line and think _oh right, JSON only, nearly forgot._\nAt one point, the same instruction appeared in three different places in the same prompt, and I genuinely thought that might help. Failures continued, unchanged.\n### A cleanup parser\nWhen prompting failed, I wrote code to clean up any mess that GPT-4 threw back at me. \nStrip markdown fences, find the JSON object if it was buried in text, and log the raw output when nothing worked.\nThis method actually worked for about a week, which was enough time for me to feel good about it.\nAnd then GPT-4 began returning structurally valid JSON with wrong key names, camelCase instead of snake_case. The parser passed it through fine, and the error surfaced three steps later.\nI was playing whack-a-mole with a model that had infinite moles.\n`**response_format**`**+**`**temperature=0**`**.** OpenAI's `response_format={\"type\": \"json_object\"}` combined with `temperature=0` brought failures from 23 down to about 9. Meaningful progress. Still not zero, and \"nine random failures per six weeks\" is not a pipeline property I could defend.\n### Function calling\nThis was the one that almost worked. Forcing output through a typed schema contract genuinely tightened things.\nI stopped checking logs every morning, stopped bracing for the Slack notification. Told someone on the team it was stable. Which, if you work in software, you know is the fastest way to jinx something.\npython\n\n```\nfunctions = [\n    {\n        \"name\": \"extract_document_metadata\",\n        \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n                \"methodology_type\": {\n                    \"type\": \"string\",\n                    \"enum\": [\"experimental\", \"observational\", \"review\", \"simulation\", \"mixed\"]\n                },\n                \"dataset_source\": {\"type\": \"string\"},\n                \"primary_metric\": {\"type\": \"string\"},\n                \"year\": {\"type\": \"integer\"},\n                \"confidence_score\": {\"type\": \"number\", \"minimum\": 0, \"maximum\": 1}\n            },\n            \"required\": [\"methodology_type\", \"dataset_source\", \"year\"]\n        }\n    }\n]\n\nresponse = openai_client.chat.completions.create(\n    model=\"gpt-4-turbo\",\n    messages=messages,\n    functions=functions,\n    function_call={\"name\": \"extract_document_metadata\"},\n    temperature=0\n)\n```\n\nThen one Tuesday, the OpenAI API had a 20-minute disruption. Pipeline failed hard; it wasn't a model issue, just a network dependency we'd never properly reckoned with.\nWe couldn't version-lock the model, couldn't run offline, couldn't answer \"what changed between the run that worked and the run that didn't?\" because the model on the other end wasn't ours.\nSitting there waiting for someone else's API to recover, I finally asked the question I should have asked months earlier: Does this specific job actually need a frontier model?\n## The Local Models Are Better Than I Expected\nI went in fully expecting to spend a day confirming they weren't good enough. I had a conclusion ready before I'd run a single test: _tried it, quality wasn't there, staying on GPT-4 with better retry logic._ A story where the first decision was still defensible.\nIt took maybe three hours to realize I'd been thinking about this wrong.\nDocument extraction into a fixed schema isn't actually a hard task for a language model. Not in the way that makes a frontier model necessary. There's no reasoning involved, no synthesis, no need for the breadth of world knowledge that makes GPT-4 worth what it costs. \nWhat it actually is: structured reading comprehension. The model reads a document and fills in fields. A well-trained 7B model does this fine. And with the right setup, specifically, seeded inference, it does it _identically_ every single run.\nI ran four models against 50 documents I'd manually annotated:\n  * **Phi-3-mini (3.8B)** : Better instruction following than I expected for a 3.8B model, but it fell apart on anything over about 3,000 tokens. I nearly picked this before I looked at the longer doc results.\n  * **Mistral 7B Instruct** : Solid everywhere, no real surprises in either direction. The Toyota Camry of local models. You'd be fine with this.\n  * **Qwen2.5-7B-Instruct** : This one won clearly. Best structured output consistency of the four by a margin wide enough that it wasn't a close call.\n  * **Llama 3.2 3B Instruct** : This is fast, but the extraction quality dropped off enough on edge cases that I wouldn't run it on production data without a lot more validation work first.\n\n\nQwen2.5 and Mistral both hit 90–95% accuracy on the annotated set, lower than GPT-4 on genuinely ambiguous documents, yes. \nBut I ran Qwen2.5 on the same 20 documents three times and went back to diff the results. No diffs. Same JSON, same values, same field order every run. \nAfter six weeks of failures I couldn't predict, that felt almost too clean to be real.\n## Before and After\nThe GPT-4 extractor at its most stable, not the original prototype, the version after months of defensive code had accumulated around it:\npython\n\n```\n# extractor_gpt4.py\nimport os\nimport json\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=os.environ[\"OPENAI_API_KEY\"])\n\nSYSTEM_PROMPT = \"\"\"You are a research document metadata extractor.\nGiven the text of a research document, extract the specified metadata fields.\nBe precise. If you are unsure about a field, use your best judgment based on context.\"\"\"\n\nEXTRACTION_SCHEMA = {\n    \"name\": \"extract_document_metadata\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"methodology_type\": {\n                \"type\": \"string\",\n                \"enum\": [\"experimental\", \"observational\", \"review\", \"simulation\", \"mixed\"]\n            },\n            \"dataset_source\": {\"type\": \"string\"},\n            \"primary_metric\": {\"type\": \"string\"},\n            \"year\": {\"type\": \"integer\"},\n            \"confidence_score\": {\"type\": \"number\"}\n        },\n        \"required\": [\"methodology_type\", \"dataset_source\", \"year\"]\n    }\n}\n\ndef extract_metadata_gpt4(document_text: str) -> dict:\n    response = client.chat.completions.create(\n        model=\"gpt-4-turbo\",\n        messages=[\n            {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n            {\"role\": \"user\", \"content\": f\"Extract metadata from this document:\\n\\n{document_text[:8000]}\"}\n        ],\n        functions=[EXTRACTION_SCHEMA],\n        function_call={\"name\": \"extract_document_metadata\"},\n        temperature=0,\n        timeout=30\n    )\n    try:\n        args = response.choices[0].message.function_call.arguments\n        return json.loads(args)\n    except (AttributeError, json.JSONDecodeError) as e:\n        raise ValueError(f\"Failed to parse GPT-4 response: {e}\")\n```\n\nLatency: 3.5–5.8s per document. Cost: ~$0.04 per call. Failure rate: ~6% after all the fixes.\nFor the replacement, I chose Ollama. A colleague mentioned it offhandedly, and the docs looked reasonable at 11 pm, which is honestly how a lot of infrastructure decisions get made. The REST API is close enough to OpenAI's that the swap took about an hour:\npython\n\n```\n# extractor_slm.py\nimport json\nimport logging\nimport os\nimport re\n\nimport requests\n\nlogger = logging.getLogger(__name__)\n\nOLLAMA_URL = os.getenv(\"OLLAMA_URL\", \"http://localhost:11434\")\nMODEL = os.getenv(\"MODEL_NAME\", \"qwen2.5:7b-instruct-q4_K_M\")\n\n# Tested empirically: 7B q4 on a T4 takes ~8s on cold first token,\n# then ~1.5s per doc chunk. 45s covers bad days.\n_TIMEOUT = 45\n\n_SYSTEM_PROMPT = \"\"\"\\\nYou are a metadata extractor for research documents.\nReturn ONLY a JSON object — no explanation, no markdown, no surrounding text.\n\nFields to extract:\n- methodology_type (required): one of experimental | observational | review | simulation | mixed\n- dataset_source (required): where the data came from\n- year (required): integer\n- primary_metric: main eval metric if present\n- confidence_score: your confidence 0.0–1.0\n\nOutput example:\n{\"methodology_type\": \"experimental\", \"dataset_source\": \"ImageNet\", \"year\": 2022, \"primary_metric\": \"top-1 accuracy\", \"confidence_score\": 0.95}\n\"\"\"\n\ndef call_ollama(doc_text: str) -> dict:\n    payload = {\n        \"model\": MODEL,\n        \"messages\": [\n            {\"role\": \"system\", \"content\": _SYSTEM_PROMPT},\n            {\"role\": \"user\", \"content\": doc_text[:6000]},\n        ],\n        \"stream\": False,\n        \"options\": {\n            \"temperature\": 0,\n            \"seed\": 42,  # determinism — this is the whole point\n        },\n    }\n\n    try:\n        resp = requests.post(f\"{OLLAMA_URL}/api/chat\", json=payload, timeout=_TIMEOUT)\n        resp.raise_for_status()\n    except requests.exceptions.Timeout:\n        raise RuntimeError(f\"Ollama timed out after {_TIMEOUT}s — is the model loaded?\")\n    except requests.exceptions.ConnectionError:\n        raise RuntimeError(f\"Can't reach Ollama at {OLLAMA_URL} — is the container running?\")\n\n    raw = resp.json()[\"message\"][\"content\"].strip()\n    cleaned = re.sub(r\"^```(?:json)?\\s*|\\s*```$\", \"\", raw).strip()\n\n    try:\n        return json.loads(cleaned)\n    except json.JSONDecodeError as e:\n        logger.error(\"Failed to parse model output: %s\\nRaw was: %s\", e, raw[:400])\n        raise\n```\n\nThe `seed: 42` in options is what actually delivers the determinism. Ollama supports seeded generation; same input, same seed, same output, every time, not approximately. `temperature=0` with a hosted API implied this but never guaranteed it because you're not controlling the runtime. Locally, you are.\n### Getting It Into GitHub Actions\nTwo things that aren't obvious until they bite you. First: GitHub Actions service containers are [network-isolated from the runner](https://docs.github.com/en/actions/using-containerized-services/about-service-containers). You can't `docker exec` into them; the model pull has to go through the REST API. Second: cache the model. A cold pull is 4.7GB and adds 3–4 minutes to every job.\nyaml\n\n```\n# The non-obvious parts — the rest is standard Actions boilerplate\n- name: Cache Ollama model\n  uses: actions/cache@v4\n  with:\n    path: ~/.ollama/models\n    key: ollama-qwen2.5-7b-q4\n\n- name: Pull SLM model\n  # Can't docker exec into service containers — use the API\n  run: |\n    curl -s http://localhost:11434/api/pull \\\n      -d '{\"name\": \"qwen2.5:7b-instruct-q4_K_M\"}' \\\n      --max-time 300\n\n- name: Warm up model\n  run: |\n    curl -s http://localhost:11434/api/generate \\\n      -d '{\"model\": \"qwen2.5:7b-instruct-q4_K_M\", \"prompt\": \"hello\", \"stream\": false}' \\\n      > /dev/null\n\n- name: Run ingestion pipeline\n  run: python pipeline/run_ingestion.py\n  env:\n    OLLAMA_URL: \"http://localhost:11434\"\n    MODEL_NAME: \"qwen2.5:7b-instruct-q4_K_M\"\n```\n\n### The Capitalization Bug I Spent Too Long On\nQwen 2.5 occasionally returns `\"Experimental\"` with a capital E despite explicit instructions not to. The `Literal` type check rejects it, and you get a validation failure on a document the model actually extracted correctly.\nI spent an embarrassingly long time on this because the error message just said \"invalid value,\" and my first instinct was to look at the document, then the extractor, before I finally looked at the validator and saw it.\nA `normalize_methodology` validator with `mode=\"before\"` fixes it before the type check runs:\npython\n\n```\n# pipeline/validation.py\nfrom typing import Literal, Optional\nfrom pydantic import BaseModel, Field, field_validator\n\nclass DocMetadata(BaseModel):\n    methodology_type: Literal[\"experimental\", \"observational\", \"review\", \"simulation\", \"mixed\"]\n    dataset_source: str = Field(min_length=3)\n    year: int = Field(ge=1950, le=2030)\n    primary_metric: Optional[str] = None\n    confidence_score: Optional[float] = Field(default=None, ge=0.0, le=1.0)\n\n    @field_validator(\"dataset_source\")\n    @classmethod\n    def strip_whitespace(cls, v: str) -> str:\n        return v.strip()\n\n    @field_validator(\"methodology_type\", mode=\"before\")\n    @classmethod\n    def normalize_methodology(cls, v: str) -> str:\n        # Qwen occasionally returns \"Experimental\" despite instructions\n        return v.lower().strip() if isinstance(v, str) else v\n```\n\n···\nI want to be straight about the tradeoffs because I hate articles that don't. \nGPT-4 is genuinely better on ambiguous documents, non-English text, anything requiring real inference. There's a class of document in our corpus, older PDFs, reports mixing multiple languages, non-standard formats, where the SLM stumbles and GPT-4 wouldn't. \nWe flag those separately and route them to a review queue. That's not seamless, but it's honest. The SLM does the 90% it's good at and doesn't get asked to do the rest.\nThe setup is also more work than `pip install openai`. First-time configuration takes a real afternoon. And Ollama doesn't auto-update models, so version management is my problem now. I've genuinely made peace with that, knowing exactly what model version ran last night and the night before is the whole point.\n···\n## What I Think Now\nThe morning the pipeline first ran clean, no notification, no re-run, just a log file showing 312 documents processed in 8.4 minutes, I checked it twice, then ran it manually again. \nI'd spent six weeks half-expecting an alert before I'd even finished my coffee. Watching it pass quietly felt genuinely strange.\nThe failure wasn't using GPT-4. The failure was treating a probabilistic system like a deterministic function. `temperature=0` reduces variance. It doesn't eliminate it. \nI understood this in theory the whole time. It took 23 failures to understand it in a way that changed how I build things.\nIf you're running an LLM in an automated pipeline and things keep breaking in ways you can't reproduce, it might not be your code or your prompts. It might be the nature of what you're depending on. \nA local model on hardware you control, seeded for determinism, is a genuinely different class of tool for this kind of job. Not better at everything but better at being the same thing twice. And for a pipeline that runs unattended every night, that's most of what actually matters.\n···\n## Before you go!\nIf this was useful, I write more about the messy reality behind building with AI, what breaks in production, what actually works, and the tradeoffs behind the fixes.\nYou can **subscribe to my**[**newsletter**](https://thatcsguy.substack.com/?r=79lldp&utm_campaign=pub-share-checklist) if you'd like more of that.\n### Connect With Me\n  * [LinkedIn](http://www.linkedin.com/in/nwekebenjamin)\n  * [Substack](https://thatcsguy.substack.com/)\n\n\n···\n### References\n  * OpenAI. _Reproducible outputs_. Retrieved from <https://platform.openai.com/docs/guides/text-generation/reproducible-outputs>\n  * Ollama. _API documentation_. Retrieved from <https://github.com/ollama/ollama/blob/main/docs/api.md>\n\n\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657271062_x78p7k.jpeg)\nWritten By\n## [Benjamin Nweke](https://towardsdatascience.com/author/benjaminnweke/)\nBenjamin Nweke is a Computer Science graduate and technical writer exploring AI, machine learning, and data science. He writes about building reliable AI systems, model evaluation, agentic architectures, and the practical challenges that appear when machine learning moves beyond experiments. His work focuses on making technically complex ideas clear, useful, and grounded in real implementation.\n[See all from Benjamin Nweke](https://towardsdatascience.com/author/benjaminnweke/)\n[Deep Dives](https://towardsdatascience.com/tag/deep-dives/)[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Document Processing](https://towardsdatascience.com/tag/document-processing/)[Llm](https://towardsdatascience.com/tag/llm/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fi-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing%2F&title=I%20Replaced%20GPT-4%20with%20a%20Local%20SLM%20and%20My%20CI%2FCD%20Pipeline%20Stopped%20Failing)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fi-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing%2F&title=I%20Replaced%20GPT-4%20with%20a%20Local%20SLM%20and%20My%20CI%2FCD%20Pipeline%20Stopped%20Failing)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fi-replaced-gpt-4-with-a-local-slm-and-my-ci-cd-pipeline-stopped-failing%2F&text=I%20Replaced%20GPT-4%20with%20a%20Local%20SLM%20and%20My%20CI%2FCD%20Pipeline%20Stopped%20Failing)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/07/18VLgJDlgx9RRIDVNkXwVDQ.png)](https://towardsdatascience.com/how-to-build-a-super-easy-handwritten-numbers-identifier-463850ab7d68/)\n## [How to build a super easy handwritten numbers identifier](https://towardsdatascience.com/how-to-build-a-super-easy-handwritten-numbers-identifier-463850ab7d68/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nA beginner friendly use of cosine similarity\n[Jose Manuel Napoles-Duarte](https://towardsdatascience.com/author/jnapoles/)\nJuly 8, 20213 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/01/16wUhgvKwtpZC_UOsW9lWIQ.jpeg)](https://towardsdatascience.com/say-goodbye-to-sql-headaches-with-this-python-tool-75099f5ff33d/)\n## [ChatGPT Doesn't Understand SQL All the Time But This Python Tool Does](https://towardsdatascience.com/say-goodbye-to-sql-headaches-with-this-python-tool-75099f5ff33d/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nSQLGlot - A one-stop SQL parsing and manipulation Python Library\n[Christopher Tao](https://towardsdatascience.com/author/christophertao/)\nJanuary 15, 202410 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/05/1WTt3WXi7tiwjjfo0Scdvgg.png)](https://towardsdatascience.com/word-embeddings-and-embedding-projector-of-tensorflow-c946b98c9b3f/)\n## [Word Embeddings and Embedding Projector of TensorFlow](https://towardsdatascience.com/word-embeddings-and-embedding-projector-of-tensorflow-c946b98c9b3f/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nTheoretical explanation and a practical example.\n[Soner Yıldırım](https://towardsdatascience.com/author/sonery/)\nMay 24, 20204 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/11/0cRwr5KUl3ZxtdGNQ-scaled.jpg)](https://towardsdatascience.com/essential-things-you-need-to-know-about-f1-score-dbd973bf1a3/)\n## [Essential Things You Need to Know About F1-Score](https://towardsdatascience.com/essential-things-you-need-to-know-about-f1-score-dbd973bf1a3/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nLearn about the key fundamentals of F1-score, one of the most important evaluation metrics in machine learning.\n[Zeya LT](https://towardsdatascience.com/author/zeyalt/)\nNovember 23, 20219 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/10/1htbJaFZFAmzhPqcMh0sQQQ.jpeg)](https://towardsdatascience.com/ai-agent-workflows-a-complete-guide-on-whether-to-build-with-langgraph-or-langchain-117025509fa0/)\n## [AI Agent Workflows: A Complete Guide on Whether to Build With LangGraph or LangChain](https://towardsdatascience.com/ai-agent-workflows-a-complete-guide-on-whether-to-build-with-langgraph-or-langchain-117025509fa0/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nLangChain and LangGraph - A deep dive on key building blocks, how they handle core pieces of their functionality, and deciding between...\n[Sandi Besen](https://towardsdatascience.com/author/sandibesen/)\nOctober 25, 202411 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/09/0YzXXZUoS6BaWK4Uo-scaled.jpg)](https://towardsdatascience.com/sampling-isnt-enough-profile-your-ml-data-instead-6a28fcfb2bd4/)\n## [Sampling isn't enough, profile your ML data instead](https://towardsdatascience.com/sampling-isnt-enough-profile-your-ml-data-instead-6a28fcfb2bd4/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nProduction logging approaches for AI and data pipelines\n[Isaac Backus](https://towardsdatascience.com/author/ibackus8/)\nSeptember 22, 20208 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/08/Graph-Engineering.jpg)](https://towardsdatascience.com/graph-engineering-isnt-about-more-connections-its-about-which-ones-get-used/)\n## [Graph Engineering Isn’t About More Connections — It’s About Which Ones Get Used](https://towardsdatascience.com/graph-engineering-isnt-about-more-connections-its-about-which-ones-get-used/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nI expected adding more communication pathways between agents to dramatically change network performance. It didn't. Figuring out why turned into a more useful article than the one I set out to write.\n[Emmimal P Alexander](https://towardsdatascience.com/author/emmimalp.alexander/)\nAugust 18, 202614 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/07/0sNu-59Hzn1eA8P8K.jpg)](https://towardsdatascience.com/significance-or-hypothesis-tests-with-python-7ed35e9ac9b6/)\n## [Significance or Hypothesis Tests with Python](https://towardsdatascience.com/significance-or-hypothesis-tests-with-python-7ed35e9ac9b6/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nCollege Statistics with Python\n[Luís Roque](https://towardsdatascience.com/author/luisroque/)\nJuly 1, 202113 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/03/1nFW3OvWQqSx-DFVzPQ_Blw-scaled.jpeg)](https://towardsdatascience.com/using-ai-you-may-want-to-start-conducting-human-rights-impact-assessments-hria-89ec0fdbce52/)\n## [Using AI? You may want to start conducting Human Rights Impact Assessments (HRIA)](https://towardsdatascience.com/using-ai-you-may-want-to-start-conducting-human-rights-impact-assessments-hria-89ec0fdbce52/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nIn a recent post I offered a few predictions on how the Biden Administration may change the AI playing field. The main takeaway is that AI...\n[Natalia Modjeska](https://towardsdatascience.com/author/nmodjeska/)\nMarch 4, 20219 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n",
  "body_base_path": "attachments/ca4739e88720/pages/",
  "body_markdown_path": "attachments/ca4739e88720/pages/archived-body.md",
  "local_html": "attachments/ca4739e88720/pages/archived-body.html",
  "body_source": "prior_capture"
}