保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Agentic AI

Why AI Engineers Are Moving Beyond LangChain to Native Agent Architectures

Frameworks accelerated the first wave of LLM apps, but production demands a different architecture. Benjamin Nweke April 30, 20267 min read Image by author (Generated with ChatGPT) Recently, I've been sitting closely with this topic, and it brought back some experiences from working on a couple of projects. Take this scenario: you ship an LLM-powered feature, the demo is clean, and all stakeholders are happy. Then three weeks into production, something breaks in a way nobody can explain. You spend an afternoon staring at logs that tell you what happened but not why. Then it turns out that the framework swallowed the context somewhere between step three and step four, and now you're reading source code you didn't write. That is not a bug report; it is a wakeup call about the architecture. Frameworks like LangChain let engineers build LLM-powered systems without first understanding how those systems work under pressure. At first, that sounds like the cavalry has arrived. But trust me, the cost doesn't show up until you're deep in a production incident, and now you're stuck wondering why your agent skipped the verification step it was supposed to run. This post is about that cost and why more engineers, after discovering it, are now building the orchestration layer by themselves. Learn this step by step with the interactive AI Agents roadmap.

Give LangChain Its Credit

I remember watching a colleague build a working RAG pipeline in about forty minutes sometime in early 2023. He went from the vector store through the retrieval chain, prompt templates, and the LLM call, all connected by lunchtime. Six months prior, that would have been at least a two-week project. Come to think of it, that's actually how and why LangChain spread so fast. Most engineers hadn't built LLM applications before. Nobody had strong opinions about the right way to structure a retrieval chain or manage conversation memory and other stuff like that. LangChain showed up with answers that were modular, composable, and documented, and of course, teams grabbed them immediately, including mine. So when I say it creates problems in production, I'm not being dismissive. It was optimized for the phase most teams were in when they adopted it. The problems came later, when the phase changed.

Where the Abstraction Breaks

When I was learning object-oriented programming in my sophomore year, one of the first concepts that clicked was abstraction: hiding the internal details of how something works and only exposing what the user needs. LangChain applies that same idea to LLM orchestration. It hides a lot of what's happening inside your system so you can move faster. But production AI systems demand something that cuts against that: clarity. You need to know exactly what your system did, in what order, with what inputs, and why. Not roughly. Exactly. Abstractions trade that visibility for speed. That’s a fair trade at first, until the hidden complexity becomes the very thing you need to understand. And it shows up in more ways than one. Debugging is worse than it sounds: When a multi-step chain gives the wrong output, you’re not just debugging your own code. You're also trying to understand the framework's execution flow and what the callback layer was doing behind the scenes. I once spent three hours tracking down a failure that turned out to be a memory module silently cutting out context. The fix itself took four minutes. Finding what caused it took half a day because the abstraction made the actual behavior invisible. Observability hits a ceiling: You can integrate LangSmith and get useful traces, but you're still seeing things through the framework's lens, limited to the spans it chooses to expose. When you need visibility into something specific to your business logic, you end up working around the framework's data model instead of just measuring what actually matters. Multi-agent state is where things really fall apart: The moment you have agents coordinating, one planning, others executing, and another verifying, shared state becomes the real problem. Who created this information, when, and is it still valid? One agent updates memory, another reads a stale version, and the coordinator makes a decision based on context that no longer matches reality. Framework-managed state tends to work just fine for the happy path and quietly breaks down at the edge cases. Production systems live in those edge cases. Latency accumulates: Every abstraction layer adds overhead through serialization, validation, callback firing, and internal routing that runs whether you need it or not. In a prototype that overhead is invisible. Under real traffic, it shows up in percentile latency, specifically in the p95 and p99 ranges where users actually feel it. The cost per call might be small, but in an agentic system making four, five, or even six model calls per user request, those small costs compound quickly. At some point, you have to ask whether that overhead is still worth what it buys you. None of this is impossible to solve inside a framework. But the fixes start to look like working around the framework instead of working with it. And once you get there, it becomes harder to tell what the framework is still giving you.

So What Does "Building It Yourself" Actually Look Like?

"Native agent architecture" sounds more complex than it really is. It just means writing the orchestration logic yourself as code you own, instead of relying on a framework's abstraction of it. State is something you define and update explicitly. Tools are clear functions you can test on their own. Memory is code you wrote, so it’s easier to debug, control, and understand what gets stored and how it gets retrieved. The model call is your code, which means you can instrument it directly and trace what matters. Sure, there will be more code upfront. But when something breaks, the failure is in your code and not somewhere inside an execution model written by somebody else. Let's not forget, complex workflows map more naturally here. Things like parallel execution, conditional branching, and long-running async tasks work much better in event-driven patterns in ways that synchronous chain execution doesn't handle cleanly. More design work upfront means less firefighting later. ··· I've seen teams rebuild a perfectly good LangChain prototype into a custom orchestration layer just because native architectures felt more “serious.” They spent three extra weeks on it and shipped the same system with more code to maintain. To me, that’s not progress. If you're checking whether a feature is worth building, then a framework gets you there faster. If three people use the system internally and nobody's pager is attached to it, the abstraction overhead is fine. The question isn’t “framework or native?” It’s what you need to optimize for right now. Fast iteration on uncertain requirements means the framework makes sense. Real users, real SLAs, agent coordination, and operational monitoring mean the native architecture earns its upfront cost. Most teams hit that turning point sooner than they expect, usually at the first serious debugging session or the first time someone asks for detailed metrics, and the honest answer is "not without a lot of extra work." That's the moment to rethink the architecture, not after six months of piling on workarounds. Frameworks are how knowledge transfers in a new field. LangChain made LLM application development accessible for a generation of engineers. That contribution is real. But maturity in a domain looks like moving from "I configure the framework to do the thing" to "I understand what the framework was doing, and I make those decisions myself." Not because frameworks are bad, but because owning your architecture means you know what’s happening under the hood. The engineers building the most reliable production AI systems aren't the ones with the most sophisticated tooling. They're the ones who can explain exactly what their system does at any point. What prompt is constructed, from what context, under what conditions, and with what fallback. That clarity is hard to maintain through thick layers of abstraction. ···

Final thoughts

Abstraction debt is quiet until it's loud. You won't notice it during the build. You'll notice it when something fails in a way the framework's error message can't explain. That moment comes earlier than you expect, usually triggered by a debugging session or a monitoring ask rather than a planning meeting. State and observability are not optional. If you can’t trace what your agent did and why, you’re not really improving the system. You’re just hoping for the best every time you redeploy. Treat orchestration as a real architectural decision. Pick it on purpose, with the tradeoffs visible. The engineers building durable AI systems aren't the ones who avoided frameworks. They're the ones who knew when to stop letting the framework decide for them. ···

Before you go!

I write more about the real engineering decisions behind AI systems, where abstractions help, where they hurt, and what it takes to build reliably. You can subscribe to my newsletter if you'd like more of that. Connect With Me

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

Benjamin Nweke

Benjamin Nweke is a Computer Science graduate and technical writer exploring AI, machine learning, and data science. He writes about building reliable AI systems, model evaluation, agentic architectures, and the practical challenges that appear when machine learning moves beyond experiments. His work focuses on making technically complex ideas clear, useful, and grounded in real implementation. See all from Benjamin Nweke AI AgentsArtificial IntelligenceLangchainMachine LearningSoftware Architecture Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Recreating an ML Master's degree with Online Courses

Deep Learning Use online courses to create your curriculum Pascal Janetzky June 14, 20217 min read

Recognizing Cats and Dogs With TensorFlow

Artificial Intelligence Create a neural network using TensorFlow to recognize cats and dogs Patrick Kalkman January 11, 202110 min read

How To Sort Pandas DataFrames

Machine Learning Sorting pandas DataFrames using one or more columns Giorgos Myrianthous July 5, 20225 min read

How to Hire an AI Consultant

Artificial Intelligence So, you know you need some machine learning/AI development. Why not just do it yourself? Well, the short answer is that specialists have... Daniel Shapiro, PhD June 27, 201711 min read

Context Engineering for AI Agents: A Deep Dive

Agentic AI How to optimize context, a precious finite resource for AI agents Clara Chong April 7, 20267 min read

How to Design Architectural Guardrails Around AI Agents

Agentic AI Agent design patterns every data engineer must know Thuwarakesh Murallie September 29, 202612 min read

Stop Ranking Agent Configs by Average Score

Agentic AI Best-worst comparisons, MaxDiff-style judging, and Plackett-Luce utility scores give agent teams a cleaner way to decide which configs to ship, prune, and route toward next. Doster Esh July 6, 202613 min read

Production-Ready LLMs Made Simple with the NeMo Agent Toolkit

Agentic AI From simple chat to multi-agent reasoning and real-time REST APIs Mariya Mansurova December 31, 202522 min read

Coding Agents Don’t Need Bigger Context Windows — They Need a Context Compiler

Agentic AI Most coding agents waste 70% of their context window on irrelevant code. A compiler would never do that. It resolves dependencies, discards unreachable code, and keeps only the interfaces where implementation isn’t needed. Coding agents do the opposite—and it’s costing them the exact thing a shrinking context budget can’t afford to lose. Emmimal P Alexander August 1, 202613 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network

Recommended Articles

Close