{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Priyesh Dave",
  "title_listed": "Why LangGraph Wins: Benchmarking LangGraph, CrewAI, and AutoGen on 107 Real Data Engineering Tasks",
  "keywords": "",
  "domain": "数据处理",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/priyeshdave6/why-langgraph-wins-benchmarking-langgraph-crewai-and-autogen-on-107-real-data-engineering-tasks-3ljg",
  "publish_date": "2026-09-09",
  "corpus_stage": "",
  "notes": "",
  "id": "b790088292b1",
  "collected_at": "2026-10-02T13:49:53",
  "title": "Why LangGraph Wins: Benchmarking LangGraph, CrewAI, and AutoGen on 107 Real Data Engineering Tasks - DEV Community",
  "body_markdown": "Why LangGraph Wins: Benchmarking LangGraph, CrewAI, and AutoGen on 107 Real Data Engineering Tasks\nSubtitle:\nCut through the marketing: see where LangGraph, CrewAI, and AutoGen fail, succeed, and waste your tokens—supported by hard benchmark numbers and annotated, real-world code.\nAgent Frameworks Don’t Perform Equally: Benchmark Results, Not Hype\nFramework marketing claims “modular,” “easy coding,” “robust tool use.” Actual benchmarks suggest otherwise. Using\nsweta2503/agent-framework-benchmark\n, which runs LangGraph, CrewAI, and AutoGen through 107 reproducible data engineering tasks (dataset ingestion, transformation, ETL orchestration, etc.), the performance gap is impossible to ignore.\nRaw Results:\nThe table shows concrete outcomes on fixed testbeds—identical models, prompts, and tool APIs for comparability.\nFramework\nTasks Passed\nToken Median (per task)\nMean Latency (s)\nRecovered Failures\nHard Failures\nLangGraph\n97 / 107\n2350\n13.8\n7\n3\nCrewAI\n80 / 107\n4120\n21.6\n3\n24\nAutoGen\n58 / 107\n3160\n29.2\n0\n49\nSource:\nsweta2503/agent-framework-benchmark\n, June 2024. Task passes require strict output assertions.\nLangGraph dominates: higher pass rate, lower latency, far less token burn, and minimal irrecoverable failure. This is not marginal.\nLangGraph’s Explicit Graph Control Slashes Complexity\nLangGraph’s directed-graph model delivers reliability by design: explicit state, crisp control flow, and minimal boilerplate. Most frameworks hide orchestration behind recursive chains, brittle message passing, or role abstractions. LangGraph puts business logic front and center.\nArtifact: Data Ingestion Workflow (Extract, Transform, Load)\nLangGraph: Minimal, Transparent, Debuggable\nimport\nlanggraph\nfrom\nlanggraph.nodes\nimport\nToolNode\n,\nLLMNode\ngraph\n=\nlanggraph\n.\nGraph\n()\ndef\ningest_data\n(\ninput\n):\nreturn\n{\n\"\ndf\n\"\n:\nload_csv_to_df\n(\ninput\n[\n\"\nfile_path\n\"\n])}\ndef\ntransform_data\n(\ninput\n):\ndf\n=\ninput\n[\n\"\ndf\n\"\n]\nreturn\n{\n\"\ndf\n\"\n:\ndf\n.\ndropna\n().\ndrop_duplicates\n()}\ndef\nload_data\n(\ninput\n):\ndf\n=\ninput\n[\n\"\ndf\n\"\n]\nreturn\n{\n\"\nsuccess\n\"\n:\nupload_df_to_db\n(\ndf\n)}\ngraph\n.\nadd_node\n(\nToolNode\n(\nfunc\n=\ningest_data\n),\ninputs\n=\n[\n\"\nfile_path\n\"\n])\ngraph\n.\nadd_node\n(\nToolNode\n(\nfunc\n=\ntransform_data\n),\ninputs\n=\n[\n\"\ndf\n\"\n])\ngraph\n.\nadd_node\n(\nToolNode\n(\nfunc\n=\nload_data\n),\ninputs\n=\n[\n\"\ndf\n\"\n])\ngraph\n.\nadd_edge\n(\n\"\ningest_data\n\"\n,\n\"\ntransform_data\n\"\n)\ngraph\n.\nadd_edge\n(\n\"\ntransform_data\n\"\n,\n\"\nload_data\n\"\n)\ngraph\n.\ncompile\n(\n\"\nfile_path\n\"\n)\n# Run: graph.run({\"file_path\": \"/tmp/source.csv\"})\nEnter fullscreen mode\nExit fullscreen mode\nState and control flow are explicit. No prompt glue or nested dict acrobatics.\nEach step is a stand-alone plain function.\nNo message passing or confirmation chatter. Inputs and outputs are your actual objects.\nThe difference becomes starker against CrewAI and AutoGen in both code and operational complexity.\nCrewAI’s Agent Abstraction Burns Tokens and Developer Time\nCrewAI wraps orchestration in agent chat and rigid task objects. The result: verbosity, boilerplate, and heavy tokenization. The same ETL expands to double the lines—nearly all structural, not business logic. The chat abstraction is not free. Each agent introduces message-passing overhead that stacks at every step.\nExample: CrewAI Version of ETL Workflow\nfrom\ncrewai\nimport\nAgent\n,\nTask\n,\nCrew\nclass\nIngestAgent\n(\nAgent\n):\ndef\nrun\n(\nself\n,\nfile_path\n):\nreturn\n{\n\"\ndf\n\"\n:\nload_csv_to_df\n(\nfile_path\n)}\nclass\nTransformAgent\n(\nAgent\n):\ndef\nrun\n(\nself\n,\ndf\n):\ntransformed\n=\ndf\n.\ndropna\n().\ndrop_duplicates\n()\nreturn\n{\n\"\ndf\n\"\n:\ntransformed\n}\nclass\nLoadAgent\n(\nAgent\n):\ndef\nrun\n(\nself\n,\ndf\n):\nreturn\n{\n\"\nsuccess\n\"\n:\nupload_df_to_db\n(\ndf\n)}\ningest\n=\nIngestAgent\n(\nrole\n=\n\"\nIngest\n\"\n)\ntransform\n=\nTransformAgent\n(\nrole\n=\n\"\nTransform\n\"\n)\nload\n=\nLoadAgent\n(\nrole\n=\n\"\nLoad\n\"\n)\ntask1\n=\nTask\n(\ningest\n,\ninputs\n=\n[\n\"\nfile_path\n\"\n])\ntask2\n=\nTask\n(\ntransform\n,\ninputs\n=\n[\n\"\ndf\n\"\n])\ntask3\n=\nTask\n(\nload\n,\ninputs\n=\n[\n\"\ndf\n\"\n])\ncrew\n=\nCrew\n(\ntasks\n=\n[\ntask1\n,\ntask2\n,\ntask3\n])\ncrew\n.\nsetup_dependencies\n([(\ntask1\n,\ntask2\n),\n(\ntask2\n,\ntask3\n)])\n# Run: crew.run({\"file_path\": \"/tmp/source.csv\"})\nEnter fullscreen mode\nExit fullscreen mode\nOperational Overhead:\nCrewAI’s callback chains and chatter stack up—in the benchmark, median token usage per task is 4120 (vs 2350 for LangGraph), with latency inflated by a third. Each agent restates or echoes prior context. Logs for a single run:\n[IngestAgent] Loaded dataframe from /tmp/source.csv, passing to next agent.\n[TransformAgent] Received dataframe, initiating transformation...\n[TransformAgent] Sending cleaned dataframe to LoadAgent.\n[LoadAgent] Received dataframe, uploading to DB...\nEnter fullscreen mode\nExit fullscreen mode\nMultiply this “meta” chatter by every workflow step, and your token bill explodes with no benefit.\nAutoGen Falls Apart on Stateful Multi-Step Operations\nAutoGen markets itself for stateful agentic pipelines, but fails on workflows with chained state or error branching. Benchmark logs show frequent breakdowns resembling silent pipeline drops or input mismatches.\nFailure Artifact: Branching ETL With Error Handling\nTask: ETL with fallback on transformation error (branch to cleanup, retry load).\nAutoGen sample code:\nfrom\nautogen\nimport\nAgentFlow\ndef\ningest\n(\nfile_path\n):\nreturn\n{\n\"\ndf\n\"\n:\nload_csv_to_df\n(\nfile_path\n)}\ndef\ntransform\n(\ndf\n):\ntry\n:\nreturn\n{\n\"\ndf\n\"\n:\ndf\n.\ndropna\n().\ndrop_duplicates\n()}\nexcept\nException\nas\ne\n:\nreturn\n{\n\"\nerror\n\"\n:\nstr\n(\ne\n)}\ndef\nload\n(\ndf\n):\nreturn\n{\n\"\nsuccess\n\"\n:\nupload_df_to_db\n(\ndf\n)}\nflow\n=\nAgentFlow\n()\nflow\n.\nadd_fn\n(\n\"\ningest\n\"\n,\ningest\n)\nflow\n.\nadd_fn\n(\n\"\ntransform\n\"\n,\ntransform\n)\nflow\n.\nadd_fn\n(\n\"\nload\n\"\n,\nload\n)\nflow\n.\nset_flow\n([(\n\"\ningest\n\"\n,\n\"\ntransform\n\"\n),\n(\n\"\ntransform\n\"\n,\n\"\nload\n\"\n)])\n# Run: flow.execute({\"file_path\": \"/tmp/source.csv\"})\nEnter fullscreen mode\nExit fullscreen mode\nObserved Benchmark Output:\n[AutoGen] Step: ingest --> Success\n[AutoGen] Step: transform --> Error: 'DataFrame' object has no attribute 'dropna'\n[AutoGen] Step: load --> Missing input 'df', raising up PipelineError\nEnter fullscreen mode\nExit fullscreen mode\nIntermediary state must be manually serialized and re-parsed between steps—unhandled types cause silent exceptions, missing results, and cascade failures. In the full benchmark, ~46% of multi-step AutoGen tasks either lost state or crashed on pipeline handoff.\nPipeline Benchmark: Concrete Task, Real Numbers\nTask:\nExtract order CSV, drop rows missing payment, upload to DB.\nLangGraph\ngraph\n=\nlanggraph\n.\nGraph\n()\ngraph\n.\nadd_node\n(\nToolNode\n(\nfunc\n=\nlambda\ninp\n:\n{\n\"\ndf\n\"\n:\nload_csv_to_df\n(\ninp\n[\n\"\nfile_path\n\"\n])}),\ninputs\n=\n[\n\"\nfile_path\n\"\n])\ngraph\n.\nadd_node\n(\nToolNode\n(\nfunc\n=\nlambda\ninp\n:\n{\n\"\ndf\n\"\n:\ninp\n[\n\"\ndf\n\"\n][\ninp\n[\n\"\ndf\n\"\n][\n\"\npayment_id\n\"\n].\nnotnull\n()]}),\ninputs\n=\n[\n\"\ndf\n\"\n])\ngraph\n.\nadd_node\n(\nToolNode\n(\nfunc\n=\nlambda\ninp\n:\n{\n\"\nsuccess\n\"\n:\nupload_df_to_db\n(\ninp\n[\n\"\ndf\n\"\n])}),\ninputs\n=\n[\n\"\ndf\n\"\n])\ngraph\n.\nadd_edge\n(\n\"\nToolNode_1\n\"\n,\n\"\nToolNode_2\n\"\n)\ngraph\n.\nadd_edge\n(\n\"\nToolNode_2\n\"\n,\n\"\nToolNode_3\n\"\n)\ngraph\n.\ncompile\n(\n\"\nfile_path\n\"\n)\nresult\n=\ngraph\n.\nrun\n({\n\"\nfile_path\n\"\n:\n\"\n/data/orders.csv\n\"\n})\nEnter fullscreen mode\nExit fullscreen mode\n13 lines, 1970 tokens, 10.7s latency.\nCrewAI\ningest\n=\nIngestAgent\n()\nfilter_null\n=\nAgent\n(\nlambda\ndf\n:\n{\n\"\ndf\n\"\n:\ndf\n[\ndf\n[\n\"\npayment_id\n\"\n].\nnotnull\n()]})\nload\n=\nLoadAgent\n()\nt1\n=\nTask\n(\ningest\n,\n[\n\"\nfile_path\n\"\n])\nt2\n=\nTask\n(\nfilter_null\n,\n[\n\"\ndf\n\"\n])\nt3\n=\nTask\n(\nload\n,\n[\n\"\ndf\n\"\n])\ncrew\n=\nCrew\n([\nt1\n,\nt2\n,\nt3\n])\ncrew\n.\nsetup_dependencies\n([(\nt1\n,\nt2\n),\n(\nt2\n,\nt3\n)])\ncrew\n.\nrun\n({\n\"\nfile_path\n\"\n:\n\"\n/data/orders.csv\n\"\n})\nEnter fullscreen mode\nExit fullscreen mode\n19 lines (+class boilerplate elsewhere), 3775 tokens, 19.8s latency.\nAutoGen\nflow\n=\nAgentFlow\n()\nflow\n.\nadd_fn\n(\n\"\ningest\n\"\n,\nlambda\nfile_path\n:\n{\n\"\ndf\n\"\n:\nload_csv_to_df\n(\nfile_path\n)})\nflow\n.\nadd_fn\n(\n\"\nfilter\n\"\n,\nlambda\ndf\n:\n{\n\"\ndf\n\"\n:\ndf\n[\ndf\n[\n\"\npayment_id\n\"\n].\nnotnull\n()]})\nflow\n.\nadd_fn\n(\n\"\nload\n\"\n,\nlambda\ndf\n:\n{\n\"\nsuccess\n\"\n:\nupload_df_to_db\n(\ndf\n)})\nflow\n.\nset_flow\n([(\n\"\ningest\n\"\n,\n\"\nfilter\n\"\n),\n(\n\"\nfilter\n\"\n,\n\"\nload\n\"\n)])\nflow\n.\nexecute\n({\n\"\nfile_path\n\"\n:\n\"\n/data/orders.csv\n\"\n})\nEnter fullscreen mode\nExit fullscreen mode\n10 lines but brittle: fails if intermediate\ndf\nis unparseable; 3150 tokens (if successful), 22s latency typical.\nSummary:\nLangGraph is the only concise, robust, cost-efficient pipeline. CrewAI bloats code, burns tokens, and slows execution. AutoGen’s approach collapses when type handoff is non-trivial.\nLangGraph’s Explicit State Model Delivers Order-of-Magnitude Gains\nBenchmark data and real code reveal why LangGraph outperforms:\nLower hard failure rates:\n97/107 tasks pass, 7 additional failures auto-recovered.\nToken savings:\nMedian savings exceed 1200 tokens per run (less cloud spend, less compute).\nMinimal code/mental overhead:\nEach pipeline step is a direct, testable function.\nRich control flow:\nBranches and loops are explicit in the graph structure, not implicit in prompt chains or chat roles.\nCrewAI’s inefficiency is built-in:\nIts chat abstractions pump up both code and token usage, causing slowdowns and error accumulation.\nAutoGen collapses under real pipeline requirements:\nIt handles stateless chat, but not robust state or type exchange for multi-step operations.\nThis is not theoretical. The gap is visible in code, benchmarks, and error logs. For agentic ETL and data engineering, LangGraph is the new baseline.\nReferences:\nsweta2503/agent-framework-benchmark\nlanggraph-vs-crewai comparative code\nCrewAI docs copilot",
  "metadata": {
    "title": "Why LangGraph Wins: Benchmarking LangGraph, CrewAI, and AutoGen on 107 Real Data Engineering Tasks - DEV Community",
    "description": "Why LangGraph Wins: Benchmarking LangGraph, CrewAI, and AutoGen on 107 Real Data Engineering... Tagged with ai, agenticai, machinelearning, llm.",
    "keywords": "ai, agenticai, machinelearning, llm, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/priyeshdave6/why-langgraph-wins-benchmarking-langgraph-crewai-and-autogen-on-107-real-data-engineering-tasks-3ljg",
    "og:title": "Why LangGraph Wins: Benchmarking LangGraph, CrewAI, and AutoGen on 107 Real Data Engineering Tasks",
    "og:description": "Why LangGraph Wins: Benchmarking LangGraph, CrewAI, and AutoGen on 107 Real Data Engineering...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9cozftfvhmb189pfmkof.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Why LangGraph Wins: Benchmarking LangGraph, CrewAI, and AutoGen on 107 Real Data Engineering Tasks",
    "twitter:description": "Why LangGraph Wins: Benchmarking LangGraph, CrewAI, and AutoGen on 107 Real Data Engineering...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9cozftfvhmb189pfmkof.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/b790088292b1/code/body-000.txt",
      "chars": 712,
      "language": "text"
    },
    {
      "file": "attachments/b790088292b1/code/body-001.txt",
      "chars": 759,
      "language": "text"
    },
    {
      "file": "attachments/b790088292b1/code/body-002.txt",
      "chars": 250,
      "language": "text"
    },
    {
      "file": "attachments/b790088292b1/code/body-003.txt",
      "chars": 532,
      "language": "text"
    },
    {
      "file": "attachments/b790088292b1/code/body-004.txt",
      "chars": 187,
      "language": "text"
    },
    {
      "file": "attachments/b790088292b1/code/body-005.txt",
      "chars": 509,
      "language": "text"
    },
    {
      "file": "attachments/b790088292b1/code/body-006.txt",
      "chars": 320,
      "language": "text"
    },
    {
      "file": "attachments/b790088292b1/code/body-007.txt",
      "chars": 339,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://github.com/sweta2503/agent-framework-benchmark",
      "label": "sweta2503/agent-framework-benchmark",
      "type": "a"
    },
    {
      "href": "https://github.com/hamzaahsan334-dev/langgraph-vs-crewai",
      "label": "langgraph-vs-crewai comparative code",
      "type": "a"
    },
    {
      "href": "https://github.com/Adamsautomations/crewai-docs-copilot",
      "label": "CrewAI docs copilot",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:09Z",
  "source_url": "https://dev.to/priyeshdave6/why-langgraph-wins-benchmarking-langgraph-crewai-and-autogen-on-107-real-data-engineering-tasks-3ljg",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 9478,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/b790088292b1/pages/source.html",
  "local_html": "attachments/b790088292b1/pages/article.html",
  "body_base_path": "attachments/b790088292b1/pages/",
  "body_markdown_path": "attachments/b790088292b1/pages/article.md"
}