{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Tae Kim",
  "title_listed": "I Built a Local LLM Rig to Escape API Bills. Then I Paid OpenAI Again.",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://dev.to/hannune/i-built-a-local-llm-rig-to-escape-api-bills-then-i-paid-openai-again-4bi7",
  "publish_date": "2026-06-13",
  "corpus_stage": "",
  "notes": "",
  "id": "68230987dbbf",
  "collected_at": "2026-10-02T14:04:10",
  "title": "I Built a Local LLM Rig to Escape API Bills. Then I Paid OpenAI Again. - DEV Community",
  "body_markdown": "I run a one-person AI shop. For 2asy.ai's filing pipeline that needs thousands of single-document extractions per cycle, the local rig lost the batch lane and OpenAI Batch won. Per-pipeline, not per-company.\nThe rule that decided it: no cross-document attention. Each filing gets its own prompt window. No string concatenation. The rule came from a Neo4j rollback I already paid for.\nQuick results.\nLocal Gemma 4 26B on llama.cpp\n(RTX 4090 + W6800): live serving fine. Batch lane blocked. vLLM has no 4-bit MoE path I need, container wants CUDA 12.9, host driver is 12.8.\nGGML_CUDA_DISABLE_GRAPHS=1\nkeeps llama.cpp alive when graph optimizer segfaults.\nOpenRouter\n: no real batch. Live pricing. At concurrency 32, latency 2 to 17 seconds, 121s timeouts, 429s.\nGemini batch SDK\n: silently inline-concatenates documents into one context. Cross-document leak. Neo4j rollback. Upstream\ngoogleapis/python-genai\nissue 1984 is not-planned.\nOpenAI Batch\n(\ngpt-5.4-mini\n): JSONL line-isolated, 50 percent off, 100-doc nano gate in 2.7 min, zero 429s, around 1 cent per document.\nThe local rig stays for live serving, ER API LLM gate, multimodal, and ablations. The batch lane moves to OpenAI.\nFull retrospective with the side-by-side table:\nhttps://hannune.ai/blog/local-llm-to-openai-batch.html",
  "metadata": {
    "title": "I Built a Local LLM Rig to Escape API Bills. Then I Paid OpenAI Again. - DEV Community",
    "description": "A solo AI operator tested llama.cpp, OpenRouter, Gemini batch, and OpenAI Batch under one strict rule for the 2asy.ai extraction pipeline. What broke and what fit. Tagged with ai, llm, openai, infrastructure.",
    "keywords": "ai, llm, openai, infrastructure, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/hannune/i-built-a-local-llm-rig-to-escape-api-bills-then-i-paid-openai-again-4bi7",
    "og:title": "I Built a Local LLM Rig to Escape API Bills. Then I Paid OpenAI Again.",
    "og:description": "A solo AI operator tested llama.cpp, OpenRouter, Gemini batch, and OpenAI Batch under one strict rule for the 2asy.ai extraction pipeline. What broke and what fit.",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fhannune.ai%2Fblog%2Fimages%2Flocal-llm-to-openai-batch.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@HannuneTech",
    "twitter:title": "I Built a Local LLM Rig to Escape API Bills. Then I Paid OpenAI Again.",
    "twitter:description": "A solo AI operator tested llama.cpp, OpenRouter, Gemini batch, and OpenAI Batch under one strict rule for the 2asy.ai extraction pipeline. What broke and what fit.",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fhannune.ai%2Fblog%2Fimages%2Flocal-llm-to-openai-batch.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://hannune.ai/blog/local-llm-to-openai-batch.html",
      "label": "https://hannune.ai/blog/local-llm-to-openai-batch.html",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:49:44Z",
  "source_url": "https://dev.to/hannune/i-built-a-local-llm-rig-to-escape-api-bills-then-i-paid-openai-again-4bi7",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 1286,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/68230987dbbf/pages/source.html",
  "local_html": "attachments/68230987dbbf/pages/article.html",
  "body_base_path": "attachments/68230987dbbf/pages/",
  "body_markdown_path": "attachments/68230987dbbf/pages/article.md"
}