{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Tae Kim",
  "title_listed": "When your calls can't leave your infrastructure: the compliance constraint that shapes the whole AI stack",
  "keywords": "",
  "domain": "数据处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://dev.to/hannune/when-your-calls-cant-leave-your-infrastructure-the-compliance-constraint-that-shapes-the-whole-ai-4kec",
  "publish_date": "2026-08-28",
  "corpus_stage": "",
  "notes": "",
  "id": "4bfc420fa6c4",
  "collected_at": "2026-10-02T14:03:52",
  "title": "When your calls can't leave your infrastructure: the compliance constraint that shapes the whole AI stack - DEV Community",
  "body_markdown": "I wrote before about\nwhy I run speech-to-text locally\n—for me it started as a personal data hygiene decision. But when I've talked to other people about local AI pipelines, a different version of the same constraint comes up: not \"I'd prefer the audio to stay local\" but \"the audio legally cannot leave our environment.\"\nThat's a different problem. A preference shapes your tooling choices. A compliance constraint shapes your architecture.\nWhat the constraint actually looks like\nThe most common version I hear: enterprise customer agreements. A company builds a software tool that serves clients in regulated industries. Those clients have data handling requirements that specify data can only be processed in specific jurisdictions or on approved infrastructure. When that company wants to add AI-powered call analysis, every cloud transcription API is immediately out of scope—not because of policy preference, but because the customer contract requires it.\nAttorney-client privilege is another version. Law firms and their clients have specific expectations about where privileged communications go. Using a third-party transcription service means the audio passes through systems outside that privileged relationship. Whether that technically breaks privilege is a legal question that most firms don't want to be the test case for.\nHealthcare is the most formalized version: PHI (protected health information) that appears in patient calls needs to stay within HIPAA-compliant infrastructure. Some cloud providers offer BAAs (Business Associate Agreements) that extend that coverage, but plenty of healthcare organizations have decided the simplest answer is a processing pipeline that stays entirely on infrastructure they control.\nThe common thread: the constraint isn't about the transcription output (text) but about the audio itself, during processing. It's a data residency and custody question.\nThe architectural implication: it's not just transcription\nHere's where this gets interesting from a systems perspective. If you accept the constraint that audio can't leave your infrastructure, and you're trying to build something useful from that audio—summaries, action items, entity extraction, follow-up flags—the constraint propagates through every step that touches the content.\nYou transcribe locally. But then what?\nIf you send the transcript to a cloud LLM for summarization, you've sent the content of the call off your infrastructure. The audio stayed local, but the information left. For most compliance constraints, this doesn't actually satisfy the requirement.\nSo the constraint ends up requiring:\nLocal transcription (Whisper or equivalent)\nLocal embeddings if you're doing semantic search across transcripts\nLocal LLM inference if you're doing summarization, extraction, or classification\nLocal storage, obviously\nYou end up needing a full local inference stack, not just a transcription component. The compliance requirement that started with \"we can't use cloud STT\" turns into \"we need to run the whole pipeline on infrastructure we control.\"\nThis is the constraint I'm designing around\nThe setup I've been building runs Whisper for transcription, bge-m3 for embeddings, and Gemma for image/document processing—all on a single local server. I covered the\nresource management side of running three models on one box\nin an earlier post.\nThe \"nothing leaves the box\" constraint wasn't an afterthought. It was the first design decision, and it determined the whole stack. Every model choice, every pipeline component was evaluated against it: does this require sending data somewhere I don't control? If yes, it's out.\nThe tradeoffs are real. Local inference is slower than cloud APIs for equivalent hardware. You own the updates and maintenance. When a better Whisper model ships, you're updating your own deployment. The TCO calculation looks different depending on how much you're processing.\nBut for the use cases where the compliance constraint is genuine—where \"it's in the customer agreement\" or \"it's a regulatory requirement\"—those tradeoffs aren't optional. The architecture has to start from the constraint.\nWhat I haven't built yet\nThe local inference stack is working. What I'm building toward is the layer that makes it actually useful: intake from multiple channels (calls, email, messages), process everything locally, surface what matters in a daily report.\nThe compliance constraint applies there too—not just transcription. Every step processes content that, in an NDA or regulated environment, can't transit external APIs. So the design for the report generation layer has to be local all the way through.\nThat's the hard part I'm working on now: not \"can I transcribe calls locally\" (yes, that's working), but \"can I build the summarization and reporting layer locally at quality that's actually useful.\"\nOther posts in this series:\nRunning three AI models on one server\n|\nWhy local STT over cloud API\n|\nMissed calls, missed revenue",
  "metadata": {
    "title": "When your calls can't leave your infrastructure: the compliance constraint that shapes the whole AI stack - DEV Community",
    "description": "I wrote before about why I run speech-to-text locally—for me it started as a personal data hygiene... Tagged with ai, selfhosted, python, architecture.",
    "keywords": "ai, selfhosted, python, architecture, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/hannune/when-your-calls-cant-leave-your-infrastructure-the-compliance-constraint-that-shapes-the-whole-ai-4kec",
    "og:title": "When your calls can't leave your infrastructure: the compliance constraint that shapes the whole AI stack",
    "og:description": "I wrote before about why I run speech-to-text locally—for me it started as a personal data hygiene...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fimages.unsplash.com%2Fphoto-1675627452781-f228fa8f678c%3Fw%3D1200%26h%3D630%26fit%3Dcrop",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@HannuneTech",
    "twitter:title": "When your calls can't leave your infrastructure: the compliance constraint that shapes the whole AI stack",
    "twitter:description": "I wrote before about why I run speech-to-text locally—for me it started as a personal data hygiene...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fimages.unsplash.com%2Fphoto-1675627452781-f228fa8f678c%3Fw%3D1200%26h%3D630%26fit%3Dcrop"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://dev.to/hannune/why-i-run-speech-to-text-locally-instead-of-calling-a-cloud-api-59j7",
      "label": "why I run speech-to-text locally",
      "type": "a"
    },
    {
      "href": "https://dev.to/hannune/running-three-ai-models-on-one-local-server-when-your-vram-doesnt-cover-all-of-them-b7g",
      "label": "resource management side of running three models on one box",
      "type": "a"
    },
    {
      "href": "https://dev.to/hannune/missed-calls-missed-revenue-the-local-first-phone-assistant-im-building-2086",
      "label": "Missed calls, missed revenue",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:49:38Z",
  "source_url": "https://dev.to/hannune/when-your-calls-cant-leave-your-infrastructure-the-compliance-constraint-that-shapes-the-whole-ai-4kec",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 4978,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/4bfc420fa6c4/pages/source.html",
  "local_html": "attachments/4bfc420fa6c4/pages/article.html",
  "body_base_path": "attachments/4bfc420fa6c4/pages/",
  "body_markdown_path": "attachments/4bfc420fa6c4/pages/article.md"
}