{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Seth Wheeler",
  "title_listed": "Checking a Cost Model Against a Stranger's Config File",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "工程实践",
  "is_paid_flag": "否",
  "url": "https://dev.to/megapixel99/checking-a-cost-model-against-a-strangers-config-file-390b",
  "publish_date": "2026-08-30",
  "corpus_stage": "",
  "notes": "",
  "id": "4710c0848e9f",
  "collected_at": "2026-10-02T13:51:57",
  "title": "Checking a Cost Model Against a Stranger's Config File - DEV Community",
  "body_markdown": "I wrote\na cost model for streaming a mixture-of-experts model off an SSD\n, and the part I was proudest of was that it gated itself twice against artifacts I had nothing to do with. Gate 1 reproduced a third-party runtime's on-disk container byte-for-byte. Gate 2 predicted the size of a model conversion published by different people using a different tool, and came within 1.1% against a 2% tolerance.\nThe experiment behind that post has a limitations section I now want to revisit. It listed an ambiguity in the model's\nconfig.json\n, bounded the damage, and moved on:\ndense_mlp_idx=2\nis ambiguous: one dense layer, or layers 0 and 1. This takes one, which moves bytes/token by 2.4%. Immaterial to the conclusion, unresolved regardless.\nThe bound was right. The assumption was wrong. Correcting it took Gate 2 from 1.1% off to\n3.4% off\n, which is outside the tolerance it had been passing, and the reason is more interesting than the arithmetic.\nThe header states the fact outright\nA config file describes a model's architecture in the vocabulary of the framework that trained it; reading one therefore means inferring what a field name meant to its author.\ndense_mlp_idx=2\ncould reasonably be \"the layer at index 2 is dense\" or \"layers below index 2 are dense\", and those give 41 and 40 mixture-of-experts layers respectively.\nI eventually downloaded the model in GGUF form (the container format llama.cpp uses) to run it. GGUF puts a block of key/value metadata at the front of the file, and that metadata is written by the conversion script after it has already resolved every question of this kind. So the answer was sitting in a file on my own disk:\ninkling.block_count\n=\n42\ninkling.dense_block_count\n=\n2\ninkling.expert_count\n=\n256\ninkling.expert_used_count\n=\n6\ninkling.expert_feed_forward_length\n=\n2048\ninkling.expert_shared_count\n=\n2\nEnter fullscreen mode\nExit fullscreen mode\ndense_block_count = 2\n. Forty mixture-of-experts layers, not forty-one.\nReading it took about forty lines of Python. The parser was wrong the first time in a way worth repeating: GGUF stores arrays of strings length-prefixed, and I read only the first few elements without consuming the rest. Every subsequent offset was then shifted, and the parse failed later with an invalid-looking type code rather than at the array that caused it. If you write one of these, consume every element even when you intend to display four.\nWhat the correction does\nThe expert size does not change, so the per-token cost moves purely through the layer count:\nassumed (1 dense)\nheader-confirmed (2 dense)\nmixture-of-experts layers\n41\n40\nrouted experts on disk\n148.58 GB\n144.96 GB\nstreamed per token\n3.482 GB\n3.397 GB\nGate 2 against a published 153.5 GB\n151.77 GB, 1.1% off\n148.23 GB,\n3.4% off\nThe model now under-predicts the published file by 5.27 GB, and the script does what I built it to do: it prints\nGATE 2: FAIL\nand refuses to emit any token-rate estimate at all. That refusal was written months earlier for exactly this case; it is the only reason I noticed, rather than quietly publishing a slightly different number.\nTwo errors, pointing opposite ways\nHere's the mechanism, and it's the whole point of the post.\nGate 2 predicts a whole container: routed experts, shared experts, attention, embeddings, dense layers. The 1.1% agreement was the sum of two errors that happened to have opposite signs.\nError one, mine.\nAn extra mixture-of-experts layer added 256 experts of 14,155,776 bytes each, inflating the prediction by 3.62 GB of routed experts; net of the dense layer it displaced,\n+3.54 GB\n.\nError two, structural and disclosed.\nThe model does not attempt the vision and audio encoders, the multi-token-prediction module, the RMS norms, or the fact that the published conversion keeps non-expert tensors at 8 bits rather than 4. With the layer count correct, those account for\n-5.27 GB\n.\nAdd +3.54 to -5.27 and you land 1.73 GB low, which reads as 1.1% and looks like a model that understands the format. Remove the inflation and the deflation stands alone at 5.27 GB, or 3.4%.\nThe residual was always that large. The agreement was partly luck, and a tolerance was never evidence that the two sides matched for the same reason.\nI want to be precise about what this does and does not undermine, because \"a gate failed\" is easy to over-read.\nGate 1 is untouched and still byte-exact.\nIt compares computed integers against a published layout with no tolerance at all: expert stride\n1,769,472\n, per-layer blob\n452,984,832\n, 256 experts, 40 layers. Equality has no room for two errors to meet inside it; that is the argument for preferring an exact gate to a tolerance wherever the domain offers one.\nThe load-bearing number got better, not worse.\nThe whole model exists to produce one figure: bytes streamed per token. That figure depends on the expert stride (Gate 1 verifies it exactly), the number of experts routed per token, and the mixture-of-experts layer count. The header now confirms the last two outright, so 3.397 GB/token rests on three measured quantities where 3.482 rested on two measured and one inferred.\nSo a gate went from pass to fail while the number I actually care about became better grounded. That is only a contradiction if you think a container-size prediction and a per-token-cost prediction are the same claim. They share inputs and they fail differently, and this is the case that separates them.\nTwo independent checks agree with the corrected figure. A separate script that estimates per-token bytes from published file sizes rather than from architecture gives 3.405 GB against 3.397 computed directly, 0.2% apart. And when I finally ran the model, the kernel's page-in counters put actual disk traffic between 1.80 and 3.96 GB per generated token, a bracket that contains it.\nWhat I'd take from this\nThe rule I had been operating under is that a gate passing within tolerance means the model is right. It doesn't. It means the sum of every error sits inside the tolerance, and a sum can be small because the terms are small, or because they cancel. Those two situations look identical from outside and behave completely differently when you fix something: one improves, the other exposes what the cancellation was hiding.\nThere's a cheaper habit buried in this, too. I spent real effort inferring what\ndense_mlp_idx\nmeant, wrote a careful bound around my uncertainty, and shipped the inference. The artifact stated the fact outright in a header I could read in forty lines.\nWhen you find yourself bounding an inference about someone else's format, check first whether some downstream tool has already resolved it and written the answer down\n, because conversion scripts have to answer these questions to do their job at all.\nAnd the disclosed limitation earns its keep again. The last post's failure was attributable because the caveat had been written in advance; this one was detectable because the tolerance and the refusal-to-proceed were written in advance. I keep finding that the value isn't in the estimate, which gets superseded. It's in having decided, before the measurement, what would count as being wrong.\nThe code for this one isn't public. It's a research repo with no release, and I'd rather link nothing than link something nobody can run.",
  "metadata": {
    "title": "Checking a Cost Model Against a Stranger's Config File - DEV Community",
    "description": "I bounded an ambiguity in a config file at 2.4% and called it immaterial. The model's own header settled it months later, and correcting the assumption pushed a gate from 1.1% off to 3.4% off, outside the tolerance it had been passing. Tagged with llm, measurement, mixtureofexperts.",
    "keywords": "llm, measurement, mixtureofexperts, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/megapixel99/checking-a-cost-model-against-a-strangers-config-file-390b",
    "og:title": "Checking a Cost Model Against a Stranger's Config File",
    "og:description": "I bounded an ambiguity in a config file at 2.4% and called it immaterial. The model's own header settled it months later, and correcting the assumption pushed a gate from 1.1% off to 3.4% off, outside the tolerance it had been passing.",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjw6eqm0irq6m4lodzb4a.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Checking a Cost Model Against a Stranger's Config File",
    "twitter:description": "I bounded an ambiguity in a config file at 2.4% and called it immaterial. The model's own header settled it months later, and correcting the assumption pushed a gate from 1.1% off to 3.4% off, outside the tolerance it had been passing.",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjw6eqm0irq6m4lodzb4a.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/4710c0848e9f/code/body-000.txt",
      "chars": 235,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://sethwheeler.dev/blog/ssd-streaming-prediction/",
      "label": "a cost model for streaming a mixture-of-experts model off an SSD",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:40Z",
  "source_url": "https://dev.to/megapixel99/checking-a-cost-model-against-a-strangers-config-file-390b",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 7270,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/4710c0848e9f/pages/source.html",
  "local_html": "attachments/4710c0848e9f/pages/article.html",
  "body_base_path": "attachments/4710c0848e9f/pages/",
  "body_markdown_path": "attachments/4710c0848e9f/pages/article.md"
}