{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Carlos Chinchilla Corbacho",
  "title_listed": "Shapes are the type system of ML, and nobody checks them for you",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://dev.to/cchinchilladev/shapes-are-the-type-system-of-ml-and-nobody-checks-them-for-you-2f8k",
  "publish_date": "2026-09-24",
  "corpus_stage": "",
  "notes": "",
  "id": "170c950e3fd4",
  "collected_at": "2026-10-02T14:05:17",
  "title": "Shapes are the type system of ML, and nobody checks them for you - DEV Community",
  "body_markdown": "Originally published at\ncchinchilla.dev\n. Part 3 of From code to weights, a 12-part series on ML fundamentals for engineers.\nPart 2\nended on a signature. A batch of sequences of embedding rows is a tensor of shape\n[B, T, d]\n. That line is the closest thing this stack has to a type. This post reads it the way you'd read a function's signature, then shows what the runtime does when the signature is wrong: usually nothing.\nI read the broadcasting rule as torch writes it. Then I wrote four shape bugs on purpose and watched which ones the runtime caught. It caught one.\nThis is part of\na series\nfor engineers moving into ML engineering. Each part names one thing that does not behave the way software behaves. This one is the type system: there isn't one.\nA shape is a signature\nA tensor is an N-dimensional array plus a description of how it sits in memory.\n[B, T, d]\nsays:\nB\nsequences in the batch,\nT\npositions in each,\nd\nnumbers at each position. Read it the way you'd read\nf(batch: List[List[Vec]])\n, with one difference that matters later: the runtime never looks at the names, only at the sizes.\nThe memory description is the part software engineers already know. A tensor of shape\n[4, 8, 16]\nhas strides\n(128, 16, 1)\n: move one step along the first axis and you skip 128 numbers. Transpose the first two axes and the shape becomes\n[8, 4, 16]\nwith strides\n(16, 128, 1)\n, the same memory read in a different order;\ndata_ptr\ndoesn't change. That's why\ntranspose\nis free and why a\nview\nthat flattens the result refuses, with a message that says so, while\nreshape\ncopies when it has to. A tensor is a pointer, a shape, and a stride tuple; the view operations you'll call are arithmetic on the last two.\nEvery operation has a shape signature of its own: what goes in, what comes out, which axis disappears.\nmean(dim=1)\non\n[B, T, d]\nreturns\n[B, d]\n; the\nT\nis gone. A linear layer maps\n[B, T, d]\nto\n[B, T, d_out]\n; only the last axis changes. Writing the signature next to the call is the discipline this post is about; nothing else will write it.\nThe rule is five lines\nTwo tensors of different shapes can still be added, and the rule that decides how is broadcasting. torch keeps a readable copy of it in Python, under\ntorch.broadcast_shapes\n, which calls\n_broadcast_shapes\nin\ntorch/_refs/__init__.py\n(lines 388-470 at v2.13.0). Stripped of the symbolic-shape machinery, the rule is this:\ncommon_shape\n=\n[\n1\n]\n*\nreduce\n(\nmax\n,\n(\nlen\n(\nshape\n)\nfor\nshape\nin\nshapes\n))\nfor\narg_idx\n,\nshape\nin\nenumerate\n(\nshapes\n):\nfor\nidx\nin\nrange\n(\n-\n1\n,\n-\n1\n-\nlen\n(\nshape\n),\n-\n1\n):\n# align from the right\nif\ncommon_shape\n[\nidx\n]\n==\n1\n:\ncommon_shape\n[\nidx\n]\n=\nshape\n[\nidx\n]\n# 1 stretches to anything\nif\nshape\n[\nidx\n]\n==\n1\n:\ncontinue\n# ... in either direction\ntorch\n.\n_check\n(\ncommon_shape\n[\nidx\n]\n==\nshape\n[\nidx\n],\n# otherwise sizes must match\nlambda\n:\nf\n\"\nAttempting to broadcast a dimension of length\n{\nshape\n[\nidx\n]\n}\nat\n{\nidx\n}\n!\n\"\n)\nreturn\ncommon_shape\nEnter fullscreen mode\nExit fullscreen mode\nAlign the shapes from the right. A size of 1 stretches to match the other. Equal sizes pass. Anything else raises. That's the whole rule. The eager path runs the same rule in C++ and raises with a shorter message. It exists so that you can add a bias of shape\n[d]\nto\n[B, T, d]\nwithout a loop or a copy.\nThe rule has one property that the rest of this post is about: it decides by size, not by meaning.\n[8, 16] + [16]\nworks because 16 equals 16.\n[8, 16] + [8]\nraises, because 8 is not 16 and 8 is not 1. Whether the 16 on the right is a sequence length or a hidden size or a batch, the rule has no idea, and neither will you when it's wrong. When two of your axes happen to be the same size, it can't tell them apart either. The data scientist has leaned on this rule in numpy for years; the software engineer is meeting a coercion rule with no declared types. Both should read it as the second thing, not the first.\nThe bug that only happens at batch size sixteen\nThe first bug. Attention scores have shape\n[B, T, T]\n, one\nT × T\nmatrix per sequence. A padding mask has shape\n[B, T]\n: which positions in each sequence are real. In its additive form (0 for real,\n-inf\nfor padding), adding it to the scores switches padding off. The correct form is\nscores + mask[:, None, :]\n: the\nNone\nmakes the mask\n[B, 1, T]\n, so it stretches across the query axis and masks the keys.\nDrop the\nNone\n. With\nB = 8\nand\nT = 16\n,\nscores + mask\nraises: 16 is not 8. With\nB = 16\nand\nT = 16\n, it runs. At\nB = 1\nit runs and is correct by accident: 1 is what the\nNone\nwould have inserted.\nThe rule aligns\n[16, 16]\nagainst\n[16, 16, 16]\nfrom the right and reads the mask as\n[1, B, T]\n. The mask meant for sequence\nb\nlands on query row\nb\nof every sequence. On random lengths I measured a third of the positions masked wrong and a fifth of the attention weight on padding. The softmax after it is finite, the loss is finite, and the run continues. No warning, because there is nothing to warn about: the rule was followed.\nThe original post has a widget for this that doesn't survive being pasted into a feed:\nmove the two sliders\nuntil\nB\nand\nT\nmeet and the verdict flips from an error to a silent result. Then pick\n[B, T, 1]\n: that mask runs at every size and is wrong at every size, because it switches off queries instead of keys. It's the loud version — a padded query's row is all\n-inf\nand its softmax is NaN. The rule is the same in all three cases.\nThe second bug is the classic one. Predictions of shape\n[256]\n, targets of shape\n[256, 1]\n, subtracted for a squared error. The rule stretches both to\n[256, 256]\n, every prediction against every target. The mean of that is a number. It trains.\nI trained a one-feature linear regression through it for two thousand steps. The loss fell and settled at 9.7. The fitted slope was 0.000, the intercept was the mean of the targets, and the R² was 0.0. The model learned to ignore its input. That's the best it can do when the loss compares it with everyone else's target. The library loss,\nF.mse_loss\n, warns about this mismatch. A hand-written\n((pred - target) ** 2).mean()\ndoesn't, and the hand-written form is the one that ends up in notebooks.\nThe third bug never raises where it happens. On a tensor of shape\n[16, 16, 64]\n,\nmean(dim=0)\nand\nmean(dim=1)\nboth return\n[16, 64]\n. Different numbers. Same shape. Nothing downstream can tell. On\n[3, 5, 7]\nthe same two calls return\n[5, 7]\nand\n[3, 7]\n, and the first thing that tries to combine them raises. The bug was there in both cases; only the sizes decided whether it was visible.\nThe fourth is the control.\n[3, 5, 7]\nplus\n[3, 5]\nfails because 7 is not 5. It was caught for the one reason the others weren't: the sizes differ.\nBe fair about the rule. It's doing what it was asked to. The alternative exists: a runtime that carries axis names and refuses to add\nT\nto\nB\n. torch shipped one as a prototype, named tensors. It never left prototype, and 2.13 removed it to cut overhead and code bloat. Part of why it never caught on: every library in the stack would have had to agree on the names, and sizes are the only thing all of them share. The rule is right. It is not a type check, and nothing in the stack is.\nPadding is a shape decision\nSequences don't come in rectangles. Three sequences of lengths 5, 3, and 1 become a\n[3, 5]\ntensor only by padding, and for those lengths 40% of the rectangle is padding. Every operation over\nT\nthen needs to know which cells are real. That knowledge is a second tensor, the mask, and it has to travel with the first.\nWhich side you pad on is part of the signature. Pad on the right and the last real token sits at index\nlength - 1\n: 4, 2, 0 for the three above. Pad on the left and it sits at 4 for all of them. Decoders that generate one token at a time prefer that; the position to read is always the last. Either choice works. Mixing them is the bug. Code written for right padding and fed a left-padded batch reads padding, or the wrong token, where it expects the last one.\nThe mask is not optional even when the model doesn't attend. Mean-pool that\n[3, 5]\nbatch without it and the length-1 sequence's vector averages one real row with four of padding. On\narange(1, 31).reshape(3, 5, 2)\nthe unmasked mean of that sequence is\n[25, 26]\n; the masked mean is\n[21, 22]\n, its only real row. Part 2's pooling code divides by the mask sum for this reason. The division is the signature, written out.\nMemory is shape times bytes\nA tensor's cost is arithmetic. Activations of shape\n[32, 2048, 4096]\nare 268 million numbers: 1.07 GB in fp32, 537 MB in bf16, 268 MB in int8. That's the bill, and you need nothing about the model to compute it: the shape and the width of one number. It decides whether a batch fits on a card.\nThe second widget on\nthe original post\nis that multiplication, live: drag the three sizes and switch the format. At the defaults one block costs 77 GB in fp32 and the two square rows are 89% of it; pull\nT\ndown to 128 and they fall to a third. Those two rows are the next part's subject.\nWhat each format gives up is also arithmetic. fp32 keeps 23 bits of mantissa. bf16 keeps 7 and the whole fp32 exponent: it writes π as 3.140625 and rounds\n1 + 0.004\nto\n1.0078125\n, but its range is fp32's, to within the last rounding step. fp16 keeps 10 bits and only five of exponent: it tops out at 65,504, and small gradients underflow to zero unless the loss is scaled first. That's why training moved to bf16 and serving reaches for int8. Each is a decision about which precision a tensor doesn't need.\nThe batch axis is the one you control at serving time, and it moves two numbers in opposite directions. I timed a\n[B, 4096]\nby\n[4096, 4096]\nmatmul on my laptop's CPU. It costs 447 µs per row at\nB = 1\nand 22 µs per row at\nB = 512\n: twenty times the throughput. The call itself went from 0.45 ms to 11.4 ms; each request waited twenty-five times longer, plus whatever it spent waiting for the batch to fill. Five repetitions on one laptop: the ratio is the finding, not the digits. Dynamic batching is the knob between those two numbers, and its setting is a product decision wearing a shape.\nWhat the ML engineer watches\nFour habits, and the first two are one habit.\nAnnotate shapes at function boundaries.\njaxtyping\nwith\nbeartype\nturns the signature into a check that runs on every call:\n@jaxtyped\n(\ntypechecker\n=\nbeartype\n)\ndef\npool\n(\nx\n:\nFloat\n[\nTensor\n,\n\"\nB T d\n\"\n],\nmask\n:\nBool\n[\nTensor\n,\n\"\nB T\n\"\n])\n->\nFloat\n[\nTensor\n,\n\"\nB d\n\"\n]:\nm\n=\nmask\n[:,\n:,\nNone\n].\nfloat\n()\nreturn\n(\nx\n*\nm\n).\nsum\n(\n1\n)\n/\nm\n.\nsum\n(\n1\n)\nEnter fullscreen mode\nExit fullscreen mode\nReduce the wrong axis inside and it raises on the return, because\n[T, d]\nis not\n[B, d]\n. Pass the column-shaped target into the squared error and it raises before the rule can stretch anything. Two of the bugs above, caught at the boundary (the reduce only when\nB\n≠\nT\n), with the shape written where the next reader looks. The batch-sixteen bug, and the wrong-axis reduce when\nB\nequals\nT\n, need the next habit.\nTest with sizes that can't collide.\nThe bug at batch size sixteen is invisible to the annotation. When\nB\nand\nT\nare both 16, the names bind to the same number and the check passes; so does the wrong-axis reduce, because\n[T, d]\nand\n[B, d]\nare the same\n[16, 64]\n. The fix is in the tests, not the code: use\nB = 3\n,\nT = 5\n,\nd = 7\n. Every dropped\nNone\nand every mix-up between\nB\n,\nT\nand\nd\nthen produces a size that matches nothing, and the runtime raises. Size-1 axes still stretch at any size, and the two\nT\ns of the scores are always equal; those need the annotation or a test that checks values. Round numbers in tests are how the bug at batch size sixteen survives to production. That's why the first two are one habit: the annotation is only as strict as the sizes it sees.\nMask and padding side are one decision.\nRecord the side next to the tokenizer config. Compute the last-token index from the mask, never from\nT\n. Reject a batch whose mask disagrees with its padding. The pooling code in Part 2 already divides by the mask sum; make sure yours does.\nPrice a tensor before allocating it.\nShape times bytes, at the batch you'll serve, in the format you'll serve it in. That multiplication has an answer before a single weight is loaded.\nNone of this is modelling work either. A shape is the one type this stack never checks: a device mismatch raises, a wrong axis broadcasts. The check is yours to write.\nPart 3 of twelve. A shape is a signature that nobody verifies: broadcasting decides by size, so a wrong axis raises only when the sizes happen to differ, and a bug can live at one batch size alone. The attention scores in the first widget and the two amber rows in the second carry\nT\ntwice, one\nT × T\nmatrix per sequence (per head, in the second).\nThe next part\nreads where that square comes from and what it costs. If you arrived here directly,\nPart 1\nreads the tokenizer and\nPart 2\nthe table a token id indexes.\nWritten with AI assistance. See\nAI policy\n.",
  "metadata": {
    "title": "Shapes are the type system of ML, and nobody checks them for you - DEV Community",
    "description": "Originally published at cchinchilla.dev. Part 3 of From code to weights, a 12-part series on ML... Tagged with machinelearning, python, ai, beginners.",
    "keywords": "machinelearning, python, ai, beginners, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/cchinchilladev/shapes-are-the-type-system-of-ml-and-nobody-checks-them-for-you-2f8k",
    "og:title": "Shapes are the type system of ML, and nobody checks them for you",
    "og:description": "Originally published at cchinchilla.dev. Part 3 of From code to weights, a 12-part series on ML...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9eiibv9r1xojdewnjb2d.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Shapes are the type system of ML, and nobody checks them for you",
    "twitter:description": "Originally published at cchinchilla.dev. Part 3 of From code to weights, a 12-part series on ML...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9eiibv9r1xojdewnjb2d.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/170c950e3fd4/code/body-000.txt",
      "chars": 612,
      "language": "text"
    },
    {
      "file": "attachments/170c950e3fd4/code/body-001.txt",
      "chars": 190,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://cchinchilla.dev/posts/fcw-tensors-shapes-type-system/",
      "label": "cchinchilla.dev",
      "type": "a"
    },
    {
      "href": "https://cchinchilla.dev/posts/fcw-embeddings-lookup-table/",
      "label": "Part 2",
      "type": "a"
    },
    {
      "href": "https://cchinchilla.dev/series/",
      "label": "a series",
      "type": "a"
    },
    {
      "href": "https://cchinchilla.dev/posts/fcw-attention-learned-weighted-average/",
      "label": "The next part",
      "type": "a"
    },
    {
      "href": "https://cchinchilla.dev/posts/fcw-tokens-models-dont-read-text/",
      "label": "Part 1",
      "type": "a"
    },
    {
      "href": "https://cchinchilla.dev/ai-policy/",
      "label": "AI policy",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:49:57Z",
  "source_url": "https://dev.to/cchinchilladev/shapes-are-the-type-system-of-ml-and-nobody-checks-them-for-you-2f8k",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 12940,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/170c950e3fd4/pages/source.html",
  "local_html": "attachments/170c950e3fd4/pages/article.html",
  "body_base_path": "attachments/170c950e3fd4/pages/",
  "body_markdown_path": "attachments/170c950e3fd4/pages/article.md"
}