{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Wesam Khallaf",
  "title_listed": "What Does `keepdim` Do in PyTorch? (And the Silent Bug When You Forget It)",
  "keywords": "",
  "domain": "机器学习",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/pytorchfromgroundup/what-does-keepdim-do-in-pytorch-and-the-silent-bug-when-you-forget-it-17p1",
  "publish_date": "2026-07-24",
  "corpus_stage": "",
  "notes": "",
  "id": "08319e9e0723",
  "collected_at": "2026-10-02T13:54:01",
  "title": "What Does `keepdim` Do in PyTorch? (And the Silent Bug When You Forget It) - DEV Community",
  "body_markdown": "You wrote\nx.sum(dim=1)\nto get a row total, divided the original tensor by it, and either got\nRuntimeError: The size of tensor a (3) must match the size of tensor b (2)\n— or, worse, got no error at all and numbers that were quietly wrong. Someone told you to add\nkeepdim=True\nand it fixed itself. But nobody said\nwhy\n.\nThe short answer\nEvery reduction —\nsum\n,\nmean\n,\nmax\n,\nmin\n— collapses a dimension, and by default that dimension\ndisappears\nfrom the shape.\nkeepdim=True\ntells PyTorch to keep it as a size-1 axis instead. A\n(2, 3)\ntensor reduced over\ndim=1\nbecomes\n(2,)\nnormally, but\n(2, 1)\nwith\nkeepdim=True\n. Same numbers either way — the only difference is a single\n1\nin the shape. That\n1\nis what lets the result\nbroadcast\nback against the original tensor.\nimport\ntorch\nm\n=\ntorch\n.\ntensor\n([[\n1.\n,\n2.\n,\n3.\n],\n[\n4.\n,\n5.\n,\n6.\n]])\nprint\n(\nm\n.\nsum\n(\ndim\n=\n1\n).\nshape\n)\n# torch.Size([2])\nprint\n(\nm\n.\nsum\n(\ndim\n=\n1\n,\nkeepdim\n=\nTrue\n).\nshape\n)\n# torch.Size([2, 1])\nEnter fullscreen mode\nExit fullscreen mode\nThat's the whole feature. The interesting part is\nwhy the missing\n1\nbreaks things\n— sometimes loudly, sometimes silently.\nFirst: what does\ndim\ndo in a reduction?\nBefore\nkeepdim\nmakes sense, you need the mental model for\ndim\n.\nThe dimension you name is the one that disappears.\nm\n=\ntorch\n.\ntensor\n([[\n1.\n,\n2.\n,\n3.\n],\n[\n4.\n,\n5.\n,\n6.\n]])\n# shape (2, 3)\nprint\n(\nm\n.\nsum\n())\n# tensor(21.)          — no dim: collapse everything to a scalar\nprint\n(\nm\n.\nsum\n(\ndim\n=\n0\n))\n# tensor([5., 7., 9.]) — collapse the rows, shape (3,)\nprint\n(\nm\n.\nsum\n(\ndim\n=\n1\n))\n# tensor([ 6., 15.])   — collapse the columns, shape (2,)\nEnter fullscreen mode\nExit fullscreen mode\ndim=0\nadds\ndown\neach column (the size-2 axis vanishes, leaving\n(3,)\n).\ndim=1\nadds\nacross\neach row (the size-3 axis vanishes, leaving\n(2,)\n). If you can predict the output shape before running, you understand reductions. The dim you name is gone.\nSo what does\nkeepdim\nactually do?\nBy default the collapsed axis is deleted.\nkeepdim=True\nleaves it behind as a length-1 placeholder:\nrow_sum\n=\nm\n.\nsum\n(\ndim\n=\n1\n)\n# shape (2,)\nrow_sum_k\n=\nm\n.\nsum\n(\ndim\n=\n1\n,\nkeepdim\n=\nTrue\n)\n# shape (2, 1)\nprint\n(\nrow_sum\n)\n# tensor([ 6., 15.])\nprint\n(\nrow_sum_k\n)\n# tensor([[ 6.],\n#         [15.]])\nEnter fullscreen mode\nExit fullscreen mode\nIdentical values. The\nkeepdim\nversion is just wrapped so each row sum sits in its own row. That shape —\n(2, 1)\ninstead of\n(2,)\n— is the entire point, because of what happens next.\nWhy size-1 matters: broadcasting back against the original\nThe most common reason to reduce a tensor is to then\nuse that result on the original\n— normalize each row to sum to 1, subtract each column's mean, divide by a per-row max. Every one of those is\noriginal (op) reduction\n, and for it to line up, the reduction has to broadcast.\nPyTorch broadcasting aligns shapes from the\nright\nand treats a\n1\nas \"stretch me to fit.\" A\n(2, 1)\nsum broadcasts cleanly against a\n(2, 3)\ntensor: the\n1\nstretches across the 3 columns. A bare\n(2,)\nsum does not line up at all.\n# Correct: keepdim=True → (2, 1) broadcasts against (2, 3)\nnormed\n=\nm\n/\nm\n.\nsum\n(\ndim\n=\n1\n,\nkeepdim\n=\nTrue\n)\nprint\n(\nnormed\n)\n# tensor([[0.1667, 0.3333, 0.5000],\n#         [0.2667, 0.3333, 0.4000]])\nprint\n(\nnormed\n.\nsum\n(\ndim\n=\n1\n))\n# tensor([1., 1.]) — every row sums to 1 ✅\nEnter fullscreen mode\nExit fullscreen mode\nDrop the\nkeepdim\nand this exact line either crashes or lies to you. Which one you get depends on the shape — and that's the part almost every tutorial skips.\nThe trap: the same bug fails two completely different ways\nCase 1 — a non-square tensor: it errors (the lucky case)\nOur\nm\nis\n(2, 3)\n. Without\nkeepdim\n, the row sum is\n(2,)\n, and PyTorch tries to align\n(2, 3)\nagainst\n(2,)\nfrom the right: 3 vs 2. Mismatch.\nm\n=\ntorch\n.\ntensor\n([[\n1.\n,\n2.\n,\n3.\n],\n[\n4.\n,\n5.\n,\n6.\n]])\n# (2, 3)\nnormed\n=\nm\n/\nm\n.\nsum\n(\ndim\n=\n1\n)\n# (2, 3) / (2,)\n# RuntimeError: The size of tensor a (3) must match the size of\n# tensor b (2) at non-singleton dimension 1\nEnter fullscreen mode\nExit fullscreen mode\nThis is the\ngood\noutcome. PyTorch stops you immediately, and the error text even names the two sizes that don't match. (If that message looks familiar, it's the same shape-mismatch error covered in\nPyTorch Broadcasting Explained\n.)\nCase 2 — a square tensor: it runs, and it's silently wrong (the dangerous case)\nNow make the tensor square. The row-sum vector is length 3, the tensor has 3 columns, so the shapes\nhappen\nto broadcast — just not the way you meant.\nm\n=\ntorch\n.\ntensor\n([[\n1.\n,\n2.\n,\n3.\n],\n[\n4.\n,\n5.\n,\n6.\n],\n[\n7.\n,\n8.\n,\n9.\n]])\n# (3, 3)\nwrong\n=\nm\n/\nm\n.\nsum\n(\ndim\n=\n1\n)\n# (3, 3) / (3,)  — no error!\ncorrect\n=\nm\n/\nm\n.\nsum\n(\ndim\n=\n1\n,\nkeepdim\n=\nTrue\n)\n# (3, 3) / (3, 1)\nprint\n(\nwrong\n.\nsum\n(\ndim\n=\n1\n))\n# tensor([0.4250, 1.2500, 2.0750])  ← NOT 1s ❌\nprint\n(\ncorrect\n.\nsum\n(\ndim\n=\n1\n))\n# tensor([1., 1., 1.])              ← correct ✅\nEnter fullscreen mode\nExit fullscreen mode\nNo exception. No warning. The row sums aren't 1 because the\n(3,)\nvector aligned to the\ncolumns\ninstead of the rows — PyTorch divided the wrong way. Your data looks normalized, your code ran clean, and the mistake surfaces as a mysteriously bad model three hours later.\nThis is why you add\nkeepdim=True\nby reflex, not by debugging.\nOn a square batch it's the difference between correct and quietly-corrupted, with nothing on screen to tell you which one you got.\nA rule you can actually remember\nWhenever you reduce a tensor and then use the result in arithmetic with the original, pass\nkeepdim=True\n.\nThat's it. If the reduction is the final answer (you just want the row totals and you're done), you don't need it. The moment the reduction feeds back into the original tensor, you do.\nDoes\nkeepdim\nwork on\nmax\n,\nmin\n, and\nargmax\ntoo?\nYes —\nmax(dim)\nand\nmin(dim)\nreturn\ntwo\ntensors (the values and their indices), and\nkeepdim\nkeeps the axis on both:\nvals\n,\nidxs\n=\nm\n.\nmax\n(\ndim\n=\n1\n,\nkeepdim\n=\nTrue\n)\nprint\n(\nvals\n.\nshape\n,\nidxs\n.\nshape\n)\n# torch.Size([3, 1]) torch.Size([3, 1])\nEnter fullscreen mode\nExit fullscreen mode\nThis is handy for a \"divide each row by its own max\" style normalization, where you need that\n(3, 1)\nshape to broadcast.\nargmax(dim, keepdim=True)\nworks the same way.\nCommon mistakes and gotchas\nForgetting\nkeepdim\non a square tensor.\nThe headline trap: it broadcasts to the wrong axis with no error. Rows-or-columns bugs on square data are almost always a missing\nkeepdim\n.\nThinking\nkeepdim\nchanges the numbers.\nIt never does. The values are identical; only the shape gains a\n1\n. If your totals changed, something else is wrong.\nReaching for\nunsqueeze\nafter the fact.\nx.sum(dim=1).unsqueeze(1)\ngives the same\n(N, 1)\nshape, but\nx.sum(dim=1, keepdim=True)\nsays it in one step and can't get the axis position wrong.\nAssuming \"it ran\" means it was right.\nAs Case 2 shows, running successfully is not the same as broadcasting correctly. Print\n.shape\non the reduction before you divide.\nUsing\nkeepdim\nwhen you don't need it.\nIf the reduced value is your final output (not fed back into the original tensor), the trailing\n1\nis just clutter — drop it.\nRecap\nkeepdim=True\nkeeps a reduced dimension as a size-1 axis instead of deleting it, so the result still broadcasts against the original tensor. Forget it and you get one of two failures: a shape-mismatch\nRuntimeError\non non-square tensors (annoying but safe), or a\nsilent wrong answer\non square tensors (dangerous). The habit that saves you: any time you reduce and then combine with the original — normalizing rows, centering columns, scaling by a per-row max — pass\nkeepdim=True\n.\nThis is one idea from my book\nPyTorch From Ground Up\n, which builds everything from tensors upward so nothing stays vague. If it helped, you can grab the\nfree PyTorch tensor cheat-sheet here\n, run every example from the book in the\ncompanion notebooks on GitHub\n, get the\ndigital edition on Leanpub\n, or find the\nfull paperback on Amazon here\n.",
  "metadata": {
    "title": "What Does `keepdim` Do in PyTorch? (And the Silent Bug When You Forget It) - DEV Community",
    "description": "You wrote x.sum(dim=1) to get a row total, divided the original tensor by it, and either got... Tagged with machinelearning, python, tutorial.",
    "keywords": "machinelearning, python, tutorial, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/pytorchfromgroundup/what-does-keepdim-do-in-pytorch-and-the-silent-bug-when-you-forget-it-17p1",
    "og:title": "What Does `keepdim` Do in PyTorch? (And the Silent Bug When You Forget It)",
    "og:description": "You wrote x.sum(dim=1) to get a row total, divided the original tensor by it, and either got...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fftj6xghyzhzii8uh24lz.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@wesamkhallaf",
    "twitter:title": "What Does `keepdim` Do in PyTorch? (And the Silent Bug When You Forget It)",
    "twitter:description": "You wrote x.sum(dim=1) to get a row total, divided the original tensor by it, and either got...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fftj6xghyzhzii8uh24lz.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/08319e9e0723/code/body-000.txt",
      "chars": 203,
      "language": "text"
    },
    {
      "file": "attachments/08319e9e0723/code/body-001.txt",
      "chars": 333,
      "language": "text"
    },
    {
      "file": "attachments/08319e9e0723/code/body-002.txt",
      "chars": 199,
      "language": "text"
    },
    {
      "file": "attachments/08319e9e0723/code/body-003.txt",
      "chars": 256,
      "language": "text"
    },
    {
      "file": "attachments/08319e9e0723/code/body-004.txt",
      "chars": 251,
      "language": "text"
    },
    {
      "file": "attachments/08319e9e0723/code/body-005.txt",
      "chars": 399,
      "language": "text"
    },
    {
      "file": "attachments/08319e9e0723/code/body-006.txt",
      "chars": 112,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://dev.to/pytorchfromgroundup/pytorch-broadcasting-explained-the-3-rules-and-the-silent-bug-that-bites-everyone-3606",
      "label": "PyTorch Broadcasting Explained",
      "type": "a"
    },
    {
      "href": "https://payhip.com/b/7ukxh",
      "label": "free PyTorch tensor cheat-sheet here",
      "type": "a"
    },
    {
      "href": "https://github.com/pytorch-from-ground-up/book_code",
      "label": "companion notebooks on GitHub",
      "type": "a"
    },
    {
      "href": "https://leanpub.com/pytorchfromgroundup",
      "label": "digital edition on Leanpub",
      "type": "a"
    },
    {
      "href": "https://www.amazon.com/dp/B0H8WMCV33",
      "label": "full paperback on Amazon here",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:51Z",
  "source_url": "https://dev.to/pytorchfromgroundup/what-does-keepdim-do-in-pytorch-and-the-silent-bug-when-you-forget-it-17p1",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 7893,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/08319e9e0723/pages/source.html",
  "local_html": "attachments/08319e9e0723/pages/article.html",
  "body_base_path": "attachments/08319e9e0723/pages/",
  "body_markdown_path": "attachments/08319e9e0723/pages/article.md"
}