{
  "source_file": "0930",
  "source_platform": "Substack",
  "author": "Arvind Narayanan & Sayash Kapoor / AI Snake Oil",
  "title_listed": "The Problem with LLM Benchmarks",
  "keywords": "LLM基准,评测,问题",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://www.aisnakeoil.com/p/the-problem-with-llm-benchmarks",
  "publish_date": "2024-09",
  "corpus_stage": "评测/评估",
  "notes": "LLM基准的问题",
  "id": "9bcaaeeed296",
  "collected_at": "2026-10-01T20:45:33",
  "title": "AI as Normal Technology | Sayash Kapoor | Substack",
  "body_markdown": "[![AI as Normal Technology](https://substackcdn.com/image/fetch/$s_!YzQY!,w_40,h_40,c_fill,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6780dda8-5879-4789-bf02-b3ea43a9f85e_1000x1000.png)](https://www.normaltech.ai/)\n# [![AI as Normal Technology](../images/013.jpeg)](https://www.normaltech.ai/)\nSubscribeSign in\n[Home](https://www.normaltech.ai/)\n[Notes](https://www.normaltech.ai/notes)\n[Get the book](https://www.normaltech.ai/p/starting-reading-the-ai-snake-oil)\n[Book exercises](https://www.normaltech.ai/p/ai-snake-oil-exercises-and-discussion)\n[About us](https://www.normaltech.ai/about)\n[At a glance](https://www.normaltech.ai/p/at-a-glance)\n## Page not found\nLatestTopDiscussions\n[A big-tent or small-tent AI safety movement?](https://www.normaltech.ai/p/a-big-tent-or-small-tent-ai-safety)\n[The unstated disagreement that underpins safety debates](https://www.normaltech.ai/p/a-big-tent-or-small-tent-ai-safety)\n4 hrs ago • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n101\n5\n12\n![](../images/014.jpeg)\n### September 2026\n[AI existential risk probabilities are (still) too unreliable to inform policy](https://www.normaltech.ai/p/p-doom)\n[How speculation gets laundered through pseudo-quantification](https://www.normaltech.ai/p/p-doom)\nSep 28 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n152\n21\n25\n![](../images/032.jpeg)\n[The AI-as-Normal-Technology view of loss-of-control incidents](https://www.normaltech.ai/p/the-ai-as-normal-technology-view)\n[A middle ground between the cybersecurity and AI safety communities](https://www.normaltech.ai/p/the-ai-as-normal-technology-view)\nSep 14 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n274\n54\n49\n![](../images/body-20b1b779921e86eee151.jpeg)\n### August 2026\n[AI agents can't yet do open-ended AI research](https://www.normaltech.ai/p/ai-agents-cant-yet-do-open-ended)\n[Early evidence from two case studies](https://www.normaltech.ai/p/ai-agents-cant-yet-do-open-ended)\nAug 5 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n194\n27\n25\n![](../images/body-8c159ebce99973a987e4.jpeg)\n### July 2026\n[What will be left for us to work on?](https://www.normaltech.ai/p/what-will-be-left-for-us-to-work)\n[My keynote at ICML 2026](https://www.normaltech.ai/p/what-will-be-left-for-us-to-work)\nJul 13 • [Arvind Narayanan](https://substack.com/@aisnakeoil)\n349\n30\n61\n![](../images/body-8d5f0c9ab934fe15ed65.png)\n[Up the Stack: How AI’s Escape From the Commodity Trap Risks Enterprise Lock-in](https://www.normaltech.ai/p/up-the-stack-how-ais-escape-from)\n[Critics and boosters are both looking in the wrong place](https://www.normaltech.ai/p/up-the-stack-how-ais-escape-from)\nJul 9 • [Arvind Narayanan](https://substack.com/@aisnakeoil)\n173\n24\n31\n![](../images/body-8672dbce29f398d63f88.png)\n### June 2026\n[Why AI hasn’t replaced software engineers, and won’t](https://www.normaltech.ai/p/why-ai-hasnt-replaced-software-engineers)\n[Coding agents as normal technology](https://www.normaltech.ai/p/why-ai-hasnt-replaced-software-engineers)\nJun 11 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n600\n89\n110\n![](../images/body-4d4af90675819fbd8cfa.png)\n### May 2026\n[Did Google’s AI agents really build an operating system for $916?](https://www.normaltech.ai/p/did-googles-ai-agents-really-build)\n[The importance of independent evaluation](https://www.normaltech.ai/p/did-googles-ai-agents-really-build)\nMay 22 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n144\n25\n11\n![](../images/body-c55a02e97e21be5c3fee.webp)\n[Do AI Risks Require Extraordinary Government Intervention?](https://www.normaltech.ai/p/do-ai-risks-require-extraordinary)\n[Let’s not skip the hard work of AI governance](https://www.normaltech.ai/p/do-ai-risks-require-extraordinary)\nMay 21 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n85\n9\n19\n![](../images/body-d27afc93fa2cece2abf8.png)\n### April 2026\n[Open-world evaluations for measuring frontier AI capabilities](https://www.normaltech.ai/p/open-world-evaluations-for-measuring)\n[Introducing CRUX, a new project for evaluating AI on long, messy tasks](https://www.normaltech.ai/p/open-world-evaluations-for-measuring)\nApr 16 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n99\n7\n14\n![](../images/body-ab273e5917edef922b91.jpeg)\n### February 2026\n[New Paper: Towards a science of AI agent reliability](https://www.normaltech.ai/p/new-paper-towards-a-science-of-ai)\n[Quantifying the capability-reliability gap](https://www.normaltech.ai/p/new-paper-towards-a-science-of-ai)\nFeb 24 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n328\n36\n71\n![](../images/body-f1c48dacaae2beeacce5.jpeg)\n[AI Won’t Automatically Make Legal Services Cheaper](https://www.normaltech.ai/p/ai-wont-automatically-make-legal)\n[Applying the AI as Normal Technology framework to legal services](https://www.normaltech.ai/p/ai-wont-automatically-make-legal)\nFeb 12 • [Justin Curl](https://substack.com/@justincurl), [Sayash Kapoor](https://substack.com/@sayash), and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n86\n13\n15\n![](../images/body-9e068808a486ef1ed733.png)\n### January 2026\n[Fact checking Moravec's paradox](https://www.normaltech.ai/p/fact-checking-moravecs-paradox)\n[The famous aphorism is neither true nor useful](https://www.normaltech.ai/p/fact-checking-moravecs-paradox)\nJan 29 • [Arvind Narayanan](https://substack.com/@aisnakeoil)\n174\n10\n25\n![](../images/body-af746494c7547caf396f.png)\n### September 2025\n[A guide to understanding AI as normal technology](https://www.normaltech.ai/p/a-guide-to-understanding-ai-as-normal)\n[And a big change for this newsletter](https://www.normaltech.ai/p/a-guide-to-understanding-ai-as-normal)\nSep 9, 2025 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n234\n29\n42\n![](../images/body-abf6fbace29ee2728fae.png)\n### July 2025\n[Could AI slow science?](https://www.normaltech.ai/p/could-ai-slow-science)\n[Confronting the production-progress paradox](https://www.normaltech.ai/p/could-ai-slow-science)\nJul 16, 2025 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n325\n37\n70\n![](../images/body-7259dbb9e459f3e9d2e0.jpeg)\n### May 2025\n[AGI is not a milestone](https://www.normaltech.ai/p/agi-is-not-a-milestone)\n[There is no capability threshold that will lead to sudden impacts](https://www.normaltech.ai/p/agi-is-not-a-milestone)\nMay 1, 2025 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n199\n37\n40\n![](../images/body-1a43f643b5b6cd283a1b.png)\n### April 2025\n[AI as Normal Technology](https://www.normaltech.ai/p/ai-as-normal-technology)\n[A new paper that we will expand into our next book](https://www.normaltech.ai/p/ai-as-normal-technology)\nApr 15, 2025 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n636\n40\n132\n![](../images/body-11e01a825e7509751d75.jpeg)\n### December 2024\n[Is AI progress slowing down?](https://www.normaltech.ai/p/is-ai-progress-slowing-down)\n[Making sense of recent technology trends and claims](https://www.normaltech.ai/p/is-ai-progress-slowing-down)\nDec 18, 2024 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n171\n15\n29\n![](../images/body-40aefaeaf590a97d144b.png)\n[We Looked at 78 Election Deepfakes. Political Misinformation is not an AI Problem.](https://www.normaltech.ai/p/we-looked-at-78-election-deepfakes)\n[Technology Isn’t the Problem—or the Solution.](https://www.normaltech.ai/p/we-looked-at-78-election-deepfakes)\nDec 13, 2024 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n119\n6\n23\n![](../images/body-d8a9c7243857d00eeb5e.jpeg)\n### November 2024\n[Does the UK’s liver transplant matching algorithm systematically exclude younger patients?](https://www.normaltech.ai/p/does-the-uks-liver-transplant-matching)\n[Seemingly minor technical decisions can have life-or-death effects](https://www.normaltech.ai/p/does-the-uks-liver-transplant-matching)\nNov 11, 2024 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n103\n11\n7\n![](../images/body-f707e0bfb4942e74aaa2.jpeg)\n### October 2024\n[FAQ about the book and our writing process](https://www.normaltech.ai/p/faq-about-the-book-and-our-writing)\n[What's in the book and how we wrote it](https://www.normaltech.ai/p/faq-about-the-book-and-our-writing)\nOct 4, 2024 • [Arvind Narayanan](https://substack.com/@arvindnarayanan) and [Sayash Kapoor](https://substack.com/@sayash)\n90\n13\n6\n![](../images/body-329c87a3b11e16b5633e.png)\n### September 2024\n[Can AI automate computational reproducibility?](https://www.normaltech.ai/p/can-ai-automate-computational-reproducibility)\n[A new benchmark to measure the impact of AI on improving science](https://www.normaltech.ai/p/can-ai-automate-computational-reproducibility)\nSep 18, 2024 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@arvindnarayanan)\n48\n2\n2\n![](../images/body-c79ff7a50bd6e4348dcf.png)\n[Start reading the AI Snake Oil book online](https://www.normaltech.ai/p/starting-reading-the-ai-snake-oil)\n[The book was published on September 24, 2024](https://www.normaltech.ai/p/starting-reading-the-ai-snake-oil)\nSep 10, 2024 • [Arvind Narayanan](https://substack.com/@arvindnarayanan) and [Sayash Kapoor](https://substack.com/@sayash)\n199\n8\n10\n![](../images/body-694a23a3a243cb15df90.png)\n### August 2024\n[AI companies are pivoting from creating gods to building products. Good.](https://www.normaltech.ai/p/ai-companies-are-pivoting-from-creating)\n[Turning models into products runs into five challenges](https://www.normaltech.ai/p/ai-companies-are-pivoting-from-creating)\nAug 19, 2024 • [Arvind Narayanan](https://substack.com/@arvindnarayanan) and [Sayash Kapoor](https://substack.com/@sayash)\n230\n24\n27\n![](../images/body-01dfb81732989705530e.png)\n© 2026 Sayash Kapoor and Arvind Narayanan · [Privacy](https://substack.com/privacy) ∙ [Terms](https://substack.com/tos) ∙ [Collection notice](https://substack.com/ccpa#personal-data-collected)\n[ Start your Substack](https://substack.com/signup?utm_source=substack&utm_medium=web&utm_content=footer)[Get the app](https://substack.com/app/app-store-redirect?utm_campaign=app-marketing&utm_content=web-footer-button)\n[Substack](https://substack.com) is the home for great culture\n",
  "metadata": {
    "title": "AI as Normal Technology | Sayash Kapoor | Substack",
    "description": "Analyzing AI as transformative but normal technology, not superintelligence. Click to read AI as Normal Technology, a Substack publication with tens of thousands of subscribers.",
    "keywords": null,
    "author": "Sayash Kapoor",
    "og:type": "article",
    "og:title": "AI as Normal Technology | Sayash Kapoor | Substack",
    "og:description": "Analyzing AI as transformative but normal technology, not superintelligence. Click to read AI as Normal Technology, a Substack publication with tens of thousands of subscribers.",
    "og:image": "https://substackcdn.com/image/fetch/$s_!0m1X!,f_auto,q_auto:best,fl_progressive:steep/https%3A%2F%2Faisnakeoil.substack.com%2Ftwitter%2Fsubscribe-card.jpg%3Fv%3D2066899008%26version%3D9",
    "og:url": "https://www.normaltech.ai/the-problem-with-llm-benchmarks",
    "twitter:title": "AI as Normal Technology | Sayash Kapoor | Substack",
    "twitter:description": "Analyzing AI as transformative but normal technology, not superintelligence. Click to read AI as Normal Technology, a Substack publication with tens of thousands of subscribers.",
    "twitter:image": "https://substackcdn.com/image/fetch/$s_!0m1X!,f_auto,q_auto:best,fl_progressive:steep/https%3A%2F%2Faisnakeoil.substack.com%2Ftwitter%2Fsubscribe-card.jpg%3Fv%3D2066899008%26version%3D9",
    "twitter:card": "summary_large_image"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 301,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!u4S7!,e_trim:10:white/e_trim:10:transparent/h_72,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F595a103f-0f5f-421a-a9ed-009cf269b7b4_3024x652.jpeg",
      "bytes": 42093,
      "reused": true,
      "local_path": "attachments/9bcaaeeed296/images/013.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!u4S7!,e_trim:10:white/e_trim:10:transparent/h_72,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F595a103f-0f5f-421a-a9ed-009cf269b7b4_3024x652.jpeg",
      "alt": "AI as Normal Technology",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!LgrS!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11d1c529-6f8d-4df3-acb6-f291eaa271f2_1024x559.jpeg",
      "bytes": 8997,
      "reused": true,
      "local_path": "attachments/9bcaaeeed296/images/014.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!LgrS!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11d1c529-6f8d-4df3-acb6-f291eaa271f2_1024x559.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!vQj7!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fedcb1dda-0882-4b56-9e2a-d6c3b131bd79_2562x1278.jpeg",
      "bytes": 8638,
      "reused": true,
      "local_path": "attachments/9bcaaeeed296/images/032.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!vQj7!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fedcb1dda-0882-4b56-9e2a-d6c3b131bd79_2562x1278.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!tFvu!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4a6ee35c-82c7-443c-b1c6-5f1f808f360d_3012x1682.jpeg",
      "bytes": 27506,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-20b1b779921e86eee151.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!tFvu!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4a6ee35c-82c7-443c-b1c6-5f1f808f360d_3012x1682.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!2o-q!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5a0117ca-8ce4-45e9-a956-62c82a45d44b_3064x1792.jpeg",
      "bytes": 13514,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-8c159ebce99973a987e4.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!2o-q!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5a0117ca-8ce4-45e9-a956-62c82a45d44b_3064x1792.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!Cea8!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F473d4546-aca2-4ff3-98e2-c7161e5e3536_3008x1734.png",
      "bytes": 25720,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/9bcaaeeed296/images/body-8d5f0c9ab934fe15ed65.png",
      "src": "https://substackcdn.com/image/fetch/$s_!Cea8!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F473d4546-aca2-4ff3-98e2-c7161e5e3536_3008x1734.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!wVRw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F01c35aa8-1081-40d7-9594-6db194483143_1792x1008.png",
      "bytes": 9260,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-8672dbce29f398d63f88.png",
      "src": "https://substackcdn.com/image/fetch/$s_!wVRw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F01c35aa8-1081-40d7-9594-6db194483143_1792x1008.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!eBfS!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F931ce348-4a6a-44e9-ad78-322dd8421bd7_1286x764.png",
      "bytes": 24462,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-4d4af90675819fbd8cfa.png",
      "src": "https://substackcdn.com/image/fetch/$s_!eBfS!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F931ce348-4a6a-44e9-ad78-322dd8421bd7_1286x764.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!OIxT!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F213a05d2-0443-42ca-85cf-0cf607ad626b_2048x1365.webp",
      "bytes": 10509,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-c55a02e97e21be5c3fee.webp",
      "src": "https://substackcdn.com/image/fetch/$s_!OIxT!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F213a05d2-0443-42ca-85cf-0cf607ad626b_2048x1365.webp",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!MKji!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F482f659f-2701-4952-8dd0-7b729be24e66_2048x1176.png",
      "bytes": 18722,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-d27afc93fa2cece2abf8.png",
      "src": "https://substackcdn.com/image/fetch/$s_!MKji!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F482f659f-2701-4952-8dd0-7b729be24e66_2048x1176.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!Vv_b!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F44ed558c-fa00-46fc-a80d-c8a70b338fc9_2992x1616.jpeg",
      "bytes": 10104,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-ab273e5917edef922b91.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!Vv_b!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F44ed558c-fa00-46fc-a80d-c8a70b338fc9_2992x1616.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!vhlu!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2e9d4b87-9068-4764-b50a-9c458b45252b_1486x960.jpeg",
      "bytes": 6237,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-f1c48dacaae2beeacce5.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!vhlu!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2e9d4b87-9068-4764-b50a-9c458b45252b_1486x960.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!GpsI!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2a2691bd-fda5-4a31-aa6c-ff0d59c79e4d_2428x1368.png",
      "bytes": 9199,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-9e068808a486ef1ed733.png",
      "src": "https://substackcdn.com/image/fetch/$s_!GpsI!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2a2691bd-fda5-4a31-aa6c-ff0d59c79e4d_2428x1368.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!ydUI!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c0502b3-b033-478e-b7f7-d224b2ff51e4_1920x1080.png",
      "bytes": 28116,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-af746494c7547caf396f.png",
      "src": "https://substackcdn.com/image/fetch/$s_!ydUI!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c0502b3-b033-478e-b7f7-d224b2ff51e4_1920x1080.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!WoEN!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd6ef1281-5124-4069-a716-955720587e38_2872x1606.png",
      "bytes": 6476,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-abf6fbace29ee2728fae.png",
      "src": "https://substackcdn.com/image/fetch/$s_!WoEN!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd6ef1281-5124-4069-a716-955720587e38_2872x1606.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!MQRX!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177c9c7a-89ac-40e2-9493-34667f829d92_4086x2296.jpeg",
      "bytes": 10506,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-7259dbb9e459f3e9d2e0.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!MQRX!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177c9c7a-89ac-40e2-9493-34667f829d92_4086x2296.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!LOkG!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49daa6d8-fe58-410c-b060-83648d15a722_640x480.png",
      "bytes": 11898,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-1a43f643b5b6cd283a1b.png",
      "src": "https://substackcdn.com/image/fetch/$s_!LOkG!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49daa6d8-fe58-410c-b060-83648d15a722_640x480.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!89bm!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac4b6c2a-d282-4124-a4fa-360ae04c9abe_2602x1550.jpeg",
      "bytes": 9909,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-11e01a825e7509751d75.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!89bm!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac4b6c2a-d282-4124-a4fa-360ae04c9abe_2602x1550.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!0fDW!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc348909d-140e-4e20-94d7-92a04e008bb2_1450x1270.png",
      "bytes": 10087,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-40aefaeaf590a97d144b.png",
      "src": "https://substackcdn.com/image/fetch/$s_!0fDW!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc348909d-140e-4e20-94d7-92a04e008bb2_1450x1270.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!ezjw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F29800e81-2362-430e-ada1-591a2d3a5228_910x1000.jpeg",
      "bytes": 16472,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-d8a9c7243857d00eeb5e.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!ezjw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F29800e81-2362-430e-ada1-591a2d3a5228_910x1000.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!eNIb!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F104a5670-c400-45bf-b02e-18d3bfe2e46d_2890x1648.jpeg",
      "bytes": 5233,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-f707e0bfb4942e74aaa2.jpeg",
      "src": "https://substackcdn.com/image/fetch/$s_!eNIb!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F104a5670-c400-45bf-b02e-18d3bfe2e46d_2890x1648.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!9h90!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F907f61bc-0697-41a3-a729-b719812a6d1a_2166x1402.png",
      "bytes": 6571,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-329c87a3b11e16b5633e.png",
      "src": "https://substackcdn.com/image/fetch/$s_!9h90!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F907f61bc-0697-41a3-a729-b719812a6d1a_2166x1402.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!9flC!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f816374-794e-4011-b986-6c5221d489de_2908x1682.png",
      "bytes": 10808,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-c79ff7a50bd6e4348dcf.png",
      "src": "https://substackcdn.com/image/fetch/$s_!9flC!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f816374-794e-4011-b986-6c5221d489de_2908x1682.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!-dim!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3b84a7d7-b736-45ad-a1c6-8162dac6d732_4480x2520.png",
      "bytes": 8102,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-694a23a3a243cb15df90.png",
      "src": "https://substackcdn.com/image/fetch/$s_!-dim!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3b84a7d7-b736-45ad-a1c6-8162dac6d732_4480x2520.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://substackcdn.com/image/fetch/$s_!Tqlc!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F29486da2-0bac-4f68-bafd-fc41d1356baa_2552x1429.png",
      "bytes": 11830,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/9bcaaeeed296/images/body-01dfb81732989705530e.png",
      "src": "https://substackcdn.com/image/fetch/$s_!Tqlc!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F29486da2-0bac-4f68-bafd-fc41d1356baa_2552x1429.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/notes"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/starting-reading-the-ai-snake-oil"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/ai-snake-oil-exercises-and-discussion"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/about"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/at-a-glance"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/a-big-tent-or-small-tent-ai-safety"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/p-doom"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/the-ai-as-normal-technology-view"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/ai-agents-cant-yet-do-open-ended"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/what-will-be-left-for-us-to-work"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/up-the-stack-how-ais-escape-from"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/why-ai-hasnt-replaced-software-engineers"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/did-googles-ai-agents-really-build"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/do-ai-risks-require-extraordinary"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/open-world-evaluations-for-measuring"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/new-paper-towards-a-science-of-ai"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/ai-wont-automatically-make-legal"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/fact-checking-moravecs-paradox"
    },
    {
      "type": "internal",
      "href": "https://www.normaltech.ai/p/a-guide-to-understanding-ai-as-normal"
    },
    {
      "type": "external",
      "href": "https://substack.com/@aisnakeoil"
    },
    {
      "type": "external",
      "href": "https://substack.com/@sayash"
    },
    {
      "type": "external",
      "href": "https://substack.com/@justincurl"
    },
    {
      "type": "external",
      "href": "https://substack.com/@arvindnarayanan"
    },
    {
      "type": "external",
      "href": "https://substack.com/privacy"
    },
    {
      "type": "external",
      "href": "https://substack.com/tos"
    },
    {
      "type": "external",
      "href": "https://substack.com/ccpa"
    },
    {
      "type": "external",
      "href": "https://substack.com/signup"
    },
    {
      "type": "external",
      "href": "https://substack.com/app/app-store-redirect"
    },
    {
      "type": "external",
      "href": "https://substack.com"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-20T17:33:18Z",
  "source_url": "https://www.aisnakeoil.com/p/the-problem-with-llm-benchmarks",
  "delivery_status": "unverified",
  "discovery_status": "http_404",
  "missing_reason": "http_404",
  "repair_error": "",
  "actual_video_files": [],
  "expected_saved_body_images": 25,
  "image_discovery": "saved_markdown",
  "source_body_markdown": "[![AI as Normal Technology](https://substackcdn.com/image/fetch/$s_!YzQY!,w_40,h_40,c_fill,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6780dda8-5879-4789-bf02-b3ea43a9f85e_1000x1000.png)](https://www.normaltech.ai/)\n# [![AI as Normal Technology](https://substackcdn.com/image/fetch/$s_!u4S7!,e_trim:10:white/e_trim:10:transparent/h_72,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F595a103f-0f5f-421a-a9ed-009cf269b7b4_3024x652.jpeg)](https://www.normaltech.ai/)\nSubscribeSign in\n[Home](https://www.normaltech.ai/)\n[Notes](https://www.normaltech.ai/notes)\n[Get the book](https://www.normaltech.ai/p/starting-reading-the-ai-snake-oil)\n[Book exercises](https://www.normaltech.ai/p/ai-snake-oil-exercises-and-discussion)\n[About us](https://www.normaltech.ai/about)\n[At a glance](https://www.normaltech.ai/p/at-a-glance)\n## Page not found\nLatestTopDiscussions\n[A big-tent or small-tent AI safety movement?](https://www.normaltech.ai/p/a-big-tent-or-small-tent-ai-safety)\n[The unstated disagreement that underpins safety debates](https://www.normaltech.ai/p/a-big-tent-or-small-tent-ai-safety)\n4 hrs ago • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n101\n5\n12\n![](https://substackcdn.com/image/fetch/$s_!LgrS!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11d1c529-6f8d-4df3-acb6-f291eaa271f2_1024x559.jpeg)\n### September 2026\n[AI existential risk probabilities are (still) too unreliable to inform policy](https://www.normaltech.ai/p/p-doom)\n[How speculation gets laundered through pseudo-quantification](https://www.normaltech.ai/p/p-doom)\nSep 28 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n152\n21\n25\n![](https://substackcdn.com/image/fetch/$s_!vQj7!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fedcb1dda-0882-4b56-9e2a-d6c3b131bd79_2562x1278.jpeg)\n[The AI-as-Normal-Technology view of loss-of-control incidents](https://www.normaltech.ai/p/the-ai-as-normal-technology-view)\n[A middle ground between the cybersecurity and AI safety communities](https://www.normaltech.ai/p/the-ai-as-normal-technology-view)\nSep 14 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n274\n54\n49\n![](https://substackcdn.com/image/fetch/$s_!tFvu!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4a6ee35c-82c7-443c-b1c6-5f1f808f360d_3012x1682.jpeg)\n### August 2026\n[AI agents can't yet do open-ended AI research](https://www.normaltech.ai/p/ai-agents-cant-yet-do-open-ended)\n[Early evidence from two case studies](https://www.normaltech.ai/p/ai-agents-cant-yet-do-open-ended)\nAug 5 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n194\n27\n25\n![](https://substackcdn.com/image/fetch/$s_!2o-q!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5a0117ca-8ce4-45e9-a956-62c82a45d44b_3064x1792.jpeg)\n### July 2026\n[What will be left for us to work on?](https://www.normaltech.ai/p/what-will-be-left-for-us-to-work)\n[My keynote at ICML 2026](https://www.normaltech.ai/p/what-will-be-left-for-us-to-work)\nJul 13 • [Arvind Narayanan](https://substack.com/@aisnakeoil)\n349\n30\n61\n![](https://substackcdn.com/image/fetch/$s_!Cea8!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F473d4546-aca2-4ff3-98e2-c7161e5e3536_3008x1734.png)\n[Up the Stack: How AI’s Escape From the Commodity Trap Risks Enterprise Lock-in](https://www.normaltech.ai/p/up-the-stack-how-ais-escape-from)\n[Critics and boosters are both looking in the wrong place](https://www.normaltech.ai/p/up-the-stack-how-ais-escape-from)\nJul 9 • [Arvind Narayanan](https://substack.com/@aisnakeoil)\n173\n24\n31\n![](https://substackcdn.com/image/fetch/$s_!wVRw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F01c35aa8-1081-40d7-9594-6db194483143_1792x1008.png)\n### June 2026\n[Why AI hasn’t replaced software engineers, and won’t](https://www.normaltech.ai/p/why-ai-hasnt-replaced-software-engineers)\n[Coding agents as normal technology](https://www.normaltech.ai/p/why-ai-hasnt-replaced-software-engineers)\nJun 11 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n600\n89\n110\n![](https://substackcdn.com/image/fetch/$s_!eBfS!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F931ce348-4a6a-44e9-ad78-322dd8421bd7_1286x764.png)\n### May 2026\n[Did Google’s AI agents really build an operating system for $916?](https://www.normaltech.ai/p/did-googles-ai-agents-really-build)\n[The importance of independent evaluation](https://www.normaltech.ai/p/did-googles-ai-agents-really-build)\nMay 22 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n144\n25\n11\n![](https://substackcdn.com/image/fetch/$s_!OIxT!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F213a05d2-0443-42ca-85cf-0cf607ad626b_2048x1365.webp)\n[Do AI Risks Require Extraordinary Government Intervention?](https://www.normaltech.ai/p/do-ai-risks-require-extraordinary)\n[Let’s not skip the hard work of AI governance](https://www.normaltech.ai/p/do-ai-risks-require-extraordinary)\nMay 21 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n85\n9\n19\n![](https://substackcdn.com/image/fetch/$s_!MKji!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F482f659f-2701-4952-8dd0-7b729be24e66_2048x1176.png)\n### April 2026\n[Open-world evaluations for measuring frontier AI capabilities](https://www.normaltech.ai/p/open-world-evaluations-for-measuring)\n[Introducing CRUX, a new project for evaluating AI on long, messy tasks](https://www.normaltech.ai/p/open-world-evaluations-for-measuring)\nApr 16 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n99\n7\n14\n![](https://substackcdn.com/image/fetch/$s_!Vv_b!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F44ed558c-fa00-46fc-a80d-c8a70b338fc9_2992x1616.jpeg)\n### February 2026\n[New Paper: Towards a science of AI agent reliability](https://www.normaltech.ai/p/new-paper-towards-a-science-of-ai)\n[Quantifying the capability-reliability gap](https://www.normaltech.ai/p/new-paper-towards-a-science-of-ai)\nFeb 24 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n328\n36\n71\n![](https://substackcdn.com/image/fetch/$s_!vhlu!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2e9d4b87-9068-4764-b50a-9c458b45252b_1486x960.jpeg)\n[AI Won’t Automatically Make Legal Services Cheaper](https://www.normaltech.ai/p/ai-wont-automatically-make-legal)\n[Applying the AI as Normal Technology framework to legal services](https://www.normaltech.ai/p/ai-wont-automatically-make-legal)\nFeb 12 • [Justin Curl](https://substack.com/@justincurl), [Sayash Kapoor](https://substack.com/@sayash), and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n86\n13\n15\n![](https://substackcdn.com/image/fetch/$s_!GpsI!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2a2691bd-fda5-4a31-aa6c-ff0d59c79e4d_2428x1368.png)\n### January 2026\n[Fact checking Moravec's paradox](https://www.normaltech.ai/p/fact-checking-moravecs-paradox)\n[The famous aphorism is neither true nor useful](https://www.normaltech.ai/p/fact-checking-moravecs-paradox)\nJan 29 • [Arvind Narayanan](https://substack.com/@aisnakeoil)\n174\n10\n25\n![](https://substackcdn.com/image/fetch/$s_!ydUI!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c0502b3-b033-478e-b7f7-d224b2ff51e4_1920x1080.png)\n### September 2025\n[A guide to understanding AI as normal technology](https://www.normaltech.ai/p/a-guide-to-understanding-ai-as-normal)\n[And a big change for this newsletter](https://www.normaltech.ai/p/a-guide-to-understanding-ai-as-normal)\nSep 9, 2025 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n234\n29\n42\n![](https://substackcdn.com/image/fetch/$s_!WoEN!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd6ef1281-5124-4069-a716-955720587e38_2872x1606.png)\n### July 2025\n[Could AI slow science?](https://www.normaltech.ai/p/could-ai-slow-science)\n[Confronting the production-progress paradox](https://www.normaltech.ai/p/could-ai-slow-science)\nJul 16, 2025 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n325\n37\n70\n![](https://substackcdn.com/image/fetch/$s_!MQRX!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177c9c7a-89ac-40e2-9493-34667f829d92_4086x2296.jpeg)\n### May 2025\n[AGI is not a milestone](https://www.normaltech.ai/p/agi-is-not-a-milestone)\n[There is no capability threshold that will lead to sudden impacts](https://www.normaltech.ai/p/agi-is-not-a-milestone)\nMay 1, 2025 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n199\n37\n40\n![](https://substackcdn.com/image/fetch/$s_!LOkG!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F49daa6d8-fe58-410c-b060-83648d15a722_640x480.png)\n### April 2025\n[AI as Normal Technology](https://www.normaltech.ai/p/ai-as-normal-technology)\n[A new paper that we will expand into our next book](https://www.normaltech.ai/p/ai-as-normal-technology)\nApr 15, 2025 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n636\n40\n132\n![](https://substackcdn.com/image/fetch/$s_!89bm!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fac4b6c2a-d282-4124-a4fa-360ae04c9abe_2602x1550.jpeg)\n### December 2024\n[Is AI progress slowing down?](https://www.normaltech.ai/p/is-ai-progress-slowing-down)\n[Making sense of recent technology trends and claims](https://www.normaltech.ai/p/is-ai-progress-slowing-down)\nDec 18, 2024 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n171\n15\n29\n![](https://substackcdn.com/image/fetch/$s_!0fDW!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc348909d-140e-4e20-94d7-92a04e008bb2_1450x1270.png)\n[We Looked at 78 Election Deepfakes. Political Misinformation is not an AI Problem.](https://www.normaltech.ai/p/we-looked-at-78-election-deepfakes)\n[Technology Isn’t the Problem—or the Solution.](https://www.normaltech.ai/p/we-looked-at-78-election-deepfakes)\nDec 13, 2024 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@aisnakeoil)\n119\n6\n23\n![](https://substackcdn.com/image/fetch/$s_!ezjw!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F29800e81-2362-430e-ada1-591a2d3a5228_910x1000.jpeg)\n### November 2024\n[Does the UK’s liver transplant matching algorithm systematically exclude younger patients?](https://www.normaltech.ai/p/does-the-uks-liver-transplant-matching)\n[Seemingly minor technical decisions can have life-or-death effects](https://www.normaltech.ai/p/does-the-uks-liver-transplant-matching)\nNov 11, 2024 • [Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)\n103\n11\n7\n![](https://substackcdn.com/image/fetch/$s_!eNIb!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F104a5670-c400-45bf-b02e-18d3bfe2e46d_2890x1648.jpeg)\n### October 2024\n[FAQ about the book and our writing process](https://www.normaltech.ai/p/faq-about-the-book-and-our-writing)\n[What's in the book and how we wrote it](https://www.normaltech.ai/p/faq-about-the-book-and-our-writing)\nOct 4, 2024 • [Arvind Narayanan](https://substack.com/@arvindnarayanan) and [Sayash Kapoor](https://substack.com/@sayash)\n90\n13\n6\n![](https://substackcdn.com/image/fetch/$s_!9h90!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F907f61bc-0697-41a3-a729-b719812a6d1a_2166x1402.png)\n### September 2024\n[Can AI automate computational reproducibility?](https://www.normaltech.ai/p/can-ai-automate-computational-reproducibility)\n[A new benchmark to measure the impact of AI on improving science](https://www.normaltech.ai/p/can-ai-automate-computational-reproducibility)\nSep 18, 2024 • [Sayash Kapoor](https://substack.com/@sayash) and [Arvind Narayanan](https://substack.com/@arvindnarayanan)\n48\n2\n2\n![](https://substackcdn.com/image/fetch/$s_!9flC!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f816374-794e-4011-b986-6c5221d489de_2908x1682.png)\n[Start reading the AI Snake Oil book online](https://www.normaltech.ai/p/starting-reading-the-ai-snake-oil)\n[The book was published on September 24, 2024](https://www.normaltech.ai/p/starting-reading-the-ai-snake-oil)\nSep 10, 2024 • [Arvind Narayanan](https://substack.com/@arvindnarayanan) and [Sayash Kapoor](https://substack.com/@sayash)\n199\n8\n10\n![](https://substackcdn.com/image/fetch/$s_!-dim!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3b84a7d7-b736-45ad-a1c6-8162dac6d732_4480x2520.png)\n### August 2024\n[AI companies are pivoting from creating gods to building products. Good.](https://www.normaltech.ai/p/ai-companies-are-pivoting-from-creating)\n[Turning models into products runs into five challenges](https://www.normaltech.ai/p/ai-companies-are-pivoting-from-creating)\nAug 19, 2024 • [Arvind Narayanan](https://substack.com/@arvindnarayanan) and [Sayash Kapoor](https://substack.com/@sayash)\n230\n24\n27\n![](https://substackcdn.com/image/fetch/$s_!Tqlc!,w_320,h_213,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F29486da2-0bac-4f68-bafd-fc41d1356baa_2552x1429.png)\n© 2026 Sayash Kapoor and Arvind Narayanan · [Privacy](https://substack.com/privacy) ∙ [Terms](https://substack.com/tos) ∙ [Collection notice](https://substack.com/ccpa#personal-data-collected)\n[ Start your Substack](https://substack.com/signup?utm_source=substack&utm_medium=web&utm_content=footer)[Get the app](https://substack.com/app/app-store-redirect?utm_campaign=app-marketing&utm_content=web-footer-button)\n[Substack](https://substack.com) is the home for great culture\n",
  "body_base_path": "attachments/9bcaaeeed296/pages/",
  "body_markdown_path": "attachments/9bcaaeeed296/pages/archived-body.md",
  "local_html": "attachments/9bcaaeeed296/pages/archived-body.html",
  "body_source": "prior_capture"
}