{
  "source_file": "0930",
  "source_platform": "个人博客",
  "author": "苏剑林",
  "title_listed": "将Softmax Attention线性化为Gated DeltaNet",
  "keywords": "线性注意力,Softmax,DeltaNet",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://spaces.ac.cn/archives/11843",
  "publish_date": "2026-07",
  "corpus_stage": "预训练/SFT",
  "notes": "线性注意力推导",
  "id": "6b486c08673c",
  "collected_at": "2026-10-02T15:15:01",
  "title": "页面没找到 - 科学空间|Scientific Spaces",
  "body_markdown": "Processing math: 0%\n![MobileSideBar](../images/body-fcd8b7d1212b1f5eab2b.png)\n## SEARCH\n## MENU\n  * [打赏](https://spaces.ac.cn/reward.html)\n  * [公式](https://spaces.ac.cn/latex.html)\n  * [天象](https://spaces.ac.cn/ac.html)\n  * [链接](https://spaces.ac.cn/links.html)\n  * [时光](https://spaces.ac.cn/me.html)\n  * [博览](https://spaces.ac.cn/science.html)\n  * [归档](https://spaces.ac.cn/content.html)\n\n\n## CATEGORIES\n  * [千奇百怪](https://spaces.ac.cn/category/Everything)\n  * [天文探索](https://spaces.ac.cn/category/Astronomy)\n  * [数学研究](https://spaces.ac.cn/category/Mathematics)\n  * [物理化学](https://spaces.ac.cn/category/Phy-chem)\n  * [信息时代](https://spaces.ac.cn/category/Big-Data)\n  * [生物自然](https://spaces.ac.cn/category/Biology)\n  * [图片摄影](https://spaces.ac.cn/category/Photograph)\n  * [问题百科](https://spaces.ac.cn/category/Questions)\n  * [生活/情感](https://spaces.ac.cn/category/Life-Feeling)\n  * [资源共享](https://spaces.ac.cn/category/Resources)\n\n\n## NEWPOSTS\n  * [慢即是快：1. Shampoo的尽...](https://spaces.ac.cn/archives/11917)\n  * [排序不等式及其推广](https://spaces.ac.cn/archives/11910)\n  * [让炼丹更科学一些（十）：单调性假设...](https://spaces.ac.cn/archives/11885)\n  * [让炼丹更科学一些（九）：经典自适应...](https://spaces.ac.cn/archives/11882)\n  * [让炼丹更科学一些（八）：多阶段训练...](https://spaces.ac.cn/archives/11879)\n  * [动量的新理解：逼近特征层面的梯度下降](https://spaces.ac.cn/archives/11875)\n  * [流形上的最速下降：7. Stief...](https://spaces.ac.cn/archives/11864)\n  * [除了交叉熵，LM Loss还有什么选择？](https://spaces.ac.cn/archives/11854)\n  * [简单谈谈K3的MoE和Attention](https://spaces.ac.cn/archives/11848)\n  * [解构Scaling Law：优化、...](https://spaces.ac.cn/archives/11833)\n\n\n## COMMENTS\n  * [yicli: 苏老师好，我有两个问题想再请教一下： 1. Full Attn...](https://spaces.ac.cn/archives/11664/comment-page-3#comment-29805)\n  * [Lindsay: 这是不是也是为什么diffusion text model质量...](https://spaces.ac.cn/archives/11854/comment-page-1#comment-29804)\n  * [Honghao: 一般结论那节中“$q_t=E[L(\\theta_t)] - L...](https://spaces.ac.cn/archives/11494/comment-page-1#comment-29803)\n  * [田奕哲: 似乎engram与其思路有点相似 都是用hash直接通过tok...](https://spaces.ac.cn/archives/11750/comment-page-1#comment-29802)\n  * [Kacent: 苏神好,我想请教一下. 如果b只被LossFree更新, ...](https://spaces.ac.cn/archives/10757/comment-page-3#comment-29801)\n  * [Dr.Ink: 受教、感谢苏神～ 最近做实验建立更深的理解、也在尝试一些改进，...](https://spaces.ac.cn/archives/11619/comment-page-1#comment-29799)\n  * [Honghao: 感谢苏神回复！](https://spaces.ac.cn/archives/11875/comment-page-1#comment-29798)\n  * [苏剑林: 我们对ds方案和qb做了充分对比，结论是qb更均衡，并且效果（...](https://spaces.ac.cn/archives/11619/comment-page-1#comment-29797)\n  * [苏剑林: 不是愿不愿意更新的问题，主要是暂时对这个方向没有什么新理解](https://spaces.ac.cn/archives/10352/comment-page-2#comment-29796)\n  * [苏剑林: 确实，更正了，感谢反馈](https://spaces.ac.cn/archives/9431/comment-page-2#comment-29795)\n\n\n## USERLOGIN\n  * [登录](https://spaces.ac.cn/admin/login.php)\n\n\n# [科学空间|Scientific Spaces](https://spaces.ac.cn)\n  * [登录](https://spaces.ac.cn/admin/login.php)\n  * [打赏](https://spaces.ac.cn/reward.html)\n  * [公式](https://spaces.ac.cn/latex.html)\n  * [天象](https://spaces.ac.cn/ac.html)\n  * [链接](https://spaces.ac.cn/links.html)\n  * [时光](https://spaces.ac.cn/me.html)\n  * [博览](https://spaces.ac.cn/science.html)\n  * [归档](https://spaces.ac.cn/content.html)\n\n\n渴望成为一个小飞侠\n  * [![](../images/body-043e6308dc873ff9fa67.png)  \n欢迎订阅](https://spaces.ac.cn/feed)\n  * [![](../images/body-942ff5c7b650f4d3d31f.png)  \n个性邮箱](https://spaces.ac.cn/archives/119)\n  * [![](../images/body-53b9d00850633d612de3.png)  \n天象信息](https://spaces.ac.cn/ac.html)\n  * [![](../images/body-896b482bdde5ec1df88a.png)  \n观测ISS](https://spaces.ac.cn/archives/41)\n  * [![](../images/body-3f7f2f2318249e4eac9a.png)  \nLaTeX](https://spaces.ac.cn/latex.html)\n  * [![](../images/body-cbbace78f2cd4551d156.png)  \n关于博主](https://spaces.ac.cn/me.html)\n\n\n欢迎访问“科学空间”，这里将与您共同探讨自然科学，回味人生百态；也期待大家的分享～\n  * [**千奇百怪** Everything](https://spaces.ac.cn/category/Everything)\n  * [**天文探索** Astronomy](https://spaces.ac.cn/category/Astronomy)\n  * [**数学研究** Mathematics](https://spaces.ac.cn/category/Mathematics)\n  * [**物理化学** Phy-chem](https://spaces.ac.cn/category/Phy-chem)\n  * [**信息时代** Big-Data](https://spaces.ac.cn/category/Big-Data)\n  * [**生物自然** Biology](https://spaces.ac.cn/category/Biology)\n  * [**图片摄影** Photograph](https://spaces.ac.cn/category/Photograph)\n  * [**问题百科** Questions](https://spaces.ac.cn/category/Questions)\n  * [**生活/情感** Life-Feeling](https://spaces.ac.cn/category/Life-Feeling)\n  * [**资源共享** Resources](https://spaces.ac.cn/category/Resources)\n\n\n  * [**千奇百怪**](https://spaces.ac.cn/category/Everything)\n  * [**天文探索**](https://spaces.ac.cn/category/Astronomy)\n  * [**数学研究**](https://spaces.ac.cn/category/Mathematics)\n  * [**物理化学**](https://spaces.ac.cn/category/Phy-chem)\n  * [**信息时代**](https://spaces.ac.cn/category/Big-Data)\n  * [**生物自然**](https://spaces.ac.cn/category/Biology)\n  * [**图片摄影**](https://spaces.ac.cn/category/Photograph)\n  * [**问题百科**](https://spaces.ac.cn/category/Questions)\n  * [**生活/情感**](https://spaces.ac.cn/category/Life-Feeling)\n  * [**资源共享**](https://spaces.ac.cn/category/Resources)\n\n\n  \n\n## 找不到相关内容。\n  \n\n### 你也许对下面的内容感兴趣\n  * [从费马大定理谈起（十）：x^3+y^3=z^3+w^3](https://spaces.ac.cn/archives/2972)\n  * [Transformer升级之路：5、作为无限维的线性Attention](https://spaces.ac.cn/archives/8601)\n  * [预训练一下，Transformer的长序列成绩还能涨不少！](https://spaces.ac.cn/archives/9787)\n  * [科学空间：2010年5月重要天象](https://spaces.ac.cn/archives/627)\n  * [[2011]一睹“食”的风采](https://spaces.ac.cn/archives/1165)\n  * [对比学习可以使用梯度累积吗？](https://spaces.ac.cn/archives/8471)\n  * [基于CNN的阅读理解式问答模型：DGCNN](https://spaces.ac.cn/archives/5409)\n  * [【NASA每日一图】甘姆星云(Gum Nebula)](https://spaces.ac.cn/archives/92)\n  * [【NASA每日一图】牧羊卫星Prometheus](https://spaces.ac.cn/archives/391)\n  * [日食当天的天气出来了](https://spaces.ac.cn/archives/17)\n\n\n### 关于站长\n![科学空间logo](../images/body-64d59c911904c60f6f14.png)\n**苏剑林|BoJone** ，科学空间博主，【数学、天文、理论物理、写作、阅读、计算机、中国象棋、厨房】爱好者（但不专业）......目前33岁，还在单调递增。希望能一直在此分享科学之美～\n**你也许会关心：**\n* [科学空间|Scientific Spaces 介绍](https://spaces.ac.cn/me.html)\n* 科学空间QQ交流群：67729435\n* 科学空间微信交流群：[spaces_ac_cn](https://spaces.ac.cn/archives/4096/)\n* 常见问题集：[《科学空间FAQ》](https://spaces.ac.cn/archives/6508/)\n### 智能搜索\n支持整句搜索！网站自动使用[结巴分词](https://github.com/fxsjy/jieba)进行分词，并结合ngrams排序算法给出合理的搜索结果。\n### 热门标签\n[attention](https://spaces.ac.cn/tag/attention/) [生成模型](https://spaces.ac.cn/tag/%E7%94%9F%E6%88%90%E6%A8%A1%E5%9E%8B/) [优化](https://spaces.ac.cn/tag/%E4%BC%98%E5%8C%96/) [模型](https://spaces.ac.cn/tag/%E6%A8%A1%E5%9E%8B/) [语言模型](https://spaces.ac.cn/tag/%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B/) [梯度](https://spaces.ac.cn/tag/%E6%A2%AF%E5%BA%A6/) [优化器](https://spaces.ac.cn/tag/%E4%BC%98%E5%8C%96%E5%99%A8/) [矩阵](https://spaces.ac.cn/tag/%E7%9F%A9%E9%98%B5/) [概率](https://spaces.ac.cn/tag/%E6%A6%82%E7%8E%87/) [网站](https://spaces.ac.cn/tag/%E7%BD%91%E7%AB%99/) [转载](https://spaces.ac.cn/tag/%E8%BD%AC%E8%BD%BD/) [微分方程](https://spaces.ac.cn/tag/%E5%BE%AE%E5%88%86%E6%96%B9%E7%A8%8B/) [分析](https://spaces.ac.cn/tag/%E5%88%86%E6%9E%90/) [天象](https://spaces.ac.cn/tag/%E5%A4%A9%E8%B1%A1/) [深度学习](https://spaces.ac.cn/tag/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0/) [积分](https://spaces.ac.cn/tag/%E7%A7%AF%E5%88%86/) [python](https://spaces.ac.cn/tag/python/) [几何](https://spaces.ac.cn/tag/%E5%87%A0%E4%BD%95/) [扩散](https://spaces.ac.cn/tag/%E6%89%A9%E6%95%A3/) [力学](https://spaces.ac.cn/tag/%E5%8A%9B%E5%AD%A6/) [无监督](https://spaces.ac.cn/tag/%E6%97%A0%E7%9B%91%E7%9D%A3/) [损失函数](https://spaces.ac.cn/tag/%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0/) [节日](https://spaces.ac.cn/tag/%E8%8A%82%E6%97%A5/) [生活](https://spaces.ac.cn/tag/%E7%94%9F%E6%B4%BB/) [不等式](https://spaces.ac.cn/tag/%E4%B8%8D%E7%AD%89%E5%BC%8F/) \n\n### 随机文章\n  * [除夕试拍星空星轨～](https://spaces.ac.cn/archives/4222)\n  * [输入梯度惩罚与参数梯度惩罚的一个不等式](https://spaces.ac.cn/archives/8796)\n  * [修改Transformer结构，设计一个更快更好的MLM模型](https://spaces.ac.cn/archives/7661)\n  * [“Cool Papers + 站内搜索”的一些新尝试](https://spaces.ac.cn/archives/10311)\n  * [今天你食了吗？（广东云浮观测日偏食之旅）](https://spaces.ac.cn/archives/30)\n  * [变分自编码器（七）：球面上的VAE（vMF-VAE）](https://spaces.ac.cn/archives/8404)\n  * [能量视角下的GAN模型（一）：GAN＝“挖坑”＋“跳坑”](https://spaces.ac.cn/archives/6316)\n  * [BLOG评论故障修复，部分数据丢失](https://spaces.ac.cn/archives/31)\n  * [有限素域上的乘法群是循环群](https://spaces.ac.cn/archives/3200)\n  * [计算夏至的精确时刻2——提高精确度](https://spaces.ac.cn/archives/906)\n\n\n### 最近评论\n  * [yicli](https://spaces.ac.cn/archives/11664/comment-page-3#comment-29805): 苏老师好，我有两个问题想再请教一下： 1. Full AttnRes 被放弃的主要原因是训练时...\n  * [Lindsay](https://spaces.ac.cn/archives/11854/comment-page-1#comment-29804): 这是不是也是为什么diffusion text model质量不好的原因？因为本质上break...\n  * [Honghao](https://spaces.ac.cn/archives/11494/comment-page-1#comment-29803): 一般结论那节中“q_t=E[L(\\theta_t)] - L(\\theta^*)”应该是“...\n  * [田奕哲](https://spaces.ac.cn/archives/11750/comment-page-1#comment-29802): 似乎engram与其思路有点相似 都是用hash直接通过tokenid来查询 把学习转成查表\n  * [Kacent](https://spaces.ac.cn/archives/10757/comment-page-3#comment-29801): 苏神好,我想请教一下. 如果b只被LossFree更新, 而不被LM head更新, 那么...\n  * [Dr.Ink](https://spaces.ac.cn/archives/11619/comment-page-1#comment-29799): 受教、感谢苏神～ 最近做实验建立更深的理解、也在尝试一些改进，后续再向你请教\n  * [Honghao](https://spaces.ac.cn/archives/11875/comment-page-1#comment-29798): 感谢苏神回复！\n  * [苏剑林](https://spaces.ac.cn/archives/11619/comment-page-1#comment-29797): 我们对ds方案和qb做了充分对比，结论是qb更均衡，并且效果（loss和下游bmk）基本无差异...\n  * [苏剑林](https://spaces.ac.cn/archives/10352/comment-page-2#comment-29796): 不是愿不愿意更新的问题，主要是暂时对这个方向没有什么新理解\n  * [苏剑林](https://spaces.ac.cn/archives/9431/comment-page-2#comment-29795): 确实，更正了，感谢反馈\n\n\n### 友情链接\n  * [Cool Papers](https://papers.cool)\n  * [数学研发](https://bbs.emath.ac.cn)\n  * [Seatop](http://www.seatop.com.cn/)\n  * [Xiaoxia](https://xiaoxia.org/)\n  * [积分表-网络版](https://kexue.fm/sci/integral/index.html)\n  * [丝路博傲](http://blog.dvxj.com/)\n  * [数学之家](http://www.2math.cn/)\n  * [有趣天文奇观](http://interesting-sky.china-vo.org/)\n  * [TwistedW](http://www.twistedwg.com/)\n  * [godweiyang](https://godweiyang.com/)\n  * [AI柠檬](https://blog.ailemon.net/)\n  * [王登科-DK博客](https://greatdk.com)\n  * [ESON](https://blog.eson.org/)\n  * [枫之羽](https://fzhiy.net/)\n  * [coding-zuo](https://coding-zuo.github.io/)\n  * [博科园](https://www.bokeyuan.net/)\n  * [孔皮皮的博客](https://www.kppkkp.top/)\n  * [运鹏的博客](https://yunpengtai.top/)\n  * [jiming.site](https://jiming.site/)\n  * [OmegaXYZ](https://www.omegaxyz.com/)\n  * [EAI猩球](https://www.robotech.ink/)\n  * [文举的博客](https://liwenju0.com/)\n  * [汇尘轩](https://kirigaya.cn/)\n  * [申请链接](https://kexue.fm/links.html)\n\n\n[![署名-非商业用途-保持一致](../images/body-b2904ced8e4508d61bed.gif)](http://creativecommons.org/licenses/by-nc-nd/2.5/cn/) 本站采用创作共用版权协议，要求署名、非商业用途和保持一致。转载本站内容必须也遵循“[署名-非商业用途-保持一致](http://creativecommons.org/licenses/by-nc-nd/2.5/cn/)”的创作共用协议。   \n© 2009-2026 Scientific Spaces. All rights reserved. Theme by [laogui](http://www.laogui.com). Powered by [Typecho](http://typecho.org). 备案号: [粤ICP备09093259号-1/2](https://beian.miit.gov.cn/ \"粤ICP备09093259号\")。 \n",
  "metadata": {
    "title": "页面没找到 - 科学空间|Scientific Spaces",
    "description": "渴望成为一个小飞侠",
    "keywords": "科学空间,科学FM,数学,科普,机器学习,深度学习,模型,优化,自然语言处理,NLP,天文,量子,费曼,路径积分,对称,",
    "author": null
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 404,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "ok",
      "url": "https://spaces.ac.cn/usr/themes/geekg/images/slide-button.png",
      "bytes": 4328,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/6b486c08673c/images/body-fcd8b7d1212b1f5eab2b.png",
      "src": "https://spaces.ac.cn/usr/themes/geekg/images/slide-button.png",
      "alt": "MobileSideBar",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://spaces.ac.cn/usr/themes/geekg/images/rss.png",
      "bytes": 19283,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/6b486c08673c/images/body-043e6308dc873ff9fa67.png",
      "src": "https://spaces.ac.cn/usr/themes/geekg/images/rss.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://spaces.ac.cn/usr/themes/geekg/images/mail.png",
      "bytes": 16811,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/6b486c08673c/images/body-942ff5c7b650f4d3d31f.png",
      "src": "https://spaces.ac.cn/usr/themes/geekg/images/mail.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://spaces.ac.cn/usr/themes/geekg/images/Saturn.png",
      "bytes": 2524,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/6b486c08673c/images/body-53b9d00850633d612de3.png",
      "src": "https://spaces.ac.cn/usr/themes/geekg/images/Saturn.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://spaces.ac.cn/usr/themes/geekg/images/iss.png",
      "bytes": 3301,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/6b486c08673c/images/body-896b482bdde5ec1df88a.png",
      "src": "https://spaces.ac.cn/usr/themes/geekg/images/iss.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://spaces.ac.cn/usr/themes/geekg/images/pi.png",
      "bytes": 939,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/6b486c08673c/images/body-3f7f2f2318249e4eac9a.png",
      "src": "https://spaces.ac.cn/usr/themes/geekg/images/pi.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://spaces.ac.cn/usr/themes/geekg/images/mlogo.png",
      "bytes": 18685,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/6b486c08673c/images/body-cbbace78f2cd4551d156.png",
      "src": "https://spaces.ac.cn/usr/themes/geekg/images/mlogo.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://spaces.ac.cn/usr/themes/geekg/images/avatar.png",
      "bytes": 56643,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/6b486c08673c/images/body-64d59c911904c60f6f14.png",
      "src": "https://spaces.ac.cn/usr/themes/geekg/images/avatar.png",
      "alt": "科学空间logo",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://kexue.fm/usr/themes/geekg/images/cc.gif",
      "bytes": 2031,
      "reused": false,
      "content_type": "image/gif",
      "local_path": "attachments/6b486c08673c/images/body-b2904ced8e4508d61bed.gif",
      "src": "https://kexue.fm/usr/themes/geekg/images/cc.gif",
      "alt": "署名-非商业用途-保持一致",
      "downloaded": true,
      "discovery": "saved_markdown"
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/reward.html"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/latex.html"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/ac.html"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/links.html"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/me.html"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/science.html"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/content.html"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/category/Everything"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/category/Astronomy"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/category/Mathematics"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/category/Phy-chem"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/category/Big-Data"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/category/Biology"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/category/Photograph"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/category/Questions"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/category/Life-Feeling"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/category/Resources"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/archives/11917"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/archives/11910"
    },
    {
      "type": "internal",
      "href": "https://spaces.ac.cn/archives/11885"
    },
    {
      "type": "external",
      "href": "https://github.com/fxsjy/jieba"
    },
    {
      "type": "external",
      "href": "https://papers.cool"
    },
    {
      "type": "external",
      "href": "https://bbs.emath.ac.cn"
    },
    {
      "type": "external",
      "href": "http://www.seatop.com.cn/"
    },
    {
      "type": "external",
      "href": "https://xiaoxia.org/"
    },
    {
      "type": "external",
      "href": "https://kexue.fm/sci/integral/index.html"
    },
    {
      "type": "external",
      "href": "http://blog.dvxj.com/"
    },
    {
      "type": "external",
      "href": "http://www.2math.cn/"
    },
    {
      "type": "external",
      "href": "http://interesting-sky.china-vo.org/"
    },
    {
      "type": "external",
      "href": "http://www.twistedwg.com/"
    },
    {
      "type": "external",
      "href": "https://godweiyang.com/"
    },
    {
      "type": "external",
      "href": "https://blog.ailemon.net/"
    },
    {
      "type": "external",
      "href": "https://greatdk.com"
    },
    {
      "type": "external",
      "href": "https://blog.eson.org/"
    },
    {
      "type": "external",
      "href": "https://fzhiy.net/"
    },
    {
      "type": "external",
      "href": "https://coding-zuo.github.io/"
    },
    {
      "type": "external",
      "href": "https://www.bokeyuan.net/"
    },
    {
      "type": "external",
      "href": "https://www.kppkkp.top/"
    },
    {
      "type": "external",
      "href": "https://yunpengtai.top/"
    },
    {
      "type": "external",
      "href": "https://jiming.site/"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-20T17:24:01Z",
  "source_url": "https://spaces.ac.cn/archives/11843",
  "delivery_status": "unverified",
  "discovery_status": "http_403",
  "missing_reason": "http_403",
  "repair_error": "",
  "actual_video_files": [],
  "expected_saved_body_images": 9,
  "image_discovery": "saved_markdown",
  "source_body_markdown": "Processing math: 0%\n![MobileSideBar](https://spaces.ac.cn/usr/themes/geekg/images/slide-button.png)\n## SEARCH\n## MENU\n  * [打赏](https://spaces.ac.cn/reward.html)\n  * [公式](https://spaces.ac.cn/latex.html)\n  * [天象](https://spaces.ac.cn/ac.html)\n  * [链接](https://spaces.ac.cn/links.html)\n  * [时光](https://spaces.ac.cn/me.html)\n  * [博览](https://spaces.ac.cn/science.html)\n  * [归档](https://spaces.ac.cn/content.html)\n\n\n## CATEGORIES\n  * [千奇百怪](https://spaces.ac.cn/category/Everything)\n  * [天文探索](https://spaces.ac.cn/category/Astronomy)\n  * [数学研究](https://spaces.ac.cn/category/Mathematics)\n  * [物理化学](https://spaces.ac.cn/category/Phy-chem)\n  * [信息时代](https://spaces.ac.cn/category/Big-Data)\n  * [生物自然](https://spaces.ac.cn/category/Biology)\n  * [图片摄影](https://spaces.ac.cn/category/Photograph)\n  * [问题百科](https://spaces.ac.cn/category/Questions)\n  * [生活/情感](https://spaces.ac.cn/category/Life-Feeling)\n  * [资源共享](https://spaces.ac.cn/category/Resources)\n\n\n## NEWPOSTS\n  * [慢即是快：1. Shampoo的尽...](https://spaces.ac.cn/archives/11917)\n  * [排序不等式及其推广](https://spaces.ac.cn/archives/11910)\n  * [让炼丹更科学一些（十）：单调性假设...](https://spaces.ac.cn/archives/11885)\n  * [让炼丹更科学一些（九）：经典自适应...](https://spaces.ac.cn/archives/11882)\n  * [让炼丹更科学一些（八）：多阶段训练...](https://spaces.ac.cn/archives/11879)\n  * [动量的新理解：逼近特征层面的梯度下降](https://spaces.ac.cn/archives/11875)\n  * [流形上的最速下降：7. Stief...](https://spaces.ac.cn/archives/11864)\n  * [除了交叉熵，LM Loss还有什么选择？](https://spaces.ac.cn/archives/11854)\n  * [简单谈谈K3的MoE和Attention](https://spaces.ac.cn/archives/11848)\n  * [解构Scaling Law：优化、...](https://spaces.ac.cn/archives/11833)\n\n\n## COMMENTS\n  * [yicli: 苏老师好，我有两个问题想再请教一下： 1. Full Attn...](https://spaces.ac.cn/archives/11664/comment-page-3#comment-29805)\n  * [Lindsay: 这是不是也是为什么diffusion text model质量...](https://spaces.ac.cn/archives/11854/comment-page-1#comment-29804)\n  * [Honghao: 一般结论那节中“$q_t=E[L(\\theta_t)] - L...](https://spaces.ac.cn/archives/11494/comment-page-1#comment-29803)\n  * [田奕哲: 似乎engram与其思路有点相似 都是用hash直接通过tok...](https://spaces.ac.cn/archives/11750/comment-page-1#comment-29802)\n  * [Kacent: 苏神好,我想请教一下. 如果b只被LossFree更新, ...](https://spaces.ac.cn/archives/10757/comment-page-3#comment-29801)\n  * [Dr.Ink: 受教、感谢苏神～ 最近做实验建立更深的理解、也在尝试一些改进，...](https://spaces.ac.cn/archives/11619/comment-page-1#comment-29799)\n  * [Honghao: 感谢苏神回复！](https://spaces.ac.cn/archives/11875/comment-page-1#comment-29798)\n  * [苏剑林: 我们对ds方案和qb做了充分对比，结论是qb更均衡，并且效果（...](https://spaces.ac.cn/archives/11619/comment-page-1#comment-29797)\n  * [苏剑林: 不是愿不愿意更新的问题，主要是暂时对这个方向没有什么新理解](https://spaces.ac.cn/archives/10352/comment-page-2#comment-29796)\n  * [苏剑林: 确实，更正了，感谢反馈](https://spaces.ac.cn/archives/9431/comment-page-2#comment-29795)\n\n\n## USERLOGIN\n  * [登录](https://spaces.ac.cn/admin/login.php)\n\n\n# [科学空间|Scientific Spaces](https://spaces.ac.cn)\n  * [登录](https://spaces.ac.cn/admin/login.php)\n  * [打赏](https://spaces.ac.cn/reward.html)\n  * [公式](https://spaces.ac.cn/latex.html)\n  * [天象](https://spaces.ac.cn/ac.html)\n  * [链接](https://spaces.ac.cn/links.html)\n  * [时光](https://spaces.ac.cn/me.html)\n  * [博览](https://spaces.ac.cn/science.html)\n  * [归档](https://spaces.ac.cn/content.html)\n\n\n渴望成为一个小飞侠\n  * [![](https://spaces.ac.cn/usr/themes/geekg/images/rss.png)  \n欢迎订阅](https://spaces.ac.cn/feed)\n  * [![](https://spaces.ac.cn/usr/themes/geekg/images/mail.png)  \n个性邮箱](https://spaces.ac.cn/archives/119)\n  * [![](https://spaces.ac.cn/usr/themes/geekg/images/Saturn.png)  \n天象信息](https://spaces.ac.cn/ac.html)\n  * [![](https://spaces.ac.cn/usr/themes/geekg/images/iss.png)  \n观测ISS](https://spaces.ac.cn/archives/41)\n  * [![](https://spaces.ac.cn/usr/themes/geekg/images/pi.png)  \nLaTeX](https://spaces.ac.cn/latex.html)\n  * [![](https://spaces.ac.cn/usr/themes/geekg/images/mlogo.png)  \n关于博主](https://spaces.ac.cn/me.html)\n\n\n欢迎访问“科学空间”，这里将与您共同探讨自然科学，回味人生百态；也期待大家的分享～\n  * [**千奇百怪** Everything](https://spaces.ac.cn/category/Everything)\n  * [**天文探索** Astronomy](https://spaces.ac.cn/category/Astronomy)\n  * [**数学研究** Mathematics](https://spaces.ac.cn/category/Mathematics)\n  * [**物理化学** Phy-chem](https://spaces.ac.cn/category/Phy-chem)\n  * [**信息时代** Big-Data](https://spaces.ac.cn/category/Big-Data)\n  * [**生物自然** Biology](https://spaces.ac.cn/category/Biology)\n  * [**图片摄影** Photograph](https://spaces.ac.cn/category/Photograph)\n  * [**问题百科** Questions](https://spaces.ac.cn/category/Questions)\n  * [**生活/情感** Life-Feeling](https://spaces.ac.cn/category/Life-Feeling)\n  * [**资源共享** Resources](https://spaces.ac.cn/category/Resources)\n\n\n  * [**千奇百怪**](https://spaces.ac.cn/category/Everything)\n  * [**天文探索**](https://spaces.ac.cn/category/Astronomy)\n  * [**数学研究**](https://spaces.ac.cn/category/Mathematics)\n  * [**物理化学**](https://spaces.ac.cn/category/Phy-chem)\n  * [**信息时代**](https://spaces.ac.cn/category/Big-Data)\n  * [**生物自然**](https://spaces.ac.cn/category/Biology)\n  * [**图片摄影**](https://spaces.ac.cn/category/Photograph)\n  * [**问题百科**](https://spaces.ac.cn/category/Questions)\n  * [**生活/情感**](https://spaces.ac.cn/category/Life-Feeling)\n  * [**资源共享**](https://spaces.ac.cn/category/Resources)\n\n\n  \n\n## 找不到相关内容。\n  \n\n### 你也许对下面的内容感兴趣\n  * [从费马大定理谈起（十）：x^3+y^3=z^3+w^3](https://spaces.ac.cn/archives/2972)\n  * [Transformer升级之路：5、作为无限维的线性Attention](https://spaces.ac.cn/archives/8601)\n  * [预训练一下，Transformer的长序列成绩还能涨不少！](https://spaces.ac.cn/archives/9787)\n  * [科学空间：2010年5月重要天象](https://spaces.ac.cn/archives/627)\n  * [[2011]一睹“食”的风采](https://spaces.ac.cn/archives/1165)\n  * [对比学习可以使用梯度累积吗？](https://spaces.ac.cn/archives/8471)\n  * [基于CNN的阅读理解式问答模型：DGCNN](https://spaces.ac.cn/archives/5409)\n  * [【NASA每日一图】甘姆星云(Gum Nebula)](https://spaces.ac.cn/archives/92)\n  * [【NASA每日一图】牧羊卫星Prometheus](https://spaces.ac.cn/archives/391)\n  * [日食当天的天气出来了](https://spaces.ac.cn/archives/17)\n\n\n### 关于站长\n![科学空间logo](https://spaces.ac.cn/usr/themes/geekg/images/avatar.png)\n**苏剑林|BoJone** ，科学空间博主，【数学、天文、理论物理、写作、阅读、计算机、中国象棋、厨房】爱好者（但不专业）......目前33岁，还在单调递增。希望能一直在此分享科学之美～\n**你也许会关心：**\n* [科学空间|Scientific Spaces 介绍](https://spaces.ac.cn/me.html)\n* 科学空间QQ交流群：67729435\n* 科学空间微信交流群：[spaces_ac_cn](https://spaces.ac.cn/archives/4096/)\n* 常见问题集：[《科学空间FAQ》](https://spaces.ac.cn/archives/6508/)\n### 智能搜索\n支持整句搜索！网站自动使用[结巴分词](https://github.com/fxsjy/jieba)进行分词，并结合ngrams排序算法给出合理的搜索结果。\n### 热门标签\n[attention](https://spaces.ac.cn/tag/attention/) [生成模型](https://spaces.ac.cn/tag/%E7%94%9F%E6%88%90%E6%A8%A1%E5%9E%8B/) [优化](https://spaces.ac.cn/tag/%E4%BC%98%E5%8C%96/) [模型](https://spaces.ac.cn/tag/%E6%A8%A1%E5%9E%8B/) [语言模型](https://spaces.ac.cn/tag/%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B/) [梯度](https://spaces.ac.cn/tag/%E6%A2%AF%E5%BA%A6/) [优化器](https://spaces.ac.cn/tag/%E4%BC%98%E5%8C%96%E5%99%A8/) [矩阵](https://spaces.ac.cn/tag/%E7%9F%A9%E9%98%B5/) [概率](https://spaces.ac.cn/tag/%E6%A6%82%E7%8E%87/) [网站](https://spaces.ac.cn/tag/%E7%BD%91%E7%AB%99/) [转载](https://spaces.ac.cn/tag/%E8%BD%AC%E8%BD%BD/) [微分方程](https://spaces.ac.cn/tag/%E5%BE%AE%E5%88%86%E6%96%B9%E7%A8%8B/) [分析](https://spaces.ac.cn/tag/%E5%88%86%E6%9E%90/) [天象](https://spaces.ac.cn/tag/%E5%A4%A9%E8%B1%A1/) [深度学习](https://spaces.ac.cn/tag/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0/) [积分](https://spaces.ac.cn/tag/%E7%A7%AF%E5%88%86/) [python](https://spaces.ac.cn/tag/python/) [几何](https://spaces.ac.cn/tag/%E5%87%A0%E4%BD%95/) [扩散](https://spaces.ac.cn/tag/%E6%89%A9%E6%95%A3/) [力学](https://spaces.ac.cn/tag/%E5%8A%9B%E5%AD%A6/) [无监督](https://spaces.ac.cn/tag/%E6%97%A0%E7%9B%91%E7%9D%A3/) [损失函数](https://spaces.ac.cn/tag/%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0/) [节日](https://spaces.ac.cn/tag/%E8%8A%82%E6%97%A5/) [生活](https://spaces.ac.cn/tag/%E7%94%9F%E6%B4%BB/) [不等式](https://spaces.ac.cn/tag/%E4%B8%8D%E7%AD%89%E5%BC%8F/) \n\n### 随机文章\n  * [除夕试拍星空星轨～](https://spaces.ac.cn/archives/4222)\n  * [输入梯度惩罚与参数梯度惩罚的一个不等式](https://spaces.ac.cn/archives/8796)\n  * [修改Transformer结构，设计一个更快更好的MLM模型](https://spaces.ac.cn/archives/7661)\n  * [“Cool Papers + 站内搜索”的一些新尝试](https://spaces.ac.cn/archives/10311)\n  * [今天你食了吗？（广东云浮观测日偏食之旅）](https://spaces.ac.cn/archives/30)\n  * [变分自编码器（七）：球面上的VAE（vMF-VAE）](https://spaces.ac.cn/archives/8404)\n  * [能量视角下的GAN模型（一）：GAN＝“挖坑”＋“跳坑”](https://spaces.ac.cn/archives/6316)\n  * [BLOG评论故障修复，部分数据丢失](https://spaces.ac.cn/archives/31)\n  * [有限素域上的乘法群是循环群](https://spaces.ac.cn/archives/3200)\n  * [计算夏至的精确时刻2——提高精确度](https://spaces.ac.cn/archives/906)\n\n\n### 最近评论\n  * [yicli](https://spaces.ac.cn/archives/11664/comment-page-3#comment-29805): 苏老师好，我有两个问题想再请教一下： 1. Full AttnRes 被放弃的主要原因是训练时...\n  * [Lindsay](https://spaces.ac.cn/archives/11854/comment-page-1#comment-29804): 这是不是也是为什么diffusion text model质量不好的原因？因为本质上break...\n  * [Honghao](https://spaces.ac.cn/archives/11494/comment-page-1#comment-29803): 一般结论那节中“q_t=E[L(\\theta_t)] - L(\\theta^*)”应该是“...\n  * [田奕哲](https://spaces.ac.cn/archives/11750/comment-page-1#comment-29802): 似乎engram与其思路有点相似 都是用hash直接通过tokenid来查询 把学习转成查表\n  * [Kacent](https://spaces.ac.cn/archives/10757/comment-page-3#comment-29801): 苏神好,我想请教一下. 如果b只被LossFree更新, 而不被LM head更新, 那么...\n  * [Dr.Ink](https://spaces.ac.cn/archives/11619/comment-page-1#comment-29799): 受教、感谢苏神～ 最近做实验建立更深的理解、也在尝试一些改进，后续再向你请教\n  * [Honghao](https://spaces.ac.cn/archives/11875/comment-page-1#comment-29798): 感谢苏神回复！\n  * [苏剑林](https://spaces.ac.cn/archives/11619/comment-page-1#comment-29797): 我们对ds方案和qb做了充分对比，结论是qb更均衡，并且效果（loss和下游bmk）基本无差异...\n  * [苏剑林](https://spaces.ac.cn/archives/10352/comment-page-2#comment-29796): 不是愿不愿意更新的问题，主要是暂时对这个方向没有什么新理解\n  * [苏剑林](https://spaces.ac.cn/archives/9431/comment-page-2#comment-29795): 确实，更正了，感谢反馈\n\n\n### 友情链接\n  * [Cool Papers](https://papers.cool)\n  * [数学研发](https://bbs.emath.ac.cn)\n  * [Seatop](http://www.seatop.com.cn/)\n  * [Xiaoxia](https://xiaoxia.org/)\n  * [积分表-网络版](https://kexue.fm/sci/integral/index.html)\n  * [丝路博傲](http://blog.dvxj.com/)\n  * [数学之家](http://www.2math.cn/)\n  * [有趣天文奇观](http://interesting-sky.china-vo.org/)\n  * [TwistedW](http://www.twistedwg.com/)\n  * [godweiyang](https://godweiyang.com/)\n  * [AI柠檬](https://blog.ailemon.net/)\n  * [王登科-DK博客](https://greatdk.com)\n  * [ESON](https://blog.eson.org/)\n  * [枫之羽](https://fzhiy.net/)\n  * [coding-zuo](https://coding-zuo.github.io/)\n  * [博科园](https://www.bokeyuan.net/)\n  * [孔皮皮的博客](https://www.kppkkp.top/)\n  * [运鹏的博客](https://yunpengtai.top/)\n  * [jiming.site](https://jiming.site/)\n  * [OmegaXYZ](https://www.omegaxyz.com/)\n  * [EAI猩球](https://www.robotech.ink/)\n  * [文举的博客](https://liwenju0.com/)\n  * [汇尘轩](https://kirigaya.cn/)\n  * [申请链接](https://kexue.fm/links.html)\n\n\n[![署名-非商业用途-保持一致](https://kexue.fm/usr/themes/geekg/images/cc.gif)](http://creativecommons.org/licenses/by-nc-nd/2.5/cn/) 本站采用创作共用版权协议，要求署名、非商业用途和保持一致。转载本站内容必须也遵循“[署名-非商业用途-保持一致](http://creativecommons.org/licenses/by-nc-nd/2.5/cn/)”的创作共用协议。   \n© 2009-2026 Scientific Spaces. All rights reserved. Theme by [laogui](http://www.laogui.com). Powered by [Typecho](http://typecho.org). 备案号: [粤ICP备09093259号-1/2](https://beian.miit.gov.cn/ \"粤ICP备09093259号\")。 \n",
  "body_base_path": "attachments/6b486c08673c/pages/",
  "body_markdown_path": "attachments/6b486c08673c/pages/archived-body.md",
  "local_html": "attachments/6b486c08673c/pages/archived-body.html",
  "body_source": "prior_capture"
}