{ "source_file": "0930", "source_platform": "个人博客", "author": "苏剑林", "title_listed": "Transformer升级之路：6、旋转位置编码的完备性分析", "keywords": "RoPE,完备性,正交矩阵", "domain": "大模型算法", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://spaces.ac.cn/archives/9403", "publish_date": "2022-12", "corpus_stage": "预训练/SFT", "notes": "RoPE 一般解的数学分析", "id": "3d322729b230", "collected_at": "2026-10-02T15:14:42", "title": "Transformer升级之路：6、旋转位置编码的完备性分析 - 科学空间|Scientific Spaces", "body_markdown": "SEARCH\nMENU\n打赏\n公式\n天象\n链接\n时光\n博览\n归档\nCATEGORIES\n千奇百怪\n天文探索\n数学研究\n物理化学\n信息时代\n生物自然\n图片摄影\n问题百科\n生活/情感\n资源共享\nNEWPOSTS\n如何让GDN2更稳定一些？\n通过微扰分析求解等式约束优化问题\n慢即是快：1. Shampoo的尽...\n排序不等式及其推广\n让炼丹更科学一些（十）：单调性假设...\n让炼丹更科学一些（九）：经典自适应...\n让炼丹更科学一些（八）：多阶段训练...\n动量的新理解：逼近特征层面的梯度下降\n流形上的最速下降：7. Stief...\n除了交叉熵，LM Loss还有什么选择？\nCOMMENTS\ncarl_lei\n: 苏神，之前我一直有一个直觉是：MoE通过不同系数加权得到的FF...\nAli Hatamizadeh\n: Thank you for looking into this...\n苏剑林\n: 对于线性层，取max能得到谱范数，然后可以推出Muon，Muo...\n苏剑林\n: Thanks for sharing! I looked in...\nPurshow\n: 苏老师，请问下取max会不会影响控制下界太宽松了呢\nbookworm\n: 天下苦秦久矣，期待苏老师的教材！\nAli Hatamizadeh\n: Hi ! I am the lead GDN-2 author...\nyixiao chen\n: 谢谢苏神！代码在https://github.com/Chyx...\n苏剑林\n: 欢迎作者！我仔细读了一下，感觉写作还是很清晰的，至少比 htt...\n苏剑林\n: 免训练估计够呛，从实用的角度看，其实也不应该相信任何免训练的可能\nUSERLOGIN\n登录\n科学空间|Scientific Spaces\n登录\n打赏\n公式\n天象\n链接\n时光\n博览\n归档\n渴望成为一个小飞侠\n欢迎订阅\n个性邮箱\n天象信息\n观测ISS\nLaTeX\n关于博主\n欢迎访问“科学空间”，这里将与您共同探讨自然科学，回味人生百态；也期待大家的分享～\n千奇百怪\nEverything\n天文探索\nAstronomy\n数学研究\nMathematics\n物理化学\nPhy-chem\n信息时代\nBig-Data\n生物自然\nBiology\n图片摄影\nPhotograph\n问题百科\nQuestions\n生活/情感\nLife-Feeling\n资源共享\nResources\n千奇百怪\n天文探索\n数学研究\n物理化学\n信息时代\n生物自然\n图片摄影\n问题百科\n生活/情感\n资源共享\n首页\n信息时代\nTransformer升级之路：6、旋转位置编码的完备性分析\n28\nDec\nTransformer升级之路：6、旋转位置编码的完备性分析\nBy\n苏剑林\n|\n2022-12-28\n|\n78492位读者\n|\n:\n在去年的文章\n《Transformer升级之路：2、博采众长的旋转式位置编码》\n中，笔者提出了旋转位置编码（RoPE），当时的出发点只是觉得用绝对位置来实现相对位置是一件“很好玩的事情”，并没料到其实际效果还相当不错，并为大家所接受，不得不说这真是一个意外之喜。后来，在\n《Transformer升级之路：4、二维位置的旋转式位置编码》\n中，笔者讨论了二维形式的RoPE，并研究了用矩阵指数表示的RoPE的一般解。\n既然有了一般解，那么自然就会引出一个问题：我们常用的RoPE，只是一个以二维旋转矩阵为基本单元的分块对角矩阵，如果换成一般解，理论上效果会不会更好呢？本文就来回答这个问题。\n指数通解\n#\n在\n《Transformer升级之路：4、二维位置的旋转式位置编码》\n中，我们将RoPE抽象地定义为任意满足下式的方阵\n\\begin{equation}\\boldsymbol{\\mathcal{R}}_m^{\\top}\\boldsymbol{\\mathcal{R}}_n=\\boldsymbol{\\mathcal{R}}_{n-m}\\label{eq:re}\\end{equation}\n然后，我们探讨了如下矩阵指数形式的解\n\\begin{equation}\\boldsymbol{\\mathcal{R}}_n=\\exp n\\boldsymbol{B}\\end{equation}\n这里的矩阵指数，不是像Softmax那样的激活函数式的element-wise运算，而是按照泰勒级数定义的“\nMatrix Exponential\n”。根据“\nBaker–Campbell–Hausdorff公式\n”，我们有\n\\begin{equation}\\begin{aligned}\n\\boldsymbol{\\mathcal{R}}_m^{\\top}\\boldsymbol{\\mathcal{R}}_n=&\\,\\big(\\exp m\\boldsymbol{B}\\big)^{\\top}\\big(\\exp n\\boldsymbol{B}\\big) = \\big(\\exp m\\boldsymbol{B}^{\\top}\\big)\\big(\\exp n\\boldsymbol{B}\\big) \\\\\n=&\\,\\exp\\left(m\\boldsymbol{B}^{\\top} + n\\boldsymbol{B} + \\frac{1}{2}mn\\left[\\boldsymbol{B}^{\\top}, \\boldsymbol{B}\\right]+\\cdots\\right)\n\\end{aligned}\\end{equation}\n这里$\\left[\\boldsymbol{A}, \\boldsymbol{B}\\right]=\\boldsymbol{A}\\boldsymbol{B}-\\boldsymbol{B}\\boldsymbol{A}$，$\\cdots$省略的都是$m,n$的三次或三次以上的项。按照式$\\eqref{eq:re}$，那么上式指数部分应该等于$(n-m)\\boldsymbol{B}$，这就推出\n\\begin{equation}\\boldsymbol{B}^{\\top} = - \\boldsymbol{B}\\end{equation}\n即要求$\\boldsymbol{B}$是反对称矩阵。\n正交通解\n#\n进一步地，我们有$(\\exp \\boldsymbol{B})^{\\top}(\\exp \\boldsymbol{B})=\\exp(\\boldsymbol{B}-\\boldsymbol{B}) = \\boldsymbol{I}$和$\\exp n\\boldsymbol{B} = \\left(\\exp\\boldsymbol{B}\\right)^n$，前者说明$\\exp\\boldsymbol{B}$是正交矩阵，后者则启示我们这是不是可以推广到任意正交矩阵？不难验证，答案是肯定的，我们有结论：\n对于任意正交矩阵$\\boldsymbol{O}$，$\\boldsymbol{\\mathcal{R}}_n=\\boldsymbol{O}^n$是满足式$\\eqref{eq:re}$的解。\n值得指出的是，在实数域内，并非所有正交矩阵能写成$\\exp\\boldsymbol{B}$的形式，所以$\\boldsymbol{O}^n$实际上比矩阵指数形式更宽泛。从\n《恒等式 det(exp(A)) = exp(Tr(A)) 赏析》\n可知$\\det(\\exp(\\boldsymbol{A})) = \\exp(\\text{Tr}(\\boldsymbol{A})) > 0$，所以能写成矩阵指数形式的正交矩阵行列式必然大于0（即等于1），事实上这个结果反过来也成立，即行列式等于1的正交矩阵，必然可以写成$\\exp\\boldsymbol{B}$的形式，其中$\\boldsymbol{B}$是反对称矩阵。（参考\n《Why can any orthogonal matrix be written as O=e^A》\n）。\n对于$\\det(\\boldsymbol{O}) = -1$的正交矩阵，我们有$\\boldsymbol{O}=\\boldsymbol{O}_+ \\boldsymbol{I}_-$，其中$\\boldsymbol{I}_-$是对角线有一个-1、剩下都是1的对角阵，$\\boldsymbol{O}_+$是$\\det(\\boldsymbol{O}_+) = 1$的正交矩阵，它可以写成$\\exp\\boldsymbol{B}$的形式，此时$\\boldsymbol{O}^n = (\\boldsymbol{O}_+ \\boldsymbol{I}_-)^n = \\boldsymbol{I}_-^n\\exp n\\boldsymbol{B}$。这也就是说，即便对于$\\det(\\boldsymbol{O}) = -1$的$\\boldsymbol{O}^n$，也只是$\\exp n\\boldsymbol{B}$的简单变换，所以接下来我们主要研究$\\exp n\\boldsymbol{B}$形式的解。\n完备分析\n#\n众所周知，我们平时所用的RoPE位置编码，是如下形式的分块对角矩阵：\n\\begin{equation}\\scriptsize{\\left(\\begin{array}{cc:cc:cc:cc}\n\\cos n\\theta_0 & -\\sin n\\theta_0 & 0 & 0 & \\cdots & \\cdots & 0 & 0 \\\\\n\\sin n\\theta_0 & \\cos n\\theta_0 & 0 & 0 & \\cdots & \\cdots & 0 & 0 \\\\\n\\hdashline\n0 & 0 & \\cos n\\theta_1 & -\\sin n\\theta_1 & \\cdots & \\cdots & 0 & 0 \\\\\n0 & 0 & \\sin n\\theta_1 & \\cos n\\theta_1 & \\cdots & \\cdots & 0 & 0 \\\\\n\\hdashline\n\\vdots & \\vdots & \\vdots & \\vdots & \\ddots & \\ddots & \\vdots & \\vdots \\\\\n\\vdots & \\vdots & \\vdots & \\vdots & \\ddots & \\ddots & \\vdots & \\vdots \\\\\n\\hdashline\n0 & 0 & 0 & 0 & \\cdots & \\cdots & \\cos n\\theta_{d/2-1} & -\\sin n\\theta_{d/2-1} \\\\\n0 & 0 & 0 & 0 & \\cdots & \\cdots & \\sin n\\theta_{d/2-1} & \\cos n\\theta_{d/2-1} \\\\\n\\end{array}\\right)}\\end{equation}\n它可以简写成\n\\begin{equation}\\begin{pmatrix}\n\\boldsymbol{R}_{n\\theta_0} & \\boldsymbol{0} & \\cdots & \\boldsymbol{0} \\\\\n\\boldsymbol{0} & \\boldsymbol{R}_{n\\theta_1} & \\cdots & \\boldsymbol{0} \\\\\n\\vdots & \\vdots & \\ddots & \\vdots \\\\\n\\boldsymbol{0} & \\boldsymbol{0} & \\cdots & \\boldsymbol{R}_{n\\theta_{d/2-1}} \\\\\n\\end{pmatrix} = \\exp n\\begin{pmatrix}\n\\boldsymbol{J}_{\\theta_0} & \\boldsymbol{0} & \\cdots & \\boldsymbol{0} \\\\\n\\boldsymbol{0} & \\boldsymbol{J}_{\\theta_1} & \\cdots & \\boldsymbol{0} \\\\\n\\vdots & \\vdots & \\ddots & \\vdots \\\\\n\\boldsymbol{0} & \\boldsymbol{0} & \\cdots & \\boldsymbol{J}_{\\theta_{d/2-1}} \\\\\n\\end{pmatrix}\\end{equation}\n其中\n\\begin{equation}\\boldsymbol{R}_{\\theta} = \\begin{pmatrix} \\cos\\theta & -\\sin\\theta \\\\ \\sin\\theta & \\cos\\theta \\end{pmatrix},\\quad \\boldsymbol{J}_{\\theta} = \\begin{pmatrix} 0 & -\\theta \\\\ \\theta & 0\\end{pmatrix}\\end{equation}\n这种选择可以说是最简单的一种，其本质原因可以说是为了降低计算量。那么，所谓完备性问题，就是要回答：如上的分块对角矩阵的特例，相比全参数$\\exp n\\boldsymbol{B}$，是否有能力上的缺失？换句话说，如果不考虑计算量，将$\\boldsymbol{B}$替换为一般的反对称矩阵，效果是否可能会有提升？\n回答这个问题不困难，事实上，对于任意偶数阶反对称矩阵，它都可以对角化为分块对角矩阵\n\\begin{equation}\\boldsymbol{\\Lambda} = \\begin{pmatrix}\n\\boldsymbol{J}_{\\theta_0} & \\boldsymbol{0} & \\cdots & \\boldsymbol{0} \\\\\n\\boldsymbol{0} & \\boldsymbol{J}_{\\theta_1} & \\cdots & \\boldsymbol{0} \\\\\n\\vdots & \\vdots & \\ddots & \\vdots \\\\\n\\boldsymbol{0} & \\boldsymbol{0} & \\cdots & \\boldsymbol{J}_{\\theta_{d/2-1}} \\\\\n\\end{pmatrix}\\end{equation}\n该结论可以参考\nSkew-symmetric matrix\n。也就是说，存在可逆矩阵$\\boldsymbol{P}$，使得$\\boldsymbol{B}=\\boldsymbol{P}\\boldsymbol{\\Lambda}\\boldsymbol{P}^{-1}$，于是\n\\begin{equation}\\exp n\\boldsymbol{B} = \\exp \\left(n\\boldsymbol{P}\\boldsymbol{\\Lambda}\\boldsymbol{P}^{-1}\\right) = \\boldsymbol{P}(\\exp n\\boldsymbol{\\Lambda})\\boldsymbol{P}^{-1}\\end{equation}\n也就是说，任意的$\\exp n\\boldsymbol{B}$与分块对角的$\\exp n\\boldsymbol{\\Lambda}$，仅仅相差一个相似变换，而我们在Self Attention中应用RoPE时，是\n\\begin{equation}\\boldsymbol{q}^{\\top}\\big(\\exp (n-m)\\boldsymbol{B}\\big)\\boldsymbol{k} = \\big(\\boldsymbol{P}^{\\top}\\boldsymbol{q}\\big)^{\\top}\\big(\\exp (n-m)\\boldsymbol{\\Lambda}\\big)\\big(\\boldsymbol{P}^{-1}\\boldsymbol{k}\\big)\\end{equation}\n由于$\\boldsymbol{q},\\boldsymbol{k}$一般都是输入$\\boldsymbol{x}$经过某个可学习的线性变换而来，$\\boldsymbol{P}^{\\top},\\boldsymbol{P}^{-1}$原则上都可以吸收到线性变换的训练参数中，因此直接设为$\\boldsymbol{q}^{\\top}\\big(\\exp (n-m)\\boldsymbol{\\Lambda}\\big)\\boldsymbol{k}$理论上不会损失一般性。\n所以，对于Self Attention来说，问题的答案是否定的。不过，如果是\n线性Attention\n，答案会有少许区别，因为线性Attention的$\\boldsymbol{q},\\boldsymbol{k}$加了个激活函数：\n\\begin{equation}\\phi(\\boldsymbol{q})^{\\top}\\big(\\exp (n-m)\\boldsymbol{B}\\big)\\varphi(\\boldsymbol{k}) = \\big(\\boldsymbol{P}^{\\top}\\phi(\\boldsymbol{q})\\big)^{\\top}\\big(\\exp (n-m)\\boldsymbol{\\Lambda}\\big)\\big(\\boldsymbol{P}^{-1}\\varphi(\\boldsymbol{k})\\big)\\end{equation}\n这就导致了$\\boldsymbol{P}^{\\top},\\boldsymbol{P}^{-1}$不一定能吸收到线性变换的训练参数中，因此对线性Attention补上两个参数矩阵，是有可能带来提升的。\n文章小结\n#\n本文简单分析了RoPE的完备性问题，表明对于Self Attention来说，目前的分块对角型RoPE不会损失一般性。\n转载到请包括本文地址：\nhttps://spaces.ac.cn/archives/9403\n更详细的转载事宜请参考：\n《科学空间FAQ》\n如果您还有什么疑惑或建议，欢迎在下方评论区继续讨论。\n如果您觉得本文还不错，欢迎\n分享\n/\n打赏\n本文。打赏并非要从中获得收益，而是希望知道科学空间获得了多少读者的真心关注。当然，如果你无视它，也不会影响你的阅读。再次表示欢迎和感谢！\n打赏\n微信打赏\n支付宝打赏\n因为网站后台对打赏并无记录，因此欢迎在打赏时候备注留言。\n你还可以\n点击这里\n或在下方评论区留言来告知你的建议或需求。\n如果您需要引用本文，请参考：\n苏剑林. (Dec. 28, 2022). 《Transformer升级之路：6、旋转位置编码的完备性分析 》[Blog post]. Retrieved from\nhttps://spaces.ac.cn/archives/9403\n@online{kexuefm-9403,\ntitle={Transformer升级之路：6、旋转位置编码的完备性分析},\nauthor={苏剑林},\nyear={2022},\nmonth={Dec},\nurl={\\url{https://spaces.ac.cn/archives/9403}},\n}\n分类：\n信息时代\n标签：\n矩阵\n,\nattention\n,\n位置编码\n,\nrope\n22 评论\n<\n生成扩散模型漫谈（十五）：构建ODE的一般步骤（中）\n|\n智能家居之热水器零冷水技术原理浅析\n>\n你也许还对下面的内容感兴趣\n如何让GDN2更稳定一些？\n排序不等式及其推广\n让炼丹更科学一些（十）：单调性假设的拆与补\n简单谈谈K3的MoE和Attention\n将Softmax Attention线性化为Gated DeltaNet\nLogSumExp和Softmax的泰勒展开\n矩阵函数近似中的暴力美学\n矩阵参数的奇异值熵越高越好吗？\n直接以FID为Loss：从梯度计算到流式训练\n如何更科学地估计矩阵的谱范数？\n发表你的看法\nDoraemonzzz\nDecember 28th, 2022\n苏神，我们的一篇工作从类似的角度讨论了这个问题，有一些实验结果：https://openreview.net/forum?id=xMWFqb5Uyk&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DICLR.cc%2F2023%2FConference%2FAuthors%23your-submissions)\n回复评论\n苏剑林\n发表于\n December 28th, 2022\n好的好的，感谢推荐，我等会拜读一下。\n回复评论\nDoraemonzzz\n发表于\n December 28th, 2022\n苏神，我添加了spaces_ac_cn微信，可否通过一下。\n回复评论\n苏剑林\n发表于\n December 28th, 2022\n已通过\n回复评论\nEnrico Shippole\nJanuary 7th, 2023\nDear Su Jianlin,\nI was wondering how you handled issues relating to length extrapolation for Rotary Embeddings and relative positional encoding for the encoder? From my understanding, Rotary Embeddings work best for the decoder since they do not length-generalize that well.\nI greatly admire and appreciate your research.\nThank you,\nEnrico\n回复评论\n苏剑林\n发表于\n January 9th, 2023\nThanks for your appreciation. In next 2 blog posts I will discuss this topic.\n回复评论\nCecilia Lee\nJanuary 11th, 2023\n式子（10）和（11）第一个括号上是不是少了一个转置\n回复评论\n苏剑林\n发表于\n January 13th, 2023\n你说$\\big(\\exp (n-m)\\boldsymbol{B}\\big)$？为什么要多一个转置？\n回复评论\ntest123\nJanuary 18th, 2023\n想请教一下公式10，左右两边是怎么相等的？\n回复评论\n苏剑林\n发表于\n January 19th, 2023\n因为式$(9)$\n回复评论\ntest123 发表于\n January 19th, 2023\n那不是应该\n\\begin{equation}\\boldsymbol{q}^{\\top}\\big(\\exp (n-m)\\boldsymbol{B}\\big)\\boldsymbol{k} = \\big(\\boldsymbol{q}^{\\top}\\boldsymbol{p}\\big)\\big(\\exp (n-m)\\boldsymbol{\\Lambda}\\big)\\big(\\boldsymbol{P}^{-1}\\boldsymbol{k}\\big)\\end{equation}\n回复评论\n苏剑林\n发表于\n January 30th, 2023\n哦哦，对的，我忘记多写个转置了。已修正，谢谢指出。\n回复评论\nMr Yan\nJuly 9th, 2023\n请教一下，如果m，n属于R^2,那这个公式会推导成什么样子。\n我现在在研究输入不是一行文本，而是一个矩阵文本的情形，需要用R^2来表示位置坐标。\n回复评论\nMr Yan 发表于\n July 9th, 2023\n抱歉，之前看过第四篇文章，忘记了，我重新拜读一遍再提问..\n回复评论\nMr Yan 发表于\n July 9th, 2023\n实验了一下RoPE 2D，我是把输入的向量分别施加x和y的一维RoPE，然后拼起来（输出的embedding比输入维度翻了一倍），然后发现确实收敛速度快了不少，最终效果也有提升。\n基本上是\n绝对位置编码 1D < RoPE 1D < 绝对位置编码 2D < RoPE 2D\n我的任务上最终精度，绝对位置编码 2D 和 RoPE 2D没特别多，但是在收敛速度上有非常明显的提升。\n大神666\n回复评论\n苏剑林\n发表于\n July 10th, 2023\n好的，恭喜，那就假设您已经解决了这个问题哈。\n回复评论\nsdz0114 发表于\n October 1st, 2025\n您好，最近在学习一些RoPE的相关知识，想咨询下您实现的RoPE 2D就是对每个token，进行X和Y的RoPE，然后concat对吗？\n回复评论\n苏剑林\n发表于\n October 3rd, 2025\n每个token的hidden state，一半的dims做X的RoPE，另一半的dims做Y的RoPE，然后拼接。\n回复评论\nwhc\nDecember 16th, 2025\nhttps://arxiv.org/abs/2502.02562 里提出两种低成本parameterize $P$的方法\n其中之一是用DFT matrix作为固定的$P$, 这样运算速度不太慢，效果上也有一些提升，尤其是机器人相关任务上\n您觉得这个工作有意思么\n回复评论\n苏剑林\n发表于\n December 18th, 2025\n感觉看场景吧，如果是文本llm下感觉没什么必要性。\n回复评论\nsk\nMay 11th, 2026\n请教个问题，反对称矩阵的分解$B=P \\Lambda P^{-1}$是根据 https://en.wikipedia.org/wiki/Skew-symmetric_matrix#Spectral_theory 中的反对称矩阵谱理论得出的吧？\n第一：该理论提到的是$A=Q \\Sigma Q^{T}$,$Q$为正交矩阵，且分解公式是Q的转置，但是文章中用的却是逆$P^{-1}$。\n第二：$\\Sigma$ 的形式如下，$\\lambda$为$A$的特征值。$\\Lambda$ 的形式如下，但是$\\theta$未必是特征值。\n$$\n\\color{blue}{\n\\Sigma=\n\\begin{pmatrix}\n0 & \\lambda_1 & 0 & 0 & \\cdots & 0 & 0 \\\\\n-\\lambda_1 & 0 & 0 & 0 & \\cdots & 0 & 0 \\\\\n0 & 0 & 0 & \\lambda_2 & \\cdots & 0 & 0 \\\\\n0 & 0 & -\\lambda_2 & 0 & \\cdots & 0 & 0 \\\\\n\\vdots & \\vdots & \\vdots & \\vdots & \\ddots & \\vdots & \\vdots \\\\\n0 & 0 & 0 & 0 & \\cdots & 0 & \\lambda_r \\\\\n0 & 0 & 0 & 0 & \\cdots & -\\lambda_r & 0 \\\\\n0 & 0 & 0 & 0 & \\cdots & 0 & 0\n\\end{pmatrix}\n}\n$$\n$$\n\\Lambda =\n\\begin{pmatrix}\nJ_{\\theta_0} & 0 & \\cdots & 0 \\\\\n0 & J_{\\theta_1} & \\cdots & 0 \\\\\n\\vdots & \\vdots & \\ddots & \\vdots \\\\\n0 & 0 & \\cdots & J_{\\theta_{d/2-1}}\n\\end{pmatrix}\n$$\n苏神，请问这中间是否有其他推导步骤？烦请不吝赐教。\n回复评论\n苏剑林\n发表于\n May 17th, 2026\n呃，其实是我当时写这个文章的时候，主要比较熟悉的还是相似矩阵的那套东西，所以就按照相似矩阵来写了，对于正交矩阵来说转置就是逆，所以也能套入到相似矩阵的框架中。\n回复评论\n取消回复\n你的大名\n电子邮箱\n个人网站（选填）\n1. 可以使用LaTeX代码，点击“预览效果”可查看效果；\n2. 可以通过点击评论楼层编号来引用该楼层；\n3. 网站可能会有点卡，如非确认评论失败，请\n不要重复点击提交\n。\n内容速览\n指数通解\n正交通解\n完备分析\n文章小结\n智能搜索\n支持整句搜索！网站自动使用\n结巴分词\n进行分词，并结合ngrams排序算法给出合理的搜索结果。\n热门标签\nattention\n生成模型\n优化\n模型\n语言模型\n梯度\n优化器\n矩阵\n概率\n网站\n转载\n微分方程\n分析\n天象\n深度学习\n积分\npython\n几何\n扩散\n力学\n无监督\n损失函数\n节日\n线性\n生活\n随机文章\nLamost下的天文夏令营\n大自然的隐身术——保护色\n概率视角下的线性模型：逻辑回归有解析解吗？\n施密特系统的校正镜方程求解\n基于树莓派Zero2W搭建一个随身旁路由\n梯度流：探索通向最小值之路\n数学基本技艺(A Mathematical Trivium)\nGlobalPointer：用统一的方式处理嵌套和非嵌套NER\n向量结合复数：常曲率曲线(1)\n新年新天象：2010年1月重要天象\n最近评论\ncarl_lei\n: 苏神，之前我一直有一个直觉是：MoE通过不同系数加权得到的FFN相比原始的大FFN（也就是直接...\nAli Hatamizadeh\n: Thank you for looking into this and featuring i...\n苏剑林\n: 对于线性层，取max能得到谱范数，然后可以推出Muon，Muon比Adam、SGD等要好，所以...\n苏剑林\n: Thanks for sharing! I looked into this improvem...\nPurshow\n: 苏老师，请问下取max会不会影响控制下界太宽松了呢\nbookworm\n: 天下苦秦久矣，期待苏老师的教材！\nAli Hatamizadeh\n: Hi ! I am the lead GDN-2 author.We've addressed...\nyixiao chen\n: 谢谢苏神！代码在https://github.com/Chyxx/CyclicFlowAtte...\n苏剑林\n: 欢迎作者！我仔细读了一下，感觉写作还是很清晰的，至少比 https://papers.cool...\n苏剑林\n: 免训练估计够呛，从实用的角度看，其实也不应该相信任何免训练的可能\n友情链接\nCool Papers\n数学研发\nSeatop\nXiaoxia\n积分表-网络版\n丝路博傲\n数学之家\n有趣天文奇观\nTwistedW\ngodweiyang\nAI柠檬\n王登科-DK博客\nESON\n枫之羽\ncoding-zuo\n博科园\n孔皮皮的博客\n运鹏的博客\njiming.site\nOmegaXYZ\nEAI猩球\n文举的博客\n汇尘轩\n申请链接\n本站采用创作共用版权协议，要求署名、非商业用途和保持一致。转载本站内容必须也遵循“\n署名-非商业用途-保持一致\n”的创作共用协议。\n© 2009-2026 Scientific Spaces. All rights reserved. Theme by\nlaogui\n. Powered by\nTypecho\n. 备案号:\n粤ICP备09093259号-1/2\n。", "metadata": { "title": "Transformer升级之路：6、旋转位置编码的完备性分析 - 科学空间|Scientific Spaces", "description": "在去年的文章《Transformer升级之路：2、博采众长的旋转式位置编码》中，笔者提出了旋转位置编码（RoPE），当时的出发点只是觉得用绝对位置来实现相对位置是一件“很好玩的事情”，并没料到其...", "keywords": "矩阵,attention,位置编码,rope", "author": null }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [], "images": [ { "status": "ok", "url": "https://spaces.ac.cn/usr/themes/geekg/images/slide-button.png", "bytes": 4328, "reused": false, "local_path": "attachments/3d322729b230/images/body-fcd8b7d1212b1f5eab2b.png", "src": "https://spaces.ac.cn/usr/themes/geekg/images/slide-button.png", "alt": "MobileSideBar", "downloaded": true }, { "status": "ok", "url": "https://spaces.ac.cn/usr/themes/geekg/images/rss.png", "bytes": 19283, "reused": false, "local_path": "attachments/3d322729b230/images/body-043e6308dc873ff9fa67.png", "src": "https://spaces.ac.cn/usr/themes/geekg/images/rss.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://spaces.ac.cn/usr/themes/geekg/images/mail.png", "bytes": 16811, "reused": false, "local_path": "attachments/3d322729b230/images/body-942ff5c7b650f4d3d31f.png", "src": "https://spaces.ac.cn/usr/themes/geekg/images/mail.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://spaces.ac.cn/usr/themes/geekg/images/Saturn.png", "bytes": 2524, "reused": false, "local_path": "attachments/3d322729b230/images/body-53b9d00850633d612de3.png", "src": "https://spaces.ac.cn/usr/themes/geekg/images/Saturn.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://spaces.ac.cn/usr/themes/geekg/images/iss.png", "bytes": 3301, "reused": false, "local_path": "attachments/3d322729b230/images/body-896b482bdde5ec1df88a.png", "src": "https://spaces.ac.cn/usr/themes/geekg/images/iss.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://spaces.ac.cn/usr/themes/geekg/images/pi.png", "bytes": 939, "reused": false, "local_path": "attachments/3d322729b230/images/body-3f7f2f2318249e4eac9a.png", "src": "https://spaces.ac.cn/usr/themes/geekg/images/pi.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://spaces.ac.cn/usr/themes/geekg/images/mlogo.png", "bytes": 18685, "reused": false, "local_path": "attachments/3d322729b230/images/body-cbbace78f2cd4551d156.png", "src": "https://spaces.ac.cn/usr/themes/geekg/images/mlogo.png", "alt": "", "downloaded": true }, { "status": "ok", "url": "https://spaces.ac.cn/usr/themes/geekg/payment/wx.png", "bytes": 36540, "reused": false, "local_path": "attachments/3d322729b230/images/body-c5d79f289db1f5d431bc.png", "src": "https://spaces.ac.cn/usr/themes/geekg/payment/wx.png", "alt": "科学空间", "downloaded": true }, { "status": "ok", "url": "https://spaces.ac.cn/usr/themes/geekg/payment/zfb.png", "bytes": 31565, "reused": false, "local_path": "attachments/3d322729b230/images/body-4345e58697a529a3d3a5.png", "src": "https://spaces.ac.cn/usr/themes/geekg/payment/zfb.png", "alt": "科学空间", "downloaded": true }, { "status": "ok", "url": "https://kexue.fm/usr/themes/geekg/images/cc.gif", "bytes": 2031, "reused": false, "local_path": "attachments/3d322729b230/images/body-b2904ced8e4508d61bed.gif", "src": "https://kexue.fm/usr/themes/geekg/images/cc.gif", "alt": "署名-非商业用途-保持一致", "downloaded": true } ], "audio": [], "videos": [], "links": [ { "href": "https://spaces.ac.cn/reward.html", "label": "打赏", "type": "a" }, { "href": "https://spaces.ac.cn/latex.html", "label": "公式", "type": "a" }, { "href": "https://spaces.ac.cn/ac.html", "label": "天象", "type": "a" }, { "href": "https://spaces.ac.cn/links.html", "label": "链接", "type": "a" }, { "href": "https://spaces.ac.cn/me.html", "label": "时光", "type": "a" }, { "href": "https://spaces.ac.cn/science.html", "label": "博览", "type": "a" }, { "href": "https://spaces.ac.cn/content.html", "label": "归档", "type": "a" }, { "href": "https://spaces.ac.cn/category/Everything", "label": "千奇百怪", "type": "a" }, { "href": "https://spaces.ac.cn/category/Astronomy", "label": "天文探索", "type": "a" }, { "href": "https://spaces.ac.cn/category/Mathematics", "label": "数学研究", "type": "a" }, { "href": "https://spaces.ac.cn/category/Phy-chem", "label": "物理化学", "type": "a" }, { "href": "https://spaces.ac.cn/category/Big-Data", "label": "信息时代", "type": "a" }, { "href": "https://spaces.ac.cn/category/Biology", "label": "生物自然", "type": "a" }, { "href": "https://spaces.ac.cn/category/Photograph", "label": "图片摄影", "type": "a" }, { "href": "https://spaces.ac.cn/category/Questions", "label": "问题百科", "type": "a" }, { "href": "https://spaces.ac.cn/category/Life-Feeling", "label": "生活/情感", "type": "a" }, { "href": "https://spaces.ac.cn/category/Resources", "label": "资源共享", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11945", "label": "如何让GDN2更稳定一些？", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11928", "label": "通过微扰分析求解等式约束优化问题", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11917", "label": "慢即是快：1. Shampoo的尽...", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11910", "label": "排序不等式及其推广", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11885", "label": "让炼丹更科学一些（十）：单调性假设...", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11882", "label": "让炼丹更科学一些（九）：经典自适应...", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11879", "label": "让炼丹更科学一些（八）：多阶段训练...", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11875", "label": "动量的新理解：逼近特征层面的梯度下降", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11864", "label": "流形上的最速下降：7. Stief...", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11854", "label": "除了交叉熵，LM Loss还有什么选择？", "type": "a" }, { "href": "https://spaces.ac.cn/archives/10699/comment-page-3", "label": "carl_lei : 苏神，之前我一直有一个直觉是：MoE通过不同系数加权得到的FF...", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11945/comment-page-1", "label": "Ali Hatamizadeh : Thank you for looking into this...", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11340/comment-page-1", "label": "苏剑林 : 对于线性层，取max能得到谱范数，然后可以推出Muon，Muo...", "type": "a" }, { "href": "https://spaces.ac.cn/archives/1328/comment-page-1", "label": "bookworm : 天下苦秦久矣，期待苏老师的教材！", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11848/comment-page-2", "label": "苏剑林 : 免训练估计够呛，从实用的角度看，其实也不应该相信任何免训练的可能", "type": "a" }, { "href": "https://spaces.ac.cn/admin/login.php", "label": "登录", "type": "a" }, { "href": "https://spaces.ac.cn/", "label": "科学空间|Scientific Spaces", "type": "a" }, { "href": "https://spaces.ac.cn/feed", "label": "欢迎订阅", "type": "a" }, { "href": "https://spaces.ac.cn/archives/119", "label": "个性邮箱", "type": "a" }, { "href": "https://spaces.ac.cn/archives/41", "label": "观测ISS", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403", "label": "Transformer升级之路：6、旋转位置编码的完备性分析", "type": "a" }, { "href": "https://spaces.ac.cn/archives/8265", "label": "《Transformer升级之路：2、博采众长的旋转式位置编码》", "type": "a" }, { "href": "https://spaces.ac.cn/archives/8397", "label": "《Transformer升级之路：4、二维位置的旋转式位置编码》", "type": "a" }, { "href": "https://en.wikipedia.org/wiki/Matrix_exponential", "label": "Matrix Exponential", "type": "a" }, { "href": "https://en.wikipedia.org/wiki/Baker%E2%80%93Campbell%E2%80%93Hausdorff_formula", "label": "Baker–Campbell–Hausdorff公式", "type": "a" }, { "href": "https://spaces.ac.cn/archives/6377", "label": "《恒等式 det(exp(A)) = exp(Tr(A)) 赏析》", "type": "a" }, { "href": "https://math.stackexchange.com/questions/2467531/why-can-any-orthogonal-matrix-be-written-as-o-ea", "label": "《Why can any orthogonal matrix be written as O=e^A》", "type": "a" }, { "href": "https://en.wikipedia.org/wiki/Skew-symmetric_matrix", "label": "Skew-symmetric matrix", "type": "a" }, { "href": "https://spaces.ac.cn/archives/线性Attention的探索：Attention必须有个Softmax吗？", "label": "线性Attention", "type": "a" }, { "href": "https://spaces.ac.cn/archives/6508", "label": "《科学空间FAQ》", "type": "a" }, { "href": "http://mail.qq.com/cgi-bin/qm_share?t=qm_mailme&email=tN7d1drY3drrx8H0xcWa19vZ", "label": "点击这里", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E7%9F%A9%E9%98%B5/", "label": "矩阵", "type": "a" }, { "href": "https://spaces.ac.cn/tag/attention/", "label": "attention", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E4%BD%8D%E7%BD%AE%E7%BC%96%E7%A0%81/", "label": "位置编码", "type": "a" }, { "href": "https://spaces.ac.cn/tag/rope/", "label": "rope", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9379", "label": "生成扩散模型漫谈（十五）：构建ODE的一般步骤（中）", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9405", "label": "智能家居之热水器零冷水技术原理浅析", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11848", "label": "简单谈谈K3的MoE和Attention", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11823", "label": "将Softmax Attention线性化为Gated DeltaNet", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11814", "label": "LogSumExp和Softmax的泰勒展开", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11787", "label": "矩阵函数近似中的暴力美学", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11767", "label": "矩阵参数的奇异值熵越高越好吗？", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11738", "label": "直接以FID为Loss：从梯度计算到流式训练", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11736", "label": "如何更科学地估计矩阵的谱范数？", "type": "a" }, { "href": "https://doraemonzzz.com/", "label": "Doraemonzzz", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20653", "label": "回复评论", "type": "a" }, { "href": "https://kexue.fm/", "label": "苏剑林", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20654", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20655", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20663", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20712", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20718", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20732", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20746", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20771", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20782", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20785", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=20816", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=22182", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=22183", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=22187", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=22207", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=28602", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=28614", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=29011", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=29020", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=29429", "label": "回复评论", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9403/comment-page-1?replyTo=29439", "label": "回复评论", "type": "a" }, { "href": "https://github.com/fxsjy/jieba", "label": "结巴分词", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E7%94%9F%E6%88%90%E6%A8%A1%E5%9E%8B/", "label": "生成模型", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E4%BC%98%E5%8C%96/", "label": "优化", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E6%A8%A1%E5%9E%8B/", "label": "模型", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B/", "label": "语言模型", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E6%A2%AF%E5%BA%A6/", "label": "梯度", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E4%BC%98%E5%8C%96%E5%99%A8/", "label": "优化器", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E6%A6%82%E7%8E%87/", "label": "概率", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E7%BD%91%E7%AB%99/", "label": "网站", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E8%BD%AC%E8%BD%BD/", "label": "转载", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E5%BE%AE%E5%88%86%E6%96%B9%E7%A8%8B/", "label": "微分方程", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E5%88%86%E6%9E%90/", "label": "分析", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E5%A4%A9%E8%B1%A1/", "label": "天象", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0/", "label": "深度学习", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E7%A7%AF%E5%88%86/", "label": "积分", "type": "a" }, { "href": "https://spaces.ac.cn/tag/python/", "label": "python", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E5%87%A0%E4%BD%95/", "label": "几何", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E6%89%A9%E6%95%A3/", "label": "扩散", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E5%8A%9B%E5%AD%A6/", "label": "力学", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E6%97%A0%E7%9B%91%E7%9D%A3/", "label": "无监督", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0/", "label": "损失函数", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E8%8A%82%E6%97%A5/", "label": "节日", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E7%BA%BF%E6%80%A7/", "label": "线性", "type": "a" }, { "href": "https://spaces.ac.cn/tag/%E7%94%9F%E6%B4%BB/", "label": "生活", "type": "a" }, { "href": "https://spaces.ac.cn/archives/1452", "label": "Lamost下的天文夏令营", "type": "a" }, { "href": "https://spaces.ac.cn/archives/471", "label": "大自然的隐身术——保护色", "type": "a" }, { "href": "https://spaces.ac.cn/archives/8578", "label": "概率视角下的线性模型：逻辑回归有解析解吗？", "type": "a" }, { "href": "https://spaces.ac.cn/archives/1257", "label": "施密特系统的校正镜方程求解", "type": "a" }, { "href": "https://spaces.ac.cn/archives/11206", "label": "基于树莓派Zero2W搭建一个随身旁路由", "type": "a" }, { "href": "https://spaces.ac.cn/archives/9660", "label": "梯度流：探索通向最小值之路", "type": "a" }, { "href": "https://spaces.ac.cn/archives/2071", "label": "数学基本技艺(A Mathematical Trivium)", "type": "a" }, { "href": "https://spaces.ac.cn/archives/8373", "label": "GlobalPointer：用统一的方式处理嵌套和非嵌套NER", "type": "a" }, { "href": "https://spaces.ac.cn/archives/1381", "label": "向量结合复数：常曲率曲线(1)", "type": "a" }, { "href": "https://spaces.ac.cn/archives/316", "label": "新年新天象：2010年1月重要天象", "type": "a" }, { "href": "https://papers.cool/", "label": "Cool Papers", "type": "a" }, { "href": "https://bbs.emath.ac.cn/", "label": "数学研发", "type": "a" }, { "href": "http://www.seatop.com.cn/", "label": "Seatop", "type": "a" }, { "href": "https://xiaoxia.org/", "label": "Xiaoxia", "type": "a" }, { "href": "https://kexue.fm/sci/integral/index.html", "label": "积分表-网络版", "type": "a" }, { "href": "http://blog.dvxj.com/", "label": "丝路博傲", "type": "a" }, { "href": "http://www.2math.cn/", "label": "数学之家", "type": "a" }, { "href": "http://interesting-sky.china-vo.org/", "label": "有趣天文奇观", "type": "a" }, { "href": "http://www.twistedwg.com/", "label": "TwistedW", "type": "a" }, { "href": "https://godweiyang.com/", "label": "godweiyang", "type": "a" }, { "href": "https://blog.ailemon.net/", "label": "AI柠檬", "type": "a" }, { "href": "https://greatdk.com/", "label": "王登科-DK博客", "type": "a" }, { "href": "https://blog.eson.org/", "label": "ESON", "type": "a" }, { "href": "https://fzhiy.net/", "label": "枫之羽", "type": "a" }, { "href": "https://coding-zuo.github.io/", "label": "coding-zuo", "type": "a" }, { "href": "https://www.bokeyuan.net/", "label": "博科园", "type": "a" }, { "href": "https://www.kppkkp.top/", "label": "孔皮皮的博客", "type": "a" }, { "href": "https://yunpengtai.top/", "label": "运鹏的博客", "type": "a" }, { "href": "https://jiming.site/", "label": "jiming.site", "type": "a" }, { "href": "https://www.omegaxyz.com/", "label": "OmegaXYZ", "type": "a" }, { "href": "https://www.robotech.ink/", "label": "EAI猩球", "type": "a" }, { "href": "https://liwenju0.com/", "label": "文举的博客", "type": "a" }, { "href": "https://kirigaya.cn/", "label": "汇尘轩", "type": "a" }, { "href": "https://kexue.fm/links.html", "label": "申请链接", "type": "a" }, { "href": "http://creativecommons.org/licenses/by-nc-nd/2.5/cn/", "label": "", "type": "a" }, { "href": "http://www.laogui.com/", "label": "laogui", "type": "a" }, { "href": "http://typecho.org/", "label": "Typecho", "type": "a" }, { "href": "https://beian.miit.gov.cn/", "label": "粤ICP备09093259号-1/2", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:53:04Z", "source_url": "https://spaces.ac.cn/archives/9403", "papers": [], "actual_video_files": [], "body_chars": 13118, "expected_body_images": 10, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/3d322729b230/pages/source.html", "local_html": "attachments/3d322729b230/pages/article.html", "body_base_path": "attachments/3d322729b230/pages/", "body_markdown_path": "attachments/3d322729b230/pages/article.md" }