{
  "source_file": "0930",
  "source_platform": "个人博客",
  "author": "Christopher Olah / colah's blog",
  "title_listed": "Transformer Circuits",
  "keywords": "Transformer,电路,可解释性",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://transformer-circuits.pub/",
  "publish_date": "2021-12",
  "corpus_stage": "预训练/SFT",
  "notes": "Transformer电路可解释性研究",
  "id": "7ff6253fd9fa",
  "collected_at": "2026-10-02T14:28:38",
  "title": "Transformer Circuits Thread",
  "body_markdown": "Transformer Circuits Thread\nAnthropic’s\nInterpretability Research\nA surprising fact about modern large language models is that nobody really knows how they work\n            internally.\n            The Interpretability team strives to change that — to understand these models to better plan for a\n            future of\n            safe AI.\nAugust 2026\nCharacterizing interference weights in a tiny language model\nWe identify interference weights in a 1-layer transformer by measuring their effect on model outputs and loss.\nJuly 2026\nVerbalizable Representations Form a Global Workspace in Language Models\nGurnee et al., 2026\nWe find that Claude maintains a small, privileged set of representations it can report on, control, and reason with, atop a much larger volume of automatic processing.\nJune 2026\nCircuits Updates — June 2026\nA short update on turn-averaged sparse autoencoders.\nMay 2026\nCircuits Updates — May 2026\nA short update on understanding features through downstream connections.\nNatural Language Autoencoders Produce Unsupervised Explanations of LLM Activations\nFraser-Taliente, Kantamneni, Ong, et al., 2026\nWe train Claude to translate its internal state into natural language.\nHeadVis\nWe develop an interactive visualization tool to help us understand the behaviors of attention heads in language models.\nApril 2026\nEmotion Concepts and their Function in a Large Language Model\nSofroniew et al., 2026\nWe find representations of emotion concepts in Claude Sonnet 4.5 and show that they causally influence its outputs.\nDecember 2025\nCircuits Cross-Post — Activation Oracles\nWe train language models to answer questions about their own activations in natural language.\nNovember 2025\nCircuits Updates — November 2025\nA short update on harm pressure.\nOctober 2025\nEmergent Introspective Awareness in Large Language Models\nLindsey, 2025\nWe find evidence that language models can introspect on their internal states.\nCircuits Updates — October 2025\nSmall updates on visual features and dictionary initialization.\nWhen Models Manipulate Manifolds: The Geometry of a Counting Task\nGurnee et al., 2025\nWe find geometric structure underlying the mechanisms of a fundamental language model behavior.\nSeptember 2025\nCircuits Updates — September 2025\nA small update on features and in-context learning.\nAugust 2025\nCircuits Updates — August 2025\nA small update: How does a persona modify the assistant’s response?\nJuly 2025\nA Toy Model of Mechanistic (Un)Faithfulness\nWhen transcoders go awry.\nTracing Attention Computation Through Feature Interactions\nKamath et al., 2025\nWe describe and apply a method to explain attention patterns in terms of\n                    feature interactions, and integrate this information into attribution graphs.\nA Toy Model of Interference Weights\nUnpacking \"interference weights\" in some more depth.\nSparse mixtures of linear transforms\nWe investigate sparse mixture of linear transforms (MOLT), a new approach to transcoders.\nCircuits Updates — July 2025\nA collection of small updates: revisiting A Mathematical Framework and applications of\n                    interpretability to biology.\nAutomated Auditing\nA note on using agents to perform automated alignment audits, including using interpretability\n                    tools.\nApril 2025\nCircuits Updates — April 2025\nA collection of small updates: jailbreaks, dense features, and spinning up on interpretability.\nProgress on Attention\nAn update on our progress studying attention.\nMarch 2025\nOn the Biology of a Large Language Model\nLindsey et al., 2025\nWe investigate the internal mechanisms used by Claude 3.5 Haiku — Anthropic's lightweight\n                    production model — in a variety of contexts.\nCircuit Tracing: Revealing Computational Graphs in Language Models\nAmeisen et al., 2025\nWe describe an approach to tracing the \"step-by-step\" computation involved when a model responds\n                    to a single prompt.\nFebruary 2025\nInsights on Crosscoder Model Diffing\nA preliminary note on using crosscoders to diff models.\nJanuary 2025\nCircuits Updates — January 2025\nA collection of small updates: dictionary learning optimization techniques.\nDecember 2024\nStage-Wise Model Diffing\nA preliminary note on model diffing through dictionary fine-tuning.\nOctober 2024\nSparse Crosscoders for Cross-Layer Features and Model Diffing\nA preliminary note on a way to get consistent features across layers, and even models.\nUsing Dictionary Learning Features as Classifiers\nA preliminary note comparing feature-based and raw-activation based harmfulness classifiers.\nSeptember 2024\nCircuits Updates — September 2024\nA collection of small updates: investigating successor heads, oversampling data in SAEs.\nAugust 2024\nCircuits Updates — August 2024\nA collection of small updates: interpretability evals, reproducing self-explanation.\nJuly 2024\nCircuits Updates — July 2024\nA collection of small updates: five hurdles, linear representations, dark matter, pivot tables,\n                    feature sensitivity.\nJune 2024\nCircuits Updates — June 2024\nA collection of small updates: topk and gated SAE investigation.\nMay 2024\nScaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet\nTempleton et al., 2024\nUsing a sparse autoencoder, we extract a large number of interpretable features from Claude 3\n                    Sonnet. Some appear to be safety-relevant.\nApril 2024\nCircuits Updates — April 2024\nA collection of small updates from the Anthropic Interpretability Team.\nMarch 2024\nCircuits Updates — March 2024\nA collection of small updates from the Anthropic Interpretability Team.\nReflections on Qualitative Research\nSome opinionated thoughts on why interpretability research may have\n                    qualitative aspects be more central than we're used to in other fields.\nFebruary 2024\nCircuits Updates — February 2024\nA collection of small updates from the Anthropic Interpretability Team.\nJanuary 2024\nCircuits Updates — January 2024\nA collection of small updates from the Anthropic Interpretability Team.\nOctober 2023\nTowards Monosemanticity: Decomposing Language Models With Dictionary Learning\nBricken et al., 2023\nUsing a sparse autoencoder, we extract a large number of interpretable features from a one-layer\n                    transformer.\nJuly 2023\nCircuits Updates — July 2023\nA collection of small updates from the Anthropic Interpretability Team.\nMay 2023\nCircuits Updates — May 2023\nA collection of small updates from the Anthropic Interpretability Team.\nInterpretability Dreams\nOur present research aims to create a foundation for mechanistic\n                    interpretability research. In doing so, it's important to keep sight of what we're trying to lay\n                    the foundations for.\nDistributed Representations: Composition & Superposition\nAn informal note on how \"distributed representations\" might be understood\n                    as two different, competing strategies — \"composition\" and \"superposition\" — with quite\n                    different properties.\nMarch 2023\nPrivileged Bases in the Transformer Residual Stream\nOur mathematical theories of the Transformer architecture suggest that\n                    individual coordinates in the residual stream should have no special\n                    significance, but recent work has shown that this observation is false in practice.\n                    We investigate this phenomenon and provisionally conclude that the per-dimension normalizers in\n                    the Adam optimizer are to blame for the effect.\nJanuary 2023\nSuperposition, Memorization, and Double Descent\nHenighan et al., 2023\nWe have little mechanistic understanding of how deep learning\n                    models overfit to their training data, despite it being a\n                    central problem. Here we extend our previous work on toy models\n                    to shed light on how models generalize beyond their training\n                    data.\nSeptember 2022\nToy Models of Superposition\nElhage et al., 2022\nNeural networks often seem to pack many unrelated concepts into a single\n                    neuron - a puzzling phenomenon known as 'polysemanticity'. In our latest interpretability work,\n                    we build toy models where the origins and dynamics of polysemanticity can be fully understood.\nJune 2022\nSoftmax Linear Units\nAn alternative activation function increases the fraction of neurons which\n                    appear to correspond to human-understandable concepts.\nMechanistic Interpretability, Variables, and the Importance of Interpretable Bases\nAn informal note on intuitions related to mechanistic interpretability.\nMarch 2022\nIn-Context Learning and Induction Heads\nOlsson et al., 2022\nAn exploration of the hypothesis that induction heads are the primary\n                    mechanism behind in-context learning. We also report the existence of a previously unknown phase\n                    change in transformers language models.\npaper\nDecember 2021\nA Mathematical Framework for Transformer Circuits\nElhage et al., 2021\nOur early mathematical framework for reverse engineering models,\n                    demonstrated by reverse engineering small toy models.\npaper\nExercises\nSome exercises we've developed to improve our understanding of how neural\n                    networks implement algorithms at the parameter level.\nnote, exercises\nVideos\nVery rough informal talks as we search for a way to reverse engineering\n                    transformers.\nlinks, videos\nPySvelte\nOne approach to bridging Python and web-based interactive diagrams for\n                    interpretability research.\ngithub link, infrastructure\nGarcon\nA description of our tooling for doing interpretability on large models.\nnote, infrastructure\nMarch 2020 - April 2021\nOriginal Distill Circuits Thread\nOur exploration of Transformers builds heavily on the original Circuits\n                    thread on Distill.\nAbout\nCan we reverse engineer transformer language models into human-understandable computer programs?\n                Inspired by the\nDistill Circuits Thread\n, we’re\n                going to try.\nWe think interpretability research benefits a lot from interactive articles (see\nActivation Atlases\nfor a striking\n                example).\n                Previously we would have submitted to Distill, but with\nDistill on Hiatus\n,\n                we're taking a page from David Ha’s approach of simply creating websites (eg.\nWorld Models\n) for research projects.\nAs part of our effort to reverse engineer transformers, we’ve created several other resources\n                besides our paper which we hope will be useful.\n                We’ve collected them on this website, and may add future content here, or even collaborations with\n                other institutions.\nSubscribe\nRSS feed",
  "metadata": {
    "title": "Transformer Circuits Thread",
    "description": null,
    "keywords": null,
    "author": null,
    "og:title": "Transformer Circuits Thread",
    "og:description": "Can we reverse engineer transformer language models into human-understandable computer programs?",
    "og:image": "https://transformer-circuits.pub/interp.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/workspace.png",
      "bytes": 84631,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-21e1644b6663f7600562.png",
      "src": "https://transformer-circuits.pub/images/workspace.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/nla.png",
      "bytes": 130879,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-79ed083106717ff88960.png",
      "src": "https://transformer-circuits.pub/images/nla.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/emotions.png",
      "bytes": 46412,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-08985c8951650effea42.png",
      "src": "https://transformer-circuits.pub/images/emotions.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/introspection.png",
      "bytes": 118184,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-2aa0f6eabc7b57225925.png",
      "src": "https://transformer-circuits.pub/images/introspection.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/linebreaks.png",
      "bytes": 167212,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-4799fc8db3b3bf293858.png",
      "src": "https://transformer-circuits.pub/images/linebreaks.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/attention-qk.png",
      "bytes": 36558,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-68b2e1aa9c1c65e7dd84.png",
      "src": "https://transformer-circuits.pub/images/attention-qk.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/biology.png",
      "bytes": 23403,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-9b529355d423f394e787.png",
      "src": "https://transformer-circuits.pub/images/biology.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/methods.png",
      "bytes": 26395,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-871a39d5f46747b74be1.png",
      "src": "https://transformer-circuits.pub/images/methods.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/scaling.png",
      "bytes": 232990,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-07b9c4ccb4332e04f56b.png",
      "src": "https://transformer-circuits.pub/images/scaling.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/towards.png",
      "bytes": 220056,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-9ab16021f123b5a6870a.png",
      "src": "https://transformer-circuits.pub/images/towards.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/memorization.png",
      "bytes": 116451,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-df0b12c2e1ad3946d2d1.png",
      "src": "https://transformer-circuits.pub/images/memorization.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/superposition.png",
      "bytes": 17753,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-89448cae37c8ab9cef06.png",
      "src": "https://transformer-circuits.pub/images/superposition.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/induction.png",
      "bytes": 19235,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-de2548ca408e816f7d1c.png",
      "src": "https://transformer-circuits.pub/images/induction.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/framework.png",
      "bytes": 46786,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-2eb44f33f6519386bc07.png",
      "src": "https://transformer-circuits.pub/images/framework.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://transformer-circuits.pub/images/circuits.png",
      "bytes": 341319,
      "reused": false,
      "local_path": "attachments/7ff6253fd9fa/images/body-1789929d99bf8fa9b158.png",
      "src": "https://transformer-circuits.pub/images/circuits.png",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://anthropic.com/",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/",
      "label": "Transformer Circuits Thread",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2026/interference_effectiveness_helpfulness/index.html",
      "label": "Characterizing interference weights in a tiny language model We identify interference weights in a 1-layer transformer b",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2026/workspace/index.html",
      "label": "Verbalizable Representations Form a Global Workspace in Language Models Gurnee et al., 2026 We find that Claude maintain",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2026/june-update/index.html",
      "label": "Circuits Updates — June 2026 A short update on turn-averaged sparse autoencoders.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2026/may-update/index.html",
      "label": "Circuits Updates — May 2026 A short update on understanding features through downstream connections.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2026/nla/index.html",
      "label": "Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations Fraser-Taliente, Kantamneni, Ong, et ",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2026/headvis/index.html",
      "label": "HeadVis We develop an interactive visualization tool to help us understand the behaviors of attention heads in language ",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2026/emotions/index.html",
      "label": "Emotion Concepts and their Function in a Large Language Model Sofroniew et al., 2026 We find representations of emotion ",
      "type": "a"
    },
    {
      "href": "https://alignment.anthropic.com/2025/activation-oracles/",
      "label": "Circuits Cross-Post — Activation Oracles We train language models to answer questions about their own activations in nat",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/november-update/index.html",
      "label": "Circuits Updates — November 2025 A short update on harm pressure.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/introspection/index.html",
      "label": "Emergent Introspective Awareness in Large Language Models Lindsey, 2025 We find evidence that language models can intros",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/october-update/index.html",
      "label": "Circuits Updates — October 2025 Small updates on visual features and dictionary initialization.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/linebreaks/index.html",
      "label": "When Models Manipulate Manifolds: The Geometry of a Counting Task Gurnee et al., 2025 We find geometric structure underl",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/september-update/index.html",
      "label": "Circuits Updates — September 2025 A small update on features and in-context learning.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/august-update/index.html",
      "label": "Circuits Updates — August 2025 A small update: How does a persona modify the assistant’s response?",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/faithfulness-toy-model/index.html",
      "label": "A Toy Model of Mechanistic (Un)Faithfulness When transcoders go awry.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/attention-qk/index.html",
      "label": "Tracing Attention Computation Through Feature Interactions Kamath et al., 2025 We describe and apply a method to explain",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/interference-weights/index.html",
      "label": "A Toy Model of Interference Weights Unpacking \"interference weights\" in some more depth.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/bulk-update/index.html",
      "label": "Sparse mixtures of linear transforms We investigate sparse mixture of linear transforms (MOLT), a new approach to transc",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/july-update/index.html",
      "label": "Circuits Updates — July 2025 A collection of small updates: revisiting A Mathematical Framework and applications of\n    ",
      "type": "a"
    },
    {
      "href": "https://alignment.anthropic.com/2025/automated-auditing/",
      "label": "Automated Auditing A note on using agents to perform automated alignment audits, including using interpretability\n      ",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/april-update/index.html",
      "label": "Circuits Updates — April 2025 A collection of small updates: jailbreaks, dense features, and spinning up on interpretabi",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/attention-update/index.html",
      "label": "Progress on Attention An update on our progress studying attention.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/attribution-graphs/biology.html",
      "label": "On the Biology of a Large Language Model Lindsey et al., 2025 We investigate the internal mechanisms used by Claude 3.5 ",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/attribution-graphs/methods.html",
      "label": "Circuit Tracing: Revealing Computational Graphs in Language Models Ameisen et al., 2025 We describe an approach to traci",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/crosscoder-diffing-update/index.html",
      "label": "Insights on Crosscoder Model Diffing A preliminary note on using crosscoders to diff models.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2025/january-update/index.html",
      "label": "Circuits Updates — January 2025 A collection of small updates: dictionary learning optimization techniques.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/model-diffing/index.html",
      "label": "Stage-Wise Model Diffing A preliminary note on model diffing through dictionary fine-tuning.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/crosscoders/index.html",
      "label": "Sparse Crosscoders for Cross-Layer Features and Model Diffing A preliminary note on a way to get consistent features acr",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/features-as-classifiers/index.html",
      "label": "Using Dictionary Learning Features as Classifiers A preliminary note comparing feature-based and raw-activation based ha",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/september-update/index.html",
      "label": "Circuits Updates — September 2024 A collection of small updates: investigating successor heads, oversampling data in SAE",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/august-update/index.html",
      "label": "Circuits Updates — August 2024 A collection of small updates: interpretability evals, reproducing self-explanation.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/july-update/index.html",
      "label": "Circuits Updates — July 2024 A collection of small updates: five hurdles, linear representations, dark matter, pivot tab",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/june-update/index.html",
      "label": "Circuits Updates — June 2024 A collection of small updates: topk and gated SAE investigation.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html",
      "label": "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet Templeton et al., 2024 Using a sparse au",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/april-update/index.html",
      "label": "Circuits Updates — April 2024 A collection of small updates from the Anthropic Interpretability Team.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/march-update/index.html",
      "label": "Circuits Updates — March 2024 A collection of small updates from the Anthropic Interpretability Team.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/qualitative-essay/index.html",
      "label": "Reflections on Qualitative Research Some opinionated thoughts on why interpretability research may have\n                ",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/feb-update/index.html",
      "label": "Circuits Updates — February 2024 A collection of small updates from the Anthropic Interpretability Team.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2024/jan-update/index.html",
      "label": "Circuits Updates — January 2024 A collection of small updates from the Anthropic Interpretability Team.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2023/monosemantic-features/index.html",
      "label": "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning Bricken et al., 2023 Using a sparse autoen",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2023/july-update/index.html",
      "label": "Circuits Updates — July 2023 A collection of small updates from the Anthropic Interpretability Team.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2023/may-update/index.html",
      "label": "Circuits Updates — May 2023 A collection of small updates from the Anthropic Interpretability Team.",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2023/interpretability-dreams/index.html",
      "label": "Interpretability Dreams Our present research aims to create a foundation for mechanistic\n                    interpretab",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2023/superposition-composition/index.html",
      "label": "Distributed Representations: Composition & Superposition An informal note on how \"distributed representations\" might be ",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2023/privileged-basis/index.html",
      "label": "Privileged Bases in the Transformer Residual Stream Our mathematical theories of the Transformer architecture suggest th",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2023/toy-double-descent/index.html",
      "label": "Superposition, Memorization, and Double Descent Henighan et al., 2023 We have little mechanistic understanding of how de",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2022/toy_model/index.html",
      "label": "Toy Models of Superposition Elhage et al., 2022 Neural networks often seem to pack many unrelated concepts into a single",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2022/solu/index.html",
      "label": "Softmax Linear Units An alternative activation function increases the fraction of neurons which\n                    appe",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2022/mech-interp-essay/index.html",
      "label": "Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases An informal note on intuitions relate",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html",
      "label": "In-Context Learning and Induction Heads Olsson et al., 2022 An exploration of the hypothesis that induction heads are th",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2021/framework/index.html",
      "label": "A Mathematical Framework for Transformer Circuits Elhage et al., 2021 Our early mathematical framework for reverse engin",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2021/exercises/index.html",
      "label": "Exercises Some exercises we've developed to improve our understanding of how neural\n                    networks impleme",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2021/videos/index.html",
      "label": "Videos Very rough informal talks as we search for a way to reverse engineering\n                    transformers. links, ",
      "type": "a"
    },
    {
      "href": "https://github.com/anthropics/PySvelte",
      "label": "PySvelte One approach to bridging Python and web-based interactive diagrams for\n                    interpretability res",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/2021/garcon/index.html",
      "label": "Garcon A description of our tooling for doing interpretability on large models. note, infrastructure",
      "type": "a"
    },
    {
      "href": "https://distill.pub/2020/circuits/",
      "label": "Original Distill Circuits Thread Our exploration of Transformers builds heavily on the original Circuits\n               ",
      "type": "a"
    },
    {
      "href": "https://distill.pub/2019/activation-atlas/",
      "label": "Activation Atlases",
      "type": "a"
    },
    {
      "href": "https://distill.pub/2021/distill-hiatus/",
      "label": "Distill on Hiatus",
      "type": "a"
    },
    {
      "href": "https://worldmodels.github.io/",
      "label": "World Models",
      "type": "a"
    },
    {
      "href": "https://transformer-circuits.pub/feed.xml",
      "label": "RSS feed",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:52:39Z",
  "source_url": "https://transformer-circuits.pub/",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 10827,
  "expected_body_images": 15,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/7ff6253fd9fa/pages/source.html",
  "local_html": "attachments/7ff6253fd9fa/pages/article.html",
  "body_base_path": "attachments/7ff6253fd9fa/pages/",
  "body_markdown_path": "attachments/7ff6253fd9fa/pages/article.md"
}