{ "source_file": "0930", "source_platform": "研究机构", "author": "BAIR Blog (UC Berkeley)", "title_listed": "Identifying Interactions at Scale for LLMs", "keywords": "可解释性,LLM,特征交互", "domain": "大模型算法", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://bair.berkeley.edu/blog/2026/03/13/spex/", "publish_date": "2026-03", "corpus_stage": "预训练/SFT", "notes": "LLM大规模交互识别（SPEX）", "id": "51f852cc57af", "collected_at": "2026-10-02T15:23:13", "title": "Identifying Interactions at Scale for LLMs – The Berkeley Artificial Intelligence Research Blog", "body_markdown": "Understanding the behavior of complex machine learning systems, particularly Large Language Models (LLMs), is a critical challenge in modern artificial intelligence. Interpretability research aims to make the decision-making process more transparent to model builders and impacted humans, a step toward safer and more trustworthy AI. To gain a comprehensive understanding, we can analyze these systems through different lenses:\nfeature attribution\n, which isolates the specific input features driving a prediction (\nLundberg & Lee, 2017\n;\nRibeiro et al., 2022\n);\ndata attribution\n, which links model behaviors to influential training examples (\nKoh & Liang, 2017\n;\nIlyas et al., 2022\n); and\nmechanistic interpretability\n, which dissects the functions of internal components (\nConmy et al., 2023\n;\nSharkey et al., 2025\n).\nAcross these perspectives, the same fundamental hurdle persists:\ncomplexity at scale\n. Model behavior is rarely the result of isolated components; rather, it emerges from complex dependencies and patterns. To achieve state-of-the-art performance, models synthesize complex feature relationships, find shared patterns from diverse training examples, and process information through highly interconnected internal components.\nTherefore, grounded or reality-checked interpretability methods must also be able to capture these\ninfluential interactions\n. As the number of features, training data points, and model components grow, the number of potential interactions grows exponentially, making exhaustive analysis computationally infeasible. In this blog post, we describe the fundamental ideas behind\nSPEX\nand\nProxySPEX\n, algorithms capable of identifying these critical interactions at scale.\nAttribution through Ablation\nCentral to our approach is the concept of\nablation\n, measuring influence by observing what changes when a component is removed.\nFeature Attribution:\nWe mask or remove specific segments of the input prompt and measure the resulting shift in the predictions.\nData Attribution:\nWe train models on different subsets of the training set, assessing how the model’s output on a test point shifts in the absence of specific training data.\nModel Component Attribution (Mechanistic Interpretability):\nWe intervene on the model’s forward pass by removing the influence of specific internal components, determining which internal structures are responsible for the model’s prediction.\nIn each case, the goal is the same: to isolate the drivers of a decision by systematically perturbing the system, in hopes of discovering influential interactions. Since each ablation incurs a significant cost, whether through expensive inference calls or retrainings, we aim to compute attributions with the\nfewest possible ablations\n.\nMasking different parts of the input, we measure the difference between the original and ablated outputs.\nSPEX and ProxySPEX Framework\nTo discover influential interactions with a tractable number of ablations, we have developed\nSPEX\n(Spectral Explainer). This framework draws on signal processing and coding theory to advance interaction discovery to scales orders of magnitude greater than prior methods. SPEX circumvents this by exploiting a key structural observation: while the number of total interactions is prohibitively large, the number of\ninfluential\ninteractions is actually quite small.\nWe formalize this through two observations:\nsparsity\n(relatively few interactions truly drive the output) and\nlow-degreeness\n(influential interactions typically involve only a small subset of features). These properties allow us to reframe the difficult search problem into a solvable\nsparse recovery\nproblem. Drawing on powerful tools from signal processing and coding theory, SPEX uses strategically selected ablations to combine many candidate interactions together. Then, using efficient decoding algorithms, we disentangle these combined signals to isolate the specific interactions responsible for the model’s behavior.\nIn a subsequent algorithm,\nProxySPEX\n, we identified another structural property common in complex machine learning models:\nhierarchy\n. This means that where a higher-order interaction is important, its lower-order subsets are likely to be important as well. This additional structural observation yields a dramatic improvement in computational cost: it matches the performance of SPEX with around\n10x fewer ablations\n. Collectively, these frameworks enable efficient interaction discovery, unlocking new applications in feature, data, and model component attribution.\nFeature Attribution\nFeature attribution techniques assign importance scores to input features based on their influence on the model’s output. For example, if an LLM were used to make a medical diagnosis, this approach could identify exactly which symptoms led the model to its conclusion. While attributing importance to individual features can be valuable, the true power of sophisticated models lies in their ability to capture complex relationships between features. The figure below illustrates examples of these influential interactions: from a double negative changing sentiment (left) to the necessary synthesis of multiple documents in a RAG task (right).\nThe figure below illustrates the feature attribution performance of SPEX on a sentiment analysis task. We evaluate performance using\nfaithfulness\n: a measure of how accurately the recovered attributions can predict the model’s output on unseen test ablations. We find that SPEX matches the high faithfulness of existing interaction techniques (Faith-Shap, Faith-Banzhaf) on short inputs, but uniquely retains this performance as the context scales to thousands of features. In contrast, while marginal approaches (LIME, Banzhaf) can also operate at this scale, they exhibit significantly lower faithfulness because they fail to capture the complex interactions driving the model’s output.\nSPEX was also applied to a modified version of the trolley problem, where the moral ambiguity of the problem is removed, making “True” the clear correct answer. Given the modification below, GPT-4o mini answered correctly only 8% of the time. When we applied standard feature attribution (SHAP), it identified individual instances of the word\ntrolley\nas the primary factors driving the incorrect response. However, replacing\ntrolley\nwith synonyms such as\ntram\nor\nstreetcar\nhad little impact on the prediction of the model. SPEX revealed a much richer story, identifying a dominant high-order synergy between the two instances of\ntrolley\n, as well as the words\npulling\nand\nlever,\na finding that aligns with human intuition about the core components of the dilemma. When these four words were replaced with synonyms, the model’s failure rate dropped to near zero.\nData Attribution\nData attribution identifies which training data points are most responsible for a model’s prediction on a new test point. Identifying influential interactions between these data points is key to explaining unexpected model behaviors. Redundant interactions, such as semantic duplicates, often reinforce specific (and possibly incorrect) concepts, while synergistic interactions are essential for defining decision boundaries that no single sample could form alone. To demonstrate this, we applied ProxySPEX to a ResNet model trained on CIFAR-10, identifying the most significant examples of both interaction types for a variety of difficult test points, as shown in the figure below.\nAs illustrated,\nsynergistic interactions\n(left) often involve semantically distinct classes working together to define a decision boundary. For example, grounding the synergy in human perception, the\nautomobile\n(bottom left) shares visual traits with the provided training images, including the low-profile chassis of the sports car, the boxy shape of the yellow truck, and the horizontal stripe of the red delivery vehicle. On the other hand,\nredundant interactions\n(right) tend to capture visual duplicates that reinforce a specific concept. For instance, the\nhorse\nprediction (middle right) is heavily influenced by a cluster of dog images with similar silhouettes. This fine-grained analysis allows for the development of new data selection techniques that preserve necessary synergies while safely removing redundancies.\nAttention Head Attribution (Mechanistic Interpretability)\nThe goal of\nmodel component attribution\nis to identify which internal parts of the model, such as specific layers or attention heads, are most responsible for a particular behavior. Here too, ProxySPEX uncovers the responsible interactions between different parts of the architecture. Understanding these structural dependencies is vital for architectural interventions, such as task-specific attention head pruning. On an MMLU dataset (highschool‐us‐history), we demonstrate that a ProxySPEX-informed pruning strategy not only outperforms competing methods, but can actually\nimprove model performance on the target task\n.\nOn this task, we also analyzed the interaction structure across the model’s depth. We observe that early layers function in a predominantly linear regime, where heads contribute largely independently to the target task. In later layers, the role of interactions between attention heads becomes more pronounced, with most of the contribution coming from interactions among heads in the same layer.\nWhat’s Next?\nThe SPEX framework represents a significant step forward for interpretability, extending interaction discovery from\ndozens to thousands of components\n. We have demonstrated the versatility of the framework across the entire model lifecycle: exploring feature attribution on long-context inputs, identifying synergies and redundancies among training data points, and discovering interactions between internal model components. Moving forwards, many interesting research questions remain around\nunifying\nthese different perspectives, providing a more holistic understanding of a machine learning system. It is also of great interest to systematically evaluate interaction discovery methods against existing scientific knowledge in fields such as genomics and materials science, serving to both ground model findings and generate new, testable hypotheses.\nWe invite the research community to join us in this effort: the code for both SPEX and ProxySPEX is fully integrated and available within the popular SHAP-IQ repository.\nhttps://github.com/mmschlk/shapiq\n(SHAP-IQ Github)\nhttps://openreview.net/forum?id=KI8qan2EA7\n(ProxySPEX NeurIPS 2025)\nhttps://openreview.net/forum?id=pRlKbAwczl\n(SPEX ICML 2025)\nhttps://openreview.net/forum?id=glGeXu1zG4\n(Learning to Understand NeurIPS 2024)", "metadata": { "title": "Identifying Interactions at Scale for LLMs – The Berkeley Artificial Intelligence Research Blog", "description": "The BAIR Blog", "keywords": "", "author": "C.K. Wolfe", "og:locale": "en_US", "og:type": "article", "og:title": "Identifying Interactions at Scale for LLMs", "og:description": "The BAIR Blog", "og:url": "http://bair.berkeley.edu/blog/2026/03/13/spex/", "og:site_name": "The Berkeley Artificial Intelligence Research Blog", "og:image": "https://bair.berkeley.edu/static/blog/spex/teaser.png", "og:image:url": "https://bair.berkeley.edu/static/blog/spex/teaser.png" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [], "images": [ { "status": "ok", "url": "https://bair.berkeley.edu/static/blog/spex/image1.png", "bytes": 669249, "reused": true, "local_path": "attachments/51f852cc57af/images/002.png", "src": "https://bair.berkeley.edu/static/blog/spex/image1.png", "alt": "different_tests", "downloaded": true }, { "status": "ok", "url": "https://bair.berkeley.edu/static/blog/spex/image2.png", "bytes": 678738, "reused": true, "local_path": "attachments/51f852cc57af/images/003.png", "src": "https://bair.berkeley.edu/static/blog/spex/image2.png", "alt": "image2", "downloaded": true }, { "status": "ok", "url": "https://bair.berkeley.edu/static/blog/spex/image3.png", "bytes": 785189, "reused": true, "local_path": "attachments/51f852cc57af/images/004.png", "src": "https://bair.berkeley.edu/static/blog/spex/image3.png", "alt": "image3", "downloaded": true }, { "status": "ok", "url": "https://bair.berkeley.edu/static/blog/spex/image4.png", "bytes": 301388, "reused": true, "local_path": "attachments/51f852cc57af/images/005.png", "src": "https://bair.berkeley.edu/static/blog/spex/image4.png", "alt": "image4", "downloaded": true }, { "status": "ok", "url": "https://bair.berkeley.edu/static/blog/spex/image5.png", "bytes": 377344, "reused": true, "local_path": "attachments/51f852cc57af/images/006.png", "src": "https://bair.berkeley.edu/static/blog/spex/image5.png", "alt": "image5", "downloaded": true }, { "status": "ok", "url": "https://bair.berkeley.edu/static/blog/spex/image6.png", "bytes": 322572, "reused": true, "local_path": "attachments/51f852cc57af/images/007.png", "src": "https://bair.berkeley.edu/static/blog/spex/image6.png", "alt": "image6", "downloaded": true }, { "status": "ok", "url": "https://bair.berkeley.edu/static/blog/spex/image7.png", "bytes": 173680, "reused": true, "local_path": "attachments/51f852cc57af/images/008.png", "src": "https://bair.berkeley.edu/static/blog/spex/image7.png", "alt": "image7", "downloaded": true }, { "status": "ok", "url": "https://bair.berkeley.edu/static/blog/spex/image8.png", "bytes": 102634, "reused": true, "local_path": "attachments/51f852cc57af/images/009.png", "src": "https://bair.berkeley.edu/static/blog/spex/image8.png", "alt": "image8", "downloaded": true } ], "audio": [], "videos": [], "links": [ { "href": "https://proceedings.neurips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html", "label": "Lundberg & Lee, 2017", "type": "a" }, { "href": "https://dl.acm.org/doi/abs/10.1145/2939672.2939778", "label": "Ribeiro et al., 2022", "type": "a" }, { "href": "https://proceedings.mlr.press/v70/koh17a/koh17a.pdf", "label": "Koh & Liang, 2017", "type": "a" }, { "href": "https://proceedings.mlr.press/v162/ilyas22a/ilyas22a.pdf", "label": "Ilyas et al., 2022", "type": "a" }, { "href": "https://papers.nips.cc/paper_files/paper/2023/hash/34e1dbe95d34d7ebaf99b9bcaeb5b2be-Abstract-Conference.html", "label": "Conmy et al., 2023", "type": "a" }, { "href": "https://openreview.net/forum?id=91H76m9Z94", "label": "Sharkey et al., 2025", "type": "a" }, { "href": "https://openreview.net/forum?id=pRlKbAwczl", "label": "SPEX", "type": "a" }, { "href": "https://openreview.net/forum?id=KI8qan2EA7", "label": "ProxySPEX", "type": "a" }, { "href": "https://github.com/mmschlk/shapiq", "label": "https://github.com/mmschlk/shapiq", "type": "a" }, { "href": "https://openreview.net/forum?id=glGeXu1zG4", "label": "https://openreview.net/forum?id=glGeXu1zG4", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:53:19Z", "source_url": "https://bair.berkeley.edu/blog/2026/03/13/spex/", "papers": [ { "status": "ok", "url": "https://proceedings.mlr.press/v70/koh17a/koh17a.pdf", "bytes": 4815350, "reused": false, "local_path": "papers/repair/1b3a5596191b2681f0ea.pdf" }, { "status": "ok", "url": "https://proceedings.mlr.press/v162/ilyas22a/ilyas22a.pdf", "bytes": 20174872, "reused": false, "local_path": "papers/repair/4422c38302e6b2d0d463.pdf" } ], "actual_video_files": [], "body_chars": 10661, "expected_body_images": 8, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/51f852cc57af/pages/source.html", "local_html": "attachments/51f852cc57af/pages/article.html", "body_base_path": "attachments/51f852cc57af/pages/", "body_markdown_path": "attachments/51f852cc57af/pages/article.md" }