MAttr (Matryoshka Attribution)

MAttr (Matryoshka Attribution) traces a language model's output back to the internal components responsible for it. Aryaman Arora, Kirill Acharya, Nathan Hu, Yanzhe Zhang, Noah Goodman, Dan Jurafsky, and Christopher Potts introduced it in "Matryoshka Attribution: Learning to Attribute Language Model Outputs to Representations and Weights" (arXiv:2609.25518, September 2026). Earlier attribution methods rely on causal interventions, which scale poorly, or on gradients and learnable masks, which often miss the components actually responsible for a behavior. The paper reframes attribution as finding nested subsets of a model's internal components that minimize a downstream loss: the "matryoshka" of the name.

MAttr learns this nested structure with a mask parametrized by a differentiable sigmoid top-k operator, without an explicit sparsity penalty or straight-through gradient tricks. Training randomizes k across steps, so one run supervises every sparsity level at once and produces a full ranking of components by attribution score rather than a single fixed circuit.

MAttr topped the official leaderboard of the Mechanistic Interpretability Benchmark (Mueller et al., 2025) on release. Its authors report that the rankings it learns transfer to related, unseen tasks. The same approach extends to weights: trained with reinforcement learning using a judge score as reward, MAttr traced refusal behavior in Llama 3.1 8B Instruct to about 1% of the parameter difference from its base model. Resetting that fraction removed refusals while leaving other capabilities intact. Code: github.com/aryamanarora/matryoshka-attribution.