---
title: "Towards interpreting any residual stream direction by scaling activation inversion"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.08465v1"
permalink: "https://feed.astralor.com/items/itm_a22807cb7aedce71fb4e9143"
published: "2026-11-05T18:54:40.000Z"
score: 59
channel: "ai"
tags: ["Towards","But","Maximally-Activating Example Metamodels","MAEMs","Our","MAEM"]
---

# Towards interpreting any residual stream direction by scaling activation inversion

## 速览

To interpret a direction in a language model's residual stream, we can look for text that strongly activates it. But when such text is found by scanning a corpus, it is limited to what that corpus contains. We develop M…

## 关键点

- To interpret a direction in a language model's residual stream, we can look for text that strongly activates it.
- But when such text is found by scanning a corpus, it is limited to what that corpus contains.
- We develop Maximally-Activating Example Metamodels (MAEMs), language models which map residual stream directions from a subject model to text exemplars that strongly activate those directions.

## 全文

To interpret a direction in a language model's residual stream, we can look for text that strongly activates it. But when such text is found by scanning a corpus, it is limited to what that corpus contains. We develop Maximally-Activating Example Metamodels (MAEMs), language models which map residual stream directions from a subject model to text exemplars that strongly activate those directions. Our objective is to maximise the cosine similarity of a supplied residual stream direction with the activations of the text generated by the MAEM. This label-free RL objective helps to scale MAEMs on diverse sources of residual stream directions. Empirically, we find the resultant metamodel produces informative exemplars: a Qwen3.6-27B MAEM inverts held-out activations better than corpus search or a natural-language autoencoder (NLA), beats corpus search (at 10M tokens) on a majority of held-out languages/domains, and activates 89% of held-out SAE features. It also identifies concepts behind steering vectors about as well as NLAs, recovers the payloads of LoRA backdoors more often than corpus search, and can even verbalise some model workspace inner monologue. Unlike explanations produced by other metamodels (e.g. NLAs), MAEM exemplars can be verified with a forward pass on the subject model, making the MAEM objective a grounded alternative for scaling the training of metamodels.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_a22807cb7aedce71fb4e9143