---
title: "训练模型而非读者：面向可验证激活解释的可解码性监督"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.20379v1"
permalink: "https://feed.astralor.com/items/itm_f136ac3711f18fe255c10a62"
published: "2026-07-22T17:10:23.000Z"
score: 79
channel: "ai"
tags: ["AI安全","Pythia-160M","Qwen-2.5-7B","ReCap","可解释性"]
---

# 训练模型而非读者：面向可验证激活解释的可解码性监督

## AI 点评

> 这篇论文值得所有关注AI可解释性和安全的从业者阅读。它直击当前最流行的解释评估方法的软肋——重构分数并不保证单个陈述的真实性，并提供了一个实用的解决方案RECAP。对于研究模型内省和红队实践的人来说，RECAP带来的独立可验证性是一大进步。

## 速览

这篇论文指出，自然语言自编码器通过重构分数评估隐藏层激活的解释，但重构分数对个别虚假陈述不敏感，导致解释可能不忠实地通过测试。作者提出了两种审计协议和RECAP方法：在目标模型旁训练线性头，使指定内容保持可解码。实验表明，RECAP显著提高了可验证性，并能在对抗攻击下保持谎言检测能力。

## 关键点

- 重构测试对个别虚假陈述不敏感，解释可能不忠实
- 提出RECAP方法，训练辅助线性头使指定内容可解码
- RECAP在小型模型和Pythia-160M上验证了有效性
- 对抗攻击下，RECAP探针仍能检测谎言（AUC 0.95）

## 全文

Natural-language autoencoders score explanations of hidden activations by reconstruction: an explanation is deemed faithful if the activation can be regenerated from it. The test is structurally insensitive to individual false claims: if flipping a claim does not change the reconstruction, the claim is never penalized. We show the test is passed in two ways, neither faithful. On a released Qwen-2.5-7B verbalizer, explanations reconstruct well above chance while ~2% of specific claims are reconstruction-dependent, so the score tracks gist, not specific facts. Under exact synthetic ground truth, the standard recipe develops co-adapted private codes (false wording the reconstruction depends on) in 5/5 runs, and fixes that leave the target model unchanged do not help. We contribute two audit protocols, the grounded-vs-true cross and the evaluator swap, and RECAP (Readable Encodings via Co-trained Auxiliary Predictors): linear heads trained alongside the target model to keep designated content decodable. On RECAP-trained sandbox models, fresh verbalizers state the designated content truly and the codes vanish, at a +0.001-nat cost. This replicates on a pretrained Pythia-160M: the content becomes reliably probe-decodable, though a fresh verbalizer conveys it only in part (truth 0.44-0.46 vs a near-zero control). For interpretability, high reconstruction does not certify individual claims. For AI safety, RECAP makes designated internal content independently checkable against probes rather than asserted by prose a model can game: an independent probe scores the verbalizer's true claims above its false ones (AUC 0.96, vs 0.82 without RECAP). Against an adversary that edits an explanation to maximize the reconstruction score while lying (suppressing ~87% of its lie penalty), the RECAP probe still flags the lies (AUC 0.95) while the control probe collapses to chance (0.51).

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_f136ac3711f18fe255c10a62