---
title: "RACE：大语言模型神经元功能一致性的可扩展统计估计框架"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.24758v1"
permalink: "https://feed.astralor.com/items/itm_75738d1c7bb3e475afdca1b2"
published: "2026-08-25T15:57:43.000Z"
score: 72
channel: "tech"
tags: ["RACE","Transformer","机制可解释性","统计估计"]
---

# RACE：大语言模型神经元功能一致性的可扩展统计估计框架

## AI 点评

> 这篇论文最值得关注的地方在于把昂贵的梯度计算换成前向传播统计，效率提升两个数量级，对于想做规模化神经元分析的研究者来说非常实用。如果你在探索模型可解释性或对齐问题，值得读一下这个框架。

## 速览

可解释性研究面临领域级神经元行为稳定性识别难题，现有方法多依赖实例级点估计或高成本计算，难以兼顾群体变异与规模化分析。RACE 提出基于前向传播的统计框架，通过残差对齐估计 Transformer 神经元在领域内的功能一致性。扰动实验表明，RACE 的领域特异性优于梯度点估计方法，并在 token 分布层面验证了所选神经元与目标领域的关联，计算开销较梯度方法降低两个数量级。

## 关键点

- RACE 采用前向传播与残差对齐，无需梯度计算即可评估神经元功能一致性。
- 在领域特异性上优于梯度点估计方法，并能在 token 分布层面验证神经元与领域关联。
- 计算开销比梯度类方法低两个数量级，支持大规模全领域分析。

## 全文

Discovering stable neuron behavior across entire domains remains a challenge in mechanistic interpretability. Existing methods often rely on instance-level point estimates or computationally expensive procedures, which either obscure population-level variability or limit scalable domain-wide analysis. We present RACE (Residual Alignment for Consistency Estimation), a forward-pass statistical framework that evaluates the domain-wide functional consistency of Transformer neurons. Perturbation experiments demonstrate that RACE achieves superior domain specificity compared to gradient-based point estimates. Meanwhile, token-distribution-level results verify the association between the selected neurons and the target domain. Furthermore, its computational overhead is two orders of magnitude lower than that of gradient-based methods.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_75738d1c7bb3e475afdca1b2