---
title: "Beyond Accuracy: What Language-Subspace Interventions Capture, Remove, and Improve"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.05278v1"
permalink: "https://feed.astralor.com/items/itm_19510ab02cf196837a193992"
published: "2026-11-03T17:58:52.000Z"
score: 60
channel: "ai"
tags: ["LLM","Beyond Accuracy","What Language-Subspace Interventions","Capture","Remove","Improve Language-subspace","Using MATH","MGSM"]
---

# Beyond Accuracy: What Language-Subspace Interventions Capture, Remove, and Improve

## 速览

Language-subspace interventions aim to improve multilingual reasoning by suppressing language-related components in mid-layer activations. Using MATH and MGSM across four open-weight LLMs, we ask three questions: do est…

## 关键点

- Language-subspace interventions aim to improve multilingual reasoning by suppressing language-related components in mid-layer activations.
- Using MATH and MGSM across four open-weight LLMs, we ask three questions: do estimated language subspaces capture translation-associated variation?
- Does projecting them out remove language information?

## 全文

Language-subspace interventions aim to improve multilingual reasoning by suppressing language-related components in mid-layer activations. Using MATH and MGSM across four open-weight LLMs, we ask three questions: do estimated language subspaces capture translation-associated variation? Does projecting them out remove language information? And do they improve accuracy without compromising response language? For the first question, we build subspaces using the existing Mean-SVD construction and our DiffPCA. Both capture substantial translation-associated variation and show similar patterns across model depth. For the second, we project out each subspace and train linear probes to classify language from the projected activations. The probes still identify language almost perfectly, showing that neither subspace exhaustively captures language information. For the third, we evaluate several intervention configurations on Qwen2.5-7B, by varying subspace creation methods and intervention operators. We find that the intervention that most improves MGSM accuracy sacrifices response-language fidelity, with only one intervention improving joint accuracy--answers that are correct and in the target language--slightly, by 1.60 points. Interestingly, subtle changes in the prompt alone can reproduce the accuracy gains albeit with significant damage to language fidelity. We argue that causal interventions must be judged on more than accuracy, to fully account for language substitution trade-offs.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_19510ab02cf196837a193992