---
title: "Reducing Logprob Without Causal Verification: Diagnosing SAE-Based PII Suppression in LLMs"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.04782v1"
permalink: "https://feed.astralor.com/items/itm_1925324b1eeabbce32f2b9ec"
published: "2026-11-03T12:22:01.000Z"
score: 58
channel: "tech"
tags: ["LLM","Reducing Logprob Without","Causal Verification","Diagnosing SAE-Based PII","Suppression","LLMs SAE-based","PII","LLMs"]
---

# Reducing Logprob Without Causal Verification: Diagnosing SAE-Based PII Suppression in LLMs

## 速览

SAE-based feature-level interventions to suppress PII leakage in LLMs are primarily evaluated by log-probability reduction. We show that this criterion cannot distinguish feature specific suppression from a magnitude ar…

## 关键点

- SAE-based feature-level interventions to suppress PII leakage in LLMs are primarily evaluated by log-probability reduction.
- We show that this criterion cannot distinguish feature specific suppression from a magnitude artifact: a direction-agnostic effect of sufficiently large activation perturbations.
- Across 36 (model, layer, PII type) cases, we compare selected SAE feature interventions with coherencematched random direction controls using logprob and rank-aware metrics.

## 全文

SAE-based feature-level interventions to suppress PII leakage in LLMs are primarily evaluated by log-probability reduction. We show that this criterion cannot distinguish feature specific suppression from a magnitude artifact: a direction-agnostic effect of sufficiently large activation perturbations. Across 36 (model, layer, PII type) cases, we compare selected SAE feature interventions with coherencematched random direction controls using logprob and rank-aware metrics. While 17 cases (47%) appear successful under logprob-only evaluation, only 1 case (2.8%) passes our multimetric framework. The rest reveal two failure modes: random directions outperform selected features, and feature selection collapse across PII types. These results indicate that existing evaluations cannot verify whether interventions target causal pathways of PII leakage. A 9,000 run causal mediation feasibility study corroborates this: the verifier separates candidates reliably, but correlation-selected candidates rarely pass it. We therefore propose a new research mission for NLP: causally grounded PII suppression through circuit-level localization and causal mediation analysis.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_1925324b1eeabbce32f2b9ec