---
title: "The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.12963v1"
permalink: "https://feed.astralor.com/items/itm_df8c3bcab736af1a5fcf86b9"
published: "2026-07-14T17:01:12.000Z"
score: 78
channel: "ai"
tags: ["Pseudo-words","Large Language Models","Tail Risk","Aggregate Accuracy","Prediction Stability","Robustness Evaluation"]
---

# The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context

## AI commentary

> This paper punctures a common intuition in LLM robustness: stable overall accuracy ≠ reliable model. With elegant experiments, it demonstrates that aggregate metrics can hide significant per-example risks. A must-read for AI engineers and evaluation researchers, especially those concerned with deployment safety.

## At a glance

Recent research reveals that state-of-the-art large language models appear robust to task-irrelevant context at the aggregate level, with little change in overall accuracy. However, this stability masks substantial per-example prediction flips: even semantically meaningless pseudo-words formed by random character combinations can markedly shift predictions on a small fraction of examples, degrading performance on some while improving it on others. This two-sided effect persists across diverse models and datasets, yet affected examples are largely model-specific. The instability is modulated by context type, length, test-time compute, and model development stage. The findings highlight tail risks concealed by aggregate accuracy, motivating per-example reliability evaluation.

## Key points

- Aggregate accuracy remains stable, but individual predictions can exhibit significant flips.
- Even meaningless pseudo-words can cause prediction changes, with both positive and negative effects.
- Affected examples are highly model-specific.
- Context type, length, test-time compute, and model development stage modulate instability.

## Full text

As large language models (LLMs) grow more capable, they are increasingly deployed in context-rich settings where task inputs are often accompanied by long, partially irrelevant context. In a controlled setting, we find that state-of-the-art models often appear robust to task-irrelevant context at the aggregate level: prepending it to benchmark questions causes little change in overall accuracy. This aggregate stability, however, masks significant per-example instability. Even semantically meaningless pseudo-words, formed by randomly combining characters, can markedly shift model predictions on a small fraction of examples, degrading performance on some while improving it on others. This two-sided effect holds consistently across a wide range of models and datasets, yet the affected examples are largely model-specific. We further show that this instability is modulated by context type, context length, test-time compute, and model development stage. Together, our findings reveal context-induced tail risks concealed by aggregate accuracy, motivating per-example reliability evaluation of language models.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_df8c3bcab736af1a5fcf86b9