---
title: "Correct Labels Are Not Enough: Reliable Metadata Can Silently Control Model Behavior"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.04395v1"
permalink: "https://feed.astralor.com/items/itm_89ce0d14b1c9851f808ab42c"
published: "2026-11-03T06:29:57.000Z"
score: 56
channel: "ai"
tags: ["Correct Labels Are","Not Enough","Reliable Metadata Can","Silently Control Model","Behavior Fine-tuning","Fine-tuning","Reward","CommonsenseQA"]
---

# Correct Labels Are Not Enough: Reliable Metadata Can Silently Control Model Behavior

## At a glance

Fine-tuning data can include metadata that predicts the correct answer, such as a reward-model score or a quality label. We ask whether a model learns to rely on this metadata even when every training label is correct. …

## Key points

- Fine-tuning data can include metadata that predicts the correct answer, such as a reward-model score or a quality label.
- We ask whether a model learns to rely on this metadata even when every training label is correct.
- We fine-tune 7-9B open-weight models from three families on multiple-choice questions in which (Reward: 5) always marks the correct option; when this annotation is moved to a wrong option at test time, models select tha…

## Full text

Fine-tuning data can include metadata that predicts the correct answer, such as a reward-model score or a quality label. We ask whether a model learns to rely on this metadata even when every training label is correct. We fine-tune 7-9B open-weight models from three families on multiple-choice questions in which (Reward: 5) always marks the correct option; when this annotation is moved to a wrong option at test time, models select that option on 97-100% of CommonsenseQA items. Removing it leaves most answers correct, so evaluation without it misses the dependence. Cue-following rises sharply with training-time reliability and develops even with neutral or nonsense annotation words, although test-time wording affects its strength. A Qwen3 experiment also finds cue dependence in option-letter scores after direct preference optimization. Audits can test for this dependence by moving or removing visible annotations while holding the task and correct answer fixed.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_89ce0d14b1c9851f808ab42c