---
title: "Swap, Replicate, Read the Mass: A Protocol for Auditing Demographic Bias in System One Decision Models"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.00496v1"
permalink: "https://feed.astralor.com/items/itm_3bdfab1e204a9b63ea3985b5"
published: "2026-10-31T00:36:15.000Z"
score: 56
channel: "ai"
tags: ["安全","Swap","Replicate","Read","Mass","Protocol","Auditing Demographic Bias","System One Decision"]
---

# Swap, Replicate, Read the Mass: A Protocol for Auditing Demographic Bias in System One Decision Models

## 速览

System One decision models replace text generation with typed, probability-scored answers and are entering production. Answers execute without a person in the loop, and no multi-model audit exists on established bias be…

## 关键点

- System One decision models replace text generation with typed, probability-scored answers and are entering production.
- Answers execute without a person in the loop, and no multi-model audit exists on established bias benchmarks.
- We provide that audit: three benchmarks: BBQ (all eleven categories), CrowS-Pairs, and SilverSpoon, adapted to the typed interface, twelve models plus a no-thinking ablation, and 1,638,000 decisions.

## 全文

System One decision models replace text generation with typed, probability-scored answers and are entering production. Answers execute without a person in the loop, and no multi-model audit exists on established bias benchmarks. We provide that audit: three benchmarks: BBQ (all eleven categories), CrowS-Pairs, and SilverSpoon, adapted to the typed interface, twelve models plus a no-thinking ablation, and 1,638,000 decisions. Every order-sensitive item runs twice with options swapped; native typed models also swap labels; ten configurations repeat under extra seeds and batch 16. We find widespread stereotype alignment, order-dependent answers, and confidence that tracks instability rather than bias. In BBQ, 97 of 143 model-by-category scores are significant and 94 aligned, while the commercial reference abstains on 99.5% of ambiguous items, hiding aligned probability mass. Option swaps flip 0.0% to 72.6% of choices; typed scoring stays below 10% except Laya (22-33%), and untuned letter prompting reaches 73%. CrowS-Pairs preference spans 0.496 to 0.816, and boolean answers can be decided by position (1.00 versus 0.005). SilverSpoon judgments shift toward lower-SES labels in five of eight decision models, inherited from the 9B backbone or induced by tuning. Audits must permute options and reverse questions by default; order-invariant scoring should become a procurement requirement.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_3bdfab1e204a9b63ea3985b5