---
title: "SFGA: A Statistics-First Gating Architecture with Adjudicative Escalation for Trustworthy SFT Data Procurement"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.18960v1"
permalink: "https://feed.astralor.com/items/itm_482c913f1bcdc3d8d6ffd8fe"
published: "2026-07-21T10:52:04.000Z"
score: 77
channel: "tech"
tags: ["LLM Bias","SFGA","SFT Data Procurement","Cost-Aware Routing","Statistical Gating","Adjudicative Debate"]
---

# SFGA: A Statistics-First Gating Architecture with Adjudicative Escalation for Trustworthy SFT Data Procurement

## AI commentary

> This paper deserves attention because it tackles a practical yet often overlooked pain point in large model training: the trust and cost trade-off in data procurement. SFGA cleverly combines fast statistical screening with LLM adjudicative debate, avoiding expensive manual review while exposing hidden biases in naive LLM evaluation. For data engineers and procurement decision-makers in AI teams, this architecture can be immediately translated into cost savings and quality control. Particularly noteworthy is the honest reporting of negative biases in the debate, reminding us to remain vigilant when using LLMs for automated decisions.

## At a glance

This paper proposes SFGA, a statistics-first gating architecture that treats SFT data procurement as a cost-aware routing problem. The system computes cheap blind measurements across three quality axes (diversity, utility, redundancy) and accepts a decision only when confidence intervals are tight, sample sizes adequate, and axes agree; otherwise, it escalates to a buy/reject adjudicative debate. On a benchmark of 12 datasets, the gate achieves 0.90 accuracy and 0.83 F1 at $0.017 per unit, undercutting the always-escalate baseline ($0.020). Experiments further reveal a con-side win rate of 0.80 and a 52% position-flip rate under advocate swapping, exposing negativity and positional biases that naive LLM judges would hide.

## Key points

- Proposes SFGA, a statistics-first gating architecture combining cheap measurements and adjudicative escalation for cost-effective SFT data procurement.
- Makes cost-aware decisions on diversity, utility, and redundancy axes, auto-passing only under high confidence, otherwise triggering debate.
- Benchmark achieves 0.90 accuracy and 0.83 F1, with cost lower than always-escalate and close to oracle upper bound.
- Reveals significant negativity bias (80% con-side win rate) and 52% position-flip rate in debate, warning of hidden biases in LLM judges.

## Full text

Procuring supervised fine-tuning (SFT) data forces a buyer to decide, before any downstream training, whether a candidate corpus is worth acquiring. We present \sys{}, a statistics-first gating architecture that treats procurement as a cost-aware routing problem over three intrinsic quality axes -- diversity, utility, and redundancy. Cheap blind measurements are summarised into per-axis estimates with confidence intervals; a gate accepts a decision only when intervals are tight, sample sizes are adequate, and the axes agree, otherwise it escalates the case to an adjudicative debate between a buy-advocate and a reject-advocate judge, resolved by a presiding verdict. On a controlled benchmark of 12 datasets ($2{\times}3{\times}2$ grid over the three axes) with 5 seeds, the gate reaches 0.90 accuracy and 0.83 $F_1$ at \$0.017 per unit, sitting between an always-verify baseline (0.75) and an oracle upper bound (0.98) while spending less than always-escalate (\$0.020). We further report honest negative diagnostics of the debate path: a con-side win rate of 0.80 ($p\approx3{\times}10^{-6}$) and a 52\% position-flip rate under advocate swapping expose negativity and positional biases that a naive LLM-judge would hide. We frame the injected-knob evaluation explicitly as a controlled synthetic benchmark for measurement fidelity and routing calibration, and delimit external validity as future work.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_482c913f1bcdc3d8d6ffd8fe