---
title: "物理AI基准冗余性的统计审计"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.25940v1"
permalink: "https://feed.astralor.com/items/itm_f0fc879d487fa993dbda4d36"
published: "2026-08-26T15:54:15.000Z"
score: 76
channel: "ai"
tags: ["arXiv","Bradley-Terry模型","基准测试","物理AI"]
---

# 物理AI基准冗余性的统计审计

## AI 点评

> 这篇论文对物理AI圈子是个提醒：我们可能一直在重复测试相同的能力，却误以为获得多元评估。对跑评测的团队，这套贪心筛选法能直接帮您砍掉冗余benchmark，省时间省算力；对做基准研究的同学，它给出了一个可复用的统计工具包，换个领域就能用。建议大家细看Bradley-Terry拟合部分，那才是排名稳定性的关键。

## 速览

物理AI模型评估常用多种基准测试，但各报告选用标准不一，导致模型-基准矩阵稀疏且基准间关系未知。该研究基于51个模型和12个物理AI基准构建矩阵，发现基准存在显著冗余：将两个替代基准合并后，51个模型中22个模型的排名变动超过3位。通过贪心选取，4个基准即可保留全部12个基准78.5%的效用，并拟合出Bradley-Terry排名。该方法仅需基准级分数和足够重叠，不局限于物理AI领域。

## 关键点

- 构建了51个模型在12个物理AI基准上的评分矩阵，并测量基准间的信息共享程度，量化了冗余现象。
- 将两个替代基准合并后，51个模型中22个模型的排名变动不少于3位，表明基准冗余显著影响报告排名。
- 通过贪心算法选出4个基准子集，可保留全部12个基准78.5%的效用，并基于该子集拟合Bradley-Terry排名。
- 该方法仅依赖有足够重叠的基准级分数，具有通用性，可推广至物理AI之外的领域。

## 全文

Physical AI models are evaluated on suites of benchmarks that differ across model reports, leaving the model-by-benchmark matrix sparse and the relationship between benchmarks unmeasured. We construct a matrix of 51 models on 12 physical AI benchmarks, selected from a registry of 51 benchmarks and 152 models by reporting density, combining scores from model cards and benchmark papers with our own evaluation runs under each benchmark's official protocol. We measure how much information the benchmarks share and show quantitative evidence of Redundancy. Redundancy affects reported rankings: collapsing the two substitute pairs into single columns moves 22 of 51 models by three or more places under an equally weighted average. We then select benchmarks greedily under a utility combining score dispersion with variance not explained by the already-selected set, and obtain a four-benchmark subset retaining 78.5\% of the utility of all 12, on which we fit a Bradley--Terry ranking. The procedure requires only benchmark-level scores with sufficient overlap and is not specific to physical AI.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_f0fc879d487fa993dbda4d36