---
title: "WorldCup Arena：在真实世界杯赛事中实现前瞻性、无泄漏的LLM评测"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.04008v1"
permalink: "https://feed.astralor.com/items/itm_4c9e04fa69cd16ece1d000d1"
published: "2026-08-04T17:59:55.000Z"
score: 76
channel: "ai"
tags: ["2026世界杯","arXiv","WorldCup Arena","大语言模型","无泄漏评测"]
---

# WorldCup Arena：在真实世界杯赛事中实现前瞻性、无泄漏的LLM评测

## AI 点评

> 作为编辑，我认为这篇文章最值得看的不是分数本身，而是它证明了一个常识：真实世界里的预测能力评测不能靠事后过滤，要靠时间本身。对做模型评测、Agent决策或体育数据产品的人来说，这个基准是很好的参考样本。

## 速览

现有LLM预测评测大多回顾已发生事件，易受记忆污染。该研究反向设计：在2026年FIFA世界杯39天赛程中，六款前沿LLM在每次开球前填写七市场预测卡，覆盖104场比赛及小组头名和夺冠池；提问时答案尚不存在，因此评测天然无泄漏，共形成4494条预测。结果显示，模型胜负准确率平均63.9%，与押注博彩热门相当；模型间共识高于正确率，多数投票无增益；比分预测趋同且低估平局与进球。数据集与评分代码已公开。

## 关键点

- 采用前瞻性实时赛事设计，提问时答案不存在，从构造上杜绝了数据泄漏。
- 六款前沿LLM对104场比赛共产生4494条预测，胜负准确率平均63.9%，与支持博彩热门策略打平。
- 模型间共识高于正确率，多数投票无法带来增益；对平局和进球数预测偏保守，比分选择高度趋同。
- 排名榜头部和底部稳定、中间层频繁更替，且数据集与评分代码已开放。

## 全文

Benchmarks that measure the forecasting ability of large language models are almost always retrospective: the event has happened, the answer is somewhere on the Web, and the evaluation must defend itself against memorisation. We report the opposite design. Over the 39 days of the 2026 FIFA World Cup, six frontier LLMs -- all with extended thinking and native server-side web search -- were asked before every kickoff, one match at a time, to fill in a seven-market prediction card for all 104 matches, plus 12 group winners and a pre-tournament outright pool; no answer existed when the question was asked, so the evaluation is leakage-free by construction rather than by filtering, and the frozen archive holds 4,494 scored predictions. What the tournament establishes is a set of behaviours the six systems share. On match outcome they average 63.9%, level with backing the bookmaker's favourite -- which is in fact what they usually do. They agree with one another far more often than they are right, so a majority vote adds nothing. They under-commit to draws and to goals, and crowd their scoreline picks onto a single prototypical result. Accuracy tracks how lopsided a fixture is rather than how much is known about it: it collapses in the closest ties, where the dossiers are richest, while questions about the tournament as a whole are answered well. On this task the current generation of frontier systems is not sharply differentiated: the standings hold up at the top and the bottom across the run and churn in the middle, and the margins stay narrow throughout. The briefing dossiers, fixtures and official results are released as a benchmark, together with the scoring code.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_4c9e04fa69cd16ece1d000d1