---
title: "自动事实核查系统鲁棒性研究：跨基准评估揭示领域与检索瓶颈"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.25934v1"
permalink: "https://feed.astralor.com/items/itm_325e79ac6bacff73c84ebe4a"
published: "2026-08-26T15:50:10.000Z"
score: 77
channel: "ai"
tags: ["AVeriTeC","ClimateCheck","SciFact","大语言模型","检索增强生成","自动事实核查"]
---

# 自动事实核查系统鲁棒性研究：跨基准评估揭示领域与检索瓶颈

## AI 点评

> 这篇论文的价值在于它用硬数据打破了‘排行榜万能’的幻觉——同一个模型在不同领域掉点可以这么严重。值得关注的不仅是技术结论，更是它给RAG和事实核查的工程实践敲了警钟：如果你的检索证据是噪音，再强的验证器也是白搭。做实际部署的团队应该认真看看它的领域迁移分析和黄金证据对比实验。

## 速览

该论文对自动事实核查（AFC）系统进行了首次全面的跨基准评估，覆盖科学、开放网页和气候领域四个数据集，对比九种模型，包括随机/稀疏基线、微调Transformer、零样本LLM及AVeriTeC 2025顶级系统。结果发现：微调模型在ClimateCheck上优于零样本LLM和顶级系统，排名高度依赖领域与指标；检索证据替换为黄金标注可将准确性提升14-22个百分点，证实检索仍是主要瓶颈。代码和数据集已开源。

## 关键点

- 跨四个数据集（科学、开放网页、气候）基准测试九种AFC模型，涵盖从基线到AVeriTeC 2025顶级系统。
- 在ClimateCheck上，微调模型超过零样本LLM和AVeriTeC 2025冠亚军，噪音证据会降低真实性预测。
- 模型排名强烈依赖领域和评估指标，SciFact上最佳模型（macro-F1 0.70）在ClimateCheck上跌至0.31。
- 使用黄金证据替代检索证据可提升14-22个百分点，证明检索环节是当前主要瓶颈。

## 全文

Automated fact-checking (AFC) systems retrieve evidence and predict claim veracity, yet evaluations omit simple baselines, systems are developed for a single benchmark and cannot be trusted to generalise across domains. No prior work cross-evaluates the full two-stage retrieve-then-verify pipeline across diverse datasets, complementing retrieval-only studies (Thakur et al., 2021) and single-stage benchmarking studies (Calamai et al., 2025). We benchmark nine models, ranging from random and sparse baselines to fine-tuned transformers, zero-shot LLMs, and the two highest-ranked systems from the AVeriTeC 2025 shared task, across four datasets spanning scientific, open-web, and climate domains. Three findings stand out: (1) on ClimateCheck claim-only and fine-tuned models outperform zero-shot LLM and top-performing AVeriTeC 2025 systems, highlighting that noisy evidence can degrade veracity prediction; (2) system rankings are strongly domain- and metric-dependent: the best model on SciFact (macro-F1 0.70) drops to 0.31 on ClimateCheck, while the AVeriTeC 2025 winner and runner-up swap rankings based on evaluation metrics and datasets; (3) replacing retrieved evidence with gold annotations improves veracity accuracy by 14-22 points across models, confirming retrieval remains primary bottleneck. We release code, pre-processed datasets, and all results to support reproducible AFC research.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_325e79ac6bacff73c84ebe4a