---
title: "多智能体系统中候选供应与答案选择如何塑造LLM裁判的价值"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.25937v1"
permalink: "https://feed.astralor.com/items/itm_dbc53d9aee941a8c3b3a2892"
published: "2026-08-26T15:52:20.000Z"
score: 73
channel: "ai"
tags: ["arXiv","LLM裁判","候选生成","多智能体系统","答案选择"]
---

# 多智能体系统中候选供应与答案选择如何塑造LLM裁判的价值

## AI 点评

> 这篇论文把多智能体系统失败归因拆得很干净，尤其适合正在做Agent框架和投票/选择机制的人读。它提醒我们：别光堆模型数量，候选池里已经有正确答案时，真正的瓶颈可能在选择规则。

## 速览

该论文将多智能体推理解构为候选生成、同伴交流与终端选择三阶段，发现正确候选常已存在但系统仍会报告错误答案。作者基于16,278道题、81,390个固定候选池和15,336道裁判可靠性测试，提出结合答案频率与裁判评估的混合选择规则，将准确率从63.82%提升至70.82-70.95%，主要靠挽救被多数错误掩盖的正确候选。研究强调裁判可靠性并非模型固有属性，而是随任务、生成器和候选罕见度变化。该工作为设计防丢失正确答案的多智能体架构提供了诊断基础。

## 关键点

- 正确候选经常已经生成，但多智能体系统仍会收敛到错误答案；
- LLM裁判的可靠性随任务、生成器和正确答案罕见度动态变化，并非固定特性；
- 结合答案频率与裁判评价改变最终选择规则，在多个基准上准确率提升约7个百分点；
- 生成更多候选的价值取决于其是否让正确答案更常见、更可识别或更可判定。

## 全文

Multi-agent systems (MAS) sometimes already have the potential to answer correctly, but still report a wrong answer. Explaining this outcome is difficult because generation, communication and final answer-selection rules usually change simultaneously. We conceptualize multi-agent reasoning as an evolutionary pipeline of candidate generation, peer communication and terminal selection, wherein consensus without quality control can exhibit patterns of memetic drift. We study two questions: (1) when an LLM judge provides effective selection pressure by supplying a signal of answer correctness for candidates generated in a multi-agent system, and (2) when using that signal improves the reported answer. To map judge reliability, we analysed 15,336 questions from MMLU-Pro, GPQA, MedXpertQA and MuSR, with Humanity's Last Exam analysed separately. To test these rules, we replayed 81,390 fixed candidate pools drawn from 16,278 questions across five benchmarks. We report three findings. (1) A correct answer is often already present among the generated candidates, but the system can still converge on and report a wrong answer. (2) Judge reliability is not a fixed trait of the model, but varies with the task, the generator and how rare the correct answer is. (3) Combining answer frequency with the judge's evaluation changed only the final answer-selection rule and raised accuracy from 63.82% to 70.82-70.95%, primarily by rescuing correct answers that were outnumbered by popular errors. In the systems studied here, the value of generating more candidates depends on whether those extra samples make correct answers present, frequent or recognisable. By isolating generation, recognition and selection, these findings establish a diagnostic basis for designing multi-agent architectures that protect generated correct answers from being lost.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_dbc53d9aee941a8c3b3a2892