---
title: "Agon：通过隐式对手评分的竞争性跨模型强化学习提升推理能力"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.07690v1"
permalink: "https://feed.astralor.com/items/itm_1e8c4b0ab67fe3218908c2fd"
published: "2026-07-08T17:49:14.000Z"
score: 79
channel: "ai"
tags: ["Agon","DeepMath","GRPO","Qwen3","对抗训练"]
---

# Agon：通过隐式对手评分的竞争性跨模型强化学习提升推理能力

## AI 点评

> 这篇论文提出的Agon方法非常巧妙，通过双模型对抗训练隐式提升了推理链质量，且无需任何人工或外部奖励模型。对于从事RLHF或推理后训练的研究者和工程师来说，这是一个值得关注的新方向。实验在多个场景中表现一致，通用性令人信服。

## 速览

Agon是一种创新的竞争性跨模型强化学习方法，通过让两个模型互为评分者来提升推理能力。与传统仅评分最终答案的GRPO不同，Agon在训练中隐式地评判推理过程，无需过程标签或奖励模型。两模型交替扮演“解题者”和“审阅者”，以超越对方为目标，形成动态对抗训练。推理时则采用级联模式：一个模型起草，另一个阅读后作答。在DeepMath硬分评测中，Agon使Qwen3模型的pass@1翻倍，效果远超未训练的混合智能体方法。该方法在编程竞赛和不同模型族（Qwen3.5、Gemma 4）上均有显著提升，揭示了对抗性共训练在推理强化中的潜力。

## 关键点

- Agon让两个竞争模型在训练中互为评分者，隐式评判推理过程，无需过程标签或奖励模型。
- 训练采用交替角色机制，两模型分别扮演解题者和审阅者，以超越对方为奖励。
- 推理时使用级联模式：一个模型起草答案，另一个阅读后给出最终答案。
- 在DeepMath硬分评测上，Agon使Qwen3的pass@1翻倍，效果远超GRPO和未训练的混合智能体方法。

## 全文

Reinforcement learning from verifiable rewards (e.g. GRPO) is the engine behind today's reasoning models, yet it grades only the final answer. On hard problems this trains models to write more rather than to think better, since the trace itself is never graded and no label for good thinking exists. We introduce Agon, which makes two competing models each other's graders. Both attempt the same problem; in alternating roles, one drafts a solution and the other reads it while solving, and each is rewarded for out-solving the other. To win, a model must out-reason a rival that has seen its work, so reasoning is judged implicitly during training, with no process labels and no reward model. Because both models are optimized, each faces a progressively stronger rival, which single-model RL cannot provide. The two need only be comparably strong and behaviorally different. At inference the pair deploys as it trains, a two-stage cascade in which one model drafts and the other answers after reading the draft. On the hard split of DeepMath with Qwen3, this doubles GRPO's pass@1, roughly eight times the gain of an untrained Mixture-of-Agents pass over the same base. The ordering replicates on competitive-programming code and across model families (Qwen3.5, Gemma 4). For now the models talk in text; the next step is to let them reason together in latent space.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_1e8c4b0ab67fe3218908c2fd