---
title: "AI 后训练代理缺失了什么：一项实证分析"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.19072v1"
permalink: "https://feed.astralor.com/items/itm_4d511bc987ec847f717107ba"
published: "2026-08-19T16:17:39.000Z"
score: 78
channel: "ai"
tags: ["AI 代理","arXiv","后训练","大型语言模型","策略评估"]
---

# AI 后训练代理缺失了什么：一项实证分析

## AI 点评

> 这篇论文最值得关注的地方在于它把“能执行”和“会决策”彻底分开，点破了当前 AI 后训练代理的根本短板。对研究自主智能体、AI 自我进化的团队来说，这是必须读的参考，它提示我们单纯堆数据和算力并不能解决策略僵化的问题。

## 速览

这篇论文探讨了 LLM 代理端到端后训练 LLM 的能力，提出这一过程混淆了执行级能力和策略级能力。通过分析大量公开的后训练轨迹，发现代理的训练策略在开始时就被锁定，剩余预算只用于局部调整。实验表明，经验驱动的脚手架能提升执行能力，人工指导能改变初始策略，但代理仍会陷入局部调整循环；增加推理算力仅对简单任务有效。结论是代理缺少的是在执行过程中自发重新评估策略的机制。

## 关键点

- 代理在后训练中策略早期锁定，仅做局部调整
- 经验脚手架提升执行能力（GSM8K +12.6，HumanEval +40.8）但策略不变
- 人类指导可改变初始策略，但训练开始后代理仍退回局部调整
- 额外推理算力对简单任务有效，对最难任务几乎无增益

## 全文

Large language model (LLM) agents can now post-train an LLM end-to-end. They can write code, launch training, evaluate checkpoints, and improve downstream performance, raising the prospect of AI-for-AI. We argue that this picture conflates two distinct capabilities: execution-level capability, iterating within a selected training strategy; and strategy-level capability, revising the high-level judgment as experimental evidence accumulates. Analyzing a large corpus of publicly released post-training trajectories, we find that across different tasks, the agent's training strategy is locked in at the very beginning, and the entire remaining budget is spent on local adjustments within the selected strategy. We then examine three natural explanations--missing experience, missing guidance, and insufficient reasoning--with escalating interventions. Extensive experiments show that (1) an experience-driven scaffold improves execution across the board (+12.6 points on GSM8K and +40.8 on HumanEval) but leaves the strategy static; (2) human guidance effectively redirects the initial strategy, yet the agent falls back into local adjustment loops once training starts; and (3) additional inference compute pays off on easier tasks but yields almost no gain on the hardest one. In conclusion, what agents lack is neither experience, guidance, nor reasoning compute, but a mechanism for spontaneously reevaluating their strategy during execution.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_4d511bc987ec847f717107ba