---
title: "OpenForgeRL：在任意环境中原生训练Harness型智能体的开源框架"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.21557v1"
permalink: "https://feed.astralor.com/items/itm_d3c071465ce8444859c6c248"
published: "2026-07-23T17:38:30.000Z"
score: 81
channel: "tech"
tags: ["AI Agent","Claude Code","Codex","OpenClaw","OpenForgeRL","强化学习"]
---

# OpenForgeRL：在任意环境中原生训练Harness型智能体的开源框架

## AI 点评

> 这篇论文值得所有做AI Agent训练的工程师和研究者关注，尤其是那些被复杂推理框架困住无法端到端训练的团队。OpenForgeRL真正打通了harness与RL之间的壁垒，让SFT/RL堆栈能原生使用现有生产级推理框架。不过要注意，错误恢复这个老大难问题依旧存在，但至少现在有了可复现的试验场。对于想部署可靠GUI或工具智能体的团队，这是个必须读的框架。

## 速览

OpenForgeRL 提出了一种轻量级代理与Kubernetes编排器结合的方案，将复杂推理框架（如Claude Code、Codex）的模型调用转化为标准RL可训练数据，实现端到端训练。在工具型、多模态GUI等任务上，仅用数百至数千样本即达到或超越大模型基线，且深入分析了不同harness与RL对行为的影响，揭示自验证与多步规划能力的提升但仍存在错误恢复短板。

## 关键点

- 提出OpenForgeRL开源框架，通过代理和K8s编排解耦训练与推理，支持任意harness与环境端到端训练。
- 在ClawEval、QwenClawBench、OSWorld-Verified等基准上达到31.7 pass^3、33.7、37.7等分数，超越同规模开源基线。
- 分析表明RL提升智能体可靠性（自验证、工具覆盖、多步规划），但错误恢复能力仍然薄弱。
- 方法高效：仅需数百至数千任务即可取得显著性能。

## 全文

Modern AI agents rely on elaborate inference harnesses such as Claude Code, Codex, and OpenClaw to drive multi-turn reasoning, tool use, and access to external systems. While powerful, these complex harnesses also make agents hard to train end-to-end with open infrastructure, whose SFT/RL stacks cannot natively express stateful, multi-process harness inference. To address this, we present OpenForgeRL, an open-source framework for training harness-based agents end-to-end in diverse environments. OpenForgeRL achieves this with a lightweight proxy that serves the harness's model calls while recording them as training data for a standard RL codebase (e.g., veRL), and a Kubernetes orchestrator that runs each rollout in its own remote container, together enabling training on any harness in any environment at scale. By decoupling training and inference, OpenForgeRL allows researchers to easily train, study, and improve agents directly in the real harnesses and environments they are deployed with. We validate our framework across diverse, complex harnesses and environments, spanning tool/claw-based agents and multimodal GUI browser- and computer-use agents. Using only hundreds to a few thousand tasks, OpenForgeClaw reaches 31.7 pass^3 and 55.9 pass@3 on ClawEval and 33.7 on QwenClawBench. OpenForgeGUI reaches 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager. Both outperform open baselines of similar size on nearly all benchmarks, and in the GUI setting match or surpass models several times larger. Beyond benchmarks, we analyze how harness choice (e.g., ZeroClaw, OpenClaw, Codex) and RL shape agent behavior. We find that some harnesses are substantially harder to learn than others, and that RL improves agentic reliability, such as self-verification, tool coverage, and completing multi-step plans, though critical abilities such as error recovery remain weak.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_d3c071465ce8444859c6c248