---
title: "Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.14004v1"
permalink: "https://feed.astralor.com/items/itm_7fb45560e7cac385c7edd981"
published: "2026-07-15T16:36:04.000Z"
score: 78
channel: "ai"
tags: ["Agent Optimization","GEPA","Meta Harness","RELAI-VCL","Terminal-Bench 2.0","Continual Learning"]
---

# Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0

## AI commentary

> This paper hits the biggest blind spot in current agent optimization evaluation: one-shot tests fail to capture real-world continuous changes. If you are building or deploying long-running agent systems, this paper is worth careful reading—the regression control idea in RELAI-VCL may become a standard component in future designs. For AI researchers, the experimental comparison and problem-finding methodology are also very enlightening.

## At a glance

This paper challenges the one-shot evaluation of agent optimization methods on static benchmarks and proposes a two-phase continual-learning framework. On hard tasks from Terminal-Bench 2.0, the authors compare three methods: GEPA, Meta Harness, and RELAI-VCL. Results show that only RELAI-VCL, which incorporates regression control, achieves compounding gains—positive transfer to unseen tasks and continued improvement after new tasks are added, reaching a lifelong average pass rate of 76.4%, significantly outperforming baselines.

## Key points

- Existing agent optimization methods are typically evaluated in a one-shot static setting, ignoring real-world continuous task arrival.
- The paper introduces a two-phase continual-learning evaluation to test whether optimization gains compound on new tasks.
- Only RELAI-VCL, which includes regression control, achieves both positive transfer and continued improvement.
- RELAI-VCL achieves 76.4% lifelong average pass rate, while GEPA and Meta Harness fall below baseline or fail to further optimize.

## Full text

Most reported gains from agent-optimization methods are one-shot: an agent is optimized against a fixed benchmark and the resulting improvement is reported as if it were a stable property of the method. This does not test the setting that matters for deployed agents, where optimization is applied recursively as new failures and new tasks appear over time. The central question this raises is whether optimizer-driven gains compound: after an agent has been optimized once, can it be optimized again on newly arrived tasks without eroding the gains the first round produced? We study this question with a two-phase continual-learning evaluation built from hard tasks in Terminal-Bench 2.0, comparing three approaches to agent-harness optimization (GEPA, Meta Harness, and RELAI's Verifiable Continual Learning, RELAI-VCL) under identical optimization budgets. All three methods improve over the baseline agent in the conventional, static, single-phase setting. However, once new tasks are introduced, the methods diverge sharply: GEPA's optimized agent transfers below the unoptimized baseline, Meta Harness transfers well but fails to improve further once given a second optimization budget, and RELAI-VCL is the only method that both transfers positively to unseen tasks and continues improving after those tasks are folded into the optimization objective, reaching the highest pass rate at every evaluated stage and the highest lifelong average pass rate overall (76.4% vs. 66.0% for GEPA, 64.6% for Meta Harness, and 58.7% for the baseline). Our key observation was that optimization gains compounded only when regression control was built into the optimization loop, providing an inductive bias against shortcut solutions that fail to generalize.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_7fb45560e7cac385c7edd981