---
title: "Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.20169v1"
permalink: "https://feed.astralor.com/items/itm_5bcda1f61096914851c3e859"
published: "2026-08-20T15:24:54.000Z"
score: 74
channel: "tech"
tags: ["Harness Optimization","LLM Agent","Task-CoEvolve","Terminal-Bench 2.1","University of Tokyo","Variance-Weighted Sampling"]
---

# Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection

## AI commentary

> This paper deserves attention from engineers working on agent system optimization. It solves the heavy-evaluation pain point with a lightweight method that is ready to use. If your team is iterating on harness code repeatedly, this approach could save you several-fold compute.

## At a glance

A team from the University of Tokyo proposes Task-CoEvolve for efficient optimization of LLM agent harnesses. Traditional methods evaluate a full fixed validation set at every iteration, incurring high costs. Task-CoEvolve uses variance-weighted sampling to focus on tasks near the agent's capability frontier and estimates full-set scores from partial evaluations by accounting for sampling probabilities. On online text classification and Terminal-Bench 2.1, it outperforms fixed-subset baselines and matches full-set search while reducing evaluation counts by 80%. Code is open-sourced.

## Key points

- Proposes Task-CoEvolve, which co-evolves validation tasks with the agent harness to dynamically select informative tasks.
- Uses variance-weighted sampling based on disagreement among candidate harnesses to focus on frontier tasks, with sampling-probability-adjusted full-set score estimation.
- Matches full-set search's final performance on benchmarks while reducing evaluation counts by 80%, significantly cutting optimization costs.

## Full text

We present a novel approach to efficient LLM agent harness optimization through adaptive validation task selection. Harness optimization iteratively rewrites the harness code based on validation performance, enabling substantial performance gains without updating the underlying model weights. Existing approaches, however, evaluate a fixed validation set in full at every iteration, incurring substantial evaluation costs even on tasks that become less discriminative as the harness evolves. We propose $\textbf{Task-CoEvolve}$, which co-evolves the validation tasks with the harness by addressing two challenges: selecting informative tasks and estimating full-set performance from partial evaluations. Task-CoEvolve builds on the observation that tasks on which candidate harnesses disagree are more informative for distinguishing among them than tasks that are consistently solved or failed. It uses variance-weighted sampling based on past outcomes to focus evaluation on tasks near the agent's capability frontier, with the sampling distribution adapting as the harness evolves. It then estimates full-set scores from the sampled tasks by accounting for their sampling probabilities, enabling consistent comparisons across iterations despite evaluating different subsets. Experiments on online text classification and Terminal-Bench 2.1 show that Task-CoEvolve consistently outperforms fixed-subset baselines and matches the final performance of full-set search while reducing the number of evaluations during optimization by 80%. Code will be released at https://github.com/Agent4Science-UTokyo/Task-CoEvolve.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_5bcda1f61096914851c3e859