---
title: "TestEvo-Bench：代码与测试协同演化的可执行实时基准"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.02469v1"
permalink: "https://feed.astralor.com/items/itm_96ef1ecbc384814057fbf4da"
published: "2026-07-02T17:35:20.000Z"
score: 61
channel: "ai"
tags: ["TestEvo-Bench","代码与测试协同演化","基准评估","测试自动化代理"]
---

# TestEvo-Bench：代码与测试协同演化的可执行实时基准

## 速览

TestEvo-Bench是一个用于评估测试自动化代理在测试与代码协同演化场景中能力的基准。它包含测试生成和测试更新两个任务轨道，每个任务都来自真实的提交历史，并配有环境配置，支持通过执行指标（如通过率、覆盖率和变异得分）进行评价。该基准是“实时”的，记录任务时间戳，并定期从开源Java项目中挖掘新任务，从而可以限制评估只使用模型训练截止日期之后的任务，降低数据泄露风险。当前快照包含746个测试生成和509个测试更新任务。实验显示，最先进的代理在测试生成上达到77.5%的成功率，在测试更新上达到74.6%，但在最新任务和有限成本下性能显著下降。

## 关键点

- 提出可执行实时基准TestEvo-Bench，聚焦测试与代码协同演化。
- 包含测试生成和测试更新两个任务轨道，依托真实提交历史。
- 支持执行指标评估（通过率、覆盖率、变异得分），并借助时间戳减少数据泄露。
- 实验显示先进代理成功率约75%，但在最新任务和成本受限时显著下降。

## 全文

Software tests and code evolve together: a code change should be followed by new or updated tests that record the new software behavior. Yet existing test generation and update benchmarks often isolate the test from the code change, and rely on static metadata that does not verify whether a test is executable or semantically tied to the code change. This makes it difficult to evaluate whether a test automation agent understands how a code change should propagate into the test suite. We introduce TestEvo-Bench, a benchmark of test and code co-evolution tasks mined from software repositories, with two tracks: in test generation, the agent shall write new tests to capture the new software behavior; in test update, the agent shall adapt failing existing tests to the changed software behavior. Each task is anchored to a real commit history and packaged with environment configuration to support execution-grounded metrics such as pass rate, coverage, and mutation score. TestEvo-Bench is also a live benchmark: each task records the timestamp of the test and code changes, and new tasks are periodically mined by our automated pipeline, so evaluation can be restricted to tasks postdating a model's training cutoff to reduce data leakage risk. The current snapshot contains 746 test generation and 509 test update tasks, curated from 59,950 candidate co-evolution records across 152 open-source Java projects. We experiment with four state-of-the-art agents that combine strong harnesses (Claude Code, Gemini CLI, and SWE-Agent) with strong foundation models (Claude Opus 4.7 and Gemini 3.1 Pro). Results show that they achieve up to 77.5% success rate on test generation and 74.6% on test update. However, success rate is materially lower on the most recent benchmark tasks and drops significantly under limited per-task cost.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_96ef1ecbc384814057fbf4da