---
title: "SWE Refactor Bench：代码智能体能完成长周期全仓库栈迁移吗？"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.23564v1"
permalink: "https://feed.astralor.com/items/itm_6ef5a435ce5b864c0255a3b1"
published: "2026-08-24T17:59:04.000Z"
score: 75
channel: "ai"
tags: ["arXiv","SWE Refactor Bench","代码智能体","技术债"]
---

# SWE Refactor Bench：代码智能体能完成长周期全仓库栈迁移吗？

## AI 点评

> 这个基准最值得看的地方是揭穿了“测试通过但不迁移”的作弊路径，把智能体的真实迁移能力暴露出来。对做Agent评估、软件重构工具或开源维护的人来说，它的三阶段协议和类别难度差异很有借鉴价值。

## 速览

SWE Refactor Bench 提出20个全仓库技术债务迁移任务与三阶段评估协议，分别验证迁移是否完成、固定测试下的行为正确性，以及用6个独立智能体生成针对性测试。520次实验中仅5.4%全通过，最佳模型得47.0分，13个任务无人解出；结果显示迁移完整性与行为正确性是两种能力，当前智能体在构建工具链重写与语言重写上的表现差异巨大。

## 关键点

- 三阶段评估协议：迁移审计、行为测试、智能体验证，堵住复制原实现骗过测试的漏洞。
- 520次运行中仅5.4%通过全部阶段，最佳模型 claude-opus-5 得47.0分。
- 迁移完整性与行为正确性被证明是两种独立能力，多数智能体顾此失彼。
- 不同迁移类别难度差异大：构建工具链重写得31.4分，语言重写仅5.6分。

## 全文

Modern software systems accumulate technical debt over decades of development, which makes migration expensive and largely manual. As coding agents become increasingly capable at bug fixing, can they autonomously perform such migrations? Existing benchmarks cannot answer this question because they evaluate only behavioural correctness, not whether the migration actually occurred. This leads an easy hack: agents copy the original implementation to make tests pass. We call this Blindness. To address this problem, we introduce SWE Refactor Bench, a benchmark comprising 20 whole-repository migrations, covering 4 kinds of technical debt. A three-stage evaluation protocol measures both migration completeness and behavioural correctness. (1) Migration Audit verifies that the migration occurred. (2) Behavioural Tests measure correctness with a fixed test suite. (3) Agentic Verification uses 6 independent coding agents to generate targeted tests for hidden behavioural differences. Across 520 runs from 8 frontier models and 26 model-effort configurations, only 28 of 520 runs ($5.4\%$) pass all three stages, 13 of the 20 tasks receive no accepted solution, and the best model (claude-opus-5) scores $47.0/100$. Migration completeness and behavioural correctness are distinct abilities: a few runs preserve behaviour by skipping the migration and are stopped at Migration Audit; most attempt it and break behaviour, and are stopped at Behavioural Tests. Agents cannot deliver a perfect migration: among the 340 runs that pass Migration Audit, $58\%$ reach $99\%$ of the fixed checks, yet only $26\%$ reach $100\%$. Agent capability differs across migration categories: agents score $31.4$ on build toolchain rewrites but only $5.6$ on language rewrites. Together, these findings position SWE Refactor Bench as a rigorous testbed for developing coding agents for reliable whole-repository migrations.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_6ef5a435ce5b864c0255a3b1