---
title: "自动化发现系统不存在普遍最优的通用框架"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.18235v1"
permalink: "https://feed.astralor.com/items/itm_05fcc563a479cb9ab9352a47"
published: "2026-07-20T17:59:37.000Z"
score: 81
channel: "ai"
tags: ["OpenEvolve","TTT-Discover","大语言模型","自动化发现系统"]
---

# 自动化发现系统不存在普遍最优的通用框架

## AI 点评

> 这篇论文非常值得关注，尤其对于从事AutoML、智能体搜索和LLM自动化研究的团队。它用大量实验告诉我们：别再盲目套用现有框架了，根据具体模型和问题灵活调整才是正道。自适应分配策略提供了即插即用的改进思路，而公开的统计基础设施可以加速后续研究的对比验证。

## 速览

本文系统分解了OpenEvolve与TTT-Discover等自动化发现系统的组成组件，通过310万次LLM推理和12个模型-问题对的30种预算匹配框架的重复试验统计分析，发现没有任何固定框架在所有场景下都表现最优，且OpenEvolve简化变体通常劣于更简单的替代方案。因此框架选择应被视为超参数而非通用配方，并提出了基于早期性能的自适应资源分配策略。研究提供了可复用的统计基础设施和基线分布供后续使用。

## 关键点

- 通过310万次LLM调用和重复试验统计，系统比较了30种自动化发现框架在12个模型-问题对上的表现。
- 没有任何固定框架在所有评估场景下均表现最优，且OpenEvolve系列框架通常不如简单替代方案。
- 早期发现进度可有效预测最终性能，据此提出自适应资源分配策略，优于固定框架和简单集成。

## 全文

Autonomous discovery systems such as OpenEvolve and TTT-Discover are often used as general-purpose harnesses. However, in practice these are composite systems combining several design choices about archives, parent selection, exploration, and budget allocation into a single recipe. Because discovery runs are expensive and inherently stochastic, existing harnesses are often compared using too few independent trials to distinguish key methodological improvements from run-to-run variance. We systematically decompose OpenEvolve-style evolutionary search and the TTT-Discover search harness into its constituent components and systematically evaluate 30 budget-matched harnesses across 12 model-problem pairs using more than 3.1 million LLM rollouts and repeated-trial statistical analysis. Our results show that discovery harnesses have a generalization problem: No fixed harness is reliably superior across the evaluated model-problem pairs, and variants of OpenEvolve generally underperform simpler alternatives. Thus, harness choice is better viewed as a hyperparameter rather than as a universal recipe, and should be tailored to the specific problem and underlying model. We also find that early discovery progress predicts final performance, and use this property to present a budget-matched adaptive-allocation experiment that starts multiple harnesses, prunes weak partial runs, and reallocates compute to stronger survivors, outperforming both commitment to a randomly sampled fixed harness and a non-adaptive harness ensemble. Together, these results motivate shifting from fixed harness selection to online adaptation guided by early performance. We release all run pools including baseline null distributions for every model-problem pair as reusable statistical infrastructure against for future harness proposals.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_05fcc563a479cb9ab9352a47