---
title: "CoPES：面向资源受限Agent大模型的协作协同进化后训练方法"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.02391v1"
permalink: "https://feed.astralor.com/items/itm_6d06728854b98aeff5ade8b3"
published: "2026-08-03T15:34:45.000Z"
score: 78
channel: "tech"
tags: ["arXiv","CoPES","GRPO","LoRA","Qwen3.5-4B","进化策略"]
---

# CoPES：面向资源受限Agent大模型的协作协同进化后训练方法

## AI 点评

> 这篇论文最戳我的点是把进化策略的「省内存但费时间」和协同进化结合起来，直接命中只有几张卡的研究者的痛点。92%的GRPO增益恢复率相当能打，而且代码开源了，想复现的人可以立刻上手。对做LLM对齐或Agent训练的小团队来说，这可能是比LoRA更值得一试的路线。

## 速览

面向使用工具的LLM智能体，梯度后训练因长轨迹而显存占用高。进化策略虽能省显存，但GPU耗时大。为此，研究者提出CoPES，将全参数空间分解为低维子空间进行协作搜索，提升优化效率。在Qwen3.5-4B数学工具智能体上，同等GPU预算下CoPES恢复GRPO验证精度增益的92%，标准ES仅67%；显存需求不足全参GRPO的八分之一，并在五个基准上全面超越标准ES和LoRA式GRPO。代码已开源。

## 关键点

- CoPES将参数空间分解为低维子空间协作搜索，在有限GPU下显著缩短进化策略训练时间。
- 在Qwen3.5-4B数学智能体上，同等GPU预算下恢复GRPO增益的92%，显存需求低于全参GRPO的1/8。
- 五个基准的pass@k指标上均优于标准ES和LoRA式GRPO，问答任务上也有优势。
- 代码已在GitHub开源。

## 全文

Tool-using large language model (LLM) agents produce long, multi-turn trajectories, making gradient-based post-training memory-intensive. Evolution strategies (ES) enable memory-efficient full-parameter post-training without backpropagation and can eventually match the performance of gradient-based reinforcement learning (RL). However, resource-constrained settings typically offer only a few GPUs, so the high GPU-hour requirements of ES translate into prohibitively long training times. To address this, we introduce Cooperative Parameter-subspace Evolution Strategy (CoPES), a cooperative coevolutionary method that decomposes the full parameter space into lower-dimensional subspaces and searches over them cooperatively to improve optimization efficiency. We post-train a Qwen3.5-4B tool-using agent for the math task and evaluate it on five benchmarks of varying difficulty. Under the GPU-hour budget of full-parameter GRPO's best validation checkpoint, CoPES recovers 92% of GRPO's validation-accuracy gain, versus 67% for standard ES, while its theoretical GPU memory requirement is less than one-eighth that of full-parameter GRPO. It consistently outperforms standard ES and LoRA-based GRPO on all evaluated pass@k metrics across the five benchmarks. Additional experiments further show the advantage of CoPES on the question-answering task. These results demonstrate an improved trade-off between memory requirements and training time for agentic LLM post-training under resource constraints. The code is open-sourced in https://github.com/MetaronWang/CoPES

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_6d06728854b98aeff5ade8b3