---
title: "CoPES: Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.02391v1"
permalink: "https://feed.astralor.com/items/itm_6d06728854b98aeff5ade8b3"
published: "2026-08-03T15:34:45.000Z"
score: 78
channel: "tech"
tags: ["arXiv","CoPES","GRPO","LoRA","Qwen3.5-4B","Evolution Strategies"]
---

# CoPES: Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training

## AI commentary

> What strikes me most is how it fuses ES's memory efficiency with coevolution to attack the 'few GPUs' bottleneck. Recovering 92% of GRPO's gain is impressive, and open-sourcing the code makes it immediately actionable. For small teams doing LLM alignment or agent training, this might be a more compelling alternative to LoRA.

## At a glance

Tool-using LLM agents generate long multi-turn trajectories, making gradient-based post-training memory-intensive. Evolution strategies (ES) enable memory-efficient full-parameter training without backpropagation but demand high GPU-hours. To address this, the authors introduce CoPES, a cooperative coevolutionary method that decomposes the full parameter space into low-dimensional subspaces and searches cooperatively. Post-training Qwen3.5-4B for math tool-use, under the same GPU-hour budget as full-parameter GRPO's best checkpoint, CoPES recovers 92% of GRPO's validation-accuracy gain versus 67% for standard ES, while requiring less than one-eighth of the GPU memory. It outperforms standard ES and LoRA-based GRPO across all pass@k metrics on five benchmarks. Code is open-sourced.

## Key points

- CoPES decomposes parameter space into low-dimensional subspaces and searches cooperatively, drastically reducing training time for ES under limited GPUs.
- On a Qwen3.5-4B math agent, it recovers 92% of GRPO's gain under the same GPU budget, with memory below 1/8 of full-parameter GRPO.
- It outperforms standard ES and LoRA-based GRPO on pass@k across five benchmarks, and also shows gains on QA.
- Code is open-sourced on GitHub.

## Full text

Tool-using large language model (LLM) agents produce long, multi-turn trajectories, making gradient-based post-training memory-intensive. Evolution strategies (ES) enable memory-efficient full-parameter post-training without backpropagation and can eventually match the performance of gradient-based reinforcement learning (RL). However, resource-constrained settings typically offer only a few GPUs, so the high GPU-hour requirements of ES translate into prohibitively long training times. To address this, we introduce Cooperative Parameter-subspace Evolution Strategy (CoPES), a cooperative coevolutionary method that decomposes the full parameter space into lower-dimensional subspaces and searches over them cooperatively to improve optimization efficiency. We post-train a Qwen3.5-4B tool-using agent for the math task and evaluate it on five benchmarks of varying difficulty. Under the GPU-hour budget of full-parameter GRPO's best validation checkpoint, CoPES recovers 92% of GRPO's validation-accuracy gain, versus 67% for standard ES, while its theoretical GPU memory requirement is less than one-eighth that of full-parameter GRPO. It consistently outperforms standard ES and LoRA-based GRPO on all evaluated pass@k metrics across the five benchmarks. Additional experiments further show the advantage of CoPES on the question-answering task. These results demonstrate an improved trade-off between memory requirements and training time for agentic LLM post-training under resource constraints. The code is open-sourced in https://github.com/MetaronWang/CoPES

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_6d06728854b98aeff5ade8b3