---
title: "Co-Evolving LLM Evaluators and Policies via DynamicRubric"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.20083v1"
permalink: "https://feed.astralor.com/items/itm_43552c643a7e637b090dd972"
published: "2026-07-22T12:35:40.000Z"
score: 80
channel: "ai"
tags: ["DynamicRubric","Post-training","Large Language Models","WeChat Search","Evaluator Co-evolution"]
---

# Co-Evolving LLM Evaluators and Policies via DynamicRubric

## AI commentary

> This paper deserves attention from anyone working on LLM alignment and post-training. It not only theoretically identifies the root cause of reward model supervision collapse, but also provides a lightweight and deployable solution. The WeChat Search deployment case is especially convincing—a true closure from lab to production. If you're struggling with vanishing reward signals in policy optimization, this method might be the answer you're looking for.

## At a glance

This paper proposes DynamicRubric, a framework addressing the bottleneck in post-training of LLMs where evaluator feedback gaps collapse as policy improves. A theoretical analysis reveals that relative score gaps are the directional signals for policy updates. DynamicRubric generates weighted binary rubric items per candidate set and aggregates judgments into response-level scores, enabling co-evolution of evaluator and policy. Experiments with 8B backbones surpass a 70B reward model and a 235B static rubric generator. The optimized model is fully deployed in WeChat Search's AI answering scenario, serving tens of millions of requests daily and improving key online metrics.

## Key points

- Identifies relative score gaps as the core signal for policy optimization, with theoretical proof from a probability allocation perspective.
- Proposes DynamicRubric: dynamically generates weighted binary rubric items per response set, enabling evaluator-policy co-evolution.
- Outperforms a 70B reward model and a 235B static rubric generator on 8B backbone models, with gains on verifiable reasoning and coding tasks.
- Deployed in WeChat Search's AI answering scenario, serving tens of millions of daily requests and improving key online metrics.

## Full text

Post-training with evaluator feedback on policy-induced samples serves as a major mechanism for improving large language models. As policies improve, these sampled responses become close in quality. These close candidates create a bottleneck for policy optimization: collapsed relative evaluator score gaps yield weak or misleading policy supervision. We theoretically characterize why these gaps matter through a probability allocation view, showing that the directional gain of shifting probability mass from one response to another is exactly the evaluator score gap between them. This identifies relative score gaps as the policy optimization signals that guide updates. Motivated by this view, we propose DynamicRubric, a response-set-conditioned evaluator--policy co-evolution framework that generates weighted binary rubric items for each candidate set and aggregates the resulting judgments into response-level scores. In our experiments with 8B backbones, DynamicRubric improves evaluator performance and provides stronger policy supervision than baselines using a 70B reward model or a 235B static rubric generator. DynamicRubric-optimized policies also show gains on verifiable reasoning and coding tasks. A DynamicRubric-optimized model is fully deployed in WeChat Search's AI answering scenario, where it serves all online traffic across tens of millions of requests per day and improves key online metrics. These results suggest a principle for evaluator-guided post-training: evaluators should evolve with the policies they supervise.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_43552c643a7e637b090dd972