---
title: "QuasiMoTTo：利用准蒙特卡洛采样实现测试时计算扩展"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.01179v1"
permalink: "https://feed.astralor.com/items/itm_18ad1cb926740f1f4278ab0b"
published: "2026-07-01T17:10:08.000Z"
score: 60
channel: "ai"
tags: ["GRPO","QuasiMoTTo","准蒙特卡洛采样","推理效率","测试时计算扩展"]
---

# QuasiMoTTo：利用准蒙特卡洛采样实现测试时计算扩展

## 速览

提升语言模型推理能力的一种可靠方法是生成大量独立采样，但独立采样导致大量冗余计算。QuasiMoTTo通过准蒙特卡洛方法生成相关性但精确的样本，替代独立同分布采样，在不牺牲并行性的前提下提高样本效率。在四个推理基准上，QuasiMoTTo以25-47%更少的样本达到相同pass@k准确率，并在策略梯度强化学习（GRPO）中匹配独立采样的性能，节省50%训练步骤。该方法通过更均匀地覆盖输出空间，减少冗余，从而提升每一批样本的学习信号。

## 关键点

- 提出相关采样替代独立同分布采样，通过准蒙特卡洛方法减少输出冗余
- 在四个推理基准上，匹配i.i.d. pass@k准确率，样本减少25-47%
- 在策略梯度强化学习GRPO中，节省50%训练步骤
- 该方法可达到任何边缘保持采样器的pass@k理论上界

## 全文

Scaling inference compute, by generating many parallel attempts per problem, is a costly but reliable lever for improving language model capabilities. By default these attempts are generated independently, wasting inference compute on redundant solutions. This waste seems unavoidable. After all, independence is what makes parallel sampling trivial to scale. However, this tradeoff is not fundamental: there is a rich design space of samplers that generate correlated but exact samples entirely in parallel. We explore this design space as an avenue for improving sample efficiency in scaling inference compute and reinforcement learning (RL). Concretely, we introduce QuasiMoTTo, which uses correlated samples as a drop-in replacement for i.i.d. samples. To generate these samples, QuasiMoTTo uses a reparameterization of autoregressive sampling as inverse-CDF sampling and draws the underlying uniforms with quasi-Monte Carlo (QMC); because QMC spreads the uniforms out more evenly than i.i.d., the resulting samples cover the output space with far less redundancy. Even though the batch is correlated, each sample is marginally distributed according to the language model, so we can use the batch for policy-gradient training. Our empirical analysis focuses on understanding how efficiently QuasiMoTTo can turn compute into performance. To evaluate correlated samplers, whose dependence breaks standard pass@k estimators, we first develop an unbiased bootstrap estimator. Across four reasoning benchmarks, QuasiMoTTo matches i.i.d. pass@k accuracy with 25-47% fewer samples. Strikingly, QuasiMoTTo often saturates an upper bound on pass@k that holds for any marginal-preserving sampler. We also apply QuasiMoTTo to policy-gradient RL (GRPO) where it matches i.i.d. performance with 50% fewer training steps. These gains come from higher coverage, which yields a stronger learning signal per batch.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_18ad1cb926740f1f4278ab0b