---
title: "Resample or Reroute? Budget-Aware Test-Time Model Selection for Large Language Models"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.08665v1"
permalink: "https://feed.astralor.com/items/itm_86b7e16995ea79a5718d805b"
published: "2026-07-09T16:34:15.000Z"
score: 81
channel: "ai"
tags: ["arXiv","Large Language Models","Cost Optimization","Model Routing","Test-Time Model Selection","Resampling"]
---

# Resample or Reroute? Budget-Aware Test-Time Model Selection for Large Language Models

## AI commentary

> This paper cleverly captures a neglected dimension in LLM serving—the competition between resampling and rerouting under the same per-query budget. The proposed RoR policy is simple yet effective, offering practical reference for any query-cost-based inference system. For AI infrastructure teams and model service providers, this is a powerful tool to reduce total cost while maintaining high correctness. It is noteworthy that verifier quality is a critical gatekeeper, reminding us to prioritize investing in high-quality verifiers in real deployment.

## At a glance

This paper proposes a budget-aware test-time model selection method that, under a fixed per-query cost budget, dynamically allocates the budget between resampling (regenerating responses from the current model) and rerouting (switching to another model) to maximize expected correctness. The authors design an online allocation policy (RoR) driven by estimated marginal correctness per unit cost. Replay experiments on an eleven-model open-weight pool over four benchmarks show that RoR attains a favorable cost-quality Pareto front compared to several baselines, with the largest gains on the most heterogeneous benchmark. Ablations show gains are verifier-gated and robust under different price vectors and label-free verifiers.

## Key points

- First to treat resampling and rerouting as competing uses of a single per-query budget, formulating budget-aware test-time model selection.
- Proposes an online Resample-or-Reroute (RoR) allocation policy based on estimated marginal correctness per unit cost.
- Experimental results on 11 open-weight models over 4 benchmarks show RoR achieves a better cost-quality Pareto front than single-route, one-commit-router, best-of-K, cascade, and random allocation baselines.
- Ablations show gains are verifier-gated and robust under different price vectors and label-free agreement verifiers.

## Full text

Routing among large language models (LLMs) trades response quality against serving cost, motivated by the reported gap between deployed routers and a per-instance oracle. Recent analysis shows that test-time resampling can recover per-instance selection headroom that no single-commit router captures; however, that guarantee holds only under an idealized oracle equipped with correctness labels and an unconstrained budget, neither of which a deployed system has. To the best of our knowledge, no previous work treats resampling the committed model and rerouting to an alternative model as competing uses of a single per-query cost budget. Therefore, this work formulates budget-aware test-time model selection: given a per-query budget and an imperfect verifier, allocate each unit of budget between resampling and rerouting so that expected correctness is maximized. An online resample-or-reroute (RoR) allocation policy driven by estimated marginal correctness per unit cost is proposed, and its behavior is grounded in the recoverability asymmetry between selection and sampling. Replay experiments on newly regenerated multi-draw correctness tensors from an eleven-model open-weight pool over four benchmarks of differing difficulty show that the proposed RoR policy attains a favorable cost-quality Pareto front relative to single-route, one-commit-router, budget-aware best-of-K, cascade, and random-allocation baselines for the tested pools, with the largest gains on the most heterogeneous benchmark; an ablation further shows the gains are verifier-gated, shrinking as verifier quality degrades, and robustness replays under a provider price vector and a label-free agreement verifier delineate where the conclusions carry over.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_86b7e16995ea79a5718d805b