---
title: "MISA-T: Scheduling Mixed RL Rollouts Beyond Prefix Locality"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.11152v1"
permalink: "https://feed.astralor.com/items/itm_e5d2ca5b48eb91c2d528c149"
published: "2026-08-11T17:10:50.000Z"
score: 76
channel: "ai"
tags: ["KV-cache scheduling","MISA-T","rollout scheduling","vLLM","RL Post-training"]
---

# MISA-T: Scheduling Mixed RL Rollouts Beyond Prefix Locality

## AI commentary

> This paper deserves attention from teams building RL infrastructure, especially those hitting inference bottlenecks in large-scale RL training. MISA-T couples KV-cache capacity scheduling with workload mixture control, addressing the real pain point of 'rollout competing for KV' rather than pure theoretical optimization. If you run vLLM or similar frameworks for multi-task RL post-training, this idea can directly inspire router improvements.

## At a glance

Modern RL post-training pipelines for LLMs increasingly combine heterogeneous rollout workloads such as RLVR, RLHF, and agentic rollouts. While prefix-aware routing improves inference efficiency through cache reuse and load balancing, it does not control how heterogeneous sessions compete for KV-cache capacity. This paper presents MISA-T, a routing-layer admission policy with adaptive session admission, workload-aware KV-capacity allocation, and residency-time-aware KV accounting. In rollout-only ablations on Step3.7 and Qwen3.6-35B-A3B, MISA-T improves rollout throughput over a sweep-tuned cache-aware vLLM Router by 53.3% and 43.6%, respectively. In a matched 50-iteration Step3.7 experiment, it increases rollout throughput by 35.6% and reduces mean iteration time by 22.8% while keeping the workload mixture close to the trainer target and achieving comparable task scores.

## Key points

- MISA-T is a routing-layer admission policy for mixed RL rollouts, addressing heterogeneous sessions' competition for KV-cache capacity.
- It combines adaptive session admission with workload-aware KV-capacity allocation, avoiding distortion of the trainer-specified workload mixture.
- It improves rollout throughput by 53.3% on Step3.7 and 43.6% on Qwen3.6-35B-A3B over a tuned vLLM Router.
- In a full 50-iteration experiment, it boosts throughput by 35.6%, cuts mean iteration time by 22.8%, while keeping task scores comparable.

## Full text

Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasingly combine rollout workloads across multiple domains and feedback paradigms. Prefix-aware routing improves inference efficiency through cache reuse and load balancing, but it does not control how heterogeneous rollout sessions compete for KV-cache capacity. When reinforcement learning with verifiable rewards (RLVR), reinforcement learning from human feedback (RLHF), and agentic rollouts share an asynchronous inference service, their distinct sequence structures, interaction patterns, and KV-residency times create substantially different serving demands. Rollout scheduling must account for this heterogeneity without distorting the workload mixture specified by the trainer. We present MISA-T, a routing-layer admission policy for mixed rollout serving. MISA-T combines adaptive session admission, workload-aware KV-capacity allocation, and residency-time-aware KV accounting. In rollout-only ablations on Step3.7 and Qwen3.6-35B-A3B, MISA-T improves rollout throughput over a sweep-tuned cache-aware vLLM Router by 53.3% and 43.6%, respectively, while maintaining high prefix-cache hit rates. In a matched 50-iteration Step3.7 experiment, it increases rollout throughput by 35.6% and reduces mean iteration time by 22.8%, while keeping the consumed workload mixture close to the trainer target and achieving comparable task scores.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_e5d2ca5b48eb91c2d528c149