---
title: "DolphinRouting: Cluster-Level Binary Routing for Disaggregated LLM Inference Serving"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.40292v1"
permalink: "https://feed.astralor.com/items/itm_eecf685d189c6ab6903196e2"
published: "2026-10-29T15:42:05.000Z"
score: 52
channel: "tech"
tags: ["LLM","推理效率","DolphinRouting","Cluster-Level Binary Routing","Disaggregated LLM Inference","Serving Prefill-decode","Prefill-decode","SLO"]
---

# DolphinRouting: Cluster-Level Binary Routing for Disaggregated LLM Inference Serving

## At a glance

Prefill-decode (PD) disaggregation isolates the two phases of large language model serving, but request-level path selection must account for prefill work, cache reuse, and decode demand. We present DolphinRouting, whic…

## Key points

- Prefill-decode (PD) disaggregation isolates the two phases of large language model serving, but request-level path selection must account for prefill work, cache reuse, and decode demand.
- We present DolphinRouting, which learns this choice from end-to-end SLO feedback under a given P/D resource allocation.
- Requests are clustered by raw prompt length, cache-adjusted input lengths on both worker pools, and output-length information.

## Full text

Prefill-decode (PD) disaggregation isolates the two phases of large language model serving, but request-level path selection must account for prefill work, cache reuse, and decode demand. We present DolphinRouting, which learns this choice from end-to-end SLO feedback under a given P/D resource allocation. Requests are clustered by raw prompt length, cache-adjusted input lengths on both worker pools, and output-length information. Offline policy search learns one binary action per cluster. Online feature extraction, cluster assignment, and table lookup select local or remote prefill. We evaluate DolphinRouting on a real PD-disaggregated cluster using three public benchmarks, a mixed workload, and real data. Across the mixed workload and real data, DolphinRouting with $k$-means reduces P90 TTFT by 5.7--7.9\% and P90 TPOT by 2.6--10.1\% relative to Dynamo.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_eecf685d189c6ab6903196e2