---
title: "ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.41160v1"
permalink: "https://feed.astralor.com/items/itm_4065d0d8d9b295c1c6eebf22"
published: "2026-10-30T02:35:39.000Z"
score: 53
channel: "tech"
tags: ["LLM","推理效率","ARK","Avoiding Routing Collisions","Cache Transfer","Disaggregated LLM Inference","Disaggregated LLM","GPU"]
---

# ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference

## At a glance

Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant…

## Key points

- Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers.
- Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization.
- In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds.

## Full text

Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization. In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds. Yet this same persistence makes coordination practical. Since these flows are long-lived, even lightweight one-to-all coordination can be amortized over their lifetime. We present ARK, a distributed elephant-flow path reservation mechanism. ARK coordinates senders to choose source ports whose hashes map concurrent flows onto distinct spines, without requiring switch changes or receiver-side packet reordering. Packet-level RDMA simulations show that ARK reduces mean and P95 FCT by up to 27.3% and 34.0% under moderate load, and further reduces mean TTFT by up to 12.9%.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_4065d0d8d9b295c1c6eebf22