---
title: "Ripple: Deadline-Aware CPU-GPU Sharing for Embodied Inference"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.01067v1"
permalink: "https://feed.astralor.com/items/itm_12d3698f4823d2db2ff5b0a0"
published: "2026-10-31T16:06:36.000Z"
score: 54
channel: "tech"
tags: ["LLM","推理效率","Ripple","Deadline-Aware CPU-GPU Sharing","Embodied Inference Unified-memory","VLM","CPU","GPU"]
---

# Ripple: Deadline-Aware CPU-GPU Sharing for Embodied Inference

## At a glance

Unified-memory edge devices increasingly run LLM/VLM inference alongside time-critical CPU tasks for perception and control. GPU memory traffic can overwhelm these tasks, yet platforms such as DGX Spark expose neither b…

## Key points

- Unified-memory edge devices increasingly run LLM/VLM inference alongside time-critical CPU tasks for perception and control.
- GPU memory traffic can overwhelm these tasks, yet platforms such as DGX Spark expose neither bandwidth counters nor direct bandwidth regulation.
- We present Ripple, a CPU-GPU co-scheduler that combines model-derived GPU memory volumes with CPU completion feedback to regulate interference.

## Full text

Unified-memory edge devices increasingly run LLM/VLM inference alongside time-critical CPU tasks for perception and control. GPU memory traffic can overwhelm these tasks, yet platforms such as DGX Spark expose neither bandwidth counters nor direct bandwidth regulation. We present Ripple, a CPU-GPU co-scheduler that combines model-derived GPU memory volumes with CPU completion feedback to regulate interference. Ripple divides inference into pausable, layer-based units and translates each CPU job's deadline slack into a GPU-byte budget, admitting work within the budgets of all overlapping jobs. Online updates track changing interference, while deadline-aware pacing and judicious SM footprint narrowing preserve inference progress. Implemented with SGLang engine hooks and a central scheduler, we evaluate Ripple on DGX Spark with four robotic CPU task mixes and LLM/VLM inference. It is the only evaluated approach that consistently meets the 2% CPU deadline-miss target and VLM deadlines across the tested deadline settings while retaining best-effort LLM throughput.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_12d3698f4823d2db2ff5b0a0