---
title: "FlashRT：引导AI Agent部署实时多模态应用的智能框架"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.18171v1"
permalink: "https://feed.astralor.com/items/itm_acbcd7090d3a031741b46f62"
published: "2026-07-20T17:12:28.000Z"
score: 80
channel: "ai"
tags: ["Agent驱动优化","FlashRT","vLLM-Omni","多模态应用"]
---

# FlashRT：引导AI Agent部署实时多模态应用的智能框架

## AI 点评

> 这篇论文最吸引我的是它用通用Agent替代了手工调优，尤其在AMD等非NVIDIA平台上效果更好。对于构建实时语音或视频生成管线的开发者来说，能大幅节省调优时间。值得关注其后续开源实现。

## 速览

FlashRT是一个Agent harness框架，通过多轮变换、中间表示验证和测量门控优化循环，引导通用编码Agent将简单的参考实现转化为高效的多GPU部署。在NVIDIA B200上实现高达70倍延迟降低和2.8倍吞吐提升；在AMD MI355X上延迟降低持平，吞吐提升达3.6倍，且对Qwen3-Omni文本转语音推理相比专家实现vLLM-Omni延迟降低65%。

## 关键点

- 提出链式程序（Chain-of-Program）范式，通过多轮变换将参考实现转化为中间表示并验证。
- 采用测量门控优化循环，自动迭代部署候选并基准测试，无需人工手动调优。
- 在NVIDIA B200和AMD MI355X上均显著超越专家实现，尤其在硬件生态不成熟的平台上效果更佳。
- 以Qwen3-Omni为例，将延迟从专家实现降低65%，展示了Agent驱动优化的通用性。

## 全文

Real-time multimodal applications, including voice agents and interactive video generation, compose heterogeneous models into pipelines whose efficient deployment requires application-specific decisions about placement, streaming, and intra-model parallelism. Existing serving systems and auto-parallelism compilers commit to limited transformations and fixed workload assumptions, so achieving high performance on a new application requires hand-crafting an efficient implementation. We present FlashRT, an agent harness that guides coding agents to lift simple developer-written reference implementations into optimized multi-GPU deployments that flexibly weigh target metrics like latency and throughput. Using a new chain-of-program paradigm, FlashRT directs a generic coding agent through a multi-pass transformation process where an agent transforms the reference into an intermediate representation (IR) to capture data dependencies and persistent-state scopes, validates this IR via a sequential interpreter, and performs static analyses to identify candidate transformations. Then, the agent iteratively implements, verifies, and benchmarks each candidate under a measurement-gated optimization loop to produce effective deployments that span different hardware budgets. Across various applications, including video world models and multimodal LLMs, FlashRT converts reference implementations into highly efficient deployments, delivering up to ~70x latency reduction and 2.8x throughput improvement on NVIDIA B200 GPUs. On AMD MI355X GPUs, FlashRT matches the peak latency reduction while increasing peak throughput improvement to 3.6x, demonstrating that agent-driven optimization can be more scalable on platforms with less mature expert optimization. In fact, for Qwen3-Omni text-to-audio inference, FlashRT reduces response latency by 65% compared to the expert vLLM-Omni implementation on AMD MI355X.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_acbcd7090d3a031741b46f62