---
title: "ReWorld：具备长时记忆的交互式世界模型"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.23565v1"
permalink: "https://feed.astralor.com/items/itm_7643e600485b77ec1eef8ee1"
published: "2026-08-24T17:59:05.000Z"
score: 78
channel: "ai"
tags: ["LoRA","ReWorld","世界模型","交互式视频生成","长时记忆"]
---

# ReWorld：具备长时记忆的交互式世界模型

## AI 点评

> 这篇把控制与记忆的矛盾摆到台面上，并用工程技巧给出了可运行解。对做交互视频生成或具身智能的人来说，位姿索引地标库和四步采样组合尤其值得参考；但预印本的评测协议仍需用更多开放场景复验。

## 速览

ReWorld 提出一种面向实时交互的世界模型，通过混合感知头注意力窗口将控制与记忆解耦，并用随机头路由与随机块丢弃让稀疏历史保持分布内。推理时以固定预算 KV 缓存配合位姿索引地标库，在 64 秒长回环中仍能还原起始视角。数据引擎统一八类来源的动作尺度，LoRA 蒸馏压缩采样至 4 步，支持 704x1280 视频流式生成，在六款最新模型中控制保真度与生成质量最佳。

## 关键点

- 控制与记忆解耦：多数注意力头限定近期，少数全局头覆盖全历史，随机头路由避免能力绑定到固定头。
- 固定预算长时记忆：位姿索引地标库配合有界 KV 缓存，64 秒/384 latent 回环测试仍能重建起始视角。
- 统一动作尺度数据引擎：八类数据源对齐到同一物理动作尺度，回文轨迹为记忆训练提供重访证据。
- 四步采样实时推理：LoRA 蒸馏让同一骨干同时支持高保真多步与实时交互模式，输出 704x1280 视频。

## 全文

An interactive world model must follow the user's actions, remember the places it has shown, and stream in real time. The tension is structural: control wants a short horizon, memory wants an unbounded one. ReWorld separates the two during training and bounds them at inference. Mixed per-head attention windows confine most heads to the recent past while a small set of global heads attends over the entire history, and random head routing keeps either capability from binding to particular heads; random chunk dropping makes sparse histories in-distribution. At inference the whole past lives under a fixed budget: a bounded KV cache backed by a pose-indexed landmark bank, from which the model retrieves the landmarks nearest the current pose. A metric-scale-aligned data engine places eight sources -- Unreal-rendered fly-throughs, game roaming, and real-world footage -- on one physical action scale, so the same key press moves the camera the same distance in every source, and palindrome trajectories supply the revisit evidence that memory training needs. Distribution-matching distillation confined to a LoRA adapter then compresses sampling to four steps: one backbone serves both a high-fidelity multi-step mode and a real-time interactive one, streaming 704x1280 video across photorealistic, game-style, and stylized worlds. Under a three-axis protocol covering action following, long-horizon recall, and video quality, against six recent interactive world models it attains the best control fidelity ($11.95^\circ$ rotation error and the best camera-motion consistency) and the best generation quality; and on minute-long out-and-back rollouts ($64$\,s, $384$ latents), its fixed 12-chunk cache still regenerates the starting view -- at rollout lengths where a sliding window has long evicted the evidence and full-KV attention runs out of memory.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_7643e600485b77ec1eef8ee1