---
title: "ReWorld: An Interactive World Model with Long-Horizon Memory"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.23565v1"
permalink: "https://feed.astralor.com/items/itm_7643e600485b77ec1eef8ee1"
published: "2026-08-24T17:59:05.000Z"
score: 78
channel: "ai"
tags: ["LoRA","ReWorld","World Model","Interactive Video Generation","Long-Horizon Memory"]
---

# ReWorld: An Interactive World Model with Long-Horizon Memory

## AI commentary

> This paper puts the control-memory tension on the table and offers a practical engineering solution. Teams working on interactive video generation or embodied AI should look closely at the pose-indexed landmark bank and four-step sampling; the preprint protocol, however, still deserves broader validation in open-world scenarios.

## At a glance

ReWorld introduces an interactive world model that separates control from long-horizon memory during training and bounds both at inference. Most attention heads are confined to the recent past while a few global heads attend over the full history; random head routing and random chunk dropping keep sparse histories in distribution. At inference, a fixed-budget KV cache backed by a pose-indexed landmark bank retrieves landmarks nearest the current pose, allowing a 12-chunk cache to regenerate the starting view on 64-second out-and-back rollouts where sliding windows have lost evidence and full-KV attention runs out of memory. A metric-scale-aligned data engine unifies eight sources, and LoRA-based distillation compresses sampling to four steps for real-time 704x1280 streaming. ReWorld achieves the best control fidelity and generation quality against six recent interactive world models.

## Key points

- Separation of control and memory: most attention heads look at the recent past, a few global heads cover the full history, and random head routing prevents either capability from binding to fixed heads.
- Bounded long-horizon memory: a pose-indexed landmark bank backed by a fixed-budget KV cache regenerates the starting view on 64-second/384-latent out-and-back rollouts.
- Metric-scale-aligned data engine: eight data sources share one physical action scale, and palindrome trajectories supply revisit evidence for memory training.
- Four-step real-time inference: LoRA-based distillation lets one backbone serve both high-fidelity multi-step generation and real-time interactive 704x1280 streaming.

## Full text

An interactive world model must follow the user's actions, remember the places it has shown, and stream in real time. The tension is structural: control wants a short horizon, memory wants an unbounded one. ReWorld separates the two during training and bounds them at inference. Mixed per-head attention windows confine most heads to the recent past while a small set of global heads attends over the entire history, and random head routing keeps either capability from binding to particular heads; random chunk dropping makes sparse histories in-distribution. At inference the whole past lives under a fixed budget: a bounded KV cache backed by a pose-indexed landmark bank, from which the model retrieves the landmarks nearest the current pose. A metric-scale-aligned data engine places eight sources -- Unreal-rendered fly-throughs, game roaming, and real-world footage -- on one physical action scale, so the same key press moves the camera the same distance in every source, and palindrome trajectories supply the revisit evidence that memory training needs. Distribution-matching distillation confined to a LoRA adapter then compresses sampling to four steps: one backbone serves both a high-fidelity multi-step mode and a real-time interactive one, streaming 704x1280 video across photorealistic, game-style, and stylized worlds. Under a three-axis protocol covering action following, long-horizon recall, and video quality, against six recent interactive world models it attains the best control fidelity ($11.95^\circ$ rotation error and the best camera-motion consistency) and the best generation quality; and on minute-long out-and-back rollouts ($64$\,s, $384$ latents), its fixed 12-chunk cache still regenerates the starting view -- at rollout lengths where a sliding window has long evicted the evidence and full-KV attention runs out of memory.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_7643e600485b77ec1eef8ee1