---
title: "Before We Adapt: The Zero-Shot Generalization Bottleneck in Continual World Models"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.41041v1"
permalink: "https://feed.astralor.com/items/itm_226d769c3de32c4a8bdaca49"
published: "2026-10-29T22:32:09.000Z"
score: 55
channel: "finance"
tags: ["AI Agent","Before","Adapt","The Zero-Shot Generalization","Bottleneck","Continual World Models","Continual","CRL"]
---

# Before We Adapt: The Zero-Shot Generalization Bottleneck in Continual World Models

## At a glance

Continual reinforcement learning (CRL) requires agents to operate under environmental changes while reusing prior knowledge whenever possible. We argue that model-based reinforcement learning provides a promising framew…

## Key points

- Continual reinforcement learning (CRL) requires agents to operate under environmental changes while reusing prior knowledge whenever possible.
- We argue that model-based reinforcement learning provides a promising framework for this setting because learned world models (WMs) can, in principle, support change detection, knowledge reuse, adaptation, and acquisiti…
- However, such benefits depend heavily on whether the learned model can generalize beyond the contexts in which it was trained.

## Full text

Continual reinforcement learning (CRL) requires agents to operate under environmental changes while reusing prior knowledge whenever possible. We argue that model-based reinforcement learning provides a promising framework for this setting because learned world models (WMs) can, in principle, support change detection, knowledge reuse, adaptation, and acquisition of new knowledge. However, such benefits depend heavily on whether the learned model can generalize beyond the contexts in which it was trained. In this work, we propose to formulate this problem with a contextual Markov decision process in which context is decomposed into behavioral and physical components. This view organizes continual WM learning into reuse, adaptation, and acquisition of new knowledge. As a first step, we study direct reuse when behavioral rules remain unchanged. Using Dyna-Q and TD-MPC in controlled FourRooms tasks, we find that zero-shot reuse can degrade substantially under physical-context changes and that this behavior depends strongly on the observation representation used by the WM. Additional ablation studies and latent-space diagnostics suggest that learned representations do not automatically provide the invariances needed for reuse. These results identify representation generalization as an important bottleneck for reliable world-model reuse in CRL.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_226d769c3de32c4a8bdaca49