---
title: "Spoofing the Latent State: Imperceptible Goal-Image Attacks Derail Planning in LeWorldModel"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.00427v1"
permalink: "https://feed.astralor.com/items/itm_4df824cd674a66a09f52a34c"
published: "2026-10-30T22:20:51.000Z"
score: 58
channel: "ai"
tags: ["AI Agent","Spoofing","Latent State","Imperceptible Goal-Image Attacks","Derail Planning","LeWorldModel Joint-embedding","JEPAs","LeWorldModel"]
---

# Spoofing the Latent State: Imperceptible Goal-Image Attacks Derail Planning in LeWorldModel

## 速览

Joint-embedding predictive architectures (JEPAs) have recently emerged as general-purpose visual world models for control. A prominent example is LeWorldModel (LeWM), which trains end-to-end from pixels using an isotrop…

## 关键点

- Joint-embedding predictive architectures (JEPAs) have recently emerged as general-purpose visual world models for control.
- A prominent example is LeWorldModel (LeWM), which trains end-to-end from pixels using an isotropic Gaussian anti-collapse regularizer (SIGReg) and plans by minimizing the squared Euclidean distance between predicted and…
- In this work, we investigate the test-time robustness of this latent planning interface.

## 全文

Joint-embedding predictive architectures (JEPAs) have recently emerged as general-purpose visual world models for control. A prominent example is LeWorldModel (LeWM), which trains end-to-end from pixels using an isotropic Gaussian anti-collapse regularizer (SIGReg) and plans by minimizing the squared Euclidean distance between predicted and goal embeddings. In this work, we investigate the test-time robustness of this latent planning interface. Analyzing the learned metric on Push-T, we first find that task-relevant object state heavily dominates latent distance: shifting the block by a single pixel displaces the embedding $3\times$ further than an equivalent agent shift, and swapping the block pose alone accounts for $99\%$ of the distance between scene pairs. However, because this pose information is distributed across a curved, high-dimensional manifold rather than a fixed linear subspace, the encoder cannot be trivially edited post-hoc to isolate or redirect it. We next demonstrate that this geometry is highly vulnerable at test time. A white-box $\ell_\infty$ perturbation of just $ε{=}4/255$ applied to a single input frame causes the encoder to register the block at an arbitrary, attacker-chosen pose in $73$--$95\%$ of scenes without disturbing the encoded agent state. Perturbing either the goal image or the live camera observation drops Push-T planning success from $87\%$ to $32\%$, whereas uniform random noise at twice the magnitude has negligible impact. This vulnerability persists across all four standard LeWM benchmarks (Push-T, Two-Room, Reacher, and OGBench-Cube). At $ε{=}8/255$, the attack bridges $82$--$97\%$ of the squared latent distance to the target embedding in every environment, inducing planning degradation proportional to each task's reliance on the spoofed entity.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_4df824cd674a66a09f52a34c