---
title: "在线神经时空记忆：动态新视角合成的实时记忆机制"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.15271v1"
permalink: "https://feed.astralor.com/items/itm_2064eae62ccac994bdf35ff3"
published: "2026-07-16T17:58:18.000Z"
score: 77
channel: "tech"
tags: ["Test-Time Training","动态新视角合成","实时渲染","神经时空记忆"]
---

# 在线神经时空记忆：动态新视角合成的实时记忆机制

## AI 点评

> 这篇论文切中了实时新视角合成的脉搏，解耦更新频率的思路既巧妙又实用，特别适合对延迟敏感的应用场景。如果你在做AR/VR或动态场景重建，这可能是你下半年最值得关注的记忆机制之一。

## 速览

针对多视角流视频的在线新视角合成，提出一种解耦记忆更新与应用的框架，通过周期性记忆更新和逐帧记忆应用，结合交叉视图注意力处理形变，并引入记忆损失和缓存策略锁定历史上下文，实现实时、高保真动态场景重建。

## 关键点

- 提出解耦记忆更新与应用的频率，以降低计算成本并满足实时性。
- 交叉视图注意力处理历史状态与当前帧间的形变。
- 引入记忆损失和记忆缓存机制，防止长期记忆漂移。
- 在动态人体运动和分钟级在线记忆场景中达到实时、最优效果。

## 全文

Online novel view synthesis from multi-view streaming videos faces a fundamental trade-off: maintaining a persistent, long-horizon memory to reconstruct temporarily occluded regions while operating under strict real-time constraints. While Test-Time Training (TTT) offers a powerful memory mechanism, standard models mandate gradient-based memory updates at every frame to adapt to the changing motion in dynamic scenes. The computational cost of heavy memory updates precludes real-time application and can lead to instability over long contexts. Given that memory updates are more demanding than memory application and video content is largely redundant, we propose to decouple the frequencies of these two processes. Our approach performs periodic memory updates while applying the memory on a per-frame basis, using cross-view attention to manage deformations between the prior memory state and the current frame. To lock in the historical context, we introduce two critical mechanisms: an auxiliary Memory Loss that forces persistent internalization of the scene, and a Memory Caching strategy that regularizes active weights against catastrophic drift. Our method demonstrates real-time, state-of-the-art performance on scenes with dynamic human motion as well as minute-scale online memorization.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_2064eae62ccac994bdf35ff3