---
title: "多模态无监督持续后训练中的视觉依赖感知框架"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.26095v1"
permalink: "https://feed.astralor.com/items/itm_ddb07d388e63b491eac8bca2"
published: "2026-08-26T17:57:04.000Z"
score: 73
channel: "ai"
tags: ["多模态大语言模型","持续学习","无监督后训练","最优传输","灾难性遗忘"]
---

# 多模态无监督持续后训练中的视觉依赖感知框架

## AI 点评

> 这篇把视觉依赖当遗忘信号的想法很新颖，相当于给持续学习装了“风向标”。对做多模态部署和边缘自适应的工程师来说，VDA中的最优传输约束值得关注，不过落地前可能得先简化复杂度。

## 速览

本文提出多模态无监督持续后训练（MU-CPT）新任务，使已部署的多模态大语言模型（MLLM）能持续从无标注流式数据中进化。现有方法统一优化目标词元，忽视了词元级视觉依赖（VD）的差异。作者发现VD的结构失真可指示跨模态灾难性遗忘，其固有异质性可引导新任务学习。为此设计VDA框架：视觉约束最优传输（VC-OT）将旧任务VD结构失真建模为最优传输问题，缓解跨模态遗忘；视觉调制适应（VMA）利用VD异质性强调视觉锚定的新任务学习。实验验证了框架在保持旧任务稳定性和新任务可塑性上的有效性。

## 关键点

- 提出MU-CPT任务，让已部署MLLM从无标注流式数据持续进化
- 揭示词元级视觉依赖（VD）的结构失真可作为跨模态灾难性遗忘的指示器
- VDA框架包含VC-OT（约束最优传输）与VMA（视觉调制适应）两大组件
- 实验证明VDA能同时保持旧任务稳定性与新任务可塑性

## 全文

In this paper, we explore a novel task of Multimodal Unsupervised Continual Post-Training (MU-CPT), enabling deployed MLLMs to continually evolve from streaming unlabeled data. Existing unsupervised post-training methods for MLLMs typically optimize target tokens uniformly, overlooking their heterogeneous visual dependence (VD). However, we reveal that token-level VD is crucial for MU-CPT. Specifically, its structural distortion serves as an indicator of cross-modal catastrophic forgetting, and its inherent heterogeneity acts as a compass to guide new-task learning. Leveraging this property, we propose a Visual Dependence-Aware (VDA) framework with two main components. First, Visually Constrained Optimal Transport (VC-OT) formulates the VD structural distortion of old-task VD during new-task learning as an optimal transport problem to mitigate cross-modal forgetting. By designing a region-aware ground cost and a dependence-stratified transport penalty, it prevents global shifts in visual focus while strictly prohibiting visual reliance from degenerating into language bias. Second, Visually Modulated Adaptation (VMA) exploits VD heterogeneity to emphasize visually grounded new-task learning, promoting new-task plasticity. Together, our method simultaneously maintains old-task stability and new-task plasticity during challenging MU-CPT. Extensive experiments under our MU-CPT setting validate the effectiveness of VDA.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_ddb07d388e63b491eac8bca2