---
title: "A Visual Dependence-Aware Framework for Multimodal Unsupervised Continual Post-Training"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.26095v1"
permalink: "https://feed.astralor.com/items/itm_ddb07d388e63b491eac8bca2"
published: "2026-08-26T17:57:04.000Z"
score: 73
channel: "ai"
tags: ["MLLM","Continual Learning","Unsupervised Post-Training","Optimal Transport","Catastrophic Forgetting"]
---

# A Visual Dependence-Aware Framework for Multimodal Unsupervised Continual Post-Training

## AI commentary

> The idea of using visual dependence as a forgetting indicator is quite novel, like adding a compass for continual learning. For engineers working on multimodal deployment and edge adaptation, the optimal transport constraint in VDA is worth attention, though simplifying complexity may be needed before real-world adoption.

## At a glance

This paper explores a novel task of Multimodal Unsupervised Continual Post-Training (MU-CPT), enabling deployed multimodal large language models (MLLMs) to continually evolve from streaming unlabeled data. Existing methods optimize target tokens uniformly, ignoring token-level visual dependence (VD). The authors reveal that structural distortion of VD serves as an indicator of cross-modal catastrophic forgetting, while its inherent heterogeneity guides new-task learning. They propose a Visual Dependence-Aware (VDA) framework with two components: Visually Constrained Optimal Transport (VC-OT) formulates VD structural distortion as an optimal transport problem to mitigate forgetting, and Visually Modulated Adaptation (VMA) exploits VD heterogeneity to promote visually grounded new-task learning. Extensive experiments validate the effectiveness in maintaining old-task stability and new-task plasticity.

## Key points

- Introduces MU-CPT, enabling deployed MLLMs to evolve from unlabeled streaming data
- Reveals that token-level visual dependence (VD) structural distortion indicates cross-modal catastrophic forgetting
- VDA framework comprises VC-OT (visually constrained optimal transport) and VMA (visually modulated adaptation)
- Experiments show VDA maintains both old-task stability and new-task plasticity

## Full text

In this paper, we explore a novel task of Multimodal Unsupervised Continual Post-Training (MU-CPT), enabling deployed MLLMs to continually evolve from streaming unlabeled data. Existing unsupervised post-training methods for MLLMs typically optimize target tokens uniformly, overlooking their heterogeneous visual dependence (VD). However, we reveal that token-level VD is crucial for MU-CPT. Specifically, its structural distortion serves as an indicator of cross-modal catastrophic forgetting, and its inherent heterogeneity acts as a compass to guide new-task learning. Leveraging this property, we propose a Visual Dependence-Aware (VDA) framework with two main components. First, Visually Constrained Optimal Transport (VC-OT) formulates the VD structural distortion of old-task VD during new-task learning as an optimal transport problem to mitigate cross-modal forgetting. By designing a region-aware ground cost and a dependence-stratified transport penalty, it prevents global shifts in visual focus while strictly prohibiting visual reliance from degenerating into language bias. Second, Visually Modulated Adaptation (VMA) exploits VD heterogeneity to emphasize visually grounded new-task learning, promoting new-task plasticity. Together, our method simultaneously maintains old-task stability and new-task plasticity during challenging MU-CPT. Extensive experiments under our MU-CPT setting validate the effectiveness of VDA.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_ddb07d388e63b491eac8bca2