---
title: "InfMasking: A Unified Variational Framework for Synergistic Multimodal Learning"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.07702v1"
permalink: "https://feed.astralor.com/items/itm_ec08194b71ecc5e31bb1def5"
published: "2026-11-05T09:24:22.000Z"
score: 61
channel: "tech"
tags: ["RAG","多模态","InfMasking","Unified Variational Framework","Synergistic Multimodal Learning","Multimodal","Infinite Masking","Before"]
---

# InfMasking: A Unified Variational Framework for Synergistic Multimodal Learning

## 速览

Multimodal models are expected to exploit synergistic information, that is, information available only when multiple modalities are observed together. In practice, however, a fusion module can fit the training objective…

## 关键点

- Multimodal models are expected to exploit synergistic information, that is, information available only when multiple modalities are observed together.
- In practice, however, a fusion module can fit the training objective by relying on a dominant modality or on information that modalities share, and the synergy between modalities is often left unused.
- We propose InfMasking, a unified multimodal learning framework based on Infinite Masking.

## 全文

Multimodal models are expected to exploit synergistic information, that is, information available only when multiple modalities are observed together. In practice, however, a fusion module can fit the training objective by relying on a dominant modality or on information that modalities share, and the synergy between modalities is often left unused. We propose InfMasking, a unified multimodal learning framework based on Infinite Masking. Before fusion, InfMasking randomly masks the features of each modality; each sampled mask produces a different partial combination of cross-modal evidence and a fused representation of the corresponding modality interactions. All masked fused representations are required to remain consistent with a reference target: the unmasked fused view when training is unsupervised, and the class proxy of the task label when labels are available. Maintaining this consistency across diverse partial combinations discourages reliance on a dominant modality or on shared information alone, and drives the fusion module to learn cross-modal interaction patterns that remain meaningful under partial observations. Since the objective is an expectation over infinitely many masks, we model the masked fused representations as Gaussian and optimize closed-form variational bounds, whose population-moment gaps are controlled by the masking-induced covariance under this model. On the evaluated benchmarks, InfMasking improves predictive performance and yields higher mean PID estimates of task-label synergy in learned modality-specific representations, alongside higher InterSHAP interaction attribution in the trained predictors. Code is available at https://github.com/brightest66/InfMasking.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_ec08194b71ecc5e31bb1def5