---
title: "DenseTwins: Dense Modality Decoupling for Multimodal Self-Supervision"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.04710v1"
permalink: "https://feed.astralor.com/items/itm_43f3466ed08708d85387ba26"
published: "2026-11-03T11:31:33.000Z"
score: 59
channel: "tech"
tags: ["推理效率","安全","多模态","DenseTwins","Dense Modality Decoupling","Multimodal Self-Supervision Multimodal","Self-supervised","Multimodal"]
---

# DenseTwins: Dense Modality Decoupling for Multimodal Self-Supervision

## 速览

Multimodal perception is central to many computer vision applications, including robotics and autonomous driving, where different sensing modalities provide complementary cues. Self-supervised learning offers a scalable…

## 关键点

- Multimodal perception is central to many computer vision applications, including robotics and autonomous driving, where different sensing modalities provide complementary cues.
- Self-supervised learning offers a scalable solution for exploiting large amounts of unlabeled multimodal data.
- However, existing methods either neglect modality-specific information by enforcing feature-space alignment between modalities or learn only global representations without preserving spatial structure.

## 全文

Multimodal perception is central to many computer vision applications, including robotics and autonomous driving, where different sensing modalities provide complementary cues. Self-supervised learning offers a scalable solution for exploiting large amounts of unlabeled multimodal data. However, existing methods either neglect modality-specific information by enforcing feature-space alignment between modalities or learn only global representations without preserving spatial structure. We propose DenseTwins, a multimodal self-supervised framework that learns dense representations by explicitly decoupling the dense feature space into shared and modality-specific components. The proposed approach combines dense cross-modal correspondences with redundancy-reduction objectives, enabling the model to capture both shared semantics and modality-specific cues within a structured dense embedding space. DenseTwins jointly optimizes an intra-modal objective across augmented views and a cross-modal objective that aligns shared features while decorrelating modality-specific information. We evaluate DenseTwins across multiple multimodal benchmarks and sensing setups, achieving state-of-the-art or competitive performance across multimodal self-supervised baselines. Code available at: https://github.com/CarloSgaravatti/DenseTwins.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_43f3466ed08708d85387ba26