---
title: "ClinFusion：面向整体医疗理解的视觉中心多模态大语言模型"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.24743v1"
permalink: "https://feed.astralor.com/items/itm_5c64f310358917c68f740e9a"
published: "2026-07-27T17:59:49.000Z"
score: 79
channel: "ai"
tags: ["ClinFusion","MedIF-Bench","医学影像理解","多模态大语言模型","级联空间感知局部融合"]
---

# ClinFusion：面向整体医疗理解的视觉中心多模态大语言模型

## AI 点评

> 这篇论文在医学多模态大模型领域找到了一个极佳的切入点——视觉中心设计。放射科医生盲评的加入让评估结果更有说服力，特别适合关注医疗AI落地和评估范式的从业者。对于研究者，级联编码器的设计和ROI评估指标值得深入借鉴。

## 速览

ClinFusion是一个视觉中心的多模态大语言模型，旨在解决医学影像理解中的异构数据与评估对齐问题。该模型提出级联空间感知局部融合编码器，统一处理2D和3D医学图像，并引入MedIF-Bench指令遵循基准和基于感兴趣区域的报告生成评估方法。在涵盖视觉问答、报告生成和指令遵循的24个基准测试中，ClinFusion在20个上超越开源医学MLLM（如Hulu-Med、Lingshu），在16个上超越GPT-5.2和Gemini-3-Flash等闭源模型。放射科医生盲评确认其报告质量最高，且提出的ROI指标与专家判断相关性最强。该系统还可扩展为代理工具调用，支持检索增强和工具辅助的临床工作流。

## 关键点

- 提出ClinFusion，采用级联空间感知局部融合编码器统一2D/3D医学图像理解。
- 构建MedIF-Bench指令遵循基准和基于感兴趣区域的报告生成评估方法。
- 在20/24个基准上超越开源医学MLLM，在13/16个基准上超越GPT-5.2和Gemini-3-Flash。
- 放射科医生盲评验证ClinFusion报告质量最高，且新评估指标与专家判断相关性最强。

## 全文

Multimodal large language models (MLLMs) hold immense potential to revolutionize clinical practice, yet deploying them in the medical domain is fundamentally a vision-centric challenge: models must absorb knowledge from heterogeneous 2D and 3D medical images, and evaluation protocols must align with radiologists' clinical practice and provide an accurate, fine-grained and factualness-driven assessment. In this paper, we introduce ClinFusion, a vision-centric MLLM designed for holistic medical understanding that systematically addresses these limitations. We propose a compositional and cascaded vision encoder architecture featuring a Cascade Spatial-Aware Locality Fusion operator that unifies diverse 2D and native 3D medical image understanding within a fused encoder. We further introduce a vision-grounded evaluation framework, including MedIF-Bench for instruction-following assessment and a region-of-interest-grounded method for clinically aligned and factualness-driven report generation evaluation. We show that ClinFusion sets a new state-of-the-art across a comprehensive suite of 2D and 3D multimodal medical benchmarks---spanning visual question answering, report generation, and instruction following---as well as textual medical tasks, outperforming leading open-source medical MLLMs (\textit{e.g.}, Hulu-Med, Lingshu) on 20 out of 24 benchmarks and demonstrating multimodal capabilities better than powerful proprietary models such as GPT-5.2 and Gemini-3-Flash on 13 out of 16 benchmarks, and can be further augmented with agentic tool use for retrieval-augmented and tool-assisted clinical workflows. A blinded evaluation by board-certified radiologists confirms that ClinFusion produces the highest-ranked reports, and validates our RoI-grounded metric as achieving the strongest correlation with expert judgment among all automatic evaluation metrics examined.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_5c64f310358917c68f740e9a