---
title: "Diagram-MMU：面向科学图示解析与理解的多模态基准"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.12262v1"
permalink: "https://feed.astralor.com/items/itm_1d3dd48b7cf9c029b557b8d2"
published: "2026-08-12T17:04:13.000Z"
score: 72
channel: "tech"
tags: ["Diagram-MMU","LaTeX/TikZ","OpenAI Prism","多模态大模型","智能体评估","科学图示理解"]
---

# Diagram-MMU：面向科学图示解析与理解的多模态基准

## AI 点评

> 这份基准最值得关注的地方在于它把评测视角从“看图问答”拉到了“看图写代码”，这正是科学写作工具实际使用中最卡脖子的环节。对研究多模态智能体和科学文档自动化的人来说，Diagram-MMU 提供了一个清晰的难度分层和失败模式参考。我个人很期待后续模型在图转码任务上的提升曲线，毕竟代码生成比选择题更能反映深层理解。

## 速览

Diagram-MMU 是一个面向科学图示多模态理解的新基准，包含 3.7 千个精选图示和 1.83 万个经人工验证的问题，覆盖六个领域。基准设计了三类任务：图示到代码解析、图示到代码编辑和图示问答，并针对每类任务设定了智能体设置。对 12 个多模态大模型的评估显示，图示到代码任务比图示问答更具挑战性，模型虽能较好理解图示，但在解析和编辑方面存在明显短板。在智能体设置下，多数模型解析与编辑性能提升，但问答性能下降，Claude-4.6 Opus 则是唯一在三类任务上均持续改进的模型。该基准有望推动 MLLM 在科学写作与协作场景中的能力提升。

## 关键点

- Diagram-MMU 包含 3.7K 图示与 18.3K 问题，覆盖六个科学领域。
- 三类核心任务：图示到代码解析、编辑与图示问答，均含智能体设置。
- 评估 12 个多模态大模型，图示到代码任务比问答难度更高。
- Claude-4.6 Opus 是唯一在智能体设置下三类任务全部提升的模型。

## 全文

Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration. For example, OpenAI Prism is a free workspace for scientific writing and collaboration. One important feature in Prism is turning scientific diagrams directly into LaTeX TikZ code. In this paper, we build a benchmark, Diagram-MMU, a multi-modal benchmark designed to assess MLLMs' ability for scientific diagram parsing and understanding. Diagram-MMU features 3.7k curated diagrams and 18.3k human-validated questions across six domains. It evaluates MLLMs on three tasks common in vibe writing workspaces: diagram-to-code parsing, diagram-to-code editing, and diagram question answering, alongside agentic settings per task. The evaluation of 12 MLLMs reveals that diagram-to-code tasks are more challenging than diagram question answering: models can reason well over diagrams but struggle to parse and edit them, underscoring the need for methods to enhance MLLMs' capability in diagram-to-code generation. Under agentic settings, most models improve parsing and editing performance but degrade on question answering, while Claude-4.6 Opus consistently improves across all three tasks. Project Page: https://vi-ocean.github.io/projects/diagram-mmu.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_1d3dd48b7cf9c029b557b8d2