---
title: "Pattern over Pixels：测量多模态代码生成中的模式补全偏差"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.03691v1"
permalink: "https://feed.astralor.com/items/itm_87d4bd0e4730e3a09f156b3f"
published: "2026-08-04T13:59:12.000Z"
score: 73
channel: "tech"
tags: ["arXiv","Design2Code","代码生成","多模态大语言模型","模式补全偏差"]
---

# Pattern over Pixels：测量多模态代码生成中的模式补全偏差

## AI 点评

> 这篇论文最值得关注的地方是它把“看起来合理”的模型输出量化成可测的偏差率，结果非常扎眼。对做前端生成产品或评测集的人来说，它是一个清晰的红旗：模型可能在重复模式面前“睁眼瞎”。我建议后续工作把偏差率当作模型卡的一项常规指标，否则这类问题很容易被漂亮的 demo 掩盖。

## 速览

多模态大语言模型（MLLM）常被用于将网页截图转换为前端代码，但重复的 UI 模式可能诱使模型生成视觉错误却与模式一致的输出。本文首次提出视觉模式补全偏差基准，基于 Design2Code 的 30 个网页构建 1440 张带扰动截图，要求模型从截图与 HTML 上下文中恢复被掩码的宽度或字号。评估五个前沿 MLLM 后发现，所有模型均强烈偏向重复基线：卡片宽度扰动平均偏差率 69.78%，文本字号扰动达 80.22%，而平均准确率分别仅为 21.17% 和 7.89%。噪声、更微妙的扰动与边界位置进一步加剧偏差。推理分析显示更多推理努力与更低偏差相关，但定性证据表明模型即使识别出异常元素，仍会用模式一致答案覆盖。该研究揭示了多模态代码生成中的具体失败模式，且严重性与视觉显著性密切相关。

## 关键点

- 首个针对视觉模式补全偏差的基准：基于 Design2Code 构建 1,440 张截图，覆盖卡片宽度与文本字号两类模式。
- 五个前沿 MLLM 均表现出严重偏差，文本字号扰动平均准确率仅 7.89%，而偏差率高达 80.22%。
- 噪声、微妙扰动和边界位置会进一步提高偏差率；更多推理努力虽与更低偏差相关，但模型即使发现异常仍会覆盖正确答案。

## 全文

Multimodal large language models (MLLMs) are increasingly used to translate webpage screenshots into front-end code, but repeated UI patterns may sway them toward visually incorrect yet pattern-consistent outputs. In this work, we test how repeated webpage patterns hurt MLLM accuracy on an objective screenshot-to-code fill-in-the-blank task. We introduce the first benchmark for visual pattern-completion bias, where one localized element in a repeated UI pattern is perturbed and the model must recover the masked width or font-size value from the screenshot and HTML context. Starting from 30 webpages curated from the Design2Code dataset, we build 1,440 evaluated screenshots spanning structural card and text-style patterns under standard and noise-overlaid conditions. We evaluate five frontier MLLMs and find that all are strongly biased toward the repeated baseline. Mean bias rate reaches 69.78% on card-width perturbations and 80.22% on text font-size perturbations, while mean accuracy is only 21.17% and 7.89%, respectively. Codex-5.3 performs best but still drops from 68.61% accuracy on cards to 13.89% on text, while Flash-3.0 reaches 96.11% bias on text. Noise, subtler perturbations, and boundary positions further increase bias rate. Reasoning analysis further shows that greater reasoning effort correlates with lower bias, yet qualitative evidence reveals that models can identify the anomalous element and still override it with the pattern-consistent answer. Our results identify a concrete failure mode in multimodal code generation and show that its severity is strongly associated with visual saliency

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_87d4bd0e4730e3a09f156b3f