---
title: "Desktop-Delta Bench：计算机使用模型能否理解桌面GUI转换？"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.26041v1"
permalink: "https://feed.astralor.com/items/itm_7982643b785eb94fc079150b"
published: "2026-07-28T17:49:51.000Z"
score: 78
channel: "ai"
tags: ["Desktop-Delta Bench","GUI转换","基准测试","计算机使用模型"]
---

# Desktop-Delta Bench：计算机使用模型能否理解桌面GUI转换？

## AI 点评

> 这篇论文切中了当前计算机使用模型评估中的盲点，对于研究GUI代理可靠性的团队来说非常值得关注。它提出的诊断层将帮助开发者更精准地定位模型失败原因，尤其适合那些正在构建桌面自动化系统的团队。

## 速览

本文提出Desktop-Delta Bench (DDB)，一个离线步级基准，包含2,013个人工验证实例，覆盖约15个应用和50个任务领域。DDB通过时间排序和前后对比两个任务，评估计算机使用模型对GUI状态转换的理解能力，填补了现有基准在诊断层上的空白。评估8个模型家族发现一致差距：排序任务最佳精确匹配率约65%，推断动作家族比定位更难。DDB为提升桌面CUA的验证、可靠性和恢复能力提供了针对性改进方向。

## 关键点

- DDB包含2,013个人工验证实例，覆盖15个应用和50个任务领域，针对三个失败维度。
- 两个任务：463个3帧时间排序实例（含105个跨轨迹诱饵）和1,550个前后对比实例。
- 8个模型家族评估显示排序未饱和：非诱饵和诱饵精确匹配率分别为65.1%和65.7%。
- 单动作任务中点击F1为0.96，拖动F1为0.76，推断动作家族比定位更难。

## 全文

Computer-use agents (CUAs) increasingly act through desktop GUIs to complete long-horizon tasks. Current benchmarks primarily measure end-task success or single-frame grounding. Neither isolates whether a model can reconstruct the causal, task-relevant transition produced by an action- crucial for rejecting stale observations, verifying progress, and recovering from failure. This is difficult because inference, remote input, app rendering, and screenshot capture are asynchronous: the next observation may be delayed, occluded, transient, or unrelated, then misread as progress and carried into subsequent planning. We introduce Desktop-Delta Bench (DDB), an offline step-level benchmark with 2,013 human-verified instances from novel, multi-app Linux trajectories across ~15 applications and 50 task domains. DDB trajectories targets 3 failure dimensions- state verification, source tracking, and context-aware control- through 2 complementary tasks: 463 3-frame temporal-ordering instances, including 105 with a cross-trajectory decoy, and 1,550 before-after pairs labeled from 5 actions + its payload. We evaluate 8 closed and open-source model families across 32 ordering and 16 single-action settings, observing consistent gaps. Ordering remains unsaturated: best non-decoy and decoy exact-match rates are 65.1% and 65.7%. Task context improves decoy identification by 6.9 percentage points but reduces non-decoy exact match by 2.2 points; error analysis reveals systematic copying of the presented A-B-C order. Single-action results show that inferring the action family is harder than locating it: click F1 is 0.96 vs, 0.76 for drag, while recognized drags are generally localized well. DDB, thus, complements end-to-end benchmarks by filling the missing diagnostic layer between GUI grounding and final task success, enabling targeted improvements to desktop CUA verification, reliability, and recovery.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_7982643b785eb94fc079150b