---
title: "Desktop-Delta Bench: Do Computer-Use Models Understand Desktop GUI Transitions?"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.26041v1"
permalink: "https://feed.astralor.com/items/itm_7982643b785eb94fc079150b"
published: "2026-07-28T17:49:51.000Z"
score: 78
channel: "ai"
tags: ["Desktop-Delta Bench","GUI Transitions","Benchmark","Computer-Use Models"]
---

# Desktop-Delta Bench: Do Computer-Use Models Understand Desktop GUI Transitions?

## AI commentary

> This paper hits a blind spot in current computer-use model evaluation and is highly relevant for teams working on GUI agent reliability. The diagnostic layer it proposes will help developers pinpoint failure causes more precisely, especially for those building desktop automation systems.

## At a glance

This paper introduces Desktop-Delta Bench (DDB), an offline step-level benchmark with 2,013 human-verified instances across ~15 applications and 50 task domains. DDB evaluates computer-use models' ability to understand GUI state transitions through two tasks: temporal ordering and before-after comparison, filling a diagnostic gap in existing benchmarks. Evaluating 8 model families reveals consistent gaps: best exact-match rates for ordering tasks are around 65%, and inferring action families is harder than localizing them. DDB enables targeted improvements for verification, reliability, and recovery of desktop CUAs.

## Key points

- DDB contains 2,013 human-verified instances across 15 applications and 50 task domains, targeting three failure dimensions.
- Two tasks: 463 three-frame temporal-ordering instances (including 105 cross-trajectory decoys) and 1,550 before-after pairs.
- Evaluation of 8 model families shows ordering is unsaturated: non-decoy and decoy exact-match rates are 65.1% and 65.7%.
- In single-action tasks, click F1 is 0.96 vs. drag F1 0.76, with inferring action families harder than localizing them.

## Full text

Computer-use agents (CUAs) increasingly act through desktop GUIs to complete long-horizon tasks. Current benchmarks primarily measure end-task success or single-frame grounding. Neither isolates whether a model can reconstruct the causal, task-relevant transition produced by an action- crucial for rejecting stale observations, verifying progress, and recovering from failure. This is difficult because inference, remote input, app rendering, and screenshot capture are asynchronous: the next observation may be delayed, occluded, transient, or unrelated, then misread as progress and carried into subsequent planning. We introduce Desktop-Delta Bench (DDB), an offline step-level benchmark with 2,013 human-verified instances from novel, multi-app Linux trajectories across ~15 applications and 50 task domains. DDB trajectories targets 3 failure dimensions- state verification, source tracking, and context-aware control- through 2 complementary tasks: 463 3-frame temporal-ordering instances, including 105 with a cross-trajectory decoy, and 1,550 before-after pairs labeled from 5 actions + its payload. We evaluate 8 closed and open-source model families across 32 ordering and 16 single-action settings, observing consistent gaps. Ordering remains unsaturated: best non-decoy and decoy exact-match rates are 65.1% and 65.7%. Task context improves decoy identification by 6.9 percentage points but reduces non-decoy exact match by 2.2 points; error analysis reveals systematic copying of the presented A-B-C order. Single-action results show that inferring the action family is harder than locating it: click F1 is 0.96 vs, 0.76 for drag, while recognized drags are generally localized well. DDB, thus, complements end-to-end benchmarks by filling the missing diagnostic layer between GUI grounding and final task success, enabling targeted improvements to desktop CUA verification, reliability, and recovery.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_7982643b785eb94fc079150b