---
title: "Interactive Reward Agent: GUI Task Evaluation via Environment-State Verification"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.25904v1"
permalink: "https://feed.astralor.com/items/itm_87590703bda1428667a01443"
published: "2026-07-28T16:01:38.000Z"
score: 79
channel: "ai"
tags: ["GUI-RewardBench","GUI Agent","Interactive Reward Agent","Task Evaluation","Environment-State Verification"]
---

# Interactive Reward Agent: GUI Task Evaluation via Environment-State Verification

## AI commentary

> This paper addresses the pain point of environment state acquisition in GUI evaluation, and the proposed IRA method is both innovative and practical. For researchers and engineers working on GUI automation and AI agent training, IRA not only provides a reliable reward signal but also inspires how to combine system tools with interface information for verification. It's worth watching its scalability to more complex cross-application scenarios.

## At a glance

This paper proposes an Interactive Reward Agent (IRA) based on a 'propose-then-verify' framework that acquires and verifies evidence from the post-execution environment by invoking system tools, application tools, and GUI tools to determine task completion. It also introduces GUI-RewardBench, a benchmark of 321 GUI task trajectories across 10 Ubuntu desktop applications. IRA achieves 86.9% accuracy on this benchmark and is applied to reinforcement learning for GUI agents, achieving a 34.0% OSWorld success rate.

## Key points

- Proposes IRA framework that addresses GUI task evaluation challenges via interactive environment-state verification
- Introduces GUI-RewardBench benchmark covering 321 task trajectories across 10 Ubuntu application categories
- IRA achieves 86.9% accuracy, outperforming existing evaluator baselines
- IRA provides effective reward signals for training GUI agents, boosting OSWorld success rate to 34.0%

## Full text

Graphical user interface task evaluation aims to determine whether a GUI agent has successfully completed a user instruction. Automated GUI task evaluation has received increasing attention because the evaluation results can serve as reward signals for both test-time scaling and post-training. However, reliable GUI task evaluation remains challenging because the judgments often require access to environment states, such as system configurations, file data, and application settings, beyond the screenshots of execution trajectories. In this paper, we propose an interactive reward agent (IRA) based on a propose-then-verify framework to acquire and verify evidence from the post-execution environment. Given a task instruction and a GUI environment after the GUI agent execution, IRA first proposes the task completion conditions and then verifies them by invoking system tools, application tools, and GUI tools. This design combines evidence from both visible interfaces and the environment state in an interactive process. We further introduce GUI-RewardBench, a benchmark of 321 GUI task trajectories spanning 10 Ubuntu desktop application categories. Experiments show that IRA achieves 86.9% accuracy on GUI-RewardBench, outperforming existing evaluator baselines. We further apply IRA to reinforcement learning of GUI agents, achieving a 34.0% OSWorld success rate, which demonstrates that IRA can provide effective reward signals for training GUI agents.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_87590703bda1428667a01443