---
title: "Test-Time Self-Evolving: Reflection-Guided On-Policy Self-Distillation for GUI Visual Grounding"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.11191v1"
permalink: "https://feed.astralor.com/items/itm_3b64c552884106b4bec48ecc"
published: "2026-08-11T17:50:25.000Z"
score: 76
channel: "ai"
tags: ["GUI agents","GUI visual grounding","Test-Time Self-Evolving framework","Test-Time Adaptation","Self-Distillation"]
---

# Test-Time Self-Evolving: Reflection-Guided On-Policy Self-Distillation for GUI Visual Grounding

## AI commentary

> What stands out most is that this paper genuinely integrates 'reflection' into test-time learning instead of mere reward tuning, addressing the pain point of blind iteration on failed exploration in RL. For teams working on GUI agents or automated testing, this means deployed models can improve themselves without waiting for data feedback loops. If you are exploring self-evolution for multimodal agents, the contrastive calibration idea here is particularly worth studying.

## At a glance

GUI visual grounding is a fundamental capability for GUI agents, but existing models typically freeze their parameters after deployment, limiting adaptation to unseen interfaces. This paper proposes a Test-Time Self-Evolving framework that enables models to improve after deployment without human-annotated ground truth, through a closed loop of Exploration, Evaluation, Reflection, and Internalization. The framework introduces an MLLM-based Reflector to assess predictions and provide reasoning reflections, and proposes Reflection-Guided On-Policy Self-Distillation to translate high-level reasoning into dense token-level supervision via a conditioned self-teacher. A Contrastive Calibration method prevents incorrect auto-regressive prefixes from corrupting supervisory signals during failed explorations. Extensive experiments across six benchmarks show an average accuracy improvement of 7.4% over the base model, marking the first work to successfully exploit on-policy self-distillation for test-time adaptation in GUI visual grounding.

## Key points

- Proposes a Test-Time Self-Evolving framework that lets GUI agents improve after deployment without human-annotated ground truth.
- Introduces an MLLM-based Reflector to incorporate exploration results and reasoning reflections into the learning loop.
- Designs Reflection-Guided On-Policy Self-Distillation with Contrastive Calibration to prevent failed explorations from corrupting supervision.
- Achieves 7.4% average accuracy improvement over six benchmarks, the first on-policy self-distillation work for test-time adaptation in GUI visual grounding.

## Full text

GUI Visual Grounding is a fundamental capability for GUI agents. Existing models typically freeze their parameters after deployment, limiting their ability to adapt to unseen interfaces. Although recent methods attempt to adapt models via test-time reinforcement learning, they cannot reflect upon failed exploration. To overcome this, we propose a Test-Time Self-Evolving framework that enables models to improve after deployment without human-annotated ground truth. It constructs a closed-loop of Exploration, Evaluation, Reflection, and Internalization. Specifically, the agent first explores unseen interfaces by predicting grounding coordinates for given instructions. To evaluate these explorations, we introduce an MLLM-based Reflector to assess the generated results and provide the corresponding reasoning reflections. To internalize reflection knowledge into the model weights, we propose Reflection-Guided On-Policy Self-Distillation, which translates high-level reasoning into dense token-level supervision via a conditioned self-teacher. Furthermore, we design a Contrastive Calibration method to prevent incorrect auto-regressive prefixes from corrupting the supervisory signals during failed explorations. Extensive experiments across six benchmarks demonstrate our framework's effectiveness, achieving an average accuracy improvement of 7.4% over the base model. To the best of our knowledge, this is the first work to successfully exploit on-policy self-distillation for test-time adaptation in GUI visual grounding. By filling the gap in post-deployment adaptation, our framework completes the self-evolving capability of GUI agents. The code will be released.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_3b64c552884106b4bec48ecc