---
title: "测试时自进化：反射引导的在线自蒸馏实现 GUI 视觉定位"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.11191v1"
permalink: "https://feed.astralor.com/items/itm_3b64c552884106b4bec48ecc"
published: "2026-08-11T17:50:25.000Z"
score: 76
channel: "ai"
tags: ["GUI 智能体","GUI 视觉定位","Test-Time Self-Evolving 框架","测试时自适应","自蒸馏"]
---

# 测试时自进化：反射引导的在线自蒸馏实现 GUI 视觉定位

## AI 点评

> 这篇论文最值得关注的是它把「反思」真正融进了测试时学习，而不只是奖励调优，解决了强化学习在失败探索中盲目迭代的痛点。对做 GUI Agent 和自动化测试的团队来说，这意味着部署后的模型可以自己变强，不用等数据回流。如果你在搞多模态智能体的自进化，这篇的对比校准思路尤其值得细读。

## 速览

GUI 视觉定位是 GUI 智能体的基础能力，但现有模型部署后参数冻结，难以适应未见过的新界面。本文提出测试时自进化框架，通过探索、评估、反思、内化闭环实现无需人工标注的部署后自适应。框架引入基于多模态大模型的反射器评估预测结果，并设计反射引导的在线自蒸馏将高层推理转化为密集 token 级监督，同时用对比校准防止错误自回归前缀污染监督信号。在六个基准上平均准确率提升 7.4%，首次成功将在线自蒸馏用于 GUI 视觉定位的测试时自适应。

## 关键点

- 提出测试时自进化框架，使 GUI 智能体在部署后无需人工标注即可持续改进。
- 引入基于 MLLM 的反射器，将探索结果与推理反思纳入学习闭环。
- 设计反射引导的在线自蒸馏与对比校准，避免失败探索污染监督信号。
- 在六个基准上平均提升 7.4%，是该领域首个在线自蒸馏测试时自适应工作。

## 全文

GUI Visual Grounding is a fundamental capability for GUI agents. Existing models typically freeze their parameters after deployment, limiting their ability to adapt to unseen interfaces. Although recent methods attempt to adapt models via test-time reinforcement learning, they cannot reflect upon failed exploration. To overcome this, we propose a Test-Time Self-Evolving framework that enables models to improve after deployment without human-annotated ground truth. It constructs a closed-loop of Exploration, Evaluation, Reflection, and Internalization. Specifically, the agent first explores unseen interfaces by predicting grounding coordinates for given instructions. To evaluate these explorations, we introduce an MLLM-based Reflector to assess the generated results and provide the corresponding reasoning reflections. To internalize reflection knowledge into the model weights, we propose Reflection-Guided On-Policy Self-Distillation, which translates high-level reasoning into dense token-level supervision via a conditioned self-teacher. Furthermore, we design a Contrastive Calibration method to prevent incorrect auto-regressive prefixes from corrupting the supervisory signals during failed explorations. Extensive experiments across six benchmarks demonstrate our framework's effectiveness, achieving an average accuracy improvement of 7.4% over the base model. To the best of our knowledge, this is the first work to successfully exploit on-policy self-distillation for test-time adaptation in GUI visual grounding. By filling the gap in post-deployment adaptation, our framework completes the self-evolving capability of GUI agents. The code will be released.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_3b64c552884106b4bec48ecc