---
title: "Inducing Task Models from Computer-Use Traces"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.20319v1"
permalink: "https://feed.astralor.com/items/itm_023f777f7dba46bb1825923d"
published: "2026-08-20T17:57:00.000Z"
score: 76
channel: "ai"
tags: ["arXiv","TMI","Task Model Induction","Computer-use agents","Trace learning"]
---

# Inducing Task Models from Computer-Use Traces

## AI commentary

> This paper is worth watching because it tackles a key pain point when agents enter real work: how to extract auditable structured knowledge from messy traces. For those working on agent learning or enterprise process automation, TMI offers a promising path, and the experimental numbers are impressive—though it's worth examining how it scales to complex real-world settings.

## At a glance

This paper introduces Task Model Induction (TMI), a framework for extracting symbolic, auditable, and reusable task models from naturalistic computer-use traces recorded passively. Unlike existing methods that assume a given task or single workflow, TMI discovers latent tasks in unconstrained traces, disentangles concurrent activity, and induces for each task a hierarchical objective model and a procedure model of control flow. On controlled human and agent trajectories, TMI recovers interleaved tasks with 0.974 agreement against ground-truth groupings and reconstructs 74.9% of observed execution steps; downstream, skills derived from TMI's task models improve held-out task accuracy by 30.0% over the strongest baseline.

## Key points

- TMI automatically discovers and disentangles concurrent tasks from raw traces without assuming tasks or workflows.
- It builds hierarchical objective models and procedure models of control flow for each latent task.
- In controlled experiments, it achieves 0.974 agreement on task grouping and 74.9% step reconstruction.
- Skills derived from TMI's task models improve downstream task accuracy by 30.0% over the strongest baseline.

## Full text

Naturalistic computer-use traces, passively recorded screenshots and mouse or keyboard actions, are a valuable resource for deriving symbolic, auditable, and reusable models of how everyday work is done. Such models matter as computer-use agents enter real work, where agents need to learn how tasks are actually performed, and organizations need to audit and reuse that knowledge. However, inducing such task models is challenging, as activity is observed only as low-level events and real-world work is multi-threaded with interleaved goals. Existing methods assume a given task or a single workflow, and produce step-level summaries rather than structured task models. We introduce Task Model Induction (TMI), which (i) discovers the latent tasks in an unconstrained trace, disentangling concurrent activity, and (ii) for each latent task, induces a task model pairing a hierarchical objective model of recursive goal decomposition with a procedure model of the control flow that organized the execution. Intrinsically, on controlled human and agent trajectories, TMI recovers interleaved tasks with 0.974 agreement against ground-truth groupings and reconstructs 74.9% of the observed execution steps, far more than the strongest workflow induction baseline. Extrinsically, skills derived from TMI's task models improve held-out task accuracy by 30.0% over the strongest baseline.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_023f777f7dba46bb1825923d