---
title: "Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.04828v1"
permalink: "https://feed.astralor.com/items/itm_22cff8136e19715b528c5590"
published: "2026-08-05T13:29:16.000Z"
score: 76
channel: "tech"
tags: ["Benchmark","Large Language Models","Skill-Use","Agentic Framework"]
---

# Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?

## AI commentary

> This benchmark turns 'whether models can actually use skills' into a measurable question, which is especially valuable for those building agent products and frameworks. The highlight is that skill use is not a fixed capability of the model but strongly depends on the harness, meaning we must design better invocation mechanisms, not just tune models. Recommended for every developer struggling with tool calling.

## At a glance

Large language model (LLM) agents increasingly rely on skills—structured documents specifying when to act, which procedure to follow, and which tools are allowed. Existing evaluations mostly judge skill quality or task success, leaving unexamined whether an agent can recognize a relevant skill and apply it on its own. We introduce Skill-Use, a benchmark that evaluates skill use under progressive disclosure, where an agent sees only a skill's name and short description and must retrieve the full procedure before following it. Skill-Use separates three facets: Trigger, Compliance, and Boundary. A Skill-Use (SU) score combines the three and credits execution only after the skill is triggered. The benchmark pairs 79 real skills with 177 executable tasks across nine domains, runs in isolated Docker sandboxes, and scores via trajectory-based rubrics. Evaluating eight LLMs under two agent harnesses, we find reliable skill use remains out of reach: the strongest configuration reaches only 0.613, and both scores and rankings shift with the harness, indicating skill use is harness-conditioned rather than a fixed model property.

## Key points

- Introduces the Skill-Use benchmark, evaluating skill use via Trigger, Compliance, and Boundary.
- Includes 79 real skills and 177 executable tasks across nine domains, run in isolated Docker sandboxes.
- Evaluates eight LLMs under two agent harnesses, with the best SU score only 0.613.
- Trigger and Compliance are independent bottlenecks, and model performance shifts with the harness.

## Full text

Large language model (LLM) agents increasingly rely on skills, structured documents that specify when to act, which procedure to follow, and which tools are allowed. Existing evaluations mostly judge the quality of a skill or its contribution to task success, leaving unexamined whether an agent can recognize a relevant skill and apply it on its own. We introduce Skill-Use, a benchmark that evaluates skill use under progressive disclosure, where an agent sees only a skill's name and short description and must retrieve the full procedure before following it. Skill-Use separates three facets of skill use. Trigger measures whether the agent invokes the relevant skill, Compliance measures how faithfully it follows the prescribed procedure, and Boundary measures whether it avoids forbidden operations. A Skill-Use (SU) score combines the three and credits execution only after the skill is triggered. Skill-Use pairs 79 real skills with 177 executable tasks across nine domains, each grounded in real files, run in an isolated Docker sandbox, and scored by a trajectory-based rubric. Evaluating eight LLMs under two agent harnesses, we find that reliable skill use remains out of reach, as the strongest configuration reaches an SU of only 0.613. Triggering and procedural compliance fail as independent bottlenecks, and both scores and model rankings shift with the harness, so skill use behaves as a capability conditioned on the harness rather than a fixed property of the model.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_22cff8136e19715b528c5590