---
title: "技能使用：大模型能否在智能体框架中真正运用技能？"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.04828v1"
permalink: "https://feed.astralor.com/items/itm_22cff8136e19715b528c5590"
published: "2026-08-05T13:29:16.000Z"
score: 76
channel: "tech"
tags: ["基准测试","大语言模型","技能使用","智能体框架"]
---

# 技能使用：大模型能否在智能体框架中真正运用技能？

## AI 点评

> 这个基准把“模型到底会不会用技能”变成了可量化的问题，对做Agent产品和框架的人特别有参考价值。亮点在于它证明技能使用不是模型自带的固定能力，而是和harness强相关，这意味着我们不能只调模型，还要设计更好的调用机制。推荐给所有被工具调用折磨的开发者。

## 速览

论文提出Skill-Use基准，评估LLM智能体在渐进式披露下使用技能的能力，从触发、合规、边界三个维度分别考察。该基准包含79个技能、177个可执行任务，覆盖9个领域，在Docker沙箱中运行。评测8个模型和2种智能体框架后发现，最强配置的SU分数仅0.613，触发和合规是独立瓶颈，且模型排名随框架变化，技能使用并非模型的固定属性。

## 关键点

- 提出Skill-Use基准，分触发、合规、边界三个维度评估技能使用。
- 包含79个真实技能和177个可执行任务，覆盖9个领域，在隔离Docker沙箱中运行。
- 评测了8个LLM和2种agent harness，最佳SU分数仅0.613。
- 发现触发和合规是独立瓶颈，且模型技能表现随harness变化。

## 全文

Large language model (LLM) agents increasingly rely on skills, structured documents that specify when to act, which procedure to follow, and which tools are allowed. Existing evaluations mostly judge the quality of a skill or its contribution to task success, leaving unexamined whether an agent can recognize a relevant skill and apply it on its own. We introduce Skill-Use, a benchmark that evaluates skill use under progressive disclosure, where an agent sees only a skill's name and short description and must retrieve the full procedure before following it. Skill-Use separates three facets of skill use. Trigger measures whether the agent invokes the relevant skill, Compliance measures how faithfully it follows the prescribed procedure, and Boundary measures whether it avoids forbidden operations. A Skill-Use (SU) score combines the three and credits execution only after the skill is triggered. Skill-Use pairs 79 real skills with 177 executable tasks across nine domains, each grounded in real files, run in an isolated Docker sandbox, and scored by a trajectory-based rubric. Evaluating eight LLMs under two agent harnesses, we find that reliable skill use remains out of reach, as the strongest configuration reaches an SU of only 0.613. Triggering and procedural compliance fail as independent bottlenecks, and both scores and model rankings shift with the harness, so skill use behaves as a capability conditioned on the harness rather than a fixed property of the model.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_22cff8136e19715b528c5590