---
title: "OmniaBench：跨多样化场景的通用AI智能体基准测试"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.14989v1"
permalink: "https://feed.astralor.com/items/itm_eb6117d3f664b16623e2bd00"
published: "2026-07-16T13:38:07.000Z"
score: 81
channel: "ai"
tags: ["AI评估","Claude Sonnet 5","GPT-5.6 Sol","OmniaBench","基准测试","通用智能体"]
---

# OmniaBench：跨多样化场景的通用AI智能体基准测试

## AI 点评

> 这篇论文对任何从事LLM Agent开发或评估的研究者与工程师而言都是必读之作。它不仅提供了覆盖ToC、ToB、ToE的丰富任务集，还通过十维能力分类和难度因子，精准定位了当前模型的弱项——规划、约束保持和自适应修正。如果你想了解自家Agent在真实场景中的上限在哪里，OmniaBench会是一个很好的试金石。

## 速览

大型语言模型正从文本生成器演变为通用智能体，但现有基准多聚焦有限场景。OmniaBench 是一个覆盖 ToC、ToB 和 ToE 共 90 个一级和 354 个二级领域的基准，通过四种途径合成 1431 个任务，引入十维能力分类和八组合原子难度因子。当前前沿模型（如 Claude-Sonnet-5 和 GPT-5.6-Sol）的总体通过率仅约 58% 和 57%，揭示了规划、约束保持和自适应修正方面的持续局限。

## 关键点

- OmniaBench 从应用商店、产品文档等来源构建了涵盖 ToC/ToB/ToE 的场景知识层次架构。
- 通过 DAG、DAG-S、Solver 和 Program 四种互补路径合成了单轮和多轮任务，共 1431 个。
- 引入十维能力分类和八组合原子难度因子，支持细粒度评估。
- 前沿模型在规划、约束保持和自适应修正上持续受限，最高通过率仅约 58%。

## 全文

Large language models are increasingly evolving from text generators into general agents capable of understanding user requests, invoking external tools, and completing complex tasks through interaction. However, existing agent benchmarks often focus on limited scenarios, tool ecosystems, or interaction formats, making it difficult to systematically characterize model capabilities across heterogeneous application settings. We introduce OmniaBench, a benchmark for evaluating general agents across diverse scenarios with explicit state spaces. We derive application-oriented scenario knowledge from app stores, product documents, industry resources, Web retrieval, and human refinement, forming a hierarchical taxonomy that spans ToC, ToB and ToE with 90 level-1 and 354 level-2 domains. Based on this taxonomy, we construct executable environments and synthesize single-turn and multi-turn tasks through four complementary routes: DAG, DAG-S, Solver, and Program. OmniaBench further introduces a ten-dimensional capability taxonomy and eight compositional atomic difficulty factors to support fine-grained evaluation and analysis. The resulting dataset contains 1,431 tasks, together with a challenging subset of 644 tasks designed to reduce evaluation cost and mitigate potential contamination of the full set after public release. The bench presents substantial challenges to current frontier models, with even Claude-Sonnet-5 and GPT-5.6-Sol achieving Overall Pass@1 scores of only 58.54 and 57.14, respectively. Further analyses reveal clear differences across domains and capabilities, as well as persistent limitations in planning, constraint maintenance, and adaptive correction. OmniaBench provides a broad and diagnostic benchmark for characterizing the capability boundaries of general agents.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_eb6117d3f664b16623e2bd00