---
title: "InsufficiencyBench：评估大模型在法律咨询中处理信息不足查询的能力"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.20220v1"
permalink: "https://feed.astralor.com/items/itm_67969febfbbe2af8090f1875"
published: "2026-08-20T16:14:47.000Z"
score: 76
channel: "ai"
tags: ["InsufficiencyBench","基准测试","大语言模型","法律AI"]
---

# InsufficiencyBench：评估大模型在法律咨询中处理信息不足查询的能力

## AI 点评

> 这个基准戳中了法律AI落地的核心痛点：真实用户查询往往充满歧义和遗漏。对法律科技公司来说，这篇论文的价值在于提供了清晰的错误分类法和可复用的评估数据，能帮助团队定向改进模型。对关注AI安全的读者，它也再次证明前沿模型在不确定性表达上的短板。

## 速览

首个专门针对法律AI查询信息不足场景的基准测试，要求模型识别查询中缺失的关键法律事实、指出缺失内容并避免过早结论。研究构建了202个测试项，覆盖6个法律领域和24个美国司法辖区，由执业律师标注。评估10个前沿模型，最佳模型在缺失要素识别上的F2分数只有0.46，中位召回率0.44。模型要么无差别回避，要么在虚构假设下作答，无法同时正确处理完整和缺陷查询。

## 关键点

- 首个专门评估法律查询信息不足场景的基准，形式化8类缺失要素与3种结构故障模式。
- 构建202个测试项，覆盖6个法律领域和24个美国司法辖区，由执业律师标注。
- 十大前沿模型表现不佳，最佳F2仅0.46，中位召回率0.44。
- 模型普遍无差别回避或基于虚构假设作答，无法同时处理完整与缺陷查询。

## 全文

Legal AI systems are increasingly used to answer legal questions, yet existing benchmarks assume queries arrive fully specified. In practice, users omit facts that materially determine the legal outcome. We introduce InsufficiencyBench, the first legal benchmark targeting query-side insufficiency: whether a model recognizes when a query lacks legally material information, identifies what is missing, and refrains from premature conclusions. We formalize a taxonomy of eight canonical missing-element categories across three structural failure modes---switch, gating, and fatal prerequisite--- and construct 202 benchmark items (58 base queries, 144 deficient variants) spanning six legal domains and 24 US jurisdictions and annotated by practising attorneys. Evaluating ten frontier models, we find that no model exceeds F2 = 0.46 on missing-element identification and that the median recall is 0.44. Models either hedge indiscriminately or answer silently under fabricated presumptions. No model both identifies and qualifies responses to deficient queries while directly addressing complete ones.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_67969febfbbe2af8090f1875