---
title: "Data Analysis in the Wild: Benchmarking Large Language Models Against Real-World Data Complexities"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.06482v1"
permalink: "https://feed.astralor.com/items/itm_c140d494710cb60e0daabd99"
published: "2026-07-07T16:43:05.000Z"
score: 79
channel: "ai"
tags: ["DataGovBench","Large Language Models","Governmental Open Data","Data Analysis","Data Benchmark","Agentic Framework"]
---

# Data Analysis in the Wild: Benchmarking Large Language Models Against Real-World Data Complexities

## AI commentary

> What makes this paper particularly noteworthy is that it pops the bubble of LLM data analysis capabilities—models excel on simple benchmarks but struggle with multi-table, reasoning-intensive, and creative real-world data. For AI product managers, data scientists, and model developers, DataGovBench is a highly valuable testbed. Whoever achieves breakthroughs on this benchmark will gain a significant edge in next-generation intelligent data analysis tools.

## At a glance

Current benchmarks for evaluating LLMs in data analysis often fail to reflect real-world settings, focusing on fact retrieval from small tables while ignoring multi-table datasets, external knowledge integration, and exploratory insight discovery. This paper introduces DataGovBench, a benchmark from governmental open data, with Table QA and Table Insight tasks. Experiments show significant performance gaps in state-of-the-art LLMs, even with agentic frameworks, indicating they are far from meeting real-world data analytics demands. Code and data are publicly available.

## Key points

- Current data analysis benchmarks are too simplistic and fail to reflect real-world complexity like multi-table datasets, external knowledge, and exploratory analysis.
- DataGovBench is introduced with two tasks: Table QA (complex decomposable questions with textual/visual answers) and Table Insight (generating expert-level findings).
- Comprehensive experiments show significant performance gaps in state-of-the-art LLMs, even with agentic frameworks.
- The results highlight severe shortcomings of LLMs in realistic data analysis, providing a challenging benchmark for future research.

## Full text

Current benchmarks for evaluating Large Language Models (LLMs) in data analysis often fail to reflect real-world settings. They typically focus on fact retrieval from small tables and overlook the challenges of large multi-tabular datasets, external knowledge integration, and exploratory insight discovery. We introduce DataGovBench, a benchmark derived from governmental open data designed to evaluate LLMs in practical scenarios. The benchmark includes two tasks: Table QA that requires solving complex decomposable questions and producing textual answers or visualizations, and Table Insight that evaluates the ability of models to generate expert-level findings through exploratory data analysis. Comprehensive experiments with state-of-the-art LLMs, both with and without agentic frameworks, reveal significant performance gaps across both tasks. These results suggest that current LLM-based systems remain far from satisfying the demands of real-world data analytics. DataGovBench provides a challenging benchmark for advancing research on LLMs capable of both answering analytical queries and discovering insights from data. Code and sample data are available at https://github.com/SoHasegawa/datagovbench.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_c140d494710cb60e0daabd99