---
title: "CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.01145v1"
permalink: "https://feed.astralor.com/items/itm_f734c4418ca6f2f1054e6693"
published: "2026-10-31T17:32:54.000Z"
score: 57
channel: "tech"
tags: ["AI Agent","推理效率","RAG","CivicBench","Benchmark","Memory Framework","Web-Based Civic Data","Analytics Municipalities"]
---

# CivicBench: A Benchmark and Memory Framework for Web-Based Civic Data Analytics

## 速览

Municipalities publish service requests, geographic boundaries, weather observations, demographic estimates, and operational guidance through independently maintained Web portals. Civic analysis must combine these sourc…

## 关键点

- Municipalities publish service requests, geographic boundaries, weather observations, demographic estimates, and operational guidance through independently maintained Web portals.
- Civic analysis must combine these sources while preserving their schemas, source links, spatial and temporal scopes, and interpretive limits.
- Tool-using language agents can perform this integration, but existing task benchmarks rarely test whether a computed claim remains supported by the records used.

## 全文

Municipalities publish service requests, geographic boundaries, weather observations, demographic estimates, and operational guidance through independently maintained Web portals. Civic analysis must combine these sources while preserving their schemas, source links, spatial and temporal scopes, and interpretive limits. Tool-using language agents can perform this integration, but existing task benchmarks rarely test whether a computed claim remains supported by the records used. We present CivicBench, a 400-task benchmark over four cities and two years that materializes civic Web data into frozen bundles for reproducible evaluation. Executable contracts score computation, source linkage, coverage, scope, qualified answers, and refusal when the requested conclusion is not supported. We also introduce Task-State Memory (TSM), which compiles tool observations into typed assertions, reconciles records only within compatible scopes, and retrieves a bounded state organized around unresolved constraints. Across nine completed model pairs, eight yield positive mean-score differences with 95\% semantic-contract-cluster bootstrap intervals excluding zero; the ninth interval crosses zero. The benchmark and memory framework make the records behind a claim inspectable at the point of decision.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_f734c4418ca6f2f1054e6693