---
title: "Agentic Coding Without the Cloud: Evaluating Open-Weight LLMs on Longitudinal Data Preparation Tasks"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.21482v1"
permalink: "https://feed.astralor.com/items/itm_d140dea79f32e7f8f5d490c4"
published: "2026-07-23T16:23:42.000Z"
score: 81
channel: "ai"
tags: ["RRBench","Open-source LLM","Data Preparation","Longitudinal Study","Privacy Governance"]
---

# Agentic Coding Without the Cloud: Evaluating Open-Weight LLMs on Longitudinal Data Preparation Tasks

## AI commentary

> This paper provides a valuable benchmark for longitudinal research under strict data governance, revealing the great potential of open-weight models in localized data preparation. If you are a researcher or engineer in population health, social science, or similar fields working with longitudinal data, this is worth reading—it demonstrates that open-weight models on consumer hardware can handle most data cleaning tasks, significantly lowering the barrier for AI applications under privacy compliance constraints.

## At a glance

This paper introduces an open-source framework for evaluating open-weight LLMs on longitudinal population study data preparation tasks. The framework includes a curated ground-truth dataset, task definitions, and automated evaluation routines. Benchmarking on 20 tasks (102 variables) shows that state-of-the-art 31-35B parameter models achieve up to 87.9% average task completion, suggesting a viable path for AI-assisted data preparation in governance-restricted settings using consumer-grade hardware.

## Key points

- Introduces an open-source evaluation framework for longitudinal data preparation tasks, including ground-truth cleaning scripts, task definitions, and automated evaluation.
- Benchmarking shows 31-35B parameter open-weight models achieve up to 87.9% average task completion, nearly saturating the benchmark.
- Demonstrates that open-weight LLMs on consumer-grade hardware can serve as a cloud-alternative for AI-assisted data preparation under governance restrictions.

## Full text

Large language models (LLMs) and agents are now widely used tools in code development, with data typically sent to third-party cloud-based models. Their adoption in research using personal data is constrained by governance requirements that typically prohibit data transmission to external services. Locally deployable open-weight models offer an alternative since sensitive data never leave the local environment. We introduce an open-source framework for evaluating the efficacy of AI agents powered by open-weight LLMs on one of the most persistent bottlenecks in research on longitudinal population studies: data preparation. The framework comprises: a curated ground-truth dataset (cleaning scripts preparing six sweeps of data from a British cohort study), task definitions encompassing tasks such as category harmonization and multi-wave merging, and automated routines for evaluating the LLM-produced R code and outputted data. We benchmark LLMs across the (consumer grade) deployment spectrum to assess their efficacy in 20 data preparation tasks (creation of 102 variables). Current state-of-the-art, 31-35B parameter models almost saturated our benchmark ("average task completion" up to 87.9%). The performance of open-weight LLMs running on consumer-grade hardware shows promise of a viable path toward AI-assisted data preparation in governance-restricted research settings. Our framework is publicly available at: https://github.com/UCL-ARC/RRBench.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_d140dea79f32e7f8f5d490c4