---
title: "Grading Needs a Rubric, Not Intelligence: Small Models Match Expensive Ones with Explicit Rubrics"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.17938v1"
permalink: "https://feed.astralor.com/items/itm_d5d71f03758d9d98a642cac8"
published: "2026-08-18T16:00:05.000Z"
score: 77
channel: "ai"
tags: ["any-to-bench","Educational Assessment","Small Language Models","Automated Grading","Rubric"]
---

# Grading Needs a Rubric, Not Intelligence: Small Models Match Expensive Ones with Explicit Rubrics

## AI commentary

> What makes this paper worth reading is how thoroughly it demystifies grading intelligence—showing with data that expensive models are unnecessary when the rubric is well-designed. For anyone building educational products, assessment systems, or any automation relying on subjective judgment, this is a directly transferable paradigm.

## At a glance

A new study finds that small language models can grade open-ended exam answers as reliably as substantially more expensive models when grading against an explicit rubric. The researchers propose the any-to-bench framework: a frontier model reads source documents once to extract questions and rubrics, while lower-cost models perform all repeated grading. Experiments show answer identity explains 95.6% of score variance, while judge identity explains only 0.2%. Raising a writer's reasoning effort moves scores by up to 0.143 of full marks, while raising a judge's reasoning effort moves scores by at most 0.006. Removing the official answer collapses reliability (ICC from 0.888 to 0.628), showing the official answer does nearly all the work within the rubric.

## Key points

- The any-to-bench framework has frontier models extract questions and rubrics once, with low-cost models handling repeated grading, significantly cutting costs.
- Answer identity explains 95.6% of score variance while judge identity explains only 0.2%, indicating the rubric drives outcomes rather than model intelligence.
- Increasing writer reasoning effort markedly affects scores, while increasing judge reasoning effort has almost no effect.
- Removing the official answer collapses grading reliability, showing it is the decisive component within the rubric.

## Full text

Small language models can grade open-ended examination answers as reliably as substantially more expensive models when they grade against an explicit rubric. We test this claim as the design principle behind any-to-bench: a frontier model reads source documents once, at ingestion, to extract each question and its rubric; lower-cost models then perform all repeated grading work. We evaluate six cost-efficient model configurations from two model families at three reasoning-effort levels. Each configuration answers 24 open-ended examination questions, and each also grades every answer sheet three times, yielding 3,456 per-question grades. Scores depend overwhelmingly on the answer being graded: answer identity explains 95.6% of score variance, whereas judge identity explains only 0.2%. Raising a writer's reasoning effort moves earned scores by as much as 0.143 of full marks, while raising a judge's reasoning effort moves assigned scores by at most 0.006. Six frontier-tier judges, added as a check, reproduce these scores and are no more reliable as a panel. Two ablations then decompose the rubric on the same questions and answers. Removing its criteria and levels while keeping the official answer changes nothing measurable. Removing the official answer as well collapses reliability (ICC 0.888 to 0.628), inflates scores, and makes judge reasoning effort matter again. The rubric is what decouples grading from judge intelligence, and within the rubric the official answer does nearly all the work. We find no evidence of length preference or same-family preference under rubric-anchored grading.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_d5d71f03758d9d98a642cac8