---
title: "M-GATE：多语言语法、翻译准确性与效率评测基准"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.03803v1"
permalink: "https://feed.astralor.com/items/itm_f9da740b5fa2bd29ae16f280"
published: "2026-08-04T15:16:05.000Z"
score: 78
channel: "ai"
tags: ["Common Crawl","M-GATE","多语言模型","语法错误检测"]
---

# M-GATE：多语言语法、翻译准确性与效率评测基准

## AI 点评

> 作为编辑，我觉得这篇论文最值得关注的是它把“会翻译”和“懂语言”拆开了。对研究多语言基础设施的团队来说，M-GATE 的对抗性语法测试和低资源回译分数会是很好的参考基线。建议做低资源模型的人去跑一下。

## 速览

M-GATE 是一个覆盖 30 种高资源到低资源语言的多语言熟练度基准，包含语法错误检测、回译和分词效率三项任务。评测 50 多个模型后发现，翻译流畅与语法熟练明显脱节：最先进模型在语法检测上的马修斯相关系数仅 0.36，且倾向于漏报错误。翻译质量与预训练数据占比高度相关（r=0.86），低资源语言劣势随模型更新逐步收窄。该基准以私有测试集持续更新，降低污染风险。

## 关键点

- M-GATE 提出三任务基准：对抗性语法错误检测、29 语回译、分词效率，覆盖 30 种语言。
- 50 多模型评测显示翻译能力与语法熟练度脱节，最佳语法 MCC 仅 0.36，且错误偏向漏报。
- 翻译质量与预训练数据占比强相关（r=0.86），低资源语言惩罚随模型迭代收窄。
- 采用私有测试集加公开榜单防止污染，并提供示例数据。

## 全文

Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency. We introduce M-GATE (Multilingual Grammar, Accuracy in Translation, and Efficiency), a benchmark of linguistic proficiency spanning 30 typologically diverse languages from high- to low-resource. M-GATE comprises three tasks: grammatical error detection on linguist-crafted, adversarially selected sentences that turn on hard, language-specific phenomena; round-trip translation of shared English sources across 29 target languages, scored by a three-provider LLM judge panel validated against professional annotators; and a supplementary tokenizer-efficiency measure. We evaluate over 50 models in more than 80 configurations. Fluency and proficiency come apart sharply: models that translate competently sit near chance on the adversarial grammar items, the best reaching a Matthews correlation coefficient (MCC) of only 0.36, and their errors lean systematically toward under-flagging, accepting ungrammatical text rather than raising false alarms. Translation quality closely tracks a language's share of pretraining data (r = 0.86 against log Common Crawl share), producing a steep low-resource penalty that is nonetheless narrowing with successive model releases. Enabling reasoning reliably improves translation, while its effect on error detection is smaller and for some models negative, so the best configuration is task-dependent. To resist contamination, test items are kept private behind a continuously updated public leaderboard, with illustrative examples released (https://m-gate.ai).

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_f9da740b5fa2bd29ae16f280