---
title: "一种表示迁移所有语言：超越原生正字法的多语言模型预训练"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.25904v1"
permalink: "https://feed.astralor.com/items/itm_7b6d2d1aa28a8d22c788b2e0"
published: "2026-08-26T15:23:18.000Z"
score: 75
channel: "ai"
tags: ["IPA","多语言模型","罗马化","自回归预训练","跨语言迁移"]
---

# 一种表示迁移所有语言：超越原生正字法的多语言模型预训练

## AI 点评

> 这篇论文值得所有做多语言模型训练的人关注，它把罗马化从‘预处理技巧’提升到了‘核心设计选择’的位置。对于团队来说，如果你在新模型里还在用原生文本做多语言预训练，可能已经输在起跑线上了。

## 速览

该研究系统比较了多语言自回归预训练中三种输入表示：原生文本、IPA音标和罗马化。在467M至1.03B参数规模、8种语言、4个类型学配对上进行实验，发现罗马化预训练跨语言迁移效果最强，且优势随规模扩大而增大；IPA优于原生文本但不如罗马化。令人意外的是，对已覆盖脚本的模型用罗马化数据微调反而有害。研究建议罗马化应作为预训练核心设计而非事后修正。

## 关键点

- 在受控条件下比较原生文本、IPA和罗马化三种输入，覆盖8种语言、4个类型学配对
- 罗马化预训练在跨语言迁移上最强，且规模越大优势越明显
- IPA多数场景优于原生文本，但弱于罗马化
- 用罗马化数据微调文本预训练模型会损害已覆盖语言的性能，仅对无脚本覆盖语言略有帮助

## 全文

Multilingual language models transfer knowledge across languages through shared subword vocabulary, a mechanism that breaks down when related languages use different writing systems. Prior work addresses this via script equalization (romanization or IPA transcription), but direct comparisons are rare; the focus has been on encoder-only models, with most work adapting existing pretrained models. We systematically compare different input representations in autoregressive multilingual pretraining, comparing orthographic text, IPA, and romanization in a controlled setup across three scales (467M, 709M, and 1.03B) on eight languages in four typologically motivated pairs. Across a wide range of downstream tasks on seen and unseen languages, romanized pretraining yields the strongest cross-lingual transfer, and the advantage over text widens with scale. IPA improves over text in most settings but trails romanization. Surprisingly, finetuning a text-pretrained model on romanized data hurts performance on languages already covered by the base model, only marginally helping when the model lacks script coverage. Our results indicate that for multilingual models spanning typologically diverse scripts, to obtain maximum benefits, romanization should be treated as a core design choice applied at pretraining rather than a post hoc fix.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_7b6d2d1aa28a8d22c788b2e0