---
title: "DenseOn与LateOn：全开放稠密和延迟交互模型用于多语言、长上下文和代码搜索"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.27178v1"
permalink: "https://feed.astralor.com/items/itm_d5cccf6532fbe2aedb44b355"
published: "2026-07-29T17:50:51.000Z"
score: 78
channel: "ai"
tags: ["DenseOn","LateOn","mmBERT-base","多语言检索","密集检索","延迟交互"]
---

# DenseOn与LateOn：全开放稠密和延迟交互模型用于多语言、长上下文和代码搜索

## AI 点评

> 这篇论文最值得看的地方在于它提供了一个完全开源的检索模型训练方案，并且通过对比实验发现了稠密模型和延迟交互模型在多语言迁移能力上的根本差异。对于从事多语言信息检索或希望复现SOTA检索模型的研究者来说，这是难得的资料；对于关注开源AI生态的人，也展示了如何用翻译训练实现跨语言泛化。

## 速览

该研究提出了一个完全开源的检索模型训练方案，重建了6.65亿对英文对比预训练数据，并训练出DenseOn（单向量稠密模型）和LateOn（ColBERT式延迟交互模型），在BEIR上分别达到56.20和57.22的nDCG@10，创下该参数量级别的新SOTA。通过翻译训练扩展到八种语言，得到mDenseOn和mLateOn（3.07亿参数）。研究发现：稠密模型在翻译语言上表现好但泛化到未见语言时下降，而延迟交互模型能更好地泛化到未见语言和文字，验证了token级匹配将翻译训练从目标语言扩展策略转变为多语言泛化方法。

## 关键点

- 基于34个公开数据源重建6.65亿英文对比预训练对，并利用挖掘的难负样本构建188万监督微调对，训练出两个1.49亿参数模型，在BEIR上取得该规模SOTA。
- 将验证后的英文数据翻译成八种语言，得到28亿对跨语言样本，训练出两个3.07亿参数模型mDenseOn和mLateOn。
- 稠密模型在翻译语言上表现强，但泛化到未见语言时性能下降；延迟交互模型则能更好地泛化到未见语言和文字，揭示token级匹配将翻译训练从目标语言扩展转变为多语言泛化方法。

## 全文

State-of-the-art retrieval models increasingly rely on closed training data, creating a reproducibility gap. We present an open end-to-end recipe for training retrieval models and study how English supervision transfers to multilingual retrieval through translate-train. We first reconstruct and curate 665M English contrastive pre-training pairs from 1.4B pairs across 34 public sources and build 1.88M supervised fine-tuning pairs with mined hard negatives. Training yields two 149M-parameter models: DenseOn, a single-vector dense model, and LateOn, a ColBERT-style late-interaction model. They achieve 56.20 and 57.22 average nDCG@10 on BEIR, respectively, setting new state-of-the-art results for this size class. We then translate the validated English data into eight languages, yielding 2.8B pairs with cross-lingual samples, and train mDenseOn and mLateOn, two 307M-parameter models built on mmBERT-base. Despite sharing their backbone, data, and objectives, their representations behave differently: the dense model is strong on English and translated languages but degrades outside translate-train support, whereas the late-interaction model generalizes better to unseen languages and scripts. This suggests that token-level matching turns translate-train from a target-language expansion strategy into a multilingual generalization recipe. We publicly release the models, datasets, and training code.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_d5cccf6532fbe2aedb44b355