---
title: "Training Foundation Models with a Single Algorithm"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.40564v1"
permalink: "https://feed.astralor.com/items/itm_75bf847957a9e2be5434c286"
published: "2026-10-29T17:42:05.000Z"
score: 54
channel: "tech"
tags: ["推理效率","RAG","Training Foundation Models","Single Algorithm Foundation","SFT","Foundation","Across"]
---

# Training Foundation Models with a Single Algorithm

## 速览

Foundation models are typically trained in stages: pretraining, midtraining, supervised fine-tuning (SFT), and reinforcement learning (RL), with changes in both data and learning algorithms. We show that this algorithmi…

## 关键点

- Foundation models are typically trained in stages: pretraining, midtraining, supervised fine-tuning (SFT), and reinforcement learning (RL), with changes in both data and learning algorithms.
- We show that this algorithmic separation is not fundamental.
- By deriving suitable RL objectives as maximum-likelihood fitting to model-generated target distributions, we place all of these stages under a single likelihood-based training algorithm, unified training.

## 全文

Foundation models are typically trained in stages: pretraining, midtraining, supervised fine-tuning (SFT), and reinforcement learning (RL), with changes in both data and learning algorithms. We show that this algorithmic separation is not fundamental. By deriving suitable RL objectives as maximum-likelihood fitting to model-generated target distributions, we place all of these stages under a single likelihood-based training algorithm, unified training. This perspective turns the foundation-model training lifecycle into a problem of choosing which data to train on, in what proportions, and when, with conventional staged training as a special case. We instantiate this framework in a unified training system that mixes static data and RL generations within a single trainer. Across controlled and real-world experiments, unified training improves the accuracy-compute tradeoff and task coverage while preserving broader capabilities, and provides a stronger starting point for continual learning. This work lays the groundwork for a unified science of foundation-model training, connecting mathematical principles, training infrastructure, and the development of model capabilities.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_75bf847957a9e2be5434c286