---
title: "PPL-Factory：面向语言建模与推理的任务感知与预算感知数据选择方法"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.18199v1"
permalink: "https://feed.astralor.com/items/itm_1d6c7c7faf8e13e237c2029e"
published: "2026-07-20T17:38:38.000Z"
score: 74
channel: "tech"
tags: ["LLM","任务感知","困惑度","微调效率","数据选择"]
---

# PPL-Factory：面向语言建模与推理的任务感知与预算感知数据选择方法

## AI 点评

> 这篇论文值得关注，因为它为高效微调提供了一种即插即用的数据选择方案，尤其适合迫切需要降低训练成本的研究团队和工程实践者。相比复杂的选择策略，PPL-Factory仅需计算困惑度，门槛低且效果显著。

## 速览

PPL-Factory框架提出了一种结合任务感知困惑度评分与数据预算感知准则的数据选择方法，用于大语言模型微调。实验表明，仅用1%训练数据即可超越现有方法，用10%数据在GSM8K上超过全数据微调0.9个点，在MATH上超过4.8个点。该方法简单、可解释，能有效降低微调计算成本。

## 关键点

- 现有数据选择方法依赖间接启发式，难以跨任务泛化
- PPL-Factory采用任务感知困惑度评分，区分语言建模与推理目标
- 加入数据预算感知准则，实现灵活选择
- 在GSM8K和MATH上以极少数据达到或超越全数据微调性能

## 全文

Not all training samples contribute equally to large language model fine-tuning. Selecting informative training samples can reduce the computational cost while preserving downstream performance. Many existing data selection methods rely on indirect heuristics, such as data quality, diversity or reasoning trace length. However, the effectiveness of these fixed criteria is task-dependent and difficult to generalize across diverse downstream tasks. Perplexity-based data selection provides a simple and model-aware solution to estimate the sample difficulty, but existing approaches typically score the entire training sequence and ignore the difference in learning objectives of language modeling and reasoning tasks. In this paper, we propose PPL-Factory, a simple and interpretable data selection framework that combines task-aware perplexity-based scores and data budget-aware selection criteria. Experiments on GSM8K demonstrate that PPL-Factory outperforms other state-of-the-art data selection methods using only $1\%$ of the training set. With $10\%$ of the data, PPL-Factory exceeds full-data fine-tuning accuracy by 0.9 on GSM8K and 4.8 on MATH. Overall, our results demonstrate that task-aware and budget-aware perplexity-based selection provides an effective and applicable approach for efficient fine-tuning.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_1d6c7c7faf8e13e237c2029e