---
title: "大语言模型中的未来置信度蒸馏：预解答表征预测后解答置信度"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.07626v1"
permalink: "https://feed.astralor.com/items/itm_3cc333911ffb12186d324525"
published: "2026-07-08T16:43:11.000Z"
score: 78
channel: "ai"
tags: ["大语言模型","学习判断","感知知悉感","未来置信度蒸馏","线性探针","置信度估计"]
---

# 大语言模型中的未来置信度蒸馏：预解答表征预测后解答置信度

## AI 点评

> 这篇论文的价值在于打破了置信度估计的传统静态视角，揭示了回答过程中的动态信息。对于从事LLM安全、可靠性以及自适应系统的研究者来说，是一个值得关注的新思路。未来置信度蒸馏的样本高效性和迁移性也使其具备工程落地潜力。

## 速览

本研究从时间维度探索大语言模型的置信度演化，对比了预解答的“感知知悉感”(FOK)和后解答的“学习判断”(JOL)。实验发现后解答置信度校准更优，且隐藏表示线性探针可提取比显式表达更丰富的置信度信息。据此提出未来置信度蒸馏，利用后解答正确性探针作为教师信号，训练仅需预解答表示的预测器，在保持校准提升的同时大幅降低成本，并展现出样本高效性和跨数据集迁移能力。

## 关键点

- 后解答置信度(JOL)比预解答置信度(FOK)校准更好、判别力更强。
- 隐藏表示的线性探针能提取比模型显式表达更丰富的置信度相关信息。
- 未来置信度蒸馏仅需预解答表示即可预测后解答置信度，显著降低推理成本。
- 该方法样本高效且在同一领域内跨数据集迁移表现良好。

## 全文

Reliable confidence estimation is essential for deploying large language models (LLMs) in confidence-aware systems, where downstream decisions such as retrieval, tool use, and adaptive computation depend on accurately estimating answer reliability. Existing approaches, however, largely treat confidence as a property of completed responses, overlooking how confidence-related information evolves throughout the answering process. In this work, we investigate confidence from a temporal perspective by comparing pre-solution Feeling-of-Knowing (FOK) and post-solution Judgement-of-Learning (JOL) confidence estimates across frontier and open-source LLMs. We show that post-solution confidence is consistently better calibrated and more discriminative than pre-solution confidence, while linear probes trained on hidden representations recover substantially richer confidence-related information than models explicitly verbalise. Building on this observation, we introduce future confidence distillation, which trains predictors operating on pre-solution hidden representations using teacher confidence estimates produced by post-solution correctness probes. Despite requiring only pre-solution representations for inference, distilled predictors recover much of the calibration improvement achieved by post-solution confidence, remain highly sample efficient, and transfer across datasets within the same domain. Together, our findings demonstrate that confidence-related information evolves throughout the answering process and can be anticipated before answer generation is complete, enabling significantly more reliable yet low-cost confidence estimation.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_3cc333911ffb12186d324525