---
title: "Future Confidence Distillation in Large Language Models: Predicting Post-Solution Confidence from Pre-Solution Representations"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.07626v1"
permalink: "https://feed.astralor.com/items/itm_3cc333911ffb12186d324525"
published: "2026-07-08T16:43:11.000Z"
score: 78
channel: "ai"
tags: ["Large Language Models","Judgement-of-Learning (JOL)","Feeling-of-Knowing (FOK)","Future Confidence Distillation","Linear Probing","Confidence Estimation"]
---

# Future Confidence Distillation in Large Language Models: Predicting Post-Solution Confidence from Pre-Solution Representations

## AI commentary

> This paper breaks the static view of confidence estimation by revealing dynamic information during the answering process. It's worth reading for researchers in LLM safety, reliability, and adaptive systems. The sample efficiency and transferability of Future Confidence Distillation also make it promising for engineering deployment.

## At a glance

This paper investigates the temporal evolution of confidence in large language models, comparing pre-solution Feeling-of-Knowing (FOK) and post-solution Judgement-of-Learning (JOL). Results show that post-solution confidence is better calibrated and more discriminative, and linear probes on hidden representations recover richer confidence information than explicit verbalization. The authors introduce Future Confidence Distillation, which trains predictors on pre-solution representations using post-solution correctness probes as teachers, achieving much of the calibration improvement with low cost, high sample efficiency, and cross-dataset transferability.

## Key points

- Post-solution confidence (JOL) is better calibrated and more discriminative than pre-solution confidence (FOK).
- Linear probes on hidden representations recover richer confidence information than models explicitly verbalize.
- Future Confidence Distillation predicts post-solution confidence using only pre-solution representations, significantly reducing inference cost.
- The method is sample-efficient and transfers well across datasets within the same domain.

## Full text

Reliable confidence estimation is essential for deploying large language models (LLMs) in confidence-aware systems, where downstream decisions such as retrieval, tool use, and adaptive computation depend on accurately estimating answer reliability. Existing approaches, however, largely treat confidence as a property of completed responses, overlooking how confidence-related information evolves throughout the answering process. In this work, we investigate confidence from a temporal perspective by comparing pre-solution Feeling-of-Knowing (FOK) and post-solution Judgement-of-Learning (JOL) confidence estimates across frontier and open-source LLMs. We show that post-solution confidence is consistently better calibrated and more discriminative than pre-solution confidence, while linear probes trained on hidden representations recover substantially richer confidence-related information than models explicitly verbalise. Building on this observation, we introduce future confidence distillation, which trains predictors operating on pre-solution hidden representations using teacher confidence estimates produced by post-solution correctness probes. Despite requiring only pre-solution representations for inference, distilled predictors recover much of the calibration improvement achieved by post-solution confidence, remain highly sample efficient, and transfer across datasets within the same domain. Together, our findings demonstrate that confidence-related information evolves throughout the answering process and can be anticipated before answer generation is complete, enabling significantly more reliable yet low-cost confidence estimation.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_3cc333911ffb12186d324525