---
title: "Aligning Language Models with Selective Prediction"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.03528v1"
permalink: "https://feed.astralor.com/items/itm_489579d3c09e0dc7eae3669a"
published: "2026-07-03T17:59:02.000Z"
score: 78
channel: "ai"
tags: ["Google DeepMind","RLSR","LLM Alignment","Selective Prediction"]
---

# Aligning Language Models with Selective Prediction

## AI commentary

> This paper cleverly introduces selective prediction into LLM alignment, moving beyond accuracy alone. If you're working on model deployment or reliability engineering, RLSR's approach is worth exploring—it teaches models 'when to keep quiet' rather than forcing wrong answers. Especially insightful for AI safety practitioners and developers in high-stakes settings.

## At a glance

This paper proposes a novel alignment framework, Reinforcement Learning for Selection Reward (RLSR), which optimizes the area under the risk-coverage curve (AURC) to improve selective prediction in large language models. By allowing the model to predict only on inputs where it is likely correct, RLSR reduces error rates and enables reliable human-AI collaboration. Experiments show that RLSR outperforms several alignment baselines on both in-domain and out-of-domain tasks.

## Key points

- Selective prediction balances risk and coverage by predicting only on trustworthy inputs, enhancing LLM reliability.
- Existing alignment methods focus on correctness or calibration; this paper is the first to adopt AURC as an alignment objective.
- The RLSR framework uses reinforcement learning to directly optimize AURC, achieving better risk-coverage trade-offs.

## Full text

Large language models (LLMs) are increasingly deployed as critical decision-making components in high-stakes real-world AI systems, rendering LLM reliability a foremost practical concern. In this paper, we focus on enhancing LLM reliability through selective prediction (SP), a strategy that allows an LLM to only predict for inputs where it is likely to be correct (i.e., coverage) and hence reduce the error rate (i.e., risk) on that portion of inputs -- flagging the remaining inputs for future human discretion. In other words, SP improves LLM reliability by balancing the risk-coverage trade-off and enabling seamless human-AI collaboration. To integrate SP into LLMs, we focus on the LLM post-training alignment stage and propose to align LLMs with SP performance metrics, in contrast with existing LLM alignment methods that focus primarily on correctness or calibration metrics. Specifically, we propose a novel alignment framework, Reinforcement Learning for Selection Reward (RLSR), which targets the area under the risk-coverage curve (AURC) -- a popular SP performance metric -- as its alignment objective. RLSR achieves substantially better risk-coverage trade-off compared to multiple alignment baselines on both in-domain and out-of-domain tasks.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_489579d3c09e0dc7eae3669a