---
title: "Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.27143v1"
permalink: "https://feed.astralor.com/items/itm_29df1c364f52b627349458d4"
published: "2026-07-29T17:15:33.000Z"
score: 80
channel: "ai"
tags: ["Mondrian Conformal Prediction","Imbalanced Classification","Human-in-the-loop","Conformal Prediction","Cost-Controlled Abstention","High-Stakes Decision"]
---

# Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark

## AI commentary

> This paper is a must-read for anyone deploying high-stakes AI systems. It reveals a critical flaw of standard conformal prediction in real-world imbalanced settings and provides a large-scale validated solution. For product managers and engineers in credit scoring, fraud detection, and healthcare, the combination of Mondrian CP with cost-controlled abstention is practically ready to use.

## At a glance

High-stakes decision systems (credit scoring, fraud detection, healthcare, industrial safety) suffer from severe class imbalance and asymmetric error costs. Standard marginal conformal prediction (CP) provides valid overall coverage but severely under-covers rare, costly minority classes (as low as 0.5%). This comprehensive benchmark on 15 real-world imbalanced tabular datasets, 7 classifiers, 3 calibration methods, and 3150 runs shows that Mondrian CP restores valid minority-class coverage, improving average minority coverage by 61.7 percentage points (p<1e-80). Combining Mondrian CP with cost-controlled abstention significantly reduces expected decision cost under realistic human review budgets. The study quantifies dataset-specific break-even thresholds, offering practical guidance for deploying distribution-free, cost-aware uncertainty quantification in high-stakes decision support.

## Key points

- Marginal conformal prediction severely under-covers minority classes in imbalanced high-stakes decisions, with coverage as low as 0.5%.
- Mondrian CP improves average minority-class coverage by 61.7 percentage points across 15 datasets.
- Combining Mondrian CP with cost-controlled abstention significantly reduces expected decision cost in human-in-the-loop settings.
- The paper provides dataset-specific break-even thresholds for practical deployment guidance.

## Full text

High-stakes decision systems in credit scoring, fraud detection, healthcare, and industrial safety require reliable uncertainty quantification under severe class imbalance and asymmetric error costs. Standard marginal conformal prediction (CP) provides valid overall coverage guarantees; however, we show that it severely under-covers rare, costly minority classes, with minority-class coverage dropping to as low as 0.5% on certain datasets. To characterize and address this limitation, we conduct a comprehensive benchmark comparing marginal CP, class-conditional (Mondrian) CP, and cost-controlled abstention mechanisms across 15 real-world imbalanced tabular datasets, 7 classification models, 3 probability calibration techniques, and 10 random seeds, resulting in 3,150 experimental runs. Our results show that Mondrian CP restores valid minority-class coverage, achieving an average minority-coverage improvement of 61.7 percentage points over marginal CP (p < 1e-80). Furthermore, combining Mondrian CP with cost-controlled abstention significantly reduces expected decision cost compared with standard decision boundaries, confidence-based rejectors, and risk-controlled rejectors under realistic human review budgets. We further quantify dataset-specific break-even thresholds at which deferring ambiguous instances to human experts becomes cost-effective. These findings provide practical guidance for deploying distribution-free, cost-aware uncertainty quantification in high-stakes decision support systems.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_29df1c364f52b627349458d4