---
title: "Hierarchical Empirical-Bayes Naive Bayes: Minimax Smoothing and Calibration with AODE Extension"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.11162v1"
permalink: "https://feed.astralor.com/items/itm_29632cba257f47ccc74b6b15"
published: "2026-08-11T17:21:31.000Z"
score: 75
channel: "ai"
tags: ["AODE","HEB-NB","Naive Bayes","Probabilistic Calibration","Empirical Bayes","Tabular Data"]
---

# Hierarchical Empirical-Bayes Naive Bayes: Minimax Smoothing and Calibration with AODE Extension

## AI commentary

> This paper appeals to both ML researchers and practitioners: adaptive smoothing addresses the long-standing hyperparameter issue in Naive Bayes, while the theoretical lower bound adds rigor. It is especially valuable for readers dealing with high-cardinality categorical features and probabilistic calibration.

## At a glance

This paper addresses the limitations of fixed-strength smoothing in Naive Bayes classifiers on high-cardinality tabular data. The authors propose Hierarchical Empirical-Bayes Naive Bayes (HEB-NB), which learns Dirichlet prior concentrations via Type-II maximum likelihood, enabling adaptive smoothing and information sharing across classes while preserving closed-form inference. Theoretically, they establish a non-asymptotic l1 error bound matching the empirical-distribution minimax rate with vanishing bias, and a lower bound showing strict separation from Laplace smoothing. They also derive excess Bayes-risk and calibration error bounds. Empirically, across 31 UCI and OpenML benchmarks, HEB-NB achieves the best average Friedman rank on probabilistic metrics, with up to 22.1% log-loss reduction on high-cardinality datasets. The extended HEB-AODE consistently improves over vanilla AODE, and combining with mutual-information weighting reduces top-1 ECE by 41%-70%, demonstrating strong gains in probabilistic accuracy and calibration.

## Key points

- Proposes HEB-NB, which adaptively learns Dirichlet prior concentrations via empirical Bayes, replacing fixed smoothing strength.
- Establishes a non-asymptotic ℓ1 error bound matching the empirical-distribution minimax rate, with strict separation from Laplace smoothing.
- Extends to HEB-AODE, transferring adaptive smoothing to relaxed first-order dependency structures with consistent gains.
- On 31 UCI/OpenML benchmarks, achieves up to 22.1% log-loss reduction on high-cardinality datasets and 41%-70% top-1 ECE reduction with mutual-information weighting.

## Full text

The Naive Bayes (NB) classifier remains a standard choice for categorical data, yet its widely used smoothing rules, such as Laplace, Lidstone, Krichevsky-Trofimov, and the $m$-estimate, all prescribe a fixed smoothing strength that ignores feature cardinality, sample size, and class imbalance, inducing a non-vanishing bias on modern high-cardinality tabular data. We propose hierarchical empirical-Bayes Naive Bayes (HEB-NB), in which each class-feature conditional probability is smoothed by a Dirichlet prior whose concentration is learned data-adaptively via Type-II maximum likelihood, enabling principled information sharing across classes while retaining closed-form inference. We further introduce HEB average one-dependence estimators (HEB-AODE), showing that the adaptive smoothing transfers cleanly to structural relaxations of NB. Theoretically, we establish a non-asymptotic $\ell_1$ error bound for HEB-NB matching the empirical-distribution minimax rate plus a vanishing data-adaptive bias, together with a matching Laplace-tight lower bound that yields a finite-sample, risk-level strict separation from Laplace. We further derive a plug-in excess Bayes-risk bound via total-variation tensorization and a population top-1 expected calibration error (ECE) corollary. Empirically, across 31 UCI and OpenML benchmarks, HEB-NB attains the best average Friedman rank on probabilistic metrics, with up to 22.1% log-loss reductions on high-cardinality datasets and consistent improvements of HEB-AODE over vanilla AODE. Combining HEB-NB with mutual-information weighting reduces top-1 ECE by 41%-70%, demonstrating substantial gains in probabilistic accuracy and calibration.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_29632cba257f47ccc74b6b15