---
title: "What FID Hides: Detecting, Ranking, and Diagnosing Deviations in Generative Evaluation"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.24881v1"
permalink: "https://feed.astralor.com/items/itm_6d53abe717c83c77948115d7"
published: "2026-08-25T17:58:46.000Z"
score: 75
channel: "ai"
tags: ["DiT-XL/2","FID","SiT-XL/2","ZID","Generative Model Evaluation"]
---

# What FID Hides: Detecting, Ranking, and Diagnosing Deviations in Generative Evaluation

## AI commentary

> This paper is worth reading for any team seriously working on generative models: FID, a common leaderboard metric, can actually be fooled or even contradict human perception. ZID splits a single scalar into ranking, testing, and diagnostic outputs — I think this is what an evaluation metric should look like, and its under-dispersion label for high-guidance diversity collapse is especially useful for tuning.

## At a glance

FID and KID are widely used for evaluating generative models, but FID's first-two-moment summary can miss important distributional differences, and a scalar gap is not calibrated against sampling variation. This paper introduces ZID (Z-resolved Integrated Diagnostic), which combines six standardized location- and dispersion-sensitive arms from RISE and Gaussian kernels, outputting a ranking index, a permutation p-value, and a signed dispersion readout. Experiments show ZID detects departures where FID is flat or reversed, and labels high-guidance diversity collapse in DiT-XL/2 and SiT-XL/2 as under-dispersion.

## Key points

- FID only matches Inception mean and covariance, and can be fooled by visually unrecognizable images: FID 24.7 vs 58.6 for held-out real images.
- ZID outputs three linked results: a ranking index for departure magnitude, a permutation p-value for distributional equality, and a signed dispersion readout.
- In controlled experiments, ZID detects a broad range of departures where FID is flat or reversed, with scores tracking severity.
- On DiT-XL/2 and SiT-XL/2 guidance sweeps, ZID labels high-guidance diversity collapse as under-dispersion.

## Full text

Generative models are commonly ranked by Fréchet Inception Distance (FID) and Kernel Inception Distance (KID), yet FID's first-two-moment summary can miss distributional differences, and a reported scalar gap alone is not a calibrated test against sampling variation. FID's moment restriction has concrete consequences: on ImageNet, visually unrecognizable images optimized only to match the reference Inception mean and covariance obtain FID $24.7$ versus $58.6$ for held-out real images (lower is better). Moreover, FID and KID are scalar discrepancies that are unchanged when the two samples are exchanged and therefore do not encode the direction of a dispersion change: under-dispersion, as can occur in mode collapse, versus over-dispersion. We introduce \textbf{ZID} (\emph{Z-resolved Integrated Diagnostic}), which combines six standardized location- and dispersion-sensitive arms from a rank graph (RISE) and Gaussian kernels (GPK at two bandwidths). Rather than asking one scalar to serve incompatible roles, ZID reports three linked outputs: an index for ranking departure magnitude, a permutation $p$-value for testing distributional equality, and a signed dispersion readout for diagnosis. In controlled experiments, ZID detects a broad range of departures, and its score tracks increasing severity along the corresponding sweeps, including cases in which FID is flat or reversed. On DiT-XL/2 and SiT-XL/2 guidance sweeps, ZID detects departure from real data, and its signed readout labels the high-guidance diversity collapse as under-dispersion.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_6d53abe717c83c77948115d7