Auditing the Risk Claims of Distributional Reinforcement Learning: Majority of Assertions Refuted

Distributional reinforcement learning agents learn full return distributions to support risk-sensitive control and safety monitoring, but this study conducts the first systematic audit of the truthfulness of their risk claims. The research team developed a method combining a decision-relevant screening metric (excess Wasserstein gap), snapshot-restart Monte Carlo ground truth, and multiple hypothesis testing control (permutation tests, bootstrap refutation, FDR). They audited three algorithms (QR-DQN, C51, IQN) across 33 runs on MinAtar. Results show that 40-95% of the strongest risk claims are statistically refuted, and the ranking of top claims is indistinguishable from random guessing. Attributing analysis reveals the risk signal is a structural artifact of training, not environment stochasticity—formed early, uncorrelated with final score, and seed-specific. At full Atari scale (Breakout), all top claims from a pretrained QR-DQN are refuted. Positive controls confirm the audit's validity: 96-100% of true claims are confirmed. Acting on CVaR advice at the most flagged states ranges from beneficial to significantly worse than chance. The authors show that risk-targeted training or ensembling does not remove the artifact; recalibration passes the audit only by nullifying the claims, indicating the risk head is uninformative. The toolkit is released, along with documentation of two silent pitfalls that produced convincing but wrong audits.