PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image

Whole-slide image (WSI) diagnosis typically involves locating suspicious regions at low magnification, examining cellular details at high magnification, and integrating multi-scale findings into a final diagnosis. However, existing pathology AI benchmarks mostly rely on pre-cropped patches or pre-extracted slide features, failing to test a model's ability to 'seek' evidence directly from gigapixel images. PathAgentBench fills this gap by decomposing pathological diagnosis into four quantifiable capabilities: cross-modal matching (aligning images with text descriptions), text-guided region localization, multi-scale reasoning (combining findings across magnifications), and autonomous exploration—where models must find diagnostic regions in whole slides without additional cues. Annotations were provided by ten board-certified pathologists across 1,822 TCGA slides and 190 private breast cancer slides. Twenty models were evaluated, including general-purpose (e.g., LLaVA), medical (e.g., BiomedCLIP), and pathology-specific (e.g., CONCH). Results are sobering: multi-scale reasoning accuracy exceeds 93%, but the best region localization IoU is below 0.09—worse than a simple center heuristic. During autonomous exploration, the high-magnification hit rate is only 2%, indicating models cannot reliably 'see' lesions. PathAgentBench not only serves as an evaluation tool but also exposes a fundamental weakness in current VLMs regarding open-world evidence acquisition, providing critical guidance for the design of next-generation pathology AI.