---
title: "KnowHal: A Knowledge-Driven Benchmark for Comprehensive Multimodal Hallucination Evaluation"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.03782v1"
permalink: "https://feed.astralor.com/items/itm_04e76e8beee3e9fbd4d8868a"
published: "2026-08-04T15:03:29.000Z"
score: 78
channel: "ai"
tags: ["KnowHal","MLLMs","Multimodal Hallucination","Knowledge Hallucination"]
---

# KnowHal: A Knowledge-Driven Benchmark for Comprehensive Multimodal Hallucination Evaluation

## AI commentary

> This paper is worth attention because it formally brings knowledge hallucination into the mainstream of multimodal evaluation, instead of leaving knowledge errors scattered across small datasets. For teams building MLLM benchmarks or improving model trustworthiness, these 1,800 paired samples are a great starting point; for general researchers, the negative-question design reminds us that models tend to be too compliant with false premises.

## At a glance

Hallucination remains a critical challenge for developing trustworthy Multimodal Large Language Models (MLLMs). Existing benchmarks mainly focus on entity, attribute, and relation hallucinations, while knowledge-related failures are often investigated separately, lacking a unified evaluation framework. KnowHal explicitly incorporates knowledge hallucination into multimodal hallucination evaluation, spanning four dimensions: entity, attribute, relation, and knowledge. It constructs paired positive and negative questions over shared images and entities, enabling controlled comparisons among perceptual errors, knowledge-related errors, and false-premise acceptance. The benchmark contains 1,800 samples across 10 domains and 50 categories, built through a semi-automated pipeline combining LLM assistance, CLIP-based filtering, and human verification. Evaluations on 14 representative MLLMs show that the knowledge dimension consistently presents the greatest challenge for nearly all models, while most models exhibit substantial performance degradation on negative questions, revealing limited robustness to false premises. By unifying four hallucination dimensions with paired question design, KnowHal addresses an important gap and enables more comprehensive assessment of hallucinations in MLLMs.

## Key points

- Introduces KnowHal, the first benchmark unifying knowledge hallucination with entity, attribute, and relation hallucination in one evaluation framework.
- Uses paired positive and negative questions over shared images and entities for controlled comparison of perceptual errors, knowledge-related errors, and false-premise acceptance.
- Contains 1,800 samples across 10 domains and 50 categories, built with LLM assistance, CLIP-based filtering, and human verification.
- Evaluates 14 MLLMs, showing knowledge dimension is the hardest challenge and negative questions significantly degrade performance.

## Full text

Hallucination remains a critical challenge for developing trustworthy Multimodal Large Language Models (MLLMs). While existing benchmarks mainly focus on entity, attribute, and relation hallucinations, knowledge-related failures are often investigated separately, lacking a unified evaluation framework across different hallucination dimensions. To overcome this, we propose \textbf{KnowHal}, a benchmark that explicitly incorporates knowledge hallucination into multimodal hallucination evaluation spanning four dimensions: entity, attribute, relation, and knowledge. KnowHal constructs paired positive and negative questions over shared images and entities, enabling controlled comparisons among perceptual errors, knowledge-related errors, and false-premise acceptance. The benchmark contains 1,800 samples across 10 domains and 50 categories, constructed through a semi-automated pipeline combining LLM assistance, CLIP-based filtering, and human verification. We evaluate 14 representative MLLMs on KnowHal and conduct extensive analyses. Results show that the knowledge dimension consistently presents the greatest challenge for nearly all evaluated models, while most models exhibit substantial performance degradation on negative questions, revealing limited robustness to false premises. By unifying four hallucination dimensions with paired question design, KnowHal addresses an important gap in existing evaluation frameworks and enables a more comprehensive assessment of hallucinations in MLLMs.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_04e76e8beee3e9fbd4d8868a