---
title: "HalluTruthQA-4K: A Fine-Grained Corpus and Annotation Process for Arabic Hallucination Detection and Truth Verification"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.03966v1"
permalink: "https://feed.astralor.com/items/itm_52a157b776cbd62b656db5d8"
published: "2026-08-04T17:33:38.000Z"
score: 80
channel: "ai"
tags: ["arXiv","HalluTruthQA-4K","Fact Verification","Hallucination Detection","Dataset Construction","Arabic"]
---

# HalluTruthQA-4K: A Fine-Grained Corpus and Annotation Process for Arabic Hallucination Detection and Truth Verification

## AI commentary

> The most notable aspect is the move from coarse binary hallucinations to character-level span annotations for a resource-poor language like Arabic. For researchers in multilingual NLP or Arabic LLM reliability, this is a ready-to-use resource. However, as a preprint, its quality and impact will ultimately be validated by the shared task outcomes and community adoption.

## At a glance

Large language models can generate fluent Arabic answers while introducing factual errors that are difficult to identify and verify. Existing Arabic hallucination resources often assign a binary label to an entire response, providing limited information about the exact erroneous content, the reason for the error, or the correct factual answer. This paper presents HalluTruthQA-4K, an expanded version of the HalluTruthQA resource containing 4,000 expert-curated Arabic question-answering instances across four knowledge-intensive domains: Islamic knowledge, history, science, and geography. Each instance pairs an Arabic question with a model-generated response, a verified reference answer, and five plausible distractors. Hallucinated responses are additionally annotated with character-level erroneous spans, human-written explanations, and hierarchical hallucination types. The corpus contains 1,643 hallucinated and 2,357 non-hallucinated responses, with 1,843 annotated erroneous spans. The paper details resource construction, annotation guidelines, taxonomy, data format, inter-annotator agreement, and corpus statistics, providing a reusable resource for hallucination detection, span-level error localization, explanation generation, factual verification, and broader evaluation of factual reliability in Arabic language models.

## Key points

- HalluTruthQA-4K contains 4,000 expert-curated Arabic QA instances across Islamic knowledge, history, science, and geography.
- Unlike existing binary-label resources, it provides character-level erroneous spans, human-written explanations, and hierarchical hallucination types.
- The corpus includes 1,643 hallucinated and 2,357 non-hallucinated responses, with 1,843 annotated erroneous spans.
- Serving as the official dataset for Track 2 of the HalluScoring 2026 shared task, it supports hallucination detection, error localization, explanation generation, and fact verification.

## Full text

Large language models can generate fluent Arabic answers while introducing factual errors that are difficult to identify and verify. Existing Arabic hallucination resources often assign a binary label to an entire response, indicating whether it is hallucinated or non-hallucinated, but provide limited information about the exact erroneous content, the reason for the error, or the correct factual answer. We present HalluTruthQA-4K, an expanded version of the HalluTruthQA resource containing 4,000 expert-curated Arabic question-answering instances across four knowledge-intensive domains: Islamic knowledge, history, science, and geography. Serving as the official dataset for Track 2 of the HalluScoring 2026 shared task, HalluTruthQA-4K extends our original corpus to 4,000 instances. Each instance pairs an Arabic question with a model-generated response, a verified reference answer, and five plausible distractors. Hallucinated responses are additionally annotated with character-level erroneous spans, human-written explanations, and hierarchical hallucination types. The corpus contains 1,643 hallucinated and 2,357 non-hallucinated responses, with 1,843 annotated erroneous spans. We describe the resource construction and annotation methodology, including question selection, controlled answer generation, candidate construction, expert annotation, independent verification, adjudication, and quality control. We also document the annotation guidelines, taxonomy, data format, inter-annotator agreement, and corpus statistics. HalluTruthQA-4K provides a reusable resource for hallucination detection, span-level error localization, explanation generation, factual verification, and the broader evaluation of factual reliability in Arabic language models.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_52a157b776cbd62b656db5d8