---
title: "Beyond the Keyword: A Human-Annotated Benchmark for Arabic Anxiety-Related Discourse"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.07949v1"
permalink: "https://feed.astralor.com/items/itm_acf3bbedf54453d126d4ce10"
published: "2026-11-05T12:45:49.000Z"
score: 59
channel: "tech"
tags: ["RAG","Beyond","Keyword","Human-Annotated Benchmark","Arabic Anxiety-Related Discourse","Arabic Twitter"]
---

# Beyond the Keyword: A Human-Annotated Benchmark for Arabic Anxiety-Related Discourse

## At a glance

Keyword retrieval returns expressions of personal anxiety together with many other uses of anxiety-related language. We present a benchmark of 1,000 Arabic Twitter/X posts retrieved with the keyword qalaq ("anxiety/worr…

## Key points

- Keyword retrieval returns expressions of personal anxiety together with many other uses of anxiety-related language.
- We present a benchmark of 1,000 Arabic Twitter/X posts retrieved with the keyword qalaq ("anxiety/worry") and annotated for six discourse functions.
- Two native Arabic-speaking annotators labeled every post independently, reaching 51.1% raw agreement (Cohen's $κ= 0.378$).

## Full text

Keyword retrieval returns expressions of personal anxiety together with many other uses of anxiety-related language. We present a benchmark of 1,000 Arabic Twitter/X posts retrieved with the keyword qalaq ("anxiety/worry") and annotated for six discourse functions. Two native Arabic-speaking annotators labeled every post independently, reaching 51.1% raw agreement (Cohen's $κ= 0.378$). We then reviewed all 489 disagreements and all 511 agreements under the same guidelines. The review changed 120 of the agreed labels (23.5%), which shows that agreements need auditing as well as disagreements. The final labels form a consensus reference; on a random sample of 200 posts, an additional annotator agreed with it on 60.5% of posts ($κ= 0.517$). Against this reference, Fanar-C-2-27B reaches 79.8% accuracy and macro-F1 0.784 ($κ= 0.749$). Its errors concentrate in the residual category (41.1% of errors; precision 0.905, recall 0.447), whose definition is internally inconsistent in the executed prompt, so the low recall cannot be attributed to the model alone. Sociopolitical Anxiety (27.1%) outnumbers Personal Emotional Expression (22.0%); these are assignment proportions in the sample, not estimates of prevalence or retrieval precision. We release the guidelines, a residual-subtype scheme, a fixed data split, label provenance, and evaluation code.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_acf3bbedf54453d126d4ce10