---
title: "Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2606.30587v1"
permalink: "https://feed.astralor.com/items/itm_39a62d7426540c6cd05dcfdb"
published: "2026-06-29T17:31:10.000Z"
score: 59
channel: "ai"
tags: ["Code Vulnerability Detection","Large Language Models","Halo Effect","Framing Effect","Cognitive Heuristics","Anchoring Effect"]
---

# Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection

## At a glance

This paper presents the first systematic exploration of cognitive heuristics in LLM-driven code vulnerability detection. The authors introduce a controlled framework that holds the code fixed and varies the surrounding context to trigger three heuristics: halo effect (author attribution), framing effect (task objectives and consequences), and anchoring effect (prior analysis results). Evaluating eight LLMs across three programming languages, they find all models are susceptible. Cross-model average susceptibility is highest for framing at 33.2%, followed by anchoring at 23.5% and halo at 18.4%. Code-level analysis reveals that vulnerabilities requiring semantic reasoning are more susceptible than those identifiable via pattern matching. Models often change their verdict from safe to vulnerable without correctly identifying the vulnerability. A proof-of-concept black-box cognitive attack can suppress up to 97% of detected vulnerabilities. The findings indicate cognitive susceptibility is a consistent and exploitable property of LLM-based vulnerability detection.

## Key points

- First systematic exploration of cognitive heuristics in LLM-based code vulnerability detection.
- Framing effect shows highest susceptibility (33.2%), followed by anchoring (23.5%) and halo (18.4%).
- Vulnerabilities requiring semantic reasoning are more susceptible than pattern-matching ones.
- Proof-of-concept black-box attack suppresses up to 97% of previously detected vulnerabilities.

## Full text

Researchers and practitioners increasingly apply Large Language Models (LLMs) for automated vulnerability detection. Recent work has shown that LLMs are susceptible to the same cognitive heuristics that bias human judgment. Yet, no work has investigated whether these heuristics affect a model's assessment of code vulnerabilities. In this paper, we present the first systematic exploration of cognitive heuristics in LLM-driven code vulnerability detection. We introduce a controlled framework that holds the code fixed and only varies the surrounding context to trigger three cognitive heuristics: the halo effect through author attribution, the framing effect through task objectives and consequences, and the anchoring effect through prior analysis results. Within this framework, we evaluate eight LLMs across three programming languages and perform both quantitative and code-level analyses. Our findings demonstrate that all evaluated models are susceptible to these heuristics. Cross-model average susceptibility is highest for framing at 33.2%, followed by anchoring at 23.5% and halo at 18.4%. Code-level analysis reveals that vulnerabilities that require semantic reasoning for detection are more susceptible to cognitive heuristics than those identifiable through pattern matching. Furthermore, models often change their verdict from safe to vulnerable based on the cognitive condition, without accurately identifying the actual vulnerability. To highlight the practical impact, we demonstrate a proof-of-concept black-box cognitive attack that can suppress up to 97% of previously detected vulnerabilities. These findings indicate that cognitive susceptibility is a consistent and exploitable property of LLM-based vulnerability detection.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_39a62d7426540c6cd05dcfdb