---
title: "Measuring Risk, Not Harm: Detecting Psychological Harm Risk Factors in LLM Conversations"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.06806v1"
permalink: "https://feed.astralor.com/items/itm_3c2b32322d7e1f17591c9fbf"
published: "2026-11-04T17:22:23.000Z"
score: 59
channel: "ai"
tags: ["LLM","安全","Measuring Risk","Not Harm","Detecting Psychological Harm","Risk Factors","LLM Conversations Psychological","Psychological"]
---

# Measuring Risk, Not Harm: Detecting Psychological Harm Risk Factors in LLM Conversations

## At a glance

Psychological harms from interactions with large language models cannot be monitored directly because they may emerge over repeated interactions and depend strongly on subjective individual user context. We propose meas…

## Key points

- Psychological harms from interactions with large language models cannot be monitored directly because they may emerge over repeated interactions and depend strongly on subjective individual user context.
- We propose measuring risk, not harm: detecting observable AI behaviors that constitute risk factors for psychological harm at the AI response level and aggregating these signals to support longitudinal monitoring.
- We operationalize eight such behaviors and construct a synthetic multi-label corpus of 47,062 AI responses and 373,873 response-factor labels, with targeted human annotations for validation and evaluation.

## Full text

Psychological harms from interactions with large language models cannot be monitored directly because they may emerge over repeated interactions and depend strongly on subjective individual user context. We propose measuring risk, not harm: detecting observable AI behaviors that constitute risk factors for psychological harm at the AI response level and aggregating these signals to support longitudinal monitoring. We operationalize eight such behaviors and construct a synthetic multi-label corpus of 47,062 AI responses and 373,873 response-factor labels, with targeted human annotations for validation and evaluation. We then compare trained embedding-based classifiers, direct LLM judging, and a classifier--judge cascade. All detectors are model-agnostic and operate only on LLM outputs, requiring no access to internal model states. We apply these detectors to several out-of-distribution human-AI conversation data sets from prior research. On held-out human-labeled data, the best cascade achieves macro precision of 0.86, recall of 0.76, and F1 of 0.79. On WildChat, it reduces LLM-judge calls by 97.1\% relative to direct judging. On DelusionEval, risk-factor detections cluster within conversations. Together, our results show that response-level behavioral risk-factor detection can provide a scalable measurement layer for longitudinal psychological-safety monitoring without conflating observable AI behavior with latent user harm.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_3c2b32322d7e1f17591c9fbf