---
title: "Reason-Mediated Behavioral Models for Auditing LLM Social Simulators"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.24649v1"
permalink: "https://feed.astralor.com/items/itm_adfe8fb8307f258931b10811"
published: "2026-07-27T16:47:27.000Z"
score: 79
channel: "tech"
tags: ["AI Evaluation","Large Language Models","Reason Audit","Social Simulator","Behavioral Model"]
---

# Reason-Mediated Behavioral Models for Auditing LLM Social Simulators

## AI commentary

> This paper hits a neglected pain point in LLM social simulator evaluation: we settle too easily for answer similarity while ignoring systematic reasoning biases. For practitioners in AI social science, synthetic data generation, and explainable AI, this audit framework helps identify simulators that sound plausible but take shortcuts. In my view, using reason states as interpretability indicators is a significant step in the right direction.

## At a glance

Large language models (LLMs) are increasingly deployed as social simulators, e.g., synthetic survey respondents. Current evaluations mainly check whether simulated outcomes match human final answers, but ignore the rationale behind them. Through a 94-person sunscreen concept test, this paper maps human open-ended rationales into signed reason states, showing that human-derived reasons substantially improve hold-out purchase intent prediction. LLM-simulated reasons, though sounding plausible, often echo the concept board instead of recovering the respondent's acceptance or rejection path. The work contributes an interpretable audit framework to assess whether a simulator's stated reasons align with human evidence.

## Key points

- Proposes a Reason State audit method that encodes human open-ended rationales into signed binary states to evaluate LLM social simulator reasoning fidelity.
- In a 94-person sunscreen concept test, human-derived reason states substantially improve purchase intent prediction, while LLM-simulated reasons often echo the concept board rather than recover actual decision paths.
- The framework does not require causal identification but provides an interpretable test of whether simulator reasons align with human evidence.

## Full text

Large language models are increasingly used as social simulators, including as synthetic survey respondents. Most evaluations ask whether simulated outcomes resemble human outcomes. We argue that this is necessary but too weak: a simulator can match the final answer while using the wrong rationale-derived reason pattern. We study this problem through a 94-person sunscreen concept test in which each respondent evaluated three product concepts and wrote open-ended rationales. We map those rationales into signed reason states $Z$, where positive signs support adoption and negative signs block it. This gives a practical audit: holding respondent descriptors $D$, category context $K$, and concept treatment $X$ fixed, do human rationale-derived reasons help predict behavior $Y$, and can an LLM simulate the same reason state without seeing the human rationale or outcome? Human rationale-derived reasons substantially improve held-out prediction of purchase intent. LLM-simulated reasons are more brittle: they often sound plausible, but frequently echo the concept board rather than recover the respondent's acceptance or rejection path. The paper contributes an evaluation framework for social simulators. Reason states do not identify natural causal effects by themselves, but they provide an interpretable test of whether a simulator's stated reasons align with human evidence.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_adfe8fb8307f258931b10811