Reason-Mediated Behavioral Models for Auditing LLM Social Simulators

Large language models (LLMs) are increasingly used as social simulators, e.g., generating synthetic survey respondents. However, existing evaluations mainly check whether simulated outcomes statistically match human answer distributions, ignoring reasoning fidelity. This paper conducts a 94-person sunscreen concept test where each respondent evaluates three product concepts and writes open-ended rationales. Researchers map those rationales into signed reason states (positive signs support adoption, negative signs block it) and build an audit framework: holding respondent descriptors, category context, and concept treatment fixed, test whether human reason states can predict purchase intent, and whether an LLM can simulate the same reason state without seeing the human rationale or outcome. Results show that human-derived reason states significantly improve hold-out purchase intent prediction (AUC gain ~8%), while LLM-simulated reasons, though often sounding plausible, frequently echo the concept board rather than recover the respondent's actual acceptance or rejection path. The paper contributes an interpretable audit framework for social simulators, emphasizing that reason states alone do not identify causal effects but provide a practical test of whether a simulator's stated reasons align with human evidence.