---
title: "SocialArena: Evaluating Social Intelligence in Large Language Models"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.39247v1"
permalink: "https://feed.astralor.com/items/itm_260f3ebeb6ec186f3146bf92"
published: "2026-10-29T03:00:43.000Z"
score: 56
channel: "tech"
tags: ["LLM","RAG","SocialArena","Evaluating Social Intelligence","Large Language Models","LLMs","Although LLMs"]
---

# SocialArena: Evaluating Social Intelligence in Large Language Models

## 速览

We introduce SocialArena, a benchmark for systematically defining and evaluating the social intelligence of large language models (LLMs). Although LLMs are increasingly deployed in socially interactive applications such…

## 关键点

- We introduce SocialArena, a benchmark for systematically defining and evaluating the social intelligence of large language models (LLMs).
- Although LLMs are increasingly deployed in socially interactive applications such as mental health counseling, companionship chatbots, and educational tutoring, existing evaluations remain fragmented focusing on isolate…
- This work makes three main contributions to address these limitations.

## 全文

We introduce SocialArena, a benchmark for systematically defining and evaluating the social intelligence of large language models (LLMs). Although LLMs are increasingly deployed in socially interactive applications such as mental health counseling, companionship chatbots, and educational tutoring, existing evaluations remain fragmented focusing on isolated capabilities or single scenarios and evaluation reliability is largely unverified. This work makes three main contributions to address these limitations. (1) We first propose a unified three-tier framework that defines social intelligence along three capability dimensions: role-playing ability, mental state reasoning, and behavioral reasoning. To operationalize this definition, we design a role-playing dialogue paradigm with explicit character thinking for evaluation, enabling simultaneous assessment of these capabilities in a manner that mirrors realistic human interactions. (2) To ensure diversity across social scenarios and thereby prevent evaluation conclusions from being confined to specific scenarios, we introduce the U-TRIMF framework, which guides us in synthesizing social contexts with broad coverage. (3) To obtain reliable evaluation, we invest $132.4K to recruit and train human annotators and adopt a human-annotated arena-style protocol, where annotators compare model responses and provide preference labels along with natural-language critiques. We hope that SocialArena can provide valuable insights and guidance for future LLM improvements.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_260f3ebeb6ec186f3146bf92