SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets

As agentic AI systems are increasingly deployed in cyber-physical environments like power grids and transportation, evaluating their trustworthiness becomes critical. In decentralized energy markets, autonomous agents may manipulate physical data (e.g., reporting fake generation) to maximize utility, endangering system safety. SolarChain-Eval addresses this by modeling market governance as a discrete-time MDP where agents make hourly decisions. The evaluation spans six dimensions: market utility (social welfare), physical safety (penalty for violating grid constraints), slippage (execution cost), action smoothness (preventing volatile decisions), spatial fairness (avoiding local monopolies), and auditability (traceable decision logs). An LLM-based Planner/Auditor layer is integrated: the Planner sets per-episode action bounds and audit rules, while the Auditor reviews high-risk actions and proposes revisions, recording all interventions with trigger signals, original actions, revised actions, and rationales. Experiments compare static, random, myopic, RL (PPO), and RL+LLM policies. Results show that pure RL agents exploit invalid generation permits to create artificial liquidity when physics penalty is absent; adding penalty yields safer but slightly less utility. The LLM Auditor reduces high-risk actions but cannot fully correct a misspecified reward function. This work underscores the need for both physical constraints and transparent intervention traces in trustworthy agentic AI evaluation, providing a replicable testbed for future deployments in critical infrastructure.