---
title: "Repair or Resample? Rethinking Failure Debugging in LLM Multi-Agent Systems"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.25920v1"
permalink: "https://feed.astralor.com/items/itm_6691e15aac6cb15ab56828ac"
published: "2026-08-26T15:33:47.000Z"
score: 76
channel: "ai"
tags: ["LLM","SymFail","SymTrace","Multi-agent Systems","Failure Debugging"]
---

# Repair or Resample? Rethinking Failure Debugging in LLM Multi-Agent Systems

## AI commentary

> This paper debunks an industry illusion: many teams fix agent failures simply by retrying. The numbers are sobering—rerun repair rate is only 6.9%, while symptom-driven targeted intervention nearly triples it. For those building multi-agent products, SymTrace and SymFail offer practical blueprints for building regression sets without relying on luck.

## At a glance

As large language model-based multi-agent systems (MASs) are increasingly applied to long-horizon complex tasks, their reliability has become the core bottleneck hindering real-world deployment. Existing debugging and repair methods typically rely on rerunning the entire execution trajectory and resampling, but whether these methods cause causal repairs or merely stochastic repairs by leveraging LLM sampling randomness remains an open question. To evaluate the effectiveness of MAS repair methods, this paper introduces SymTrace, a controlled evaluation framework that records MAS execution traces and establishes intervention anchors. During replay, it reconstructs the pre-anchor execution from logs and only regenerates the downstream trajectory, enabling reliable reproduction of MAS failures. The authors also construct SymFail, a dataset of 536 human-annotated failure trajectories with graph-linked locations, categories, and trace evidence. A large-scale empirical study across three mainstream MAS frameworks reveals that existing unguided rerun methods are highly unreliable, with low failure reproduction and repair rates of only 67.97% and 6.90%, respectively. Building on these findings, they further explore a symptom-driven intervention method that successfully repairs 20.15% of failed cases, a 191.89% improvement over state-of-the-art repair methods. This study provides actionable insights for MAS debugging and repair research, paving the way for robust deployment of multi-agent systems.

## Key points

- SymTrace provides a controlled evaluation framework that records execution traces and uses intervention anchors to reliably reproduce MAS failures via log replay and downstream regeneration.
- SymFail dataset includes 536 human-annotated failure trajectories with graph-linked locations, categories, and trace evidence.
- Empirical results show unguided rerun methods are unreliable: 67.97% failure reproduction and 6.90% repair rate; symptom-driven intervention raises repair to 20.15%, a 191.89% improvement over SOTA.

## Full text

As large language model (LLM)-based multi-agent systems (MASs) are increasingly applied to long-horizon complex tasks, their reliability has emerged as the core bottleneck hindering their real-world deployment. Existing MAS debugging and repair methods typically rely on rerunning and resampling the entire execution trajectory. However, a fundamental question remains to be answered: do these methods causally repair MAS failures or merely stochastically repair by leveraging the randomness of LLM sampling? To evaluate the effectiveness of MAS repair methods, we introduce SymTrace, a controlled evaluation framework that records the MAS execution trajectory and establishes intervention anchors. During replay, it effectively reconstructs the execution before the anchor using recorded logs and only regenerates the downstream trajectory, thereby enabling the reliable reproduction of MAS failures. We further construct the dataset SymFail, comprising 536 human-annotated failure trajectories with graph-linked locations, categories, and trace evidence. Based on these foundations, we conduct a large-scale empirical study across three mainstream MAS frameworks. Our findings reveal that existing unguided rerun methods are highly unreliable, exhibiting low failure reproduction and repair rates (only 67.97% and 6.90%, respectively). Building upon these findings, we further explore the effectiveness of a symptom-driven intervention method, which successfully repairs 20.15% of the failed cases (a 191.89% improvement to state-of-the-art repair methods). This study aims to provide actionable insights for MAS debugging and repair research, paving the way for the robust deployment of multi-agent systems.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_6691e15aac6cb15ab56828ac