When Local Monitors Miss Compositional Harm: Diagnosing Distributed Backdoors in Multi-Agent Systems

Quick overview

As multi-agent, tool-using LLM systems are deployed, a common safety net is a runtime monitor that checks each message, tool call, or step on its own. This paper reveals a fundamental hole: a distributed backdoor splits a harmful payload across agents, so every local check passes while the assembled object is the attack. The monitor can be right on every step and still miss the attack. The problem is not splitting itself, but 'local benignness': no fragment carries the harm, and what is left looks like ordinary benign traffic. The authors formalize an 'observability boundary': a monitor catches only what its view can tell apart from benign traffic. Experiments prove that once fragments look benign in the monitored view, no detector on that view can catch them. The signal returns only when the monitor sees the assembled object. A monitor trained only on benign traffic recovers the attack's code structure across held-out encodings (0.874 mean AUROC). A decoded-view gate blocks every tested attack given the encoding family. But seeing more is not enough: full-trace monitors and decoders still fail unless they reach the representation where the payload is exposed. Local safety is not global safety when harm is compositional, and the open problem is finding that representation.

• Distributed backdoors split harmful payloads across agents; local monitors can be correct on every step yet miss the attack.
• Formalized 'observability boundary': a monitor captures only what its view can distinguish from benign traffic.
• A monitor trained on benign traffic recovers attack code structure (mean AUROC 0.874); a decoded-view gate blocks tested attacks.
• Local safety is not global safety when harm is compositional; the open problem is finding the representation where payload is exposed.