代理记忆中毒检测:基于取证轨迹签名的行为不变性方法
评分
58精选理由
提出了一种新颖的行为不变性检测方法,实验充分(多模型验证、高AUC、可实时阻断),且具有实际应用前景,对AI安全领域贡献突出。
AI 点评
「该工作提供了一种无需模型内部访问即可检测LLM代理记忆中毒攻击的鲁棒方法,且能区分不同攻击类型,对代理安全防御具有重要实践价值。」
速览
该论文发现LLM代理在持久性记忆中毒攻击下存在行为不变性:成功攻击必须调用memory_recall_fact再调用email_send_email,而非渗出会话极少有此顺序。基于此规则的检测方法AUC达0.9563,随机森林在19个轨迹特征上AUC提升至0.9904。跨9个模型(7B-120B参数)验证,6/9分割AUC=1.000,且对前沿模型(GPT-4.1、GPT-4o)无需重训练即可泛化。仅使用前缀的变体AUC=0.934,可实现实时阻断。提示注入攻击产生不同轨迹(得分0.541),可区分攻击类型。
- 发现LLM代理在记忆中毒攻击下的行为不变性:内存读取调用必须在邮件发送之前。
- 基于此不变性的简单规则检测AUC=0.9563,随机森林模型AUC=0.9904。
- 跨9个模型验证,6/9分割AUC=1.000,对GPT-4.1/4o无需重训练泛化。
- 前缀检测变体AUC=0.934,可实现实时阻断;提示注入攻击轨迹不同(得分0.541),可区分攻击类型。
正文
Computer Science > Cryptography and Security
arXiv:2606.30566v1 (cs) [Submitted on 29 Jun 2026]Title:Forensic Trajectory Signatures for Agent Memory Poisoning Detection
Authors:Jun Wen Leong View a PDF of the paper titled Forensic Trajectory Signatures for Agent Memory Poisoning Detection, by Jun Wen Leong View PDF HTML (experimental)Abstract:We discover a behavioral invariant in LLM agents under persistent memory poisoning: in architectures where routing information is retrieved through observable memory-tool invocations, successful attacks require calling memory_recall_fact before email_send_email, a transition that non-exfiltrating sessions rarely exhibit. Under the evaluated architecture, this invariant follows from the attack's information-retrieval dependency rather than being merely an empirical correlation, and suppressing it breaks the attack. A simple rule exploiting this invariant alone achieves AUC = 0.9563. A Random Forest classifier over 19 trajectory features refines it to AUC = 0.9904 (BCa 95% CI [0.987, 0.993], N=10,000 resamples), demonstrating that the attack imprints on multiple independent behavioral channels. The signature is overdetermined: removing all recall-related features (half the feature set) leaves AUC unchanged at 0.990, confirming that memory poisoning induces a distributed trajectory signature rather than a single observable anomaly. Cross-model hold-out on 9 models (7B-120B parameters) confirms AUC = 1.000 on 6/9 hold-out splits, with all three exceptions mechanistically explained. The invariant generalizes to frontier models (GPT-4.1, GPT-4o) without retraining. A strictly prefix-only variant achieves AUC = 0.934, suggesting that real-time blocking is feasible with moderate degradation. The boundary is forensically useful: prompt-injection attacks that bypass memory produce a distinct trajectory (score = 0.541), enabling incident responders to distinguish memory-channel attacks from prompt-injection attacks using tool-call logs alone.
| Comments: | 11 pages, 4 figures. Companion note to arXiv:2605.08442 |
| Subjects: | Cryptography and Security (cs.CR); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.30566 [cs.CR] |
| (or arXiv:2606.30566v1 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2606.30566 Focus to learn more arXiv-issued DOI via DataCite |
Submission history
From: Jun Wen Leong [view email][v1] Mon, 29 Jun 2026 17:09:03 UTC (63 KB)
Full-text links:
Access Paper:
-
View a PDF of the paper titled Forensic Trajectory Signatures for Agent Memory Poisoning Detection, by Jun Wen Leong
- View PDF
- HTML (experimental)
- TeX Source
Current browse context:
cs.CR < prev | next >new | recent | 2026-06 Change to browse by: cs
cs.LG
References & Citations
export BibTeX citation Loading...BibTeX formatted citation
× Data provided by:Bookmark
Bibliographic and Citation Tools
Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, MediaCode, Data and Media Associated with this Article
alphaXiv Toggle alphaXiv (What is alphaXiv?) Links to Code Toggle CatalyzeX Code Finder for Papers (What is CatalyzeX?) DagsHub Toggle DagsHub (What is DagsHub?) GotitPub Toggle Gotit.pub (What is GotitPub?) Huggingface Toggle Hugging Face (What is Huggingface?) ScienceCast Toggle ScienceCast (What is ScienceCast?) DemosDemos
Replicate Toggle Replicate (What is Replicate?) Spaces Toggle Hugging Face Spaces (What is Spaces?) Spaces Toggle TXYZ.AI (What is TXYZ.AI?) Related PapersRecommenders and Search Tools
Link to Influence Flower Influence Flower (What are Influence Flowers?) Core recommender toggle CORE Recommender (What is CORE?) About arXivLabsarXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)相关条目
精选DeepSeek V4 在 OpenRouter 上的代币份额翻倍,智能体工作负载成主要驱动力
DeepSeek V4 的快速采用表明市场对高性能、低成本推理模型的强烈偏好,尤其是智能体场景将重塑 AI 部署格局。
91
精选启用两项设置使ARC-AGI-3基准测试分数提升三倍
这项发现太关键了!它表明模型的性能不仅取决于训练,还取决于推理时的配置。对于AI工程师和研究者来说,这是一个必须注意的优化方向。特别是那些在资源受限设备上部署模型的团队,可能会因此找到低成本提升能力的方法。
91
精选GPT-5.6:前沿智能与前沿效率的融合
这条消息值得关注,因为它标志着大模型竞争从单纯拼参数转向拼效率。对于AI开发者和企业决策者来说,GPT-5.6意味着可以用更少预算获得更高性能,尤其适合那些被推理成本挡在门外的长上下文或Agent场景。OpenAI这一步,可能重新定义‘前沿’的标准。
91
精选LFM2.5-Encoder:面向CPU快速长上下文推理的新模型
这篇博文让我眼前一亮,因为它直接解决了大模型在CPU上跑长上下文的痛点。对于没有高端GPU的团队或个人开发者,LFM2.5-Encoder系列提供了可立即使用的开源方案。尤其推荐给做RAG、文档搜索或代码分析的朋友——现在用普通服务器就能处理百万级token的文档了。
91
精选GPT-Red:OpenAI通过自我对弈实现AI红队自动化,提升模型鲁棒性
GPT-Red 是红队测试从人工到自动化的里程碑,特别适合安全工程师和对齐研究者关注。它直接降低了测试成本,但也引发了对“自动攻击”本身的伦理讨论——值得所有关注AI治理的人深入思考。
91
精选OpenAI推出GPT-5.5生物漏洞赏金计划
这个计划值得关注,因为它首次将AI漏洞赏金制度拓展到生物安全领域,不仅为安全社区提供了新的研究方向,也为AI风险治理提供了实际演练场。对模型安全研究者、生物安全专家以及关心AI双重用途的从业者尤其重要。
91
精选vLLM推出原生速度Transformers建模后端,大幅提升推理性能
这项更新对于AI工程团队非常关键:无需格式转换意味着能更快地迭代模型,而2-5倍的速度提升直接降低GPU成本。如果你正在做LLM在线服务或者批量推理,这几乎是必读的技术更新。
90
精选Google DeepMind 发布手语转文本模型 SL2T,助力聋人用户
这则消息的意义不仅在于技术突破,更在于AI开始真正关注边缘群体的需求。对开发者而言,SL2T的架构思路和技术路径值得深入研究;对无障碍倡导者来说,这是推动行业标准的重要案例。