---
title: "Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.14006v1"
permalink: "https://feed.astralor.com/items/itm_960fa450a96feb34dd32c266"
published: "2026-07-15T16:36:54.000Z"
score: 79
channel: "ai"
tags: ["Agentic Misalignment","AI Security","AI System Evaluation","Prompt Injection","Penetration testing","Behavioral Objective Violation"]
---

# Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation

## AI commentary

> This paper precisely hits the pain point of current AI system security assessment: traditional penetration testing only focuses on resource compromise, ignoring the unique behavioral manipulation risks of AI. For practitioners in AI security, red-blue teaming, and model auditing, this is a must-read baseline that provides a systematic framework and reusable testing workflow.

## At a glance

This paper proposes a shift in penetration testing for AI-enabled systems from traditional resource compromise to behavioral objective violation. The authors define AI-enabled penetration as the feasible induction of AI-governed behavior that violates operational objectives, extending conventional testing to adversarial pathways such as prompt injection, data poisoning, sensor manipulation, and tool misuse. A workflow covering identification of objectives, mapping behavior, analyzing influence surfaces, defining failure criteria, scenario-based testing, and reporting evidence is presented, with an example of an AI-enabled SOC assistant. This provides a technical framework for evaluating adversarial success in deployed AI systems.

## Key points

- Traditional penetration testing (resource compromise) is insufficient for AI-enabled systems, as adversaries can manipulate behavior without directly compromising infrastructure.
- Reframes AI penetration testing as objective-driven behavioral evaluation, covering adversarial pathways such as prompt injection, data poisoning, retrieval poisoning, and agentic misalignment.
- Proposes a seven-step workflow: identify operational objectives, map AI-governed behavior, analyze adversarial influence surfaces, define behavioral failure criteria, execute scenario-based tests, and report evidence linking adversarial action to objective violation.
- A running example of an AI-enabled SOC assistant illustrates penetration through behavioral influence rather than infrastructure compromise.

## Full text

Penetration testing traditionally evaluates whether adversaries can exploit weaknesses in software, infrastructure, configurations, or operational controls to achieve security-relevant compromise. This paradigm remains necessary for AI-enabled systems, but it is no longer sufficient. In such systems, adversaries may influence prompts, retrieved content, sensor inputs, training data, memory, tools, or human-AI interaction loops to alter system behavior without directly compromising the underlying infrastructure. This paper reframes penetration testing for AI-enabled systems as objective-driven behavioral evaluation. We define an AI-enabled system as one in which learned models materially influence behavior affecting operational outcomes, and we define AI-enabled penetration as the feasible induction of AI-governed behavior that violates one or more operational objectives under an explicit threat model. This definition preserves conventional penetration testing while extending it to adversarial pathways such as prompt injection, indirect prompt injection, data poisoning, sensor manipulation, retrieval poisoning, tool misuse, and agentic misalignment. We further propose a testing workflow that identifies operational objectives, maps AI-governed behavior, analyzes adversarial influence surfaces, defines behavioral failure criteria, executes scenario-based tests, and reports evidence linking adversarial action to objective violation. A running example involving an AI-enabled security operations center assistant illustrates how penetration may occur through behavioral influence rather than infrastructure compromise. Together, the definitions, workflow, and example provide a technical framework for evaluating adversarial success in deployed AI-enabled systems.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_960fa450a96feb34dd32c266