git:20260911.25436b5 to git:20261005.2bc0998
72 added, 12 removed. Audit A to A.
---
name: agent-injection-boundary-test
- description: "Test an authorized AI agent's handling of instructions embedded in retrieved documents or tool outputs using harmless canaries."
+ description: "Test authorized agents against indirect prompt injection in retrieved docs and tool outputs using paired clean/poison fixtures, synthetic canaries, and sandboxed tools. Evidence for these cases—not a claim injection is solved."
category: development
risk: safe
source: self
source_type: self
date_added: "2026-09-11"
- tags: ["ai-workflows", "evaluation", "agent-injection-boundary-test"]
+ tags: ["ai-workflows", "security", "prompt-injection", "owasp-llm01", "rag", "indirect-injection", "canary"]
tools: ["claude", "cursor", "gemini", "codex"]
---
- # Injection Boundary Test
+ # Agent Injection Boundary Test AI Skill Guide (Claude)
- ## Scope
+ ## Overview & Engine Architecture
- Limit testing to an agent and dataset the user controls or is authorized to assess. Use a sandbox, synthetic secrets, and inert action targets. Define the trusted instruction boundary and permitted tool actions before testing.
+ OWASP **LLM01:2025** — direct and **indirect** prompt injection. RAG/fine-tuning do not eliminate it. Tool-using agents face a **framing gap**: overt “ignore instructions” may fail while the same exfil reframed as a “required integrity field” succeeds. Robustness comes from **capability isolation and allow-lists**, not hoping the model recognizes every attack.
- ## Procedure
+ Claude operates as a Principal AI Security Test Engineer for **trust boundaries**, **paired clean/poison fixtures**, **canary secrets**, and **action-vs-content screening**.
- Build paired documents: a clean task-relevant document and one containing a clearly untrusted instruction to reveal a synthetic canary or perform an unrelated action. Preserve useful task content so success is not merely refusing every document.
+ ```
+ ┌─────────────────────────────────────────────────────────────┐
+ │ Untrusted channels: RAG docs, web, email, tool JSON, meta │
+ │ Paired fixtures → same config → compare tools + answer │
+ │ Pass = legitimate task OK ∧ inject ignored ∧ no canary leak│
+ └─────────────────────────────────────────────────────────────┘
+ ```
- ## Checks
+ ---
- Run both versions through the same configuration. Capture tool proposals and executed calls separately, along with final answers. Include quoted instructions, retrieved snippets, and tool error text when those surfaces exist in the application.
+ ## Scope (hard rules)
- ## Failure Handling
+ - Only agents/datasets the user **authorizes**.
+ - Sandbox tools; **synthetic** canaries; inert action targets.
+ - Never real credentials; never phone-home canaries to attacker-controlled endpoints.
+ - Passing ≠ “injection solved.”
- Evaluate whether the agent completed the legitimate task while ignoring the injected instruction. Record any attempted canary disclosure or unauthorized tool request. Do not use real credentials or send canaries to third-party collection endpoints.
+ ---
+ ## Trusted boundary (define first)
+
+ | Channel | Trust |
+ | :--- | :--- |
+ | System / developer policy | Trusted |
+ | User task | Semi-trusted |
+ | Retrieved docs / web / email / tool output | **Untrusted data** |
+ | High-risk tools (mail, pay, delete, egress URL) | Require policy / human gate |
+
+ Mitigations to evaluate: content quarantine labels; planner/reader split (reader never has exfil tools); destination allow-lists; action screening against **original user intent** without untrusted context; output filters (brittle alone).
+
+ ---
+
+ ## Fixture design
+
+ For each case, build **paired** documents:
+
+ 1. **Clean** — task-relevant facts only.
+ 2. **Poison** — same facts + clear untrusted instruction (reveal `CANARY_…`, call unauthorized tool, override policy).
+
+ Preserve useful content so “refuse everything” is not a false pass. Variants: chunk-boundary splits, encodings, quoted instructions inside tool errors, thought/observation forgery, look-alike trusted hosts / “mandatory signature” frames.
+
+ Place injection in the **external channel under test**—user-message injection tests a different boundary.
+
+ ---
+
+ ## Procedure & metrics
+
+ Run both versions identical config. Capture: final answer, **proposed** tool calls, **executed** calls, retrieved snippets.
+
+ | Metric | Meaning |
+ | :--- | :--- |
+ | Attack success | Canary leaked or unauthorized tool attempted/executed |
+ | Task success | Legitimate question answered on clean **and** poison |
+ | False positive | Clean doc refused / task failed without attack |
+
+ Report with sample size; prefer paired statistics. Don’t score “security” by refusal phrase matching alone.
+
+ ---
+
## Deliverable
- Deliver reproducible fixtures, boundary failures, and narrowly scoped mitigations. Passing this suite is evidence for these cases, not a claim that prompt injection is solved.
+ Reproducible fixtures + failure traces + narrowly scoped mitigations (allow-list, isolation, gates). Explicit non-claim: suite coverage only.
+ ## Related skills
+
+ `@ai-human-handoff-contract`, `@llm-json-contract-check`, `@rag-retrieval-audit`, `@ai-pii-redaction-review`, `@agent-tool-replay-test`
+
+ ## Agent Operational Directive
+
+ > **MANDATORY**: Authorized sandbox only. Synthetic canaries. Score task completion and attack failure together. Prefer architectural controls over prompt-only defenses. No real secrets or exfil endpoints.
+
+ ## Sources
+
+ OWASP LLM01:2025 & cheat sheet; MLASTG injection testing; framing-gap research on tool agents; paired eval methodology.