adversarial-testing git:20260527.c466ff8A

Immutable. This exact content is served forever at /api/v1/blob/f313a20ad755b999.

---
name: adversarial-testing
description: Use when authoring evals/predicates (Scout, TDD-Engineer), framing an audit (Auditor), or justifying which phases run (PhaseAdvisor/router). Codifies Google's 4-phase adversarial-testing methodology mapped onto evolve-loop's eval, audit, routing, and red-team surfaces.
---

# Adversarial Testing for evolve-loop

> Canonical methodology reference. Derived from Google's [Adversarial Testing for Generative AI](https://developers.google.com/machine-learning/guides/adv-testing), mapped onto this repo's existing inward-facing adversarial machinery (mutation testing, adversarial auditor, EGPS exit-code grounding). Referenced by: `agents/evolve-scout.md` (§eval integrity), `agents/evolve-tdd-engineer.md` (test diversity), `agents/evolve-auditor.md` (input categories), `go/internal/core/router_proposer.go` (`buildRoutingPrompt`), `acs/red-team/`. This file is the single source of truth — those consumers reference it, they do not re-derive it.

## Table of Contents

1. [Methodology overview (4 phases)](#1-methodology-overview)
2. [Phase 1 — Identify adversarial inputs](#2-phase-1--identify-adversarial-inputs)
3. [Phase 2 — Seed → synthesize → diversify](#3-phase-2--seed--synthesize--diversify)
4. [Phase 3 — Generate & annotate outputs](#4-phase-3--generate--annotate-outputs)
5. [Phase 4 — Report & mitigate](#5-phase-4--report--mitigate)
6. [Diversity checklist (eval authoring)](#6-diversity-checklist-eval-authoring)
7. [Phase-advisor rubric](#7-phase-advisor-rubric)
8. [Auditor framing extensions](#8-auditor-framing-extensions)
9. [Red-team predicate catalogue](#9-red-team-predicate-catalogue)
10. [Anti-gaming reference](#10-anti-gaming-reference)

## 1. Methodology overview

Adversarial testing proactively tries to *break* a system with inputs most likely to elicit bad output. Google's loop has four phases; this is how each maps to evolve-loop:

| Google phase | Generic meaning | evolve-loop surface |
|---|---|---|
| 1. Identify inputs | Find explicit + implicit adversarial inputs | Eval/predicate authoring; auditor's hunt list |
| 2. Find/create test sets | Seed → synthesize → require diversity | `.evolve/evals/`, `acs/cycle-N/`, `acs/regression-suite/` |
| 3. Generate & annotate | Auto-classifier + human/judge on uncertain | EGPS exit codes (auto) + adversarial Auditor (judge) |
| 4. Report & mitigate | Summarize, feed findings back as new tests | Incident reports → `acs/red-team/` standing predicates |

**Core principle:** avoid obvious attack patterns that surface filters already catch; hunt the rare-but-harmful, innocuous-looking edge cases. In this repo that means: a tautology (`exit 0`) is the *obvious* attack the linter already blocks — the *real* danger is a predicate that looks rigorous but would also pass on an empty repo.

## 2. Phase 1 — Identify adversarial inputs

Two input classes, per Google. Spend effort on the second.

| Class | Definition | evolve-loop examples |
|---|---|---|
| **Explicit** | Obviously hostile; surface filters usually catch | `exit 0`, `echo PASS`, grep-on-inlined-literal, confidence < 0.85 reported as PASS |
| **Implicit** (innocuous-but-harmful) | Looks valid; slips past filters | predicate that passes on a GREEN build *and* on an EMPTY repo; build that touches the right files but the change is a no-op (rename/whitespace/comment); a "new" eval that re-skins last cycle's eval (diversity collapse) |

Authoring rule: for every criterion, write down both *what success looks like* and *the cheapest way a lazy/gaming implementation could fake it* — then test that the fake fails.

## 3. Phase 2 — Seed → synthesize → diversify

Build test sets the way Google prescribes: a few hand-written seeds, expanded, then checked for diversity.

| Step | Action |
|---|---|
| Seed | Hand-write the happy-path predicate that proves the behavior exists |
| Synthesize | Add the negative case (input that must be rejected) and the edge/OOD case (empty, boundary, malformed) |
| Diversify | Across a feature's evals, vary the command verbs and the level of abstraction — do not test the same thing five ways |

**Diversity axes** (all four expected on any non-trivial feature):

| Axis | Question | Cheap signal |
|---|---|---|
| Lexical | Different command verbs / phrasings? | not all `grep`; not all the same binary |
| Semantic | Different behaviors covered? | distinct ACs, not one AC restated |
| Negative | Is there a case that must FAIL? | `! cmd`, `exit 1` expected, `stdout_absent` |
| Edge / OOD | Boundary / error inputs? | empty `""`, `0`/`-1`/max, `invalid`/`missing`/`corrupt` |

## 4. Phase 3 — Generate & annotate outputs

Dual annotation, mirroring Google's auto-classifier + human-for-uncertain split:

| Layer | Role | Mechanism |
|---|---|---|
| Auto (deterministic) | The verdict floor | EGPS: every AC is an executable predicate; `acs-verdict.json` `red_count==0` *is* the ship decision (`docs/architecture/egps-v10.md`) |
| Judge (LLM) | Catches what code cannot | Adversarial Auditor (Opus, cross-family from Builder's Sonnet) — see §8 |

The two are complementary: exit codes are objective but blind to "passes-on-empty-repo"; the judge is subjective but catches abstraction-level fraud. Neither alone is sufficient.

## 5. Phase 4 — Report & mitigate

Google's loop closes by feeding findings back as new test cases. In evolve-loop:

1. A gaming attempt that lands becomes a documented incident (`docs/incidents/cycle-NNN.md`).
2. Each incident's detection signal is converted to a **standing red-team predicate** in `acs/red-team/` (§9) that fires every cycle.
3. Promotion: this-cycle predicates (`acs/cycle-N/`) graduate to `acs/regression-suite/` post-ship; the regression suite only grows.

The mitigation is structural, not narrative — a documented incident with no standing predicate is an unfinished mitigation.

## 6. Diversity checklist (eval authoring)

Referenced by `agents/evolve-scout.md` and `agents/evolve-tdd-engineer.md`. Enforced (advisory) by `evolve eval diversity-check`.

- [ ] Each eval has ≥1 command that **inspects the workspace** (not echo/grep-on-literal).
- [ ] Each non-trivial feature has ≥1 **negative case** (an input that must be rejected / a command expected to exit non-zero).
- [ ] Each non-trivial feature has ≥1 **edge / OOD case** (empty, boundary, malformed input).
- [ ] Evals for the same module do **not share all command verbs** (lexical diversity — guards against diversity collapse).
- [ ] For each criterion, the **cheapest gaming fake** was identified and a test makes it fail.

Suite-level gate (`CheckDiversity`), keyed on the high-precision **negative-case** signal: a cohesive suite (3–12 evals) with **zero** negative cases → `HALT`; a smaller or archive-scale zero-negative suite → advisory `WARN`; any suite with ≥1 negative case → `PASS`. Edge-case counts are reported but do not gate (keyword-based, noisier). Heuristics are a *complement* to mutation testing (`mutate-eval.sh` kill-rate ≥0.8), not a replacement.

## 7. Phase-advisor rubric

The routing advisor proposes which optional phases run; the kernel (`router.Route()`) clamps. This rubric is the advisor's decision policy — `buildRoutingPrompt()` renders it inline so the LLM reasons from the same table.

| Signal (from digest) | Phase it justifies | Threshold |
|---|---|---|
| `scout.carryover_count` high | skip `scout` (work already queued) | carryover ≥ 3 |
| `scout.item_count == 0` | end cycle early (no-ship; legitimate) | item_count == 0 |
| `build.diff_files_touched` / `diff_loc` large | insert `plan-review` | files ≥ 10 OR loc ≥ 500 |
| `build.acs_red` ≥ 1 | insert `tester` after build | acs_red ≥ 1 |
| `build.severity_max` HIGH+ | insert `tester` after build | severity ≥ HIGH |
| `audit.verdict == FAIL` | insert `retrospective` | always (kernel also enforces) |
| `audit.confidence` low | insert `retrospective` regardless of verdict | confidence < 0.85 |
| `cycle_size == trivial` | skip `tdd` (conditional-mandatory exemption) | trivial only |

**Forbidden attack (explicit, must be named in the prompt):** never propose reaching `ship` without `audit`. The kernel rejects it, but the advisor must not attempt it — naming the attack vector is the Phase-1 "explicit adversarial input" defense, not a reliance on the clamp silently correcting.

Every proposal carries a one-sentence `justification`, captured on the `RouterDecision` (even when the proposal is clamped) and persisted in the recorded `routing-decision-N.json` artifact, whose SHA256 is bound in the `routing_decision` ledger entry — so the shadow soak can diff advisor-rationale against the static path, tamper-evidently (ADR-0024 problem #2).

## 8. Auditor framing extensions

Referenced by `agents/evolve-auditor.md`; rendered by `adversarialAuditFraming()` in `go/internal/subagent/run.go`. Extends the base "guilty until proven innocent" framing with Google's explicit/implicit taxonomy.

**Hunt list (implicit class — where real defects hide):**

- A predicate that passes on the GREEN build but would *also* pass on an EMPTY repo (does not actually require the feature).
- A build that touches the right files but the change is a no-op (rename, comment, whitespace).
- A "new" eval that shares ALL command verbs with the prior cycle's eval for the same module (diversity collapse).
- A check that verifies the wrong level of abstraction (tests the test, not the behavior).
- A new file verified to *exist* but not verified to be *non-empty* or *correct*.

**Per-criterion evidence requirement** (replaces "I see no problems"): for EACH acceptance criterion, cite exactly one of — (a) test output line (name + exit code + excerpt), (b) diff hunk (file:line + behavior), (c) a command the auditor ran + its output. Citing only (b) is allowed only for behavior-preserving refactors. A criterion with no citation → FAIL for that criterion, regardless of overall impression.

## 9. Red-team predicate catalogue

Standing predicates in `acs/red-team/` that fire every cycle, each encoding a past gaming pattern as a live test. Format follows the EGPS contract (`docs/architecture/egps-v10.md`); fixture pattern follows `acs/regression-suite/rhds-end-to-end/001-all-5-layers-fire.sh`.

| Predicate | Asserts | Catches |
|---|---|---|
| `rt-001-ledger-role-completeness.sh` | last cycle has scout+builder+auditor ledger entries | cycle 102-111: auditor never invoked |
| `rt-002-no-batch-state-write.sh` | `lastCycleNumber` advanced by exactly 1 (read from ledger, not state.json) | cycle 132-141: batch 132→141 jump |
| `rt-003-workspace-artifacts-present.sh` | recent cycle has `acs/cycle-N/` with ≥1 `.sh` | empty-cycle fabrication |
| `rt-004-challenge-token-in-reports.sh` | scout/build reports contain the ledger's challenge token | report forgery |
| `rt-005-phase-plan-clamp-logged.sh` | applied clamps are recorded in the ledger (no silent suppression) | the "clamp is the entire safety story" gap |
| `rt-006-no-tautological-eval.sh` | no `.evolve/evals/` file returns HALT from quality-check | AC-by-grep tautology |
| `rt-007-eval-diversity-floor.sh` | `evolve eval diversity-check` ≠ HALT | diversity collapse |

## 10. Anti-gaming reference

evolve-loop's three-tier trust model (`docs/concepts/trust-architecture.md`) is the enforcement backbone this methodology feeds:

| Tier | Mechanism | This skill contributes |
|---|---|---|
| 1 — Structural | phase-gate, role-gate, ship-gate, ledger SHA-chain | §9 red-team predicates |
| 2 — OS isolation | sandbox-exec / bwrap, per-cycle worktree | (unchanged) |
| 3 — Workflow defaults | mutation testing, adversarial audit, triage | §6 diversity, §7 rubric, §8 framing |

Source incidents (the adversarial corpus): `docs/incidents/cycle-102-111.md`, `docs/incidents/cycle-132-141.md`.