git:20260506.b1f0fef to git:20260909.6778255

30 added, 145 removed. Audit A to A.

---

name: criterion-verification-map

- description: "Transform acceptance criteria into plan-time runnable verification commands (behavioral, API, UI, error, config, data, contract types) with expected evidence shapes, then execute at verify time and assemble evidence bundles with honest completeness subsections (untested paths, known limitations, adversarial cases covered). Use when planning implementation against issue acceptance criteria or verifying completeness. This skill MUST be consulted because deferring verification to later causes incomplete PRs, and suppressing evidence gaps prevents the verdict judge from reasoning about gaps."

+ description: "Transform acceptance criteria into plan-time runnable verification commands (behavioral, API, UI, error, performance, config, data, contract types) with expected evidence shapes and risk areas, then execute at verify time and assemble the evidence bundle with its mandatory completeness subsections, including test inputs/expected values taken from test source and risk-map coverage. Use when planning implementation against issue acceptance criteria or verifying completeness. This skill MUST be consulted because deferring verification to later causes incomplete PRs, and suppressing evidence gaps prevents the verdict judge from reasoning about gaps."

allowed-tools: Bash, Read, Grep, Glob, TaskCreate, TaskList, TaskUpdate, TaskGet

context: fork

agent: general-purpose

---

# Criterion Verification Map

- Domain skill that treats acceptance criteria as **eval sources**. Each criterion produces a concrete, runnable check at plan time — not at verify time. Planning a criterion without a runnable command is a blocking error.

- ## Iron Law

- **EVERY ACCEPTANCE CRITERION IS AN EVAL SOURCE.** At plan time, each criterion must produce a runnable verification command. No criterion is deferred to verify time with "we'll figure out how to test this later." No criterion passes by assumption. No criterion is "too obvious to verify."

- ## Criteria as Eval Sources (Plan Time)

- An acceptance criterion is useful only if it can be evaluated mechanically. At plan time, the criterion must be transformed into:

- 1. **A verification type** (classification — see table below)

- 2. **A runnable command** (the exact bash/test/curl/script invocation that will be executed at verify time)

- 3. **Expected evidence shape** (what the command's output must contain to count as PASS)

- 4. **What the criterion does NOT promise** (non-goals scoped to this criterion — prevents scope creep and false-positive verdicts)

- If any of the four cannot be filled in at plan time, the criterion is not ready. Escalate through the Spec Validation Gate in the `start.md` EXPLORE phase, not here.

+ ## Contract

- ### Criterion Classification Table

+ Iron law: **every acceptance criterion is an eval source: at plan time it must produce a runnable verification command, or planning is blocked.** Invoked by `/flow:start` at the Spec Validation Gate and PLAN (Phases 1-2) to classify each criterion and emit the task's verification fields, and at VERIFY (Phase 4) to run them and assemble the evidence bundle shaped by [`references/evidence-bundle-format.md`](../../references/evidence-bundle-format.md). Returns task fields at plan time, the bundle at verify time. Permitted skips: none; an unmappable criterion is escalated through the Spec Validation Gate.

- |---------------|-------------|-------------------|-----------------|

+ ## Classification

- ## Plan-Time Task Creation

+ | Type | Signal words | Method |

+ |---|---|---|

+ | behavioral | "when X then Y", "should return" | unit/integration test |

+ | api | "status code", "endpoint", "header" | curl/fetch |

+ | ui | "displays", "renders", "layout" | screenshot + analysis |

+ | error | "error message", "invalid", "fails gracefully" | invalid-input test |

+ | performance | "within N ms", "timeout" | benchmark/timing |

+ | config | "config", "setting" | build/load test |

+ | data | "transforms", "output matches" | run with test data |

+ | contract | "schema", "type", "signature" | validator / type-check |

- Per `commands/start.md` Phase 2 (PLAN), tasks are **atomic**: implementation, test, and evidence collection are bundled into one task. This skill's role at plan time is to produce the verification command and expected evidence shape that gets embedded into that atomic task's description.

+ ## Plan-time task fields

- For each acceptance criterion, the atomic task description must include:

+ Tasks are atomic (`commands/start.md` Phase 2). Emit per criterion:

```

Criterion: {full criterion text}

- Verification type: {type from classification table}

+ Verification type: {type from the table}

Verification command: {exact runnable command}

Expected evidence: {what successful output looks like}

Does NOT promise: {non-goals scoped to this criterion}

+ Risk areas: {risk-map rows whose area this criterion's logic touches, verbatim | none | (disabled by specFirst.riskMap)}

```

- The atomic task flows through implementation → test → evidence collection within Phase 3 (CODE). Evidence is captured at task-completion time.

+ `Does NOT promise` comes from the journal's `### Non-goals`, `Risk areas` from `### Risk map` ([`references/specification-journal-format.md`](../../references/specification-journal-format.md)). `implementation-planner` adds one discriminating test per risk row (input, expected, source). Any unfillable field: escalate through the Spec Validation Gate.

- ## Evidence Collection Protocol

+ ## Evidence collection protocol (verify time)

- During VERIFY phase, for each atomic task's verification command:

+ For each atomic task:

1. `TaskUpdate(taskId, status: "in_progress")`

- 2. Execute the verification command captured at plan time

- 3. Capture output as evidence

+ 2. Run the verification command

+ 3. Capture output verbatim

4. `TaskUpdate(taskId, status: "completed", result: "EVIDENCE_COLLECTED")`

- ## Evidence Bundle Format

- After all verification commands have run, assemble the evidence bundle — a structured text document that the verdict-judge agent receives. Every criterion block MUST include all fields below. The "Does NOT promise" field is captured at plan time; the three completeness subsections (What was NOT tested, Known limitations, Negative/adversarial cases covered) are populated at verify time and may not be omitted — they force the implementer to state the shape of the evidence honestly so the judge can reason about gaps.

- ```markdown

- ## Evidence Bundle for Issue #{N}

- Generated: {timestamp}

- Branch: {branch name}

- Commits: {count} since branch creation

- ### Criterion 1: {full criterion text}

- - **Type**: {behavioral|api|ui|error|performance|config|data|contract}

- - **Verification command**: `{command that was run}`

- - **Evidence**:

- ```

- {raw output from verification command}

- ```

- - **What the criterion does NOT promise**:

- - {non-goal 1 — e.g. "does not guarantee idempotency across retries"}

- - {non-goal 2 — e.g. "does not cover the admin flow"}

- - {non-goal 3 — e.g. "does not handle concurrent writes"}

- - **Screenshot**: {path, if UI type — otherwise omit}

- - **What was NOT tested**: {Explicit list of related behaviors, inputs, code paths, environments, or configurations that this evidence does not cover. Never "N/A" — if you cannot think of anything, you have not thought hard enough. State at minimum: untested environments, untested edge inputs, untested concurrency/scale conditions, untested integrations.}

- - **Known limitations of this evidence**: {How the evidence could be misleading even though it looks positive. Examples: "test uses a mocked external API," "smoke test only hits the happy path," "screenshot was taken at desktop viewport only," "timing numbers taken on an idle machine, not under load." If the verification command is self-reported (agent-run test output), state that explicitly.}

- - **Negative/adversarial cases covered**: {List the specific failure modes, invalid inputs, and abuse cases this evidence demonstrates the system rejects or handles safely. Examples: "rejects empty email with 400," "returns 401 on expired token," "displays error state on network failure." If none were tested, state "none" — do not leave blank — and expect the verdict-judge to treat this as a gap.}

- ### Criterion 2: {full criterion text}

- - **Type**: {type}

- - **Verification command**: `{command}`

- - **Evidence**:

- ```

- {output}

- ```

- - **What the criterion does NOT promise**:

- - {non-goal items}

- - **What was NOT tested**: {as above}

- - **Known limitations of this evidence**: {as above}

- - **Negative/adversarial cases covered**: {as above}

- {repeat for all criteria — every criterion MUST have all four subsections}

- ```

- ### Why "Does NOT Promise" Is a First-Class Field

- A verdict judge receiving only criterion text and evidence tends to over-credit passing commands — e.g., a passing unit test is treated as proof the whole behavior is correct, when the test only covered one path. The "does NOT promise" field explicitly fences each criterion's scope so the judge does not inflate a narrow PASS into a broad guarantee. It also gives reviewers and future readers a shared understanding of what was intentionally out of scope.

- Populate this field at **plan time** from the non-goals captured in the EXPLORE phase Specification capture sub-step. Each criterion inherits the global non-goals plus any criterion-specific non-goals discovered during planning.

- <!--

- SECTION BOUNDARY — VERIFY-TIME EXTENSIONS

- The three verify-time completeness subsections below were added by

- Issue #42. They are populated at verify time, not plan time. The

- plan-time contract above this line remains stable.

- -->

- ### Completeness Subsections Are Mandatory

- The verdict-judge treats any criterion missing "Does NOT promise" or any of the three completeness subsections ("What was NOT tested", "Known limitations of this evidence", "Negative/adversarial cases covered") as having incomplete evidence and will FAIL that criterion. Do not omit them. If a subsection is genuinely empty (e.g., no adversarial cases tested), write "none" explicitly rather than removing the heading.

- ## What the Evidence Bundle Does NOT Include

- The bundle is passed to the verdict-judge agent, which must judge independently. Therefore:

- - **NO diff** — the judge doesn't see the code changes

- - **NO decision journal** — the judge doesn't see the rationale

- - **NO planning notes** — the judge doesn't see why approaches were chosen

- - **NO self-review findings** — the judge evaluates from spec + evidence only

- The judge receives ONLY:

- 1. The acceptance criteria (from the issue)

- 2. The evidence bundle (from this skill)

- 3. The holdout-validation output (from the holdout-validation skill — added in v2.0 to detect self-review claims that don't match file state)

- ## Verification Method Examples

- **Behavioral (test output)**:

- ```bash

- npm run test -- --grep "user authentication" 2>&1 | tail -20

- ```

- **API endpoint (curl)**:

- ```bash

- curl -s -w "\nHTTP_STATUS:%{http_code}" http://localhost:3000/api/login \

- -H "Content-Type: application/json" \

- -d '{"email":"test@example.com","password":"wrong"}' 2>&1

- ```

- **UI rendering (screenshot path)**:

- ```

- Screenshot saved to: .screenshots/login-page-desktop.png

- Visual analysis: Login form visible with email and password fields, submit button enabled.

- ```

+ Then assemble one `## Criterion {N}` section each. `### Type` and `### Does NOT promise` are the plan-time fields; `### What was NOT tested`, `### Known limitations of this evidence`, `### Negative/adversarial cases covered` are authored honestly now (`none — {reason}` is positive; blank is an auto-FAIL). `### Visual analysis` is copied per viewport from the `visual-verification` result tasks (`Viewport:`/`Screenshot:`/`Result:`/`Observed:` blocks), never re-derived; non-`ui` types write `none — criterion type {type} has no visual surface`.

- **Error handling**:

- ```bash

- npm run test -- --grep "invalid credentials" 2>&1 | tail -10

- ```

+ Two subsections come from test source, never memory:

- **Contract (schema/type)**:

- ```bash

- npx tsc --noEmit 2>&1 | tail -20

- # or

- npx ajv validate -s schemas/payload.json -d fixtures/sample.json

- ```

+ - **`### Test inputs and expected values`**: open every test file the command or its output cites. For each test that ran, record one `| Test | Input | Expected | Source of expected |` row from its source: literal input, literal asserted value, its origin (spec/criterion text, reference implementation, hand computation, existing fixture, external standard). Implementation-captured values are written `implementation output`; the judge FAILs them. `none — {reason}` only for `ui`/`config` types.

+ - **`### Risk map coverage`**: for each `Risk areas` row, find the test whose input is its discriminating check and write `<area> → <test file:line>`. A row with no such test is a gap: add it or escalate. `none — {reason}` only when no row maps here; `none — risk map disabled (specFirst.riskMap=false)` only when the journal subsection is the disabled marker.

- ## Integration with Start Command

+ Validate the bundle against the reference (every mandatory subsection present, `none` only where permitted) before `Agent(verdict-judge)`.

- The criterion-verification-map skill is invoked in two phases of `/flow:start`:

+ ## Judge isolation

- 1. **EXPLORE / PLAN phase (plan time)** — Classify each criterion, produce a runnable verification command, and capture "does NOT promise" non-goals. These become the `Verification command` and `Does NOT promise` fields of the atomic task created in PLAN.

- 2. **VERIFY phase (verify time)** — Execute the verification commands captured at plan time and assemble the evidence bundle.

+ The verdict-judge has no file tools and receives ONLY acceptance criteria, evidence bundle, and holdout-validation output — never diff, journal, planning notes, self-review findings, test source, or screenshots; tests reach it as `### Test inputs and expected values` rows, screens as `### Visual analysis` text.