--- name: mutation-testing description: Use when adding or reviewing a gate, guard rule, CI grep, or test assertion in this plugin's own scripts — and whenever a suite is green and you need to know whether that means anything. Runs scripts/mutate.sh, which breaks the code on purpose and reports which assertions failed to notice. Triggers from code-reviewer, verification-engineer, and any change under scripts/ or hooks/. --- # Mutation testing `defect-hunting` §3 says a new rule is not done until you have watched it fail, in three steps. That instruction was followed by hand, when someone remembered. On 2026-07-29 the suite read **385 passed, 0 failed** while containing a `grep -E` with a PCRE lookahead (a syntax error, stderr to `/dev/null`, `|| ok` every time), two doctor assertions that a demoted gate could not turn red, and a hook that stood down in exactly the incident it was written for. **A green suite is evidence only to the extent its assertions can go red.** `scripts/mutate.sh` is that, executable. ## Running it ```sh sh scripts/mutate.sh # all 16 catalogued mutations (~1 min each — the whole suite runs) sh scripts/mutate.sh --list # the catalogue, plus what it cannot test and why sh scripts/mutate.sh --only M04 # one mutation, while you iterate on an assertion sh scripts/mutate.sh --sample 4 # what CI runs on every PR ``` Exit `0` all caught · `1` something SURVIVED · `2` could not run (baseline not green, anchor drifted). Three verdicts matter: - **CAUGHT (n assertions)** — the gate bites. - **SURVIVED** — the gate can be deleted and the suite stays green. **That is a hole**, and it is a finding at the same severity as the bug the gate was supposed to catch. - **CAUGHT, but NOT by the assertion written for it** — some unrelated assertion noticed. Not a hole today; the named guard is decorative, and the next refactor that touches the unrelated one takes the coverage away silently. Fix the named assertion. ## The rule **A new gate ships with a mutation proving its assertion bites.** Adding a check to `board-doctor`, `ship-gate`, `verify-done`, `spawn-gate`, a hook, or `scripts/test.sh` is not done until `scripts/mutate.sh --only ` prints CAUGHT and names *your* assertion. Adding one is four fields in the catalogue at the top of `scripts/mutate.sh`: ``` M17@@scripts/your-gate.sh@@@@@@ ``` - The find text must occur **exactly once** in the file. Zero or many exits 2 — a mutation anchored at an arbitrary one of several sites proves nothing, and a drifted anchor must be loud rather than a phantom survivor. - Break the *behaviour*, not the message. Inverting an exit code, demoting a finding to a warning, flipping `-gt` to `-lt`, `if false`, neutering a regex, or returning the fallback instead of refusing are the seven shapes in the catalogue — all seven are things that really shipped here. - The last field is a substring of the real label in `scripts/test.sh`, so the tool can tell "the suite noticed" from "the assertion I wrote noticed". ## When you cannot test it Some assertions need Xcode, a simulator, or a network. **Do not fake coverage.** Add them to `excluded()` in `scripts/mutate.sh` with the reason; they are printed with every score and counted in neither numerator nor denominator. A mutation score that quietly omits the untestable parts is the same lie as a green suite full of decorative assertions. A gate with no fixture that reaches it belongs there too, with that reason — reporting it as a hole would be a false alarm, and a tool that cries wolf gets switched off. ## Safety Mutations are applied to a copy in `$TMPDIR`; the real tree is never written to. `.git` is not copied (in a worktree it points at the real repository). Restore is a trap, and on exit the tool compares `git status --porcelain` against the snapshot it took at start and fails loudly if it moved. ## Manual fallback No Node or a hostile environment: pick the assertion you doubt, edit the line it guards to invert it, run `sh scripts/test.sh`, and confirm **that assertion by name** is in the FAIL list. Then `git checkout -- ` — never a repo-wide revert. That is `defect-hunting` §3's three steps done by hand: confirm the edit landed, confirm the fixture reaches it, watch it go red and green again.