test-discipline · diff
git:20260910.9cf5842 to git:20260911.f640e80
25 added, 23 removed. Audit A to A.
---
- name: "test-discipline"
- description: "Update tests when changing APIs — no exceptions"
- domain: "quality"
- confidence: "high"
- source: "earned (test assertion sync violations)"
+ name: test-discipline
+ description: Design assertions that prove the actual contract and fail on realistic regressions
+ domain: quality
+ confidence: high
+ source: current regression tests and retained quality incident lessons
---
## Context
- When APIs or public interfaces change, tests must be updated in the same commit. When test assertions reference file counts or expected arrays, they must be kept in sync with disk reality. Stale tests block CI for other contributors.
+ Use this skill whenever a change affects an API, public interface, workflow contract, generated
+ artifact, or user-observable behavior. A passing status-only test is not evidence that the
+ intended artifact or contract is correct.
## Patterns
- - **API changes → test updates (same commit):** If you change a function signature, public interface, or exported API, update the corresponding tests before committing
- - **Test assertions → disk reality:** When test files contain expected counts (e.g., `EXPECTED_FEATURES`, `EXPECTED_SCENARIOS`), they must match the actual files on disk
- - **Add files → update assertions:** When adding docs pages, features, or any counted resource, update the test assertion array in the same commit
- - **CI failures → check assertions first:** Before debugging complex failures, verify test assertion arrays match filesystem state
+ - Update tests in the same change as an API, signature, export, or public-interface change.
+ - Make assertions name and inspect the offending artifact, output, or configuration field—not just
+ an exit code or broad success status.
+ - Mutation-test critical gates by changing the real source or emitted artifact and proving the
+ assertion turns red.
+ - Exercise every caller path for a changed parser, normalizer, or helper; isolated unit coverage
+ can miss the integration path that invokes it differently.
+ - Reproduce working-tree-only failures locally when the contract depends on index, filesystem,
+ line-ending, or generated-output state.
+ - Keep expected inventory arrays synchronized with the files they enumerate.
## Examples
- ✓ **Correct:**
- - Changed auth API signature → updated auth.test.ts in same commit
- - Added `distributed-mesh.md` to features/ → added `'distributed-mesh'` to EXPECTED_FEATURES array
- - Deleted two scenario files → removed entries from EXPECTED_SCENARIOS
-
- ✗ **Incorrect:**
- - Changed spawn parameters → committed without updating casting.test.ts (CI breaks for next person)
- - Added `built-in-roles.md` → left EXPECTED_FEATURES at old count (PR blocked)
- - Test says "expected 7 files" but disk has 25 (assertion staleness)
+ A workflow-compilation test mutates the source workflow, compiles it, and inspects the emitted
+ safe-output contract. A repair-script test constructs the actual index/blob/worktree state that
+ the repair routine must handle instead of a simplified fixture.
## Anti-Patterns
- - Committing API changes without test updates ("I'll fix tests later")
- - Treating test assertion arrays as static (they evolve with content)
- - Assuming CI passing means coverage is correct (stale assertions can pass while being wrong)
- - Leaving gaps for other agents to discover
+ - Calling a test sufficient because it observes only a successful process exit
+ - Mutating a hand-built fixture instead of the source or artifact protected by the gate
+ - Testing a helper while omitting production callers
+ - Treating a local checkout defect as something CI alone can validate