test-gap-review ยท diff
git:20260717.dc9a72d to git:20260718.188cd11
10 added, 22 removed. Audit A to A.
---
name: test-gap-review
- description: Review tests and verification for missing behavior coverage or false confidence. Use during validation when changed behavior may lack meaningful normal, boundary, failure, cleanup, or side-effect coverage, or when required checks were skipped or claimed without evidence.
+ description: Review whether existing tests and verification credibly prove a scoped behavior contract. Use during validation when evidence may omit important behavior or provide false confidence.
---
# Test Gap Review
- Determine whether verification proves the promised behavior rather than implementation trivia.
+ Audit verification sufficiency without silently becoming a test-writing or production implementation workflow.
## Workflow
- 1. Identify the observable behavior and invariants that must be proven.
- 2. Inspect existing tests and checks for that behavior.
- 3. Look for missing normal paths, important boundaries, failure paths, preserved state, cleanup, and external side effects.
- 4. Reject weak assertions, fixtures that cannot fail meaningfully, implementation-detail tests, unsupported manual claims, and skipped required commands.
- 5. Add or strengthen tests only where they materially improve confidence.
- 6. Run the relevant checks.
- 7. Report any remaining verification gap and residual risk.
-
- ## Verification Preference
-
- Prefer, in order:
-
- 1. the repository's authoritative test command;
- 2. validation required by the current contract;
- 3. focused unit or integration tests;
- 4. focused manual verification with concrete evidence;
- 5. an explicit blocker report.
+ 1. Establish the authoritative behavioral contract, comparison scope, and claimed verification.
+ 2. Map normal behavior, important boundaries, meaningful failures, preserved state, cleanup, and external effects to existing evidence.
+ 3. Identify weak assertions, fixtures that cannot fail meaningfully, implementation-detail tests, unsupported manual claims, and skipped required commands.
+ 4. Report each material gap with the affected claim, missing evidence, consequence, smallest proving check, and residual risk.
+ 5. If remediation is authorized, use `test-design` for test changes and `behavior-implement` for production behavior; otherwise remain read-only.
- For measurable claims, report:
+ For each material claim report:
```text
VERIFIED | NOT VERIFIED | INCONCLUSIVE
Claim: ...
Evidence: ...
Reasoning: ...
```
- Use `INCONCLUSIVE` when the baseline is invalid, the signal is noisy, the environment differs materially, or the check failed for an unrelated reason.
+ Do not chase coverage percentages, add fake-confidence tests, broaden infrastructure, or change production behavior directly.
- Do not chase coverage percentages, add fake-confidence tests, snapshot unstable output without an established reason, or broaden test infrastructure unnecessarily.
+ Finish when material claims have supported statuses and remaining verification risk is explicit.