autoqa git:20260721.5cedab0A

Immutable. This exact content is served forever at /api/v1/blob/0261501c844c504b.

---
name: autoqa
description: Self-QA a repo's running application end-to-end — discover how it runs, build a feature×modality test plan, execute it against a live instance, and produce a witnessed pass/fail report. Use when the user says "autoqa", "QA this repo/branch", "verify every feature works", "release readiness check", "test this like a QA engineer would", or another skill needs an automated QA pass before sign-off.
---

# autoqa — QA any repo against its own running app

You are the QA engineer for this repo. Every verdict is **witnessed** — it points at an
artifact (an HTTP response, a page snapshot, a log line, a DB row) that *shows* the result,
not merely a file that exists. The full witness contract is in Hard rules below.

## Phase 0 — RESOLVE

Establish three facts before anything else:

1. **Target repo** — path or URL the user pointed at (ask only if truly absent).
2. **Target instance** — a running deployment to test against (URL/port), or the
   instruction to bring one up locally.
3. **Repo config** — look for `AUTOQA.md` at the repo root or under `docs/`. It is the
   repo's pre-answered discovery: how to run, how to auth, where the feature catalog
   lives, known env caveats. **If found, read it now and skip straight to Phase 2.**
   Missing config means full Phase 1 — and a repo you QA repeatedly earns one: write
   `AUTOQA.md` from what Phase 1 taught you and it becomes the config for every later run.

Done when: repo path, instance URL (or "must boot"), and config-or-none are stated.

## Phase 1 — DISCOVER

Read the repo the way a new engineer would, in this order, stopping when the three questions
below are answered. Full source-priority list and what each source answers:
[references/discovery.md](references/discovery.md).

- **How does it run?** Dockerfile / compose / Procfile / Makefile / CI workflows / README.
- **How do I authenticate?** env samples, auth middleware, dev-token conventions, CLAUDE.md / AGENTS.md.
- **What are the features?** a feature catalog or spec doc if the repo ships one (use it —
  it beats inference), else routes/pages/CLI entrypoints enumerated from code.

Done when: you can write down the run command, an auth recipe, and a feature inventory —
each traced to the file that taught you it.

## Phase 2 — PLAN

First write the **feature inventory** — a numbered list of every feature discovery found.
Then build the matrix from it: one row per feature, columns = modality, entry point, check,
pass criterion, witness to capture. The matrix must have ≥1 row per inventory item; the
report states the coverage arithmetic (`N discovered / M rows / K untested`) so a reader
sees nothing was silently dropped.

- **Entry point is a gating column, decided here — not at execute time.** For each feature,
  name the path a user takes to reach it: the button, link, route, or client API call. A
  feature whose entry point you cannot trace is not tested — its disposition is
  `SKIPPED (unreachable — candidate dead code)`, and it never executes. Reachability is a
  planning property; deciding it now is what stops you from running a router endpoint
  nothing navigates to and mistaking its breakage for a bug.
- **Disposition per row**: DEEP (run the end-to-end check), SMOKE (load the surface, assert
  its key content), UNTESTED (needs a fixture you don't have), SKIPPED (unreachable, or the
  branch predates the feature). A run that covers only headline features is a smoke pass —
  the report says so rather than implying full coverage.
- **Modalities**: API (endpoint calls), UI (browser tooling — chrome-devtools MCP,
  Playwright, or whatever the session has), CLI (the repo's own binaries). Cover every
  modality the app actually has; a web app QA'd only through its API is half-tested.
- **Drive each feature by its traced entry point** — through the UI, or the API call the
  client actually issues — not by a raw endpoint you found in the router.
- **Pass criteria are concrete**: status codes, visible text, row counts, terminal job
  states — never "looks right".
- Order rows: boot/health/auth first (everything else depends on them), then core money
  paths, then edge/regression rows.

Done when: the inventory list exists, the matrix has ≥1 row per inventory item, every row
carries all five columns, and every executable row names a traced entry point.

## Phase 3 — EXECUTE

Execute only rows whose entry point was traced in PLAN. Run the matrix top to bottom
against the live instance.

- Instance not up? Bring it up exactly as discovery taught — respect the repo's own
  runbook (secret-injection wrappers, port maps) over generic docker commands. If it
  cannot be brought up at all (missing secrets, port conflict, boot crash), the whole run
  is `BLOCKED` — report what failed to boot and stop; never force a ship/don't-ship verdict
  on an app you never ran.
- Write the report and evidence side by side: report at `<scratch>/autoqa-report.md`,
  evidence in `<scratch>/autoqa-evidence/`, witness paths relative to that shared parent so
  they resolve. Name witness files by row: curl output with status codes, page snapshots or
  screenshots, log excerpts.
- A failing row gets one diagnosis pass: is it the app, the env, or your check? Fix
  env/check mistakes and rerun; app failures stay failed and get a one-line cause. (Rows
  whose entry point couldn't be traced were already SKIPPED in PLAN — you never reach here
  for dead code.)
- Async work (jobs, builds) is polled to a terminal state, capped at a stated timeout; on
  expiry the row is UNTESTED with the elapsed time — never a pass, never an infinite poll.
- Leave the instance as healthy as you found it; if you restarted anything, re-verify
  health before reporting.

Done when: every matrix row is PASS, FAIL (with cause), UNTESTED (with reason), or SKIPPED —
each with a witness that shows the asserted result — or the run is BLOCKED with the boot
failure recorded.

## Phase 4 — REPORT

Write the report from the template in
[references/report-template.md](references/report-template.md): verdict table (feature,
modality, result, witness path), failure triage (release blocker vs env quirk vs test
bug), and the one-paragraph bottom line a release owner can act on.

Done when: the report file exists next to the evidence dir, every table row's witness path
resolves, and the bottom line states ship / don't-ship / ship-with-caveats / blocked.

## Hard rules

- **No witness, no verdict.** A witness must *show* the asserted result — the 200 in the
  captured status line, the expected text in the snapshot — not merely exist. A named file
  that doesn't show the result is not a witness. Reruns beat inference.
- **Reachability is decided in PLAN, not blamed in EXECUTE** — a feature whose user entry
  point you can't trace is SKIPPED before it runs; a break behind a path nothing navigates
  to is dead code to remove, not a bug to fix.
- **Coverage is counted, not claimed** — the report shows `discovered / rows / untested` so
  a dropped feature is visible arithmetic, not a silent gap.
- **The repo's runbook outranks your habits** — a project whose docs wrap startup in a
  secret-injection command never gets a bare `docker compose up`.
- **Report failures as found** — a QA pass that only reports greens is a failed QA pass;
  triage severity honestly instead of softening results.