v1.0.0 to v1.0.0

1 added, 0 removed. Audit B to B.

---

name: pre-push-gate-debugging

description: "Use when a pre-push quality gate blocks git push or a lane fails: read the gate's own logs first (reproduce one lane), run single lanes for fast iteration, test the working tree you intend to push, handle E2E/infra cross-run state contamination, worktree node_modules gotchas, and build a manual repro harness when lane output hides the real error."

version: 1.0.0

author: ai-badger

license: MIT

platforms: [linux, macos, windows]

+ scope: optIn

metadata:

hermes:

tags: [gates, debugging, ci, pre-push]

related_skills: [debug-issue, commit-reminder]

---

# Pre-push verification gate debugging

Repos with heavy local gates block `git push` until every lane passes (lefthook pre-push hook

calling a verify script that runs docs, unit, E2E, infra and frontend lanes). This skill is for

operating and debugging that gate without burning whole-gate cycles. The methods here

are generic; keep a repo-specific playbook beside the gate script (lane list + commands,

log layout, repro steps) as this skill's reference when one exists.

## Detect and understand the gate

- `lefthook.yml` → `pre-push` → `script` + `runner` (+ `root`). Read the script's header

comments FIRST: usage, lane names, exit codes, env vars (`VERIFY_MODE`, `VERIFY_SKIP`),

self-test, and the "reproduce / bypass / logs" failure contract.

- A blocked push ends with `🥊 quality gate (Ns)` and `exit status 1`. The hook buffers lane

output — never diagnose from the push output alone.

## Workflow

1. **Read the gate's own logs first.** The failure block prints:

- `reproduce: bun <script> <lane>` — run exactly this to re-run one lane (minutes, not the whole gate).

- `logs: <dir>/<lane>.log` — the lane's full output; `<dir>/progress.log` is the run timeline

(mode, which lanes ran, PASS/FAIL per lane).

- Log dir is per-worktree (hash of the worktree path) — find it by mtime (`ls -t`), not by

recomputing the hash.

2. **Run one lane** for fast iteration: `bun <script> <lane>` (e.g. `api-e2e`). Only re-run the

full gate (`<script> pre-push`) when the lane is green.

3. **The gate tests the WORKING TREE, not the pushed ref.** If you push from a different

checkout than the one you want verified, the lanes verify the wrong bytes. Gate-test the

tree you intend to push.

4. **E2E/infra lanes**:

- They start their own AppHost (Docker: emulator containers) OR **reuse a live dev stack**

that is already ready (probed first). Reuse is convenient but carries state forward.

- **Cross-run state contamination**: emulator data persists across suite runs on a reused

stack. Durable orchestration instance IDs are PERMANENTLY TAKEN after a terminal state,

so a later run scheduling the same deterministic ID 409s or silently dedupes. Symptom: a

test passes alone, then a DIFFERENT test fails on a later full run with no code change.

Fix: fresh ephemeral volumes (kill the reused stack, let the lane start its own).

5. **Worktree gotchas**: a bare `git worktree add` has NO node_modules → tooling lanes fail

(`tsc` TS2688 "Cannot find type definition file for 'bun'", bun test resolution errors).

Fix: symlink the main checkout's node_modules into the worktree. Never commit node_modules.

6. **Manual repro harness** (when the lane's test output hides the real error — e.g. an

orchestration ends `failed` with an EMPTY error field):

- Start the infra exactly as the lane does (see the reference / lane source for the AppHost

command and env).

- Build the host under test and run it with the E2E settings (the lane copies a

`local.settings.e2e.json`-style file into the build output before the tests run).

- Drive the failing flow with curl, sending the app's auth header (see reference).

- Read the HOST's own logs: orchestration/activity failures appear there with the full

exception ("Task 'X' failed with an unhandled exception: ..."), never in the HTTP status

payload. `failed` + empty `error` in the API body = the detail is in the host logs.

7. **Auto-repair**: some gates try to regenerate drifted artefacts (baseline.json, index.json)

before failing. If repair "declined", the cause is NOT drift — read the lane log.

## Gotchas

- **A push can LOOK hung while the pre-push hook is simply running.** `git push 2>&1 | tail` buffers everything, so the lefthook banner (`🥊 lefthook ... hook: pre-push`) never shows and the full gate (which can take 5+ minutes) looks like a network stall — zero output, no timeout. Diagnose by re-running the push UNPIPED or in background and reading the first lines: a banner means the gate is working, not hanging. If the user has said "skip gates" / "just push", `git push --no-verify` is the sanctioned bypass and returns instantly — do not keep waiting on a gate the user already waived.

- `full` vs `limited` mode changes which lanes run; limited skips infra/e2e and prints that

coverage is still owed. Unset `VERIFY_MODE` for the real gate.

- `VERIFY_SKIP=<lane>` and `--no-verify` bypass the gate entirely — prefer fixing the lane.

- A 401 with "Missing or invalid x-functions-key" can really mean a missing auth header on your

curl, not a missing key — check the app's principal/auth scheme before assuming.

- Don't re-run the full gate to test one fix — single-lane runs are minutes faster.

- Force-pushing a rebased PR branch re-triggers remote CI; the local gate still runs against

your working tree, so keep the local tree on the branch you are pushing.

- **A green full-tree lint does NOT prove the changed-file lint lane passes.** If the gate's

lint lane lints only changed files (git-diff scope) while your manual run covered every

tracked file, the lane can flag findings the full run reports clean — e.g. a W0108

`unnecessary-lambda` on a freshly added `resolve=lambda: f()` that a whole-repo

`pylint $(git ls-files ...)` rated 10.00/10. Reproduce with the actual lane

(`<script> <lane>`, e.g. `verify.sh pylint`) before pushing, not with a full-tree equivalent.

- **The interpreter you run the gate with determines what the gate's subprocesses can import.**

Gates that spawn `sys.executable` (scaffold/regeneration lanes) inherit the python that

launched the gate. On macOS the bare `python3` is often the CommandLineTools build (no

site-packages, e.g. `ModuleNotFoundError: No module named 'jsonschema'`) — always run lanes

with the repo's venv python, or rely on the gate's own `_resolve_python` fallback.

- **A worktree's own broken `.venv` shadows the main checkout's good one.** If the gate's

python resolution prefers `$PWD/.venv` and the worktree has a bare/broken venv (e.g. no pip,

no deps), every lane fails with import errors even though the main checkout's venv is

populated. Fix: delete the worktree `.venv` so resolution falls back to the main checkout's

venv. A stray venv you did not create is the first suspect when a fresh worktree's lanes

fail on missing modules.

- **`git rebase --continue` hangs: the pre-commit hook wedges in rebase context.** Lefthook's

pre-commit can hang indefinitely while a rebase is being continued (observed with the

code-review-graph hook; the git process times out waiting on it and the rebase state survives

the kill). Replay the rest of the rebase with hooks off:

`GIT_EDITOR=true git -c core.hooksPath=/dev/null rebase --continue`. This is safe because the

pre-push gate re-runs every relevant lane on the pushed tree — enforcement happens at push,

not at rebase. A normal commit on a clean branch is unaffected; only the rebase context hangs.

- **The gate runs SCOPED lanes, not the full list.** `progress.log` shows e.g. `PUSH lanes: docs terraform`

when a branch only touches docs+infra — the gate picks lanes by changed paths. Don't read a

scoped run as proof the E2E/dotnet lanes passed, and don't be surprised a scoped run finished in

seconds. `self-test` tooling tests ride inside the `docs` lane.

- **Playwright-style e2e lanes reuse a live dev server on a fixed port** (`reuseExistingServer:

!CI`). A wholesale failure pattern — every page test timing out while the [WebServer] log floods

`Cannot find module` / TS2307 for imports — is a SERVER-level failure, not your change. The

reused server may be a zombie from a removed worktree (its node_modules symlink is gone), or the

gate's own server hit a transient gap while a concurrent `npm install` in the main checkout

churned package dirs. Diagnose in this order: (1) who owns the port — `lsof -nP -iTCP:<port>

-sTCP:LISTEN`, then `lsof -p <pid> | grep cwd` (never kill a process whose cwd is the owner's

checkout); (2) does the failing module exist on disk; (3) re-run the e2e lane alone for fast

feedback; (4) re-push. Keep a repo-specific playbook (port, lane commands, worktree

symlink quirks) beside the gate when the project has one.

- **After an auto-repair commit, push AGAIN.** The gate's repair path regenerates drifted

artifacts (e.g. `docs/meta/baseline.json`, `trust-index.json`), commits

`chore(gate): regenerate stale artifacts`, and exits non-zero with "push again" — the first

push deliberately fails. Re-push the same ref (plain push; after a rebase, `--force-with-lease`).

- **A lane can be red because the gate tests a working tree polluted by ANOTHER task's

uncommitted files — the repair path refuses to touch them, so the lane stays red until that

task commits.** Repair declines with "not regenerating: <path> has uncommitted changes, and

the repair may only commit bytes it wrote itself". This is not your diff. Prove it before

bypassing: (1) your branch diff contains no file in the failing lane's scope

(`git show --stat HEAD` / `git diff origin/main...HEAD`); (2) the lane's own output shows

your files clean. Then the gate's failure block itself names the

sanctioned escape: `VERIFY_SKIP=<lane> git push` — stage-scoped, NOT `--no-verify`

(skips every lane) and NOT fixing the other task's files. The skipped lane still runs on

GitHub CI only if CI has it; a docs lane is often local-only, so the PR is unaffected.

- **`git push --delete <branch>` also runs the pre-push gate** and can fail on it; bypass with

`git -c core.hooksPath=/dev/null push origin --delete <branch>` — deleting a remote ref needs

no verification.

## References

The gate script's own header + logs are the primary reference (usage, lane names, exit

codes, reproduce/bypass commands). Keep a repo-specific playbook beside the gate script

with the lane list and commands, log layout, AppHost command, E2E auth header + config,

manual repro steps, and any repo-local drift/state files — and update it when the gate

changes.