--- name: p9 primitive: P9 category: orchestration description: | P9 — Broomva productive-wait primitive (the wait optimizer). Convert any blocking external operation — PR CI checks, push-triggered deploys, builds, long-running index ops — into work on the next priority. The reference implementation is a PR CI watcher: drains a context-scoped deferred-work queue while `gh pr checks --watch` runs in the background, classifies failures, and self-heals known categories. Non-PR waits get the same lifecycle via `p9 wait-for` (deploy readiness, long extractions — poll a predicate command with heartbeats and re-arm). Every watcher and wait upholds the termination invariant (BRO-1701): on success, failure, OR kill it reports state + next action and pushes through the configured notify channels — killed watchers never die silently; `p9 stuck-scan` catches live-but-wedged ones. Merge authorization stays with the existing control metalayer (.control/policy.yaml). when_to_use: | Automatically after every `git push` that opens or updates a PR. After a push that triggers a non-PR deploy or any long-running external operation: `p9 wait-for --cmd ''` instead of polling by hand. Hard rule: the agent MUST apply productive-wait discipline before `sleep` is ever an option. --- # P9 — Productive Wait (Wait-Optimizer Skill) ## Cardinal rule > **Never `sleep` on a blocking wait.** Whether you're waiting on PR CI, > a push-triggered deploy, a long build, or an index sync — convert the > wait into productive work on the next priority. For PR CI, `p9 watch ` > spawns the observer in the background and the agent pulls work from the > wait-queue. For non-PR waits, `p9 wait-for --cmd ''` > gives the wait the same lifecycle (state row, heartbeat, termination > report, re-arm). Sleep is a footgun — it burns clock time the agent could > be using to validate definitions, refresh the knowledge graph, or draft > the next slice. ## When to invoke | Trigger | Action | |---|---| | `git push` opens or updates a PR | `p9 watch --background` immediately | | `run_in_background` task notification fires for the watcher | `p9 status --pr ` to read terminal state | | `gh pr checks` returned non-zero | `p9 heal --classify` to inspect failure | | Push triggers a deploy / long non-PR operation | `p9 wait-for --cmd '' --detach` (or `--preset railway\|vercel`) | | Fresh session picking up after a kill/crash/reboot | `p9 report` to read state + next action; `p9 rearm` to re-arm dead watchers | | A watcher/wait looks wedged | `p9 stuck-scan` — structured dump + notification | | About to `sleep` | **Don't.** Pull from `p9 wait-queue pop` instead | ## Parallel agent sessions (BRO-1529) P9 state lives in one shared dir (`~/.config/broomva/p9/`). Concurrent agents stay collision-free by **scoping every record to a session id**. > **Contract:** each parallel agent session/worktree/wave-plan MUST export > `BROOMVA_P9_SESSION=` before calling `p9`. Fanout (P5) > worktrees, `bstack wave` plans, and autonomous runs each set their own. > If unset, p9 falls back to a single persisted id (`session-default.id`) — > i.e. backward-compatible **global** behavior, *not* isolation. No env var ⇒ > no parallel safety. What the session id buys you: | Dimension | Behavior | |---|---| | **Concurrency ceiling** | `max_concurrent_prs` is counted **per session**, and over the scope `ci_watch.max_concurrent_prs_scope` names (`repo`, the default, or `global`). A session's *own* second watch in the same scope still blocks. | | **PR identity** | Keyed by `(repo, pr)` — the same PR number in two repos never collides, in the state table *and* in the ceiling count. | | **Wait-queue** | `pop`/`list`/`clear` default to the **current session's** view (its items + legacy-unowned). `--all` crosses sessions. This is what "context-scoped" finally means in code. | | **Watcher de-dup** | A second `p9 watch` on a PR that already has a **live** watcher is refused (`--force` to supersede). A **dead** watcher is superseded automatically once aged, or now via `--adopt`. | ### Repo identity (BRO-1988) Every lifecycle command (`watch`, `merge-ready`, `merge-status`, `auto-merge`, `abandon`) resolves the repo **once, before it reads state**, and uses that one answer for both the read key and the write key. Resolution order: 1. `--repo OWNER/REPO` 2. `BROOMVA_P9_REPO` — the deterministic hook for tests and for harnesses that already know the target. `-` pins "no repo"; an **empty value means unset** (shell convention), so `export BROOMVA_P9_REPO=$(cmd_that_failed)` degrades to normal detection instead of silently dropping into repo-less state. A value that does not reduce to `OWNER/REPO` gets a stderr notice. 3. `gh repo view` → `git remote get-url origin` from cwd (memoized per process). The `git` fallback matters wherever `gh` is absent or unauthenticated (CI). Any spelling normalizes to `owner/name` — `https://host/o/r.git`, `git@host:o/r.git`, `ssh://git@host/o/r.git`, trailing slashes — and comparison is case-insensitive, so one logical repo never splits into two keys. Host parsing is generic, not a github.com allowlist: an allowlist made every repo on a GHE/GitLab host collapse to a single key. The host is stripped rather than keyed on, matching `gh --repo` (which takes a bare `OWNER/REPO` and gets its host from `GH_HOST`). **Ceiling scope** — `ci_watch.max_concurrent_prs_scope: repo | global` (default `repo`). `repo` counts in-flight PRs per `(session, repo)`; `global` is the pre-BRO-1988 cross-repo count — one bounded merge train across every repo, which is what the design spec describes. `repo` is the default because a global count let an in-flight PR in one repo refuse `p9 watch` in an unrelated one, and since `watch` is the only transition into GREEN that made the whole lifecycle unreachable there. The tradeoff is real and is why the knob exists: at `max_concurrent_prs: 1` across N repos a session holds N watchers, and the defer-into-wait-queue discipline stops firing cross-repo. Spec reconciliation is tracked separately. With no resolvable repo the count falls back to global — an ambiguous identity must not silently disable the ceiling. **Rows with no recorded repo.** Rows written before repo stamping carry `repo: ""`, and p9 does **not** migrate them. It keeps that key: `""` is the row's true identity and a perfectly good one — it collides with no real repo, so it can neither shadow one nor (under a repo-scoped ceiling) hold its slot. `current_pr_state(pr, "")` still reaches such a row, and `reap` / `rearm` still drain it. Nothing is discarded; p9 simply declines to invent the one thing it does not know. Attributing them to the ambient repo was tried and reverted, because: - it put a `gh repo view` call on every state read; - it let `p9 rearm` re-watch the PR against the ambient repo — and omitting `--repo` from the child argv does **not** help, because the child calls `resolve_repo(None)` itself and, sharing cwd and env, resolves the same value. The observed chain ended in `gh pr merge` on a PR nobody targeted; - and when the ambient repo genuinely had a PR of the same number, the guess *shadowed* the real row — recreating the exact defect BRO-1988 fixes. So a row with no recorded repo is **folded and not re-armed**. There is no correct repo to re-arm against; that is the whole content of "no repo recorded". Folding frees the concurrency slot, and recovery is an explicit `p9 watch --repo --adopt` — a human naming the repo p9 could not. ### Lifecycle / self-healing - **`p9 reap`** — reconcile dead-watcher rows (pid gone) to `ABANDONED`, freeing the concurrency slot a crashed/closed session would otherwise hold forever. `--now` ignores the grace window; `--no-reconcile` skips the gh enrichment query. `watch` and `status` run a liveness-only reap as a cheap preflight, so the ceiling self-heals without manual `cleanup`. - **`p9 watch --adopt`** — re-watch a PR whose prior watcher pid is gone (orphan recovery after a session ends mid-watch). - **Queue TTL** — items are pruned once their PR reaches a terminal state, or after `BROOMVA_P9_QUEUE_TTL_DAYS` (default 14). - **`p9 heal --apply`** — run the classified `heal_command` under `heal.lock` (serialized workspace-wide, so a heal in a parallel session can't race on shared codegen/cache). Auto-classifiable failures only; `--dry-run` prints the command. `--classify` stays read-only. ## Wait-time work selection (priority order) When the watcher is running, drain work from these sources in priority order (higher = pulled first): 1. **session** — TODOs already on the agent's TaskList tagged `wait_ok=true`. 2. **memory** — items from `~/.claude/.../memory/MEMORY.md` flagged "needs follow-up" within the last 24h. 3. **graph** — knowledge-graph entities adjacent to files-touched-in-PR (BFS depth 1 via `bookkeeping.py query`). 4. **docs** — cross-refs from the current PR's diff (mentioned files not yet updated). 5. **linear** — tickets in the current cycle, label-matched to PR's Linear ID. ### Isolation tier (per spec §5.5) Each pop returns the inferred isolation tier: | Work type | Tier | Where it happens | |---|---|---| | research, docs, knowledge-graph mutations, Linear updates | `none` | current worktree, no separate branch | | code that's independent of the in-flight PR | `worktree` | new P5 worktree off main | | code that depends on the in-flight PR | `stacked_branch` | branch off `feat/X+1` from `feat/X` HEAD | | anything touching `CLAUDE.md` / `AGENTS.md` / `.control/` | `blocked` | **not** auto-handled; surface to user | ## Wakeup protocol When the bg task notification fires: ```text 1. p9 status --pr --json 2. parse `to_state`: - GREEN → p9 merge-ready ; defer to control metalayer - RED_CLASSIFIED → p9 heal --classify; if classified+evaluator-positive, apply heal_command (in PR scope only); push amend; loop - RED_UNCLASSIFIED, ESCALATED → notify user via Linear ticket; stop healing, keep watcher alive in case human pushes a fix - ABANDONED → surface failure to user; remove watcher; skip cleanup ``` ### The watcher exit code is necessary-not-sufficient (BRO-1489) `GREEN` only means `gh pr checks --watch` exited 0 — which it does on a *subset* of checks (required-only) and *before* async bot reviews (CodeRabbit) settle. Observed three times on bstack PR #78: exit 0 while the PR was `UNSTABLE` / had a pending review. `p9 merge-ready` therefore **verifies the real merge predicate** before marking `MERGE_READY`: it queries `gh pr view --json mergeable,mergeStateStatus,reviewDecision` plus a best-effort `gh api graphql` unresolved-thread count, and is ready iff `mergeStateStatus ∈ {CLEAN, UNSTABLE}` with no `CHANGES_REQUESTED` and zero unresolved review threads. `BLOCKED`/`DIRTY`/`BEHIND`/`DRAFT`/`UNKNOWN`, an open thread, or any gh error → refused (fail-safe). Pass `--no-verify` to skip (test/offline only). Query it directly without transitioning state: ```text p9 merge-status [--json] # exit 0 iff merge-ready; prints the verdict + reason ``` ## Termination conditions The agent exits the heal loop when **any** of: - `to_state ∈ {MERGED, ESCALATED, ABANDONED}` (terminal) - `attempt ≥ ci_heal.max_attempts` (default 5) - evaluator returned `stalled=true` for two consecutive cycles - user interrupt (Ctrl-C in terminal, or chat message) - session ends (the `Stop` hook leaves watchers running for next session pickup) ## Examples ### Example 1 — Green on first try (happy path) ```bash $ git push origin feat/my-change $ gh pr create ... ; PR=42 $ p9 watch $PR --background watcher_id=ab12cd34ef56 pid=78901 pr=42 repo=broomva/workspace # Run watcher in foreground/background; meanwhile drain queue $ p9 wait-queue pop {"id": "...", "source": "graph", "item": "verify entities adjacent to ...", "isolation_tier": "none"} # ... agent does the work ... # bg task notification fires; check terminal state $ p9 status --pr 42 --json {"open_prs": [{"pr": 42, "to_state": "GREEN", ...}]} $ p9 merge-ready 42 PR #42 marked MERGE_READY (control metalayer authorizes merge) # control-gate-hook authorizes; agent runs `gh pr merge` ``` ### Example 2 — Lint-failure self-heal ```bash $ p9 status --pr 42 --json {"open_prs": [{"pr": 42, "to_state": "RED_CLASSIFIED", "attempt": 0}]} $ p9 heal 42 --classify {"failure_type": "lint", "classified": true, "confidence": 0.8, "heal_command": "bun run lint:fix", "rationale": "matched lint at confidence 0.80"} # agent runs heal_command, scoped to PR diff files $ bun run lint:fix $ git commit -am "fix(lint): heal CI" $ git push --force-with-lease # only if existing P6 policy permits $ p9 watch 42 --background # new WATCHING cycle; attempt=1 ``` ### Example 3 — Unclassified-failure escalation ```bash $ p9 heal 42 --classify {"failure_type": "unclassified", "classified": false, "confidence": 0.0, "heal_command": null, "rationale": "no rubric pattern matched"} # Agent does NOT attempt to heal. Creates a Linear ticket via MCP: # title: "[P9 ESCALATION] PR #42: feat/my-change" # body: failure signature + log excerpt # label: ci-heal-escalation # Watcher stays running — if a human pushes a fix, watcher resumes and # the next green check transitions to MERGE_READY. ``` ## Background-work visibility (BRO-1701) ### Termination invariant (hard rule) > **On watcher termination — success, failure, OR kill — P9 always reports > state + next action.** Killed watchers must not die silently. Every exit path of `p9 watch` and `p9 wait-for` (green, red, timeout, SIGTERM/SIGINT/SIGHUP, unexpected exception) folds a state event, prints a `P9-TERMINATION-REPORT {json}` line to **stderr** (stdout stays machine-parseable), and pushes through the notify channels. The report carries `state`, `cause`, and a concrete `next_action` (e.g. ABANDONED → "re-arm: p9 watch --adopt"). SIGKILL and machine death can't be trapped — that path is covered by `p9 reap` (emits the same report shape when it reconciles a dead row) and `p9 rearm`. Read-side: `p9 report [--pr ] [--json]` renders the latest report for every tracked watcher/wait — this is what a fresh session (or the Tier-1 #1 post-notification reconcile rule) consumes to learn what happened and what to do next. Report fields are additive-only. `gh pr checks --watch` output now lands in `$P9_HOME/logs/watch-.log` (not the void): the log's mtime is a progress signal for `stuck-scan` and its tail rides along in reports — full details to understand *why*, not just *that*, something died. ### Notify channels (push-to-phone) `p9 notify [--body ...]` and every termination/stuck event fan out to channels in `$P9_HOME/notify.json`: ```json {"channels": [ {"type": "ntfy", "topic": "broomva-p9", "url": "https://ntfy.sh"}, {"type": "webhook", "url": "https://example.com/hook"}, {"type": "command", "cmd": "scripts/p9-escalate-notify.sh"} ]} ``` - **ntfy** — reaches a phone with zero infra (install the ntfy app, subscribe to the topic). Quick-config without a file: `export BROOMVA_P9_NTFY_TOPIC=<topic>`. - **webhook** — generic JSON POST (`{title, body, payload}`). - **command** — JSON on stdin to any hook script; this is the seam for Omnara, Telegram, Discord, or claude-remote-sessions relays. - Escalation-class events (`termination:escalated`, `stuck`, and any kind containing `escalat`) additionally fire the policy's `ci_heal.escalation_channel.notify_hook` (previously dead config — now invoked). Delivery is best-effort and per-channel isolated: a failing channel is recorded and skipped, never raised — a notification must never take down the watcher it reports on. Every attempt (even with zero channels) appends an audit row to `$P9_HOME/notify.jsonl`; that audit floor is what makes the termination invariant verifiable after the fact. **In-session protocol (PushNotification/Omnara):** when an agent session receives a termination report or stuck dump while the user is off-terminal, it MUST surface it through the harness `PushNotification` tool (or the Omnara session surface) — the file-level channels cover the no-session case; the harness tool covers the live-session case. ### Non-PR waits — `p9 wait-for` ```bash p9 wait-for railway-deploy --preset railway --interval 30 --timeout 1800 --detach p9 wait-for vercel-deploy --preset vercel --target <deployment-url> --detach p9 wait-for extraction --cmd 'test -f /tmp/extract.done' --interval 60 --timeout 7200 --detach ``` Polls the predicate command until exit 0 (`SUCCEEDED`), deadline (`TIMED_OUT`), first-poll exit 126/127 (`FAILED` fast), or signal (`KILLED`) — each terminal state folds + reports + notifies exactly like a PR watch. State lives in `$P9_HOME/waits.jsonl` (its own stream — PR-state consumers never see wait states); every poll touches a heartbeat file that `stuck-scan` reads. Presets are convenience templates over `--cmd` — deploy-CLI output shapes drift, so verify against your installed CLI and fall back to an explicit `--cmd` when they do. ### Re-arm after kill — `p9 rearm` `p9 rearm [--dry-run] [--now]` scans for dead-but-unfinished work: PR rows whose watcher pid is gone re-enter via a detached `p9 watch --adopt`; dead waits are folded ABANDONED and re-spawned from their recorded argv (with `rearmed_from` lineage). This closes the loop the July-1 leverage audit flagged: watchers killed before their notification fires now leave a report AND come back. ### Stuck-detector — `p9 stuck-scan` `p9 stuck-scan [--threshold-min N] [--json]` flags **live** watchers/waits with no progress (state event / watch-log mtime / heartbeat) inside the threshold (default 45 min, env `BROOMVA_P9_STUCK_MIN`): structured failure dump (pid, ages, log tail, next action) + notification, deduped to one per stall episode (`--renotify` overrides; a new episode starts when progress moves). Dead pids are reap/rearm territory, not stuck. Exit code 1 when anything is stuck — wire it into cron/governor loops as a cheap probe. ## Cardinal invariant (hard rule) > **P9 never silently drops state.** Every failure produces (a) a > `state.jsonl` event, (b) a Linear ticket, or (c) both. If P9 cannot > write to `state.jsonl` AND cannot reach Linear, it crashes loudly > (exit 99) — degraded silent operation is forbidden. ## See also - Spec: `docs/superpowers/specs/2026-05-04-p9-ci-watcher-design.md` - Rubric: `references/scoring-rubric.md` - CLI: `scripts/p9.py` (run `python3 scripts/p9.py --help`) - Related primitives: P1 (Conversation Bridge), P2 (Control Gate), P3 (Linear Tickets), P4 (PR Pipeline), P5 (Parallel Agents), P6 (Knowledge Bookkeeping), P8 (Branch + Worktree Janitor), P10 (Worktree Hygiene Discipline), P11 (Empirical Feedback Loop).