ci-triage · diff
git:20261003.258a0fd to git:20261004.2c80ffa
2 added, 2 removed. Audit A to A.
---
name: ci-triage
description: Monitor and triage the POST-PUSH automated-review surface for the whiteboard repo — GitHub Actions CI, CodeRabbit, AccessLint, the WIP app, CodeQL, and Dependabot — into Tasks / whiteboard issue documents. Use after the integrator pushes, when CI/PR checks complete or a review bot comments, to decide what is real and file/fix it. Complements the local lefthook pre-push gate (which runs before push); this covers what only the cloud sees.
---
# CI & automated-review triage (whiteboard)
`lefthook` (pre-push) catches local typecheck, lint and architecture-guard breakage before push; it runs no test suite and no build. This skill covers the layer ONLY the cloud sees after push: CI workflows + the GitHub-App review bots on the PR. The integrator monitors them, separates signal from noise, and files actionable items into the Task list / whiteboard `type: issue` documents (or fixes quick ones on the spot).
## What runs on this repo (verified surface)
| Source | What it is | How to read it |
|--------|-----------|----------------|
| **`ci-gate`** | The one required check over every `ci.yml` job (`check` lint/typecheck, the test jobs, `verify` build + smokes, `extension-smokes`, …) | the head commit's check-runs (see Monitor); logs: `gh run view <run-id> --log-failed` |
| **CodeRabbit** | AI PR review (line comments + summary). **Skips while the PR title contains `WIP`/draft** | `gh api repos/{owner}/{repo}/pulls/<PR>/reviews`, `.../issues/<PR>/comments`, `.../pulls/<PR>/comments` |
| **AccessLint** | accessibility review app | its check + PR review comments |
| **WIP** | reports work-in-progress from the title — **informational, measured not to block a merge** (#1201: with WIP the sole non-success check, `mergeable_state` read `unstable`, never `blocked`; Draft is the one mechanical merge block) | its check-run stays `queued`/`in_progress`; remove `WIP`/`(WIP)` from the title to settle it + un-skip CodeRabbit |
| **CodeQL** | security code-scanning | **primary (scope-free):** its findings post as PR review comments by `github-advanced-security[bot]` — `gh api repos/{owner}/{repo}/pulls/<PR>/comments --jq '.[]\|select(.user.login=="github-advanced-security[bot]")'`. **enrichment (may 404 on scope / "no analysis"):** `gh api repos/{owner}/{repo}/code-scanning/alerts` (needs `security_events`/admin) for severity/state |
| **Dependabot** | dependency-bump PRs + security alerts | **handled by its own flow** — the `dependabot-triage` workflow + `dependabot-review` skill (semver×ecosystem classify, conflict-cascade-safe merge, `pnpm audit --prod` gate). Don't triage it here. |
Note: some security APIs need `gh auth refresh -s security_events` (or admin) — if a fetch 404s on scope, skip that source and note it rather than failing the triage.
## Reading GitHub: REST, and "could not read" is not "settled"
Everything here goes through `gh api` (REST). The GraphQL-backed subcommands — `gh pr view`, `gh pr checks`, `gh pr list`, `gh pr status`, `gh pr ready`, `gh pr comment`, `gh repo view` — answer `HTTP 403: GitHub GraphQL is not available` in a Claude Code web session, which is a normal contributor path. `gh run view`, `gh run list` and `gh api` are REST and work there. `{owner}`, `{repo}` and `{branch}` in a `gh api` endpoint fill from the local remote and branch with no network call.
A watch loop must never turn a failed or empty read into a result. The shape that did — `s=$(gh pr checks … 2>/dev/null || echo '[]')`, then "no pending, stop" — exited at once on a 403 and reported a clean PR. Below, a failed read retries and, after five in a row, ends as UNKNOWN; an empty check-run list (nothing started yet) is waiting, not done.
## Monitor (live)
Watch the PR head's check-runs until they settle, emitting each terminal result (use the `Monitor` tool):
```bash
PR=<PR>; prev=""; fails=0
while true; do
if sha=$(gh api "repos/{owner}/{repo}/pulls/$PR" --jq .head.sha) &&
s=$(gh api --paginate "repos/{owner}/{repo}/commits/$sha/check-runs" \
--jq '.check_runs[] | [.name, .status, (.conclusion // "")] | @tsv'); then
fails=0
cur=$(awk -F'\t' '$2=="completed" {print $1": "$3}' <<<"$s" | sort)
comm -13 <(echo "$prev") <(echo "$cur") # emit only newly-settled checks
prev=$cur
# No rows yet means the runs have not been created: keep waiting.
[ -n "$s" ] && [ -z "$(awk -F'\t' '$2!="completed"' <<<"$s")" ] && break
else
fails=$((fails+1))
[ "$fails" -ge 5 ] && { echo "UNKNOWN: GitHub unreadable ${fails}x — nothing is known to be settled"; exit 1; }
fi
sleep 30
done
```
PushNotification when a check flips to `failure` — that changes what the integrator does next.
- **Caveat (Draft PRs): the `WIP` check stays pending forever while the PR is a Draft** — the WIP app flags Drafts as work-in-progress regardless of the title (removing `(WIP)` from the title is NOT enough; only `gh pr ready <PR>` settles it). `AccessLint` can also linger on Drafts. So a "wait until ALL checks settle" loop never exits on a Draft. **Watch the gating check (`ci-gate`) specifically instead:**
+ **Caveat (Draft PRs): the `WIP` check stays pending forever while the PR is a Draft** — the WIP app flags Drafts as work-in-progress regardless of the title (removing `(WIP)` from the title is NOT enough; only taking the PR out of Draft settles it). `AccessLint` can also linger on Drafts. So a "wait until ALL checks settle" loop never exits on a Draft. **Watch the gating check (`ci-gate`) specifically instead:**
```bash
PR=<PR>
while true; do
sha=$(gh api "repos/{owner}/{repo}/pulls/$PR" --jq .head.sha) || { sleep 60; continue; }
b=$(gh api --paginate "repos/{owner}/{repo}/commits/$sha/check-runs" \
--jq '.check_runs[] | select(.name=="ci-gate") | (if .status=="completed" then .conclusion else "pending" end)') || b=""
# An unreadable answer is "" and keeps waiting; only a real conclusion ends it.
case "$b" in ""|pending) ;; *) echo "ci-gate: $b"; break ;; esac
sleep 60
done
```
`ci-gate` ends `success`, `failure` or `cancelled`; anything but `success` is the integrator's cue.
## Triage (the `ci-triage` workflow)
`Workflow({ scriptPath: '.claude/workflows/ci-triage.workflow.mjs', args: { pr: 56 } })` gathers failed-check logs + CodeRabbit/AccessLint comments, triages each source (real vs noise, severity), adversarially verifies the load-bearing ones, and returns a deduped backlog. Read-only — the integrator files the survivors.
## Triage rubric (signal vs noise)
- **CI failure (`ci-gate` or a job under it)** → almost always REAL and blocking. Read `--log-failed`, reproduce locally, fix on the spot (it gates merge). A flaky-isolation failure (see `audit-test-fixture-dedup`) is the one exception — re-run before treating as real.
- **CodeRabbit** → high recall, variable precision. Treat each comment as a CANDIDATE: keep correctness/security/contract points; drop style nits already covered by Biome and "consider"-grade suggestions that don't apply. Verify against the actual code before filing (it hallucinates context).
- **AccessLint** → real a11y findings on UI diffs; keep, file under the touched component.
- **CodeQL** (`github-advanced-security[bot]` PR comments) → security; verify the data-flow is real (not an already-sanitized path). A finding on an **untrusted-input path** (a parser/regex over user/document content — ReDoS, injection) is REAL and **blocking**: fix it on the branch (red test first) before merge, don't just task-track it. Genuinely lower-severity or false-positive ones → task-track / dismiss with a recorded rationale.
- **Dependabot** → not triaged here; use the `dependabot-triage` workflow + `dependabot-review` skill (PRs + alerts, security-first, `pnpm audit --prod` gate).
## Filing (integrator)
Per finding: blocking/CI → fix now; real but not blocking → `TaskCreate` (track=task) or a whiteboard `type: issue` document (backlog) per the `ticketing` skill; noise → dismiss (resolve the CodeRabbit thread with a one-line why). Don't refile what's already a Task/issue.
## CodeRabbit: on-demand trigger + rate-limit re-queue
CodeRabbit does **not** auto-review a **Draft** PR or a PR whose title contains `WIP`/`(WIP)` (it posts "Review skipped"). Two consequences:
- **Monitoring**: the `WIP` GitHub-App check stays *pending forever* while the title says WIP, so a "wait until all checks settle" monitor never exits. Prefer a **Draft PR** for merge-blocking (Draft blocks merge natively) and keep the title WIP-free, so every check settles cleanly.
- **Triggering CodeRabbit on a Draft**: it must be invoked explicitly with a PR comment:
```bash
gh api "repos/{owner}/{repo}/issues/<PR>/comments" -f body="@coderabbitai review" # review the latest changes (REST form of `gh pr comment`)
# other commands: "@coderabbitai full review" (re-review everything),
# "@coderabbitai pause" / "resume", "@coderabbitai resolve" (resolve its threads),
# "@coderabbitai configuration"
```
**`@coderabbitai review` is INCREMENTAL** — it does not re-review commits CodeRabbit already reviewed, so on an already-reviewed PR it just replies "✅ Review finished" with no new findings (verified on #56). For a real pass over accumulated work, use **`@coderabbitai full review`**. (Plain `review` is also stated to apply "only when automatic reviews are paused".)
**Strategic timing (free OSS tier = strict rate limits).** Do NOT trigger on every commit. Trigger at meaningful points: a milestone batch landed + green, just before requesting human review, or after a large fold. One review per accumulated batch, not per push.
**CodeRabbit EDITS one sticky comment; it does not post a new one per state.**
Watch the comment BODY, never "is there a new `coderabbitai[bot]` comment".
The same comment id carried, in order: the rate-limit warning, then
`Currently processing new changes in this PR`, then the finished review. A
monitor waiting for a new comment saw the "I will review" acknowledgement and
then nothing for 26 minutes, while the result was being written into a
comment posted half an hour earlier. Poll
`gh api repos/{o}/{r}/issues/comments/<comment-id>` (or the issue-comments
list, reading `.body` and `.updated_at` of the one it already owns) and match
on its text:
| body contains | state |
|---|---|
| `Review limit reached` | rate-limited, not started |
| `Currently processing` | running |
| `Actionable comments posted: N` | finished |
**The body alone is not enough — take a BASELINE before triggering.** That
same persistence is what makes the table a trap on a re-review: the comment
already holds the PREVIOUS run's `Actionable comments posted: N`, so a monitor
that only matches text reports "finished" on its first poll, before the new
review has written anything. Record the sticky comment's `id` and its
`updated_at` BEFORE posting the trigger, and accept a terminal state only from
a body whose `updated_at` is newer than that baseline:
```bash
base=$(gh api repos/{o}/{r}/issues/<PR>/comments \
--jq '[.[]|select(.user.login=="coderabbitai[bot]")]|last|{id,updated_at}')
# ... post @coderabbitai review, then poll that id and require
# .updated_at > the recorded one before believing the body.
```
A PR with no `coderabbitai[bot]` comment yet has no baseline and no trap —
there the first comment to appear is this run's.
Inline findings arrive separately as PR **review comments**
(`gh api repos/{o}/{r}/pulls/<PR>/comments`), so a run with zero of those and
a finished sticky body is a genuine clean review — not a monitor that missed
something.
**Rate-limit → re-queue at the stated recovery time.** When rate-limited, CodeRabbit replies with a comment saying when it will reset (e.g. "rate limit … try again in N minutes" / a timestamp). Flow:
1. Record the sticky comment's `id` + `updated_at` as the baseline, THEN
trigger `@coderabbitai review`.
2. Watch that comment per the table above — a state counts only once its
`updated_at` is newer than the baseline.
3. If it says **rate-limited**, parse the reset time and `ScheduleWakeup({ delaySeconds: <until reset + small buffer> })` — on wake, re-post `@coderabbitai review`. Loop until it actually reviews (cap the retries).
4. If it **starts reviewing**, let it finish, then triage its comments via this skill / the `ci-triage` workflow. `Currently processing` can also STALL: one run sat on it for ~50 minutes with the comment's `updated_at` frozen at the acknowledgement. **Cut it off at 20 minutes with no movement in `updated_at`** — clear of a slow-but-live run (the ones that finish move within a few minutes, which is what CodeRabbit's own copy promises) and well short of the one dead run measured — then treat it as unavailable and fall back to the standing decision that AI review's absence does not block a merge (`dev-flow.md`), rather than waiting indefinitely. A deadline off two data points is a starting number, not a law: widen it if a run is ever seen to recover past it, and say so here.
- Bypass note: removing a PR from Draft (`gh pr ready <PR>`) is a deliberate human step (it signals "ready for human review/merge") — don't auto-undraft.
+ Bypass note: removing a PR from Draft is a deliberate human step (it signals "ready for human review/merge") — don't auto-undraft. It is also not an agent step in a web session: GitHub's REST API has no route that marks a pull request ready for review (only a GraphQL mutation does, which a web session cannot reach), so the human does it from the PR page.