git:20260927.9852324 to git:20260929.bb5ca16

---

name: review-reviewers

description: Outcome-based analysis of tend's CI behavior — checks whether tend's outputs were accepted or rejected, escalating to session logs only when outcomes look wrong.

argument-hint: "<owner/repo>"

metadata:

internal: true

---

# Review Reviewers

Analyze tend's CI behavior on the target repo over the window Step 1 returns. Focus on **outcomes** — what the bot produced publicly and whether it was accepted — rather than internal session mechanics. Create PRs or issues on tend when outcomes reveal behavioral problems.

## First steps

Load `/tend-ci-runner:run-tend` first — it contains CI security rules, the per-action skills, and comment formatting. This skill opens PRs and issue comments on tend, so those rules apply. Judge the target repo's runs against its own instructions first: the repo wins over tend's defaults, per **First Steps** there.

## Core principle: outcomes over internals

The bot's job is to produce useful outputs: reviews, triage comments, fix commits, issue responses. Judge them first by whether they were **accepted** (merged, kept, acted on) or **rejected** (reverted, closed, corrected, disagreed with). Escalate to session-log inspection only when an outcome signal points at a problem to diagnose.

Log parsing and outcome checking produce bulk text; run them in a subagent so the context that makes the judgments — evaluating findings against the gates, deciding whether to act, drafting PRs — holds the conclusions rather than the transcripts.

Expected behavior that looks suspicious — designed no-ops, designed silence, re-review after a fix commit — is listed under **Non-issues** in `review-gates.md`. Read it before a finding becomes a PR.

## Target repo

**Target repo:** $ARGUMENTS

Analysis targets a consumer repo whose CI runs are analyzed. Findings result in PRs/issues on the current repo (tend) to improve skills and workflows.

Use `-R $ARGUMENTS` for commands that access the target repo (querying runs, PRs, issues). Commands without `-R` default to tend.

@review-gates.md

## Evidence accumulation

Evidence lives in one secret gist per target repo and month, indexed by the

monthly `review-reviewers-tracking` issue on Tend. Prepare it and read the

current and previous month's evidence:

This skill lives in tend's own `.claude/skills/`, so the bundled plugin's

scripts are reached through `$SCRIPTS` — Claude leaves `CLAUDE_PLUGIN_ROOT`

unset in the shell, Codex exports it. Each block below sets it.

```bash

SCRIPTS="${CLAUDE_PLUGIN_ROOT:-/home/tend-sandbox/tend-marketplace/plugins/tend-ci-runner}/scripts"

uv run --script "$SCRIPTS/review_reviewers.py" prepare-evidence "$ARGUMENTS"

```

The command finds or creates both index and gist, announces a new gist once,

persists their ids, and prints both evidence windows and the gist's

`gist_url`, which the PR bodies and summary below link.

After applying the gates, write this run's findings in the format from

`@review-gates.md` to `$TMPDIR/findings.md`. Append a `## Run

$GITHUB_RUN_ID` heading every run, including an all-clear window; the heading

is the audit trail future runs use. Then append it:

```bash

SCRIPTS="${CLAUDE_PLUGIN_ROOT:-/home/tend-sandbox/tend-marketplace/plugins/tend-ci-runner}/scripts"

uv run --script "$SCRIPTS/review_reviewers.py" append-evidence

```

The command refuses a findings file that does not name this run, fetches the

latest gist content, and appends without replacing prior evidence.

## Step 1: Setup

Resolve the **target repo's** bot login and load its repo-specific instructions upfront — both are needed throughout. `gh api user` returns the *analysis* bot (e.g., `tend-agent` when review-reviewers runs on tend), which is typically **not** the target repo's bot — filtering reviews/comments by the wrong login produces false "no bot output" negatives. Read `bot_name` from the target repo's `.config/tend.yaml`:

```bash

BOT_LOGIN=$(gh api "repos/$ARGUMENTS/contents/.config/tend.yaml" --jq '.content' 2>/dev/null \

| base64 -d 2>/dev/null \

| yq '.bot_name // ""' 2>/dev/null)

if [ -z "$BOT_LOGIN" ]; then

echo "ERROR: could not resolve bot_name from $ARGUMENTS/.config/tend.yaml" >&2

exit 1

fi

echo "BOT_LOGIN=$BOT_LOGIN (target: $ARGUMENTS)"

```

Read the target repo's own instructions to understand what the bot was told to do:

```bash

gh api "repos/$ARGUMENTS/contents/.claude/skills/running-tend/SKILL.md" \

--jq '.content' | base64 -d

```

If the file doesn't exist, try the legacy overlay paths and the repo's root project instructions (`.claude/skills/running-tend.md`, `.claude/CLAUDE.md`, `CLAUDE.md`, `AGENTS.md`). Understanding the repo's instructions is essential context for evaluating outcomes — without it, you'll misjudge authorized behavior as a violation.

Then check whether tend is still enabled there — it decides how to read an

empty run list below:

```bash

gh api --paginate "repos/$ARGUMENTS/actions/workflows" \

--jq '.workflows[] | select(.path | test("/tend-")) | "\(.state)\t\(.path)"'

```

`disabled_manually` across every `tend-*` workflow means tend is switched off

there. Say so in the summary; it is not an all-clear. There are no runs for

Steps 2 and 3 to key on, while Step 4 still applies — its dedup check is what

keeps a repo that stays switched off from opening the same issue every day.

A mix of states is ordinary — a consumer may disable one workflow — and

`disabled_inactivity` is GitHub expiring a cron in a repo with no recent

pushes, not a decision. Note either and carry on.

Then list recently completed tend CI runs on the target repo:

```bash

SCRIPTS="${CLAUDE_PLUGIN_ROOT:-/home/tend-sandbox/tend-marketplace/plugins/tend-ci-runner}/scripts"

TARGET_REPO=$ARGUMENTS uv run --script \

"$SCRIPTS/list_recent_runs.py" review-reviewers

```

The script discovers `tend-*` workflows by default. Prefix arguments replace that default rather than adding to it, so a call that widens the set names `tend-` too.

If empty, record the run as all-clear per **Evidence accumulation** above, then skip to Step 6 — all-clear only where the check above showed tend enabled.

If the script printed a `WARNING:` on stderr, the list is known-incomplete — the window was clamped, no anchor was found, or a workflow hit the fetch limit. Record a coverage gap naming the missing span instead of an all-clear, whether or not the list came back empty; the next run's floor advances past that span regardless, so an unrecorded gap is never revisited. If the script *fails* (non-zero exit, e.g. a transient API error), re-run it once; if it fails again, record the window as a coverage gap the same way — this run still concludes green, so the next tick anchors on it and never revisits the span.

**State the window you analyzed.** Its floor is the previous successful run of this workflow — normally a day back, since the workflow runs daily, and further after a missed tick, up to a 49h cap beyond which the script warns and records a coverage gap. Scope every claim to the window — "no problems since 08:12Z", never "no problems" — and say plainly when the run was dispatched to check on something that landed before it.

## Step 2: Survey outcomes

Check outcomes across all runs from Step 1: map runs to PRs and issues, and check acceptance signals. This is the bulk reading, so run it in a subagent per **Core principle: outcomes over internals**, briefed with a prompt like:

> Survey bot outcomes on `$ARGUMENTS` for the following runs: [run IDs from Step 1].

> The bot's login is `$BOT_LOGIN`.

>

> For each run, determine:

> 1. Did the bot produce visible output (review, comment, issue action, commit)?

> 2. If yes, was the output accepted or rejected?

>

> **Sweep the window repo-wide before mapping any run, and report the row counts.** The per-run mapping below walks run → branch → PR → endpoint; a break anywhere in that chain returns empty for *every* run at once, and uniform silence reads as a quiet hour rather than as a broken query. These calls take no run ID, so they fail independently of it:

>

> ```bash

> WINDOW_START=<window start, ISO 8601>

> WINDOW_END=<window end, ISO 8601>

> IN_WINDOW="[.[] | select(.user.login == \"$BOT_LOGIN\")

> | select((.created_at // .submitted_at) >= \"$WINDOW_START\"

> and (.created_at // .submitted_at) <= \"$WINDOW_END\")] | length"

>

> gh api "repos/$ARGUMENTS/issues/comments?since=$WINDOW_START&per_page=100" --jq "$IN_WINDOW"

> gh api "repos/$ARGUMENTS/pulls/comments?since=$WINDOW_START&per_page=100" --jq "$IN_WINDOW"

>

> CANDIDATES=$(gh -R $ARGUMENTS pr list --state all --limit 100 \

> --search "updated:>$WINDOW_START" --json number --jq '.[].number')

> echo "$CANDIDATES"

> for pr in $CANDIDATES; do

> gh api "repos/$ARGUMENTS/pulls/$pr/reviews?per_page=100" --jq "$IN_WINDOW"

> done | jq -s add

> ```

>

> Filter the comment calls on `created_at` at both ends. `since` is an `updated_at` floor with no ceiling, so unfiltered it also returns comments written days earlier and merely edited in the window, plus everything posted between the window end and now — the session's own runtime keeps that gap open, so on a busy repo it catches rows in most windows. Those are rows the per-run walk was right not to reach, and a check that contradicts a correct walk every run stops being believed. Leave `CANDIDATES` unbounded above: `updated:` matches the PR's own `updated_at`, so a range drops any PR that got bot output inside the window and was touched after it, and over-inclusion in a candidate list costs nothing. `--limit 100` is load-bearing — `gh pr list` defaults to 30 and truncates silently.

>

> The comment endpoints cover conversation and inline-review comments only; neither returns review submissions, and an empty-body `APPROVE` is `tend-review`'s most common output. Without the fourth count an approvals-only window reports `0, 0` and satisfies the gate below with two zeros carrying no signal. Report all four numbers at the top of your summary. If the sweep found in-window rows your per-run walk did not reach, the walk is wrong — re-map from the PR numbers the sweep found rather than reporting those runs as silent.

>

> **Acceptance needs a non-bot actor — name it.** For every acceptance or rejection signal, report the login that produced it: who merged, who reviewed, who replied, who pushed the follow-up. `$BOT_LOGIN` reviewing, replying to, or pushing to its own PR is the bot talking to itself, not acceptance. A non-bot merge *is* acceptance even when every commenter is the bot — the most common shape here is a maintainer merging a bot PR without ever posting. Reserve `bot-only — no human signal` for threads where `$BOT_LOGIN` is the only login across every surface. One `gh pr view` covers them all — merge actor, reviews, inline comments, conversation comments, commits:

>

> ```bash

> gh -R $ARGUMENTS pr view <pr> --json number,state,author,mergedBy,reviews,comments,commits --jq '{

> pr: .number, state, author: .author.login, merged_by: .mergedBy.login,

> reviews: [.reviews[] | {login: .author.login, state}],

> logins: ([.mergedBy.login, .reviews[].author.login, .comments[].author.login,

> .commits[].authors[].login] | map(select(. != null and . != "")) | unique)

> }'

> ```

>

> `logins` is the bot-only test: `["$BOT_LOGIN"]` or empty means no human touched the thread. Anything else, name that login and say which surface it came from. Every inline review comment belongs to a review record — including a standalone reply posted through the replies endpoint — so `reviews` covers inline commenters as well as submitted reviews, and its `state` gives the accept/reject direction. `comments` and `reviews` paginate in full. `commits` does not: `gh pr view` selects `commits(first: 100)` and never fetches a second page, oldest-first, so past 100 commits the newest drop out — exactly where a human follow-up push sits. On a PR that long, read the authors from `gh api "repos/$ARGUMENTS/pulls/<pr>/commits" --paginate` (itself capped at 250) before calling the thread bot-only.

>

> A comments-only check is not enough on its own: it misses both a silent maintainer merge (`mergedBy` set, nobody commenting) and a human `CHANGES_REQUESTED` that carries no inline comments.

>

> **How to map runs to outputs:**

> - `tend-review`: `gh -R $ARGUMENTS run view <run-id> --json headBranch` → find PR via

> `gh -R $ARGUMENTS pr list --head <branch> --state all` → check bot reviews via

> `gh api repos/$ARGUMENTS/pulls/<pr>/reviews`

> - `tend-notifications`: check for bot comments/issue-close events inside the window from Step 1

> - `tend-mention`: map run to issue/PR from triggering comment, check for bot replies

> - `tend-mention-relay`: no output of its own; the review event it re-posts is handled by the `tend-mention` `repository_dispatch` run below

> - `tend-mention` on `repository_dispatch` (the relay path for review events): there is no triggering comment and `headBranch` is the default branch, so neither route above resolves it. Read the target off the `verify` job's log, where the step env block prints the relayed payload:

> ```bash

> JOB=$(gh api "repos/$ARGUMENTS/actions/runs/<run-id>/jobs" --jq '.jobs[] | select(.name == "verify") | .id')

> gh api "repos/$ARGUMENTS/actions/jobs/$JOB/logs" | grep -E 'PAYLOAD_(KIND|PR|ID):'

> ```

> `PAYLOAD_PR` is the issue/PR number and `PAYLOAD_KIND` is the relayed event (`pull_request_review`, `pull_request_review_comment`). Read the gate's verdict off the `handle` job, which is gated on `should_run`: `handle` with conclusion `skipped` means the engagement gate declined and no agent booted — expected silence, not missing output. Do not read it off `React to mention`, which is skipped on every relayed `pull_request_review` regardless of the verdict (a review submission has no single comment to react to) and on a comment relay admitted for participation rather than a mention.

> - `tend-ci-fix`: map run → PR via `headBranch`, check for bot commits

>

> **Negative outcome signals** — report any sign the bot's output was rejected, corrected, or ignored. Common shapes (use judgment for signals not listed):

> - Human reviewer posted CHANGES_REQUESTED after bot approved

> - PR closed without merge shortly after bot approved

> - Bot posted no review despite a `tend-review` run completing on an open PR — first check whether designed silence applies (draft or self-authored PR), then inspect the session logs if it does not.

> - Subsequent commits reversed changes the bot approved

> - Bot-closed issue was reopened

> - Fix commit was reverted or CI still failing after bot pushed

> - Human replied to bot with correction or complaint

> - Bot comment contains corruption (literal `${`, unescaped bangs, backslash-backticks, broken heredoc markers)

>

> **Corruption-scan recipe.** Save bot bodies to a file, then scan with `grep`:

>

> ```bash

> mkdir -p "$TMPDIR/bot-output" && : > "$TMPDIR/bot-output/all.txt"

> # Issue/PR comments (issue_comment endpoint)

> for n in <pr-or-issue-numbers>; do

> gh api "repos/$ARGUMENTS/issues/$n/comments?per_page=100" \

> --jq ".[] | select(.user.login == \"$BOT_LOGIN\" and .created_at > \"<window-start>\") | \"=== #$n issue-comment \(.id) ===\n\(.body)\n\"" \

> >> "$TMPDIR/bot-output/all.txt"

> done

> # Issue bodies (when bot opened the issue this window)

> for n in <bot-opened-issues>; do

> gh api "repos/$ARGUMENTS/issues/$n" \

> --jq "select(.user.login == \"$BOT_LOGIN\" and .created_at > \"<window-start>\") | \"=== ISSUE #$n body ===\n\(.body)\n\"" \

> >> "$TMPDIR/bot-output/all.txt"

> done

> # PR bodies (only when bot opened the PR this window)

> for n in <bot-opened-prs>; do

> gh api "repos/$ARGUMENTS/pulls/$n" \

> --jq "select(.user.login == \"$BOT_LOGIN\" and .created_at > \"<window-start>\") | \"=== PR #$n body ===\n\(.body)\n\"" \

> >> "$TMPDIR/bot-output/all.txt"

> done

> # PR reviews + inline review comments — any PR the bot reviewed/commented on, not just

> # bot-opened. tend-review's output ships on human-authored PRs (the most common surface)

> # which would never appear in <bot-opened-prs>.

> for n in <pr-numbers-bot-reviewed>; do

> gh api "repos/$ARGUMENTS/pulls/$n/reviews" \

> --jq ".[] | select(.user.login == \"$BOT_LOGIN\" and .submitted_at > \"<window-start>\") | \"=== PR #$n review \(.id) state=\(.state) ===\n\(.body)\n\"" \

> >> "$TMPDIR/bot-output/all.txt"

> gh api "repos/$ARGUMENTS/pulls/$n/comments?per_page=100" \

> --jq ".[] | select(.user.login == \"$BOT_LOGIN\" and .created_at > \"<window-start>\") | \"=== PR #$n inline-comment \(.id) ===\n\(.body)\n\"" \

> >> "$TMPDIR/bot-output/all.txt"

> done

> grep -nF '${' "$TMPDIR/bot-output/all.txt" # literal ${...} interpolation failure

> grep -nP '\\!' "$TMPDIR/bot-output/all.txt" # backslash-bang corruption

> grep -nP '\\`' "$TMPDIR/bot-output/all.txt" # backslash-backtick corruption

> grep -nE 'blob/main/.*#L[0-9]' "$TMPDIR/bot-output/all.txt" # un-pinned line links

> grep -nF 'anthropics/' "$TMPDIR/bot-output/all.txt" # wrong-owner URL

> ```

>

> Cover all four bot-output surfaces: issue comments, issue bodies, PR bodies, and reviews/inline review comments. Comments-only scans miss corruption that ships in a survey-issue or PR body.

>

> **Report format** — return a structured summary:

> ```

> ## Runs with no bot output (skipped)

> - <run-id>: <workflow> — <reason> (e.g., "no artifacts", "notification no-op")

>

> ## Runs with accepted output

> - <run-id>: <workflow> on PR #N — bot reviewed, PR merged by <login>

>

> ## Runs with bot-only threads (no human signal)

> - <run-id>: <workflow> on PR #N — participants: [<login>, ...]

>

> ## Runs with concerning output

> - <run-id>: <workflow> on PR #N — <signal> by <login> (e.g., "human posted CHANGES_REQUESTED")

>

> ## Sanity check

> <note if zero bot activity found across all runs — may indicate systemic failure>

> ```

A report of little or no bot output is only usable if it carries the repo-wide sweep counts — without them, run the sweep block yourself before believing it, because a silent window and a broken per-run walk produce the same summary. Absence is not a finding on its own either: don't reason from it toward a conclusion the sweep would contradict.

Review the survey's summary, and verify any actor attribution before it enters a finding — a survey that credits the bot's own reply to a human turns a self-conversation into a false all-clear. Route on the buckets: if concerning outcomes exist, continue to Step 3. Otherwise judge the bot-only threads on their content — a self-review chain that went wrong is a finding even with no human in it, and one that read fine is not — and if nothing there concerns you and there are no sanity-check flags, skip to Step 6 (summary).

## Step 3: Investigate concerning outcomes

For runs with negative outcome signals (or suspicious lack of output), download and inspect the specific session logs, in a subagent as above.

**Also escalate whenever a finding will name *which run* produced an output.** A timestamp falling inside a run's start/end span does not attribute the output to it: several runs are live at once — a long scheduled run that opened the PR, the event-triggered handle answering a review on it, a racing sibling — and any of them can post. Attributing by wall-clock inclusion credits the wrong run, and the run ID then ships in a public comment. Confirm from the posting run's own log before a run ID enters a finding.

Grep for the write *shape*, not the endpoint — `/comments` and `/reviews` are the GET paths too, and reading the thread is the first thing nearly every run does. Don't truncate the tool input: a `git push` or `gh issue comment` often sits hundreds of characters into a longer command. Scan every `*.jsonl` under the run directory, since a subagent posts from its own `subagents/agent-*.jsonl`.

```bash

WRITES='gh (pr|issue) (comment|review|create|edit)|--method (POST|PATCH|PUT)|-X (POST|PATCH|PUT)|comments/[0-9]+/replies|git push|resolveReviewThread'

# Claude logs

for f in $(find "$TMPDIR/session-logs/<run-id>" -name '*.jsonl'); do

jq -r 'select(.type == "assistant") | .message.content[]? | select(.type == "tool_use") | "\(.name): \(.input | tostring)"' "$f"

done | grep -E "$WRITES"

# Codex logs — same idea against the rollout schema (/install-tend:debug-tend-run)

for f in $(find "$TMPDIR/session-logs/<run-id>" -name '*.jsonl'); do

jq -r 'select(.payload.type == "custom_tool_call") | "\(.payload.name): \(.payload.input)"' "$f"

done | grep -E "$WRITES"

```

The run whose log contains the posting call is the author. Presence is the strong signal; before concluding a run posted *nothing*, confirm you ran the variant matching the artifact you downloaded — the wrong one returns empty regardless of what the run did — and that the write you're chasing has a shape `WRITES` covers.

Brief the investigation with a prompt like:

> Investigate session logs for run <run-id> on `$ARGUMENTS`.

>

> Download: `gh run download <run-id> -R $ARGUMENTS --pattern 'claude-session-logs*' --pattern 'codex-session-logs*' --dir "$TMPDIR/session-logs/<run-id>/"` (both patterns are passed because the artifact prefix depends on the target repo's harness — Claude uploads `claude-session-logs*`, Codex uploads `codex-session-logs*`)

>

> The concerning outcome was: <signal from Step 2>.

>

> If your answer will name **which run produced an output**, say which output and confirm it from the log before answering — run <paste the `WRITES` recipe above> over every `*.jsonl` under the download directory, and quote the matching tool call verbatim rather than summarising it. Don't use the truncated query below for this: the write often sits hundreds of characters into a longer command.

>

> Load `/install-tend:debug-tend-run` and use the parsing reference that

> matches the downloaded artifact's harness.

>

> Focus narrowly: what decision did the bot make that led to this bad outcome? Trace the decision

> chain in the JSONL for the specific problematic action. Don't parse the entire session.

> CI polling (sleep loops checking `gh pr checks`) in session logs is expected bot behavior — do

> not flag it.

>

> Report: what the bot decided, what evidence it used, and what went wrong.

Evaluate the diagnosis against the repo-specific instructions from Step 1. Determine whether the failure is structural (same conditions always produce this failure) or stochastic (probabilistic model behavior that might not recur).

## Step 4: Deduplicate

```bash

```

A merged fix stays dormant in the target repo until the next release tags, so observing the bug there is not evidence the fix is missing — filing it again is churn on something already landed.

## Step 5: Act on findings

**Prefer PRs over issues.** A PR with a clear description is immediately actionable.

- **PR** (default): Branch `review-reviewers/$GITHUB_RUN_ID-<target-repo-name>-<topic-slug>`, fix, commit, push, create with label `claude-behavior`. `$GITHUB_RUN_ID` alone is not a unique branch name: every matrix leg of a run carries the same one, and a single leg may open more than one PR (see the limit below). The target's repo name (the part after the `/`) keeps two legs from racing the same ref; the topic slug keeps one leg's own PRs from doing the same. Write the description for a maintainer deciding whether the current change fixes the general behavior gap, following **Reader-facing prose** in `/tend-ci-runner:run-tend`. The evidence gist already carries the run history, outcome evidence, and gate assessment; link it and include only what the reader needs to understand this change. Don't also create a separate issue.

- **Issue** (fallback): Only for problems too large or ambiguous to fix directly. Include run ID, outcome evidence, root cause analysis.

Group multiple findings by broad theme. **Limit to a couple of PRs per run** — if you have more findings, pick the highest-confidence ones and record the rest in the evidence gist.

PR/issue bodies should link to the evidence gist (`gist_url`) so reviewers can see the accumulated history behind the finding.

**Do not poll CI** after creating a PR. The `tend-review` and `tend-ci-fix` workflows handle PRs independently. Exit after pushing and creating the PR.

## Step 6: Summary

Report results in the conversation log and save a markdown summary to `$GITHUB_STEP_SUMMARY` (a later workflow step copies this into the GitHub Actions step summary). Include the `gist_url` at the top so maintainers viewing the run page can click through to the full evidence log:

```bash

# Author $GITHUB_STEP_SUMMARY, starting:

#

# ## Review-reviewers summary

#

# Evidence: <gist_url>

#

# ...

```

If no problems found (or none passed the gates), report "all clear" with: runs analyzed, outcomes checked, brief quality assessment, and a link to the evidence gist for any below-threshold findings recorded this run.