--- name: copilotloop description: > Iteratively satisfy GitHub Copilot's code review on a GitHub PR, but treat every Copilot comment SKEPTICALLY: verify each finding against the real code first, fix ONLY the genuinely-correct ones, and rebut + resolve false positives WITHOUT changing correct code. Repeat until no unresolved Copilot comments remain. Use when the user says "satisfy Copilot", "clear the Copilot review", "fix Copilot comments", "Copilot loop", or wants to iterate on Copilot PR feedback. compatibility: GitHub only (Copilot code review is a GitHub feature). Requires git + jq + gh (GitHub CLI) authenticated, Copilot code review enabled, and permission to request reviewers on the repository. metadata: version: "1.1" allowed-tools: Bash(gh:*) Bash(git:*) --- # Copilotloop Drive a GitHub PR until Copilot has no unresolved review comments, **but do not cargo-cult its suggestions.** Every comment is a *claim to verify*, not an instruction to obey. A wrong suggestion applied is worse than the comment itself. ## How Copilot differs from other review bots - **No check-run, no score.** Copilot posts an overview comment and a PR review with inline review comments. Detection is by polling the reviews endpoint and paginated review threads. "Satisfied" means zero unresolved Copilot threads and zero unaddressed body findings after each finding is fixed or rebutted with a recorded reply. - **Request through the reviewers API.** Copilot does not use a reliable slash-command trigger. Request `copilot-pull-request-reviewer[bot]` through the pull request reviewers endpoint. A repository may also be configured to review new pushes automatically. - **Copilot may repeat comments.** A re-review can repeat comments that were resolved or downvoted, so match findings to the current head and prior replies before reopening work, including body findings that have no inline thread. - **Copilot is concise and can reason from the diff alone.** Its characteristic failure is confidently asserting a bug based only on the changed hunk, without the surrounding file or the call sites. That makes "read the whole file before believing it" the single highest-value check. ## Not for - GitLab / Perforce. For other review bots this catalog ships `codexloop`, `geminiloop`, `coderabbitloop`, and `qodoloop`; for CI failures rather than review comments, use `ci-fix-loop`. ## 0. Resolve the Copilot bot login (do this first) The bot login is normally stable, but confirm it from the PR rather than assuming it: ```bash gh api repos/{owner}/{repo}/pulls//reviews --paginate --jq '.[].user.login' | sort -u gh api repos/{owner}/{repo}/pulls//comments --paginate --jq '.[].user.login' | sort -u ``` Use `copilot-pull-request-reviewer[bot]` as `BOT`. Confirm it appears after the first requested review; if GitHub uses a different login for the installation, use the exact Copilot-looking login returned by the probe rather than guessing. **Finding nothing here is NOT a stop condition.** On a PR Copilot has never reviewed there is no bot login to find yet. Fall through to step 2A, request the reviewer, and re-run this probe once a review lands. Only conclude Copilot is unavailable after the bounded wait expires with no review. ## 1. Identify the PR ```bash gh pr view --json number,headRefName,headRefOid -q '{number,branch:.headRefName,head:.headRefOid}' ``` Switch to the PR branch if not already on it. Capture `OWNER`/`REPO` (`gh repo view --json owner,name`). ## 2. The loop (max 5 iterations) Keep an explicit iteration counter and stop at 5: the cap is a real bound to enforce, not a figure of speech. Each pass through A–G is one iteration; on hitting the cap, go straight to the report and list what is still unresolved rather than starting a sixth. ### A. Ensure a fresh Copilot review on the current head ```bash HEAD_SHA=$(gh pr view --json headRefOid -q .headRefOid) # Only trigger if no Copilot review exists for this exact SHA: HAVE=$(gh api repos/{owner}/{repo}/pulls//reviews --paginate | jq -s --arg bot "$BOT" --arg head "$HEAD_SHA" \ '[.[][] | select(.user.login==$bot and .commit_id==$head)] | length') if [ "$HAVE" = "0" ]; then gh api --method POST repos/{owner}/{repo}/pulls//requested_reviewers \ -f 'reviewers[]=copilot-pull-request-reviewer[bot]' fi ``` Poll for the review of THIS head to land. No check-run exists, so poll the reviews endpoint, and poll it on a **deadline**, never `while true`: a review that never arrives must end the skill with an honest timeout, not hang it. ```bash # 10-minute deadline, one retry, then give up. DEADLINE/RETRIES are the # enforcement of the bounds this skill claims: do not drop them. wait_for_review() { # $1 = attempt label local deadline=$(( SECONDS + 600 )) while [ "$SECONDS" -lt "$deadline" ]; do R=$(gh api repos/{owner}/{repo}/pulls//reviews --paginate | jq -s --arg bot "$BOT" --arg head "$HEAD_SHA" \ '[.[][] | select(.user.login==$bot and .commit_id==$head)] | sort_by(.submitted_at, .id) | last') if [ -n "$R" ] && [ "$R" != "null" ]; then return 0; fi echo "waiting for Copilot review of $HEAD_SHA ($1)..."; sleep 15 done return 1 } if ! wait_for_review "first wait"; then echo "no Copilot review after 10m, retrying once" # say the retry out loud gh api --method POST repos/{owner}/{repo}/pulls//requested_reviewers \ -f 'reviewers[]=copilot-pull-request-reviewer[bot]' if ! wait_for_review "after retry"; then echo "Copilot did not review $HEAD_SHA after a retry; stopping and reporting." exit 1 # honest timeout, never a success claim fi fi ``` Copilot usually reviews quickly, but large diffs can take longer; the deadline is bounded. Report the retry in the final summary; two silent timeouts are the failure mode this guard exists to prevent. ### B. Fetch the findings - **Summary**: read the Copilot overview comment and review `.body`, including all matching reviews for the current head. Extract and classify actionable claims even when they have no inline thread; an approval assessment or lack of inline comments does not make those findings complete. Track each body finding across iterations by source review/comment ID and item, with its classification, fix commit or evidence-based rebuttal, PR-level reply URL, and completion status. Link duplicates to the same finding so they are counted once; retain pending findings after a push. - **Unresolved inline comments** on the current head: ```bash gh api repos/{owner}/{repo}/pulls//comments --paginate \ --jq ".[] | select(.user.login==\"$BOT\") | {id, path, line, body}" ``` Also pull the review threads + their resolved state via GraphQL (see step F) so you only act on unresolved ones. Mark each unresolved inline comment as seen, so the bot and a watching human can tell it is being handled. GitHub's reaction set is fixed; `eyes` is the only "looking at it" value: ```bash gh api --method POST repos/{owner}/{repo}/pulls/comments//reactions -f content=eyes ``` Body findings (no inline thread) have no comment to react to; track them in step B's register and give each a PR-level reply in step F instead. ### C. Critically evaluate EACH finding (the core of this skill) For every inline or body finding, **verify the claim against the actual code and repo conventions before touching anything.** Read the whole file, not just the diff hunk Copilot saw, plus the types and the call sites. Then classify: 1. **CORRECT + actionable**: the finding is real and the fix improves the code. → fix it (step D). 2. **FALSE POSITIVE / technically wrong**: the claim doesn't hold. → do **NOT** change code; write a specific, evidence-based reply (cite the exact code/line/behavior that disproves it), then resolve its thread or record completion for a body finding (step F). 3. **Valid but out-of-scope / stylistic nit** that conflicts with repo convention or the PR's intent → briefly decline with evidence of that convention or scope, then resolve its thread or record completion for a body finding (step F). Do not expand the PR's scope to satisfy a nit. **Hard rules:** - **Never modify correct code just to silence Copilot.** Prefer a reasoned rebuttal. - When uncertain whether a claim holds, **investigate** (read more code, run the type-checker / tests) rather than assume Copilot is right. Default to skepticism. - If a suggested change would break other call sites, alter public behavior, or contradict a verified repo convention, it is a category-2 rebuttal, not a fix. - Never fabricate identifiers to satisfy a comment (e.g. a Linear/ticket prefix). If Copilot asks for a ticket reference and none exists, say so; do not invent one. **Copilot's common failure modes to watch for (default these to category 2):** - Diff-local reasoning: asserts a bug that the unchanged surrounding code already handles. - "This can be null/undefined here" where the type or an earlier guard already rules it out. - Invented race conditions or error paths with no actual trigger. - Suggestions that compile-break or break other callers. - Security/perf warnings with no exploit path or measurable cost. - Restating library/framework semantics incorrectly. - Style demands that contradict the repo's existing, consistent pattern. ### D. Apply fixes: category 1 only Make the minimal correct change. Re-run the local gate if the repo has one (typecheck/tests) before moving on. ### E. Commit and push FIRST, before resolving anything Order matters. A resolved thread is a claim that the fix is on the branch, so the push has to succeed before the claim is made: otherwise a failed commit or push leaves the PR unfixed with the finding marked resolved, and nobody looks at it again. If step D changed code: ```bash # Stage ONLY the files your fixes touched: never `git add -A`, which sweeps up # unrelated work and untracked secrets sitting in the worktree. git status --short # look before you stage git add [...] # the files named in the findings you fixed git commit -m "address Copilot review feedback (copilotloop iteration N)" git push ``` Author the commit per the repo's norms (e.g. the user's identity; no AI attribution if that is the convention). Confirm the push actually landed before continuing: ```bash git rev-parse HEAD gh pr view --json headRefOid -q .headRefOid # must match ``` If they differ, stop: the fix is not on the PR, so nothing may be resolved yet. ### F. Reply to every addressed finding and resolve its inline threads Only now, with the fixes pushed, reply and resolve. Fetch unresolved threads, **following pagination**: a PR with more than 100 threads will otherwise look clean while unresolved findings sit on page two: ```bash # Loop until hasNextPage is false, passing endCursor back in as $cursor. CURSOR=null while : ; do PAGE=$(gh api graphql -F cursor="$CURSOR" -f query=' query($cursor: String) { repository(owner: "OWNER", name: "REPO") { pullRequest(number: PR_NUMBER) { reviewThreads(first: 100, after: $cursor) { pageInfo { hasNextPage endCursor } nodes { id isResolved comments(first: 1) { nodes { databaseId author { login } path body } } } } } } }') echo "$PAGE" # collect nodes from every page before deciding the PR is clean PI='.data.repository.pullRequest.reviewThreads.pageInfo' [ "$(echo "$PAGE" | jq -r "$PI.hasNextPage")" = "true" ] || break CURSOR=$(echo "$PAGE" | jq -r "$PI.endCursor") done ``` Reply on a thread's comment via `gh api repos/{owner}/{repo}/pulls//comments -f body="..." -F in_reply_to=`, then resolve: ```bash gh api graphql -f query='mutation { resolveReviewThread(input: {threadId: "THREAD_ID"}) { thread { isResolved } } }' ``` Then record the disposition of each inline finding as a reaction on Copilot's own comment: the learning signal, and a visible marker for a human scanning the PR: - 👍 (`+1`): the finding was accepted and fixed. - 👎 (`-1`): the finding was rebutted as a false positive. **This is the correction signal.** ```bash gh api --method POST repos/{owner}/{repo}/pulls/comments//reactions -f content=+1 # rebutted finding: gh api --method POST repos/{owner}/{repo}/pulls/comments//reactions -f content=-1 ``` Reply first, then react, then resolve: the reaction is not a substitute for the reply, and a resolved thread with no reply loses the reasoning. Resolve a thread only for comments authored by `$BOT` that you have fixed or rebutted: never blanket-resolve, and never resolve a human reviewer's thread. Threads you are **rebutting** need no push, so they may be replied to and resolved regardless of whether step D changed code. For each body finding, post a PR-level reply (for example, using `gh pr comment --body-file `) identifying the source review/comment and item. Record the pushed fix commit and validation, or an evidence-based rebuttal, including the reason for declining an out-of-scope finding. Record the reply URL in the tracking entry before marking it complete. One reply may cover multiple explicitly identified findings. Fixed body findings require the same successful push check as fixed threads; rebuttals need no push. If a finding also has an inline thread, that thread still requires a reply and resolution. ### G. Re-review Pushing re-triggers Copilot when automatic review of new pushes is on; otherwise request Copilot again through the reviewers endpoint. Go back to **A** with the new head SHA. If step D changed nothing (all comments were rebutted), skip the push, ensure all Copilot threads are resolved and all tracked body findings have recorded PR-level replies and are complete, then exit. ## 3. Exit conditions Stop when **any** is true: - Zero unresolved `$BOT` threads and zero pending body findings remain across iterations. Every finding was fixed or rebutted with evidence, every inline thread was replied to and resolved, and every body finding has a recorded PR-level reply. (There is no score to hit: this is "done".) - Max iterations (5) reached: report what remains. - Copilot never responded after one retry: report the timeout honestly; do not claim success. ## 4. Report ```text Copilotloop complete. PR: # Bot login: Iterations: N Findings fixed: N (inline + body findings, deduplicated) Findings rebutted: N (inline + body findings, with recorded evidence-based replies) Remaining: 0 (unresolved inline + pending body findings, deduplicated) ``` If it stopped at max iterations, list the remaining threads and body findings with your current assessment (fix-pending vs disputed) so a human can arbitrate.