review-pr · diff

v1.0.0 to v3.0.0

198 added, 140 removed. Audit A to A.

What this update changes in the checks

No check changed: every fixed check gives the same answer before and after this update.

A tripped check is a label, not a verdict: read the matched text and the lines below. This week's updates that newly trip a check.

---
name: review-pr
description: >-
- Forensic, timeline-first pull request review that reconstructs the commit
- narrative before forming opinions. Use when the user says "review PR",
- "review this PR", "look at PR #N", "what happened in this PR", "why did
- they change X", "is this PR good", "review the diff", "check this branch",
- or any variation of wanting to understand, evaluate, or critique a pull
- request. Also use when asked to compare branches, analyze a contributor's
- changes, or assess whether changes were necessary.
+ Forensic, stage-aware pull request review. Reconstructs the commit narrative,
+ tests architectural premises against project reality, detects over-engineering,
+ and produces a ready-to-post review comment. Use when the user says "review PR",
+ "review this PR", "review PR #N", "is this PR ready to merge", "review the
+ diff", or any variation of wanting to critically evaluate a pull request for
+ merge readiness. Do NOT use for branch summaries or "what does this PR do"
+ questions — those are explanation tasks, not reviews.
license: MIT
metadata:
author: jcottam
- version: "1.0.0"
+ version: "3.0.0"
---
# Review PR
- A forensic, timeline-first approach to pull request review. The core insight: most bad reviews happen because the reviewer looks at the final diff in isolation and either misses the *why* behind changes or doesn't notice that intermediate churn netted to zero.
+ Forensic, stage-aware pull request review. Reconstructs the commit narrative,
+ tests the PR's architectural premises against project reality, and produces a
+ review comment the user can post directly.
- Reconstruct the narrative before forming opinions.
+ Two core insights:
+ 1. Most bad reviews happen because the reviewer looks at the final diff in
+ isolation and misses the *why*.
+ 2. Most over-engineered PRs happen because the contributor builds for imagined
+ future requirements instead of the current problem.
## Phase 1 — Orientation
Before reading any code, establish context.
- ### 1. Identify the hosting platform
-
- Determine whether the repo uses GitHub, Azure DevOps, GitLab, or another host. Check git remotes:
+ ### 1. Get PR metadata
```bash
git remote -v
```
- Use the appropriate tooling:
- - **GitHub** → resolve the PR and branches first:
+ **GitHub (primary path):**
```bash
gh pr view <N> --json baseRefName,headRefName,title,body,commits,files,statusCheckRollup,reviews
```
- Use `baseRefName` as `<base-branch>` and `headRefName` as `<pr-branch>` for all subsequent commands. Do not guess the base branch.
+ **Fallback (no `gh` CLI or non-GitHub remote):**
- - **Azure DevOps** → `az repos pr show --id <N>` or git log with branch inspection
- - **GitLab** → `glab mr view <N>`
- - **No CLI access** → fall back to `git log` on the branch
+ ```bash
+ git log --oneline <base-branch>..<pr-branch>
+ git diff --stat <base-branch>...<pr-branch>
+ ```
- ### 2. Read the PR description and commit messages
+ Use the resolved base/head branch names for all subsequent commands.
- The PR description and commit messages are the contributor's stated intent — hypotheses you test against the code. Extract what the PR *claims*; do not treat claims as established fact until the net diff confirms them.
+ ### 2. Read project conventions
+ Before forming opinions, read the project's own rules:
+ - `AGENTS.md` — architecture, boundaries, "always do / never do" lists
+ - `.cursor/rules/` — workspace rules that apply to all changes
+ - `CONTRIBUTING.md`, `.github/PULL_REQUEST_TEMPLATE.md` if present
+
+ If none of these exist, infer conventions from the existing codebase: look at
+ 2–3 files in the same directory as the PR's primary changes. Match naming,
+ error handling patterns, test structure, and abstraction level already
+ established.
+
+ These are the standards the PR should be measured against, not generic best
+ practices.
+
+ ### 3. Read the PR description and commit messages
+
```bash
git log <base-branch>..<pr-branch> --reverse --format="%h %s%n%n%b"
```
- Extract:
- - What the PR claims to do (from title/description)
- - How many commits are on the branch
- - Whether the commit messages tell a story (tried X → fixed Y → finalized Z)
-
- Form your own judgment of scope and risk **before** reading existing review comments or the user's stated opinion of the PR.
+ Extract what the PR *claims* to do. Do not treat claims as fact until the
+ diff confirms them. Form your own judgment of scope and risk before reading
+ existing review comments or the user's opinion.
- ### 3. Check CI status
+ ### 4. Check CI status
- Before deep code review, inspect `statusCheckRollup` (GitHub) or equivalent CI status. If checks are failing, lead with that — a correct-looking diff that breaks the build is still a blocker.
+ If checks are failing, lead with that.
- ### 4. Get the shape
+ ### 5. Get the shape
```bash
git diff --stat <base-branch>...<pr-branch>
```
- Note which files were touched and their rough magnitude. Flag anything unexpected:
- - Utility/shared files touched by a "feature" PR
- - Large line counts in files unrelated to the stated goal
- - Test files present or absent
+ Flag anything unexpected: utility files touched by a feature PR, large line
+ counts unrelated to the stated goal, test files present or absent.
## Phase 2 — Timeline Reconstruction
- This is the most valuable and most frequently skipped step.
-
- ### Walk commits in order
-
```bash
git log <base-branch>..<pr-branch> --reverse --oneline
```
- For each commit, read the diff:
-
- ```bash
- git show <sha> --stat
- git show <sha> -- <file-of-interest>
- ```
+ ### Scaling strategy
- Build a mental timeline:
- - What was attempted first?
- - Were there course corrections?
- - Did the contributor fix their own mistakes within the branch?
+ | PR size | Approach |
+ |---------|----------|
+ | **≤15 commits, ≤30 files** | Full reconstruction — read every commit's `--stat`, inspect files of interest |
+ | **16–40 commits or 31–80 files** | Sample — read first commit, last commit, and any commit whose message signals a pivot ("revert", "actually", "try different approach"). Focus on the 5 highest-churn files. |
+ | **>40 commits or >80 files** | Shape-only — read `--stat` for all commits but only inspect the 3–5 files with the largest net diff. Note in findings that full reconstruction was not feasible. |
- ### Identify reversals and churn
+ ### Reconstruction
- Look for patterns where commit N adds something and commit N+M removes it. This is normal healthy engineering — it means the contributor showed their work. Do not penalize them for it.
+ For each commit in scope, read `git show <sha> --stat` and inspect files of
+ interest.
- Flag it only if the churn is *not* cleaned up (i.e., intermediate complexity still exists in the final state).
+ Build a timeline: what was attempted first, were there course corrections,
+ did the contributor fix their own mistakes within the branch?
- ### Compare net diff to merge base
+ **Reversals and churn** — commit N adds something, commit N+M removes it.
+ Normal and healthy. Flag only if intermediate complexity survives in the final
+ state.
- The net diff is what actually lands on the target branch:
+ **Net diff is ground truth:**
```bash
git diff <base-branch>...<pr-branch>
```
- This is the ground truth. Individual commit diffs are useful for understanding *why*, but the net diff is what matters for the review verdict.
+ Individual commits explain *why*; the net diff is what lands.
## Phase 3 — Categorize Changes
- For each file touched in the net diff, classify the change:
+ For each file in the net diff:
- | Category | Definition | Review posture |
+ | Category | Definition | Review actions |
|----------|-----------|----------------|
- | **Primary** | Directly serves the PR's stated goal | Full scrutiny on correctness |
- | **Supporting** | Necessary plumbing to enable primary changes | Check it's minimal, not over-engineered |
- | **Opportunistic fix** | A real bug fix discovered along the way | Verify the bug was real; consider if it should be a separate PR |
- | **Scope creep** | Unrelated improvement or refactor | Flag for discussion, don't block |
- | **Churn** | Added and then reverted within the same PR | Confirm it truly nets to zero in the final diff |
+ | **Primary** | Directly serves the PR's stated goal | Read full file (Phase 4). Check correctness, edge cases, test coverage. Evaluate naming, error paths, and interaction with existing code. |
+ | **Supporting** | Necessary plumbing for primary changes | Verify it's the minimum change needed. Check: could the primary change work without this? Is the API surface minimal? |
+ | **Opportunistic fix** | Real bug fix discovered along the way | Confirm the bug exists on main. Check the fix is correct. Flag for separate PR if it complicates revert. |
+ | **Scope creep** | Unrelated improvement or refactor | Note in findings. Don't block merge unless it creates risk. Suggest splitting. |
+ | **Churn** | Added and reverted within the same PR | Run `git diff <base>...<head> -- <file>` to confirm net-zero. If not net-zero, reclassify. |
- When a change touches a **shared utility** (libraries, helpers, common components), apply extra scrutiny:
- - Is the API change justified by more than one caller?
- - Could the same result be achieved without modifying the shared surface?
- - Are existing callers still correct?
+ Shared utility changes get extra scrutiny: is the API change justified by more
+ than one caller? Could the same result be achieved without modifying the shared
+ surface?
- ## Phase 4 — Evaluate
+ ## Phase 4 — Read the Current State
- Phase 4 is a draft assessment. Phase 5 may surface issues missed here; Phase 6 must reflect the updated picture.
+ For files categorized as **Primary**, read the full file on the PR branch:
- ### Review discipline
+ ```bash
+ git show <pr-branch>:<path/to/file>
+ ```
- - **Test the PR's premise, don't validate it.** Check whether the net diff actually proves what the description claims before evaluating implementation details.
- - **Seek disconfirming evidence.** Before approving, actively look for reasons the change could fail (missing edge cases, broken callers, rollback risk). If you find none, say so explicitly in Phase 6.
- - **Confidence tags on findings.** Tag each substantive concern or approval reason as `[high confidence]`, `[moderate confidence]`, `[low confidence]`, or `[unknown]`. Example: "Race on concurrent writes [high confidence]" vs "May increase memory under load [moderate confidence — no benchmark run]."
- - **Independent assessment.** Do not anchor on the PR author's framing, existing review comments, or the user's opinion until you have formed your own view from the diff and timeline.
+ Diffs show what changed but hide whether the result is coherent. Check for
+ leftover artifacts, naming consistency, dead imports. Understanding the full
+ file prevents forming architectural opinions from incomplete context — a diff
+ that looks over-engineered may match patterns already established in the file.
- ### Primary changes
+ For large files (>500 lines), read at minimum the 50 lines surrounding each
+ change hunk plus the file's imports/exports.
- - Does the implementation match the stated intent?
- - Are schemas, types, and contracts correct?
- - Are edge cases handled?
- - Is there test coverage for new behavior?
+ ## Phase 5 — Evaluate Proportionality
- ### Supporting changes
+ This is where most reviewers stop at "is the code correct?" and miss "is the
+ code necessary?"
- - Is this the minimum plumbing needed?
- - Could the primary goal be achieved without this supporting change?
- - Is the abstraction level appropriate (not over-engineered for a single use case)?
+ ### Over-engineering detection
- ### Opportunistic fixes
+ For each piece of infrastructure the PR introduces, ask:
- - Was the original bug actually real? (Check git blame, production behavior, existing tests)
- - Is the fix correct?
- - Should it have been a separate PR for cleaner history?
+ 1. **What problem does this solve right now?** Not "could solve someday" —
+ right now, for current users at current scale.
+ 2. **Is there evidence the problem exists?** Usage data, incident reports,
+ user complaints, or is it anticipatory?
+ 3. **What's the simplest thing that could work?** If the answer is 50 lines
+ and the PR ships 500, the 450-line delta needs justification.
+ 4. **Does this match the project's stage?** Early-stage projects should
+ optimize for speed of iteration, not theoretical scalability.
- ### Scope creep
+ Common over-engineering patterns:
+ - **Versioned internal APIs** (`/v1/`, `schemaVersion`) in monorepos that
+ deploy atomically
+ - **Multi-tier auth/rate-limiting** for traffic patterns that don't exist yet
+ - **Custom observability layers** when the stack already has tracing
+ - **Microservice boundaries** (self-calling HTTP, message dispatch) inside a
+ monolith
+ - **Caching infrastructure** for access patterns with no repeated reads
+ - **Abstract interfaces** with a single implementation
+ - **"Kept for future callers"** exports that nobody calls
- - Does it make the PR harder to review or revert?
- - Is it entangled with the primary changes or cleanly separable?
- - Note it but don't block unless it introduces risk.
+ When flagging over-engineering, always steelman first: state the strongest case
+ for the infrastructure, then explain why the current reality doesn't justify
+ it.
- ### Churn
+ ### Stage-aware evaluation
- - Confirm the net diff shows zero residue from the intermediate state.
- - If residue exists (dead parameters, unused imports, vestigial config), flag it.
+ | Project stage | Bias toward | Accept | Push back on |
+ |---------------|------------|--------|--------------|
+ | **Early build** | Simplicity, speed of iteration | Thin wrappers, direct calls, minimal abstraction | Infra for imagined scale, premature versioning |
+ | **Growth** | Reliability, observability | Caching, rate limiting, structured errors | Over-abstraction, speculative microservice splits |
+ | **Scale** | Performance, resilience | Multi-tier systems, circuit breakers, versioned APIs | Unnecessary rewrites of working code |
- ## Phase 5 — Read the current state
+ Read `AGENTS.md` and the repo structure to assess stage. If unclear, ask the
+ user.
- For any file with significant changes, read the **current state** of the file (not just the diff). Diffs show what changed but hide whether the result is coherent. A clean diff can still produce incoherent code.
+ ### Architecture challenge
- ```bash
- git show <pr-branch>:<path/to/file>
- ```
+ Test the PR's architectural premises:
+ - Does the deployment model support the abstraction? (Microservice patterns in
+ a monolith? Versioned APIs with atomic deploys?)
+ - Does the PR introduce infrastructure that duplicates what the stack already
+ provides?
+ - Does a new constraint (e.g., requiring a pre-resolved object instead of a
+ string) have downstream consequences for other callers?
+ - If the PR changes a workflow or pipeline, what breaks for existing
+ consumers?
- Check:
- - Does the file read well as a whole?
- - Are there leftover artifacts from the PR's development process?
- - Is naming consistent with the rest of the codebase?
+ ### Standard evaluation
- If Phase 5 reveals issues missed in Phase 4, Phase 6 must reflect the updated assessment.
+ Also evaluate:
+ - **Correctness** — Does the implementation match the stated intent? Edge
+ cases? Test coverage?
+ - **Supporting changes** — Minimum plumbing needed?
+ - **Scope creep** — Entangled with primary changes or cleanly separable?
+ - **Project conventions** — Does it follow AGENTS.md boundaries? Are shared
+ files (AGENTS.md, workflow-steps, etc.) updated when required?
- ## Phase 6 — Synthesize
+ Tag substantive concerns with confidence levels: `[high confidence]`,
+ `[moderate confidence]`, `[low confidence]`.
- ### Tone
+ ### Fast path — clean PRs
- - Start with findings, not praise. No "great PR", "nice work", or "solid approach" openers.
- - **Lead with the strongest concern** when the verdict is Request changes or Needs discussion. Steelman the contributor's approach first, then explain why it still fails the bar.
- - **Separate code from contributor.** Critique the diff; do not attribute intent or competence.
- - **Show reasoning for the verdict.** Cite specific files, lines, or commit SHAs — not vague impressions.
+ If after Phases 1–5 no substantive concerns emerge (correct implementation,
+ proportional scope, follows conventions, has tests), skip to Phase 7 with a
+ short approval. Do not manufacture findings to justify the review's existence.
+ A clean PR gets:
- Produce a structured summary with these sections:
+ - Confirmation of what it does (one sentence)
+ - Verdict: **Approve**
+ - No comment to post (or a one-line "LGTM — [what you verified]")
- ### 1. What the PR does
- One to two sentences. What lands on the target branch?
+ ## Phase 6 — Collaborative Refinement
- ### 2. Commit timeline
- Ordered narrative of what happened on the branch. Include course corrections — they show the contributor's reasoning process.
+ Present findings to the user, then **expect corrections**. The reviewer often
+ lacks context the user has:
- ### 3. Net impact
- What actually changes after churn is removed. Distinguish "files touched" from "meaningful changes."
+ - The branch may have forked before certain features existed on the target
+ (apparent "removals" are merge gaps, not regressions)
+ - A decision that looks wrong from the diff may have been discussed and agreed
+ offline
+ - The contributor may have constraints you can't see from the code
- ### 4. Change categories
- Map each significant file change to its category (primary, supporting, opportunistic fix, scope creep, churn).
+ When the user corrects an assumption, update your assessment immediately. Drop
+ the finding — don't hedge or footnote it.
- ### 5. Concerns
- Organized by severity. Tag blockers and substantive concerns with confidence levels; skip tags on pure style nits unless the nit reflects a real convention violation.
+ This is iterative. The first pass surfaces raw findings; dialogue refines them
+ into the actual review.
- - **Correctness** — bugs, logic errors, missing edge cases
- - **Scope** — changes that don't belong in this PR
- - **Complexity** — over-engineering, unnecessary abstractions
- - **Risk** — shared utility changes, migration concerns, rollback difficulty
+ ## Phase 7 — Produce the PR Comment
- ### 6. Verdict
- One of:
- - **Approve** — changes are correct and well-scoped
- - **Approve with nits** — minor suggestions that don't block merge
- - **Request changes** — specific issues that need addressing before merge
- - **Needs discussion** — architectural or scope questions that need team input
+ After collaborative refinement, produce a comment the user can post directly.
- Always include reasoning for the verdict. Add these sub-bullets **proportionally**:
+ ### Format
- - **Steelman:** strongest case for merging as-is. Required for Request changes / Needs discussion. Optional one-liner for Approve with nits. Skip for trivial Approve (typo fixes, single-line changes).
- - **Disconfirming evidence checked:** what you looked for and what you found. Required before any Approve verdict on non-trivial PRs; one sentence is enough for small changes.
+ Each concern is one block: location, problem, fix. No preamble.
+ ```
+ **1. [File or area]** — [Problem in one sentence]. [What to do about it.]
+
+ **2. [File or area]** — [Problem]. [Fix.]
+
+ **Housekeeping:**
+ - [Item]
+ - [Item]
+
+ **What should land:** [Subset to merge now. What to split out.]
+ ```
+
+ ### Tone
+
+ - No openers. No "overall" paragraph. No "great work" buffer.
+ - Lead with the strongest concern.
+ - Critique the diff, not the contributor.
+ - If asking the contributor to reconsider a tradeoff, frame as a question.
+
+ ### Verdict (internal, not in the posted comment)
+
+ One of:
+ - **Approve** — correct and well-scoped
+ - **Approve with nits** — minor suggestions, don't block merge
+ - **Request changes** — specific issues that must be fixed before merge
+ - **Split** — correct work but too large or entangled; specify which subset to merge now and what to extract into follow-up PRs
+ - **Needs discussion** — architectural questions that need team input before proceeding
+
## Principles
- - **Never judge a diff without reading commit messages first.** The diff is the "what"; the message is the "why." You need both.
- - **Always check the net diff, not just individual commits.** A 3-commit PR can have 500 lines of churn that nets to 50 lines of real change.
- - **Distinguish "code in the PR" from "code landing on the target branch."** Follow-up fixups within the same branch are self-correcting, not problems.
- - **Don't penalize showing work.** A commit history of "tried X → realized Y was better → switched to Y" is healthy engineering.
- - **Flag shared-utility changes with extra scrutiny.** When a PR touches code that other features depend on, verify the change is justified and existing callers still work.
- - **Assume good intent.** A wrong assumption that was later corrected is not incompetence — it's iterative development.
- - **Separate the review from the reviewer.** Comment on the code, not the person.
- - **Accuracy over agreeability.** A review that misses a real bug to avoid friction is a failed review. Deliver blockers plainly.
+ - Never judge a diff without reading commit messages first.
+ - Always check the net diff, not just individual commits.
+ - Don't penalize showing work — course corrections within a branch are healthy.
+ - Infrastructure should be justified by current reality, not anticipated need.
+ - Assume good intent. Challenge the code, not the person.
+ - Accuracy over agreeability. A review that misses a real problem to avoid
+ friction is a failed review.