github-pr-reviewer · diff
git:20261019.87f109e to git:20261027.f79208b
49 added, 517 removed. Audit A to A.
---
name: github-pr-reviewer
description: >
Create an automation that reviews GitHub pull requests when a configured
reviewer is requested or a trigger label is applied. Starts one OpenHands
review conversation per request with the pull request's exact head checked
out, and publishes the review to GitHub.
triggers:
- /pr-reviewer:setup
---
- # GitHub PR Reviewer Automation
-
- ## Agent Canvas catalog
-
- For new Agent Canvas installations, use the **GitHub code review** catalog
- entry. Its deterministic `worker.py` delegates each requested exact head to a
- stable conversation using the selected agent profile. It supports GitHub
- reviewer-request events and scheduled label scans. The manual upload flow below
- remains for existing deployments and is deprecated for new installations.
-
- Create a cron automation that watches one or more GitHub repositories for pull
- requests with a review trigger label, starts an OpenHands review conversation
- once per label event, and publishes the AI review to GitHub.
- Windows PowerShell equivalents for the setup, packaging, upload, and API-check shell snippets are in `references/windows.md`.
-
- The automation script is deterministic: PR discovery, label-event tracking,
- head eligibility, state persistence, stale-result suppression, the repository
- checkout, and its removal are all handled in Python. The LLM is invoked only for
- the review itself.
-
- Before any review conversation is created, a scheduled scan evaluates the
- current head's GitHub-required checks, read through the GraphQL `isRequired`
- signal for the pull request. That signal is the merge policy's own source of
- truth and is pull-request-scoped, so it stays correct for a stacked PR whose
- symbolic base branch carries no branch rules of its own:
-
- - Only checks GitHub reports as required decide the scheduled gate. An optional
- workflow that fails - including one that fails before creating any check run -
- does not block a head whose required checks pass.
- - A completed required run on the exact head whose conclusion is `failure`,
- `cancelled`, or `timed_out` blocks the review. Any other unrecognized
- conclusion fails closed as a block rather than silently approving.
- - A completed required run whose conclusion is `success`, `neutral`, or
- `skipped` does not block. A required `commit status` context is classified
- alongside required check runs.
- - A `queued` or `in_progress` required run on the exact head, and a required
- context that has not reported a current-head check run at all, make the run
- exit with a **waiting on checks** outcome. Unfulfilled is never approval. The
- worker does not hold a slot polling; the next scheduled scan or explicit
- review request retries.
- - When the required-check signal cannot be read or is empty, the gate falls back
- to every current-head check run and workflow run, so a red head still blocks.
- In that fallback, workflow runs are read as well as check runs, because a
- workflow can fail before creating any check run - a workflow-level error, or a
- `pull_request` run whose jobs never start. Such a run leaves a failed check
- suite with no check runs under it, so the commit's check-run rollup and
- `gh pr checks` both report success and only the workflow run reveals the red
- CI. A workflow run whose check suite already reported check runs is left to
- those runs, so a workflow is never counted twice.
- - A scheduled scan considers the trigger label **and** every open PR, including drafts,
- that still holds an outstanding `all-hands-bot` review request. That is what
- resumes a request made while CI was running: the request is keyed by its own
- `review_requested` event, so repeated scans reuse one conversation and one
- review instead of creating duplicates, and a PR whose request was answered or
- withdrawn simply drops out of `requested_reviewers`.
- - The scheduled scan also reviews open, non-draft PRs that **nobody requested**
- and that carry no trigger label, once their current head has no completed
- review by this account. That is what keeps reviewing PRs after the outstanding
- requests are exhausted. The delivery key is the stable
- `scan:{repository}:{number}:{head}`, so repeated scans reuse one conversation
- and one native review, and a changed head becomes eligible again under its new
- SHA.
- - Each scheduled scan classifies the **whole open backlog**, including
- unrequested PRs. Blocked, pending, draft, or already-reviewed heads therefore
- cannot hide an eligible head behind an inspection window. The separate
- `max_new_per_run` quota limits only newly created review conversations, not the
- number of PRs inspected; explicit requests and trigger labels retain priority
- when eligible candidates are drained.
- - A gate stop on an **unrequested** head posts no managed comment. A PR nobody
- asked about that is merely red or pending is simply skipped without starting an
- agent, which is what keeps a scan over a large backlog from posting a gate
- comment on every PR. The managed comment is the answer to an explicit request,
- so a requested or labeled head that is red or pending still gets its
- explanation.
- - Workflow runs are read as well as check runs, because a workflow can fail
- before creating any check run - a workflow-level error, or a `pull_request`
- run whose jobs never start. Such a run leaves a failed check suite with no
- check runs under it, so the commit's check-run rollup and `gh pr checks` both
- report success and only the workflow run reveals the red CI. A workflow run
- whose check suite already reported check runs is left to those runs, so a
- workflow is never counted twice.
- - Runs attributed to any other (obsolete) head SHA are ignored, so a stale
- failure cannot block the push that fixed it. This applies to workflow runs
- too.
- - Only the latest run of each logical check or workflow counts. A logical check
- is its name plus the reporting app identity, and a logical workflow is its
- name plus workflow ID. The latest is chosen by the run ID (the reliable
- creation sequence) with the start time as a tie-break, so a re-run that fixed
- a check supersedes its earlier failure on the same SHA while a newer queued or
- in-progress re-run supersedes an earlier success and makes the head wait.
- Ordering by the run ID keeps a run whose start time is still absent in its
- true creation position in both directions.
-
- An explicit `all-hands-bot` review request is the intake-policy exception: the
- caller asked for that head by name, so the request dispatches even when required
- CI is red or pending, and the gate leaves no explanatory comment. Draft, scope,
- exact-head, and delivery-deduplication safeguards still apply. A blocked or
- waiting scheduled stop consumes no trigger: once the head's required checks are
- non-blocking, the scheduled scan, or a new `all-hands-bot` review request,
- starts the normal review.
-
- The gate needs no configured list of check names. When it stops a review it
- leaves one concise explanation on the PR, identified by a hidden marker carrying
- the head SHA and gate category, so a later run for a different head updates that
- comment instead of posting another. Only a marker this reviewer account authored
- counts as its own comment; every other PR comment is untrusted and can neither
- suppress the explanation nor be edited. If the repository's own workflow already
- posted a deterministic remediation comment that names the same current-head
- checks, the gate adds nothing; a disclosure about some other check does not
- suppress it. The gate applies to scheduled label scans; an explicit
- `all-hands-bot` review request bypasses it by design.
-
- Each explicit review request - a re-applied trigger label, or a new reviewer
- request in event mode - is a fresh review. The conversation for a PR is reused,
- but its earlier turns must not be trusted as current: before deciding a verdict
- the reviewer re-fetches the mutable GitHub state (the exact head, the PR body,
- review comments and threads, review requests, the linked issues' bodies and
- labels, and the current-head Actions results) and ignores any earlier finding,
- verdict, or label/priority claim that state no longer supports. Repository
- analysis the conversation already did, such as reading `AGENTS.md`, stays
- useful and is not repeated.
-
- The waiting and blocked explanations name the retry the deployment actually has.
- A scheduled run proves a scan is configured, so it says the next scan will
- retry. An event-only run does not, so it tells the reader to remove the
- outstanding `all-hands-bot` request and request `all-hands-bot` again instead of
- promising a scan that does not exist: GitHub will not accept a second request
- while the first is still outstanding.
-
- The gate leaves only one explanation per head. When the same head keeps the same
- hidden marker but the retry wording changes -- an event-only comment on a head
- whose automation is later switched to a cron scan -- the scheduled run rewrites
- that managed comment in place, so the comment always names the retry that is
- actually deployed, and an unchanged body is left untouched.
-
- ## Bounded intake per scheduled scan
-
- A scheduled scan drains outstanding reviewer requests fairly, but starting an
- agent for every eligible pull request at once would overload the deployment, so
- the scan starts at most `MAX_NEW_PER_RUN` new review conversations, counted
- **across every configured repository** rather than per repository. The default
- is `2`, and the rendered `config.json` overrides it through the
- `max_new_per_run` key that `github-issue-to-pr` and `gitlab-issue-to-mr` already
- use.
-
- - Eligible candidates are ordered deterministically: an explicit `all-hands-bot`
- request (or a trigger label) is ordered before an unrequested PR, then by the
- oldest `all-hands-bot` `review_requested` event for a requested PR or the PR's
- own creation time (oldest first) for an unrequested one, then by repository,
- then by pull-request number, so the oldest work drains first and a later scan
- reaches the remainder.
- - The bound counts the conversations a scan **starts**. A delivery that only
- deduplicates or reports an already-running conversation reuses a runtime and
- consumes no slot, so repeated scans make progress on the backlog instead of
- re-spending the bound on work already in flight.
- - Reaching the bound never cuts the scan short. The scan still evaluates the
- exact-head checks of the remaining candidates, reconciles completed reviews,
- and runs the maintainer handoff. A candidate whose checks are pending or
- failing starts no conversation and consumes no slot, so it cannot block a later
- green candidate. Only an explicit request or a trigger label gets a waiting or
- blocked gate comment; an unrequested head that is merely red or pending is
- skipped silently.
- - A dispatch that raises is reported and does not consume a slot or abort the
- scan, so the candidates behind it are still considered.
- - The explicit `review_requested` event path and the trigger-label scan are
- unchanged: an explicit request still starts its conversation immediately, and
- only the scheduled scan's new conversations are bounded.
-
- The review prompt starts with a scope gate: using the repository's own guidance
- (its scope categories and ownership boundaries, not a list of individual PR
- numbers), the reviewer decides whether the change belongs in this repository
- and has the product/architecture direction it needs. When it does not, the review
- stops with a single `event: COMMENT` review that says whether the change should
- move repositories, close, or receive a maintainer decision, and ends with the
- `🛑 MAINTAINER DECISION REQUIRED` verdict. That outcome is **neither an approval
- nor a change request**: it does not approve or merge the PR. The completion
- handler recognizes the verdict and requests one configured maintainer through the
- same handoff used after an approval. Both paths require a successfully fetched
- same-repository closing issue labeled `priority:medium` or `priority:high`; a
- missing link or priority label, low/normal priority, unresolved reference, or PR
- reference does not authorize handoff. One qualifying issue is sufficient even
- with other low-priority or missing references. Existing manual review requests
- are unchanged. An in-scope change continues the existing review unchanged.
-
- Before approving a change to user-visible UI behavior, the prompt requires live
- evidence from a real running application when the repository's guidance demands
- it: a screenshot, screen recording, or equivalent capture of the running app
- that exercises the production-facing path. Unit tests, CSS-token or contract
- assertions, generated mockups, and reconstructed captures may support the review
- but cannot substitute for that evidence. When the required evidence is missing,
- the reviewer publishes one `event: COMMENT` review naming exactly what is
- missing and ends with `🔄 CHANGES REQUESTED`, so approval is withheld and the
- deterministic maintainer handoff does not fire. Non-UI changes keep the existing
- requirement of the real command and its observed output, with tests insufficient
- as sole proof.
-
- Every review ends with the LLM profile and model of the conversation that wrote
- it, after the verdict: `LLM profile: <profile> · Model: <model>`. The agent does
- not write this footer. The prompt gives it a hidden marker naming its own
- conversation, `<!-- openhands-review-run: <conversation-id> -->`, to copy under
- the disclosure, and the deterministic script appends the footer to the reviews
- that carry exactly that marker and no other. Login, head, and submission time
- cannot tell two automations that share a bot account apart, so a review without
- the marker, with another run's marker, or with several markers is never edited.
- The catalog worker reads the values back from the marked conversation: the
- model it runs, and the LLM profile its agent profile referenced at launch
- (`unknown` once that agent profile has been edited). It never substitutes the
- scanner's own settings, and a reused conversation keeps reporting the LLM it
- was launched with. A review is stamped once, so a later agent profile edit
- cannot rewrite its footer, and a provenance failure is logged without holding
- up the verdict or the maintainer handoff. The completion handler reads the
- verdict above the footer.
-
- The script prepares each review's workspace before the agent starts: the pull
- request's head commit is downloaded as a tarball and extracted to a directory of
- its own, which becomes the conversation's working directory. The agent is told
- not to clone, fetch, check out, or delete anything, and the script removes the
- checkout once the conversation has stopped. Nothing accumulates between runs.
-
- ---
-
- The script imports shared GitHub transport from
- `scripts/github_client.py`, installed with this skill. Include it beside
- `main.py` when packaging manually, as shown below; catalog bundles include it
- automatically.
-
- ## Prerequisites
-
- ### Required secret
-
- Verify that the following secret is set in **OpenHands Settings -> Secrets**:
-
- | Secret name | Token type | Minimum permissions |
- |---|---|---|
- | `GITHUB_PERSONAL_ACCESS_TOKEN` | Classic PAT | `repo` for private repos or `public_repo` for public repos |
- | `GITHUB_PERSONAL_ACCESS_TOKEN` | Fine-grained PAT | Contents: Read, Metadata: Read, Pull requests: **Read and Write**, Issues: Read and Write |
-
- Pull-request **write** access is required because the agent publishes a pull
- request review, not just an issue comment. The Agent Canvas catalog worker may
- also request a configured human reviewer after approval. A token with only Pull
- requests: Read will poll happily and then fail at the point of publishing or
- requesting the handoff.
-
- When several repositories are monitored, the token must cover all of them.
-
- On OpenHands Cloud and Enterprise, the Agent Canvas catalog template needs no
- saved secret when GitHub is connected under Settings -> Integrations: a secret
- saved under the configured name is used if present, otherwise the connected
- account's token, and the review is published by that account. The requested
- reviewer should be that account's login. The steps below apply to the local
- setup through this skill.
-
- Check with:
- ```bash
- curl -s https://api.github.com/user \
- -H "Authorization: Bearer $GITHUB_PERSONAL_ACCESS_TOKEN" \
- | python3 -c "import json,sys; d=json.load(sys.stdin); print(d.get('login') or d.get('message'))"
- ```
-
- If the token is missing or invalid, inform the user and stop.
-
- ---
-
- ## Setup Workflow
-
- Follow these steps in order.
-
- ### Step 1 - Verify `GITHUB_PERSONAL_ACCESS_TOKEN`
-
- Run the `curl` check above.
-
- - If absent: *"GITHUB_PERSONAL_ACCESS_TOKEN is not set. Please add it in
- OpenHands Settings -> Secrets."* Stop.
- - If the API returns `{"message": "Bad credentials"}`: tell the user the
- token is invalid and ask them to update it. Stop.
-
- ### Step 2 - Collect repositories
-
- Ask: *"Which GitHub repositories should be monitored?
- (Format: `owner/repo`, e.g. `myorg/backend`. List several separated by commas to
- review them all from one automation.)"*
-
- Validate access to **each** repository:
- ```bash
- curl -s "https://api.github.com/repos/{owner}/{repo}" \
- -H "Authorization: Bearer $GITHUB_PERSONAL_ACCESS_TOKEN" \
- | python3 -c "
- import json, sys
- d = json.load(sys.stdin)
- if 'message' in d:
- print('ERROR:', d['message'])
- else:
- print(f\"Accessible. Private: {d.get('private')}. Permissions: {d.get('permissions')}\")
- "
- ```
-
- Record every accepted repository into `REPOS = ["{owner}/{repo}", ...]`. If one
- repository fails the check, say which and ask whether to continue without it.
-
- Each repository is polled independently and keeps its own state, so pull-request
- numbers never collide between them. The trigger label, tone, and schedule are
- shared by all of them; a repository needing different settings wants its own
- automation.
-
- ### Step 3 - Collect trigger label
-
- Ask: *"Which PR label should trigger a review?
- (Press Enter for the default: `openhands-review`.)"*
-
- Record the answer as `TRIGGER_LABEL`. If the label does not exist yet, tell the
- user that GitHub will still record the event once the label is created and
- applied to a PR.
-
- The automation reviews a PR when it sees the latest matching `labeled` event for
- that label. To request another review later, remove and re-apply the label.
-
- ### Step 4 - Collect review tone
-
- Ask: *"What review tone should the reviewer use?
- 1. Thorough (default) - comprehensive coverage of correctness, security, tests, style
- 2. Concise - high-signal only, skips minor style feedback
- 3. Friendly - constructive and encouraging
- (Press Enter for Thorough, or type your choice or any custom style description)"*
-
- Map the choice to `REVIEW_TONE`:
-
- | Answer | `REVIEW_TONE` | `REVIEW_STYLE_INSTRUCTIONS` |
- |---|---|---|
- | 1 / Enter | `"thorough"` | `""` |
- | 2 | `"concise"` | `""` |
- | 3 | `"friendly"` | `""` |
- | Custom text, e.g. `strict but kind` | `"thorough"` | the custom text verbatim |
-
- ### Step 5 - Collect cron schedule
-
- Ask: *"How often should the automation poll for labeled PRs?
- (Press Enter for the default: every 5 minutes.
- Use a cron expression for a different interval, e.g. `0 * * * *` = hourly)"*
-
- Default: `*/5 * * * *`.
-
- Record as `CRON_SCHEDULE`.
-
- ### Step 6 - Generate the automation script
-
- Read `scripts/main.py` from this skill's directory. Apply exactly seven constant
- substitutions near the top of the file:
-
- > The script also reads a `config.json` shipped beside it, if there is one, over
- > these constants. That is how the catalog entry
- > (`automations/catalog/github-pr-reviewer/`) configures an unmodified copy,
- > since a declarative host cannot rewrite Python. This setup path substitutes the
- > constants and ships no `config.json`, so the two never collide.
+ # Github Pr Reviewer
- | Placeholder | Replace with |
- |---|---|
- | `REPOS = ["owner/repo"]` | `REPOS = ["{owner_repo}", ...]` - one entry per repository collected in Step 2 |
- | `TRIGGER_LABEL = "openhands-review"` | `TRIGGER_LABEL = "{trigger_label}"` |
- | `REVIEW_TONE = "thorough"` | `REVIEW_TONE = "{review_tone}"` |
- | `REVIEW_STYLE_INSTRUCTIONS = ""` | `REVIEW_STYLE_INSTRUCTIONS = "{style_instructions}"` |
- | `REPO_REVIEW_GUIDE_PATH = ".agents/skills/custom-codereview-guide.md"` | leave unchanged to auto-load a repo review guide at this path, or set to `""` to disable |
- | `MAX_NEW_PER_RUN = 2` | leave unchanged to bound a scheduled scan to two new review conversations across all repositories, or raise it if the deployment can hold more agents at once |
- | `DEFAULT_OPENHANDS_URL = "http://localhost:8000"` | leave unchanged unless the user has a preference |
+ This automation now runs from the executable automation plugin at `plugins/github-pr-reviewer`. Do not copy or package code from `skills/github-pr-reviewer`; the skill is only a setup guide for creating the plugin-spec automation.
- Use a safe string writer such as `json.dumps(value)` when inserting user-provided
- repository names, labels, or style instructions into Python string literals.
- `json.dumps(list_of_repos)` produces the whole `REPOS` list safely in one step.
+ ## Create with the plugin-spec endpoint
- Run these commands from this skill's directory and write the customized script
- to a temporary build directory:
- ```bash
- mkdir -p /tmp/pr-reviewer-build
- cp -L scripts/github_client.py /tmp/pr-reviewer-build/github_client.py
- # write the customized main.py to /tmp/pr-reviewer-build/main.py
- ```
+ Use `POST /api/automation/v1/plugin-spec` with this plugin source:
- Validate syntax before packaging:
- ```bash
- python3 -m py_compile /tmp/pr-reviewer-build/main.py && echo "Syntax OK"
+ ```json
+ {
+ "source": "github:OpenHands/extensions",
+ "repo_path": "plugins/github-pr-reviewer",
+ "ref": "main"
+ }
```
- Fix any syntax errors before proceeding.
-
- ### Step 7 - Package and upload
-
- Determine the Automation backend URL and auth from the `<RUNTIME_SERVICES>`
- block in your system context:
- - **OPENHANDS_HOST**: the Automation backend `url_from_agent`
- - **Auth**: `X-Session-API-Key: $OPENHANDS_AUTOMATION_API_KEY`
-
- ```bash
- tar -czf /tmp/pr-reviewer.tar.gz -C /tmp/pr-reviewer-build .
+ `ref` is `main` while this plugin layout is unreleased. After an `openhands-extensions` release includes this plugin, prefer the matching release tag, for example `v<extensions-version>`. If setup code can inspect the installed package version, parameterize this value as `v{version('openhands-extensions')}`; otherwise default to `main`.
- TARBALL_PATH=$(curl -s -X POST \
- "${OPENHANDS_HOST}/api/automation/v1/uploads?name=github-pr-reviewer" \
- -H "X-Session-API-Key: $OPENHANDS_AUTOMATION_API_KEY" \
- -H "Content-Type: application/gzip" \
- --data-binary @/tmp/pr-reviewer.tar.gz \
- | python3 -c "import json,sys; print(json.load(sys.stdin)['tarball_path'])")
+ For local Agent Canvas development from this checkout, use a local source instead:
- echo "Uploaded: $TARBALL_PATH"
+ ```json
+ {
+ "source": "/absolute/path/to/OpenHands/extensions",
+ "repo_path": "plugins/github-pr-reviewer"
+ }
```
- ### Step 8 - Register the automation
+ ## Request template
- ```bash
- curl -s -X POST "${OPENHANDS_HOST}/api/automation/v1" \
- -H "X-Session-API-Key: $OPENHANDS_AUTOMATION_API_KEY" \
- -H "Content-Type: application/json" \
- -d "{
- \"name\": \"GitHub PR Reviewer: {repo_summary} label {trigger_label}\",
- \"trigger\": {\"type\": \"cron\", \"schedule\": \"{cron_schedule}\"},
- \"tarball_path\": \"$TARBALL_PATH\",
- \"entrypoint\": \"python3 main.py\",
- \"timeout\": 600
- }" | python3 -m json.tool
+ ```json
+ {
+ "name": "Create an automation that reviews GitHub pull requests when a configurable trigger label is applied",
+ "plugin": {
+ "source": "github:OpenHands/extensions",
+ "repo_path": "plugins/github-pr-reviewer",
+ "ref": "main"
+ },
+ "config": {
+ "repos": [
+ "OpenHands/extensions"
+ ],
+ "trigger_label": "openhands-review",
+ "review_tone": "concise",
+ "github_token_secret": "GITHUB_PERSONAL_ACCESS_TOKEN",
+ "trigger_reviewer": "all-hands-bot",
+ "maintainers": []
+ },
+ "trigger": {
+ "type": "cron",
+ "schedule": "*/15 * * * *",
+ "timezone": "UTC"
+ },
+ "enabled": false
+ }
```
- Use the single repository as `{repo_summary}` when there is one, and something
- like `3 repos` when there are several. A poll now downloads a tarball per queued
- review, so the timeout allows for that; a run never waits for a review to
- finish, only for it to be started.
-
- Record the returned `id`.
-
- ### Step 9 - Confirm
-
- Tell the user:
-
- > ✅ **GitHub PR Reviewer** is running!
- >
- > - Automation ID: `{id}`
- > - Repositories: `{owner}/{repo}`, ... (one line each)
- > - Trigger label: `{trigger_label}`
- > - Review tone: `{tone}`
- > - Polling schedule: `{cron_schedule}`
- > - State file per repository:
- > `~/.openhands/workspaces/automation-state/github_pr_reviewer_label_event_{id}_{owner}__{repo}.json`
- >
- > Apply the `{trigger_label}` label to a pull request to queue a review. Each
- > label event is processed once. To request another review, remove and re-apply
- > the label.
- >
- > The review is published as a pull request review on the head commit, with
- > inline comments where a finding maps to a changed line.
-
- ---
-
- ## Runtime Behaviour (per poll)
-
- Each cron run executes `main.py`, which resolves and validates
- `GITHUB_PERSONAL_ACCESS_TOKEN` once, then processes every repository in `REPOS`
- independently. One repository failing does not stop the others; the run fails
- only if every repository fails.
-
- For each repository:
-
- 1. Loads that repository's state (see `references/state-schema.md`).
- 2. Verifies repository access.
- 3. Lists open PRs, newest-updated first.
- 4. For each open PR carrying `TRIGGER_LABEL`:
- - Refetches current PR metadata to avoid acting on stale list data.
- - Finds the latest matching GitHub `labeled` issue event.
- - Skips the event if it has already been tracked.
- - Downloads the PR's head commit as a tarball and extracts it to
- `{WORKSPACE_BASE}/repositories/{owner}__{repo}/pr-{number}-{sha12}`. The
- archive is checked as it is unpacked: a single root, no absolute or `..`
- paths, and symlinks skipped rather than materialised.
- - Records `default` and the concrete model from the server settings already
- used to start the conversation, for the review footer.
- - Starts an OpenHands conversation **whose working directory is that
- checkout**, under an id chosen before it starts, with a review prompt
- carrying PR metadata, the exact head SHA, label event details, the
- requirement to re-fetch the current mutable GitHub state before deciding
- a verdict, and that conversation's review marker.
- - Posts an acknowledgement comment with the label event, head SHA, and
- conversation link.
- - Records the review in state with `status: "active"` and the checkout path.
- - If the checkout or the conversation cannot be created, the checkout is
- removed and nothing is recorded, so the next poll retries the label event.
- 5. For each active review conversation:
- - Marks it closed without posting if the PR has closed or merged.
- - Suppresses stale results if the PR head SHA changed after the review was
- queued.
- - When the conversation reaches `idle`, `finished`, `error`, or `stuck`,
- verifies a submitted review by the token's own user at that head SHA and
- appends the recorded LLM profile and model to each review carrying this
- conversation's marker before marking the review complete. Reviews without
- that marker are confirmed but never edited. If no review exists, the
- agent's final response is posted as a comment with provenance. Failed
- verification or publication is retried on the next poll.
- - Abandons a conversation that has not reached a terminal status within two
- hours, so its checkout can be reclaimed.
- 6. Removes the checkout of every finished review, but only after confirming the
- conversation has stopped - deleting it under a running agent would remove its
- working directory. When that cannot be confirmed the directory is left alone
- and the next poll tries again.
- 7. Saves that repository's state atomically.
-
- The completion callback fires once for the whole run.
-
- ---
-
- ## Additional Resources
-
- - **`references/state-schema.md`** - State JSON schema, field definitions, and
- review lifecycle diagram.
- - **`scripts/main.py`** - The complete automation script. Customize the five
- constants at the top before packaging.
- - **`tests/test_main.py`** - Unit tests for the checkout, its removal, and state
- handling. Run them from the skill root with `python -m pytest tests/` after
- editing the script.
+ The authoritative config schema is in `plugins/github-pr-reviewer/.plugin/plugin.json` under `automation.config_schema`. Use that schema to validate any user-provided values before creating the automation.
- ---
+ ## Runtime notes
- ## Troubleshooting
+ - Entrypoint: `python3 plugin_runner.py worker.py` in `scripts`.
+ - Agent profile: `default`.
+ - Secondary plugins: `[]`.
- | Symptom | Likely cause | Fix |
- |---|---|---|
- | Bot never queues reviews | Trigger label not present or no matching `labeled` event | Apply the configured label to the PR |
- | "Bad credentials" in run logs | Token expired | Rotate and update `GITHUB_PERSONAL_ACCESS_TOKEN` |
- | 404 on repo access | Repo name wrong or no access | Re-check the entry in `REPOS` and the token's permissions |
- | One repository is skipped, others work | That repository failed its access check | Read the `=== owner/repo ===` block in the run log |
- | Same PR not reviewed after new commits | Label event was already processed | Remove and re-apply the trigger label |
- | Review paused with a failing-check comment | A current-head required check reported `failure`, `cancelled`, or `timed_out` | Fix the named checks and push; the review starts on the new head, or request `all-hands-bot` to review immediately |
- | Review reported waiting on checks | A current-head required check is `queued` or `in_progress`, or has not reported yet | No action; a later scan or a new review request retries |
- | Optional workflow failed but no review was paused | The failed workflow is not required, so the required-only scheduled gate ignored it | No action; only GitHub-required checks gate scheduled discovery |
- | Only a few reviews start on a large backlog | The per-scan `max_new_per_run` bound (default 2) reached | No action; later scans drain the remaining oldest eligible PRs, or raise `max_new_per_run` if the deployment can hold more agents |
- | Review result never posts | Conversation still running or stuck | Open the conversation link from the acknowledgement comment |
- | Stale review suppressed | PR head SHA changed while the agent was reviewing | Re-apply the trigger label after the latest commit |
- | Review arrives as a plain comment, not a review | Publishing failed, so the script posted the text as a fallback | Check that the token has Pull requests: Read and Write |
- | Agent reports it cannot clone the repo | Prompt asked it not to; the workspace is already the checkout | No action - the code is at the head SHA in its working directory |
- | Checkouts remain under `repositories/` | Their conversations had not stopped yet | They are removed by a later poll once the conversation is terminal |
+ The old skill-local script packaging flow is deprecated. Use the plugin spec directly so OpenHands/automation resolves the plugin, writes `config.json`, and derives the runtime entrypoint from the plugin manifest.