test-browser · diff
git:20260717.87dea44 to git:20260909.f6f5227
38 added, 8 removed. Audit A to A.
---
name: test-browser
description: "This skill should be used when running end-to-end browser tests on pages affected by a PR. Uses agent-browser CLI to map changed files to routes and capture screenshots."
---
# Browser Test
<command_purpose>Run end-to-end browser tests on pages affected by a PR or branch changes using agent-browser CLI.</command_purpose>
> **Positioning (ADR-049, #4834):** for **structural-UI** diffs (nav/rail/dashboard shell), the *pre-merge* visual check is now the committed, CI-blocking `nav-states-*.e2e.ts` gate run by `/soleur:qa` Step 2.6 — not this skill. Running `test-browser` only *after* ship is the deferral trap that let the #4810 layout bugs reach prod. Treat this skill as a **post-ship smoke** (broad page reachability + console-error sweep) that complements, never replaces, the pre-merge `/soleur:qa` gate.
## CRITICAL: Use agent-browser CLI Only
**DO NOT use Chrome MCP tools (mcp__claude-in-chrome__*).**
This skill uses the `agent-browser` CLI exclusively. The agent-browser CLI is a Bash-based tool from Vercel that runs headless Chromium. It is NOT the same as Chrome browser automation via MCP.
If calling `mcp__claude-in-chrome__*` tools, STOP. Use `agent-browser` Bash commands instead.
- When working in a worktree, always pass absolute paths to Playwright MCP screenshot tools -- relative filenames are resolved from the main repo root (the MCP server's CWD), so screenshots land in the main repo instead of the active worktree.
## Introduction
<role>QA Engineer specializing in browser-based end-to-end testing</role>
This skill tests affected pages in a real browser, catching issues that unit tests miss:
- JavaScript integration bugs
- CSS/layout regressions
- User workflow breakages
- Console errors
## Prerequisites
<requirements>
- Local development server running (e.g., `bin/dev`, `rails server`, `npm run dev`)
- agent-browser CLI installed (see Setup below)
- Git repository with changes to test
</requirements>
## Setup
**Check installation:**
```bash
command -v agent-browser >/dev/null 2>&1 && echo "Installed" || echo "NOT INSTALLED"
```
**Install if needed:**
```bash
npm install --prefix ~/.local -g agent-browser@0.22.3
agent-browser install # Downloads Chrome for Testing (~300MB)
```
**On Linux (Ubuntu 23.10+, containers, VMs), export the no-sandbox launch flag
before the first command** — otherwise Chrome cannot launch and `open` hangs:
```bash
export AGENT_BROWSER_ARGS="--no-sandbox"
```
See the `agent-browser` skill for detailed usage and the "Chrome fails to launch
/ `open` hangs" troubleshooting entry.
## Main Tasks
### 0. Verify agent-browser Installation
Before starting ANY browser testing, verify agent-browser is installed:
```bash
command -v agent-browser >/dev/null 2>&1 && echo "Ready" || (echo "Installing..." && npm install --prefix ~/.local -g agent-browser@0.22.3 && agent-browser install)
```
If installation fails, inform the user and stop.
### 1. Ask Browser Mode
<ask_browser_mode>
Before starting tests, ask the user if they want to watch the browser:
Use AskUserQuestion with:
- Question: "Do you want to watch the browser tests run?"
- Options:
1. **Headed (watch)** - Opens visible browser window so tests can be observed
2. **Headless (faster)** - Runs in background, faster but invisible
Store the choice and use `--headed` flag when user selects "Headed".
</ask_browser_mode>
### 2. Determine Test Scope
<test_target> $ARGUMENTS </test_target>
<determine_scope>
**If PR number provided:**
```bash
gh pr view [number] --json files -q '.files[].path'
```
**If 'current' or empty:**
```bash
git diff --name-only main...HEAD
```
**If branch name provided:**
```bash
git diff --name-only main...[branch]
```
</determine_scope>
### 3. Map Files to Routes
<file_to_route_mapping>
Map changed files to testable routes:
| File Pattern | Route(s) |
|-------------|----------|
| `app/views/users/*` | `/users`, `/users/:id`, `/users/new` |
| `app/controllers/settings_controller.rb` | `/settings` |
| `app/javascript/controllers/*_controller.js` | Pages using that Stimulus controller |
| `app/components/*_component.rb` | Pages rendering that component |
| `app/views/layouts/*` | All pages (test homepage at minimum) |
| `app/assets/stylesheets/*` | Visual regression on key pages |
| `app/helpers/*_helper.rb` | Pages using that helper |
| `src/app/*` (Next.js) | Corresponding routes |
| `src/components/*` | Pages using those components |
Build a list of URLs to test based on the mapping.
</file_to_route_mapping>
### 4. Verify Server is Running
<check_server>
Before testing, verify the local server is accessible:
+ ### Preflight: verify the plugin install before any snapshot
+
+ The redactor is reached through `${CLAUDE_PLUGIN_ROOT}`. An ambient value pointing at a
+ directory that is not a Soleur install would resolve to a path that does not exist — or, worse,
+ to one an attacker chose. Verify plugin IDENTITY and halt if it does not hold (ADR-179 decision 2);
+ a `test -f` on the script alone is a shape check and was measured bypassable.
+
```bash
+ [ -f "${CLAUDE_PLUGIN_ROOT}/.claude-plugin/plugin.json" ] \
+ && grep -q '"name"[[:space:]]*:[[:space:]]*"soleur"' "${CLAUDE_PLUGIN_ROOT}/.claude-plugin/plugin.json" \
+ || { echo "SOLEUR_SNAPSHOT_HALT reason=plugin-root-unverified root=[${CLAUDE_PLUGIN_ROOT}]" >&2
+ echo " Cannot locate the snapshot redactor, so no accessibility snapshot may be taken here." >&2
+ echo " Root EMPTY: no Soleur plugin is loaded in this session. Install it and start a NEW session." >&2
+ echo " Root set but wrong: a repo checkout is not an install. Run 'claude plugin update soleur', then RESTART Claude Code." >&2
+ echo " Nothing has been captured yet, so nothing has leaked." >&2
+ exit 2; }
+ ```
+
+ ```bash
agent-browser open http://localhost:3000
- agent-browser snapshot -i
+ agent-browser snapshot -i 2>&1 | python3 "${CLAUDE_PLUGIN_ROOT}/skills/agent-browser/scripts/redact-a11y-snapshot.py"
```
If server is not running, inform user:
```markdown
**Server not running**
Please start the development server:
- Rails: `bin/dev` or `rails server`
- Node/Next.js: `npm run dev`
Then run `/test-browser` again.
```
</check_server>
### 5. Test Each Affected Page
<test_pages>
For each affected route, use agent-browser CLI commands (NOT Chrome MCP):
**Step 1: Navigate and capture snapshot**
```bash
agent-browser open "http://localhost:3000/[route]"
- agent-browser snapshot -i
+ agent-browser snapshot -i 2>&1 | python3 "${CLAUDE_PLUGIN_ROOT}/skills/agent-browser/scripts/redact-a11y-snapshot.py"
```
**Step 2: For headed mode (visual debugging)**
```bash
agent-browser --headed open "http://localhost:3000/[route]"
- agent-browser --headed snapshot -i
+ agent-browser --headed snapshot -i 2>&1 | python3 "${CLAUDE_PLUGIN_ROOT}/skills/agent-browser/scripts/redact-a11y-snapshot.py"
```
**Step 3: Verify key elements**
- - Use `agent-browser snapshot -i` to get interactive elements with refs
+ - Use `agent-browser snapshot -i 2>&1 | python3 "${CLAUDE_PLUGIN_ROOT}/skills/agent-browser/scripts/redact-a11y-snapshot.py"` to get interactive elements with refs
+
+ **Credential safety (#7947).** An accessibility snapshot serializes the **value**
+ of input fields, including a value the agent never typed (a password manager's
+ autofill, a static `value=`, a generated-credential panel). On any page carrying
+ a password or credential field, route the snapshot through the redactor —
+ `agent-browser snapshot -i 2>&1 | python3 "${CLAUDE_PLUGIN_ROOT}/skills/agent-browser/scripts/redact-a11y-snapshot.py"`
+ — and never capture a page that is displaying a credential value: a screenshot
+ is safe for a `type=password` field but **not** for a readonly `type=text`
+ credential panel, which renders in clear. Full rule and its measured ceiling:
+ `plugins/soleur/skills/agent-browser/SKILL.md` §"Credential safety on a login or
+ credential page".
+
- Page title/heading present
- Primary content rendered
- No error messages visible
- Forms have expected fields
**Step 4: Test critical interactions**
```bash
- agent-browser click @e1 # Use ref from snapshot
- agent-browser snapshot -i
+ agent-browser click @e1 # Use ref from snapshot 2>&1 | python3 "${CLAUDE_PLUGIN_ROOT}/skills/agent-browser/scripts/redact-a11y-snapshot.py"
+ agent-browser snapshot -i 2>&1 | python3 "${CLAUDE_PLUGIN_ROOT}/skills/agent-browser/scripts/redact-a11y-snapshot.py"
```
**Step 5: Take screenshots**
```bash
agent-browser screenshot page-name.png
agent-browser screenshot --full page-name-full.png # Full page
```
</test_pages>
### 6. Human Verification (When Required)
<human_verification>
Pause for human input when testing touches:
| Flow Type | What to Ask |
|-----------|-------------|
| OAuth | "Please sign in with [provider] and confirm it works" |
| Email | "Check inbox for the test email and confirm receipt" |
| Payments | "Complete a test purchase in sandbox mode" |
| SMS | "Verify the SMS code was received" |
| External APIs | "Confirm the [service] integration is working" |
Use AskUserQuestion:
```markdown
**Human Verification Needed**
This test touches the [flow type]. Please:
1. [Action to take]
2. [What to verify]
Did it work correctly?
1. Yes - continue testing
2. No - describe the issue
```
</human_verification>
### 7. Handle Failures
<failure_handling>
When a test fails:
1. **Document the failure:**
- Screenshot the error state: `agent-browser screenshot error.png`
- Note the exact reproduction steps
2. **Ask user how to proceed:**
```markdown
**Test Failed: [route]**
Issue: [description]
Console errors: [if any]
How to proceed?
1. Fix now - help debug and fix
2. Create todo - Add to todos/ for later
3. Skip - Continue testing other pages
```
3. **If "Fix now":**
- Investigate the issue
- Propose a fix
- Apply fix
- Re-run the failing test
4. **If "Create todo":**
- Create `{id}-pending-p1-browser-test-{description}.md`
- Continue testing
5. **If "Skip":**
- Log as skipped
- Continue testing
</failure_handling>
### 8. Test Summary
<test_summary>
After all tests complete, present summary:
```markdown
## Browser Test Results
**Test Scope:** PR #[number] / [branch name]
**Server:** http://localhost:3000
### Pages Tested: [count]
| Route | Status | Notes |
|-------|--------|-------|
| `/users` | Pass | |
| `/settings` | Pass | |
| `/dashboard` | Fail | Console error: [msg] |
| `/checkout` | Skip | Requires payment credentials |
### Console Errors: [count]
- [List any errors found]
### Human Verifications: [count]
- OAuth flow: Confirmed
- Email delivery: Confirmed
### Failures: [count]
- `/dashboard` - [issue description]
### Created Todos: [count]
- `005-pending-p1-browser-test-dashboard-error.md`
### Result: [PASS / FAIL / PARTIAL]
```
</test_summary>
### 9. Cleanup
<cleanup>
Playwright MCP writes screenshots to the main repo root when invoked from a worktree, so clean both locations. If any test failures were recorded with `error.png`, upload or review those screenshots before running cleanup.
```bash
# Remove session screenshots from current working directory
rm -f *.png
# If in a worktree, also clean the main repo root
MAIN_REPO=$(git rev-parse --show-superproject-working-tree 2>/dev/null)
if [[ -n "$MAIN_REPO" ]]; then
rm -f "$MAIN_REPO"/*.png
fi
```
**Do NOT delete** files under `plugins/soleur/docs/` -- those are legitimate assets.
</cleanup>
## Quick Usage Examples
```bash
# Test current branch changes
/test-browser
# Test specific PR
/test-browser 847
# Test specific branch
/test-browser feature/new-dashboard
```
## agent-browser CLI Reference
**ALWAYS use these Bash commands. NEVER use mcp__claude-in-chrome__* tools.**
```bash
# Navigation
agent-browser open <url> # Navigate to URL
agent-browser back # Go back
agent-browser close # Close browser
# Snapshots (get element refs)
- agent-browser snapshot -i # Interactive elements with refs (@e1, @e2, etc.)
- agent-browser snapshot -i --json # JSON output
+ agent-browser snapshot -i 2>&1 | python3 "${CLAUDE_PLUGIN_ROOT}/skills/agent-browser/scripts/redact-a11y-snapshot.py" # Interactive elements with refs (@e1, @e2, etc.)
+ agent-browser snapshot -i --json 2>&1 | python3 "${CLAUDE_PLUGIN_ROOT}/skills/agent-browser/scripts/redact-a11y-snapshot.py" # JSON output
# Interactions (use refs from snapshot)
agent-browser click @e1 # Click element
agent-browser fill @e1 "text" # Fill input
agent-browser type @e1 "text" # Type without clearing
agent-browser press Enter # Press key
# Screenshots
agent-browser screenshot out.png # Viewport screenshot
agent-browser screenshot --full out.png # Full page screenshot
# Headed mode (visible browser)
agent-browser --headed open <url> # Open with visible browser
agent-browser --headed click @e1 # Click in visible browser
# Wait
agent-browser wait @e1 # Wait for element
agent-browser wait 2000 # Wait milliseconds
```