browser-harness · diff
v1.0.1 to v1.1.0
161 added, 147 removed. Audit A to A.
---
name: browser-harness
description: >
- Self-healing browser automation framework that connects LLMs directly to Chrome via CDP.
- Use when the user needs an AI agent to autonomously complete web browser tasks, write adaptive
- helper code during execution, contribute domain skills (site-specific playbooks), or integrate
- with Browser Use Cloud. Triggers on:
+ Self-healing browser automation framework that connects LLM agents directly to Chrome via CDP.
+ Use when the user needs autonomous browser tasks, clean browser verification, Codex or Antigravity
+ browser control, Claude-safe screenshots, adaptive helper code in `agent_helpers.py`, domain skills,
+ or Browser Use Cloud escalation. Triggers on:
browser-harness, self-healing browser, llm browser automation, cdp agent, chrome devtools agent,
- agent browser task, browser-use harness, browser automation agent, domain skills browser.
+ codex browser automation, antigravity browser automation, claude screenshot error, claude image error,
+ agent browser task, browser-use harness, domain skills browser.
allowed-tools: Bash Read Write Edit Glob Grep WebFetch
- compatibility: Requires Python 3.10+, Chrome with remote debugging enabled. Setup via Claude Code prompt. Optional Browser Use Cloud tier (free: 3 concurrent browsers, proxies, captcha solving).
+ compatibility: Requires Python 3.10+ and Chrome/Chromium with remote debugging enabled. Works from Claude Code, Codex CLI, Antigravity (`agy`), Gemini CLI, and OpenCode when the agent can edit/run the local Python workspace. Includes a Claude-safe screenshot patch for PIL file-handle and image-size issues.
license: MIT
metadata:
- tags: browser-harness, browser-automation, self-healing, cdp, chrome-devtools-protocol, llm-browser, agent-browser, domain-skills, claude-code
- version: "1.0.1"
+ tags: browser-harness, browser-automation, self-healing, cdp, chrome-devtools-protocol, llm-browser, codex, antigravity, claude-code, claude-vision-safe, screenshot, domain-skills
+ version: "1.1.0"
source: https://github.com/browser-use/browser-harness
- platforms: Claude, Codex, OpenCode
+ platforms: Claude Code, Codex, Antigravity, Gemini CLI, OpenCode
---
- # browser-harness — self-healing LLM browser automation
+ # browser-harness - self-healing LLM browser automation
> **Keyword**: `browser-harness` · `self-healing browser` · `llm browser automation` · `cdp agent`
>
- > Direct WebSocket connection between your LLM agent and Chrome via Chrome DevTools Protocol — no intermediary layers, no fragile selectors. The agent writes and improves its own helper code on every run.
+ > Direct WebSocket connection between an LLM agent and Chrome via Chrome DevTools Protocol. The agent can inspect the page, write helper code, reuse domain skills, and verify the task without an extra browser abstraction layer.
- Browser Harness is most useful when you choose the **smallest automation mode that fits the job**:
- - start with existing domain skills before writing new helper code
- - let the agent discover and document selectors before hardcoding them
- - use Browser Use Cloud only when local Chrome or proxies are insufficient
- - prefer `agent_helpers.py` edits over rewriting core package code
+ Browser Harness is the canonical replacement for the removed `agent-browser` skill in this catalog. Use it for clean browser verification, autonomous browser tasks, and platform-portable CDP control across Claude Code, Codex, Antigravity, Gemini CLI, and OpenCode.
## When to use this skill
- - The user needs an LLM agent to autonomously complete multi-step browser tasks (login, form fill, scrape, navigate)
- - The user wants self-healing automation where the agent writes missing helper functions during execution
- - The user wants to contribute or consume community domain skills (site-specific playbooks in `agent-workspace/domain-skills/`)
- - The user needs direct CDP access to Chrome without Playwright/Selenium overhead
- - The user wants to integrate with Browser Use Cloud for concurrent browsers, proxies, or captcha solving
- - The user wants to route complex browser workflows to an agent that can adapt when the DOM changes
+ - The user needs an LLM agent to complete a multi-step browser workflow: login, navigation, form fill, data extraction, download, or verification.
+ - The workflow needs a clean browser profile or repeatable CDP verification instead of the user's already-open browser state.
+ - The user is running from Codex CLI or Antigravity and needs a local browser harness the agent can operate with shell/Python commands.
+ - Claude reports image/screenshot/tool errors when browser screenshots are written, resized, or re-opened.
+ - The target DOM changes and the agent should add or repair helpers in `agent-workspace/agent_helpers.py`.
+ - The task benefits from site-specific domain skills in `agent-workspace/domain-skills/`.
+ - Browser Use Cloud is justified for concurrent browsers, proxies, or captcha solving on allowed targets.
## Do not use this skill when
- - The task is simple HTML extraction without JS rendering → route to `scrapling`
- - The task is browser-based UI verification or screenshot capture → route to `agent-browser`
- - The task is web scraping with selector drift recovery → route to `scrapling`
- - The task is UI annotation or rendered-UI feedback → route to `agentation`
- - The task is Playwright/Puppeteer script authoring without agent autonomy → use those tools directly
+ - The task is simple HTML extraction without browser state or JS interaction -> route to `scrapling`.
+ - The task is exact human UI annotation or pointing at a rendered issue -> route to `agentation`.
+ - The task must reuse the user's already-open authenticated Chrome profile -> route to `playwriter`.
+ - The task is React component source capture -> route to `react-grab`.
+ - The task is ordinary Playwright/Puppeteer script authoring without agent autonomy -> use that stack directly.
## Instructions
- ### Step 1: Install browser-harness
+ ### Step 1: Choose the execution packet
- Browser Harness is designed to be set up by an AI agent. Paste the following prompt into Claude Code:
+ Pick one primary packet before writing commands:
- ```
- Set up https://github.com/browser-use/browser-harness for me
- ```
+ - **local-cdp**: local Chrome/Chromium with `--remote-debugging-port=9222`.
+ - **codex-cdp**: Codex CLI controls the same local checkout and CDP endpoint.
+ - **antigravity-cdp**: Antigravity (`agy`) uses the same workspace and Chrome debugging endpoint.
+ - **claude-vision-safe**: screenshot capture must use the safe image pipeline below.
+ - **domain-skill**: add or repair a site-specific helper in `agent-workspace/domain-skills/`.
+ - **cloud-browser**: Browser Use Cloud is needed and allowed.
- The agent will:
- 1. Clone the repository and read `install.md`
- 2. Guide you to enable Chrome remote debugging at `chrome://inspect/#remote-debugging`
- 3. Grant WebSocket connection permissions when prompted
- 4. Verify the CDP connection is live
+ ### Step 2: Install browser-harness
- **Manual clone (if preferred):**
+ Browser Harness can be set up by an agent from any platform that can run shell commands:
+
```bash
git clone https://github.com/browser-use/browser-harness.git
cd browser-harness
- python3 -m venv .venv && source .venv/bin/activate
+ python3 -m venv .venv
+ source .venv/bin/activate
pip install -e .
```
- **Requirements:**
+ Claude Code can also use the project-native setup prompt:
+
+ ```text
+ Set up https://github.com/browser-use/browser-harness for me
+ ```
+
+ Requirements:
+
- Python 3.10+
- - Chrome / Chromium with `--remote-debugging-port=9222`
+ - Chrome or Chromium
+ - `http://localhost:9222/json` reachable from the agent runtime
- ### Step 2: Enable Chrome remote debugging
+ ### Step 3: Enable Chrome remote debugging
+ Use a separate profile so the harness can safely create clean sessions:
+
```bash
# macOS
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
--remote-debugging-port=9222 --user-data-dir=/tmp/chrome-debug
# Linux
google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-debug
- # Windows
- "C:\Program Files\Google\Chrome\Application\chrome.exe" ^
- --remote-debugging-port=9222 --user-data-dir=%TEMP%\chrome-debug
+ # Windows PowerShell
+ & "C:\Program Files\Google\Chrome\Application\chrome.exe" `
+ --remote-debugging-port=9222 --user-data-dir="$env:TEMP\chrome-debug"
```
- Verify: open `http://localhost:9222/json` — you should see a JSON list of open tabs.
-
- ### Step 3: Run browser tasks
-
- Give your agent a natural-language browser task:
+ Verify:
- ```
- "Log into example.com with credentials X and download the monthly report"
- "Navigate to GitHub, open the first open issue, and summarize it"
- "Fill in the contact form at example.com with the following data: ..."
+ ```bash
+ curl -s http://localhost:9222/json
```
- The agent will:
- 1. Connect to Chrome via CDP WebSocket
- 2. Look up existing helpers in `agent-workspace/agent_helpers.py`
- 3. Write new helper functions for missing actions
- 4. Execute the task and verify completion
+ ### Step 4: Platform-specific notes
- ### Step 4: Extend with domain skills
+ | Platform | Use browser-harness when | Setup note |
+ |----------|--------------------------|------------|
+ | Claude Code | You need autonomous browser work or Claude-safe screenshots | Apply the screenshot patch before image-heavy work |
+ | Codex CLI | You need local CDP automation from a repo task | Keep `.venv` inside the checkout and run commands from that shell |
+ | Antigravity (`agy`) | You need the same browser harness from Antigravity workflows | Ensure `agy` can see the checkout and `localhost:9222` |
+ | Gemini CLI / OpenCode | You need portable browser automation without platform-specific MCP wiring | Use the same local CDP and Python workspace |
- Domain skills are site-specific playbooks contributed by the community. They live in:
+ For Codex and Antigravity, do not assume Claude Code plugin commands exist. Prefer explicit local commands:
- ```
- agent-workspace/domain-skills/
- ├── github.py # GitHub-specific helpers
- ├── linkedin.py # LinkedIn navigation patterns
- └── your-site.py # Add your own
+ ```bash
+ cd ~/browser-harness
+ source .venv/bin/activate
+ python -c "import browser_harness; print('browser-harness OK')"
+ curl -s http://localhost:9222/json
```
- **Add a domain skill:**
- ```python
- # agent-workspace/domain-skills/mysite.py
+ ### Step 5: Apply the Claude-safe screenshot patch
- def login(page, username: str, password: str):
- """Log into mysite.com."""
- page.goto("https://mysite.com/login")
- page.fill("#username", username)
- page.fill("#password", password)
- page.click("button[type=submit]")
- page.wait_for_url("**/dashboard")
+ If Claude throws image recognition, image upload, PNG read, or tool errors around screenshots, patch `src/browser_harness/helpers.py` so screenshots are decoded and resized in memory, and PIL file handles are closed before saving overlays.
+
+ Required changes:
+
+ ```diff
+ diff --git a/src/browser_harness/helpers.py b/src/browser_harness/helpers.py
+ --- a/src/browser_harness/helpers.py
+ +++ b/src/browser_harness/helpers.py
+ @@
+ -import base64, importlib.util, json, math, os, sys, time, urllib.request
+ +import base64, importlib.util, io, json, math, os, sys, time, urllib.request
+ @@
+ - img = Image.open(path)
+ + with Image.open(path) as src:
+ + img = src.copy()
+ @@
+ - open(path, "wb").write(base64.b64decode(r["data"]))
+ + data = base64.b64decode(r["data"])
+ if max_dim:
+ from PIL import Image
+ - img = Image.open(path)
+ + img = Image.open(io.BytesIO(data))
+ if max(img.size) > max_dim:
+ img.thumbnail((max_dim, max_dim))
+ - img.save(path)
+ + buf = io.BytesIO()
+ + img.save(buf, format="PNG")
+ + data = buf.getvalue()
+ + with open(path, "wb") as f:
+ + f.write(data)
```
- The agent discovers and uses domain skills automatically — no registration needed.
+ Why this matters:
- ### Step 5: Browser Use Cloud (optional)
+ - `Image.open(path)` keeps a lazy file handle unless copied or closed.
+ - Claude image/tool pipelines are more likely to fail when a PNG is opened, rewritten, then reopened by the agent in quick succession.
+ - In-memory resize via `io.BytesIO` avoids the write-read-write cycle.
+ - Writing once with `with open(path, "wb")` produces a stable file for Claude vision upload.
- For concurrent browsers, residential proxies, or captcha solving:
+ Recommended screenshot call for Claude:
```python
- from browser_harness import BrowserUseCloud
-
- client = BrowserUseCloud(api_key="YOUR_API_KEY")
- result = client.run("Log into example.com and extract the dashboard data")
- print(result)
+ path = capture_screenshot(max_dim=1800)
```
- Free tier includes:
- - 3 concurrent browsers
- - Residential proxies
- - Captcha solving
+ Use `max_dim=1800` on high-DPI displays to stay under common 2000px-per-side image limits.
- Sign up at [browser-use.com](https://browser-use.com).
+ ### Step 6: Run browser tasks
- ### Step 6: Customize agent_helpers.py
+ Give the agent a natural-language task:
- The agent writes and edits `agent-workspace/agent_helpers.py` during execution. You can also edit it manually:
+ ```text
+ Open the local app, complete the signup form, and verify that the dashboard appears.
+ Navigate to GitHub, open the first open issue, and summarize the acceptance criteria.
+ Fill in the contact form at example.com and confirm the success message.
+ ```
- ```python
- # agent-workspace/agent_helpers.py
+ The agent should:
- def click_accept_cookies(page):
- """Dismiss cookie banners on common sites."""
- selectors = [
- "#accept-cookies", ".cookie-accept", "[data-testid=accept-btn]"
- ]
- for sel in selectors:
- try:
- page.click(sel, timeout=2000)
- return True
- except:
- continue
- return False
- ```
+ 1. Connect to Chrome via CDP.
+ 2. Inspect tabs and page state.
+ 3. Reuse existing helpers in `agent-workspace/agent_helpers.py`.
+ 4. Add missing helpers in `agent-workspace/agent_helpers.py` or `agent-workspace/domain-skills/`.
+ 5. Verify completion with text, URL, DOM state, screenshot, or downloaded artifact evidence.
- **Core package** (`src/browser_harness/`) is protected — do not edit it directly. All customization goes through `agent_helpers.py` and `domain-skills/`.
+ ### Step 7: Extend with domain skills
- ## Examples
+ Domain skills are site-specific playbooks. Keep them small and reusable:
- ### Example 1: Autonomous login and data extraction
- ```
- Prompt to agent: "Go to https://dashboard.example.com, log in with user@example.com / pass123,
- navigate to Reports > Monthly, and download the CSV for April 2026"
+ ```text
+ agent-workspace/domain-skills/
+ ├── github.py
+ ├── linkedin.py
+ └── your-site.py
```
- ### Example 2: Form automation
- ```
- Prompt to agent: "Fill out the contact form at https://example.com/contact with:
- Name: John Doe, Email: john@example.com, Message: 'Request for proposal'"
- ```
+ Example:
- ### Example 3: Add a domain skill via agent
- ```
- Prompt to agent: "Create a domain skill for https://github.com that can:
- 1. Open a repository by name
- 2. Create a new issue with a title and body
- 3. Add a label to an existing issue
- Save it to agent-workspace/domain-skills/github.py"
+ ```python
+ def login(page, username: str, password: str):
+ """Log into mysite.com."""
+ page.goto("https://mysite.com/login")
+ page.fill("#username", username)
+ page.fill("#password", password)
+ page.click("button[type=submit]")
+ page.wait_for_url("**/dashboard")
```
- ### Example 4: Browser Use Cloud
+ ### Step 8: Browser Use Cloud escalation
+
+ Use Browser Use Cloud only when local Chrome is insufficient and the target permits automation:
+
```python
from browser_harness import BrowserUseCloud
- client = BrowserUseCloud(api_key="bu_...")
- result = client.run("Search for 'browser automation python' on Google and return the top 5 links")
+
+ client = BrowserUseCloud(api_key="YOUR_API_KEY")
+ result = client.run("Extract the dashboard data and return a CSV summary")
+ print(result)
```
## Best practices
- 1. Let the agent write helper code — avoid manual `agent_helpers.py` edits that conflict with agent-generated code.
- 2. Enable Chrome remote debugging before starting any browser task; verify with `http://localhost:9222/json`.
- 3. Prefer existing domain skills over blank-slate prompts for well-known sites.
- 4. Use Browser Use Cloud for tasks that need proxies or captcha bypass — do not attempt to bypass anti-bot measures on sites that prohibit it.
- 5. Keep `agent_helpers.py` lean — split large helpers into domain-skill files once they become site-specific.
- 6. For stateless HTML-only scraping, route to `scrapling` instead — CDP overhead is unnecessary.
- 7. Verify task completion explicitly in your prompt ("confirm the form was submitted successfully") to trigger the agent's self-check.
-
- ## Local patches
-
- The following fixes are applied to `src/browser_harness/helpers.py` in local installations to prevent image-handling errors with Claude's vision API:
+ 1. Start with `local-cdp`; escalate only when the local CDP endpoint cannot satisfy the job.
+ 2. Keep core package edits minimal. Put ordinary workflow logic in `agent_helpers.py` or domain skills.
+ 3. Apply the Claude-safe screenshot patch before image-heavy Claude Code runs.
+ 4. For Codex and Antigravity, prefer explicit shell/Python commands over Claude-only plugin instructions.
+ 5. Treat every browser task as incomplete until the agent records final evidence.
+ 6. Use `scrapling` for stateless scraping and `playwriter` for already-open authenticated browser reuse.
+ 7. Do not bypass site terms, robots, rate limits, or authorization boundaries.
- - **Added `io` import** — required for `io.BytesIO` usage in screenshot pipeline.
- - **`click_at_xy` debug overlay** — use `with Image.open(path) as src: img = src.copy()` to avoid holding a file lock on the PNG after opening.
- - **`capture_screenshot`** — decode CDP data to `bytes` first, resize in-memory via `io.BytesIO` (never write then re-read the file), then write once with `with open(path, "wb")`. Prevents double-write and avoids unclosed file handle errors that trigger Claude tool errors.
+ ## Quick verification
- Apply with:
```bash
- cd ~/browser-harness && git apply <patch-file>
- # or edit src/browser_harness/helpers.py directly per the diff above
+ cd ~/browser-harness
+ source .venv/bin/activate
+ python -c "import browser_harness; print('browser-harness OK')"
+ curl -s http://localhost:9222/json
```
## References
- [browser-use/browser-harness GitHub](https://github.com/browser-use/browser-harness)
- - [scrapling](../scrapling/SKILL.md) — for stateless HTML/JS scraping without agent autonomy
- - [agent-browser](../agent-browser/SKILL.md) — for clean disposable browser verification sessions
- - [agentation](../agentation/SKILL.md) — for rendered-UI feedback and UI annotation
+ - [scrapling](../scrapling/SKILL.md) — stateless HTML/JS scraping without agent-owned browser state
+ - [playwriter](../playwriter/SKILL.md) — running-browser reuse when existing login/session state matters
+ - [agentation](../agentation/SKILL.md) — rendered-UI feedback and human annotation packets