CLAUDE.md ยท diff
git:20260421.f0e04a4 to git:20260722.d08d231
42 added, 65 removed. Audit A to A.
# Claude Code Instructions for abx-dl
- ## Project Overview
-
- abx-dl is a CLI tool for downloading URLs using ArchiveBox plugins. Plugins are symlinked from `/Users/squash/Code/ArchiveBox/archivebox/plugins` during development.
+ Use [`AGENTS.md`](./AGENTS.md) as the canonical repository guide. Keep this
+ checkout on `main`, use `uv` for Python commands, and exercise real CLI,
+ plugin, subprocess, network, and filesystem paths in tests. Do not add skips,
+ xfails, retries, fake hooks, fake binaries, fake buses, or fallback binary
+ resolution.
- ## Development Setup
+ ## Development setup
```bash
- # Install dependencies
- uv sync --group dev
-
- # Activate venv (or use uv run prefix)
- source .venv/bin/activate
+ set -Eeuo pipefail
+ uv sync --locked --dev --no-sources
+ help_output="$(uv run --no-sync --no-sources abx-dl --help)"
+ grep -q 'Usage:' <<<"$help_output"
```
- ## Running Tests
+ ## Inspect the installed plugin surface
- ### Manual testing in a temp directory
```bash
- cd /tmp && rm -rf abx-dl-test && mkdir abx-dl-test && cd abx-dl-test
- abx-dl dl --plugins=chrome,title 'https://example.com'
-
- # Or test all plugins
- abx-dl dl 'https://example.com'
-
- # Check output
- ls -la
- cat title/title.txt
- cat index.jsonl | jq -s '.'
+ set -Eeuo pipefail
+ plugin_info="$(uv run --no-sync --no-sources abx-dl plugins wget)"
+ grep -q 'WGET_BINARY=wget' <<<"$plugin_info"
+ grep -q 'on_Snapshot__06_wget' <<<"$plugin_info"
```
- ### Running pytest
- ```bash
- uv run pytest tests/
- uv run pytest -xvs tests/ # verbose with output
- ```
+ ## Verification
- ## Linting
+ The normal CI workflow runs the complete repository suite on Linux and macOS
+ with every supported Python minor version, every documentation snippet, every
+ test directory from the pinned `abx-plugins` release, and a real all-plugin
+ crawl. The Docker workflow builds and tests both amd64 and arm64 images and
+ runs every snippet marked `docker_required`.
+ Run the repository's complete static verification before publishing changes:
+
```bash
- uv run ruff check abx_dl/
- uv run ruff format abx_dl/
- uv run mypy abx_dl/
+ set -Eeuo pipefail
+ uv run --no-sync --no-sources prek run --all-files
```
- ## Rolling a New Release
-
- 1. **Bump version in pyproject.toml**
- ```bash
- # Edit version = "X.Y.Z" in pyproject.toml
- ```
-
- 2. **Commit, tag, and push**
- ```bash
- git add pyproject.toml
- git commit -m "Bump to vX.Y.Z"
- git push
- git tag -a vX.Y.Z -m "vX.Y.Z"
- git push origin vX.Y.Z
- ```
-
- 3. **Create GitHub release**
- ```bash
- gh release create vX.Y.Z --title "vX.Y.Z" --notes "Release notes here"
- ```
-
- 4. **PyPI publish** happens automatically via GitHub Actions when a tag is pushed (requires trusted publisher configured on PyPI)
-
- ## CI/CD Workflows
-
- - `.github/workflows/test.yml` - Parallel plugin tests on PR/push to main
- - `.github/workflows/publish.yml` - PyPI publishing on tags/releases
-
- ## Key Architecture Notes
+ ## Releases
- - **Plugins with Crawl hooks** (like chrome) self-install their dependencies - the executor skips pre-checking binaries for these
- - **Background hooks** (`.bg.js` suffix) run as daemons and are cleaned up via PID files at the end
- - **Binary discovery**: Hooks output `{"type": "Binary", "name": "...", "abspath": "..."}` to register binary paths
+ Releases are automatic from `main`. Bump the version and its lockfile, commit,
+ and push. The release workflow waits for the exact commit's test workflow,
+ creates the matching GitHub release and tag, waits for the exact Docker build,
+ and publishes the package through PyPI trusted publishing. Do not create or
+ push tags locally.
- ## Common Issues
+ ## Architecture
- - **Chrome plugin skipped**: Fixed in v1.0.4 - plugins with Crawl hooks skip dependency pre-check
- - **Hook file conflicts**: Each hook writes to `{hook_name}.stdout.log` etc. to avoid conflicts when multiple hooks run in same plugin dir
+ - Plugin discovery comes from the installed `abx-plugins` package or the
+ explicit `ABX_PLUGINS_DIR` override.
+ - Required binaries are emitted as `BinaryRequestEvent` records and resolved
+ only by `abxpkg`; host binaries are considered before managed providers.
+ - Resolved host and managed binaries are projected into
+ `ABXPKG_LIB_DIR/env/bin`. `ABXPKG_LIB_DIR/bin` is only a human convenience
+ surface and must never be used programmatically.
+ - Crawl setup and snapshot hooks run through the unified event state machine.
+ Chrome readiness uses its plugin-owned file mechanism rather than a generic
+ bus-level readiness event.