CLAUDE.md · diff

git:20261003.b442173 to git:20261003.99158f2

2 added, 0 removed. Audit A to A.

# job-search plugin — contributor guide
This repo is the **engine** (a Claude Code plugin). It must never contain anyone's personal data: no real names,
resumes, emails, sheet ids, connector UUIDs, or real run output. Personal data lives in a user's data folder
(`JOB_SEARCH_HOME`, found by walking up from the cwd to a folder with `config.yaml` + `data/`).
## Layout
- `skills/job-search/SKILL.md` — the orchestration Claude follows; `references/` loaded on demand.
- `.claude-plugin/plugin.json` (`mcpServers`) — starts `scripts/mcp_server.py` (stdlib MCP core; `mcp_tools.py` tools,
`mcp_content.py` resources and prompts). New tools register with `@tool` in `mcp_tools.py`; mock the network in tests.
+ `homes.py` lists the data folders `setup.py link` registered (`$XDG_CONFIG_HOME/trawlnet/homes`): only those get their
+ `.venv` python run by the server.
- `skills/job-search/scripts/` — stdlib + PyYAML engine. `common.py` (paths, config + pack merge, parsing),
`jobsearch.py` (plan/feeds/filter/decide/shortlist/publish/…), `sources/` (one module per board; see below), `db.py` (SQLite state +
migrations), `tracker.py` (csv/gsheets), `firecrawl.py` (credit budget), `discover.py`, `wwr_verify.py`,
`setup.py` (stdlib only: init/link/env/doctor).
- `sources/` — `__init__.py` holds `BOARDS` (lookup order), `ATS_FETCHERS`, description routing and `fetch_all`; `net.py`
the HTTP helpers tests patch; `board.py` the `Board` record each board module declares as `BOARD`; `generic.py` the
custom career-page scraper.
- `templates/` — rendered into a data folder by `setup.py` (config, preferences, CLAUDE.md, agents).
- `packs/<cc>.yaml` — all country specifics. `seeds/` — discovery seed lists + blocklist.
- `examples/data-folder/` — fictional persona; its run is generated by `tests/test_pipeline.py`.
## Rules
- Country-specific strings go in packs, user-specific ones in the data folder's config/preferences — never in code.
- Keep token cost down: deterministic work belongs in scripts; agents read `data/scoring-context.md`, not config.
- Schema changes to `jobs.db`: bump `db.SCHEMA` and add a `MIGRATIONS` step. Never break old data folders
(legacy keys like `sheet:`, `min_inr_lpa`, `accept_any_india_location` are still read).
- New network sources: descriptive User-Agent, pacing, no bot-detection bypass; undocumented endpoints are opt-in.
- Adding a board: a module in `sources/` with `fetch_*` (and `*_description` if the list has no text) and a `BOARD`; add
it to `BOARDS` in `sources/__init__.py`, `jobsearch.SOURCE_RANK`, and (token boards) `discover.py` patterns/validation
and `generic._JOB_HOSTS`. `link` is for documented token boards only; undocumented ones check the opt-in themselves.
- Tests are offline. Before committing: `python -m unittest discover -s tests` and `python tests/pii_scan.py`.
Coverage (needs `pip install coverage`; ~100% of the engine is covered, CI fails under 95%):
`COVERAGE_FILE=$PWD/.coverage coverage run -m unittest discover -s tests && coverage combine && coverage report`.
New engine code needs tests; mock the network (`sources.net.get`, `get_json`, `post_json`, `discover._get`).
- CI runs Python 3.9 and 3.12; 3.9 rejects syntax newer code allows (e.g. a backslash inside an f-string expression).
Before every push also run the suite on 3.9 (`/usr/bin/python3 -m venv <scratch>/py39 && <scratch>/py39/bin/pip install
PyYAML`, then `<scratch>/py39/bin/python -m unittest discover -s tests`); the newest local Python is not enough.
- Regenerate the example run after changing filter/shortlist/publish output:
`JS_WRITE_EXAMPLE=1 python -m unittest discover -s tests -p test_pipeline.py`.
- Regression-check filter changes: run old and new `filter` on the same raw `feeds.jsonl` with an empty seen list
in scratch data folders and diff key → reason maps.
- Data copied from other projects (seeds, lists) needs a compatible license and an entry in `THIRD_PARTY_NOTICES.md`.