AGENTS.md · diff

git:20260501.5824c48 to git:20260815.2c8eba9

166 added, 27 removed. Audit A to A.

# AGENTS.md

- Repo-local agent instructions live in [AGENT_RULES.md](./AGENT_RULES.md).

+ Entry point for any coding agent working in this repository. The full

+ rulebook is [AGENT_RULES.md](./AGENT_RULES.md); this file is the short

+ orientation plus the commands you will actually run.

- Any coding agent working in this repository must:

+ ## Before anything else: build the local index

+ A `git clone` does not give you a working library. Two retrieval artifacts are

+ deliberately gitignored, and `.claude/commands/` is not committed either:

+ ```bash

+ python3 scripts/bootstrap.py

+ ```

+ Without this, `scripts/search_knowledge.py` returns zero results for every

+ query **and still exits 0** — which looks like an empty library rather than a

+ missing index. `pipelines/lexical_index.search_index` returns `[]` when

+ `vector_index/lexical.sqlite` is absent; nothing errors. Since step 1 of every

+ skill workflow is "search local coverage first", a skipped bootstrap silently

+ turns that gate into a rubber stamp.

+ `bootstrap.py --verify-only` re-runs only the checks (sub-second) and prints

+ whether retrieval and the slash commands are live. It writes no tracked files.

+ See `docs/installing.md`.

+ ## Rules every agent follows

1. Read `AGENT_RULES.md` before creating or materially revising a skill.

2. Treat `SKILL.md` frontmatter as the canonical skill metadata source.

- 3. Apply `standards/skill-content-contract.md` (what to say) and `standards/skill-authoring-style.md` (how to say it) when authoring or editing SKILL.md content.

- 4. Use `python3 scripts/search_knowledge.py` before creating a new skill or claiming a coverage gap.

- 5. Run `python3 scripts/skill_sync.py` and `python3 scripts/validate_repo.py` after skill changes.

- 6. Never hand-edit generated files in `registry/`, `vector_index/`, or `docs/SKILLS.md`.

+ 3. Apply `standards/skill-content-contract.md` (what to say) and

+ `standards/skill-authoring-style.md` (how to say it) when authoring or

+ editing SKILL.md content.

+ 4. Use `python3 scripts/search_knowledge.py` before creating a new skill or

+ claiming a coverage gap.

+ 5. Run `python3 scripts/skill_sync.py` and `python3 scripts/validate_repo.py`

+ after skill changes.

+ 6. Never hand-edit generated files.

+ ### What "generated" means, precisely

+ Do not hand-edit: everything under `registry/`, `docs/SKILLS.md`,

+ `docs/queue-progress.md`, `standards/validation-gates.md`,

+ `.claude/skills/**/references/skill-index.md`, and the retrieval artifacts

+ `vector_index/chunks.jsonl` / `lexical.sqlite` / `*embeddings.jsonl` (all

+ gitignored).

+ `vector_index/` is **not** wholesale generated. Three files in it are tracked

+ and hand-maintained:

+ ```

+ $ git ls-files vector_index

+ vector_index/manifest.json

+ vector_index/query-fixtures.json

+ vector_index/query-variants.json

+ ```

+ `query-fixtures.json` is where you add the retrieval fixture every new skill

+ needs (see AGENT_RULES.md step 5). `manifest.json` is written by the sync

+ engine — leave it alone.

+ ## Syncing: pick the right flag

+ | Command | When |

+ |---|---|

+ | `python3 scripts/skill_sync.py --skill skills/<domain>/<name>` | After editing one skill. Validates that skill first, then writes artifacts. |

+ | `python3 scripts/skill_sync.py --all --skip-embeddings` | Repo-wide rebuild during normal work. |

+ | `python3 scripts/skill_sync.py --all` | Only when you intend to rebuild embeddings. |

+ `config/retrieval-config.yaml` has `embeddings.enabled: true`, so a bare

+ `--all` will attempt a chunk-level encode. The config records the measured

+ cost: "First-time build is ~2:20 on M-series CPU", and `.githooks/pre-commit`

+ notes the full encode "takes ~3 hours" when the content-hash cache is cold.

+ That is why the pre-commit hook passes `--skip-embeddings` and why you should

+ too, unless rebuilding embeddings is the point. Rebuild them deliberately with

+ `python3 scripts/build_index.py`.

## Working on agents (not skills)

- Agents live under `agents/<slug>/AGENT.md`. Before editing or adding one:

+ Agents live under `agents/<slug>/AGENT.md`. There are 76 of them —

+ 48 active run-time, 14 build-time, 14 deprecated run-time stubs — plus

+ `agents/_shared/`, which is shared material, not an agent. Before editing or

+ adding one:

1. Read `agents/_shared/AGENT_CONTRACT.md` for the frontmatter spec, required

section order, confidence rubric, and structured citation/output formats.

2. For skill-builder agents, also read `agents/_shared/SKILL_BUILDER_CORE.md`.

- 3. Reuse probes from `agents/_shared/probes/` instead of hand-rolling MCP

- queries.

- 4. Use refusal codes from `agents/_shared/REFUSAL_CODES.md` in output envelopes.

+ 3. Reuse probes from `agents/_shared/probes/` (7 files) instead of hand-rolling

+ MCP queries.

+ 4. Use refusal codes from `agents/_shared/REFUSAL_CODES.md` in output

+ envelopes.

- After editing any `AGENT.md` run:

+ After editing any `AGENT.md`:

```bash

python3 scripts/validate_repo.py --agents

```

- `validate_repo.py --all` runs both skill and agent validation (slow; skill

- compilation + query fixtures). Use `--agents` or `--skills-only` for fast

- iteration.

+ Measured at 0.4 s across all 76 agents. `--skills-only` (the default) is the

+ slow half: a full skill pass is roughly 2 minutes per quarter-shard. `--all`

+ runs both.

- ## Install the git hooks (strongly recommended)

+ ### Known gap in the agent contract — do not paper over it

+ `AGENT_CONTRACT.md` line 132 tells every agent to resolve skills through the

+ MCP `search_skill` tool ("Skill-first, never freestyle"). In practice:

+ - Only 3 of 76 `AGENT.md` files mention `search_skill` at all

+ (`audit-router`, `automation-migration-router`, `release-planner`), and none

+ of the other 45 active run-time agents issues it.

+ - `.claude-plugin/plugin.json` ships `skills` and `commands` only. It declares

+ no `mcpServers`, and there is no `.mcp.json` in the repo, so installing the

+ plugin does **not** wire the MCP server. Most users do not have

+ `search_skill`.

+ - Both routers gate a hard refusal on it: audit-router step 1 says "at runtime

+ the router still verifies via `search_skill` … If any citation is

+ unresolvable, STOP with `REFUSAL_NEEDS_HUMAN_REVIEW`". A caller without the

+ MCP server cannot satisfy that check.

+ Treat the contract line as an aspiration for MCP-connected callers, not as

+ something the shipped path performs. When you edit a router, prefer a refusal

+ condition a plugin-only caller can actually evaluate — the same citation set is

+ already enforced statically by `_validate_citations` in

+ `pipelines/agent_validators.py` at PR time.

+ ## Install the git hooks (recommended)

One-time setup per clone:

```bash

python3 scripts/install_hooks.py

```

- This configures `core.hooksPath` to `.githooks/` and installs two hooks:

+ This sets `core.hooksPath` to `.githooks/` and installs two hooks:

- - **`pre-commit`** — runs `skill_sync --changed-only` and `validate_repo --changed-only`. Fast (<5s).

- - **`pre-push`** — runs the full 4-shard validation CI runs. Catches cross-cutting drift (stale registries, schema-enum drift, missing query fixtures) that `--changed-only` can't see. 10–20s.

+ - **`pre-commit`** — `skill_sync --changed-only --skip-embeddings` then

+ `validate_repo --changed-only`. Fast on small commits.

+ - **`pre-push`** — `validate_repo --skills-only --shard N/4

+ --skip-fixture-retrieval` for each of 4 shards, sequentially. Catches

+ cross-cutting drift (stale registries, schema-enum drift, missing query

+ fixtures) that `--changed-only` cannot see.

- Bypass for WIP: `git commit --no-verify` / `git push --no-verify`. CI still gates merge.

+ **The pre-push hook is slow.** Its own banner says "~10–20s"; measured on this

+ corpus (1,027 skills) a single shard is 1 m 58 s, so the four sequential shards

+ are roughly 8 minutes. Budget for that, or push with `--no-verify` on WIP

+ branches — CI still gates merge.

+ `install_hooks.py` takes no arguments and performs the install when run, so

+ there is no `--help` to inspect; it is idempotent.

## Reading / updating the skill queue

- `BACKLOG.yaml` is the authoritative machine-readable queue (the row data formerly

- inside `MASTER_QUEUE.md`, migrated 2026-05-01). Do not hand-edit it — use:

+ `BACKLOG.yaml` is the authoritative machine-readable queue (646 entries), the

+ row data that used to live inside `MASTER_QUEUE.md` before the 2026-05-01

+ migration. Do not hand-edit it — use:

```bash

python3 scripts/queue_reader.py --summary

python3 scripts/queue_reader.py --next --status TODO,RESEARCHED

python3 scripts/queue_reader.py --set-status IN_PROGRESS \

--id <entry-id> --actor "<agent-name>@<host>"

```

+ Recognised statuses (`scripts/queue_reader.py` `STATUSES`): `TODO`,

+ `RESEARCHED`, `RESEARCH`, `IN_PROGRESS`, `DONE`, `DUPLICATE`, `BLOCKED`,

+ `UPDATE`, `SHIPPABLE`. `DONE` is accepted for transitional updates but the

+ dashboard ignores it — the filesystem under `skills/` is authoritative for

+ "is this skill built?".

The generated dashboard at `docs/queue-progress.md` shows status counts, drift

between queue and disk, oldest TODO, and the next 10 picks. It regenerates on

every `skill_sync.py` run.

- `MASTER_QUEUE.md` is now a short prose intro + agent workflow doc. The

- historical row data is preserved in git history and reproducible from a

- pre-migration `MASTER_QUEUE.md` via `scripts/_migrations/migrate_queue_to_yaml.py`.

+ `MASTER_QUEUE.md` is now a short prose intro plus queue-specific agent

+ workflow. The pre-migration table is recoverable from git

+ (`git show 5824c4801^:MASTER_QUEUE.md`) and reproducible via

+ `scripts/_migrations/migrate_queue_to_yaml.py`.

+ **Stale pointers to be aware of:** nine build-time agents still describe

+ reading or writing TODO rows in `MASTER_QUEUE.md` (`grep -l MASTER_QUEUE

+ agents/*/AGENT.md` → the 6 skill-builders plus `orchestrator`, `task-mapper`,

+ `currency-monitor`), and `commands/request-skill.md` step 5 still tells the

+ caller to add a table row there. Those tables no longer exist. Route queue

+ reads and writes through `queue_reader.py`.

+ ## Changing skill descriptions? Check the shipped roster

+ Only `.claude/skills/salesforce-<domain>/references/skill-index.md` ships to a

+ plugin install — a GitHub-sourced install has no FTS5 index and no embeddings.

+ A term that appears in a skill body but not in its 220-character gloss reaches

+ zero users, silently.

+ ```bash

+ python3 scripts/check_gloss_coverage.py <term> --domain <domain>

+ ```

+ Exits 1 when packages mention the term but do not route on it, so it can gate a

+ wave. Run it after any change to skill `description` or `triggers`.

## Running agent evals

```bash

- python3 evals/agents/scripts/run_agent_evals.py --structure # lint fixtures

- python3 evals/agents/scripts/run_agent_evals.py --agent field-impact-analyzer --output path/to/output.json

+ python3 evals/agents/scripts/run_agent_evals.py --structure

+ python3 evals/agents/scripts/run_agent_evals.py \

+ --file evals/agents/fixtures/field-impact-analyzer/case-rename-billingcity.yaml

+ python3 evals/agents/scripts/run_agent_evals.py \

+ --file <fixture.yaml> --grade --envelope <produced-envelope.json>

```

- Fixture format lives in `evals/agents/framework.md`.

+ `--structure` lints all 53 fixtures. The runner makes no network or model

+ calls: the agent runs in the caller's model and you hand the envelope it

+ produced to `--grade`. The only flags are `--structure`, `--file`, `--grade`,

+ `--envelope`. Fixture format lives in `evals/agents/framework.md`.

+ Skill-output evals are separate: `python3 evals/scripts/run_evals.py

+ --structure` lints the 10 flagship files under `evals/golden/`.