AGENTS.md git:20261001.de36cdfA

Immutable. This exact content never changes and is served at /api/v1/blob/415f41c7527b99d7.

# claude-jev

A Claude Code plugin. Five hooks send small judgments to TypeSafe's Jev, a
System One model that returns typed answers instead of text. `README.md`
explains what each hook decides and why. Read it before changing behavior.

## Vision

`VISION.md` is the acceptance policy for what this plugin becomes and refuses to
become. Check a non-trivial change against its align/resist pair before
building it, and cite it when resisting.

## Layout

| Path | What lives there |
|---|---|
| `scripts/jev.py` | API client and CLI. Every other script imports it. |
| `scripts/prompt_router.py` | Measurement reference for the routing eval; production runs `src/prompt-router.ts` under node type stripping |
| `scripts/rules.py` | `PostToolUse` on edits, `PreToolUse`/`PostToolUse` on Bash to record shell writes, and `Stop` — rule enforcement |
| `hooks/register.ts` | Experimental function-hooks module: `session.compact` -> `node --experimental-strip-types src/compactor.ts rows`, and the `/claude-jev` settings pane. A bridge, not a second implementation. |
| `plugin.json` | Agent Plugins 1.0.0 identity. Claude Code still loads `.claude-plugin/plugin.json` and `hooks/`. |
| `src/` | The TypeScript hook implementations the plugin runs, executed directly as source under node type stripping (needs node >= 22.18) |
| `src/compact/strategy.ts` | Shared compaction strategy: checks, thresholds, `selectBlocks`. `src/compactor.ts` is the Claude rows bridge. |
| `adapters/afk/` | The AFK host adapter: a TypeScript implementation of the same hooks with its own manifest, `hooks.json`, and README. Its known-gaps list is the contract — do not claim parity that table does not state. |
| `adapters/pi/` | The Pi adapter. `package.json` `pi.extensions` points at `adapters/pi/jev.ts` (`session_before_compact` and `/jev`). Selection imports `src/compact/strategy.ts`. Block shaping, the `<read-files>` index, the 14k+2k split, and Pi logs stay in the adapter. |
| `scripts/comparators.py` | ast-grep lookups the rule hook adds to a judgment |
| `scripts/observed.py` | Scores what a past turn actually did |
| `scripts/stats.py` | The Stats row in `/claude-jev`, or `python3 scripts/stats.py` — scores live decisions from the three logs under `~/.claude`: `jev-router-log.jsonl` (router, subagent, rules), `jev-compact-log.jsonl`, `jev-calls.jsonl` (every API call, written by `jev.ask`) |
| `scripts/check_no_comments.py` | Hard ban: fails if `scripts/`, `eval/`, or `hooks/` source has a `#` / `//` / `/* */` comment |
| `eval/` | Offline measurement. See `eval/README.md`. |
| `.claude/skills/` | Maintainer skills for this repository, not shipped. `release` cuts a version: changelog section to bump, tag, push, GitHub release. |
| `hooks/hooks.json` | Hook registration. New hook means an entry here. `modules` names the function-hooks module; older Claude Code ignores the key. |

## Invariants

- **Hooks fail open.** A failure never blocks or alters a session. Command
  hooks exit 0 and print nothing on any error, missing key, or timeout; keep
  the `except Exception: return` at the top of every hook `main`. The `rows`
  bridge answers `{"fallback": ...}` on stdout and exits 0, and
  `hooks/register.ts` then calls `next(e)` so the built-in summary runs.
- **The one external program is ast-grep**, pinned by version and sha256 in
  `scripts/comparators.py`, fetched to `~/.claude/jev-bin` by a detached
  process outside the hook's budget, and never required: every comparator
  answers `""` without it, and the judgment proceeds as before.
- **Standard library only, per host.** `scripts/` and the Python files in
  `eval/` are Python 3 standard library only: no third-party imports, and
  `urllib.request` is the HTTP client.
- The hooks in `src/`, the shared modules in `adapters/afk/src/shared/`, and
  `eval/compact.ts` are TypeScript with zero runtime dependencies. They use
  Node's standard modules only; `typescript`, `oxlint`, and `@types/node` are
  devDependencies. The shipped plugin runs them as source under node type
  stripping, with no build step.
- `hooks/register.ts` holds no judgment. It runs
  `node --experimental-strip-types src/compactor.ts rows`, falls through to
  `next(e)` on failure, and draws the `/claude-jev` pane. Its debug-log line
  is the `summary` string the compactor sends, and the pane's key source,
  provider, and last call come from `python3 scripts/jev.py status`.
- **One key variable per provider:** each entry in `PROVIDERS`
  (`scripts/jev.py`) names its URL, key prefix, and variable
  (`TYPESAFE_API_KEY`, `OPENROUTER_API_KEY`). A new provider is a new entry
  there; do not add any other variable. The `provider` `userConfig` field pins
  one; on `auto` the variables are read in `PROVIDERS` order and the key's
  prefix picks.
  Every other tunable is a module-level constant, or a `userConfig` field in
  `.claude-plugin/plugin.json` when the user sets it from `/claude-jev` or
  `/config`. Claude Code hands those fields to hooks as
  `CLAUDE_PLUGIN_OPTION_<FIELD>`, its own variables, read only through
  `jev.plugin_option()`; `hooks/register.ts` passes the saved key to
  the compactor under the same name.
  `CLAUDE_CONFIG_DIR` is Claude Code's own variable, not a plugin tunable:
  `jev.config_dir()` honors it and every path under the user's config
  directory goes through that helper, never through a literal `~/.claude`.
- **No code comments.** Line, block, and JSDoc comments are banned in
  `scripts/`, `eval/`, and `hooks/` source (`.py`, `.ts`, `.js`). Shebangs
  and LICENSE text stay. Python module/function docstrings are documentation
  strings, not comments, and stay where Conventions require them. Markdown
  under `docs/`, `README.md`, and `AGENTS.md` is prose, not code
  comments — leave it. Rationale for thresholds and constants (`ACT`,
  `MIN_CONFIDENCE`, `KEEP_THRESHOLD`, and the rest) lives in eval results,
  docs, or PR evidence — not inline comments. Changing a threshold without
  eval evidence is still a guess. Enforce with
  `python3 scripts/check_no_comments.py` (exit 1 on any hit).
- **`scripts/observed.py` is the shared scorer.** `eval/replay.py` and
  `scripts/stats.py` both call it. Editing it moves every accuracy number in
  `README.md`.
- **Probes pay for live calls.** `jev.ask` is assigned in exactly one
  place, the `cached_ask` wiring in `eval/rules_eval.py`'s `cmd_run`, which
  wraps the real client with the cache. Eval and hook verification never
  stubs, mocks, or monkeypatches the ask path — a stub proves wiring, not
  behavior — and cost is bounded by shrinking the sample, never by faking
  the client. Enforce with `python3 scripts/check_no_stubs.py` (exit 1 on
  any `.ask =` assignment outside the sanctioned wiring).
- Hook scripts import siblings through `sys.path.insert(0, dirname(__file__))`.
  Keep that, because Claude Code runs them from arbitrary directories.

## Conventions

- Each script's module docstring states the hook it serves, the fail-open
  contract, and the env var. Keep that shape when you add one.
- Question definitions live in `scripts/jev.py` (`intent_bundle`,
  `subagent_bundle`) or next to the hook that asks them. Put the meaning in
  the `instructions` and `criteria` text — Jev never sees the key names.
- Write rules in this file as bullets, one instruction each. `rules.py` parses
  instruction files bullet by bullet, and a rule buried inside a prose
  paragraph classifies poorly.
- The user-facing entry point is the `/claude-jev` pane in `hooks/register.ts`.
  There is no `skills/` or `commands/` directory; do not add one. A new
  setting or report is a row in that pane.
- `.agents/skills/` is canonical for shared agent skills; symlink them into tool `skills/` dirs, never copy.

## Verify

There is no test suite. The local / script contract for this repo is:

```bash
python3 -m compileall -q scripts eval
python3 scripts/check_no_comments.py
python3 scripts/check_no_stubs.py
echo '{"prompt":"hi","transcript_path":""}' | CLAUDE_CONFIG_DIR="$(mktemp -d)" python3 scripts/prompt_router.py; echo "exit=$?"
```

`compileall` plus stdin hook smoke (exit 0, fail open) is what a default
cloud agent or Grok Bot run can claim. Ruff is a maintainer check, required
for a release and for CI, and not part of that python3-only claim:

```bash
ruff format --check .
ruff check .
```

`ruff.toml` configures it. It is not a dependency file: no hook imports
Ruff.

The AFK adapter has its own contract, and claiming it requires running it:

```bash
cd adapters/afk && npm ci && npm run build
echo '{"session_id":"test","cwd":"'$(pwd)'"}' | node dist/session-start.js; echo "exit=$?"
```

`npm ci && npm run build` is what a doc-only run can claim for
`adapters/afk/`; the second line is its no-key hook smoke, and the keyed
hooks are not part of any default claim. Live Claude Code session verification is out-of-band: it needs a
machine with `claude` and `TYPESAFE_API_KEY`, and is not claimed as proved
by those local checks alone.

For a scripted launch → doctor → drive → evidence → cleanup loop (isolated
`HOME`, feature map under `.cursor/skills/verify-claude-jev/features/`), use
the project-local Cursor skill:

```bash
.cursor/skills/verify-claude-jev/bin/control-jev launch
.cursor/skills/verify-claude-jev/bin/control-jev doctor
# then drive one feature from features/; evidence under
# .cursor/skills/verify-claude-jev/artifacts/<RUN_ID>/
.cursor/skills/verify-claude-jev/bin/control-jev cleanup
```

See `.cursor/skills/verify-claude-jev/SKILL.md`. Keep the map honest with
`/maintain-verification-skill` when hooks or skills change.

`python3 scripts/comparators.py which` prints the ast-grep the rule hook
would run, or `(none)`; `fetch` downloads the pinned one. The hook must exit
0 within its budget either way — check with the binary renamed away.

Every hook must exit 0 on a malformed or empty event. Feed the script you
changed a matching JSON event on stdin and check the exit code.

- Run a hand-fed hook event with `CLAUDE_CONFIG_DIR="$(mktemp -d)"`. Hooks
  log under the config directory, and a test event in the real one lands in
  the `/claude-jev` stats with no transcript behind it.

The `rows` bridge answers bad input with `{"fallback": ...}` and exit 0:

```bash
echo '' | node --experimental-strip-types src/compactor.ts rows
```

Out-of-band (not claimed on Grok Bot / default cloud agents): exercise the
function-hooks module end to end on a machine with Claude Code and a real
key — load the plugin from disk, compact, and read the debug log:

```bash
export CLAUDE_CODE_ENABLE_FUNCTION_HOOKS=1
claude -p "..." --session-id "$ID" --plugin-dir "$PWD"
claude -p "/compact" --resume "$ID" --plugin-dir "$PWD" -d
grep 'jev-compact\|core never ran' ~/.claude/debug/"$ID".txt
```

"a hook's N messages stand ... core never ran" means the summary was
replaced; "built-in summary runs" means the bridge fell through.

To measure a routing or threshold change, run the relevant eval and compare
against the table in `README.md`:

```bash
python3 eval/replay.py run --variant v7_no_unclear --sample 250
python3 eval/rules_eval.py run --sample 250 --seed 0
python3 eval/rules_eval.py report --sweep
node --experimental-strip-types eval/compact.ts compact --synth 60 --seed 0 --workers 8 --max-inflight 12
```

The compaction command is the only way to measure the TypeScript compactor.
It prints re-fetch coverage and planted-constraint survival in one gate and
exits 2 below either floor, so a wording that keeps paths but drops what the
user said, or the reverse, cannot pass on one number. It also fails on any
failed backend chunk: the compactor is fail-open, so a chunk that never
answers leaves its blocks unscored, and unscored rows are kept, which raises
coverage.

`--seed 0 --sample 250` selects the same real edits as the numbers in
`README.md`; keep it when comparing. A rule-question wording change misses
the answer cache and re-costs the run.

- `run --out` moves the answer file, not the cache, so a re-score times the cache
  and its latency row is meaningless. Set `JEV_RULES_CACHE` to a fresh path to
  re-ask every question; the live 247-edit pass is ~320 calls and ~40s.

The rules eval judges each record at its `sha`, and reads
`eval/global_CLAUDE.md` in place of `~/.claude/CLAUDE.md`. Change a case's
sha only when a rule-file change in that repo is the thing being measured.

Second decision models: name them `backend:model` (bare id = System One) in
`--decision-model` / `--judge-model`; unknown backends raise, and caches key
on model so providers are interchangeable. The decisions endpoint 503s above
~24 requests in flight, so pair `--workers 8 --max-inflight 12`. When the
TypeSafe account is out of credits, pin
`CLAUDE_PLUGIN_OPTION_PROVIDER=openrouter`: provider order does not fall
through.

These call `api.typesafe.ai` with real past prompts and cost money. Ask before
running a full sweep.

## Data and numbers

- `eval/observed/` and `eval/authored/` are gitignored. They hold extracted
  transcripts and repo-specific cases, and they do not survive a clone. Do not
  commit them or write code that assumes they exist.
- Numbers in `README.md` and `eval/README.md` come from eval runs. Change one
  only with a run behind it, and say which run.
- Bump `version` in `.claude-plugin/plugin.json` and root `plugin.json` together
  for a behavior change, and add the change under `## [Unreleased]` in
  `CHANGELOG.md`. Those two versions stay the same; `/release` writes both.
  The AFK adapter carries its own version in `adapters/afk/.claude-plugin/plugin.json`,
  mirrored in its `package.json`, and bumps by the same rule. Releases go
  through `/release` (`.claude/skills/release`), which refuses an empty
  section.