git:20260821.b88b609 to git:20260822.30d0abb

3 added, 2 removed. Audit A to A.

# Project Overview (for AI Agent)

## Goal

Provide the `modlens` CLI tool that converts image sources (local path or remote URL) into structured text evidence for non-vision LLM workflows.

## Scope

The contract: an image the user hands over is read once, and that read leaves text later turns can quote. The image is in the conversation because the user pasted it, dropped a path, or gave a URL.

Do not add:

- a camera, screenshot capture, or hotkeys

- CDP, or holding a browser session

- computer-use (screen plus keyboard and mouse)

- a pixel toolbox (grounding, crop, pixel-diff, reconstructing a UI)

- bounding boxes or confidence scores (dropped from the schema on purpose)

Visual parsing is the only job. Web search and page fetching live in `modsearch`.

## Technical Approach

- **Six vision providers behind one interface** (`src/providers/index.ts`). Subprocess providers implement `buildInvocation` + `parseOutput` (antigravity-cli, claude-cli, kimi-cli); in-process API providers implement `execute` (gemini-api, openai, anthropic). `antigravity-cli` is the zero-config default, and `kimi-cli` runs only when named, since it spends a subscription.

- **Schema-enforced JSON output** wherever the backend allows: `--json-schema` on the Claude and Antigravity CLIs (kimi-cli has no such flag and uses the template), `responseJsonSchema` on gemini-api, a forced tool call on anthropic. The openai route uses a template-instance prompt (weak gateways echo raw schemas back) plus shape validation that fails loudly.

- - **Layered config**: CLI flags > `~/.modlens/config.json` (managed by `modlens config init/set/show`, 0600, masked rendering) > built-ins. Since 3.17.0 a provider's settings come from one source, whole: the file when it mentions that provider, the bound environment variables (`GEMINI_API_KEY`, `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`) when it does not. They used to merge field by field, and an endpoint and a key are one credential, so drawing halves from two places built a pairing that existed in neither. `MODLENS_MODEL`, `MODLENS_HARNESS` and the proxy conventions are unaffected.

+ - **Layered config**: CLI flags > `~/.modlens/config.json` (managed by `modlens config init/set/show`, 0600, masked rendering) > built-ins. Since 3.17.0 a provider's settings come from one source, whole: the file when it mentions that provider, the bound environment variables (`GEMINI_API_KEY`, `GEMINI_BASE_URL`, `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`) when it does not. They used to merge field by field, and an endpoint and a key are one credential, so drawing halves from two places built a pairing that existed in neither. `apiKey` (file or env) accepts a comma-separated list and rotates after auth, rate-limit, or quota failures. Quota cooldown state lives in `~/.modlens/state.json`. `MODLENS_MODEL`, `MODLENS_HARNESS` and the proxy conventions are unaffected.

- **Vendor knobs pass through, they are not modelled**: `<provider>.extraBody` (or `--extra-body`) deep-merges a JSON object into the request body of the three API providers, which is how thinking gets turned off. No per-vendor table lives in the code, because the spelling differs per gateway and a wrong guess either 400s or is ignored silently. The fields carrying the image, the prompt, and the schema are reserved (`src/util/extraBody.ts`).

- **Paste recovery across harnesses**: `modlens recover-paste` pulls pasted image bytes out of local session storage (pastes never hit a regular temp file). It supports Claude Code and Pi (JSONL transcripts) and OpenCode (SQLite), detects Codex and defers to its on-disk temp files, and scopes to the harness it runs inside via process ancestry. Exact targeting via `--session`, else newest-image-timestamp scanning. Storage layouts are each harness's internals, so treat this as best-effort.

- **Single responsibility**: visual parsing only. Web search and page fetching live in `modsearch`.

```bash

pnpm install

```

## Code Organization

```

src/

- ├── main.ts # CLI entry: analyze (default), guard, recover-paste, doctor, config subcommands

+ ├── main.ts # CLI entry: analyze (default), guard, recover-paste, doctor, config, state subcommands

├── analyzer.ts # orchestration: input resolution, config merge, provider dispatch

├── config.ts # layered config load/set/show/init

+ ├── cooldown.ts # quota cooldown store at ~/.modlens/state.json

├── doctor.ts # offline diagnostics: Node, provider readiness, selection, harness, config perms

├── prompt.ts # vision prompt (local/remote agent modes + inline api mode)

├── schema.ts # vision result JSON schema (single source of truth)

├── imageInput.ts # base64/mime helpers (local + remote image bytes)

├── util/json.ts # shared JSON helpers (parse, extract, truncate)

├── guard/ # invocation guard (issue #15): rules (deny globs), modelSniff (active model from session storage), index (signal precedence)

├── recoverPaste/ # paste recovery: adapters/{claude,pi,opencode}, detect, jsonl, index

└── providers/

├── index.ts # provider interface + registry (5 providers + aliases)

├── antigravity.ts # agy subprocess provider

├── claudeCli.ts # claude subprocess provider (Read-only tools)

├── geminiApi.ts # Gemini Developer API

├── openaiCompat.ts # any OpenAI-compatible multimodal endpoint

└── anthropicApi.ts # Claude API (forced tool call)

```

Tests are co-located: modules get an adjacent `*.test.ts` (vitest), the CLI assembly in `main.ts` included. A few pure-data modules such as `schema.ts` carry none, since there is no behavior to pin. The CLI is exposed via `dist/main.js` (vite lib build, Node built-ins auto-externalized).

## Skills Directory

```

skills/modlens/

├── SKILL.md # triggering + per-harness path finding + workflow

└── references/

├── output-schema.md # output contract

└── configure.md # per-provider setup recipes the agent can execute

```

## CLI Usage

```bash

modlens -i screenshot.png # default provider (antigravity-cli)

modlens -i screenshot.png -p gemini-api # fastest free route (5-10s)

modlens recover-paste --session <uuid> # Claude Code pasted-image recovery

modlens doctor # offline config/routing diagnosis (--json for machine output)

modlens config show

```

## Verification

- `pnpm typecheck && pnpm test` for unit-level checks; `pnpm build` must produce a single `dist/main.js`.

- Real end-to-end runs consume the user's provider quota (agy, API keys, Claude subscription). Ask before running them in bulk.

## Evals

- Every experiment leaves a reproducible artifact. `evals/` holds seed cases (`evals/cases/<id>/case.json`) and a runner (`pnpm eval`, or `pnpm eval --dry-run` to validate without a provider call). Each live run writes one evidence artifact per case to `evals/results/<date>/` (git-ignored), recording the command, tool version and commit, provider and model, input SHA-256, raw output, expected points and scoring, latency, usage, and any error or degradation. Format and fields are documented in `evals/README.md`.

- Evals are local and on-demand: they spend real quota and never run in CI. The runner only checks containment and schema shape; whether a model obeyed a prompt injection is a human read of the artifact.

## Operational Docs (`docs/`)

1. Operational docs use front-matter metadata (`summary`, `read_when`).

2. Before creating a new doc, run `pnpm docs:list` to review the existing index.

3. Existing docs: `troubleshooting` (every error this CLI prints, with causes and fixes), `harness-setup` (how a pasted image reaches the model in each harness), `security` (recovered-image privacy, permissions passed to engines, untrusted image content), `testing`, `commit`, `research-gemini-claude-skills` (historical, Gemini CLI era).

## .gitignore must include

- `node_modules/`

- `dist/`

- `skills/**/outputs/`

- common logs/cache/system files