CLAUDE.md git:20260922.4cfe2f3B

Immutable. This exact content is served forever at /api/v1/blob/0c30e3ff105f0361.

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Purpose

> Provenance note: llm-redact was developed privately before its public
> v1.0.0 debut; the public version numbering starts there. This document
> and the docs/ tree deliberately avoid the private lineage's internal
> version numbers — phase labels like "R2"/"R4" name that history instead.

llm-redact is an LLM information redactor: it prevents private information from being sent to LLMs by agentic tools. It substitutes placeholders for private information on outgoing requests, then restores the real values in incoming responses, transparently to the agentic tool's user.

## Commands

```bash
uv sync                                        # install deps (incl. dev group)
uv run pytest                                  # all tests (live tests deselected)
uv run pytest tests/test_rehydrate.py -x      # run one file
uv run pytest tests/test_vault.py::test_reverse_lookup   # run one test
uv run ruff check . && uv run ruff format --check .      # lint + format check
uv run ruff format .                           # apply formatting
uv run mypy                                    # strict type-check of src/
uv run llm-redact serve [--config PATH] [--port N] [--session NAME] [--log-format json]  # run the proxy
uv run llm-redact serve --check                # full startup build, no socket: the deploy/reload gate
uv run llm-redact config show [--path]         # effective config as TOML; env overrides named
uv run llm-redact status [--json] [--ca CA --cert C --key K]  # query a running proxy (mTLS flags)
uv run llm-redact sessions list|prune --older-than 90d   # vault lifecycle
uv run llm-redact lookup «EMAIL_001» | --value V         # local secret lookup
uv run llm-redact vault gen-key                # LLM_REDACT_VAULT_KEY value
uv run llm-redact vault set-key                # store the key in the OS keychain (keyring extra)
uv run llm-redact audit verify [--json]        # verify the tamper-evident audit hash-chain
uv run llm-redact audit decrypt FILE [--out P]  # decrypt a downloaded .ndjson.fernet backup object
uv run llm-redact license show|verify [--key K]  # decoded license payload / exit 0 valid, 1 invalid, 2 expired
uv run llm-redact users invite|verify|list|revoke  # named seats (llm-redact-pro); codes via [email] SMTP or --print-code
uv run python scripts/issue_license.py gen-key|sign …  # vendor-side signing (cryptography dev-dep; llm-redact-pro docs/licensing.md)
uv run llm-redact init [--yes --tools claude,codex --vault sqlite]  # setup wizard
uv run llm-redact run [--tools claude,ollama] -- claude -p "hi"  # env-injecting wrapper; ephemeral proxy if none running
uv run llm-redact doctor [--json]              # read-only diagnostics incl. detector-build dry-run; non-zero exit on FAILs
uv run llm-redact service install|uninstall|status [--print-only]  # launchd/systemd user unit
uv run llm-redact plugin install|uninstall|status claude|codex|opencode|cursor [--print-only --force --proxy-url URL --install-proxy]  # agent slash-command plugins
./scripts/oracle_smoke.sh [docker|podman]      # opt-in Oracle RDBMS-vault battery (needs vault-oracle extra + a container engine)
uv run python scripts/render_plugins.py       # re-render plugins/llm-redact + .claude-plugin/marketplace.json from plugin_assets.py (pinned by test)
uv run llm-redact completions bash|zsh|fish    # shell completion scripts
uv run llm-redact fips-check                   # host FIPS posture report
scripts/render_diagrams.sh                     # re-render docs/diagrams/*.mmd -> committed PNGs (needs node; README embeds the PNGs)
uv run --with playwright python scripts/capture_screenshots.py  # re-capture docs/screenshots/*.png (fixture traffic only)
uv run python scripts/capture_plugin_screenshots.py       # re-render docs/screenshots/plugins/*.svg terminal shots (fixture traffic only)
uv run python scripts/history_sweep.py         # audit ALL git history with the production detectors (docs/history-hygiene.md)
uv run python scripts/fake_upstream.py --port 9999 [--mangle]  # fake provider for manual e2e
uv run python -m llm_redact.bench --check      # recall + fp-corpus precision gates
uv run coverage run -m pytest && uv run python scripts/complexity_gate.py --check  # every CC>1 function must be executed (assurance gate; allowlist scripts/complexity_allowlist.py)
uv run python -m llm_redact.bench --latency --check   # + in-process overhead benchmark (CI form)
uv run python scripts/live_smoke.py            # live-API tests (needs keys, costs money)
LLM_REDACT_DOGFOOD=1 uv run python scripts/dogfood_claude.py  # real claude CLI e2e (needs auth, costs money)
./scripts/container_smoke.sh docker            # container e2e (needs a container engine)
```

`pyproject.toml` is the single source of project metadata and tool configuration (ruff, pytest, mypy). Runtime dependencies are deliberately only httpx, starlette, and uvicorn; config/CLI use stdlib (`tomllib`, `argparse`, dataclasses). Do not add pydantic or FastAPI — the proxy must forward unknown JSON fields verbatim, never validate or reshape them.

## Architecture

One request flows: agentic tool → proxy (`proxy.py`) → provider adapter redacts → upstream LLM API → adapter rehydrates → tool.

- **Outbound (redaction)**: `detection/` (regex rules + validators; optional NER behind the same `Detector` protocol via `[detection.ner]` — backends spacy (`ner` extra), gliner, and presidio (`presidio_ner.py`: `PRESIDIO_TYPE_MAP` folds EMAIL_ADDRESS/PHONE_NUMBER/US_SSN/IBAN_CODE/CREDIT_CARD into the built-in placeholder types so one value never gets two token identities; `score_threshold` valid for gliner+presidio, rejected for spacy)) finds values; `redactor.py` resolves overlapping detections (longest/highest-priority wins; NER runs at priority 120 so structured rules win ties) and substitutes vault-issued placeholders. Per-rule modes (`[detection.modes]`): configured by RULE NAME, dispatched by DETECTOR TYPE via `build_modes` (rules sharing a type must agree; unknown names are build-time errors, deferred past `parse_config` like `enabled`). Dispatch happens AFTER overlap resolution — the winner's mode governs. USER DENY STRINGS (`[detection] deny` sugar / `[[detection.deny_strings]]`, `detection/deny.py`): literal substrings, case-insensitive unless flagged, tier 0 — a two-phase `_resolve_overlaps` makes them win EVERY overlap (a rule match merely starting earlier cannot claim the span), they bypass the allowlist, and they ALWAYS redact (modes never apply, even on type collision — structural in `redact_text`, not config). With no deny entries the old sweep runs unchanged (property-tested). Deny values must not contain guillemets; error messages name positions, never values. PER-TYPE ALLOWLISTS (`[detection.allowlist_by_type]` TYPE = [values], `Allowlist.allows_for`): exact values allowed for ONE detector type only; deny (tier 0) bypasses these too (pinned by test). LANGUAGE SCOPING (`[detection] languages`, unset = ALL = historical behavior): national-id rules carry ISO 639-1 tags (`RegexRule.languages`; None = universal — emails/IPs/vendor/credit_card/iban/phone always run); a tagged rule with no overlap is NOT BUILT (`active_rule_names` is the single list build_detectors instantiates AND the editor/status report — `language_inactive_rules`), the NER type-suppression set is computed from the ACTIVE list (a type whose only rule is scoped out stays out), and an enabled NER whose `language` is outside the list is a parse-time ConfigError; codes are lowercased+sorted at parse. Validated national-id rules (ALL grouped-display-form + checksum; bare digit runs never fire): `canadian_sin` (grouped 9 digits, Luhn; only area 8 rejected — area 0 stays allowed because 046 454 286 is the government's own example), `uk_nino` (HMRC grammar + invalid-prefix blacklist {BG,GB,NK,KN,TN,NT,ZZ}; the QQ reserved prefix is excluded by the grammar itself), `aadhaar` (4-4-4, Verhoeff, lead 2-9; the corpus derives Verhoeff from the D5 group law so table typos fail the recall gate), `australian_tfn` (3-3-3, ATO mod-11 — SAME grammar as canadian_sin: SIN wins exact-span ties by registration order, and TFN corpus values lead with 8 so _sin_ok rejects them), `spanish_dni` (DNI/NIE mod-23 control letter; solid form safe because the letter validates; declares no prefilter literals like credit_card), `french_nir` (spaced display form incl. Corsican 2A/2B, mod-97 key; priority 90 because the loose 13-19-digit card grammar covers a spaced NIR and 1/10 of NIRs are Luhn-valid — the tie must go to the NIR-specific grouping), `german_steuer_id` (spaced 2-3-3-3, repeat-structure + ISO 7064 MOD 11,10), `brazilian_cpf` (dotted 000.000.000-00, dual mod-11, all-same-digit rejected), `italian_codice_fiscale` (16-char grammar + day range incl. +40 female form + mod-26 check letter; the corpus transcribes the odd-position table INDEPENDENTLY and RSSMRA85T10A562S pins both; omocodia out of scope), `belgian_nn` (Rijksregisternummer dotted YY.MM.DD-NNN.CC, two check digits = 97 - body mod 97 with the post-2000 '2' century prefix accepted, month/day 0 allowed; 93.05.18-223.61 pins it), `finnish_hetu` (henkilötunnus DDMMYY + century sign + NNN + mod-31 check char; the sign letter makes the solid form safe like codice_fiscale — the corpus builds the check alphabet by FILTERING out G/I/O/Q so a typo fails recall, 131052-308T pins it), `nhs_number` (UK 3-3-4 SPACED display form only, mod-11 with 10-invalid; the space form is disjoint from the phone grammar — the hyphenated form is a phone shape, not NHS; 943 476 5919 pins it), `norwegian_fnr` (fødselsnummer DDMMYY-NNNNN spaced/hyphenated, DOUBLE mod-11 with either check landing on 10 = invalid; weights transcribed independently in the corpus), `korean_rrn` (resident registration number YYMMDD-SBBBBNC hyphenated 6-7 form ONLY, weighted mod-11 + month/day/gender-digit gates; priority 90 like french_nir because the 13-digit hyphenated span is also a valid credit_card grammar and ~1/10 pass Luhn — the RRN grouping must win the exact-span tie), `singapore_nric` (NRIC/FIN [STFG]+7digits+check letter, S/T and F/G share weight-2765432 tables with T/G adding 4; the bracketing letters make the solid form safe, S1234567D pins it). Dutch BSN deliberately absent (customarily bare 9 digits; the 11-test passes 1/11 of random runs). CRYPTO WALLET ADDRESSES (`detection/wallet_checksums.py`, all checksum-vetoed, priority 20): `eth_address` (ETH_ADDRESS, `0x`+40hex — EIP-55 mixed-case keccak-256 checksum; all-lower/all-upper accepted on shape, single-nibble null/burn rejected; keccac-256 is VENDORED because hashlib only ships NIST SHA3 which uses a different pad byte — pinned to the empty/`abc` Keccak vectors), `btc_address` (BTC_ADDRESS, base58check P2PKH `1…`/P2SH `3…`, double-SHA256, version 0x00/0x05, declares no anchor like iban), `btc_bech32` (BTC_ADDRESS shared, `bc1…` segwit — BIP-173 bech32 for witver 0, BIP-350 bech32m for v1+ taproot). The corpus reuses the ENCODE side of the same primitives, so correctness rests on the independent published spec vectors in test_wallet_checksums.py (a self-consistent keccak bug would otherwise pass recall). Vendor tokens are prefix-anchored ONLY — Cloudflare/Vercel have no prefix and are deliberately excluded; `sb_publishable_` and similar non-secret forms are deliberately unmatched; the GitLab routable-token family beyond `glpat-` (`gitlab_token`: glrt/glcbt/gldt/glptt/glagent/glimt/glsoat), `google_oauth_client_secret` (GOCSPX-), `sentry_token` (sntrys_/sntryu_), `xai_key` (xai-), `perplexity_key` (pplx-), `hashicorp_vault_token` (hvs./hvb.), `langsmith_key` (lsv2_pt_/lsv2_sk_), `replicate_token` (r8_), `pinecone_key` (pcsk_), `new_relic_key` (NRAK-), `grafana_service_account` (glsa_…_hex), `jina_key` (jina_), and `telegram_bot_token` (8-10 digit id : 35-char secret, no single anchor) are covered; `aws_access_key_id` matches ASIA (STS temporary) as well as AKIA; the legacy unprefixed GitLab CI_JOB_TOKEN and 64-hex Sentry token stay out (no anchor). NER is MULTI-BACKEND: `[detection.ner] backends = [..]` runs any combination of spacy/gliner/presidio/stanza/hf concurrently (stanza = `stanza_ner.py`, Stanford Stanza `Pipeline(lang, processors="tokenize,ner")`, multilingual, NO confidences like spacy; hf = `hf_ner.py`, transformers `pipeline("token-classification", aggregation_strategy="simple")`, any Hub model, emits confidences so score_threshold applies — `_confidence_backends = gliner/presidio/hf`; both build-dispatched in `build_detectors` and validated in `known_backends`) (legacy single `backend` = one-element form; `[detection.ner.models]` per-backend model overrides — the legacy `model` key applies only when exactly one backend is active; score_threshold rejected only when NO confidence backend is active). PER-TYPE SINGLE SOURCE OF TRUTH: a placeholder type disabled at the rule level is suppressed for NER emissions too (TypeFilteredDetector wraps every NER backend; PERSON has no builtin rule so never suppressed); the dashboard NER card has per-backend checkboxes/models and a live folded-type state line. NER language/model (`[detection.ner] language`, `model`): spaCy pipeline name for spacy/presidio (default en_core_web_sm), HF id for gliner (default urchade/gliner_small-v2.1); presidio wires language through NlpEngineProvider and `analyze(language=...)`; presidio's EmailRecognizer validates TLDs against the public-suffix list, so `.example` addresses score zero (real-analyzer smoke test uses `.com`). HOT-PATH PREFILTER (`regex_rules.py`): rules declare `required` literal CNF (necessary conditions; skipped entirely when unsatisfied — one shared `PreparedText` caches the lowered haystack) and `anchors` (every match starts with the literal → find-then-match via `pattern.match(text, pos)`, which keeps \b/lookbehinds correct; CI anchors fall back to plain finditer when lowering changes string length, e.g. 'İ'). Equivalence is machine-checked: per-rule soundness tests over the recall corpus plus a differential fast-vs-naive suite — a wrong literal declaration is a silent recall bug, so never add one without its generator coverage. credit_card/iban deliberately declare nothing. CUSTOM-RULE VALIDATORS (`detection/validators.py`, `[[detection.custom_rules]] validator=`): a named checksum/format gate resolved in `build_detectors` and hung on the RegexRule's `validator` — `luhn`/`mod97` (accepts plain MOD-97-10 AND the IBAN move-first-4 layout)/`verhoeff`/`jwt` (base64url header+payload decode to JSON objects)/`entropy` (≥3.5 bits/char, ≥16 chars, no whitespace — the loose-pattern-plus-check recipe made user-facing); an unknown name is a build-time ValueError (deferred past parse_config like modes). Custom rules also accept `required` (each literal → a single-alternative CNF clause) and `anchors` passed straight through to RegexRule — a wrong hint is the user's silent-recall risk, opt-in and off by default; all three round-trip through the emitter and the editor view. `warn` counts the type and deliberately forwards the value upstream (no vault write); `block` raises `BlockedRequest` (type only, never the value) which `handle()` turns into a provider-shaped **400** (not 403 — SDK UX) before any upstream contact, on CHAT and REDACT_ONLY routes alike. `jsonwalk.py` applies redaction to every string value in the request body (skipping `STRUCTURAL_KEYS` like `model`/`role`/base64 `data` — but plaintext documents with `source.type == "text"` have their `data` scanned, and the OpenAI list envelope `{"object":"list","data":[...]}` has its `data` array WALKED because those are content items, not base64 — image responses lack the `object:list` marker so their `b64_json` stays skipped) so system prompts, nested content blocks, and tool results are covered without hardcoded body shapes. Oversized redactable bodies are rejected 413 fail-closed (`max_body_bytes`).
- **Vault** (`vault.py`): deterministic — same `(session, type, value)` always yields the same `«TYPE_NNN»` token (format in `placeholders.py`); the mapping never leaves the machine. `InMemoryVault` (default) dies with the process; `SqliteVault` (`[vault] backend = "sqlite"`, `--session`) persists across restarts with 0600/0700 permissions, WAL + `synchronous=FULL` (counter reuse after a lost write would silently rehydrate the wrong secret) + `busy_timeout` (concurrent WAL writers wait, not error), and a write-through cache keeping reverse lookups off the DB on the streaming hot path. WRITE-FAULT FAIL-CLOSED: `placeholder_for` rolls back and re-raises on ANY sqlite write error (not just IntegrityError) so a disk-full/IO fault under `isolation_level=None` can't leave the open `BEGIN IMMEDIATE` wedging the next request; caches write only after COMMIT (nothing poisoned) and n is `MAX(n)+1` read fresh, so a retry reissues the SAME dense number (test_vault_faults.py). `VaultManager`s hand out per-session views over one shared connection (LRU-cached ~64; eviction drops only the cache) and persist the Responses `response_id → session` map (`response_sessions`, pruned ~10k). `[vault] encryption = "fernet"` (`crypto` extra, `LLM_REDACT_VAULT_KEY`) stores a domain-separated HMAC index + Fernet ciphertext (schema v3); the MEMORY backend honors fernet too (EncryptedInMemoryVault: HMAC index + ciphertext in RAM, decrypt per reverse lookup, deliberately NO plaintext cache; key resolution and fail-closed behavior shared); migration is one-transaction encrypt-in-place followed by `wal_checkpoint(TRUNCATE)` + `VACUUM` (else plaintext lingers and the at-rest claim is false); wrong/missing key fails closed AT OPEN; caches stay plaintext-keyed so the hot path is unchanged. KEY RESOLUTION (`vault_crypto.from_env`): env var (`LLM_REDACT_VAULT_KEY`) → key command (`LLM_REDACT_VAULT_KEY_CMD`, `key_from_command`: run via shell, stdout stripped, for service units with no `$(...)` shell — a failing command logs by exception TYPE only since CalledProcessError carries captured output that could hold the key, timeout `_CMD_TIMEOUT_S`) → OS keychain (`keyring` extra, `llm-redact vault set-key`, service "llm-redact"/item "vault-key") → fail closed; every non-env source that errors counts as key-absent (never a silent downgrade, never a traceback), and error messages name the key SOURCE without echoing key material. Lifecycle CLI (`vault_cli.py`): `sessions prune` deletes WHOLE sessions only (partial deletion would let MAX(n)+1 reissue a live token number); for RDBMS backends `sessions list|prune` and `lookup` are CONFIG-driven (an explicit --db always means a sqlite file) while `vault verify/rotate-key/backup` refuse with guidance (server engines bring their own tooling; MVCC makes rotate-in-place dishonest). RDBMS BACKENDS (`vault_rdbms.py`, llm-redact-pro docs/vault-rdbms.md, Pro): `[vault] backend = postgresql|mysql|oracle|dbapi` + `[vault.rdbms]` (dsn URL-form; LLM_REDACT_VAULT_DSN env override; password from `password_env`; `module` names the DB-API 2.0 driver for dbapi; `cloud` declares managed placement) — ONE RdbmsStore over DB-API 2.0 with a paramstyle adapter (named/pyformat/qmark/format/numeric) and a portable SQL subset: NO upserts, NO FOR UPDATE — dense counters are MAX(n)+1 inside the transaction resolved by the UNIQUE (session,type,n) constraint + bounded retry, caches write only after COMMIT, one reconnect-retry per self-contained op (a lost commit-ack re-finds the committed row). Schema deltas from sqlite, both deliberate: fixed-width `original_key` (HMAC index when encrypted, SHA-256 otherwise) replaces raw values in the PK (MySQL/Oracle can't index unbounded text), and the ENCRYPTION MODE IS FIXED AT CREATION (llm_redact_meta marker; server MVCC keeps old row versions no rewrite could scrub — no migration, no rotation; fresh schema to change). OFF-BOX RULE: non-local DSN (incl. recognized managed hosts and /cloudsql/ sockets) without fernet = startup ConfigError; LLM_REDACT_VAULT_REMOTE_PLAINTEXT=1 is the hatch, surfaced in /status (vault.remote_plaintext), posture, doctor; dbapi DSNs are opaque → doctor WARNs locality unverifiable. Drivers are extras (vault-postgres=psycopg3, vault-mysql=PyMySQL utf8mb4-forced, vault-oracle=oracledb thin with fetch_lobs=False); missing extra = ConfigError with hint. Tests: the battery runs against stdlib sqlite3-as-dbapi, fake psycopg/pymysql/oracledb over sqlite3 (fault injection + reconnect), and REAL postgres:16 + mysql:8 CI services (the `rdbms` job); scripts/oracle_smoke.sh is the Oracle opt-in. DSNs never logged/echoed.
- **Sessions** (`sessions.py`, `[vault] session_mode = "per-conversation"`): OPEN-CORE SPLIT (R2) — Free `sessions.py` ships only `StaticSessionRouter` (one shared namespace, `mode="static"`, inert `resolve`/`record_response_id` stubs) plus a fail-closed `build_session_router` (per-conversation without the `llm-redact-pro` package is a `ConfigError`, never a silent downgrade to the shared namespace); the per-conversation `SessionRouter` lives in `llm_redact_pro.sessions` (registered through the registry, reusing the Free `StaticSessionRouter` for static mode). The proxy binds only to the `plugin_api.SessionRouter` Protocol (`mode`/`resolve`/`record_response_id`). The pro router now lives in the separately-installed `llm-redact-pro` package (R4 physical split); its mutation coverage (`resolve`/`_canonical`/`orphan_session_id`) runs in the pro repo's CI, and this repo's mutmut config covers only the Free codecs + vault. The pro `SessionRouter` derives `conv-<16hex>` ids from a domain-separated SHA-256 of the first user message (whole `input` for Responses; only the hash is ever stored/logged; `model` excluded so mid-conversation switches don't fork the namespace). Isolation is strict by design — there is NO fallback lookup across sessions: token names collide across sessions by construction (every session has an «EMAIL_001»), so any fallback hit would silently restore another conversation's secret. ORPHAN SESSIONS (pre-debut security fix): an UNMAPPED Responses `previous_response_id` or unmapped `GET /v1/responses/{id}` routes to a UNIQUE, empty session derived from the response id (`orphan_session_id`, domain-separated `_ORPHAN_DOMAIN`), NOT the configured static fallback — in per-conversation mode the static session is itself populated by the conversations/batch/no-anchor paths, so a provider-echoed token could rehydrate a DIFFERENT secret bound there (a never-wrong-value violation); the orphan session is empty by construction so an echoed stale token misses and passes through verbatim (compaction stance). Static mode (default) uses one prebuilt `RequestContext`, so the hot path is unchanged; per-conversation costs one hash + dict lookups (thin Redactor/Rehydrator wrappers share detectors/allowlist/counters). COMPACTION FORKS: a NEW session whose first message already contains placeholder tokens is the history-compaction signature — counted and surfaced (`compaction_forks` in /status, `llm_redact_compaction_forks_total`, dashboard pill, specific INFO line). Relinking the fork was spiked and REJECTED — every design fails the never-wrong-value bar; `docs/compaction-relink.md` is the record, do not re-attempt without reading it.
- **Inbound (rehydration)** (`rehydrate.py`): non-streaming bodies via jsonwalk (with a `key_overrides` escape-aware path for OpenAI `arguments`, which is raw JSON source); streaming via `StreamingRehydrator`, which buffers a bounded partial-token suffix across chunk boundaries (`«EM` → `«EMAIL_` → `«EMAIL_001»`). `json_source=True` channels normalize `«`/`»` guillemet escapes (even-backslash positions only; split escapes held back ≤5 chars) and re-escape restored values so the argument stream stays valid JSON source. Fuzzy matching (`[rehydration] fuzzy`, default on) restores mangled tokens (`«email_001»`, `«EMAIL-1»`, `« EMAIL_001 »`) via canonicalize-then-vault-lookup — a miss always passes through verbatim; bracket swaps are deliberately out of scope. `substitute_tokens()` is the single substitution path shared by streaming and non-streaming.
- **SSE** (`sse.py`): incremental byte-level parser/serializer. The proxy always operates on the raw byte stream — never assume typed objects; this is the known failure mode that broke other proxies.
- **Providers** (`providers/`): `ProviderAdapter` maps paths to routing kinds (CHAT / REDACT_ONLY / NONE) and maps SSE events to rehydrator channels. Anthropic: `text_delta`/`thinking_delta`/`input_json_delta` per block index; flush on `content_block_stop` (leftovers become synthetic deltas *before* the stop event). MESSAGE BATCHES: POST /v1/messages/batches is REDACT_ONLY (requests[].params walked; note injected per entry); GET .../results is CHAT rehydrated line-by-line through the NDJSON path (`handles_ndjson`, content types in proxy `_JSONL_CONTENT_TYPES`; whole-string restoration — lines are complete); poll/list/cancel/delete pass through (pinned). OPENAI FILES/BATCHES: POST /v1/files is REDACT_ONLY multipart (`multipart.py`, byte-faithful codec — parse returns None outside the canonical CRLF grammar and the proxy forwards verbatim; JSONL file-part lines redacted per line, batch `body` and fine-tune lines noted, BlockedRequest anywhere rejects the WHOLE request 400 via the shared blocked_response closure); GET /v1/files/{id}/content is CHAT via `rehydrate_raw_body` (buffered non-JSON branch); /v1/batches + file metadata pass through, and provider inference maps /v1/files + /v1/batches to openai so disabled-provider 502 covers them. Batch flows use the static session (async fetches have no anchor). MCP CONNECTOR: mcp_servers[] (Messages) and tools[].type=="mcp" (Responses/Realtime) are provider-directed config — stripped BEFORE redaction (strip_mcp_tools/restore_mcp_tools in providers/base.py) so the provider receives the real credential and nothing in the block is counted; MCP call CONTENT still flows (Responses gained response.mcp_call_arguments.delta/done channels + mcp bookkeeping events in KNOWN_EVENT_TYPES). docs/api-coverage.md is the endpoint matrix, pinned by tests/test_api_coverage.py in BOTH directions (route + doc sync). OpenAI chat: `delta.content`, reasoning-model chain-of-thought (`delta.reasoning_content` — DeepSeek/vLLM/Groq/xAI — and `delta.reasoning` — OpenRouter — on their own per-choice channels, `_REASONING_FIELDS`), and `tool_calls[].function.arguments` per choice; flush on `finish_reason`/`[DONE]` (leftover reasoning tokens map back via `_leftover_to_delta`). Non-streaming reasoning fields are already covered by the generic jsonwalk — only the streaming deltas needed the per-channel path. OpenAI Responses (`openai_responses.py`, Codex CLI): `output_text`/`refusal`/`function_call_arguments` deltas per item; `*.done` events flush and rehydrate their repeated full value; `response.completed` flushes all and rehydrates the embedded response; `GET /v1/responses/{id}` is also rehydrated. Its event shapes are pinned by hand-authored fixtures — verify against the live API when possible. CONVERSATIONS API (the stateful item store paired with Responses): POST /v1/conversations + POST/GET /v1/conversations/{id}/items + GET /v1/conversations/{id} are all CHAT (item content redacted out, stored/echoed content rehydrated back); DELETE passes through. Conversations use the STATIC vault session (async reads have no first-message anchor — the batch/realtime stance, forced in sessions.py `resolve`), so redact/rehydrate always agree (never wrong-value) at the cost of a per-conversation namespace. The list-items response rides the `object:list` jsonwalk envelope walk above; note injection is suppressed for conversations (bodies carry `items`, not `messages`). Gemini (`gemini.py`): channels keyed `(candidate, text|thought)`; NO [DONE] sentinel — `finishReason` is the only flush point (leftovers append to the candidate's last matching part; `_stream_rehydrated` discards stream-end leftovers); `functionCall.args` is a parsed object (plain walk, not JSON source); the no-`alt=sse` form returns a buffered JSON *array* whose elements split tokens → list-aware `rehydrate_body` runs streaming channels across elements; drift detector = KNOWN_CHUNK/CANDIDATE/PART_KEYS subsets (events are unnamed). CONTEXT CACHING + BATCH MODE: `POST /v1beta/cachedContents` (create, `_GEMINI_CACHED_CREATE`) and `models/{m}:batchGenerateContent` are REDACT_ONLY — the cached prompt and inlined batch requests are content walked by the generic jsonwalk, the responses carry only a cache/operation name (nothing to rehydrate); both are stored/async (no first-message anchor) so `sessions.py resolve` forces the STATIC session for `:batchGenerateContent` and `/cachedContents` paths (batch stance), and per-cache GET/PATCH/DELETE + list pass through; no note injection (would corrupt a batch body / change a cache). Vertex uses a different batch API (BatchPredictionJob) and is out of scope. Azure (`azure_openai.py`): OpenAIAdapter subclass, only `matches()` differs; needs `[providers.azure]` set (502 until then). Vertex AI (`vertex.py`): GeminiAdapter subclass, only `matches()` differs (v1/v1beta1, publishers/models + endpoints, with/without projects/locations prefix); needs `[providers.vertex]` set (502 — the host embeds the region); registered after Gemini, matchers proven disjoint. Claude-on-Vertex (`claude_vertex.py`, `ClaudeVertexAdapter`): AnthropicAdapter subclass reusing ALL Messages redaction/rehydration/note-injection — only `matches()` differs (`publishers/anthropic/models/{m}:rawPredict|:streamRawPredict`, anchored to the anthropic publisher because Llama/others use rawPredict with different bodies; body carries `anthropic_version: vertex-2023-10-16`, no `model`); `name = "vertex"` so it shares the same upstream as the Gemini VertexAdapter (disjoint matchers — rawPredict vs generateContent — proven by test); registered after VertexAdapter. EMBEDDINGS are REDACT_ONLY on OpenAI `/v1/embeddings`, Azure embeddings paths, and Gemini `embedContent|batchEmbedContents` (input redacted, vector response verbatim); `wants_system_note(kind, path)` gates note injection — CHAT-only by default, Anthropic keeps it on count_tokens and Gemini/Vertex on countTokens, embed* bodies are never touched (injecting would corrupt them). AWS Bedrock (`bedrock.py`, bearer-key auth; SigV4 is a permanent non-goal — body rewriting breaks the signature): four `/model/{id}/…` runtime routes, all CHAT; model ids can be percent-encoded ARNs, so `matches` runs greedy on the decoded path while the proxy forwards `scope["raw_path"]` (raw-path forwarding applies to ALL providers); needs `[providers.bedrock]` set (502 — the host embeds the region). Note injection is positively-recognized-shapes-only: `anthropic_version` ⇒ Messages-style (shared `inject_anthropic_system_note`), Converse-signature messages (type-less keyed-union content blocks) ⇒ append `{"text": NOTE}` to `system`; anything else untouched (a wrong-schema field corrupts a native invoke body; a missing note only weakens token preservation). Streams are binary vnd.amazon.eventstream (`eventstream.py`: CRC32 = zlib.crc32, both CRCs validated, headers keep type codes so untouched frames re-serialize byte-identical, `feed()` never loses bytes on error — `residual` = everything unreturned); the proxy's third response branch (adapter flag `handles_eventstream`) degrades to verbatim pass-through on any framing violation (unrestored placeholders are safe; guessing at corrupt frames is not). converse-stream channels `("text"|"reasoning"|"tool" json_source, contentBlockIndex)` flush on contentBlockStop with synthetic delta frames; invoke-with-response-stream `chunk` frames base64-wrap Claude Messages events → shared `rehydrate_messages_payload` (extracted from anthropic.py); unrecognized inner shapes forwarded verbatim. Cohere (`cohere.py`, `[providers.cohere]` default `https://api.cohere.com`): `/v2/chat` CHAT (messages redacted; non-streaming response rehydrated by the generic jsonwalk with the `arguments` JSON-source override; streaming SSE rehydrated per channel — content-delta `delta.message.content.text` keyed `(text,index)`, tool-plan-delta `delta.message.tool_plan` keyed `(tool_plan,)`, tool-call-delta `delta.message.tool_calls.function.arguments` keyed `(args,index)` json_source; flush on `content-end`/`tool-call-end` per index and `message-end` for all, leftovers → synthetic deltas before the end event), `/v2/embed` + `/v2/rerank` REDACT_ONLY, legacy `/v1/chat` + `/v1/generate` CHAT (v1 `event_type:text-generation` stream on a single `("v1text",)` channel, flush on `stream-end`); note injection = a system message (v2 messages[]) or `preamble` (v1 chat), never for v1 generate's bare prompt. Drift set `KNOWN_COHERE_EVENT_TYPES` (live test); unrecognized events forward verbatim (worst case an unrestored placeholder, never a wrong/leaked value). Ollama NATIVE API (`ollama.py`, default upstream `http://127.0.0.1:11434`; its OpenAI-compat `/v1` routes still go through the openai adapter): `/api/chat` + `/api/generate` CHAT, `/api/embed` + deprecated `/api/embeddings` REDACT_ONLY; streams are `application/x-ndjson` (verified in ollama's server/routes.go — stream:false returns plain application/json, so the response-content-type branch stays unambiguous) → `ndjson.py` line codec + the proxy's fourth response branch (adapter flag `handles_ndjson`, `_stream_rehydrated_ndjson`); channels: chat `message.content`, generate `response`; the `done:true` line is the flush point (leftovers fold into its content); `tool_calls` arguments are PARSED OBJECTS (plain walk, not json_source); a line that fails json.loads is forwarded byte-identically. Ollama note injection appends to an EXISTING system message (chat, last one) or `system` string (generate) and NEVER creates one — a fresh `system` would override the Modelfile SYSTEM template and change model behavior. CUSTOM PROVIDERS (`custom.py`, `[providers.custom.NAME]`, served under `/custom/NAME/`): `_CustomPrefixMixin` wraps the full OpenAI (`CustomOpenAIAdapter`) and Responses (`CustomResponsesAdapter`) surface per named upstream — each independently enable/disable-able (fail-closed 502), several side by side (vLLM + LM Studio + OpenRouter). Every path-sensitive hook (`matches`/`wants_system_note`/`rehydrate_raw_body`) routes through `_canonical`, which re-anchors the stripped inner path at the LAST `/v1/` when present (Groq `/openai/v1`, OpenRouter `/api/v1`, Fireworks `/inference/v1`) and otherwise prepends `/v1` (a tool that put `/v1` in upstream_base_url leaves the inner path without it) — without normalization a non-`/v1` base path matched NOTHING and the request was silently forwarded UNREDACTED; a genuinely unknown tail still falls through to NONE (pass-through) via the wrapped EXACT matcher, so this only ever promotes real OpenAI endpoints. Unmatched subpaths under a configured prefix pass through to THAT upstream (the client addressed it explicitly). PER-PROVIDER DETECTION OFF: `[providers.NAME] detection = false` forwards that provider's requests UNREDACTED (no detection/deny/block/note — like warn mode, docs must never imply protection) while REHYDRATION stays active; applies to realtime WS frames too; surfaced via `providers_detection_off` in /status, a per-request INFO log line, a dashboard ⚠ marker, and an editor checkbox. MCP EXEMPT SERVERS: `[detection.mcp] exempt_servers = [names]` — MCP content blocks addressed to those servers (Anthropic server_name, OpenAI server_label) bypass detection via a stash-around-redaction in base `prepare_request` (stash_exempt_mcp_blocks / restore_exempt_mcp_blocks, position-based restore decided on the ORIGINAL so sentinel-shaped user data can't confuse it, restored BEFORE note injection which may restructure lists); Anthropic mcp_tool_result names no server — exempt only when tool_use_id correlates to an exempt mcp_tool_use in the same body, else redacted (fail-closed); HTTP bodies only. PROVIDER DISABLE: `[providers.NAME] enabled = false` fails CLOSED — checked in `handle()` BEFORE the body is read, answers a proxy-generated 502 (`adapter.error_body` shape); it must never fall through to pass-through, which would forward traffic unredacted (covers inferred pass-through paths too, pinned by test). `?key=` query auth passes through but httpx/uvicorn loggers print full URLs — cli.py silences both; never log URLs with queries.
- **Realtime WS** (`realtime.py`, `realtime` extra = websockets>=13, which serves BOTH uvicorn's ws protocol (auto-enabled when importable — without it uvicorn refuses upgrades entirely) and the upstream client): a catch-all `WebSocketRoute` relays OpenAI Realtime (`/v1/realtime`, beta AND GA event names in the channel tables) and Gemini Live (`BidiGenerateContent`, JSON over text OR binary frames — re-serialize in the SAME frame type) to the provider's wss endpoint derived from `[providers.*]` by scheme swap. Outbound: jsonwalk over EVERY client event with `_REALTIME_STRUCTURAL_KEYS` (base64 `audio` + enums skipped; a missed redaction is a leak, so walking everything is the default); `BlockedRequest` closes the connection 1008 (type only). Inbound: `rehydrate_message` returns a frame LIST (flush leftovers become synthetic delta frames); channels keyed (item_id, kind, output_index, content_index) for OpenAI, (modelTurn, text|thought) + outputTranscription for Gemini; flush on `*.done`/`response.done`/turnComplete; item/session echo events rehydrate whole (the client owns the originals). Close codes mirror BOTH directions — websockets' iterator raises ConnectionClosed for non-OK codes (handled, else the client would see 1000). Headers/query/subprotocols pass through and are NEVER logged (wss URLs carry `?key=`; browser clients carry keys in subprotocols). Refusals (unknown path — there is no default WS upstream, disabled provider, missing extra, reserved path) are accept-then-close 1011 so the reason reaches the client. Note injection appends ONLY to existing instruction fields (session updates replace instructions wholesale — creating one clobbers the server default), idempotently. Static vault session ONLY (no first-message anchor at upgrade time). One `record_request` per connection at close (method "WS"); detections use a per-connection tee counter (the HTTP diff trick corrupts across awaits). Drift: `KNOWN_REALTIME_EVENT_TYPES` / `KNOWN_LIVE_SERVER_KEYS` asserted by live tests. Testing: WS cannot ride ASGITransport — test_realtime_relay.py runs uvicorn on port 0 + a fake `websockets.serve` upstream (the test_tls pattern); adapter sweeps split at every offset.
- **Proxy** (`proxy.py`): catch-all route; non-JSON or unrecognized traffic forwards verbatim (never break the tool). Streaming-vs-JSON handling branches on the upstream *response* content-type, not the request's `stream` flag. UPSTREAM FAULTS fail closed: transport errors on `send`/buffered `aread` return a recorded, provider-shaped **502** via the `upstream_fault_response` closure (never a bare 500 or a leaked upstream connection), and the three streaming `finally` blocks suppress an `aclose()` that itself raises on a broken stream so a mid-stream drop ALWAYS finalizes (record_request + `llm_redact_upstream_errors_total`); the full fault catalogue is docs/resilience.md, and stream-end-mid-token flushes the partial placeholder verbatim (test_stream_truncation.py). Auth headers pass through untouched and are never logged; log lines contain only path, status, and detection counts. Reserved `/__llm-redact/*` paths (dashboard at `/`, status, metrics, recent, sessions, audit, config) are answered by the first statement of `handle()` and provably never forwarded (`/recent` and `/audit` are `_host_allowed`-gated like `/events`/`/sessions` — pre-debut DNS-rebinding fix; `/metrics` + health probes stay open for monitoring); every reserved reply is stamped in that one place with browser-hardening headers (`_SECURITY_HEADERS`: strict CSP `default-src 'none'` + inline script/style + `connect-src 'self'` — exactly what the self-contained dashboard needs — plus `X-Frame-Options: DENY`, nosniff, `Referrer-Policy: no-referrer`; `setdefault` so a handler's own header wins; forwarded upstream traffic is never touched). GET-only with THREE exceptions sharing one guard chain (`_guarded_post_json`: layered Host-validation (DNS rebinding) → Origin → per-process CSRF token in a custom header (forces preflight; OPTIONS→405 no CORS) → content-type → 1 MiB cap): `POST /__llm-redact/config` (the dashboard's config editor), `POST /__llm-redact/sessions/prune` (whole idle sessions only, sqlite only, evicts manager views, never the active static session — the CLI prunes it with the proxy stopped), and `POST /__llm-redact/preview` (config dry-run: runs the LIVE detectors/allowlist/modes over caller-supplied `{"text"}` on a THROWAWAY InMemoryVault, returning `{redacted, detections, warnings, blocked}` — no upstream request, no vault/metrics/audit write; the caller's own text comes back masked so no new value leaves the box, but warn-mode values remain in `redacted` (honest); like the config-editor GET it is exempt from the metadata-only self-check. `llm-redact preview` is the CLI twin — loads config from disk, scans stdin/`--text` entirely locally, `--json` for machine output; both surfaced in the dashboard preview card). Config edits merge over FILE truth (env overrides never baked in; host/port/vault/audit/log/tls/otel/users/email readonly), validate through `parse_config` + dry-run detector build, re-verify by reparsing the emitted TOML (`config_write.py`, hand-rolled emitter — `_toml_str` is `json.dumps`; the default `enabled` list is deliberately omitted to stay open-ended), write atomically with one `.bak`, and hot-apply via `apply_config` (shared with SIGHUP reload, returns restart-required names). SIGHUP triggers `ProxyState.reload()`: hot-swappables are built fully then swapped in one assignment block; vault/audit/host/port changes warn "require restart"; a bad config file never crashes the proxy (registered via `loop.add_signal_handler`, guarded for Windows).
- **Routing (pro)**: rule-based upstream routing, fallback chains, cooldowns, plan-limit detection, model rewrite/restore, `/v1/models` discovery and monthly budgets are the llm-redact-pro routing layer; the core holds the SEAM only. Config SHAPES live here (`config.py`: `UpstreamConfig`/`RouteRule`/`RoutingConfig`/`PricesConfig` + the `[upstreams]`/`[routing]`/`[prices]` parser, invariants I-1..I-6, `resolve_credentials`, the emitter in `config_write.py`) — parsed LAST, file-only in the editor (`_FILE_PRESERVED_KEYS`, 400 on a POST naming them, preserved from file truth in the merge, re-validated in the dry-run via `resolve_credentials` + `Router.validate` or a build-and-close probe), hot on SIGHUP (`apply_config` builds/reconfigures/drops the router AFTER the detector+adapter builds and closes the displaced router at swap time; parser warnings are logged by the core at startup and on every reload). The CONTRACT is `plugin_api` (`Router` → `RoutePlan` → `HopRequest`/`HopResult`/`HopDecision` → `RouteDelivery`, plus `RouteInbound`/`RouteRefusal`/`LocalAnswer`; pinned by tests/test_plugin_api_surface.py — change it deliberately and bump the pro floor); the registry factory `build_router(config, tier)` returns None unless `[routing] enabled = true`, and the Free default then raises ConfigError naming the package (never a silent one-upstream downgrade; `[upstreams]` alone stays warn-inert). The DRIVER in `proxy.py` is policy-free: `handle()` consults `state.router` only when it is not None (two `is None` tests per request — the unrouted path is byte-identical, pinned by tests/test_routing_seam.py); `local_answer` sits after the disabled-provider 502 / named-user 403 gates and before the body read; `plan()` runs BEFORE redaction (the first upstream's `inject_system_note` governs the prepared body) and returns None for providers outside the router's protocols (legacy path), a `RouteRefusal` (502 no_route), or a plan; `_handle_routed` = `local_refusal` (404 count_tokens, BEFORE the audit START row) → `_begin_audit_guarded` → `begin()` (402 budget, WITH the token, or hop 1) → issue/`decide` loop (`_issue_hop` sends, pre-reads bodies `_deliver` would buffer so a mid-body drop is still re-issuable, counts faults by upstream NAME and reports the exception TYPE; `_discard` closes every undelivered response; `plan.wait` sleeps retry-same; `metrics.reissues[(from,to)]` bumps when a hop's `reissued_from` is set) → `delivery()` hooks (`observe_event`/`observe_line` after the adapter's rehydration and before serialization, `observe_payload` on buffered JSON, `mark_failed("transport"|"stream_error")`, `x-llm-redact-*` headers merged) → `finish_route` (`metrics.routed[(upstream, rule or "-")]` on EVERY routed outcome incl. 502s, then the router's `finish` = spend) → `record_request(route=row)` (the recent/events row's `route` key, None on the legacy path; audit rows unchanged). The Bedrock eventstream branch has no hook (never routed). Log lines carry the fixed-key `_route_log_suffix` (`rule upstream hops auth class reissue`) — names/ids/modes/classes only. Surfaces are core: `/status` `routing` block (the router's, `{"enabled": false}` without one), the two Prometheus counters, `routes list|test` / `spend` / `doctor --offline` parsers (lazy-import dispatch to `llm_redact_pro.routes_cli`, the requires-package line on ImportError), `_print_routing`/`_routing_posture`, the doctor `_check_routing` shell (PASS not-configured/disabled, FAIL without the package, else `llm_redact_pro.routing_doctor.routing_checks` rows), the `routes`/`spend` plugin commands, the dashboard pill/card, the Anthropic 402/404 error types, the user guide. `config.example.toml` carries a 6-line pointer, never the enabled block; core docs never link a `docs/routing.md` (it lives in the pro package). Tests stay keyless via `tests/fake_router.py` (scripted `FakeRouter`/`FakePlan`/`FakeDelivery` registered on a bare `Registry`).
- **Licensing & users (FOSS core)**: THE CORE ENFORCES NOTHING — the AGPL-3.0 relicense removed `features.py` (`check_license`/`pro_features`/`required_clouds`) entirely: no tier gates, no seat caps, no cloud entitlements, no k8s gate, no non-loopback licensing gate (binding beyond loopback is purely the `validate_bind_security` mTLS policy). `proxy._resolve_license_info` resolves the key for INFORMATIONAL surfacing only (/status `license` block, dashboard pill — keyless shows "nothing gated", or "pro features need a key" when llm-redact-pro is installed (its factories refuse paid subsystems on the Free tier), doctor PASS line, `llm-redact status`) and hands it to the pro plugin; warnings (key set without pro, expiry grace, 14-day grace window) still log loudly and `refresh_license_warnings` still ticks daily. What fails closed is PACKAGE PRESENCE: a config that requests a subsystem only llm-redact-pro implements (vault encryption/cipher, RDBMS vaults, the audit log + sinks, OTel, per-conversation sessions, named users, `[audit] required`) raises ConfigError from the registry/factory seams naming the feature AND the package — never a silent downgrade (pinned both directions by tests/test_no_gates.py; formerly-gated free-repo capabilities — non-loopback, k8s, Bedrock/Azure/Vertex adapters — are pinned to build keyless). Free `licensing.py` keeps the DATA (License/ResolvedLicense, TIER_ORDER/TIER_USER_CAPS/CLOUDS, grace windows, FREE) + the thin registry-dispatched `resolve_license` DELEGATOR (free_defaults returns Free with a loud "llm-redact-pro not installed" notice when a key is set; the pro plugin registers the real Ed25519 resolver, and PRO'S OWN FACTORIES honor the key's tier/seats/expiry — tiers are commercial packaging of the pro package, enforced there, never here). Resolution env → `[license] key` → `key_file`; `[license]` stays in config and `license show|verify` stays in the CLI (informational; the verify core lives in pro). Cloud placement (`cloud_detect.py`) is detection-only now (env override + probes, no gating consumer in-core). NAMED USERS: the store implementation lives in pro (`users.py` here keeps the Protocol + fail-closed `build_users_store` — Free tier ⇒ None, no users.db ever); IDENTITY extraction/scrubbing is unchanged (`/u/<key>/` prefix + `x-llm-redact-user` header, both scrubbed segment-based in `_extract_user_key` before routing/logging/forwarding, header in _SKIP_REQUEST_HEADERS + WS _HOP_HEADERS). Enforcement (`user_enforcement_required`) is now ACCESS CONTROL, not licensing: 2+ verified users in the (pro) registry ⇒ provider-shaped 403 without a valid key (WS: accept-then-close); the old non-loopback trigger is GONE; counts read the LIVE registry. Attribution via the `_REQUEST_USER` contextvar into recent/events/audit rows is unchanged. The suite runs keyless and pins the no-gates contract; `tests/test_open_core_free_coverage.py` still drives every free-code function whose only e2e path rides a pro feature (complexity gate, empty allowlist). LICENSED-FEATURES SIGNAL (`registry.pro_package_installed`, a pure find_spec probe): doctor prints an informational `license` line ("licensed-features package not installed (FOSS core is complete; pro-only config fails closed)" / "installed (<plugins> active)", NEVER a FAIL), /status carries `license.package_installed` + `license.plugins`, the ONE loud case — package present but its plugin did NOT register — stays a doctor WARN + dashboard opt-out line. LICENSES: this repo is GNU AGPL-3.0-only (LICENSE = verbatim text; pyproject `AGPL-3.0-only`; pre-relicense private-lineage artifacts remain MIT); contributions require the CLA (docs/CLA.md, sign-off = agreement — keeps the dual-license model legal); llm-redact-pro is proprietary (per-seat commercial subscription license, no redistribution). Threat model: distribution control is the primary protection; verification never phones home.
- **Ops surface**: `metrics.py` hand-rolls Prometheus text (stdlib only). `DurationHistogram.observe()` increments every covering bucket, so stored counts are ALREADY cumulative — `render()` must not re-sum. `record_request()` always updates metrics AND the in-memory recent ring buffer (`deque(maxlen=200)`, served newest-first at `GET /__llm-redact/recent`, same row shape as audit — the dashboard's recent table reads it so the tail works without the audit DB), and writes an audit row only when enabled; 413s, pass-through, upstream-fault 502s, and stream-close finalization included. Upstream transport faults are counted per provider (`llm_redact_upstream_errors_total`, /status `upstream_errors_total`, `LlmRedactUpstreamErrors` alert). `record_request` also fans the row out to `GET /__llm-redact/events` SSE subscribers (per-subscriber bounded `asyncio.Queue(100)`; QueueFull silently drops — the dashboard's EventSource prepends rows while the 3 s poll stays authoritative and self-heals; host-check gated like /sessions; 15 s comment keepalives; NEVER drive this endpoint through ASGITransport to completion in tests — the infinite stream hangs; test_events_endpoint.py speaks raw ASGI and cancels) and, when `[otel]` is enabled, to `otel.py` (`otel` extra; `build_telemetry` → scoped SDK providers — never the process globals; spans get explicit start/end computed from the measured duration so streaming needs no context threading; metadata-only, same contract as audit; restart-only + editor-readonly; enabled-without-extra is a startup ConfigError with the install hint; lifespan shutdown flushes the batched exporters). `GET /__llm-redact/sessions` lists vault sessions (metadata only) via `VaultManager.sessions_summary()`. `[log] format = "json"` (`log.py` JsonFormatter, restart-only, `serve --log-format` overrides) emits one object per line — a FIXED value-free key set ts/level/logger/service/version/message (+exception), never record extras (the never-widen-content invariant, pinned by test); json mode passes `log_config=None` to uvicorn so its records propagate to the root JSON handler. OPS ASSETS (`deploy/`): prometheus-scrape.yml + prometheus-alerts.yml (warn-mode-forwarding is the honesty alert) + grafana-dashboard.json + k8s-sidecar.yaml (same-pod sidecar BINDING 127.0.0.1 with exec healthz/readyz probes — a pre-debut fix corrected the 0.0.0.0 bind that contradicted its never-exposed claim; runAsNonRoot/readOnlyRootFilesystem/drop-ALL) + `helm/llm-redact/` (the Helm chart, `mode: sidecar|standalone` from one chart — SIDECAR BINDS 127.0.0.1 (pod-shared netns makes "never exposed" structural; kubelet httpGet dials the POD IP so sidecar probes are EXEC probes, the Dockerfile-HEALTHCHECK one-liner; no INSECURE_BIND needed) while standalone binds 0.0.0.0 + httpGet + INSECURE_BIND-or-mTLS (extraVolumes/extraVolumeMounts exist to mount the [tls] Secret — without them the mTLS advice was unwireable); standalone renders Deployment+Service+optional HPA(autoscaling/v2, CPU Resource metrics + optional memory)+optional ServiceMonitor+optional PDB; optional NetworkPolicy (sidecar default-deny / standalone allowFrom) + ServiceAccount (IRSA/WI annotations) + imagePullSecrets; tool.enabled defaults FALSE (placeholder image would ImagePullBackOff — NOTES warns if enabled unedited); NOTES also warns on sqlite-without-persistence and on an RDBMS backend with the STOCK image — the `-rdbms` GHCR variant (Dockerfile ARG EXTRAS; release.yml builds+cosigns both) adds vault-postgres/vault-mysql/crypto; the hardened container spec is shared via `_helpers.tpl` `llm-redact.proxyContainer`; the LOAD-BEARING `llm-redact.validate` guardrail `{{ fail }}`s a standalone autoscaled/multi-replica render on a per-pod memory/sqlite vault — replicas would issue divergent tokens (never-wrong-value), so a shared RDBMS backend is forced; Chart appVersion == `__version__`, pinned; k8s needs NO license — the FOSS core is ungated, the chart adds no gate); `test_deploy_assets.py` pins that the dashboard/alerts reference only emitted metrics, the k8s manifest keeps its hardening, and (stdlib-always + helm-gated test tiers) the chart's hardening/guardrail/appVersion hold — the CI `helm` job installs helm PINNED+checksum-verified and renders both presets for real. The systemd unit (`service_cli.py`) ships a sandbox (NoNewPrivileges/ProtectSystem=strict/empty CapabilityBoundingSet/@system-service filter/ReadWritePaths scoped to XDG data). `dashboard.html` is package data loaded once via importlib.resources (self-contained, textContent-only DOM writes); packaging is verified by a unit test *and* a CI wheel/sdist listing step — twine can't catch missing package data (the same CI step asserts the LICENSE file ships in both distributions).
- **Audit** (`audit.py`, `[audit]`, default off): separate metadata-only SQLite DB (types+counts/path/duration — never values, never placeholder ids); `synchronous=NORMAL` vs the vault's FULL (losing an audit row is acceptable; losing a vault row is not). TAMPER-EVIDENT CHAIN (`[audit] tamper_evident`, default off): each row stores `chain_hash = HMAC-SHA256(key, prev_hash \x00 canonical_row)` (`_chain_serialize` = compact JSON of the stored fields incl. the pre-serialized detections/rehydrations, so record and verify hash identical bytes); the `chain_hash` column is always in `_SCHEMA` (NULL when off) and `_ensure_chain_column` migrates pre-column DBs; ZERO-LOSS MODE (`[audit] required`, Pro, requires `enabled`): the proxy's `begin_audit` durably commits a write-ahead START row BEFORE any upstream contact (HTTP + realtime WS) and refuses the request with a provider-shaped 503 on `AuditWriteError` (the audit-storage twin of the upstream-fault 502); `record_request(audit_token=...)` finalizes the END row (buffered, all three streaming finalizers, WS close), an END-side fault after the response is committed logs CRITICAL (refusal impossible), startup fails closed when the built AuditLog lacks the `begin`/`finalize` pair (the `WriteAheadAudit` sub-Protocol, resolved once in ProxyState — old pro or a required config with no log both refuse), and proxy-LOCAL replies (413/400/403/502) stay best-effort rows — the guarantee is scoped to upstream contact. Default stays fail-open; the in-memory tip advances ONLY after a successful insert (a swallowed write can't orphan the next link). Key is env-only (`LLM_REDACT_AUDIT_HMAC_KEY`, SHA-256 → 32 bytes, `audit_hmac_key_from_env`); `tamper_evident` without it is a startup ConfigError in ProxyState (fail closed — a keyless chain is attacker-recomputable). `verify()` walks ascending: the oldest surviving chained row is an UNTRUSTED ANCHOR (its predecessor may have been legitimately pruned, so a first-row mismatch is adopted, not flagged) and every row after it is cryptographically checked → `ChainVerification(ok, checked, broken_at, reason)`; `llm-redact audit verify [--json]` (exit 0/1/2) runs it offline. Surfaced in /status (`audit.tamper_evident`), the editor readonly card, dashboard, and a doctor FAIL when the key is absent. S3 SINK (`audit_s3.py`, `[audit.s3]`, default off, restart-only): batches the SAME metadata-only rows as NDJSON objects to aws (virtual-hosted host from bucket+region) / minio / ceph (endpoint_url, path-style) / gcs (Google Cloud Storage via its S3-compatible XML API at storage.googleapis.com, path-style, HMAC interop keys — reuses the AWS SigV4 signer unchanged since GCS interop accepts it); credentials from env vars ONLY (never the config file), provider-keyed via `credential_env_names` (aws/minio/ceph → AWS_*, gcs → GCS_HMAC_ACCESS_ID/GCS_HMAC_SECRET); hand-rolled SigV4 on stdlib hmac/hashlib pinned by the official AWS docs test vector (query strings unsupported by design; key prefixes restricted to URI-unreserved chars so the canonical URI never needs encoding); every failure WARNs-and-drops (missing creds warn ONCE, naming the provider's own env vars), buffer bounded at 10k rows drop-oldest, flush loop starts in the lifespan and aclose() flushes at shutdown; doctor checks credential PRESENCE only (provider-aware); surfaced in /status (batches_uploaded/rows_dropped) and the editor's readonly card. GCS-native GOOG4/OAuth is deliberately NOT used (the S3-interop path needs no new signer). AZURE BLOB SINK (`[audit.azure]`, `AzureAuditSink`, separate section — SharedKey is not an S3 API): hand-rolled SharedKey HMAC-SHA256 over the documented StringToSign (12 fixed lines + canonicalized x-ms-* headers + `/account/container/blob` resource; Date blank because x-ms-date is signed; locale-proof `rfc1123`), account key from `AZURE_STORAGE_KEY` (env only), `endpoint_url` overrides the host for Azurite; the signer is pinned by an INDEPENDENT StringToSign transcription in tests (checksum-table discipline), mis-canonicalization degrades to a 403 the sink WARNs-and-drops on. The proxy holds parallel `audit_s3`/`audit_azure` optionals; the lifespan runs a flush task per active sink and aclose()s each; /status and the editor readonly card carry both `audit.s3` and `audit.azure` blocks. Multiple sinks can run at once. BATCH ENCRYPTION (`[audit.s3]`/`[audit.azure]` `encryption = "fernet"`, Pro with the sink): `encode_batch` uploads each NDJSON batch as ONE Fernet token (`.ndjson.fernet`, application/octet-stream) ahead of BOTH signers; key env-only (`LLM_REDACT_AUDIT_ENC_KEY`, SHA-256 → Fernet key via `audit_enc_key_from_env` — the HMAC-key recipe); enabled-without-key/extra = startup ConfigError in ProxyState, and a key that vanishes at runtime WARNs once and DROPS batches (NEVER a plaintext fallback — pinned: nothing uploads at all); `llm-redact audit decrypt FILE [--out]` reads objects back (exit 1 on wrong key, no garbage output); doctor checks per-sink posture, /status carries `audit.s3/azure.encryption`. Rehydration counting threads a `Counter` through `substitute_tokens`.
- **Env & containers**: `LLM_REDACT_HOST/PORT/CONFIG` overrides (CLI > env > file); config search adds `/etc/llm-redact/config.toml`. The Dockerfile sets `LLM_REDACT_HOST=0.0.0.0` (container netns only — the documented publish spec is `-p 127.0.0.1:8787:8787`) plus `LLM_REDACT_INSECURE_BIND=1` (the documented hatch for exactly that confined case) and `XDG_DATA_HOME=/data` for vault/audit persistence; the native default stays 127.0.0.1. Released GHCR images are MULTI-ARCH (amd64+arm64 via QEMU+buildx in release.yml; CI smoke jobs stay native amd64). `perf` extra = uvloop; uvicorn `loop="auto"` auto-selects it — serve must NEVER pass an explicit `loop=` (that would break the auto-selection the extra relies on).
- **Client CLIs** (`run_cli.py`, `doctor_cli.py`): `llm-redact run -- cmd` injects the tool→env-var map (shared `TOOL_EXPORTS` in init_cli.py, incl. ollama→OLLAMA_HOST); an already-running proxy is reused and NEVER killed; otherwise an ephemeral `serve` subprocess runs for the child's lifetime (torn down on every exit path). POINTED-AT PROXIES: `LLM_REDACT_PROXY_URL` (or `run --proxy-url`) names an EXISTING proxy — `run` uses it verbatim (must answer /status; never auto-starts beside it), `status` queries it (same --ca/--cert/--key mTLS flags), `validate_proxy_url` allows plain http on LOOPBACK ONLY (remote = https, or prompts cross the network in cleartext), and log lines print scheme://host:port only (an /u/<key> mistakenly embedded never echoes). SIGINT is deliberately ignored in the wrapper (the child gets it from the terminal process group; forwarding would double-signal); SIGTERM is forwarded; the child's exit code propagates. Plain-http loopback only ([tls] set ⇒ refuse). `doctor` is read-only PASS/WARN/FAIL (config parse, bind policy, proxy reachability + version skew, port occupancy via a bind probe, vault/audit 0600/0700 perms, fernet-without-key, missing extras with install hints, and a COVERAGE-POSTURE check `_check_posture` that WARNs for every configured opt-out — warn-mode rules, per-provider detection off, MCP exempt servers, language-scoped-out national-ids — or one PASS when none), exits non-zero on FAIL, never prints values. `status` mirrors this at runtime: `_print_posture` reads the live /status honesty fields (warn counts, `providers_detection_off`, `mcp_exempt_servers`, `language_inactive_rules`, `compaction_forks`, S3/Azure `rows_dropped`, disabled providers) into a loud posture block, silent only when nothing is opted out. Both are enumerated in completions.py's COMMANDS (a coverage test enforces parser↔completions sync). AGENT PLUGINS (`plugin_assets.py` single source, `plugin_cli.py` installer, docs/plugins.md): ten slash commands mirroring the dashboard + config editor (status/recent/sessions/config-show/config-edit/preview/doctor/audit/users/guide — users NEVER runs `users verify` itself: the per-user key prints once and belongs to the invitee; `guide` displays the packaged user guide, also served at `/__llm-redact/guide`) rendered per tool — Claude Code (checked-in `plugins/llm-redact/` + repo-root `.claude-plugin/marketplace.json`, regenerated by scripts/render_plugins.py and pinned BOTH directions by test_plugins.py; copy-install goes to ~/.claude/commands as flat llm-redact-*.md), Codex (~/.codex/prompts, top-level only), OpenCode (~/.config/opencode/commands), Cursor (~/.cursor/commands — PLAIN markdown: no frontmatter, $ARGUMENTS rewritten to prose; render_cursor). EVERY body opens with PROXY_GUARD (missing `llm-redact` CLI ⇒ stop and ask approval before installing — never install uninvited; pinned by test), and `plugin install` ends with a proxy posture hint line (probe injectable for tests). config-edit mirrors the editor guardrails (effective config → file edit → `serve --check` gate → SIGHUP → posture read-back) and is user-invocation-only; `lookup` is deliberately NEVER a plugin command (an agent reading a secret value would send it upstream — pinned by test); body `llm-redact` invocations are sync-tested against completions COMMANDS; install never overwrites modified files without --force, uninstall removes only managed files. PROXY SETUP STEP (`proxy_setup` in plugin_cli, probe/runner/ask injectable): `plugin install --proxy-url URL` points at an existing proxy (probed, validated, prints the LLM_REDACT_PROXY_URL export; the posture hint names the env var WITHOUT echoing its value); `--install-proxy` = `init --yes` (only when no config exists) + `service install`, explicit ask only; interactive single prompt (install/point/skip) ONLY on a tty with no flags and nothing answering; scripted default stays skip-with-hint. The agent-side PROXY_GUARD in command bodies is unchanged.
- **TLS & FIPS**: `[tls]` (restart-only, config-editor readonly): certfile+keyfile always together; `client_ca` = MUTUAL TLS (uvicorn `ssl_cert_reqs=CERT_REQUIRED`). `validate_bind_security` in config.py is the fail-closed bind policy `serve` runs pre-socket: non-loopback ⇒ full mTLS trio required unless `LLM_REDACT_INSECURE_BIND=1`; unresolvable hostnames count as non-loopback. `_origin_allowed` accepts https origins only when TLS is on; `status --ca/--cert/--key` builds an `ssl.SSLContext` (httpx 0.28 dropped per-request `cert=`). The real-socket mTLS e2e (test_tls.py) generates a throwaway PKI via cryptography (importorskip) and runs uvicorn on port 0 in a thread. FIPS (`fips.py`, `llm-redact fips-check`, docs/fips.md): approved-algorithm-selection posture, validation belongs to the HOST; wording rule "never certified" is machine-enforced by tests/test_fips_posture.py alongside a banned-primitive scan (md5/sha1/rc4/3des) of src/.
- **Bench** (`bench/`, `python -m llm_redact.bench`): deterministic runtime-generated corpus (never committed — positives are secret-shaped); exact span+type scoring; `--check` gates recall==1.0 per rule (functional regression gate) AND the false-positive corpus. `bench/fp_corpus/` (repo root, vendored, ruff-excluded, never shipped) holds real-world NEGATIVES; `fp_scan.py` gates exact per-file counts against `MANIFEST.toml` in both directions. When adding a detection rule: add its generator to `bench/corpus.py`, run the fp gate, and update the manifest (with justification) only for hits you judge legitimate.

## Testing conventions

The load-bearing suites are the chunk-split sweeps: `test_rehydrate.py` and `test_sse.py` split inputs at every offset and assert streaming output equals non-streaming output; they are parametrized over `fuzzy` and json-source-escape modes. When touching `rehydrate.py`, `sse.py`, or an adapter's event handling, extend those sweeps rather than adding single-case tests; `test_eventstream.py` mirrors the convention for the binary codec (golden fixtures assembled longhand, never via the codec's own serializer), `test_ndjson.py` for the NDJSON line codec, and `test_properties.py` (hypothesis, deadline=None) probes the same invariants with random inputs — it extends, never replaces, the sweeps. Integration tests (`test_proxy_integration.py`) run the real app in-process against fake upstreams via `httpx.ASGITransport` — NOTE: ASGITransport joins response body parts into one chunk, so chunk-boundary behavior can't be pinned through it (test_provider_bedrock.py's `_ChunkedUpstream` custom transport exists for exactly that). The mTLS suite (`test_tls.py`) is the one real-socket exception: throwaway PKI via cryptography (importorskip), uvicorn on port 0 in a thread. NER unit tests inject fake models (spaCy nlp / GLiNER / a fake Presidio analyzer) so the suite runs without any extra installed. The RESILIENCE suites prove fault behavior: `test_upstream_faults.py` (connect/timeout/mid-body drops fail closed with a recorded 502; the streaming generators finalize on a mid-stream drop), `test_stream_truncation.py` (a stream ending mid-token flushes the partial verbatim — `feed(prefix)+flush()` == `rehydrate_text(prefix)` at every truncation), `test_vault_faults.py` (write-fault rollback, dense counter, corrupted-ciphertext fail-closed), and `test_soak_concurrency.py` (a `soak` marker, deselected by addopts, run as its own CI step — cross-session isolation + counter density + bounded LRU eviction under many connections); faults are injected with fake transports/conns, never real sockets or disks. Live-API tests (`-m live`) are deselected by default via addopts and double-gated on `LLM_REDACT_LIVE=1` + API keys; the `/v1/responses` drift detector asserts observed event types ⊆ `KNOWN_EVENT_TYPES`, and the Bedrock one (AWS_BEARER_TOKEN_BEDROCK) asserts ConverseStream `:event-type`s ⊆ `KNOWN_CONVERSE_EVENT_TYPES`. The ner CI job installs presidio alongside spaCy and runs test_presidio.py's real-AnalyzerEngine smoke test (skips wherever the extra or model is absent); OTel tests inject fake tracer/meter the same way.

Placeholders use guillemets (`«»`) precisely because they don't collide with code/markdown; don't change the token format without updating `viable_prefix_start` holdback logic, the fuzzy grammar (`FUZZY_PLACEHOLDER_RE`/`canonicalize`), and the split sweeps together.

## Known limitations (deliberate scope)

Bracket-swap/ASCII-guillemet mangles (`[EMAIL_001]`, `<<EMAIL_001>>`) are never restored — too collision-prone with real code. Base64 media is not decoded and scanned (images can't leak through text regexes; PDFs would need parser deps) — on realtime streams that includes voice audio: only the TEXT modality is redacted, and what a user says aloud goes to the provider as-is. Realtime WS connections always use the static vault session (per-conversation anchors don't exist at upgrade time). Synthetic flush events on the Responses stream omit `sequence_number`. Originals that arrive *pre-escaped inside request JSON-source strings* are captured in escaped form. Per-conversation mode: history compaction rewrites the first-message anchor and forks a fresh session — deliberately fails pass-through-verbatim (never wrong-value); forks are counted and surfaced (`compaction_forks`), and relink is a rejected design (docs/compaction-relink.md). Bare-digit phones/SSNs never fire (collision-prone); street addresses and passport/DL numbers are out of scope (no reliable grammar); `+`-prefixed integer literals (INT_MAX in code) match the solid E.164 phone form — pinned in the fp-corpus manifest. Warn mode is observation only: the matched value (and anything a longer warn-mode match overlaps) IS forwarded upstream — docs must never imply protection. The same honesty applies to the other deliberate opt-outs: `[providers.NAME] detection = false` (whole provider unredacted, rehydration active), `[detection.mcp] exempt_servers` (named MCP servers' blocks), and `[detection] languages` (scoped-out national-id rules are not built) — all surfaced in /status, never silent. Vault encryption is at-rest only (memory holds plaintext) and the v2→v3 migration is one-way. RDBMS vaults fix the encryption mode at schema creation (no encrypt-in-place, no key rotation — server MVCC would keep scrubbing-proof old row versions; stand up a fresh schema instead), and `vault verify/rotate-key/backup` are sqlite-only. Multi-instance sharing of one vault database is safe but pays a round trip per first-sight allocation. The config editor rewrites the TOML file without preserving comments (one .bak kept). Supported platforms are Linux and macOS (both CI-tested); Windows is unsupported (SIGHUP reload alone would not save it). Non-loopback binding is fail-closed behind mutual TLS (`[tls]` certfile+keyfile+client_ca, enforced by `serve`); keep the proxy on 127.0.0.1 unless you operate a client-cert PKI — the written threat model lives in `docs/threat-model.md`.