# proxy — the byte-safe standalone `caveman` proxy (commercial, binary-distributed)

A base-URL-swap reverse proxy: match → authenticate → inspect → byte-safe transform → upstream
→ meter. Single-operator, BYOK, **zero cloud dependencies**. It shares its provider adapters with
the managed gateway (the managed gateway imports them from here). `caveman start` launches the
`caveman-proxy` binary.

## Layout
- `providers/` — the shared, public byte-safe adapter set: `Adapter` interface + `Base` embed + `UsageScanner`/`ParseUsageBytes` (`adapter.go`), and `anthropic`/`openai`/`gemini`/`azureopenai`/`bedrock`/`vertex`/`openaicompat`. `ResolveUpstreamURL` takes `providers.RouteContext` (no control-plane coupling). Anthropic + OpenAI carry both prefixed and bare routes (`/v1/messages`, `/v1/chat/completions`). `vertex` preserves caller OAuth bearer or Express-mode Google API-key credentials for Gemini + Claude on Vertex AI (no signing, no environment-key fallback, no custom usage parser). `MatchRoute` is POST-only; read-only model discovery (`GET /v1/models`, `/v1/models/{id}`) is a separate GET-only allowlist, `Base.MetadataRoutes`, declared by `openai`/`anthropic`/`openaicompat` and forwarded unchanged with no body, no transform and no spend row.
- `internal/gateway/` — the request lifecycle (`server.go` + `proxy.go`) behind three injected seams: `Authenticator`, `CredentialResolver`, `TelemetrySink`. Ports the managed loop with the fail-open fix.
- `internal/config/` — `caveman.yaml` loader + BYOK env-key resolution; unknown mode fails closed to `record`.
- `internal/store/` — `~/.caveman/caveman.db` SQLite spend store (`modernc.org/sqlite`, cgo-free); implements `TelemetrySink`.
- `internal/standalone/` — wiring: static `Auth`, BYOK `Creds`, adapter set, and the always-on SSRF-guarded client.
- `internal/identity/` — who calls the framework middleware routes (legacy token, token map, OIDC/JWT, mTLS) and the reloadable TLS listener config.
- `cmd/caveman-proxy/` — binary: `serve` (default), `stats`, and content-blind
  `agent-evidence --session --build --plan`. Evidence query returns only exact
  provider usage, request hashes, declared context/plan identity, ordered
  provider-prefix component hashes, actual transform IDs/counts, and CCR handle;
  basis is always `inferred`, verified dollars always zero.

## Conventions
- Build/test: `make product-build PRODUCT=proxy` / `make product-test PRODUCT=proxy`.
- Tests inject a plain `*http.Client` to reach loopback stubs; the binary uses the SSRF-guarded client.
- New provider/optimizer work goes in `providers/` (shared) — change it once, both proxies get it.

## Gotchas (honesty invariants — correctness, not style)

- **listener lifetime is not session lifetime**: `serve` never exits because a
  wrapper, native session, heartbeat, or idle timer expires. Legacy
  `CAVEMAN_NATIVE_IDLE_TIMEOUT` does not arm shutdown. The CLI never restarts a
  shared listener to change mode/recovery; an incompatible new wrap runs direct.
  Failed local startup also runs direct. Native session correlation entries may
  age out without affecting API traffic.
- **no default generation deadline**: `CAVE_GATEWAY_UPSTREAM_TIMEOUT_MS` defaults
  to `0` (no total request deadline). A positive value is an explicit operator
  cap and includes response streaming. Client cancellation still cancels upstream;
  connection setup, inbound header/upload limits, and idle keep-alive socket
  cleanup remain bounded separately. The replacement bound for an upstream that
  connects and then goes silent is the transport's response-header deadline
  (`CAVE_GATEWAY_RESPONSE_HEADER_TIMEOUT_MS`, default 900000, `0` disables) —
  headers only, so it never truncates a live stream.
- **response protocol controls streaming**: SSE/event-stream responses flush
  headers and chunks even without a JSON `stream` flag. Encoded requests keep
  `Content-Encoding` and bypass transforms. Interrupted response copies record
  an error and abort HTTP framing; they never become a clean successful EOF.
- **replay only when non-delivery is proven**: proxy transport retries are
  limited to connection-setup failures — `*net.OpError` with `Op` `dial` or
  `proxyconnect` (a proxy that is down fails as the latter, never as a dial). A failed upload/header read or truncated response
  does not prove an inference was unprocessed; do not automatically replay it.
  An explicit transformed-request 4xx still retries once with original bytes —
  except a 429 that says it is a rate limit (`Retry-After` or
  `anthropic-ratelimit-*`), which is returned as-is; an accepted retry pins the
  conversation raw (`internal/gateway/raw_pin.go`).
  See `docs/technical/proxy-reliability.md` at the repository root.
- **byte-safe**: `record` mode never transforms; on transform error the ORIGINAL bytes are forwarded (HTTP 200, fail-open) — never a 400. The one carve-out: bytes the provider already cached in replaced form are re-sent replaced, because forwarding the original there busts the prefix (see cache safety).
- **request-wide opt-out**: `x-cave-transforms: caveman.pass-through.v1` suppresses every request transform path — compress, pixel, and provider-native — not only compiled plan routes. Tests cover all three modes.
- **no-fake-savings**: standalone records `Basis: "inferred"` on every row; it never writes `verified` and never re-projects to a monthly figure.
- **practice join**: local learn sinks carry additive `practice_id`; one
  fail-closed mapping table owns sink→practice and unknown sinks keep `""`.
  The historical `subagent_overuse` sink is count-only and deliberately has no
  practice id: spawn count cannot reactivate the retired
  `context-exploration-offload` opportunity or prove any spawn unnecessary.
- **local trial heuristics are not actuation evidence**: a model name never emits
  the retired `model-right-sizing` id, and provider plus positive cost never
  emits a cache move because neither proves stable-prefix eligibility. Legacy
  rows for those identities are hidden at read time. Compression replay reports
  one trial's local engine `estimated_engine_o200k` before/after shape with zero
  dollars and low confidence; it is not provider-counted, a rate, an invoice,
  causal/verified savings, or task-outcome evidence.
- **Anthropic automatic caching is experimental observation only**:
  `anthropic-automatic-prompt-cache` is a typed, default-off manual policy
  experiment and may add only Anthropic's top-level 5-minute marker on direct
  Messages API requests. Managed traffic additionally requires server-attested
  official Anthropic origin; custom or provenance-unknown origins lose the flag
  before the adapter. It is mutually
  exclusive with the explicit `anthropic-cache-breakpoints` transform and any
  caller `cache_control`; Bedrock and count-tokens requests stay byte-identical.
  An applied marker records only its optimizer id plus actual provider usage and
  cost. It has no practice, recipe, generic mode/candidate activation, ledger
  tuple, inferred savings, or verified-savings path (cache-only and forged IDs
  are excluded from the counted-baseline method too); evaluate it by manual
  paired observation because shared provider cache state can contaminate an A/B.
- **SSRF always on**: `standalone.StandaloneHTTPClient` guards every upstream dial (not gated on `CAVE_ENV`) using `ssrf.SelfHostedConfig` — NOT ManagedConfig, which ignores the allowlist and would make the escape hatch a silent no-op. `CAVE_SSRF_ALLOWLIST` opts loopback/private hosts back in (local model servers like Ollama; `localhost` as an entry covers 127.0.0.0/8 + ::1); metadata/link-local stay blocked in every mode. Provider traffic honours `HTTPS_PROXY`/`NO_PROXY` by default (#1001) — `upstream_proxy: env|off|<url>` / `CAVE_UPSTREAM_PROXY` — via `ssrf.Config.Proxy`, which validates IP-literal/localhost destinations before proxy selection and dials the (operator-configured) proxy address unguarded; `ssrf.NewHTTPClient` itself stays direct unless a caller sets `Proxy`, and a managed-mode Config carrying one is refused loudly (every request fails with `ssrf.ErrProxyInManagedMode`) rather than silently dialing direct. Adapters that pre-flight a resolved endpoint (bedrock, vertex) read the selector off the request context (`providers.WithUpstreamProxy`, published by `gateway.Handler` from the upstream transport) and use `ssrf.ValidateURLNoResolve` for proxied destinations: a proxy-only network has no outbound DNS, so resolving there failed the request before the proxy was ever consulted. `ca_bundle` / `CAVE_CA_BUNDLE` plus inherited `SSL_CERT_FILE`/`REQUESTS_CA_BUNDLE`/`NODE_EXTRA_CA_CERTS` append private roots for TLS inspection through `shared/platform/cabundle` (fail-closed parser shared with chhttp).
- **inbound token is consumed, never forwarded**: `CAVEMAN_AUTH_TOKEN` gates non-loopback listens (a middleware token map, OIDC issuer or TLS client CA also makes one legal; provider routes still accept only the token, and without it `Auth.closed` refuses every provider request off loopback); `standalone.Auth` deletes the matching `x-cave-api-key`/`Authorization` header before `Creds.Resolve` so the shared token can never be forwarded to a provider or classified as a provider credential; a bearer that is not the token survives untouched (Claude OAuth, `/chatgpt/`). `Authenticate` runs BEFORE `matchAdapter` so the 401/404 split cannot be used to enumerate routes, and every 401 goes through `Server.rejectUnauthorized`, which counts `cave_proxy_unauthorized_total` and warns with the path and remote host only. Health and the no-op `/caveman/keepalive` beacon stay unauthenticated, `/metrics` too unless `CAVEMAN_METRICS_TOKEN` is set; the `X-Caveman-Instance` header on `/health/live` is published only on a loopback bind. A token on a LOOPBACK listener is honored too (the local `caveman wrap` path sends none, so `runServe` warns). Bedrock credentials then come from `shared/platform/awscreds` (env → web identity → container → IMDSv2), never from the request; a PARTIAL env pair fails closed instead of falling through to an ambient role, and the winning source is logged once.
- **middleware identity is server-side and per route**: `middleware.Config.Identify` resolves an `identity.Principal` from the credential alone (bearer decides when present, else a verified client certificate); `ServeHTTP`'s `decode` checks `Principal.Allows(scope.Namespace)` on EVERY scoped route, so a new route that skips `decode` skips authorization. The legacy token stays `single_operator` with every namespace; JWT and certificate principals may never be named `single_operator`. Token and hash comparisons are constant-time over every entry. A token map or TLS file that fails to reload keeps the previous one. Tests: `identity/*_test.go` (negative JWT and mTLS cases), `middleware/identity_test.go`.
- **Postgres middleware store keeps SQLite's single-writer semantics per authority**: `store.PostgresMiddleware` serializes writers of one authority with a transaction-scoped advisory lock taken by `Scope` (then re-read), `SaveScope`, `Revoke`, `PurgeBatch`, `Receipt` and `SaveOriginal`; the sweep is one-replica (try-lock) and skips busy authorities and row-locked scopes. Counters are statement-level transition-table triggers striped by txid; a per-row trigger goes quadratic on bulk statements. Lock keys include `current_schema()`. Run the store suite against both backends with `CAVEMAN_TEST_POSTGRES_URL`; `middleware_concurrency_test.go` fails without the locks.
- **Auth scheme is preserved**: a key from an inbound `Authorization: Bearer` keeps `Scheme:"bearer"` on the `providers.Credential`; Anthropic and Gemini forward bearer credentials as bearer credentials (Claude/Gemini OAuth breaks if remapped to an API-key header). BYOK env keys and inbound `x-api-key` keep provider API-key mapping.
  On a named compat mount the header follows the wire protocol of the path:
  `/compat/<name>/v1/messages` gets `x-api-key` plus a default
  `anthropic-version`, every other path gets Bearer. A real inbound Bearer stays
  Bearer on every path. OpenCode Go rejects Bearer
  on `/v1/messages` (401 `Missing API key.`, 2026-09-03). The gateway env
  fallback puts the key in the header that the adapter emitted, so the upstream
  never gets two credentials.
- **fail-closed**: unknown route → 404; unknown mode → `record`.
- **subscription AND oauth compression is NOT account-gated**: non-PAYG sessions from Claude Code, Codex ChatGPT, Gemini CLI, and other routed clients take live-zone compression with no Caveman account, entitlement, or seat. `CAVEMAN_WRAP_ENTITLED` and every `WrapEntitled` field are **deleted**, not merely ignored — do not reintroduce them. Exactly four conditions remain, all technical and all fail-closed (`liveZoneCompressionAllowed`): the operator `subscription_compress` switch (empty/`live_zone` allow, `off` and any unknown value close it), the adapter must implement schema-aware `PrefixStabilizer` zones, recovery must run through the agent's own MCP `caveman_retrieve`, and a durable prefix cache must be wired. The dedicated Codex `/chatgpt/responses` route uses the OpenAI Responses stabilizer while preserving OAuth and `ChatGPT-Account-ID` headers; transformed 4xx responses retry once with exact original bytes. Legacy savings fields remain **tokens-only** — `compression_tokens_before/after` + `estimated_engine_o200k`, never booked compression dollars. New request-comparison and price-snapshot fields support separately labeled API equivalents in local stats; those values never enter subscription spend or legacy saved-dollar fields. LOCAL wrap only; managed gateway non-PAYG behavior is unchanged.
- **cache safety is byte-stable replacement**: THE INVARIANT (#1105): for accepted requests P then R of one conversation, if R's client bytes over P's cached prefix (system, tools and messages through P's last `cache_control` marker; the whole prompt for implicit caches) equal P's, R's forwarded bytes over that range equal P's. Sole exception: a request the provider rejected transformed and accepted via the raw retry; later requests are held to that raw request (raw pin). `internal/gateway/cache_prefix_invariant_test.go` (scenarios + `FuzzCachePrefixInvariant`) proves it; keep it green. A compressed live-zone turn becomes prefix on the next request, so the replacement is a pure function of segment content (deterministic compressor + content-hash CCR marker), memoised in the durable replacement cache, and the FIRST decision for a block wins — replaced or raw (`RawDecisionHandle`). A frozen block with no decision (a `--resume` history, an evicted row) goes out raw and raw becomes its decision; a failed memo write keeps the block raw for the process (`rawMemory`); new rows are remembered only once the request's splice holds. Substitution is never gated: the derived epoch gate is gone, and an explicit framework epoch may veto only NEW compression. Pixel renders and the tool-schema strip are memoised the same way. New compression stays live-zone-only. The runtime tripwire (`prefix_monitor.go`) checks the same rule on live traffic and records `requests.cache_bust_cause`: `client` (DEBUG), `raw_retry` (WARN), `caveman` (ERROR `caveman changed bytes the provider already cached`, counted as `caveman_cache_bust_requests` and raised by `caveman status`) — any `caveman` row is a bug. Cross-turn stability covers **anthropic, openai, azureopenai/openaicompat, and gemini** through `ExtractStabilizable`; `bedrock` and `vertex` expose no compressible blocks and stay pass-through. Anthropic uses declared `cache_control`; OpenAI/Gemini use latest-user/latest-tool zones against implicit provider caches. Replacement cache is SQLite spend store and must retain `journal_mode(WAL)` + `busy_timeout`.
- **pixel mode**: S4 lossy text→PNG (`pxpipe` port). Default allowlist is `claude-fable-5,gpt-5.6` via `CAVE_PIXEL_MODELS`; original request is always in CCR before transformed bytes are sent; savings stay inferred-only; an error on new content sends that content raw and records it, never undoing an earlier turn's render (first decision wins, as for compression).
- **boundary**: this is public code — it must never import the managed-cloud lane. `make check-boundaries` enforces it.

See ../../CLAUDE.md (root)