git:20260904.426c934 to git:20260904.4387701

# CLAUDE.md - charts/curie

The umbrella Helm chart: Langfuse + Postgres + Valkey + ClickHouse + RustFS +

OTel Collector, plus the Agent Sandbox substrate and its security rails. Full

component and rail detail in `charts/curie/README.md`.

## Load-bearing invariants

- **Values, not templates, for anything that varies by environment.** Per

the platform-level rule this repo also follows: resource limits, replica

counts, and probe settings belong in `values.yaml` / a values overlay, not

hardcoded in `templates/`. A value safe on a 4 GB scratch cluster can OOMKill

on a smaller install.

- **One recorded exception: `templates/mail-adapter.yaml`**, which hardcodes

`replicas: 1` and `strategy: type: Recreate`. The reason is correctness, not

sizing: the adapter is one serialized SQLite writer on a single-writer

`ReadWriteOnce` or `ReadWriteOncePod` PVC.

A second replica or a rolling update creates two writers and defeats the

ownership/lease invariant even on storage that happens to multi-attach. Any

count other than 1 is wrong at every cluster size, which is exactly what

makes it unlike the resource limits and probe settings the invariant is

about, and `Recreate` is required for the same reason (a rolling update runs

two pods for the duration of every upgrade).

There is deliberately **no `mailAdapter.replicas` key**: a values key that

must never be changed advertises a knob that silently breaks reply routing,

which is worse than no knob. Pinned behaviorally by

`ci/mail-adapter-wiring-assertions.sh` assertion 10, which asserts

`spec.replicas` is 1 even under `--set mailAdapter.replicas=3`. Horizontal

scale for this adapter needs an Accepted shared-store/multi-writer design;

when it lands, this exception is removed rather than extended. Do not

generalize it into a rule about stateful services -- one named file, one

named reason.

- **Second named exception: `templates/langfuse.yaml`**, which hardcodes

`replicas: 1` and `strategy: type: Recreate` on both the web and worker

Deployments. The reason is correctness, not sizing: both images run Prisma

and ClickHouse migrations at container boot with no cross-process lock, so

a RollingUpdate of a single replica (default `maxSurge: 25%`, which rounds

up to one extra pod) can apply two migration sets to the same database and

leave Prisma in a failed state (#2216). Any count other than 1, or any

strategy other than Recreate, is wrong at every cluster size. There is

deliberately **no `langfuse.web.replicas` / `langfuse.worker.replicas` /

strategy values key**. Pinned by `ci/langfuse-recreate-assertions.sh`.

Horizontal scale for Langfuse needs an Accepted out-of-band migrator; when

that lands, this exception is removed rather than extended.

- **Mail-adapter egress is a separate fail-closed rail.** Enabling

`mailAdapter.deploy` requires at least one

`mailAdapter.agentmail.httpsCidrs` entry. Its single egress-only policy allows

DNS, this release's API pods, and those CIDRs on TCP 443; with `api.deploy`

false, `mailAdapter.apiEgress.httpsCidrs` and `.port` replace the pod selector

with an explicit narrow BYO-API peer. It never selects a runner sandbox and

never allows the Kubernetes API. The runtime pod mounts no ServiceAccount

token and has no RBAC. Prefix-0 and prefix-1 routes fail render, including

split default routes. Do not turn provider DNS into a broad CIDR or add a

private-network allow: use current provider ranges or a controlled egress

proxy with a stable range.

- **Every backing store follows the same toggle + BYO idiom.** `<store>.deploy`

(default `true`) gates whether the in-chart resource renders; flipping it

to `false` repoints consumers (Langfuse env, the collector config) at the

BYO `host`/`port`/`auth`/`existingSecret` fields on the same block. A new

backing store must follow this exact pattern -- do not add a store with a

different enable/disable shape.

- **Values keys are camelCase, not hyphenated.** Go templates cannot

dot-index a hyphenated key. Keep this consistent across any new values

additions.

- **Placement-class lookups go through `curie.placement.class`; never

index `.Values.placement.<class>` directly in a template.** Helm's

values coalescing deletes a key whose replayed value is YAML null, so

`helm upgrade --reuse-values` on a release created before placement

classes existed -- which stored `placement: null` -- leaves

`.Values.placement` nil, and a direct dereference crashes the render

before any cluster mutation (#2008). A nil or missing tree/class

degrades to the chart's empty defaults; a tree or class that is present

but not a map is refused with `fail`, deliberately -- `fromYaml` returns

an error map rather than an error for a non-map document, so softening

that refusal into a default would silently drop every scheduling

constraint the operator asked for. A new pod surface added to the chart

uses the helper with its class name rather than a fresh direct index.

- **Fail-closed egress, always.** `security.networkPolicy.allowedEgress` is

empty by default; an unset allowlist must never mean allow-all. If you add

a new egress destination the runner needs, it goes into this allowlist

explicitly -- never widen the default-deny baseline itself. The exception

is a BYO object store (`rustfs.deploy: false`): that allow is

`rustfs.egress` (and `rustfs.stsEgress` on the key-free path), required at

render, not mixed into the model-API allowlist.

- **NetworkPolicy allows are additive, never restrictive-intersecting

(#765, ADR-0067).** A second NetworkPolicy selecting the same pods can only

widen what Rail 1 permits, never narrow it -- there is no such thing as one

policy overriding another. This is why the runner SandboxTemplate sets

`spec.networkPolicyManagement: Unmanaged` whenever Rail 1 is on: it stops the

vendored controller from reconciling its own separately-managed, broader

egress policy for the same pods. Do not add any other NetworkPolicy-adjacent

mechanism (another controller, an operator, a second chart) that could select

`component: runner-sandbox` pods without checking it does not reintroduce

this exact union-defeats-default-deny failure mode.

- **The preflights are mandatory Helm hooks, not advisory scripts.** The

CPU-AVX/ClickHouse-pin check (`preflights.avxCheck`), the

NetworkPolicy-enforcement probe (`preflights.networkPolicyProbe`), and the

controller-ready gate (`preflights.controllerReady`, which fails the install

if the vendored agent-sandbox controller cannot sync its cluster-scope

NetworkPolicy informer -- issue #350) block a broken install. Do not make

any of them skippable by default, and do not add a new cluster-dependent

assumption (a CNI feature, a kernel feature, an RBAC grant the controller

needs to start) without a matching preflight -- an assumption that silently

fails on a customer cluster is exactly the failure mode these exist to

prevent.

- **gVisor needs `runsc` on the node; the chart cannot install it.** On a

cluster without it, use the ready-made overlay

`-f charts/curie/values-e2e-nogvisor.yaml` (sets `runtimeClassName=""` and

disables the gVisor preflight, leaves every other rail on) rather than

hand-editing `security.gvisor.*` -- the overlay is the supported opt-out

path for e2e/scratch clusters.

- **CRDs in `crds/` are vendored, never templated.** Helm does not

upgrade or delete `crds/` content; a teardown needs a manual

`kubectl delete crd <name>`. Do not move CRD definitions into `templates/`

to make them "manageable" -- that changes install ordering guarantees

Helm's `crds/` convention provides.

- **The controller (`agentSandbox.controller.deploy`) is cluster-scoped.**

Install it from exactly one release per cluster; leave it `false` on any

cluster that already runs `agent-sandbox`. Do not default this to `true`

in a values file intended for a shared/multi-release cluster.

- **The runner image is `IfNotPresent` + prewarm, NOT `Always`.** Images pull

from GHCR; the four Deployment-managed services default to `Always` (fresh

`:latest` on every rollout), but the runner must not -- a sandbox pod is

created per Slack thread, and an in-boot image download can blow the

worker's claim timeout (live incident 2026-07-06). The runner-prewarm

DaemonSet (`agentSandbox.runner.prewarm`) pulls the runner image at

install/upgrade instead, and every `helm upgrade` rolls it to refresh the

cache. Do not flip the runner to `Always` and do not disable the prewarm

on `:latest`-tag clusters without accepting stale-image risk.

- **`values.schema.json` is deliberately permissive, not a full contract.**

The chart had no values schema at all before issue #1388; Helm now

validates the ENTIRE coalesced values tree against `values.schema.json`

on lint, template, install, and upgrade -- not just the keys a given

operation touches. Because of that blast radius, the schema stays

draft-07 with no top-level `required` and no `additionalProperties:

false`, and it types only four bounded values: the three worker knobs

(`worker.claimTimeoutSeconds`, `worker.routeTtlSeconds`,

`worker.suspendedRouteTtlSeconds`) and the approval-chat attester's explicit

nonblank contract (`api.approvalChatAttesterSecret`). Adding a `required` or

`additionalProperties: false` constraint, or typing any other existing

key, would fail every install whose values file happens not to match the

new shape -- broaden the schema only for a key you are prepared to

validate across every current values file and overlay.

- Known trap: `api.githubAppId` must stay untyped. It reaches the

coalesced values tree as a JSON string when `curie cluster

github-app` sets it via `--set-string` (`cli/src/github_app.rs`,

deliberately, to dodge Helm's float64 round-trip on a bare numeric

`--set`), but as a number if anyone sets it with a plain `--set` or an

unquoted YAML override. Typing it either way in the schema breaks the

other path.

- Corollary: `--set-string` on any of the three bounded worker knobs now

fails by design -- a JSON string violates `type: integer` /

`type: number` even though the rendered env var is itself a quoted

string. The attester remains a string, but an explicitly supplied empty

or whitespace-only string fails by design; omitting it still delegates to

the managed-secret generator.

- The schema is not the only gate. Helm drops nil-valued keys during

values coalescing before schema validation runs, so

`--set worker.routeTtlSeconds=null` is not caught by the schema; it

renders an empty `CURIE_ROUTE_TTL_SECONDS` env value and is caught

instead by the worker's own boot-time refusal. Both checks are

load-bearing -- the schema catches a bad value early for the common

cases, the worker's refusal is the backstop for the coalescing gap the

schema cannot see. `placement` is the second instance of this exact

coalescing gap -- a release created before placement classes existed

can store `placement: null`, and `--reuse-values` on upgrade replays

it, deleting the key -- and the `curie.placement.class` helper (see

the values-keys invariant above) is its template-level backstop

(#2008). `worker.publication.githubHttpsCidrs` is a third instance:

`minItems: 1` catches `[]`, but a coalesced nil deletes the key and a

`range` over it used to emit an empty `to:` (allow-all on 443 for the

tokenless publication job). The template `fail` in

`publication-owner.yaml` is the backstop (#2321).

## Verify

Static / chart-authoring checks (they render manifests but NEVER run a container,

so they cannot catch a bug that only surfaces at runtime):

```bash

helm lint charts/curie

helm template charts/curie -f charts/curie/values-dev.yaml # chart-authoring check, no cluster contact

```

Runtime check (the cheap default for a chart/sandbox/bundle change): installs a

trimmed slice, runs the bundle-fetch init pair, and exec-asserts on the runner:

```bash

curie dev chart-runtime-e2e # implemented by scripts/chart-runtime-e2e.sh

curie dev chart-runtime-e2e --force # same, on a scratch context not named k8scratch

```

A ticket whose AC is a runtime check (like #56, the bundle-fetch credential

isolation) is only satisfied by running this and pasting its output -- lint /

template do not exercise the init container or the live runner.

It refuses any kube context not named `k8scratch`, because it installs and

uninstalls a real release; `--force` is the override, and points at a disposable

cluster only. The other script flags (`--namespace`, `--release`, `--chart`,

`--runner-image`, `--expect-vulnerable`, `--keep`) are not yet exposed through

`curie dev`; reach them with `bash scripts/chart-runtime-e2e.sh --help`.

Cluster verification (a disposable local cluster, `kind` or `k3s`):

```bash

helm install curie-dev charts/curie -n curie-dev --create-namespace \

-f charts/curie/values-dev.yaml

kubectl get pods -n curie-dev -w

helm test curie-dev -n curie-dev # re-runs the three preflights + the security probe suite

kubectl logs -n curie-dev job/curie-dev-preflight-avx

kubectl logs -n curie-dev job/curie-dev-security-probe # rails 1, 2, 4

kubectl logs -n curie-dev curie-dev-security-probe-hardening # rail 3

```