# CLAUDE.md — context for AI collaborators This file is for Claude Code (and similar coding agents) working in the `rote` repository. Read it before making non-trivial changes. It captures the project's mental model, architectural invariants, gotchas, and the canonical examples you should imitate. If you're a human reading this, you probably want [README.md](README.md) or [CONTRIBUTING.md](CONTRIBUTING.md) instead — this file is deliberately oriented toward an AI that needs a fast mental model before it starts editing code. --- ## What rote is `rote` is a CLI that takes an Anthropic-style Skill (a `SKILL.md` + `references/`) and compiles it into a runnable background pipeline: 1. An LLM agent (itself defined as a skill at [`skills/rote-compile/`](skills/rote-compile/)) reads the source skill and applies a structured rubric. 2. The agent produces a `pipeline.yaml` (runtime-agnostic IR) plus extracted Python modules and typed LLM-judge signatures. 3. A runtime adapter (DBOS by default; Temporal, Cloudflare Workflows, DBOS-TS, and Inngest too; more planned) consumes the IR and emits the runtime's native code shape — a single `main.py` DBOS app, a `workflow.py` + `activities.py` pair for Temporal, a TypeScript `WorkflowEntrypoint` class plus `wrangler.jsonc` / `package.json` for Cloudflare, or an Inngest function app that mounts into an existing Node/Next.js service. The result is a deterministic workflow that replaces a fuzzy 10-20 minute agent loop. That's the whole product in one paragraph. --- ## Three-layer architecture Every piece of code in this repo belongs to exactly one layer. **Respect the layer boundaries** — most of the project's value comes from keeping them clean. ### Layer 1 — The compiler agent (the "brain") - **Lives at:** [`skills/rote-compile/`](skills/rote-compile/) - **Consists of:** a `SKILL.md` plus five rubric files (`node-kinds.md`, `crystallization-heuristics.md`, `ir-schema.md`, `llm-judge-extraction.md`, `implementation.md`) - **What it does:** classifies every step of a source skill into one of 5 node kinds, extracts deterministic procedures into Python modules, designs typed signatures for LLM-judge steps, and assembles the whole thing into a `pipeline.yaml` - **Runs under:** a pluggable driver layer (see Layer 2) **When you edit this layer:** you are changing how the agent behaves. Changes are testable by running `rote compile` end-to-end against `examples/bdr-outreach/skill` and diffing the output against the previously committed snapshot in `examples/bdr-outreach/runs/`. ### Layer 2 — The driver layer (the "runtime for the agent") - **Lives at:** [`src/rote/compiler/drivers/`](src/rote/compiler/drivers/) - **Protocol:** every driver implements `CompilerDriver` with `name`, `is_available() -> (bool, str)`, and `async run(skill_dir, compiler_skill_dir, work_dir) -> DriverResult` - **Three concrete drivers:** `ClaudeDriver` (subprocess to `claude -p`), `CodexDriver` (subprocess to `codex exec`), `AnthropicApiDriver` (in-process via the `anthropic` SDK) - **The contract is the filesystem:** every driver writes `work_dir/pipeline.yaml` and returns a `DriverResult` pointing at it. Do not invent new return-shape fields or side-channel comms. **When you edit this layer:** you are changing how the agent is invoked, not what it does. Changes are testable by mocking subprocess spawning (see [`tests/test_claude_driver.py`](tests/test_claude_driver.py)) or the Anthropic SDK client (see [`tests/test_anthropic_driver.py`](tests/test_anthropic_driver.py)). ### Layer 3 — The runtime adapters (the "code emitter") - **Lives at:** [`src/rote/adapters/`](src/rote/adapters/) - **Five adapters today:** - `DbosAdapter` ([`src/rote/adapters/dbos.py`](src/rote/adapters/dbos.py)) — the CLI default; emits a single durable Python app (`main.py` with `@DBOS.workflow`/`@DBOS.step`, plus `signatures/*.py`, `extracted/*.py`, `dbos-config.yaml`, `README.md`) — no orchestrator process, state in SQLite/Postgres - `TemporalAdapter` ([`src/rote/adapters/temporal.py`](src/rote/adapters/temporal.py)) — emits Python (`workflow.py` + `activities.py`) - `CloudflareAdapter` ([`src/rote/adapters/cloudflare.py`](src/rote/adapters/cloudflare.py)) — emits TypeScript (`src/workflow.ts` extending `WorkflowEntrypoint`, plus `signatures/*.ts`, `extracted/*.ts`, and a `wrangler.jsonc` / `package.json` / `tsconfig.json`) - `PythonAdapter` ([`src/rote/adapters/python.py`](src/rote/adapters/python.py), registered as `python`) — the maximum-legibility target: emits a plain, orchestrator-free Python script (`main.py` with one function per node, visible inline retry loops, stdlib `ThreadPoolExecutor` for parallel waves, plus `extracted/`, `signatures/`, `requirements.txt`). **Refuses pipelines with `hitl_gate` nodes at emit time** (via the derived `Pipeline.requires_durable_execution` property) — a plain script cannot durably park for human approval; the error points at `--runtime dbos`. Shares the Pydantic signature / extracted-stub emitters with the DBOS adapter via [`src/rote/adapters/_py_common.py`](src/rote/adapters/_py_common.py). - `DbosTsAdapter` ([`src/rote/adapters/dbos_ts.py`](src/rote/adapters/dbos_ts.py), registered as `dbos-ts`) — emits TypeScript for DBOS Transact (`src/main.ts` with `DBOS.registerWorkflow` / `DBOS.registerStep` function wrappers, plus `signatures/*.ts`, `extracted/*.ts`, and `package.json` / `tsconfig.json` / `dbos-config.yaml`). Shares the TS emission machinery in [`src/rote/adapters/_ts_common.py`](src/rote/adapters/_ts_common.py) with the Cloudflare adapter. Note: the DBOS **TS** SDK is Postgres-only (no SQLite parity with DBOS Python), and its `DBOS.recv` defaults to a 60s timeout — the emitter always passes the IR timeout explicitly. Targets DBOS 3 (Python) / Transact v5 (TS); see the DBOS 3 queue gotcha below. - `InngestAdapter` ([`src/rote/adapters/inngest.py`](src/rote/adapters/inngest.py), registered as `inngest`) — emits a TypeScript Inngest app (one durable `inngest.createFunction` in `src/inngest/pipeline.ts` running the DAG waves, `step.waitForEvent` per HITL gate on a `/` event, plus a framework-neutral `inngest/node` serve entrypoint and a README documenting the Next.js mount). Shares `_ts_common.py` with the other TS adapters. Note: Inngest v4 retries are **function-level only** — the emitter maps max-across-nodes and comments the per-node deltas (see the Inngest gotcha below). - **What they do:** consume a validated `Pipeline` IR and write runtime-native code into an output directory - **Never run an agent loop.** Code emission is pure template substitution. The agent's job ends when the IR is valid; the adapter's job begins there. **When you edit this layer:** you are changing the output code format or adding a new runtime. Changes are testable with unit tests covering emission (AST / textual invariants), plus a real runtime smoke test — `tests/test_temporal_e2e.py` for Temporal (time-skipping `WorkflowEnvironment`); `tests/test_cloudflare_e2e.py` for Cloudflare (runs `npm install` + `tsc --noEmit` on the emitted output, gated by `@pytest.mark.slow`); `tests/test_dbos_ts_e2e.py` for DBOS TypeScript (`npm install` + `tsc --noEmit`, plus a live run on the real DBOS TS runtime against a Docker Postgres); `tests/test_inngest_e2e.py` for Inngest (`npm install` + `tsc --noEmit`, plus a live run through both HITL gates against the real Inngest dev server, `inngest-cli dev`). --- ## The IR is the source of truth [`src/rote/ir.py`](src/rote/ir.py) defines the Pydantic models that every layer agrees on. If you're uncertain about a field's semantics, read that file — it's ~290 lines, fully typed, and the authoritative reference (not the rubric's `ir-schema.md`, which is agent-facing prose that can drift). **The five node kinds:** | Kind | Required field | Where the LLM lives | | --- | --- | --- | | `pure_function` | `impl` | Not involved | | `external_call` | `impl` | Not involved | | `llm_judge` | `signature` (legacy path) and/or `signature_spec` (structured) | Typed signature (DSPy / BAML in Python; Zod + vendor SDK in TS) | | `agent_loop` | `tools` | Bounded agent loop | | `hitl_gate` | `signal` | Durable suspend/resume | **The `signature_spec` model** ([`src/rote/ir.py`](src/rote/ir.py)) carries a JSON Schema for input + output, a Jinja-style prompt template, and the LLM client config (anthropic / openai). It's the runtime-agnostic alternative to the legacy `signature: 'path/to/file.py:Class'` form, which only works for Python adapters. Cloudflare requires `signature_spec`; Temporal accepts either and prefers `signature_spec` when present. **The canonical skill that covers all 5 kinds:** the BDR outreach example in `examples/bdr-outreach/skill/`. If you need an IR that exercises every code path, its hand-drafted baseline is at `examples/bdr-outreach/expected/pipeline.yaml`. --- ## Gotchas (things you will absolutely trip on if you don't read this) ### `claude -p` and the `ANTHROPIC_API_KEY` trap Claude Code's print mode (`claude -p`) has a documented behavior where `ANTHROPIC_API_KEY` **always wins** over any active OAuth session. If your system has the env var set, `claude -p` will use API-key billing even though the user expected subscription auth. **The fix:** `ClaudeDriver._build_child_env()` scrubs `ANTHROPIC_API_KEY` and `ANTHROPIC_AUTH_TOKEN` from the child environment before spawning. **Do not remove this.** If you need API-key auth to Anthropic, use `AnthropicApiDriver` instead — it's a separate driver precisely so the Claude driver can force subscription auth. ### YAML 1.1 parses `on:` as boolean `True` PyYAML follows YAML 1.1, which has historical bool keys (`on`, `off`, `yes`, `no`). A field literally named `on` in a YAML file becomes the Python value `True`, not the string `"on"`. This **broke the first IR test run** when I tried to use `on:` for retry trigger conditions. **The fix:** the IR uses `retry_on:` (with the `_on` suffix), not `on:`. Propagate this convention if you add new fields. ### Loop body sub-nodes are excluded from top-level waves An `agent_loop` node can declare a `loop_body: [node_id, ...]` listing sub-nodes invoked per iteration. Those sub-nodes also exist as top-level entries in the `nodes:` list (so they can be tested in isolation) but the adapter's `_execution_waves` function filters them out of the top-level DAG — otherwise the workflow would dispatch them twice. See `rote.adapters.temporal._execution_waves` for the reference implementation. ### Driver factories accept `**kwargs` The driver registry's factories have signature `**kwargs -> CompilerDriver` because `Compiler(model=...)` needs to pass kwargs through at construction time. Drivers that don't care about a particular kwarg should swallow it via `**kwargs` in their `__init__`, not hard-fail. **Do not change the factory signature to typed keyword args** — that breaks the registry's polymorphism. ### Emitted code touches MCP only through an explicit `mcp:` binding Two regimes, don't mix them up: - **Nodes without an `mcp:` binding must never reference MCP** in any adapter or language — no `mcp` imports, no `@mcp.tool()` decorators, no MCP SDK dependencies. Enforced per adapter: Temporal — `tests/test_temporal_adapter.py::test_emitted_activities_never_reference_mcp` (Python AST walk); Cloudflare — `tests/test_cloudflare_adapter.py::test_emitted_files_never_reference_mcp` (every `.ts` file, comments + string literals stripped). Copy the matching test when adding an adapter. - **Nodes WITH an `mcp:` binding** (DBOS adapter, `--backend mcp`, the default) emit a *working* FastMCP call: the step goes through the emitted `extracted/_rote_mcp.py` helper — the verbatim source of `rote.mcp._runtime_helper` (never hand-edit the emitted copy; fix the module) — which resolves the endpoint (env > rote registry > IR) and credentials (`rote mcp login` token store > registry static headers > unauthenticated) at runtime. On DBOS Python AND DBOS-TS, auth failures **park the workflow durably** instead of failing it (`RoteMcpAuthNeeded` → `DBOS.recv` on `rote:auth:`; `rote mcp login` — or `rote mcp release` without a login — discovers parked workflows via the app registry + the `rote_auth_status` event and releases them, one Python code path for both runtimes). Traps if you touch this: a parked DBOS workflow is just PENDING (no WAITING status — discovery *requires* the advertised event); parallel-wave siblings can surface stale auth failures after the release signal was already consumed — hence the retry-once-before-parking shape in the emitted wrappers; and the cross-language leg only works in DBOS's *portable* serialization — the emitted TS `setEvent` passes `{ serializationType: "portable" }` and the Python release `send` passes `WorkflowSerializationFormat.PORTABLE` (the defaults, superjson and pickle, are mutually unreadable). In TS, detect the auth error by `name` string, never `instanceof` (serialized errors reconstruct as plain `Error`s). See [`docs/mcp-client.md`](docs/mcp-client.md). ### Sync Python runtimes run async work on ONE background loop python and dbos steps are plain functions, so every coroutine they need (an `async def` extracted function, a dbos MCP step, a legacy async judge) goes through the emitted `_run_async`, which submits it to one daemon-thread event loop per process. Do not "simplify" it back to `asyncio.run`: that makes and closes a loop per call, so a module-level `httpx.AsyncClient` fails from its second call on ("Event loop is closed"), which is every fan_out element and every retry; and it raises inside a running loop, where `run_pipeline` lands when async code imports it. `_call_impl` comes in two forms from one place (`_py_common.call_impl_helper`, `is_async=True` for temporal), and both check the *result* for awaitability, so a plain wrapper returning a coroutine works on every runtime (`tests/test_call_impl_helper.py`). ### A missing MCP endpoint is configuration: never retried, never parked `resolve_url` / `resolveUrl` raise `RoteMcpConfigError` (Python and all three TS helpers, detected by `name` in TS like the auth error) when no endpoint resolves. Every runtime's retry opt-out excludes it next to the auth error, so a step fails on its first attempt with a message naming `ROTE_MCP__URL`. Agent loops reach the same gap through tool discovery, so `bindAgentTools` / `_mcp_config_for` raise the config error when it explains a missing tool, and loop steps carry a name-based opt-out (by name so `main` never imports MCP code, invariant #3). It must not join the park path: a parked run waits for a login, and no login supplies an endpoint. Before this, DBOS spent 4 attempts and surfaced only `DBOSMaxStepRetriesExceeded` (`tests/test_mcp_config_error_e2e.py`). On Cloudflare local dev the instance status then reads only `WorkflowFatalError`; the real message is in workerd's console output, which `rote.runners.cloudflare` reads back from `wrangler-dev.log`. ### Never return fastmcp's `result.data` from an MCP step `_result_payload` in `rote.mcp._runtime_helper` must NOT be "simplified" to `return result.data`. Re-verified on fastmcp 4.1.1 against real TS-SDK servers (v1 and v2) and five hosted servers, it fails both ways: - **None for text-only results.** fastmcp hydrates `.data` only from structured content. FastMCP servers auto-generate output schemas (which is why FastMCP fixtures never caught this), but real-world servers, the TS SDK in particular, return JSON as text and `.data` is silently None. Found live on the first real-server run: every MCP step returned None. - **Unpicklable for typed results.** With an output schema, `.data` is a generated `types.Root` class: DBOS cannot pickle it, and keys that are not identifiers come back renamed (`display-name` → `display_name`). Found live against `docs.mcp.cloudflare.com`. The payload is `structured_content` (the wire value, with fastmcp's `{"result": ...}` wrapper removed only when the server marked it), else the parsed text blocks. Two things to keep: the marker is `_meta.fastmcp.wrap_result` on current fastmcp but only the `x-fastmcp-wrap-result` output-schema flag on 2.x/3.0 servers, so the schema is still needed; and the TS helpers apply the same rule through `_FASTMCP_UNWRAP_TS` (one source spliced into the Node and Workers helpers). The text fallback is shared the same way: `_MCP_TEXT_PAYLOAD_TS` (spliced into the Node, Workers and binding helpers) applies `_result_payload`'s exact text rules, which are every text block parsed, non-JSON kept verbatim, several blocks as a list, and none as null. The TS helpers used to read only the first block, throw on non-JSON text and return `{}` for no text, so the same MCP step returned different values per runtime (`tests/test_mcp_text_payload_parity_e2e.py` runs both helpers against one real server). The helper's explicit `list_tools()` is not an extra round trip, because the MCP SDK's `call_tool` lists tools itself on a fresh session to validate the result (measured, and pinned by `test_one_tools_list_per_step`). Regressions: `tests/test_mcp_client.py:: test_result_payload_is_plain_json_from_a_real_fastmcp_server` and `tests/test_mcp_ts_server_e2e.py` (real TS-SDK servers, slow). ### fastmcp 4 hides HTTP status codes; the helper hooks the transport MCP SDK v2 turns any non-2xx answer into a bare `MCPError` (-32603, "Server returned an error response") with no cause chain, and the HTTP stack is httpx2, not httpx. So a 401 can no longer be found by walking the exception for `httpx.HTTPStatusError`; that check silently matched nothing and park-on-auth stopped working. `_HttpStatusWatch` builds the transport's httpx2 client with a response hook and remembers whether the latest answer was a 401. Regression: `tests/test_mcp_client.py:: test_call_mcp_tool_maps_a_real_401_to_auth_needed` (a real localhost 401, with a 500 as the negative half). Any new `except httpx.` around a fastmcp call is dead code for the same reason. Two smaller fastmcp 4 traps: `client.ping()` raises "Method not found" on the sessionless 2026-07-28 era, which `Client(url)` negotiates by default (use an authenticated `list_tools()` to trigger OAuth), and an SDK v2 `ServerSession` is per request, so anything that remembers a client across requests must hold the `Connection` instead (see `rote serve`'s list_changed delivery, which also has to publish on a `subscriptions/listen` bus for modern clients). ### MCP 2026-07-28: what keeps both eras and every registration mode working Seven things, each with a test that fails when it is undone (`tests/test_mcp_era_matrix_e2e.py`, `tests/test_mcp_oauth_registration_e2e.py`, `tests/test_mcp_park_e2e.py`). See [`docs/mcp-client.md`](docs/mcp-client.md). - **The TS helpers must pass `versionNegotiation: { mode: "auto" }`.** `@modelcontextprotocol/client` 2.x defaults to `legacy`, which fails against a modern-only server exactly like v1 (`-32022`). Checking the import line proves nothing; the era matrix asserts the era each server saw. A 401 is `SdkHttpError.status`; v1's `err.code === 401` reads an SdkErrorCode string in v2 and never matches, which silently turns park-on-auth into a failure. - **The CIMD document's redirect URIs and `CALLBACK_PORTS` are one fact.** A CIMD server compares redirect URIs by exact string match and no spec defines a loopback port wildcard, so the login binds a published port. `tests/test_mcp_client.py::test_cimd_document_matches_what_the_login_offers` pins `docs/oauth/rote-cli.json` to the code, and the copy served by rote-cloud's marketing site must match it too. Add ports to the document before the CLI uses them, never after. Selecting a port must bind it (`bind_callback_sockets`), and the callback server must run on those sockets: a check now and a bind later lost the port to a concurrent login in CI after the redirect URI already named it. - **rote refuses registrations the SDK would send on its own.** Without a `registration_endpoint` the SDK POSTs a guessed `/register` (GitHub: not its authorization server). `RoteOAuth._refuse_registration` stops that, the emitted helper refuses any registration (it parks instead), and an issuer-bound pre-registered client that the SDK dropped is not silently replaced by CIMD or DCR. The tests assert on the authorization server's hit log, because the failure is a request that should never have been sent. - **Refresh goes to the stored `token_endpoint`.** Before discovery the SDK refreshes at `/token`. For a path issuer or an AS on another host that 404s, and an unattended run parks on a credential that was fine. `retarget_token_request` is shared by the CLI and the emitted helper. FastMCP fixtures cannot catch this: their AS is the server's own origin root. - **Never offer the CIMD document unverified.** The SDK does not fall back to DCR when a CIMD server rejects it, so an unpublished or stale document breaks logins that DCR serves. `_check_offered_document` fetches it once per process when the AS advertises CIMD and withdraws the URL unless it equals the packaged `client_metadata.json`. - **A token file is bound to an endpoint and an issuer.** Files are keyed by server name, so `stored_for_url` gates every read (else a repointed name gets the old bearer token), and `_with_bound_issuer` fills `client_info.issuer` from the file (the SDK treats a missing one as matching every issuer). - **`RoteMcpInputRequired` is not an auth error.** A server that wants input mid-call (`input_required`, or `-32021` for an undeclared capability; TS SDK servers send the latter because rote declares no capabilities) fails the step on every runtime, and every retry opt-out exempts it. Making it a `RoteMcpAuthNeeded` would park the run for 30 days waiting for a login that answers nothing. ### An mcp-bound node is a BARE tool call — post-processing must split The default backend emits an mcp-bound step as exactly "call the tool, return the result". If a skill step is "call X, then filter/match/ reshape", the transformation half must be its own `pure_function` node or it silently vanishes at emission (live failure: match-by-name logic lost, workflow crashed dereferencing a declared-but-never-computed key). The rubric (`node-kinds.md`, external_call) mandates the split; `rote.probe.cross_check` flags violations as `output_mismatch` (node declares `output` keys the observed tool result can't provide) and `rote compile` prints them as WARNINGs. ### `temperature` is rejected outright by current Claude models A judge carrying `temperature` 400s on every call against a current Anthropic model (`invalid_request_error: \`temperature\` is not supported on this model`). The compiler used to write `temperature: 0.0` on judges as a determinism instinct, which made every compiled pipeline dead on arrival the moment `ROTE_MODEL_` pointed at a current model — found live, first real gateway run. Two halves of the fix, keep both: the rubric (`llm-judge-extraction.md`) says never set `temperature` — determinism comes from schema-locked decoding (forced tool call / `json_schema`), not the sampler — and when the IR *does* set one, emitted judges render it as a `ROTE_TEMPERATURE_` env knob whose empty value drops the parameter entirely (`_sampling_kwargs()` in Python, `...(TEMPERATURE.trim() ? … : {})` in TS). Don't "simplify" it back to a literal keyword argument: a baked constant cannot be escaped without a re-emit, which defeats the model-override knob next to it. ### First-run output: three contracts that live in one place each Found by a fresh-install run of the published wheel, where every one of these read fine to someone who already knew the answer: - **Install hints come from `rote.extras.install_hint`.** The PyPI distribution is `rote-cli`; `rote` is somebody else's package, so a hand-typed `pip install rote[api]` installs the wrong thing. `tests/test_first_run_output.py` fails on the bare form, and on a hand-typed `rote-cli[` outside `rote/extras.py`. - **A failed subprocess reports its tail** (`rote.eval.empirical.failure_excerpt`). The exception is at the end; a `[:800]` head slice showed a newcomer DBOS's startup banner cut off mid-word and nothing else. - **`--json` always puts one object on stdout.** `main()` wraps every `--json` handler; a nonzero exit with empty stdout gets `{"ok": false, "exit_code", "error"}` built from the `error:` line the handler printed. New error paths only need the usual `print(f"error: ...", file=sys.stderr)`. `rote run` also refuses stubs up front (`rote.runners.preflight_pipeline`, reading the emit manifest via `rote.adapters._common.unimplemented_stubs`), so a stub never burns a durable runtime's retry budget before failing. A stub is a function raising with `STUB_SENTINEL` (`[rote stub]`, in every stub rote emits) or, in Python, one whose whole body is `raise NotImplementedError` (the shape compile agents write), at any depth, methods included, `@abstractmethod` excepted. The marker counts only where it is executable: in Python as a `raise` argument (AST), in TS after `strip_ts_comments`. A raw-text match kept a filled-in module that merely mentions the marker in a comment listed, and `rote run` refused it. Never go back to a `NotImplementedError` substring match either: a real module that raises it on an unsupported branch was refused with no way past it. `--allow-stubs` is the escape hatch. ### Vendor SDK errors do not survive a durable step boundary `anthropic.APIError` / `openai.APIError` subclasses take keyword-only `response`/`body` arguments, so `BaseException.__reduce__`'s `(cls, args)` cannot rebuild them. A runtime that persists a failed step by pickling the exception — DBOS does — therefore replaces the real status and body with `TypeError: APIStatusError.__init__() missing 2 required keyword-only arguments` at the join. That masked the temperature 400 above completely. Emitted judges wrap the SDK call and re-raise through `_durable_error()`, a plain `RuntimeError` carrying type, HTTP status, model, node id, and the original message (regression: `tests/test_dbos_adapter.py::test_judge_translates_sdk_errors_so_they_survive_a_durable_step`, which asserts the translated error actually pickle-round-trips). This bites hardest on the fan_out enqueue path, where the failure is observed in a *different* process from where it was raised — any durability boundary is also an error-fidelity boundary. ### The TS tool runner's `done()` deadlocks; use `runUntilDone()` `client.beta.messages.toolRunner(...)` returns an async iterator plus two ways to wait on it, and they are not interchangeable. `done()` *observes* a loop somebody else drives — its own docs say it "works even if the async iterator hasn't yet started, and will wait for an instance to start" — so a single-consumer caller that never iterates hangs forever, silently, with no output and no error. `runUntilDone()` consumes the iterator itself. The Python twin's `until_done()` has no such split, which is exactly why this is easy to port wrong. Found by running it (rc=124, zero output); tsc and the SDK's types are both happy with the deadlocking version. Related, same file: report the turns that actually ran, not the cap. `AgentResult.iterations` counts assistant messages (`runner.params.messages.filter(m => m.role === "assistant").length`) and is `number | null` — the Workers AI lane returns null because `runWithTools` genuinely does not report it. Echoing back `maxIterations` made every loop look saturated. The Python `_run_agent_via_sdk` has no public `params` accessor, so it drives the runner's iterator itself and counts each yielded message, summing usage over every turn. Then `until_done()` on the drained runner returns the final turn without another request. ### An agent_loop's tools carry no server — `tool_servers` supplies it `Node.tools` are bare names (`zoominfo_search_contacts`): one loop may reach several servers, and an `MCPBinding` is a single server/tool pair. That gap was not cosmetic. `Pipeline.required_mcp_servers` derives from server *names*, so a pipeline whose only MCP usage is an agent loop reported **zero** required servers — `analyze`/`emit`/`compile` and the `rote mcp login` advisories all said there was nothing to authenticate. BDR is exactly that shape, so the flagship example demonstrated the bug. `Node.tool_servers` (tool name → server, agent_loop only, keys validated ⊆ `tools`) closes it. Three things now depend on it: - **The advisories are correct.** `required_mcp_servers` counts both declarations — a node's `mcp:` binding and a loop's `tool_servers`. BDR reports five servers instead of none. - **A resolved tool binds from its own server and no other.** Two servers exporting one tool name is precisely what an allowlist cannot disambiguate; first-wins discovery would pick whichever answered first. Python narrows its `--allowedTools` cross product the same way. - **Workers works at all.** There is no registry in workerd — `resolveUrl(env, server, …)` reads env bindings — so the server name *must* be known at emit time. Three sources feed the emitted loop, in descending authority: the IR's `tool_servers` (knowledge), other nodes' `mcp:` bindings (a guess — the loop probably reaches the servers the rest of the pipeline does), and at run time the local registry plus `ROTE_MCP_SERVERS` (the escape hatch). Only the first is trustworthy, which is why only it feeds the advisories. A partial map is valid and an unresolved tool still works via discovery; the emitted module names what it could not resolve rather than failing later with a bare "no server provides this tool". Two behaviors to preserve: - **Nothing guesses.** `rote.probe.enrich_pipeline` writes `tool_servers` from what the baseline actually *called* (`cross_check`'s `resolved_agent_tools`), and leaves a tool served by two servers in one trial unresolved — pinning either would silently send the loop to the wrong endpoint. An authored value always beats inference. - **Discovery is best-effort, binding is not.** A server that is down may not be one this loop needed, so it is recorded and skipped; a declared tool no reachable server provides is fatal. When an auth failure plausibly explains the gap, that failure is rethrown so the workflow parks and `rote mcp login` releases it, instead of dying with a misleading "unknown tool". ### `rote run` must see a park, and only the runner can release a local one A parked run used to look like a hang under `rote run`. The run-side watcher (`rote.mcp.parked`) reads each runtime's park signal: the DBOS `rote_auth_status` event (now `{awaiting, node}`), and on Inngest and Cloudflare a `rote-auth-park {json}` line the emitted park prints before its `waitForEvent`, because neither local dev server reports what a run waits on. Traps, all found by running it: - **The prompt goes to stderr.** `input()` writes its prompt to stdout, which is `rote run`'s result JSON. On a pty with stdout piped the user is asked a question they never see and the run waits forever. `_stderr_input` exists for this; don't "simplify" it back to `input`. - **Replays reprint the marker.** Both TS runtimes re-execute workflow code after a release, so the line prints again. Dedupe on its `wait` field (the wait's step id), never on server or node. - **`rote mcp login` cannot reach a `rote run` dev server.** The registry-wide release targets Inngest's default port and the Cloudflare REST API. So the watcher releases its own run when the server's token file or registry entry changes. On DBOS it waits a few seconds first, because login already released registered DBOS apps. - **A login writes the token file twice.** Client registration lands first with no token. Releasing on that write wakes the run into the same failure, it parks again, and the finished login then looks like "no change". `credentials_settled` waits for an access token. - **A Worker never sees the local token store.** On Cloudflare the watcher shows the export-to-`.dev.vars` fix and neither offers a login nor self-releases, since a login there would "succeed" and change nothing the Worker can read. - **An auth park looks like a gate park to the Cloudflare stability heuristic.** While the watcher reports an active park, the runner must not spend a gate payload on it. - Non-TTY never prompts and never opens a browser. The browser login is offered only when stdin and stderr are both terminals. ### A Cloudflare agent_loop is parkable, so it leaves its parallel wave Its tools are MCP tools, so `bindAgentTools` and every in-loop tool call raise the same auth signal a bound step does — an agent loop therefore gets the park-on-auth wrapper on Cloudflare (invariant #3's park-on-auth would otherwise not hold on the logged-in default runtime). Two specifics: - The release event is derived from the failure at run time (`authEventType(err, fallback)` parses the server out of the message) because one loop may reach several servers and `ROTE_MCP_SERVERS` can add more. Own properties do not reliably survive the `NonRetryableError` re-throw; the message does. - Parkable steps stay OUT of `Promise.all` (see the parallel-wave gotcha), so an agent loop no longer runs concurrently with its wave siblings. That is a real throughput cost on the slowest node kind, accepted because `waitForEvent` inside a promise combinator is undocumented and its timeout *throws*, which would reject every sibling. ### Emitted judges do not contain the vendor call A judge module holds its Pydantic models, prompt, schema and operator knobs; the call itself goes through `call_judge` in `signatures/_rote_inference.py` — the verbatim source of `rote.inference._runtime_helper` (never hand-edit the emitted copy; fix the module — byte-equality test in `tests/test_inference.py`). That helper resolves *who pays*: `claude-cli` (subscription, local only), `api` (the user's key), `rote-cloud` (tenant account), selected by `ROTE_INFERENCE_` > `ROTE_INFERENCE` > auto-detect (cheapest-to-the-user first). See [`docs/inference-runtime.md`](docs/inference-runtime.md). Traps if you touch this: - **The provider is not in the IR and must never be.** A `pipeline.yaml` is shared, committed, and sometimes third-party — a billing decision encoded there travels to people it doesn't belong to. `signature_spec.client` stays (wire format is behavior); the provider is a deployment fact. Consequence to preserve: the same emitted artifact runs on a subscription locally and a key in CI with no re-emit and no diff. - **`build_subscription_env()` lives in the helper**, not beside the Claude driver, and `drivers/claude.py` / `eval/*` import it back. Two copies is exactly how the `ANTHROPIC_API_KEY` scrub gets fixed in one place and silently drifts back to key billing in the other. - **The refusal is at `rote deploy`, not `rote emit`.** Emit cannot know where the directory is headed — that's the point of the bullet above. Don't "move it earlier". - **An explicit endpoint rules out `claude-cli` in auto-detect.** The proven AI Gateway config is `ROTE_BASE_URL_` + a bearer token; a laptop with `claude` on PATH would otherwise ignore it silently. - **The helper is stdlib-only at module level** (vendor SDKs import lazily inside functions) and imports nothing from rote. That constraint is what makes the verbatim copy safe. - **No custom exception classes in the helper.** A custom class only unpickles where its module is importable, and the observer of a failed durable step isn't always the process that raised it — the same failure mode as the vendor-SDK gotcha below. Plain `RuntimeError` carrying type / status / model / node / provider. ### `claude -p` gets schema-locked output, and costs 37k tokens if you let it `claude -p --json-schema --output-format json` is a **forced tool call** under the hood (`stop_reason: "tool_use"`) and returns the validated object in `structured_output` — the subscription lane has the same structural guarantee as the SDK lane, not "prose that parses". Prompt goes on **stdin**, not argv (judge prompts carry whole documents; argv is size-limited and visible in `ps`). The default invocation ships Claude Code's entire coding-agent system prompt: measured ~37k cache-creation tokens and 11.5s for a one-sentence judge. `--system-prompt ` + `--tools ""` + `--setting-sources ""` was supposed to fix that, and on a machine with claude.ai connectors it did not: Claude Code 2.1.302 still loaded the user's connector and plugin MCP schemas (`--tools` does not touch MCP tools) plus the auto-memory index of the repository the pipeline ran from. A real "say hi" judge through the helper measured 30.3k tokens, $0.183 cold; one greeted the developer by name from that memory. The connectors load asynchronously, so the same call ranged from 1.2k to 28k tokens run to run. The helper now passes one shared `_CLI_ISOLATION_FLAGS` tuple on BOTH paths. Each flag was measured by removing it from the full set: | Removed | Judge tokens (input + cache) | | --- | --- | | nothing | 1,250 | | `--strict-mcp-config` | 30,300 | | `--setting-sources ""` | 25,600 | | `--settings '{"autoMemoryEnabled": false}'` | 3,070 (from a repo with memory) | `--setting-sources ""` does not cover auto memory; only the setting does (or `CLAUDE_CODE_DISABLE_AUTO_MEMORY=1`, rejected because `build_subscription_env` is shared with the compiler). `--bare` would do all of it but refuses OAuth, which is the whole lane; `--safe-mode` measured the same 1.25k but is documented as a troubleshooting mode whose scope may change, so the explicit flags win. The agent loop takes the same tuple plus its own `--mcp-config`: strict mode means it sees only the servers rote resolved. It also now passes `--tools ""` when it has servers, because `--tools` removes only built-ins: before, the loop carried 31 coding tools (Bash, Write, Agent...) at ~31k tokens a turn. A real two-turn loop against a local fastmcp server went from 120k tokens / $0.38 to 2.8k / $0.012 with the tool still bound and called. Removing any flag silently restores its overhead. Also: the CLI reports API failures *inside a zero exit* (`is_error: true` in the envelope), so returncode alone calls a failed judge a success. The compiler's `ClaudeDriver` passes `--strict-mcp-config` too, with no `--mcp-config`: it wires no MCP (live compile-time MCP is the in-process drivers' job), and on this machine the user's connectors put 357 tools into the first compile request, past 200k tokens, which the subscription rejected outright ("Usage credits are required for long context requests"). Strict, that request is 70.7k. The compiler still inherits the user's settings, CLAUDE.md, auto memory, output style, permission mode and 25 unused built-in tools (11k tokens with all of those isolated too); that is a separate change because it alters what the compile agent is told, and needs a real compile eval to land. ### A prompt-cached run reports almost nothing in `input_tokens` `claude -p` always runs prompt-cached, so `input_tokens` holds only the few tokens that were neither written to nor read from the cache. A measured one-word reply reported `input_tokens: 3` beside `cache_creation_input_tokens: 122956`, and `total_cost_usd: 0.7378`. The compiler driver summed the plain pair only, with a docstring explaining that cache fields were dropped "to match the api driver". That reasoning is the bug: the api driver sends no `cache_control` at all, so two fields are complete for it and catastrophically incomplete here. Compiling the rote-compile skill against itself reported a 22-minute, 62-turn run as `tokens in=113 out=1919; cost $0.029`. Three things now hold, each with a regression test verified by breaking the code (`tests/test_claude_driver.py`, `tests/test_cli.py`, `tests/test_compiler.py`): - **All four buckets are counted.** `_usage_delta` returns `{input, output, cache_write, cache_read}`, and every display adds the cache buckets into the reported input rather than hiding them. Claude repeats input/cache usage across content blocks sharing one `message.id`; count each ID once. Assistant `output_tokens` is a placeholder: show output as pending until final `modelUsage` (whole query, including subagents) or `usage` (main loop) arrives. Recovery without final usage retains input/cache and marks output incomplete. - **The cost key is `total_cost_usd`.** There is no `cost_usd` key in the result envelope, so the old read returned `None` on every run while the estimate sat one key away. Prefer the CLI's whole-query estimate to rote's single-model estimate; neither is authoritative billing data. When changing this logic, consult the [SDK cost-tracking contract](https://code.claude.com/docs/en/agent-sdk/cost-tracking). - **Cache rates span the whole fetch.** models.dev publishes them per model and `_parse_models_dev` already read them, but `build_catalog` kept only the three tier representatives'. The compiler's default model is not one, so a default compilation had nothing to price its cache with. `PricingCatalog.rates_for` is the accessor that covers every model; `model_price_for` stays narrow. An older on-disk cache has no such key and degrades to "no published rate" rather than reading as free. - **An unpriceable run reports no cost, not a wrong one.** When a run touched the cache and the rates are genuinely unknown, `_cost_usd` returns `None`. Billing cache reads at the plain input rate would overstate them roughly tenfold. The general lesson, and it generalizes past this file: a usage number is only as complete as the fields you chose to read, and a field you never read fails silently and looks like a small number. ### Sonnet is the default, not Opus Both `ClaudeDriver` and `AnthropicApiDriver` default to `claude-sonnet-4-6` rather than Opus, because Opus is ~5× more expensive and Sonnet is fully capable of following the structured rubric. We learned this the hard way — the first two BDR runs with Opus exhausted a Claude Max/Pro "extra usage" budget in two attempts (~$3.50 each). Don't "fix" the default to Opus unless you have specific evidence that a skill needs it. ### `DEFAULT_MAX_TURNS = 60` BDR-scale skills need ~25 tool calls minimum, realistically 40-50 with exploration. The initial default of 30 caused an `error_max_turns` failure on the first real run. 60 leaves headroom. Don't reduce it without measuring. ### Cloudflare local-dev `status` stays `"running"` while parked on `waitForEvent` The Cloudflare e2e test (`tests/test_cloudflare_e2e.py`) needs to know when a workflow instance has parked on a `step.waitForEvent` so it can deliver the resume signal. The intuitive check — `status.status === "waiting"` — does **not** work in local dev. The top-level `status` field stays `"running"` the entire time; only the human-readable `wrangler workflows instances describe` output distinguishes "💤 Waiting for event". **The fix:** the test infers parking by step-output stability. Once `__LOCAL_DEV_STEP_OUTPUTS` reaches the expected count *and* stops growing for a few consecutive polls, the workflow is necessarily either at a HITL gate or done. See `_wait_until_parked_or_complete` in the e2e test for the canonical implementation. Production behavior may differ — this is a local-dev quirk; document it if anything changes upstream. Related: `__LOCAL_DEV_STEP_OUTPUTS` includes both `step.do` results *and* `waitForEvent` payloads in the same array. Don't rely on the length matching node count; compare node-name **sets** instead. ### Cloudflare `step.do` rejects `Record` returns `step.do`'s generic is `T extends Rpc.Serializable`, and `Record` does **not** satisfy it — `unknown` values aren't structurally serializable, so annotating an emitted stub with that return type breaks overload resolution at every `step.do` call site. Leaving the return inferred is no better: a throwing function declaration infers `Promise`, and `void as Record<...>` fails TS2352 at the data-flow reference sites. **The fix:** emitted stubs declare `Promise` (honest for an always-throwing stub; `never` satisfies any constraint and casts to anything). **The trap that fix sets:** because `never` casts to anything, a reference-site cast that is wrong for real types compiles against the stubs and fails only once the user types an implementation. All three TS runtimes shipped `"": _result as Record`, which is TS2352 the moment the node returns `Promise` (found by an eval on ops-report; `rote run` passed because casts are erased). A single `as Record` only compiles from a type that overlaps a Record: an anonymous object type does, an `interface` (no implicit index signature), an array, or a scalar does not. So exit results are returned uncast (`exit_return_ts_lines`) and field access is plain `_result["field"]` (`ref_to_ts_expr`), both in `_ts_common.py`. Indexing `never` is legal, so the stubs still compile. Don't "fix" the field access with `as unknown as Record<...>`: it compiles everywhere precisely because it turns off tsc's field-name check, so an IR field the user's type names differently becomes `undefined` at run time instead of a TS7053. Checking the stubs compile proves nothing here. `tests/test_ts_typed_impls_e2e.py` replaces them with typed implementations of every output shape and runs `tsc` on all three runtimes (positive half), then renames one referenced field and requires tsc to fail on exactly that (negative half). The negative half is not optional: on Inngest, `step.run` returns `Jsonify`, an anonymous object type that overlaps a Record, so a restored `as Record` field cast passes the positive half there and only the drift case catches it. Run the suite after touching any TS emitter's typing story. ### A parallel wave is not automatically parallel in the emitted code `_execution_waves` says which nodes MAY run concurrently; each adapter decides whether to take it, and the two mistakes here are easy to reintroduce because neither fails a test that didn't exist yet. Cloudflare computed waves and then emitted every `step.do` sequentially — the concurrency survived only as a `// ─── Wave N ───` comment, on the runtime that is also the logged-in default. It now emits `await Promise.all([...])`, which is Cloudflare's documented primitive (the ✅ example in "Rules of Workflows"; their own visualizer has a `ParallelNode` for it; no documented per-instance cap on concurrent steps). Three rules to preserve: - **Never wrap the wave in an outer `step.do`.** That advice is specific to `Promise.race`/`any`, where losing branches are discarded. Wrapping burns a step, subjects the whole wave to the 1 MiB step-result cap, and collapses per-node retry policies into one. - **`Promise.all`, not `allSettled`** — opposite of DBOS-TS, on purpose. Reaching the rejection means that node already exhausted its own retries, and every sibling's result is persisted under its own step name, so a workflow retry replays survivors from cache. (DBOS-TS uses `allSettled` for a different reason: a bare `Promise.all` can crash the Node process on unhandled rejections.) - **HITL gates and MCP-parkable steps stay sequential.** Both suspend on `waitForEvent`, whose behavior inside a promise combinator is undocumented — and whose timeout *throws*, which would reject the whole wave. Temporal had the mirror-image bug: `_emit_wave_call` emitted `retry_policy=`, but the `asyncio.gather` branch emitted only the timeout, so a node silently lost its declared retry budget the moment it gained a sibling in its wave — and parallel fetch waves are exactly where the flaky network calls live. If you add a parallel branch to any adapter, diff it against the single-node branch field by field. ### Inngest v4: no per-step retries, no 3-arg `createFunction`, and a lying dev-server endpoint Three traps hit while building the Inngest adapter, all verified empirically against `inngest` 4.11.0 + `inngest-cli` 1.34.0: 1. **`createFunction` is two-argument in v4.** Triggers live in the options object: `createFunction({ id, retries, triggers: [{ event }] }, handler)`. The classic `createFunction(opts, { event }, handler)` form from docs/blog posts **throws at runtime**. tsc alone won't catch a regression to the old form in emitted-string templates — the e2e's live run does. 2. **Per-step retry/timeout config does not exist.** `StepOptions` is `{ id, name?, parallelMode? }`. The IR's per-node `RetryPolicy` maps to a single function-level `retries` (max across nodes, clamped 0–20); per-node deltas, `backoff`, `retry_on`, and per-node `timeout` are emitted as comments. Don't "fix" the adapter to pass per-step retry options — there's nowhere to put them. 3. **`GET /v1/events/{id}/runs` lies while a run is parked.** During probing it reported `status: "Completed"` while the run was parked on `step.waitForEvent`. Use it only to discover the `run_id`, then poll `GET /v1/runs/{run_id}` (truthful `Running`/`Completed`). The run's *return value* is only available via the dev server's GraphQL API (`POST /v0/gql`, `run(runID:){output}` → `RunComplete` op JSON); the v1 REST `output` field comes back empty. Also: with `--no-poll` the dev server's single startup sync can race the app boot and never register it — the e2e sends an explicit `PUT` to the app's serve handler to force registration. See `tests/test_inngest_e2e.py`. ### DBOS 3 queues exist only after launch; emitted apps start through `launch()` DBOS 3 (Python) and DBOS Transact v5 (TS) removed direct queue construction: `Queue("name")` raises `DBOSException` at import, and `DBOS.register_queue` needs a launched runtime because queues now live in the system database. A workflow enqueued on a queue nobody registered does not fail; it sits in `ENQUEUED` forever. So the emitted `main.py` declares `QUEUE_NAME`, enqueues via `_enqueue` (a thin `DBOS.enqueue_workflow(QUEUE_NAME, ...)` wrapper), and exposes `launch()` = `DBOS.launch()` + `DBOS.register_queue(QUEUE_NAME)`. Any in-process launcher, tests included, must call `main_module.launch()`. A bare `DBOS.launch()` hangs the first parallel wave, which shows up only as a test timeout. Two more v5 traps on the TS side: the SDK no longer reads `dbos-config.yaml` at launch (only the `dbos` CLI does, so `rote._dbos.dbos_ts_system_database_url` ignores its `system_database_url` key), and starting or enqueuing a *step* directly now throws, which is one reason the DBOS-TS emitter keeps parallel waves on `Promise.allSettled` instead of queues. ### A node's constants reach its impl, and a tool only by name Several nodes can share one `impl` and differ only in `constants` (ops-report: three fetch nodes, one `fetch_file_content`, three `file_id`s). The Python runtimes passed no constants at all, so every fetch received `{}` and no implementation could tell the files apart. Three rules now, each learned the hard way: - **Merge in the step body, not the workflow's payload literal.** A `pure_function` / `external_call` step calls `impl(**{**payload, })` (`_py_common.impl_kwargs_expr`, used by dbos, python and temporal). The payload literal only reaches DAG callers. An agent loop calls its `loop_body` sub-nodes as tools with a payload the model built, so BDR's `enrich_contact_batch` lost `batch_size` the first time this was tried. Constants come last so the model cannot override a pinned value, and they never appear in the tool schema it sees. Judges are excluded: their input model rejects unknown keys. - **A constant reaches an MCP tool only when `mcp.args` names it.** Without `args`, the step sends its payload as the tool arguments, and a tool rejects arguments it does not declare (FastMCP: "Unexpected keyword argument"). `_common.mcp_tool_arguments` decides the arguments once for every runtime, Python and TS, so a tool gets identical arguments everywhere (`tests/test_constants_reach_every_caller.py`). - **The impl contract is checked at emit time.** An `extracted/` function that takes neither the constant nor `**kwargs` raises TypeError on its first call, possibly after earlier steps had side effects. The Python adapters run `require_constant_parameters` BEFORE writing anything, on the module text the runtime dir will hold (`EmitWriter.landed_content`: the agent's source, or a user edit the writer would preserve). They refuse the emit naming the node, the parameter and the full path of the file to fix, and `rote compile` turns that into exit 1 with the `rote emit` retry command. `check_contracts` reports the same break as `constant_not_accepted`. Most committed `examples/*/runs/` predate this and hard-code their constants, so they no longer emit on a Python runtime. On Temporal the check is skipped when `extracted_module` is not the emitted package (an explicit `TemporalAdapterConfig`, as the BDR snapshot uses): those modules still receive constants and would fail with TypeError at run time. - **Constants drive the logic.** An impl uses the value it is passed; a module-level copy is only the default for a direct call. A module that accepts a constant and then applies its own value turns the IR's constant into dead config (expense-audit did this until `test_editing_a_constant_in_the_ir_changes_the_rule`). Related, found by the loop test: the SDK tool runner does not serialize a dict tool result, it iterates it, so the model saw only the keys. `_runtime_helper._text_result` JSON-encodes a local tool's result, as the TS twin already did. ### A fan_out element's step name must carry its index Cloudflare and Inngest key a durable step by its **name**, so a fan_out node emits `` `[${_index}]` ``, not `""`. Reusing one name for every element is the nastiest failure shape available here: Cloudflare returns element 0's *cached* result for all N elements, so the run succeeds, the output has the right length, and every entry is identical — no error, anywhere. Inngest's memoization collapses the fan to a single execution the same way (its e2e asserts `count == FAN_OUT_ELEMENTS` for fan_out nodes and `== 1` for the rest, which is what catches it). DBOS is the exception on purpose: both its Python and TS SDKs identify a step by execution order, so repeated calls to one registered step are already distinct — no suffix needed, and adding one would be meaningless rather than harmful. Second, on Cloudflare only: a **parkable** fan_out node (MCP-bound) runs its elements *sequentially*, not in `Promise.all`, for the same reason a parkable step leaves its parallel wave — `waitForEvent` inside a promise combinator is undocumented and its timeout *throws*, which would reject every other element. ### A judge's `inputs:` must satisfy its input schema, and a fan passes one element Every adapter builds a judge's payload from its `inputs:` keys and nothing else (`{"thread": _item}` for a fan, plus shared params), then validates it against `signature_spec.input_schema`. So a `required` property `inputs:` does not bind fails *every* call. A bound key the schema does not declare is also rejected, open schema or not: Python silently drops it, and before the Zod emitter honored `additionalProperties` the TS runtimes rejected every call with it. A binding the contract cannot name is a mistake on every runtime. Nothing caught that before run time: the IR loaded, every adapter emitted, and the first real eval of deal-monitor failed all of its fanned `classify_thread_step` calls on a missing `account_name`. `Pipeline._validate_judge_input_bindings` now rejects it at load. Three things to preserve: - **The fanned compiler shape is the usual offender.** The compiler tends to flatten the element into top-level schema fields (`thread_id`, `messages`) while binding the list under one name (`threads`), or to declare a per-element value beside the element. Neither can work, because the grammar has no per-element reference; the per-element value must travel inside the element (the upstream step puts `account_name` on each `MatchedThread`). The rubric says so with the real wrong shapes; keep them there. - **It resolves the element with `rote.ir.fan_out_element_param`**, the same function the emitters call, which is why that function moved out of `rote.adapters._common`. A validator with its own resolution rule could pass a pipeline the emitters dispatch differently. - **Two exemptions, both principled.** Loop-body sub-nodes are called by their parent loop with model-chosen arguments, so their `inputs:` describe rather than bind. A legacy `signature:`-only judge has no schema in the IR to check. Don't widen the check to `Node.input_schema` on other kinds: that is observed documentation, not a contract any adapter enforces. Unknown-key handling is behavioral emitted code, so it must match across languages: Zod gets `.strict()` exactly where Pydantic gets `extra="forbid"` (`additionalProperties: false`), strips otherwise, and a property-less object is a free-form record on both. `tests/test_judge_schema_parity_e2e.py` runs one schema through the emitted Pydantic model and real `zod` and compares verdicts and parsed values; touch either converter and run it. `tests/test_ir.py::test_every_committed_pipeline_validates` sweeps every `pipeline.yaml` under `examples/`, so a snapshot that would fail its first judge call cannot be committed. ### An undefined fanned list must name the node, on every runtime Emitted code routes the fanned list through a guard — `fanOutList` in TS, `_fan_out_list` in Python — that throws naming the node id and the IR reference. Without it the failure is `Cannot read properties of undefined (reading 'map')` (TS) or `'NoneType' object is not iterable` (Python): no node, no reference, inside generated code the user never wrote. Found by running it — the first live fan_out run on Cloudflare died exactly that way, and the cause is almost always an upstream node that didn't return the key the IR says it does. Both guards are emitted only when the pipeline has a fan_out node, and they are *behavioral* emitted code like `_serialize`: if they drift, the same broken pipeline reports differently per target. Diagnostics are part of the emitted contract — a guard on one runtime and not another is its own parity gap. Related trap in the **tests**: a live e2e's mocks must return the fanned key as a non-empty list. Three TS e2e suites had `exclusion_check_sequence` returning `passed: []`, which made BDR's fan a correct no-op — the node never ran and every assertion still passed. `tests/_helpers.py`'s `fan_out_source_keys` now derives those mock values from the IR using the adapters' own resolution, so a mock cannot disagree with the emitted code about which input is the list. ### Eval pricing: no hardcoded prices, and two traps found empirically `rote.eval.pricing` fetches current models + official prices at eval time (models.dev primary, OpenRouter cross-check; 24h cache in `~/.cache/rote/pricing.json`; loud `PricingError` when offline — never a baked-in table). Two things you will get wrong if you touch it: 1. **"Most expensive = flagship" is false.** Previous-generation flagships linger in catalogs at higher prices than the current one (Opus 4.1 at $15/Mtok vs Fable 5 at $10/Mtok). Flagship detection restricts to the provider's *current generation* — models released within `_CURRENT_GENERATION_WINDOW` of the provider's own newest release date. The anchor is the data's newest release, never the wall clock, so tier detection is a pure function of the payload and testable with fixtures. Regression: `tests/test_eval_pricing.py::test_tier_detection_matches_current_lineup`. 2. **Community catalogs lag launches.** On Sonnet 5's launch week, LiteLLM's price file still carried the 4.6-generation price while models.dev and OpenRouter agreed on the new one — that's why LiteLLM was dropped and why the cross-check annotates the scorecard's `source` string with a WARNING on >2% disagreement instead of trusting either side silently. 3. **Cloudflare Workflows infrastructure has no price API.** Steps and storage started billing on 2026-08-10. `rote.eval.infra_pricing` fetches the one table both Cloudflare pricing pages render, the raw `src/content/partials/workflows/workflows-pricing.mdx` from `cloudflare/cloudflare-docs@production`. It caches the raw file for 24h in `~/.cache/rote/` and re-parses it on every load. The parser is strict, and both halves of that matter. A missing row or a missing `Workers Paid` column raises `PricingError`, because a skipped row prices that unit at nothing. Each cell must also match a closed grammar anchored at both ends. A prefix match read "per additional 100K steps" as a per of 100, overstating the price 1000x. The wording has already changed once (2026-09-09). Steps per run (`cloudflare_steps_per_run`) are a Range: one per node, the fan width per fan_out node, one per HITL gate, none for `loop_body` sub-nodes. The fan width is the token estimate's own `fan_out_widths` (eval.yaml iterations, else the prior), so steps and LLM dollars never assume different element counts. Counting a gate's wait as a step is labeled a conservative assumption, since the docs don't say. Retention defaults to 30 days: the emitted wrangler.jsonc sets no `default_retention`, so the account maximum applies (limits.mdx footnote 2). The 2026-09-10 changelog's seven-day default for new Workflows contradicts that page, and this takes the larger figure. CPU and state size are stated priors. `rote eval --runtime cloudflare|rote-cloud` adds the section, as does a `compile --out` dir holding only `runtime/cloudflare`, or a compile that emits for cloudflare. Regressions in `tests/test_eval_infra_pricing.py`: `test_a_table_missing_any_row_is_rejected`, `test_an_unknown_unit_phrase_is_rejected_not_misread`. Related invariant: the eval sidecar (`eval.yaml`, per-step agent-turn estimates written by the compiler) is deliberately NOT part of the IR — it describes the *source skill's* behavior as an agent, not the pipeline. Don't move it into `pipeline.yaml`; invariant #1 (the IR is runtime-agnostic and behavior-only) applies. ### ClaudeDriver recovers `pipeline.yaml` if subprocess errors after writing The agent's deliverable is a file on disk, not a clean exit code. A real-world failure mode hit during this project: the agent wrote a complete `pipeline.yaml` at turn N, then ran a self-validation turn at N+1 that hit `ECONNRESET` — the subprocess returned nonzero, the orchestrator's `TemporaryDirectory` cleanup ran, ~$2.50 of Sonnet 4.6 work disappeared. The driver now checks for `pipeline.yaml` **before** failing on returncode. If the file exists, the run succeeded materially; the subprocess error is surfaced in `metadata["subprocess_warning"]` rather than treated as fatal. Don't undo this — losing $2 runs to a network blip is unacceptable. See `tests/test_claude_driver.py::test_nonzero_exit_with_pipeline_yaml_recovers_run`. --- ## Canonical examples to imitate When adding new code, the fastest way to stay consistent is to copy the shape of an existing canonical example. | What you want to do | Imitate | | --- | --- | | Add a new node kind to the IR | `rote.ir.Node` — add an enum variant, a kind-specific required field, and a branch in `_validate_kind_specific_fields` | | Add a Python-emitting runtime adapter | `src/rote/adapters/temporal.py` (~570 lines) | | Add a TS-emitting runtime adapter | `src/rote/adapters/cloudflare.py` (~750 lines) — includes the JSON-Schema-to-Zod converter and signal-name validator | | Add a subprocess-based driver | `src/rote/compiler/drivers/claude.py` + `tests/test_claude_driver.py` | | Add an in-process driver | `src/rote/compiler/drivers/anthropic_api.py` + `tests/test_anthropic_driver.py` | | Add a new CLI subcommand | `_cmd_emit` in `src/rote/cli.py` (simple) or `_cmd_compile` (which calls into the Compiler orchestrator) | | Test a Python-emitting adapter end-to-end | `tests/test_temporal_e2e.py` (uses Temporal's `WorkflowEnvironment.start_time_skipping`) | | Test a TS-emitting adapter end-to-end | `tests/test_cloudflare_e2e.py` (real `npm install` + `tsc --noEmit`; `@pytest.mark.slow`) | | Test the CLI via subprocess | `tests/test_cli.py::test_subprocess_emit_bdr` | | Add a rubric file | Any file in `skills/rote-compile/references/` — keep them 200-300 lines, concrete, with BDR examples for every point | --- ## Invariants that must hold These are the things that, if violated, break the project's design. If a change of yours would violate any of them, stop and reconsider. 1. **The IR is runtime-agnostic.** No Temporal-specific or Inngest-specific concepts in `rote.ir`. If a field is meaningful only to one runtime, it belongs in that adapter's config, not the IR. 2. **Drivers contract on the filesystem only.** `work_dir/pipeline.yaml` is the deliverable. No stdout parsing, no side channels. 3. **Emitted code touches MCP only where the IR declares it.** There are exactly two declarations, and no third: a node's `mcp:` binding, and an `agent_loop`'s `tools:` (which the IR documents as MCP tool names — a loop that declares any pulls in the MCP helper even when no node in the pipeline carries a binding). Everything else never references MCP in any adapter or language, enforced by AST/text tests that **name** the legitimate sites rather than skipping files that happen to mention MCP — so an unexpected reference still fails, and an expected site that stops referencing MCP fails too (`tests/_helpers.py::assert_no_mcp_in_ts`'s `expected=`). Both declarations emit a working, *never-interactive* client call — auth problems raise/park, they never open a browser from workflow code. 4. **Mandatory nodes cannot become conditional.** The IR validator rejects `mandatory: true` on `agent_loop` nodes, and the Temporal adapter emits mandatory nodes as unconditional activities. Adding a conditional-skip mechanism would destroy the "MANDATORY prose check becomes impossible-to-skip code" value prop. 5. **The repo is PH-free and secret-free.** `scripts/sanity-check.sh` is the source of truth. Run it before any push. If you add a new example skill or a new test fixture, run the script afterward. 6. **Two adapters before declaring the IR generic.** *(Satisfied as of v0.2.0.)* The IR was stress-tested by the Cloudflare adapter, which emits TypeScript with a single-class programming model (vs. Temporal's workflow + activities split). Two pressure points surfaced: - The retry `backoff` enum already maps cleanly to Cloudflare's categorical enum — no IR change needed. - HITL signal names need a `[A-Za-z0-9_-]+` constraint for Cloudflare's `waitForEvent`, but this is enforced at adapter emission time, not in the IR. - LLM signatures needed a structured cross-language form — solved by adding `signature_spec` (JSON Schema + prompt) alongside the legacy Python-path `signature` field. The Inngest adapter added a third pressure point without requiring an IR change: Inngest v4 has **no per-step retry/timeout config**, so the per-node `RetryPolicy` degrades to a function-level budget (max across nodes) with per-node deltas emitted as comments. That's a lossy mapping documented at the adapter layer — acceptable because the IR stays the richer form; a runtime that can't express a policy documents the gap rather than forcing the IR down to the lowest common denominator. Future adapters (Restate, etc.) shouldn't require IR changes for their core programming model — but if one does, that's a real signal to revisit the IR shape rather than papering over it in the adapter. 7. **IR string fields that reach emitted code are charset-constrained at the IR boundary, not in the adapters.** A `pipeline.yaml` is untrusted input (it's LLM-generated from a possibly third-party `SKILL.md`, or shared directly), yet every adapter splices certain fields *verbatim* into emitted source and filenames. `Node.id` (emitted as `async def {id}` / `@activity.defn(name=…)` and as `signatures/{id}.py`), `signal` (a Temporal signal-handler method / DBOS topic), and the two halves of `impl`/`signature` (`from … import {symbol}` + call site, and the module path) are all validated to a safe identifier / relative-path shape by `field_validator`s in `rote.ir` — see `_IDENTIFIER_RE`, `_validate_impl_ref`, and the `Node._validate_id/_validate_signal/ _validate_impl_signature` validators. Prose fields (`description`) can't be charset-restricted, so they're escaped at emission via `rote.adapters._common.safe_docstring_line` (Python docstrings) and `safe_block_comment_line` (TS JSDoc). Emitted-file writes also pass through `resolve_within` as defense-in-depth. **Do not add a new IR string field that an adapter emits verbatim without giving it an equivalent validator**, and don't move these checks down into a single adapter — the IR boundary is where all three (and every future) adapter inherit them. Regression coverage: `tests/test_ir.py::test_node_id_must_be_a_safe_identifier`, `::test_impl_symbol_injection_is_rejected`, and `tests/test_temporal_adapter.py::test_malicious_description_cannot_break_out_of_docstring`. --- ## Tests vs. evals: the dividing line is inference **Anything that does not require inference is a test.** Automated, run on every push, and kept fast. **Anything that costs tokens is an eval.** Run deliberately and periodically — `rote eval --run`, the compiler against a real skill, a live gateway judge. Never wired into CI. This is not a style preference. A token-spending check in an automated suite bills someone on every push, gets slower as it grows, and fails for reasons that have nothing to do with the change under test. Both halves of the split are enforced, not just documented: - `tests/conftest.py::_no_inference_in_tests` scrubs every vendor credential from the environment and makes a bare-name `shutil.which` lookup of `claude` / `codex` return None. The second half is the one that matters: `claude -p` authenticates from an OAuth session, so having no API key is *not* protection — not finding the binary is. Two DBOS e2e suites burned real subscription inference on every run until it was caught by wall-clock timing. - `tests/test_no_inference_in_tests.py` verifies the guard itself, because its failure mode is silent spending that no other assertion would notice. - Only *bare-name* lookups are blocked, so a test that writes a fake `claude` stub and passes its absolute path still works — a guard that forced tests to route around it would soon be disabled. Consequences worth keeping in mind: - **An e2e test is still a test.** The TypeScript suites run real `npm install`, real `wrangler dev`, a real Inngest dev server and a real Postgres — no inference, so they are automated (see the `TypeScript e2e` / `DBOS-TS e2e` CI jobs). `@pytest.mark.slow` says what *toolchain* a suite needs, never whether it should run. - **Emitted judges and agent loops get stubbed, not called.** The live suites point `ROTE_BASE_URL_` at a local Messages stub and set a fake key. That is what makes it possible to prove the agent-loop machinery end to end without paying per run. - **Speed is a feature of the test half only.** The fast suite is ~18s; keep it there. Evals are allowed to be slow and expensive because nothing blocks on them. --- ## Testing discipline: make assertions capable of failing Five real defects in this repo shipped past a green suite because a test observed something *adjacent* to the behavior. This is the single most common failure mode here, so it gets its own section. | Defect | What the test checked | What it should have checked | | --- | --- | --- | | bare `--tools` killed the whole subscription lane | `"--tools" in command` | the flag's **value** | | DBOS e2e mocked agent loops via an unreachable branch | that the overlay file was written | that the mock was actually *called* | | BDR's fan was a no-op in 4 live e2e suites | that the workflow completed | that the fanned node ran **N** times | | per-node `timeout:` ignored (temporal, cloudflare gates) | that *a* timeout was emitted | that the **declared** one was, and the default was not | | `1h` emitted as `"1 hours"` | nothing — no per-node timeout test existed | — | Three habits that catch these: - **Assert the value, not the presence.** A flag, key, or node id appearing in the output says nothing about what it was set to. - **Give a negative half.** Pair "the declared value is present" with "the default is absent" — otherwise emitting both still passes. - **Make fixtures non-degenerate.** An empty list makes a fan, a loop, or a retry a *correct* no-op that no assertion can catch. Fixture collections that drive iteration count carry >1 element (`tests/_helpers.py::FAN_OUT_ELEMENTS`), and their values are derived from the IR (`fan_out_source_keys`) so a mock cannot disagree with the emitted code about which input is which. **Verify a new test by breaking the code.** Not by inspection — apply the regression the test is meant to catch and confirm it fails. Any test written for a bug fix should be run against the *unfixed* code once. When the fix spans runtimes, check the split: the fan_out parity suite fails on the five broken runtimes and passes on `dbos`, which already worked, proving it recognizes correct behavior rather than just matching new strings. A periodic mutation sweep is the systematic version, and it works well here because emission is pure template substitution — mutate a module, run `pytest tests/`, and any mutation that survives is an untested behavior. Three sweeps so far, 64 mutations, 48 caught on the first pass: - **Adapters** (18): retry policy, timeouts, step naming, payload threading, MCP allowlist narrowing, the `ANTHROPIC_API_KEY` scrub, invariants #4 and #7. Gaps found: per-node `timeout:` was ignorable on both temporal and cloudflare gates with nothing failing. - **Drivers / inference / MCP / probe** (17): the flags that keep `claude -p` cheap, `is_error` inside a zero exit, fastmcp's `result.data`, probe's never-guess rule, and the cost-critical driver defaults. - **`rote.eval` / `rote.config` / `rote.cli`** (29): dollar arithmetic, tier detection, the read-only MCP gate, the four-layer config precedence, and the login-aware compile default. Config precedence and pricing tier detection were already airtight — every mutation died. Six gaps found, all in the two places where a wrong number or a wrong exit code looks like success (see below). Two lessons from the second sweep worth keeping: - **The agent-loop CLI path is the weak twin.** `_runtime_helper` carries two copies of the `claude -p` flag block (judge, agent loop), and mutations died on the judge copy while surviving on the loop copy — `--setting-sources` and `is_error` were covered on one and not the other, and `--system-prompt` on neither. That asymmetry is exactly how the bare `--tools` bug shipped. When you touch one copy, test both. - **A symbolic assertion cannot pin a constant.** `args[args.index("--model") + 1] == DEFAULT_MODEL` verifies the flag is wired up and says nothing about the constant, because both sides move together. `DEFAULT_MODEL` could be flipped to Opus (~5x cost) and `DEFAULT_MAX_TURNS` halved (the original `error_max_turns` failure) with the suite fully green. `tests/test_cost_defaults.py` pins the *property* — not-Opus, `>= 60` — so a legitimate model bump passes and the expensive regression fails. Three more from the third sweep: - **A self-comparison is the same tautology in disguise.** `pipeline_cost_usd` had `cost.high == pytest.approx(expected_high)` recomputed from the fixture rates and killed every mutation. `agent_run_cost_usd` was only ever compared against *itself* (`with_cache.mid < without_cache.mid`) — an ordering that holds under any transformation applied to both sides, so the divisor could become 1e3 instead of 1e6, or the cache-write premium could vanish, with the suite green. Cost numbers are what a user makes a spending decision on; assert the dollars. - **Asserting the loop bound leaves the per-call figures unpinned.** The `agent_loop` estimate asserted `calls.high` and nothing else, so both the token and the wall-time lines could drop `agent_loop_turns_per_iteration` (a 3x understatement) undetected — in the turn-dominated regime the loop-aware model exists to capture. - **Strictness is a property of a layer, not of one command.** Four commands call `load_layers()`; only `rote config` asserted exit 2, so compile's handler could `return 0` — reporting success while doing nothing. The replacement test parametrizes over all four, and the next command to read config fails there instead of shipping a silent fallback. Watch for **equivalent mutants** — the fan_out tier ordering in `fan_out_element_param`, and `args.no_deploy` in `_cmd_compile`'s cloud-vs-local branch, both survive because they are provably unobservable, not because they are untested. Both are commented in place; confirm the claim empirically before writing one off, since "probably equivalent" is exactly what a real gap looks like. --- ### The suite never writes into examples/ The committed runtime snapshots under `examples/bdr-outreach/expected/runtimes//` are compared, not regenerated: adapter suites emit into `tmp_path`, and `tests/test_committed_snapshots.py` checks that a fresh emission (with the same `extracted/` sources `rote emit` reads) reproduces each one byte for byte. The suites used to emit straight into those dirs, so every pytest run rewrote committed files, and without the example's sources it replaced the real implementations with stubs. That dirtied the merge-queue worktree and blocked merges. A session hook in `tests/conftest.py` now fails the run if anything under `examples/` changed (`__pycache__` and `node_modules` excepted). When an adapter change is intended, regenerate with the `rote emit` command the failing snapshot test prints, and commit the result. ## Workflow expectations ### Running tests ```sh .venv/bin/pytest tests/ # full suite, ~1s .venv/bin/pytest tests/test_ir.py # fast iteration on IR changes ``` The suite is intentionally fast (no real API calls, no real subprocess spawns, no real LLM invocations). Every slow thing is mocked. Keep it that way — if a new test needs to be slow, gate it behind a `@pytest.mark.slow` decorator and document why. ### Running the real compiler ```sh .venv/bin/rote compile examples/bdr-outreach/skill \ --runtime temporal \ --out /tmp/bdr-compiled ``` ~13 minutes wall clock on BDR. Expect 30-40 turns with Sonnet 4.6. The output landing at `/tmp/bdr-compiled` (or wherever you point `--out`) is what you'd commit as a new regression snapshot if the rubric or IR changed in a way that requires re-compiling. ### Sanity-checking before commit ```sh ./scripts/sanity-check.sh ``` Must exit 0. If it exits non-zero, either the scan found real leaks (fix them) or the pattern needs adjustment (be careful — err on the side of false positives over false negatives). ### Before a PR - [ ] `pytest tests/` — all tests pass - [ ] `./scripts/sanity-check.sh` — clean - [ ] `ruff check . && ruff format .` - [ ] `mypy src/rote` — strict, no ignores - [ ] If the rubric or IR changed materially: re-run the compiler on BDR and diff the snapshot - [ ] Commit message explains the *why*, not just the *what* --- ## What's stubbed vs. working Don't waste time debugging stubs. These are intentional. **Intentionally stubbed:** - The BDR example's `extracted/*.py` modules raise `NotImplementedError` — users fill them in with real API client code; the compiler produces scaffolding, not production code **Working end-to-end:** - `fan_out` on ALL SIX runtimes: a node marked `fan_out: true` is invoked once per element of its bound list, and the result binds as a list in input order. Which input is the list is resolved by `rote.ir.fan_out_element_param` (fan_out edge marker > incoming-edge source > only node-bound param; ambiguity is an emit-time error, never a guess). It lives in the IR module on purpose: two adapters disagreeing about which input is the list would make one pipeline mean two different things, and the judge-binding validator must agree with them too. Per runtime: DBOS enqueues one durable step per element; python uses `pool.map`; temporal `asyncio.gather` over one activity execution each; Cloudflare and Inngest `Promise.all` over per-element steps; DBOS-TS `Promise.allSettled` + `unwrap`. See the fan_out gotchas below for the two traps. - `agent_loop` on ALL runtimes. Python got the real loop first; the three TypeScript runtimes now emit one too — MCP tools bound through whichever MCP helper that runtime already emits, `loop_body` sub-nodes bound as callables, and the whole thing handed to `runAgentLoop` in the verbatim `signatures/_roteInference.ts` (`rote.adapters._ts_common.ROTE_INFERENCE_HELPER_TS`). Proven live in `tests/test_ts_agent_loop_e2e.py`: real fastmcp server, real emitted module compiled by the emitted toolchain, real Anthropic SDK tool runner against a local Messages endpoint. - IR load + validate - `rote emit` (IR → runtime code; DBOS default), with hash-guarded re-emission: every adapter writes through `rote.adapters._common.EmitWriter`, which records what rote wrote in `.rote-manifest.json` and, on re-emit, preserves any file the user edited (fresh content lands in a `.new` sibling and the CLI reports it). Never write emitted files with bare `write_text` in an adapter — route through the writer. Emission also sources the agent's REAL `extracted/.py` implementations found beside the pipeline.yaml verbatim into the runtime dir (dbos, python and temporal adapters, `rote.adapters._py_common.resolve_extracted_source`); IR-derived stubs are only the fallback. Temporal writes its package through the shared `write_extracted_package` (python uses it too) and its activities call plain functions in a worker thread and await coroutine functions, since agent-written modules are plain. Only the BDR snapshot points temporal at a user-maintained package, through an explicit `TemporalAdapterConfig(extracted_module="expected.extracted")`. Known gap: TS runtimes can't consume Python modules at all (the compiler would have to emit TS implementations). - `rote compile` (SKILL.md → IR → Temporal code), stamping a `provenance.json` sidecar (section hashes; see `rote.skill_source`). Every run streams NDJSON progress to `/progress.jsonl` (`--progress-file` just moves it) and prints the `tail -f` watch command as its first stderr line — agents driving a run relay that line to the human (AGENTS.md). - `rote compile --update` — incremental re-compilation: diffs the skill against the stamped provenance, re-derives only nodes whose source sections changed, enforces that unchanged nodes' ids survive, and merges file-level so user-filled stubs are kept. No section changes → no agent run at all. - Per-node inference overrides in emitted judges: `ROTE_MODEL_` / `ROTE_BASE_URL_` (plus `ROTE_TEMPERATURE_` when the IR sets one) env vars beat the IR defaults on all four runtimes, and `signature_spec.base_url` reaches any OpenAI-compatible endpoint. Node.source (provenance) is excluded from the pipeline hash on purpose — metadata must not re-version in-flight workflows. **Proven config — judges through a Cloudflare AI Gateway** (no Anthropic key at all; Unified Billing pays provider rates from CF credits and logs every call's tokens + cost): `ROTE_BASE_URL_=https://api.cloudflare.com/client/v4/accounts//ai`, `ROTE_MODEL_=anthropic/claude-sonnet-5`, and the CF API token in `ANTHROPIC_AUTH_TOKEN` — the SDK sends it as `Authorization: Bearer`, which is what that endpoint wants; passing it as `ANTHROPIC_API_KEY` (→ `x-api-key`) gets a 401, contradicting Cloudflare's own JS example. Two more traps: only *current-generation* model ids resolve (`anthropic/claude-sonnet-4-5` from the CF docs 404s), and `@cf/*` Workers AI models are rejected by the `/v1/messages` route entirely — they only work on `/v1/chat/completions`. Routing to a *named* gateway needs a `cf-aig-gateway-id` header, which emitted clients can't inject; the account's default gateway needs no header. - All three compiler drivers: `ClaudeDriver` (`claude -p`), `CodexDriver` (`codex exec` — workspace-write sandbox, no env scrubbing since Codex login is only overridden by `CODEX_API_KEY`, smoke-tested against the real CLI), `AnthropicApiDriver` - `rote analyze` — the `plan` to `compile`'s `apply`: runs the compiler, then prints a structural report (node-kind breakdown, roteness matching `rote eval`, HITL gates, targetable runtimes) instead of emitting runtime code. `--json` for machines; `--out` keeps the IR for a later `rote emit`. - MCP requirements manifest: `Pipeline.required_mcp_servers` (derived, never stored — same rationale as `requires_durable_execution`) feeds an `mcp_servers` entry — `{server, nodes, tools, auth}` — in `analyze`/`emit`/`compile` output (human, `--json`, and the progress-file summary line), joined against the local registry + token store. Deliberately **advisory-only**: rote never hard-gates on auth (park-on-auth is the backstop); it prints `rote mcp login` recommendations for servers that would park a run. - Data-flow threading: nodes declare `inputs:` (param → source reference, grammar in `rote.ir.parse_input_ref`) and all three adapters — Temporal, Cloudflare, and DBOS — thread real payloads through the DAG, with HITL gate resume payloads participating as the gate's result — validated empirically in the runtime e2e tests - Park-on-auth (ALL MCP-capable runtimes — DBOS Python, DBOS-TS, Inngest, Cloudflare): MCP-backed steps with a missing/dead credential suspend the run durably; `rote mcp login ` (or `rote mcp release `) releases every parked run across the apps recorded in `~/.local/share/rote/apps.json` (written at emit/compile time). Release channels differ per runtime: DBOS = discovery + per-workflow send (portable serialization); Inngest = one broadcast event per pipeline (events fan out; no discovery needed, but NO buffering for unstarted waits — retry-once covers the race); Cloudflare = REST blast to every non-terminal instance (no broadcast, but events buffer per-instance, so blasting is race-free; needs CLOUDFLARE_API_TOKEN/ACCOUNT_ID, local dev uses `wrangler … send-event --local`). Retry opt-outs also differ: should_retry (DBOS-py) / shouldRetry (DBOS-TS) / NonRetriableError (Inngest) / NonRetryableError (Cloudflare — and its waitForEvent THROWS on a 24h-default timeout, hence the explicit "30 days"). Proven live per runtime: `tests/test_mcp_park_e2e.py`, `test_mcp_park_ts_e2e.py`, `test_mcp_park_inngest_e2e.py`, `test_mcp_park_cf_e2e.py` (the last also runs the TS MCP SDK live inside workerd). - `rote run` — one-off local execution of either side, built as a thin detection/dispatch layer (`rote.runners`) over the proven trial primitives: a skill dir runs via `rote.eval.baseline.run_baseline_trial` (registry MCP injection + read-only gate), an emitted dir via `rote.eval.empirical.run_pipeline_trial` (python in-process; dbos with cross-process `DBOSClient` gate delivery — payloads are collected up front, safe because DBOS notifications persist per-topic; cloudflare under `wrangler dev --local` via `rote.runners.cloudflare`, driving the emitted `src/index.ts` router with parked-then-send gate delivery in topological gate order — parking is inferred from `__LOCAL_DEV_STEP_OUTPUTS` stability because local-dev status lies, see the Cloudflare gotcha; inngest via `rote.runners.inngest` — emitted serve entrypoint + managed `inngest-cli dev`, forced PUT registration, and gate events RE-BROADCAST every poll tick because Inngest drops events with no active waitForEvent, with output read from the dev server's GraphQL RunComplete op; dbos-ts via `rote.runners.dbos_ts` — the emitted main.ts shares dbos-py's one-shot CLI contract, signals go Python→TS over the portable serialization channel, and an unreachable Postgres falls back to a throwaway Docker container; NOTE the TS SDK prints its startup banner to STDOUT ahead of the result JSON, hence the trailing-JSON extraction; temporal via `rote.runners.temporal` — no subprocess dance at all: temporalio's WorkflowEnvironment.start_local() manages a real dev server, the worker runs in-process on the emitted workflow.py/activities.py with UnsandboxedWorkflowRunner, and signals send up front because Temporal buffers them server-side). ALL SIX emitted runtimes are now locally runnable. Detection understands both the `compile --out` layout (`runtime//`) and bare emitted dirs (marker files — see `rote.runners._detect_runtime`). - `rote deploy` — push-deploy wrappers (`rote.deploy`): cloudflare via `npx wrangler deploy` (whoami-preflight surfaces the session account first — ambient wrangler state on this machine is the WORK account), dbos/dbos-ts via `npx dbos-cloud app deploy` (verified current: no `app register` step anymore; CI auth = `login --with-refresh-token`). Temporal/Inngest/python have NO push model — the command prints current hosting guidance with doc URLs instead of pretending (Temporal Cloud hosts only the server; Inngest syncs via authenticated POST to api.inngest.com/v2/apps//syncs, the old unauthenticated PUT-to-your-endpoint flow is deprecated). The `rote-cloud` target (`rote.deploy_rote_cloud`) bundles the emitted `src/workflow.ts` with `npx esbuild` (esm/neutral, worker-first conditions, `cloudflare:workers` + `node:*` external — must match the platform's reference bundler) and POSTs the manifest-derived DeployPayload to `/v1/pipelines` with a tenant bearer token; the emitted cloudflare `manifest.json` is REQUIRED (no TS-regex fallback). Proven live against the platform running under `vite dev`. - `rote login` / `logout` / `whoami` (`rote.cloud_auth`) — OAuth 2.0 device flow (RFC 8628) against the platform's better-auth `deviceAuthorization` plugin. BOTH UXes ride the one grant path: browser available → auto-open `verification_uri_complete` (code pre-filled, one Approve click); `--device`/headless → printed code + URL. There is deliberately NO localhost-callback listener — no second server-side mechanism to secure. The polled session token is traded at `POST /v1/cli/keys` for a durable tenant `rote_…` API key; the CLI never persists a session token. Credential at `~/.local/share/rote/cloud.json` (0600, atomic; `ROTE_CLOUD_CRED_PATH` test override — conftest sets it suite-wide). Resolution everywhere: flag > `ROTE_CLOUD_URL`/`ROTE_CLOUD_TOKEN` > stored login. `logout` revokes server-side (`DELETE /v1/cli/keys`, the key revokes itself) but ALWAYS clears the local store; `whoami` verifies live via `GET /v1/me`. - Layered defaults (`rote.config`) + `rote init` + `rote config`: `runtime`/`deploy`/`agent`/`model` resolve flag > `ROTE_*` env > project `rote.yaml` (walk-up discovery, stops at `.git`; `ROTE_PROJECT_CONFIG_PATH` overrides) > user `~/.config/rote/config.yaml` (`ROTE_CONFIG_PATH` overrides — conftest isolates both suite-wide) > the command's built-in. Config files are strict: unknown key or invalid value = loud exit 2, never a silent fallback. `rote init` is the interactive wizard that writes them (the ONLY interactive command besides login; exits 2 without a TTY; I/O injected for tests — and its `input_fn` default resolves at call time, not def time, or monkeypatched `builtins.input` never lands). `rote config` prints each key's value + winning source. Compile deploy semantics: config `deploy: rote-cloud` + logged out fails fast BEFORE the agent spends money; an explicit `--runtime` flag downgrades a configured cloud deploy with a warning instead of blocking; contradictory config (non-cloudflare runtime + rote-cloud deploy, both from files) is exit 2. - `rote run` surfaces bundled dev UIs: the temporal runner starts its dev server with `ui=True` on an explicit `find_free_port()` port (`ui_port=None` picks one we couldn't print) and the inngest runner prints the dev-server dashboard URL — both live only while the run lasts. - Cloud-side compilation (`rote.cloud_compile` + `_cmd_compile_cloud`): logged in + flagless → the compilation runs ON rote cloud — bundle sync with per-file sha diff-skip, start-or-attach (an active compilation for the same skill attaches instead of double-spending; Ctrl-C cancels a run we started but only detaches from one we attached to), server events streamed through the same progress renderer/JSONL sink as local, artifacts downloaded into the standard `--out` layout. `--local`/`--cloud` flags (mutually exclusive); local-only flags (`--agent`, `--backend`, `--baseline*`, `--yes`, `--no-eval`) reject in cloud mode with a `--local` pointer. Config interplay: a config-pinned local runtime or `deploy: none` opts out of the cloud default (one-line reason printed); config `agent`/ `model` NEVER leak into cloud mode — the server owns driver choice and validates models against its own lineup. Server contract lives in rote-cloud (`/v1/skills` + bundle endpoints, `/v1/skills/:id/ compilations`, container job) — the server must be deployed before a CLI release that enables this. E2E-proven against local dev (including a drained-tenant "insufficient credits" path). - Login-aware LOCAL compile default (applies when the cloud path is opted out but a login exists): no explicit `--runtime` → emit `cloudflare` and auto-deploy to rote cloud after emission (`--no-deploy` opts out; a deploy failure exits 1 but keeps all local artifacts and prints the `rote deploy … --target rote-cloud` retry). Logged out → today's `dbos` default with a one-line hint, never a prompt (CI-safe). The parser's `--runtime` default is now `None` — resolution happens at the top of `_cmd_compile`; don't reintroduce a static default in the parser. - `rote register` + `rote serve` (compiled pipelines as MCP tools, FastMCP 4.x, stdio + Streamable HTTP — see [`docs/mcp-trigger.md`](docs/mcp-trigger.md)) - `rote eval` + the auto-emitted `compiled/scorecard.md` (static before/after estimate of speed, cost, determinism; live-fetched prices — see the eval gotchas below) - `rote eval --run` (empirical mode: real trials of both sides — `claude -p` for the skill, the emitted python/DBOS app for the pipeline with cross-process gate signaling via `DBOSClient`; real judge usage captured through the emitted `$ROTE_USAGE_LOG` hook; measurements appended to `~/.local/share/rote/eval-corpus.jsonl`) - 1228 tests (1201 fast + 27 slow). Run with `pytest tests/` (fast only — what runs by default). Slow tests cover the runtime e2e suites (Temporal, Cloudflare, DBOS, DBOS-TS, Inngest, MCP-over-stdio); the TS ones require a Node toolchain, DBOS-TS needs Docker, and Inngest downloads the `inngest-cli` binary. Run them with `pytest tests/ -m slow`. If you find something in the "working" column that doesn't work, file it as a bug. If you find something in the "stubbed" column that frustrates you, that's the roadmap — pick it up. --- ## What this file is not - **Not a README.** The [README.md](README.md) is the public-facing front door; it explains what `rote` is to someone landing on the repo cold. - **Not CONTRIBUTING.md.** The [CONTRIBUTING.md](CONTRIBUTING.md) is the human-facing collaboration guide. - **Not design doc.** The canonical design record is [`docs/agent-runtime.md`](docs/agent-runtime.md), which captures the driver-layer decisions with research citations. Read it if you're touching the driver layer. - **Not the rubric.** The compiler's rubric lives in [`skills/rote-compile/references/`](skills/rote-compile/references/) — that's what the agent reads. This CLAUDE.md is what *you* read. If information in this file contradicts information in `ir.py`, the test suite, or `scripts/sanity-check.sh`, **those are the source of truth and this file is out of date.** Please fix it.