CLAUDE.md · diff
git:20261019.d1e5050 to git:20261021.751a5a4
2671 added, 1881 removed, shown as one replacement because the pair was too large to align, first 3,000 lines shown. Audit A to A.
# CLAUDE.md — context for AI collaborators
This file is for Claude Code (and similar coding agents) working in
the `rote` repository. Read it before making non-trivial changes.
It captures the project's mental model, architectural invariants,
gotchas, and the canonical examples you should imitate.
If you're a human reading this, you probably want
[README.md](README.md) or [CONTRIBUTING.md](CONTRIBUTING.md)
instead — this file is deliberately oriented toward an AI that
needs a fast mental model before it starts editing code.
---
## What rote is
`rote` is a CLI that takes an Anthropic-style Skill (a `SKILL.md` +
`references/`) and compiles it into a runnable background pipeline:
1. An LLM agent (itself defined as a skill at
[`skills/rote-compile/`](skills/rote-compile/)) reads the
source skill and applies a structured rubric.
2. The agent produces a `pipeline.yaml` (runtime-agnostic IR) plus
extracted Python modules and typed LLM-judge signatures.
3. A runtime adapter (DBOS by default; Temporal, Cloudflare
Workflows, DBOS-TS, and Inngest too; more planned) consumes the
IR and emits the runtime's native code shape — a single `main.py`
DBOS app, a `workflow.py` + `activities.py` pair for Temporal, a
TypeScript `WorkflowEntrypoint` class plus `wrangler.jsonc` /
`package.json` for Cloudflare, or an Inngest function app that
mounts into an existing Node/Next.js service.
The result is a deterministic workflow that replaces a fuzzy
10-20 minute agent loop. That's the whole product in one paragraph.
---
## Three-layer architecture
Every piece of code in this repo belongs to exactly one layer.
**Respect the layer boundaries** — most of the project's value
comes from keeping them clean.
### Layer 1 — The compiler agent (the "brain")
- **Lives at:** [`skills/rote-compile/`](skills/rote-compile/)
- **Consists of:** a `SKILL.md` plus five rubric files
(`node-kinds.md`, `crystallization-heuristics.md`, `ir-schema.md`,
`llm-judge-extraction.md`, `implementation.md`)
- **What it does:** classifies every step of a source skill into
one of 5 node kinds, extracts deterministic procedures into
Python modules, designs typed signatures for LLM-judge steps, and
assembles the whole thing into a `pipeline.yaml`
- **Runs under:** a pluggable driver layer (see Layer 2)
-
- **When you edit this layer:** you are changing how the agent
- behaves. Changes are testable by running `rote compile` end-to-end
- against `examples/bdr-outreach/skill` and diffing the output
- against the previously committed snapshot in
- `examples/bdr-outreach/runs/`.
-
- ### Layer 2 — The driver layer (the "runtime for the agent")
-
- - **Lives at:** [`src/rote/compiler/drivers/`](src/rote/compiler/drivers/)
- - **Protocol:** every driver implements `CompilerDriver` with
- `name`, `is_available() -> (bool, str)`, and
- `async run(skill_dir, compiler_skill_dir, work_dir) -> DriverResult`
- - **Three concrete drivers:** `ClaudeDriver` (subprocess to `claude -p`),
- `CodexDriver` (subprocess to `codex exec`), `AnthropicApiDriver`
- (in-process via the `anthropic` SDK)
- - **The contract is the filesystem:** every driver writes
- `work_dir/pipeline.yaml` and returns a `DriverResult` pointing at it.
- Do not invent new return-shape fields or side-channel comms.
-
- **When you edit this layer:** you are changing how the agent is
- invoked, not what it does. Changes are testable by mocking
- subprocess spawning (see
- [`tests/test_claude_driver.py`](tests/test_claude_driver.py)) or
- the Anthropic SDK client (see
- [`tests/test_anthropic_driver.py`](tests/test_anthropic_driver.py)).
-
- ### Layer 3 — The runtime adapters (the "code emitter")
-
- - **Lives at:** [`src/rote/adapters/`](src/rote/adapters/)
- - **Five adapters today:**
- - `DbosAdapter`
- ([`src/rote/adapters/dbos.py`](src/rote/adapters/dbos.py))
- — the CLI default; emits a single durable Python app (`main.py`
- with `@DBOS.workflow`/`@DBOS.step`, plus `signatures/*.py`,
- `extracted/*.py`, `dbos-config.yaml`, `README.md`) — no
- orchestrator process, state in SQLite/Postgres
- - `TemporalAdapter`
- ([`src/rote/adapters/temporal.py`](src/rote/adapters/temporal.py))
- — emits Python (`workflow.py` + `activities.py`)
- - `CloudflareAdapter`
- ([`src/rote/adapters/cloudflare.py`](src/rote/adapters/cloudflare.py))
- — emits TypeScript (`src/workflow.ts` extending
- `WorkflowEntrypoint`, plus `signatures/*.ts`, `extracted/*.ts`,
- and a `wrangler.jsonc` / `package.json` / `tsconfig.json`)
- - `PythonAdapter`
- ([`src/rote/adapters/python.py`](src/rote/adapters/python.py),
- registered as `python`) — the maximum-legibility target: emits a
- plain, orchestrator-free Python script (`main.py` with one function
- per node, visible inline retry loops, stdlib `ThreadPoolExecutor`
- for parallel waves, plus `extracted/`, `signatures/`,
- `requirements.txt`). **Refuses pipelines with `hitl_gate` nodes at
- emit time** (via the derived `Pipeline.requires_durable_execution`
- property) — a plain script cannot durably park for human approval;
- the error points at `--runtime dbos`. Shares the Pydantic
- signature / extracted-stub emitters with the DBOS adapter via
- [`src/rote/adapters/_py_common.py`](src/rote/adapters/_py_common.py).
- - `DbosTsAdapter`
- ([`src/rote/adapters/dbos_ts.py`](src/rote/adapters/dbos_ts.py),
- registered as `dbos-ts`) — emits TypeScript for DBOS Transact
- (`src/main.ts` with `DBOS.registerWorkflow` / `DBOS.registerStep`
- function wrappers, plus `signatures/*.ts`, `extracted/*.ts`, and
- `package.json` / `tsconfig.json` / `dbos-config.yaml`). Shares the
- TS emission machinery in
- [`src/rote/adapters/_ts_common.py`](src/rote/adapters/_ts_common.py)
- with the Cloudflare adapter. Note: the DBOS **TS** SDK is
- Postgres-only (no SQLite parity with DBOS Python), and its
- `DBOS.recv` defaults to a 60s timeout — the emitter always passes
- the IR timeout explicitly. Targets DBOS 3 (Python) / Transact v5
- (TS); see the DBOS 3 queue gotcha below.
- - `InngestAdapter`
- ([`src/rote/adapters/inngest.py`](src/rote/adapters/inngest.py),
- registered as `inngest`) — emits a TypeScript Inngest app (one
- durable `inngest.createFunction` in `src/inngest/pipeline.ts`
- running the DAG waves, `step.waitForEvent` per HITL gate on a
- `<pipeline>/<signal>` event, plus a framework-neutral
- `inngest/node` serve entrypoint and a README documenting the
- Next.js mount). Shares `_ts_common.py` with the other TS
- adapters. Note: Inngest v4 retries are **function-level only** —
- the emitter maps max-across-nodes and comments the per-node
- deltas (see the Inngest gotcha below).
- - **What they do:** consume a validated `Pipeline` IR and write
- runtime-native code into an output directory
- - **Never run an agent loop.** Code emission is pure template
- substitution. The agent's job ends when the IR is valid; the
- adapter's job begins there.
-
- **When you edit this layer:** you are changing the output code
- format or adding a new runtime. Changes are testable with unit
- tests covering emission (AST / textual invariants), plus a real
- runtime smoke test — `tests/test_temporal_e2e.py` for Temporal
- (time-skipping `WorkflowEnvironment`); `tests/test_cloudflare_e2e.py`
- for Cloudflare (runs `npm install` + `tsc --noEmit` on the emitted
- output, gated by `@pytest.mark.slow`); `tests/test_dbos_ts_e2e.py`
- for DBOS TypeScript (`npm install` + `tsc --noEmit`, plus a live run
- on the real DBOS TS runtime against a Docker Postgres);
- `tests/test_inngest_e2e.py` for Inngest (`npm install` +
- `tsc --noEmit`, plus a live run through both HITL gates against the
- real Inngest dev server, `inngest-cli dev`).
-
- ---
-
- ## The IR is the source of truth
-
- [`src/rote/ir.py`](src/rote/ir.py) defines the Pydantic models that
- every layer agrees on. If you're uncertain about a field's semantics,
- read that file — it's ~290 lines, fully typed, and the authoritative
- reference (not the rubric's `ir-schema.md`, which is agent-facing
- prose that can drift).
-
- **The five node kinds:**
-
- | Kind | Required field | Where the LLM lives |
- | --- | --- | --- |
- | `pure_function` | `impl` | Not involved |
- | `external_call` | `impl` | Not involved |
- | `llm_judge` | `signature` (legacy path) and/or `signature_spec` (structured) | Typed signature (DSPy / BAML in Python; Zod + vendor SDK in TS) |
- | `agent_loop` | `tools` | Bounded agent loop |
- | `hitl_gate` | `signal` | Durable suspend/resume |
-
- **The `signature_spec` model** ([`src/rote/ir.py`](src/rote/ir.py))
- carries a JSON Schema for input + output, a Jinja-style prompt
- template, and the LLM client config (anthropic / openai). It's the
- runtime-agnostic alternative to the legacy `signature: 'path/to/file.py:Class'`
- form, which only works for Python adapters. Cloudflare requires
- `signature_spec`; Temporal accepts either and prefers `signature_spec`
- when present.
-
- **The canonical skill that covers all 5 kinds:** the BDR outreach
- example in `examples/bdr-outreach/skill/`. If you need an IR that
- exercises every code path, its hand-drafted baseline is at
- `examples/bdr-outreach/expected/pipeline.yaml`.
-
- ---
-
- ## Gotchas (things you will absolutely trip on if you don't read this)
-
- ### `claude -p` and the `ANTHROPIC_API_KEY` trap
-
- Claude Code's print mode (`claude -p`) has a documented behavior
- where `ANTHROPIC_API_KEY` **always wins** over any active OAuth
- session. If your system has the env var set, `claude -p` will use
- API-key billing even though the user expected subscription auth.
-
- **The fix:** `ClaudeDriver._build_child_env()` scrubs
- `ANTHROPIC_API_KEY` and `ANTHROPIC_AUTH_TOKEN` from the child
- environment before spawning. **Do not remove this.** If you need
- API-key auth to Anthropic, use `AnthropicApiDriver` instead — it's
- a separate driver precisely so the Claude driver can force
- subscription auth.
-
- ### YAML 1.1 parses `on:` as boolean `True`
-
- PyYAML follows YAML 1.1, which has historical bool keys (`on`,
- `off`, `yes`, `no`). A field literally named `on` in a YAML file
- becomes the Python value `True`, not the string `"on"`. This
- **broke the first IR test run** when I tried to use `on:` for
- retry trigger conditions.
-
- **The fix:** the IR uses `retry_on:` (with the `_on` suffix), not
- `on:`. Propagate this convention if you add new fields.
-
- ### Loop body sub-nodes are excluded from top-level waves
-
- An `agent_loop` node can declare a `loop_body: [node_id, ...]`
- listing sub-nodes invoked per iteration. Those sub-nodes also
- exist as top-level entries in the `nodes:` list (so they can be
- tested in isolation) but the adapter's `_execution_waves` function
- filters them out of the top-level DAG — otherwise the workflow
- would dispatch them twice. See `rote.adapters.temporal._execution_waves`
- for the reference implementation.
-
- ### Driver factories accept `**kwargs`
-
- The driver registry's factories have signature `**kwargs ->
- CompilerDriver` because `Compiler(model=...)` needs to pass
- kwargs through at construction time. Drivers that don't care about
- a particular kwarg should swallow it via `**kwargs` in their
- `__init__`, not hard-fail. **Do not change the factory signature
- to typed keyword args** — that breaks the registry's polymorphism.
-
- ### Emitted code touches MCP only through an explicit `mcp:` binding
-
- Two regimes, don't mix them up:
-
- - **Nodes without an `mcp:` binding must never reference MCP** in any
- adapter or language — no `mcp` imports, no `@mcp.tool()` decorators,
- no MCP SDK dependencies. Enforced per adapter:
- Temporal — `tests/test_temporal_adapter.py::test_emitted_activities_never_reference_mcp`
- (Python AST walk); Cloudflare —
- `tests/test_cloudflare_adapter.py::test_emitted_files_never_reference_mcp`
- (every `.ts` file, comments + string literals stripped). Copy the
- matching test when adding an adapter.
- - **Nodes WITH an `mcp:` binding** (DBOS adapter, `--backend mcp`, the
- default) emit a *working* FastMCP call: the step goes through the
- emitted `extracted/_rote_mcp.py` helper — the verbatim source of
- `rote.mcp._runtime_helper` (never hand-edit the emitted copy; fix
- the module) — which resolves the endpoint (env > rote registry > IR)
- and credentials (`rote mcp login` token store > registry static
- headers > unauthenticated) at runtime. On DBOS Python AND DBOS-TS,
- auth failures **park the workflow durably** instead of failing it
- (`RoteMcpAuthNeeded` → `DBOS.recv` on `rote:auth:<server>`;
- `rote mcp login` — or `rote mcp release` without a login — discovers
- parked workflows via the app registry + the `rote_auth_status` event
- and releases them, one Python code path for both runtimes). Traps if
- you touch this: a parked DBOS workflow is just PENDING (no WAITING
- status — discovery *requires* the advertised event); parallel-wave
- siblings can surface stale auth failures after the release signal was
- already consumed — hence the retry-once-before-parking shape in the
- emitted wrappers; and the cross-language leg only works in DBOS's
- *portable* serialization — the emitted TS `setEvent` passes
- `{ serializationType: "portable" }` and the Python release `send`
- passes `WorkflowSerializationFormat.PORTABLE` (the defaults,
- superjson and pickle, are mutually unreadable). In TS, detect the
- auth error by `name` string, never `instanceof` (serialized errors
- reconstruct as plain `Error`s). See
- [`docs/mcp-client.md`](docs/mcp-client.md).
-
- ### Sync Python runtimes run async work on ONE background loop
-
- python and dbos steps are plain functions, so every coroutine they need
- (an `async def` extracted function, a dbos MCP step, a legacy async
- judge) goes through the emitted `_run_async`, which submits it to one
- daemon-thread event loop per process. Do not "simplify" it back to
- `asyncio.run`: that makes and closes a loop per call, so a module-level
- `httpx.AsyncClient` fails from its second call on ("Event loop is
- closed"), which is every fan_out element and every retry; and it raises
- inside a running loop, where `run_pipeline` lands when async code
- imports it. `_call_impl` comes in two forms from one place
- (`_py_common.call_impl_helper`, `is_async=True` for temporal), and both
- check the *result* for awaitability, so a plain wrapper returning a
- coroutine works on every runtime (`tests/test_call_impl_helper.py`).
-
- ### A missing MCP endpoint is configuration: never retried, never parked
-
- `resolve_url` / `resolveUrl` raise `RoteMcpConfigError` (Python and all
- three TS helpers, detected by `name` in TS like the auth error) when no
- endpoint resolves. Every runtime's retry opt-out excludes it next to the
- auth error, so a step fails on its first attempt with a message naming
- `ROTE_MCP_<SERVER>_URL`. Agent loops reach the same gap through tool
- discovery, so `bindAgentTools` / `_mcp_config_for` raise the config error
- when it explains a missing tool, and loop steps carry a name-based opt-out
- (by name so `main` never imports MCP code, invariant #3). It must not join
- the park path: a parked run
- waits for a login, and no login supplies an endpoint. Before this, DBOS
- spent 4 attempts and surfaced only `DBOSMaxStepRetriesExceeded`
- (`tests/test_mcp_config_error_e2e.py`). On Cloudflare local dev the
- instance status then reads only `WorkflowFatalError`; the real message is
- in workerd's console output, which `rote.runners.cloudflare` reads back
- from `wrangler-dev.log`.
-
- ### Never return fastmcp's `result.data` from an MCP step
-
- `_result_payload` in `rote.mcp._runtime_helper` must NOT be
- "simplified" to `return result.data`. Re-verified on fastmcp 4.1.1
- against real TS-SDK servers (v1 and v2) and five hosted servers, it
- fails both ways:
-
- - **None for text-only results.** fastmcp hydrates `.data` only from
- structured content. FastMCP servers auto-generate output schemas
- (which is why FastMCP fixtures never caught this), but real-world
- servers, the TS SDK in particular, return JSON as text and `.data` is
- silently None. Found live on the first real-server run: every MCP
- step returned None.
- - **Unpicklable for typed results.** With an output schema, `.data` is
- a generated `types.Root` class: DBOS cannot pickle it, and keys that
- are not identifiers come back renamed (`display-name` →
- `display_name`). Found live against `docs.mcp.cloudflare.com`.
-
- The payload is `structured_content` (the wire value, with fastmcp's
- `{"result": ...}` wrapper removed only when the server marked it), else
- the parsed text blocks. Two things to keep: the marker is
- `_meta.fastmcp.wrap_result` on current fastmcp but only the
- `x-fastmcp-wrap-result` output-schema flag on 2.x/3.0 servers, so the
- schema is still needed; and the TS helpers apply the same rule through
- `_FASTMCP_UNWRAP_TS` (one source spliced into the Node and Workers
- helpers). The text fallback is shared the same way: `_MCP_TEXT_PAYLOAD_TS`
- (spliced into the Node, Workers and binding helpers) applies
- `_result_payload`'s exact text rules, which are every text block parsed,
- non-JSON kept verbatim, several blocks as a list, and none as null. The
- TS helpers used to read only the first block, throw on non-JSON text and
- return `{}` for no text, so the same MCP step returned different values
- per runtime (`tests/test_mcp_text_payload_parity_e2e.py` runs both
- helpers against one real server). The helper's explicit `list_tools()` is not an extra round
- trip, because the MCP SDK's `call_tool` lists tools itself on a fresh
- session to validate the result (measured, and pinned by
- `test_one_tools_list_per_step`). Regressions: `tests/test_mcp_client.py::
- test_result_payload_is_plain_json_from_a_real_fastmcp_server` and
- `tests/test_mcp_ts_server_e2e.py` (real TS-SDK servers, slow).
-
- ### fastmcp 4 hides HTTP status codes; the helper hooks the transport
-
- MCP SDK v2 turns any non-2xx answer into a bare `MCPError` (-32603,
- "Server returned an error response") with no cause chain, and the HTTP
- stack is httpx2, not httpx. So a 401 can no longer be found by walking
- the exception for `httpx.HTTPStatusError`; that check silently matched
- nothing and park-on-auth stopped working. `_HttpStatusWatch` builds the
- transport's httpx2 client with a response hook and remembers whether
- the latest answer was a 401. Regression: `tests/test_mcp_client.py::
- test_call_mcp_tool_maps_a_real_401_to_auth_needed` (a real localhost
- 401, with a 500 as the negative half). Any new `except httpx.` around a
- fastmcp call is dead code for the same reason.
-
- Two smaller fastmcp 4 traps: `client.ping()` raises "Method not found"
- on the sessionless 2026-07-28 era, which `Client(url)` negotiates by
- default (use an authenticated `list_tools()` to trigger OAuth), and an
- SDK v2 `ServerSession` is per request, so anything that remembers a
- client across requests must hold the `Connection` instead (see
- `rote serve`'s list_changed delivery, which also has to publish on a
- `subscriptions/listen` bus for modern clients).
-
- ### MCP 2026-07-28: what keeps both eras and every registration mode working
-
- Seven things, each with a test that fails when it is undone
- (`tests/test_mcp_era_matrix_e2e.py`, `tests/test_mcp_oauth_registration_e2e.py`,
- `tests/test_mcp_park_e2e.py`). See [`docs/mcp-client.md`](docs/mcp-client.md).
-
- - **The TS helpers must pass `versionNegotiation: { mode: "auto" }`.**
- `@modelcontextprotocol/client` 2.x defaults to `legacy`, which fails
- against a modern-only server exactly like v1 (`-32022`). Checking the
- import line proves nothing; the era matrix asserts the era each server
- saw. A 401 is `SdkHttpError.status`; v1's `err.code === 401` reads an
- SdkErrorCode string in v2 and never matches, which silently turns
- park-on-auth into a failure.
- - **The CIMD document's redirect URIs and `CALLBACK_PORTS` are one fact.**
- A CIMD server compares redirect URIs by exact string match and no spec
- defines a loopback port wildcard, so the login binds a published port.
- `tests/test_mcp_client.py::test_cimd_document_matches_what_the_login_offers`
- pins `docs/oauth/rote-cli.json` to the code, and the copy served by
- rote-cloud's marketing site must match it too. Add ports to the
- document before the CLI uses them, never after. Selecting a port must
- bind it (`bind_callback_sockets`), and the callback server must run on
- those sockets: a check now and a bind later lost the port to a
- concurrent login in CI after the redirect URI already named it.
- - **rote refuses registrations the SDK would send on its own.** Without
- a `registration_endpoint` the SDK POSTs a guessed
- `<resource-origin>/register` (GitHub: not its authorization server).
- `RoteOAuth._refuse_registration` stops that, the emitted helper refuses
- any registration (it parks instead), and an issuer-bound pre-registered
- client that the SDK dropped is not silently replaced by CIMD or DCR.
- The tests assert on the authorization server's hit log, because the
- failure is a request that should never have been sent.
- - **Refresh goes to the stored `token_endpoint`.** Before discovery the
- SDK refreshes at `<resource-origin>/token`. For a path issuer or an AS
- on another host that 404s, and an unattended run parks on a credential
- that was fine. `retarget_token_request` is shared by the CLI and the
- emitted helper. FastMCP fixtures cannot catch this: their AS is the
- server's own origin root.
- - **Never offer the CIMD document unverified.** The SDK does not fall back
- to DCR when a CIMD server rejects it, so an unpublished or stale
- document breaks logins that DCR serves. `_check_offered_document`
- fetches it once per process when the AS advertises CIMD and withdraws
- the URL unless it equals the packaged `client_metadata.json`.
- - **A token file is bound to an endpoint and an issuer.** Files are keyed
- by server name, so `stored_for_url` gates every read (else a repointed
- name gets the old bearer token), and `_with_bound_issuer` fills
- `client_info.issuer` from the file (the SDK treats a missing one as
- matching every issuer).
- - **`RoteMcpInputRequired` is not an auth error.** A server that wants
- input mid-call (`input_required`, or `-32021` for an undeclared
- capability; TS SDK servers send the latter because rote declares no
- capabilities) fails the step on every runtime, and every retry opt-out
- exempts it. Making it a `RoteMcpAuthNeeded` would park the run for 30
- days waiting for a login that answers nothing.
-
- ### An mcp-bound node is a BARE tool call — post-processing must split
-
- The default backend emits an mcp-bound step as exactly "call the tool,
- return the result". If a skill step is "call X, then filter/match/
- reshape", the transformation half must be its own `pure_function` node
- or it silently vanishes at emission (live failure: match-by-name logic
- lost, workflow crashed dereferencing a declared-but-never-computed
- key). The rubric (`node-kinds.md`, external_call) mandates the split;
- `rote.probe.cross_check` flags violations as `output_mismatch` (node
- declares `output` keys the observed tool result can't provide) and
- `rote compile` prints them as WARNINGs.
-
- ### `temperature` is rejected outright by current Claude models
-
- A judge carrying `temperature` 400s on every call against a current
- Anthropic model (`invalid_request_error: \`temperature\` is not
- supported on this model`). The compiler used to write
- `temperature: 0.0` on judges as a determinism instinct, which made
- every compiled pipeline dead on arrival the moment `ROTE_MODEL_<ID>`
- pointed at a current model — found live, first real gateway run.
-
- Two halves of the fix, keep both: the rubric
- (`llm-judge-extraction.md`) says never set `temperature` — determinism
- comes from schema-locked decoding (forced tool call / `json_schema`),
- not the sampler — and when the IR *does* set one, emitted judges
- render it as a `ROTE_TEMPERATURE_<NODE_ID>` env knob whose empty value
- drops the parameter entirely (`_sampling_kwargs()` in Python,
- `...(TEMPERATURE.trim() ? … : {})` in TS). Don't "simplify" it back to
- a literal keyword argument: a baked constant cannot be escaped without
- a re-emit, which defeats the model-override knob next to it.
-
- ### First-run output: three contracts that live in one place each
-
- Found by a fresh-install run of the published wheel, where every one of
- these read fine to someone who already knew the answer:
-
- - **Install hints come from `rote.extras.install_hint`.** The PyPI
- distribution is `rote-cli`; `rote` is somebody else's package, so a
- hand-typed `pip install rote[api]` installs the wrong thing.
- `tests/test_first_run_output.py` fails on the bare form, and on a
- hand-typed `rote-cli[` outside `rote/extras.py`.
- - **A failed subprocess reports its tail** (`rote.eval.empirical.failure_excerpt`).
- The exception is at the end; a `[:800]` head slice showed a newcomer
- DBOS's startup banner cut off mid-word and nothing else.
- - **`--json` always puts one object on stdout.** `main()` wraps every
- `--json` handler; a nonzero exit with empty stdout gets
- `{"ok": false, "exit_code", "error"}` built from the `error:` line the
- handler printed. New error paths only need the usual
- `print(f"error: ...", file=sys.stderr)`.
-
- `rote run` also refuses stubs up front (`rote.runners.preflight_pipeline`,
- reading the emit manifest via `rote.adapters._common.unimplemented_stubs`),
- so a stub never burns a durable runtime's retry budget before failing.
- A stub is a function raising with `STUB_SENTINEL` (`[rote stub]`, in
- every stub rote emits) or, in Python, one whose whole body is `raise
- NotImplementedError` (the shape compile agents write), at any depth,
- methods included, `@abstractmethod` excepted. The marker counts only
- where it is executable: in Python as a `raise` argument (AST), in TS
- after `strip_ts_comments`. A raw-text match kept a filled-in module that
- merely mentions the marker in a comment listed, and `rote run` refused
- it. Never go back to a `NotImplementedError` substring match either: a
- real module that raises it on an unsupported branch was refused with no
- way past it. `--allow-stubs` is the escape hatch.
-
- ### Vendor SDK errors do not survive a durable step boundary
-
- `anthropic.APIError` / `openai.APIError` subclasses take keyword-only
- `response`/`body` arguments, so `BaseException.__reduce__`'s
- `(cls, args)` cannot rebuild them. A runtime that persists a failed
- step by pickling the exception — DBOS does — therefore replaces the
- real status and body with `TypeError: APIStatusError.__init__()
- missing 2 required keyword-only arguments` at the join. That masked
- the temperature 400 above completely.
-
- Emitted judges wrap the SDK call and re-raise through
- `_durable_error()`, a plain `RuntimeError` carrying type, HTTP status,
- model, node id, and the original message (regression:
- `tests/test_dbos_adapter.py::test_judge_translates_sdk_errors_so_they_survive_a_durable_step`,
- which asserts the translated error actually pickle-round-trips). This
- bites hardest on the fan_out enqueue path, where the failure is
- observed in a *different* process from where it was raised — any
- durability boundary is also an error-fidelity boundary.
-
- ### The TS tool runner's `done()` deadlocks; use `runUntilDone()`
-
- `client.beta.messages.toolRunner(...)` returns an async iterator plus two
- ways to wait on it, and they are not interchangeable. `done()` *observes*
- a loop somebody else drives — its own docs say it "works even if the async
- iterator hasn't yet started, and will wait for an instance to start" — so
- a single-consumer caller that never iterates hangs forever, silently, with
- no output and no error. `runUntilDone()` consumes the iterator itself.
- The Python twin's `until_done()` has no such split, which is exactly why
- this is easy to port wrong. Found by running it (rc=124, zero output);
- tsc and the SDK's types are both happy with the deadlocking version.
-
- Related, same file: report the turns that actually ran, not the cap.
- `AgentResult.iterations` counts assistant messages
- (`runner.params.messages.filter(m => m.role === "assistant").length`) and
- is `number | null` — the Workers AI lane returns null because
- `runWithTools` genuinely does not report it. Echoing back
- `maxIterations` made every loop look saturated. The Python
- `_run_agent_via_sdk` has no public `params` accessor, so it drives the
- runner's iterator itself and counts each yielded message, summing usage
- over every turn. Then `until_done()` on the drained runner returns the
- final turn without another request.
-
- ### An agent_loop's tools carry no server — `tool_servers` supplies it
-
- `Node.tools` are bare names (`zoominfo_search_contacts`): one loop may
- reach several servers, and an `MCPBinding` is a single server/tool pair.
- That gap was not cosmetic. `Pipeline.required_mcp_servers` derives from
- server *names*, so a pipeline whose only MCP usage is an agent loop
- reported **zero** required servers — `analyze`/`emit`/`compile` and the
- `rote mcp login` advisories all said there was nothing to authenticate.
- BDR is exactly that shape, so the flagship example demonstrated the bug.
-
- `Node.tool_servers` (tool name → server, agent_loop only, keys validated
- ⊆ `tools`) closes it. Three things now depend on it:
-
- - **The advisories are correct.** `required_mcp_servers` counts both
- declarations — a node's `mcp:` binding and a loop's `tool_servers`. BDR
- reports five servers instead of none.
- - **A resolved tool binds from its own server and no other.** Two servers
- exporting one tool name is precisely what an allowlist cannot
- disambiguate; first-wins discovery would pick whichever answered first.
- Python narrows its `--allowedTools` cross product the same way.
- - **Workers works at all.** There is no registry in workerd —
- `resolveUrl(env, server, …)` reads env bindings — so the server name
- *must* be known at emit time.
-
- Three sources feed the emitted loop, in descending authority: the IR's
- `tool_servers` (knowledge), other nodes' `mcp:` bindings (a guess — the
- loop probably reaches the servers the rest of the pipeline does), and at
- run time the local registry plus `ROTE_MCP_SERVERS` (the escape hatch).
- Only the first is trustworthy, which is why only it feeds the advisories.
- A partial map is valid and an unresolved tool still works via discovery;
- the emitted module names what it could not resolve rather than failing
- later with a bare "no server provides this tool".
-
- Two behaviors to preserve:
-
- - **Nothing guesses.** `rote.probe.enrich_pipeline` writes `tool_servers`
- from what the baseline actually *called* (`cross_check`'s
- `resolved_agent_tools`), and leaves a tool served by two servers in one
- trial unresolved — pinning either would silently send the loop to the
- wrong endpoint. An authored value always beats inference.
- - **Discovery is best-effort, binding is not.** A server that is down may
- not be one this loop needed, so it is recorded and skipped; a declared
- tool no reachable server provides is fatal. When an auth failure
- plausibly explains the gap, that failure is rethrown so the workflow
- parks and `rote mcp login` releases it, instead of dying with a
- misleading "unknown tool".
-
- ### `rote run` must see a park, and only the runner can release a local one
-
- A parked run used to look like a hang under `rote run`. The run-side
- watcher (`rote.mcp.parked`) reads each runtime's park signal: the DBOS
- `rote_auth_status` event (now `{awaiting, node}`), and on Inngest and
- Cloudflare a `rote-auth-park {json}` line the emitted park prints before
- its `waitForEvent`, because neither local dev server reports what a run
- waits on. Traps, all found by running it:
-
- - **The prompt goes to stderr.** `input()` writes its prompt to stdout,
- which is `rote run`'s result JSON. On a pty with stdout piped the user
- is asked a question they never see and the run waits forever.
- `_stderr_input` exists for this; don't "simplify" it back to `input`.
- - **Replays reprint the marker.** Both TS runtimes re-execute workflow
- code after a release, so the line prints again. Dedupe on its `wait`
- field (the wait's step id), never on server or node.
- - **`rote mcp login` cannot reach a `rote run` dev server.** The
- registry-wide release targets Inngest's default port and the
- Cloudflare REST API. So the watcher releases its own run when the
- server's token file or registry entry changes. On DBOS it waits a few
- seconds first, because login already released registered DBOS apps.
- - **A login writes the token file twice.** Client registration lands
- first with no token. Releasing on that write wakes the run into the
- same failure, it parks again, and the finished login then looks like
- "no change". `credentials_settled` waits for an access token.
- - **A Worker never sees the local token store.** On Cloudflare the
- watcher shows the export-to-`.dev.vars` fix and neither offers a login
- nor self-releases, since a login there would "succeed" and change
- nothing the Worker can read.
- - **An auth park looks like a gate park to the Cloudflare stability
- heuristic.** While the watcher reports an active park, the runner must
- not spend a gate payload on it.
- - Non-TTY never prompts and never opens a browser. The browser login is
- offered only when stdin and stderr are both terminals.
-
- ### A Cloudflare agent_loop is parkable, so it leaves its parallel wave
-
- Its tools are MCP tools, so `bindAgentTools` and every in-loop tool call
- raise the same auth signal a bound step does — an agent loop therefore
- gets the park-on-auth wrapper on Cloudflare (invariant #3's park-on-auth
- would otherwise not hold on the logged-in default runtime). Two specifics:
-
- - The release event is derived from the failure at run time
- (`authEventType(err, fallback)` parses the server out of the message)
- because one loop may reach several servers and `ROTE_MCP_SERVERS` can
- add more. Own properties do not reliably survive the `NonRetryableError`
- re-throw; the message does.
- - Parkable steps stay OUT of `Promise.all` (see the parallel-wave gotcha),
- so an agent loop no longer runs concurrently with its wave siblings.
- That is a real throughput cost on the slowest node kind, accepted
- because `waitForEvent` inside a promise combinator is undocumented and
- its timeout *throws*, which would reject every sibling.
-
- ### Emitted judges do not contain the vendor call
-
- A judge module holds its Pydantic models, prompt, schema and operator
- knobs; the call itself goes through `call_judge` in
- `signatures/_rote_inference.py` — the verbatim source of
- `rote.inference._runtime_helper` (never hand-edit the emitted copy; fix
- the module — byte-equality test in `tests/test_inference.py`). That
- helper resolves *who pays*: `claude-cli` (subscription, local only),
- `api` (the user's key), `rote-cloud` (tenant account), selected by
- `ROTE_INFERENCE_<NODE_ID>` > `ROTE_INFERENCE` > auto-detect
- (cheapest-to-the-user first). See [`docs/inference-runtime.md`](docs/inference-runtime.md).
-
- Traps if you touch this:
-
- - **The provider is not in the IR and must never be.** A
- `pipeline.yaml` is shared, committed, and sometimes third-party — a
- billing decision encoded there travels to people it doesn't belong
- to. `signature_spec.client` stays (wire format is behavior); the
- provider is a deployment fact. Consequence to preserve: the same
- emitted artifact runs on a subscription locally and a key in CI with
- no re-emit and no diff.
- - **`build_subscription_env()` lives in the helper**, not beside the
- Claude driver, and `drivers/claude.py` / `eval/*` import it back. Two
- copies is exactly how the `ANTHROPIC_API_KEY` scrub gets fixed in one
- place and silently drifts back to key billing in the other.
- - **The refusal is at `rote deploy`, not `rote emit`.** Emit cannot
- know where the directory is headed — that's the point of the bullet
- above. Don't "move it earlier".
- - **An explicit endpoint rules out `claude-cli` in auto-detect.** The
- proven AI Gateway config is `ROTE_BASE_URL_<ID>` + a bearer token; a
- laptop with `claude` on PATH would otherwise ignore it silently.
- - **The helper is stdlib-only at module level** (vendor SDKs import
- lazily inside functions) and imports nothing from rote. That
- constraint is what makes the verbatim copy safe.
- - **No custom exception classes in the helper.** A custom class only
- unpickles where its module is importable, and the observer of a
- failed durable step isn't always the process that raised it — the
- same failure mode as the vendor-SDK gotcha below. Plain
- `RuntimeError` carrying type / status / model / node / provider.
-
- ### `claude -p` gets schema-locked output, and costs 37k tokens if you let it
-
- `claude -p --json-schema <schema> --output-format json` is a **forced
- tool call** under the hood (`stop_reason: "tool_use"`) and returns the
- validated object in `structured_output` — the subscription lane has the
- same structural guarantee as the SDK lane, not "prose that parses".
- Prompt goes on **stdin**, not argv (judge prompts carry whole
- documents; argv is size-limited and visible in `ps`).
-
- The default invocation ships Claude Code's entire coding-agent system
- prompt: measured ~37k cache-creation tokens and 11.5s for a
- one-sentence judge. `--system-prompt <one line>` + `--tools ""` +
- `--setting-sources ""` was supposed to fix that, and on a machine with
- claude.ai connectors it did not: Claude Code 2.1.302 still loaded the
- user's connector and plugin MCP schemas (`--tools` does not touch MCP
- tools) plus the auto-memory index of the repository the pipeline ran
- from. A real "say hi" judge through the helper measured 30.3k tokens,
- $0.183 cold; one greeted the developer by name from that memory. The
- connectors load asynchronously, so the same call ranged from 1.2k to
- 28k tokens run to run.
-
- The helper now passes one shared `_CLI_ISOLATION_FLAGS` tuple on BOTH
- paths. Each flag was measured by removing it from the full set:
-
- | Removed | Judge tokens (input + cache) |
- | --- | --- |
- | nothing | 1,250 |
- | `--strict-mcp-config` | 30,300 |
- | `--setting-sources ""` | 25,600 |
- | `--settings '{"autoMemoryEnabled": false}'` | 3,070 (from a repo with memory) |
-
- `--setting-sources ""` does not cover auto memory; only the setting
- does (or `CLAUDE_CODE_DISABLE_AUTO_MEMORY=1`, rejected because
- `build_subscription_env` is shared with the compiler). `--bare` would
- do all of it but refuses OAuth, which is the whole lane; `--safe-mode`
- measured the same 1.25k but is documented as a troubleshooting mode
- whose scope may change, so the explicit flags win.
-
- The agent loop takes the same tuple plus its own `--mcp-config`:
- strict mode means it sees only the servers rote resolved. It also now
- passes `--tools ""` when it has servers, because `--tools` removes only
- built-ins: before, the loop carried 31 coding tools (Bash, Write,
- Agent...) at ~31k tokens a turn. A real two-turn loop against a local
- fastmcp server went from 120k tokens / $0.38 to 2.8k / $0.012 with the
- tool still bound and called. Removing any flag silently restores its
- overhead. Also: the CLI reports API failures *inside a zero exit*
- (`is_error: true` in the envelope), so returncode alone calls a failed
- judge a success.
-
- The compiler's `ClaudeDriver` passes `--strict-mcp-config` too, with no
- `--mcp-config`: it wires no MCP (live compile-time MCP is the
- in-process drivers' job), and on this machine the user's connectors put
- 357 tools into the first compile request, past 200k tokens, which the
- subscription rejected outright ("Usage credits are required for long
- context requests"). Strict, that request is 70.7k. The compiler still
- inherits the user's settings, CLAUDE.md, auto memory, output style,
- permission mode and 25 unused built-in tools (11k tokens with all of
- those isolated too); that is a separate change because it alters what
- the compile agent is told, and needs a real compile eval to land.
-
- ### A prompt-cached run reports almost nothing in `input_tokens`
-
- `claude -p` always runs prompt-cached, so `input_tokens` holds only the
- few tokens that were neither written to nor read from the cache. A
- measured one-word reply reported `input_tokens: 3` beside
- `cache_creation_input_tokens: 122956`, and `total_cost_usd: 0.7378`.
-
- The compiler driver summed the plain pair only, with a docstring
- explaining that cache fields were dropped "to match the api driver". That
- reasoning is the bug: the api driver sends no `cache_control` at all, so
- two fields are complete for it and catastrophically incomplete here.
- Compiling the rote-compile skill against itself reported a 22-minute,
- 62-turn run as `tokens in=113 out=1919; cost $0.029`.
-
- Three things now hold, each with a regression test verified by breaking
- the code (`tests/test_claude_driver.py`, `tests/test_cli.py`,
- `tests/test_compiler.py`):
-
- - **All four buckets are counted.** `_usage_delta` returns
- `{input, output, cache_write, cache_read}`, and every display adds the
- cache buckets into the reported input rather than hiding them.
- Claude repeats input/cache usage across content blocks sharing one
- `message.id`; count each ID once. Assistant `output_tokens` is a
- placeholder: show output as pending until final `modelUsage` (whole
- query, including subagents) or `usage` (main loop) arrives. Recovery
- without final usage retains input/cache and marks output incomplete.
- - **The cost key is `total_cost_usd`.** There is no `cost_usd` key in the
- result envelope, so the old read returned `None` on every run while the
- estimate sat one key away. Prefer the CLI's whole-query estimate to
- rote's single-model estimate; neither is authoritative billing data.
- When changing this logic, consult the
- [SDK cost-tracking contract](https://code.claude.com/docs/en/agent-sdk/cost-tracking).
- - **Cache rates span the whole fetch.** models.dev publishes them per
- model and `_parse_models_dev` already read them, but `build_catalog`
- kept only the three tier representatives'. The compiler's default
- model is not one, so a default compilation had nothing to price its
- cache with. `PricingCatalog.rates_for` is the accessor that covers
- every model; `model_price_for` stays narrow. An older on-disk cache
- has no such key and degrades to "no published rate" rather than
- reading as free.
- - **An unpriceable run reports no cost, not a wrong one.** When a run
- touched the cache and the rates are genuinely unknown, `_cost_usd`
- returns `None`. Billing cache reads at the plain input rate would
- overstate them roughly tenfold.
- - **A judge's usage row names the model the CLI ran, not the alias.**
- `claude -p --model sonnet` runs `claude-sonnet-5-5`, and only the
- envelope's `modelUsage` says so. The emitted helper logs one row per
- `modelUsage` entry with the CLI's `costUSD` as `cost_usd`, which
- pricing takes as is: the CLI bills a 1h cache write at twice the 5m
- rate a catalog publishes. A row logged under `sonnet` matched no
- catalog entry, so the whole subscription lane went unpriced.
-
- The general lesson, and it generalizes past this file: a usage number is
- only as complete as the fields you chose to read, and a field you never
- read fails silently and looks like a small number.
-
- ### Sonnet is the default, not Opus
-
- Both `ClaudeDriver` and `AnthropicApiDriver` default to
- `claude-sonnet-4-6` rather than Opus, because Opus is ~5× more
- expensive and Sonnet is fully capable of following the structured
- rubric. We learned this the hard way — the first two BDR runs with
- Opus exhausted a Claude Max/Pro "extra usage" budget in two
- attempts (~$3.50 each). Don't "fix" the default to Opus unless you
- have specific evidence that a skill needs it.
-
- ### `DEFAULT_MAX_TURNS = 60`
-
- BDR-scale skills need ~25 tool calls minimum, realistically 40-50
- with exploration. The initial default of 30 caused an
- `error_max_turns` failure on the first real run. 60 leaves headroom.
- Don't reduce it without measuring.
-
- ### Cloudflare local-dev `status` stays `"running"` while parked on `waitForEvent`
-
- The Cloudflare e2e test (`tests/test_cloudflare_e2e.py`) needs to know
- when a workflow instance has parked on a `step.waitForEvent` so it
- can deliver the resume signal. The intuitive check —
- `status.status === "waiting"` — does **not** work in local dev. The
- top-level `status` field stays `"running"` the entire time; only the
- human-readable `wrangler workflows instances describe` output
- distinguishes "💤 Waiting for event".
-
- **The fix:** the test infers parking by step-output stability. Once
- `__LOCAL_DEV_STEP_OUTPUTS` reaches the expected count *and* stops
- growing for a few consecutive polls, the workflow is necessarily
- either at a HITL gate or done. See `_wait_until_parked_or_complete`
- in the e2e test for the canonical implementation. Production behavior
- may differ — this is a local-dev quirk; document it if anything
- changes upstream.
-
- Related: `__LOCAL_DEV_STEP_OUTPUTS` includes both `step.do` results
- *and* `waitForEvent` payloads in the same array. Don't rely on the
- length matching node count; compare node-name **sets** instead.
-
- ### Cloudflare `step.do` rejects `Record<string, unknown>` returns
-
- `step.do`'s generic is `T extends Rpc.Serializable<T>`, and
- `Record<string, unknown>` does **not** satisfy it — `unknown` values
- aren't structurally serializable, so annotating an emitted stub with
- that return type breaks overload resolution at every `step.do` call
- site. Leaving the return inferred is no better: a throwing function
- declaration infers `Promise<void>`, and `void as Record<...>` fails
- TS2352 at the data-flow reference sites.
-
- **The fix:** emitted stubs declare `Promise<never>` (honest for an
- always-throwing stub; `never` satisfies any constraint and casts to
- anything).
-
- **The trap that fix sets:** because `never` casts to anything, a
- reference-site cast that is wrong for real types compiles against the
- stubs and fails only once the user types an implementation. All three TS
- runtimes shipped `"<exit>": <exit>_result as Record<string, unknown>`,
- which is TS2352 the moment the node returns `Promise<string>` (found by
- an eval on ops-report; `rote run` passed because casts are erased). A
- single `as Record<string, unknown>` only compiles from a type that
- overlaps a Record: an anonymous object type does, an `interface` (no
- implicit index signature), an array, or a scalar does not. So exit
- results are returned uncast (`exit_return_ts_lines`) and field access is
- plain `<id>_result["field"]` (`ref_to_ts_expr`), both in `_ts_common.py`.
- Indexing `never` is legal, so the stubs still compile. Don't "fix" the
- field access with `as unknown as Record<...>`: it compiles everywhere
- precisely because it turns off tsc's field-name check, so an IR field the
- user's type names differently becomes `undefined` at run time instead of
- a TS7053.
-
- Checking the stubs compile proves nothing here.
- `tests/test_ts_typed_impls_e2e.py` replaces them with typed
- implementations of every output shape and runs `tsc` on all three
- runtimes (positive half), then renames one referenced field and requires
- tsc to fail on exactly that (negative half). The negative half is not
- optional: on Inngest, `step.run` returns `Jsonify<T>`, an anonymous
- object type that overlaps a Record, so a restored `as Record` field cast
- passes the positive half there and only the drift case catches it. Run
- the suite after touching any TS emitter's typing story.
-
- ### A parallel wave is not automatically parallel in the emitted code
-
- `_execution_waves` says which nodes MAY run concurrently; each adapter
- decides whether to take it, and the two mistakes here are easy to
- reintroduce because neither fails a test that didn't exist yet.
-
- Cloudflare computed waves and then emitted every `step.do` sequentially
- — the concurrency survived only as a `// ─── Wave N ───` comment, on the
- runtime that is also the logged-in default. It now emits
- `await Promise.all([...])`, which is Cloudflare's documented primitive
- (the ✅ example in "Rules of Workflows"; their own visualizer has a
- `ParallelNode` for it; no documented per-instance cap on concurrent
- steps). Three rules to preserve:
-
- - **Never wrap the wave in an outer `step.do`.** That advice is specific
- to `Promise.race`/`any`, where losing branches are discarded. Wrapping
- burns a step, subjects the whole wave to the 1 MiB step-result cap, and
- collapses per-node retry policies into one.
- - **`Promise.all`, not `allSettled`** — opposite of DBOS-TS, on purpose.
- Reaching the rejection means that node already exhausted its own
- retries, and every sibling's result is persisted under its own step
- name, so a workflow retry replays survivors from cache. (DBOS-TS uses
- `allSettled` for a different reason: a bare `Promise.all` can crash the
- Node process on unhandled rejections.)
- - **HITL gates and MCP-parkable steps stay sequential.** Both suspend on
- `waitForEvent`, whose behavior inside a promise combinator is
- undocumented — and whose timeout *throws*, which would reject the whole
- wave.
-
- Temporal had the mirror-image bug: `_emit_wave_call` emitted
- `retry_policy=`, but the `asyncio.gather` branch emitted only the
- timeout, so a node silently lost its declared retry budget the moment it
- gained a sibling in its wave — and parallel fetch waves are exactly where
- the flaky network calls live. If you add a parallel branch to any
- adapter, diff it against the single-node branch field by field.
-
- ### A Postgres container is "ready" over the socket before it is ready
-
- The `postgres` image's init phase runs a socket-only server, then
- restarts it. `pg_isready` over the socket says ready during that phase,
- so an app connecting right after it gets "Connection terminated
- unexpectedly". Check over TCP (`pg_isready -h 127.0.0.1`), which answers
- only once the final server is up. This only showed in CI, where nothing
- sat between readiness and the first connection.
-
- It turned into a hang because the emitted DBOS-TS `main()` called
- `DBOS.launch()` before its `try`. A launch that fails part way leaves
- pools and timers alive, so without the `finally` shutdown the process
- never exits and the runner reports a bare timeout. Launch stays inside
- the `try` (regression:
- `tests/test_dbos_ts_e2e.py::test_a_failed_launch_exits_instead_of_hanging`).
-
- ### Inngest v4: no per-step retries, no 3-arg `createFunction`, and a lying dev-server endpoint
-
- Three traps hit while building the Inngest adapter, all verified
- empirically against `inngest` 4.11.0 + `inngest-cli` 1.34.0:
-
- 1. **`createFunction` is two-argument in v4.** Triggers live in the
- options object: `createFunction({ id, retries, triggers: [{ event }] },
- handler)`. The classic `createFunction(opts, { event }, handler)` form
- from docs/blog posts **throws at runtime**. tsc alone won't catch a
- regression to the old form in emitted-string templates — the e2e's
- live run does.
- 2. **Per-step retry/timeout config does not exist.** `StepOptions` is
- `{ id, name?, parallelMode? }`. The IR's per-node `RetryPolicy` maps
- to a single function-level `retries` (max across nodes, clamped 0–20);
- per-node deltas, `backoff`, `retry_on`, and per-node `timeout` are
- emitted as comments. Don't "fix" the adapter to pass per-step retry
- options — there's nowhere to put them.
- 3. **`GET /v1/events/{id}/runs` lies while a run is parked.** During
- probing it reported `status: "Completed"` while the run was parked on
- `step.waitForEvent`. Use it only to discover the `run_id`, then poll
- `GET /v1/runs/{run_id}` (truthful `Running`/`Completed`). The run's
- *return value* is only available via the dev server's GraphQL API
- (`POST /v0/gql`, `run(runID:){output}` → `RunComplete` op JSON); the
- v1 REST `output` field comes back empty. Also: with `--no-poll` the
- dev server's single startup sync can race the app boot and never
- register it — the e2e sends an explicit `PUT` to the app's serve
- handler to force registration. See `tests/test_inngest_e2e.py`.
-
- ### DBOS 3 queues exist only after launch; emitted apps start through `launch()`
-
- DBOS 3 (Python) and DBOS Transact v5 (TS) removed direct queue
- construction: `Queue("name")` raises `DBOSException` at import, and
- `DBOS.register_queue` needs a launched runtime because queues now live
- in the system database. A workflow enqueued on a queue nobody registered
- does not fail; it sits in `ENQUEUED` forever. So the emitted `main.py`
- declares `QUEUE_NAME`, enqueues via `_enqueue` (a thin
- `DBOS.enqueue_workflow(QUEUE_NAME, ...)` wrapper), and exposes
- `launch()` = `DBOS.launch()` + `DBOS.register_queue(QUEUE_NAME)`. Any
- in-process launcher, tests included, must call `main_module.launch()`.
- A bare `DBOS.launch()` hangs the first parallel wave, which shows up
- only as a test timeout. Two more v5 traps on the TS side: the SDK no
- longer reads `dbos-config.yaml` at launch (only the `dbos` CLI does, so
- `rote._dbos.dbos_ts_system_database_url` ignores its
- `system_database_url` key), and starting or enqueuing a *step* directly
- now throws, which is one reason the DBOS-TS emitter keeps parallel waves
- on `Promise.allSettled` instead of queues.
-
- ### A node's constants reach its impl, and a tool only by name
-
- Several nodes can share one `impl` and differ only in `constants`
- (ops-report: three fetch nodes, one `fetch_file_content`, three
- `file_id`s). The Python runtimes passed no constants at all, so every
- fetch received `{}` and no implementation could tell the files apart.
- Three rules now, each learned the hard way:
-
- - **Merge in the step body, not the workflow's payload literal.** A
- `pure_function` / `external_call` step calls
- `impl(**{**payload, <constants>})` (`_py_common.impl_kwargs_expr`, used
- by dbos, python and temporal). The payload literal only reaches DAG
- callers. An agent loop calls its `loop_body` sub-nodes as tools with a
- payload the model built, so BDR's `enrich_contact_batch` lost
- `batch_size` the first time this was tried. Constants come last so the
- model cannot override a pinned value, and they never appear in the tool
- schema it sees. Judges are excluded: their input model rejects unknown
- keys.
- - **TS passes them the same way, at every call site.** Cloudflare,
- Inngest and DBOS-TS call the impl directly, so there is no step body
- to merge in. Instead `payload_ts_literal` / `fan_out_ts_binding` add
- `impl_constant_entries_ts` after the inputs, and the agent-loop module
- wraps a loop_body tool call as `{ ...args, <constants> }`. Both take a
- required `with_constants` / `mcp_backed_ids`, so a new call site has to
- decide, and an MCP-backed call passes False. A TS impl reads its
- constants from `input`; hard-coding them makes the IR's value dead
- config, as on Python. `tests/test_constants_parity_ts_e2e.py` runs the
- Python parity fixture live on all three TS runtimes.
- - **A constant reaches an MCP tool only when `mcp.args` names it.**
- Without `args`, the step sends its payload as the tool arguments, and a
- tool rejects arguments it does not declare (FastMCP: "Unexpected
- keyword argument"). `_common.mcp_tool_arguments` decides the arguments
- once for every runtime, Python and TS, so a tool gets identical
- arguments everywhere (`tests/test_constants_reach_every_caller.py`).
- - **The impl contract is checked at emit time.** An `extracted/` function
- that takes neither the constant nor `**kwargs` raises TypeError on its
- first call, possibly after earlier steps had side effects. The Python
- adapters run `require_constant_parameters` BEFORE writing anything, on
- the module text the runtime dir will hold (`EmitWriter.landed_content`:
- the agent's source, or a user edit the writer would preserve). They
- refuse the emit naming the node, the parameter and the full path of
- the file to fix, and `rote compile` turns that into exit 1 with the
- `rote emit` retry command. `check_contracts` reports the same break as
- `constant_not_accepted`. Most committed `examples/*/runs/` predate this
- and hard-code their constants, so they no longer emit on a Python
- runtime. On Temporal the check is skipped when `extracted_module` is
- not the emitted package (an explicit `TemporalAdapterConfig`, as the
- BDR snapshot uses): those modules still receive constants and would
- fail with TypeError at run time.
- - **Constants drive the logic.** An impl uses the value it is passed; a
- module-level copy is only the default for a direct call. A module that
- accepts a constant and then applies its own value turns the IR's
- constant into dead config (expense-audit did this until
- `test_editing_a_constant_in_the_ir_changes_the_rule`).
-
- Related, found by the loop test: the SDK tool runner does not serialize a
- dict tool result, it iterates it, so the model saw only the keys.
- `_runtime_helper._text_result` JSON-encodes a local tool's result, as the
- TS twin already did.
-
- ### A fan_out element's step name must carry its index
-
- Cloudflare and Inngest key a durable step by its **name**, so a fan_out
- node emits `` `<id>[${_index}]` ``, not `"<id>"`. Reusing one name for
- every element is the nastiest failure shape available here: Cloudflare
- returns element 0's *cached* result for all N elements, so the run
- succeeds, the output has the right length, and every entry is
- identical — no error, anywhere. Inngest's memoization collapses the fan
- to a single execution the same way (its e2e asserts
- `count == FAN_OUT_ELEMENTS` for fan_out nodes and `== 1` for the rest,
- which is what catches it). DBOS is the exception on purpose: both its
- Python and TS SDKs identify a step by execution order, so repeated
- calls to one registered step are already distinct — no suffix needed,
- and adding one would be meaningless rather than harmful.
-
- Second, on Cloudflare only: a **parkable** fan_out node (MCP-bound)
- runs its elements *sequentially*, not in `Promise.all`, for the same
- reason a parkable step leaves its parallel wave — `waitForEvent` inside
- a promise combinator is undocumented and its timeout *throws*, which
- would reject every other element.
-
- ### A judge's `inputs:` must satisfy its input schema, and a fan passes one element
-
- Every adapter builds a judge's payload from its `inputs:` keys and
- nothing else (`{"thread": _item}` for a fan, plus shared params), then
- validates it against `signature_spec.input_schema`. So a `required`
- property `inputs:` does not bind fails *every* call. A bound key the
- schema does not declare is also rejected, open schema or not: Python
- silently drops it, and before the Zod emitter honored
- `additionalProperties` the TS runtimes rejected every call with it. A
- binding the contract cannot name is a mistake on every runtime.
- Nothing caught that before run time: the IR loaded, every adapter
- emitted, and the first real eval of deal-monitor failed all of its
- fanned `classify_thread_step` calls on a missing `account_name`.
-
- `Pipeline._validate_judge_input_bindings` now rejects it at load. Three
- things to preserve:
-
- - **The fanned compiler shape is the usual offender.** The compiler
- tends to flatten the element into top-level schema fields
- (`thread_id`, `messages`) while binding the list under one name
- (`threads`), or to declare a per-element value beside the element.
- Neither can work, because the grammar has no per-element reference;
- the per-element value must travel inside the element (the upstream
- step puts `account_name` on each `MatchedThread`). The rubric says so
- with the real wrong shapes; keep them there.
- - **It resolves the element with `rote.ir.fan_out_element_param`**, the
- same function the emitters call, which is why that function moved out
- of `rote.adapters._common`. A validator with its own resolution rule
- could pass a pipeline the emitters dispatch differently.
- - **Two exemptions, both principled.** Loop-body sub-nodes are called by
- their parent loop with model-chosen arguments, so their `inputs:`
- describe rather than bind. A legacy `signature:`-only judge has no
- schema in the IR to check. Don't widen the check to `Node.input_schema`
- on other kinds: that is observed documentation, not a contract any
- adapter enforces.
-
- Unknown-key handling is behavioral emitted code, so it must match across
- languages: Zod gets `.strict()` exactly where Pydantic gets
- `extra="forbid"` (`additionalProperties: false`), strips otherwise, and a
- property-less object is a free-form record on both.
- `tests/test_judge_schema_parity_e2e.py` runs one schema through the
- emitted Pydantic model and real `zod` and compares verdicts and parsed
- values; touch either converter and run it.
-
- `tests/test_ir.py::test_every_committed_pipeline_validates` sweeps every
- `pipeline.yaml` under `examples/`, so a snapshot that would fail its
- first judge call cannot be committed.
-
- ### Optional pipeline inputs are left out; required ones fail on entry
-
- `Pipeline.optional_input_fields` (listed in `input.optional`, or a schema
- property the schema's `required` omits) and `required_input_fields`
- (everything else a node reads, plus the declared required) are derived,
- never stored. Emitted code relies on the pair:
-
- - **An omitted optional field is absent from the payload, not None.**
- Python used to emit `pipeline_input["f"]`, so ops-report's optional
- `report_date` raised `KeyError` whenever a caller left it out. Reading
- it with `.get` was the obvious fix and the wrong one: an explicit None
- overrides the impl's own default (`def f(d="today")`) and fails a
- judge input field that has a schema `default`. Python payloads now
- spread each input field in with the emitted `_input_field`, which adds
- the key only when the run supplied it. TS gets the same effect for free,
- since an undefined field is dropped from the JSON payload.
- - **Every runtime checks the input on entry** with the emitted
- `_missing_inputs_error` / `missingInputsError`. The check names missing
- required fields, and it reports a non-object input (a run started with
- no payload at all) instead of a TypeError inside generated code. It is
- emitted whenever a node reads the pipeline input (`needs_input_check`).
- The failure must not retry. Temporal raises
- `ApplicationError(..., non_retryable=True)`, because a plain exception
- in workflow code fails the workflow task and Temporal retries that
- forever. Inngest throws `NonRetriableError`, because its retries are
- function-level.
-
- `tests/test_pipeline_input_parity.py` runs the Python runtimes for real
- both ways. `tests/test_pipeline_input_ts_e2e.py` runs the missing-input
- case live on Inngest, Cloudflare and DBOS-TS, and proves Inngest fails on
- the first attempt. The Inngest runner now reports the failed run's error
- name and message from the dev server, where it used to show only a log
- tail that held the end of a stack trace.
-
- ### An undefined fanned list must name the node, on every runtime
-
- Emitted code routes the fanned list through a guard — `fanOutList` in
- TS, `_fan_out_list` in Python — that throws naming the node id and the
- IR reference. Without it the failure is `Cannot read properties of
- undefined (reading 'map')` (TS) or `'NoneType' object is not iterable`
- (Python): no node, no reference, inside generated code the user never
- wrote. Found by running it — the first live fan_out run on Cloudflare
- died exactly that way, and the cause is almost always an upstream node
- that didn't return the key the IR says it does.
-
- Both guards are emitted only when the pipeline has a fan_out node, and
- they are *behavioral* emitted code like `_serialize`: if they drift, the
- same broken pipeline reports differently per target. Diagnostics are
- part of the emitted contract — a guard on one runtime and not another is
- its own parity gap.
-
- Related trap in the **tests**: a live e2e's mocks must return the fanned
- key as a non-empty list. Three TS e2e suites had `exclusion_check_sequence`
- returning `passed: []`, which made BDR's fan a correct no-op — the node
- never ran and every assertion still passed. `tests/_helpers.py`'s
- `fan_out_source_keys` now derives those mock values from the IR using the
- adapters' own resolution, so a mock cannot disagree with the emitted code
- about which input is the list.
-
- ### Eval pricing: no hardcoded prices, and two traps found empirically
-
- `rote.eval.pricing` fetches current models + official prices at eval
- time (models.dev primary, OpenRouter cross-check; 24h cache in
- `~/.cache/rote/pricing.json`; loud `PricingError` when offline — never
- a baked-in table). Two things you will get wrong if you touch it:
-
- 1. **"Most expensive = flagship" is false.** Previous-generation
- flagships linger in catalogs at higher prices than the current one
- (Opus 4.1 at $15/Mtok vs Fable 5 at $10/Mtok). Flagship detection
- restricts to the provider's *current generation* — models released
- within `_CURRENT_GENERATION_WINDOW` of the provider's own newest
- release date. The anchor is the data's newest release, never the
- wall clock, so tier detection is a pure function of the payload and
- testable with fixtures. Regression:
- `tests/test_eval_pricing.py::test_tier_detection_matches_current_lineup`.
- 2. **Community catalogs lag launches.** On Sonnet 5's launch week,
- LiteLLM's price file still carried the 4.6-generation price while
- models.dev and OpenRouter agreed on the new one — that's why
- LiteLLM was dropped and why the cross-check annotates the
- scorecard's `source` string with a WARNING on >2% disagreement
- instead of trusting either side silently.
- 3. **Cloudflare Workflows infrastructure has no price API.** Steps and
- storage started billing on 2026-08-10. `rote.eval.infra_pricing`
- fetches the one table both Cloudflare pricing pages render, the raw
- `src/content/partials/workflows/workflows-pricing.mdx` from
- `cloudflare/cloudflare-docs@production`. It caches the raw file for
- 24h in `~/.cache/rote/` and re-parses it on every load. The parser is
- strict, and both halves of that matter. A missing row or a missing
- `Workers Paid` column raises `PricingError`, because a skipped row
- prices that unit at nothing. Each cell must also match a closed
- grammar anchored at both ends. A prefix match read "per additional
- 100K steps" as a per of 100, overstating the price 1000x. The
- wording has already changed once (2026-09-09). Steps per run
- (`cloudflare_steps_per_run`) are a Range: one per node, the fan width
- per fan_out node, one per HITL gate, none for `loop_body` sub-nodes.
- The fan width is the token estimate's own `fan_out_widths` (eval.yaml
- iterations, else the prior), so steps and LLM dollars never assume
- different element counts. Counting a gate's wait as a step is labeled
- a conservative assumption, since the docs don't say. Retention
- defaults to 30 days: the emitted wrangler.jsonc sets no
- `default_retention`, so the account maximum applies (limits.mdx
- footnote 2). The 2026-09-10 changelog's seven-day default for new
- Workflows contradicts that page, and this takes the larger figure.
- CPU and state size are stated priors. `rote eval --runtime
- cloudflare|rote-cloud` adds the section, as does a `compile --out` dir
- holding only `runtime/cloudflare`, or a compile that emits for
- cloudflare. Regressions in `tests/test_eval_infra_pricing.py`:
- `test_a_table_missing_any_row_is_rejected`,
- `test_an_unknown_unit_phrase_is_rejected_not_misread`.
-
- Related invariant: the eval sidecar (`eval.yaml`, per-step agent-turn
- estimates written by the compiler) is deliberately NOT part of the IR
- — it describes the *source skill's* behavior as an agent, not the
- pipeline. Don't move it into `pipeline.yaml`; invariant #1 (the IR is
- runtime-agnostic and behavior-only) applies.
-
- ### ClaudeDriver recovers `pipeline.yaml` if subprocess errors after writing
-
- The agent's deliverable is a file on disk, not a clean exit code. A
- real-world failure mode hit during this project: the agent wrote a
- complete `pipeline.yaml` at turn N, then ran a self-validation turn
- at N+1 that hit `ECONNRESET` — the subprocess returned nonzero, the
- orchestrator's `TemporaryDirectory` cleanup ran, ~$2.50 of Sonnet 4.6
- work disappeared.
-
- The driver now checks for `pipeline.yaml` **before** failing on
- returncode. If the file exists, the run succeeded materially; the
- subprocess error is surfaced in `metadata["subprocess_warning"]`
- rather than treated as fatal. Don't undo this — losing $2 runs to a
- network blip is unacceptable. See
- `tests/test_claude_driver.py::test_nonzero_exit_with_pipeline_yaml_recovers_run`.
-
- ### The compile verifies its deliverable, and the agent repairs it
-
- `rote.compiler.verify.verify_deliverable` runs on every natural stop of
- the agent and once more at the end: IR validation, `check_contracts`
- (symbol, payload signature, Python 3.11 parse), the compile-only rules
- (every `pure_function` implemented and named by a test, or a stub whose
- `NotImplementedError` message says why), and `python -m pytest tests`
- from the work dir. Failures go back into the SAME conversation as one
- user turn, up to `MAX_VALIDATION_REPAIRS`. Things to preserve:
-
- - **The compile-only rules stay out of `check_contracts`.** That one
- also runs at `rote emit` on hand-written and older IRs, where an
- unwritten module is the documented stub path. Moving the rules into
- it would flag every expected/ example on every emit.
- - **`ast.parse(feature_version=(3, 11))` does not catch PEP 701
- f-strings** (a backslash, comment, or reused quote inside `{}`): that
- changed the tokenizer, not the grammar. `newer_syntax_problems` walks
- the 3.12+ token stream instead. Its corpus test is differential across
- the CI matrix: on 3.11 it checks the labels against the real parser.
- - **Generated tests run with `PYTHONDONTWRITEBYTECODE=1`.** A repair
- can rewrite a module within the same second at the same size, and a
- stale `.pyc` then makes the fixed code test red.
- - **An update verifies an overlay** of the previous output's code under
- the work dir, because an update's work dir holds only what changed.
- - **A deliverable that still fails is returned, not raised.** It is the
- output of a paid run. The CLI exits 1 and skips the cloud deploy.
- - **Verification never raises.** It runs inside the driver's repair loop
- (where only `DriverError` is caught) and before the work dir leaves its
- temp home, so an exception there deletes the paid work. Any crash
- becomes a `verification_error` finding; a non-UTF-8 module is a
- `not_utf8` finding.
- - **A rejected IR goes to `<out>.rejected`, never into `<out>`.** A
- re-run into an existing `--out` must not replace the last good
- pipeline.yaml and the stubs the user filled in since.
- - **Generated tests are untrusted code and run in an OS sandbox, or not
- at all** (`rote.compiler.sandbox`). The agent wrote them from a
- SKILL.md that may be third-party (invariant #7), and rote runs them
- automatically, up to three times. An env allowlist alone was shown to
- stop nothing: `pwd.getpwuid` names the real home, the network is open,
- and the ssh-agent socket and Keychain CLI are reachable. So:
- - macOS: `sandbox-exec` with a generated deny-by-default profile.
- Reads: the work dir; the interpreter's stdlib, its site-packages
- dirs, and as single files the binary, libpython and `pyvenv.cfg`
- (`sandbox.current_layout`, derived from `sysconfig`, never "the dir
- above the binary", which for `~/bin/python` is the home and for
- `/usr/bin/python3` all of `/usr`); `/` as a literal; `/System`,
- `/usr/lib`, `/usr/share`, dyld; a few devices. Writes: the work dir and
- a private scratch dir. Exec: literals only, the binary and a framework
- build's `Python.app` launcher. No network. A layout whose roots
- include `/`, a home, an ancestor of one, or anything holding
- `~/.ssh`, `~/.config/rote` or `~/.local/share/rote` is refused
- (`sandbox.refusal`), which means no sandbox, so unverified. `file-read-metadata` is allowed
- everywhere because path resolution needs it, so a test can learn that
- a path exists, not what it holds.
- - Linux: `bwrap --unshare-all` (network included), read-only binds of
- the system lib dirs and interpreter paths, binds of the work dir and
- scratch dir, tmpfs `/tmp`, `--clearenv`, no home. CI installs it via
- `.github/actions/bubblewrap` and lifts Ubuntu's AppArmor userns
- restriction; `tests/test_sandbox.py` fails instead of skipping when
- `CI` is set. No `unshare -n` fallback: without a mount namespace
- rebuilt the way bwrap does it, the home stays readable.
- - The child runs `python -S` (no `.pth` processing) with PYTHONPATH set
- to the work dir and pytest's site-packages, and pytest's config,
- rootdir and conftest boundary pinned to the work dir: walking up
- reads the user's project, which the sandbox denies.
- - The env is still an allowlist (`verify.generated_test_env`: PATH,
- locale, an empty HOME), not a scrub list.
- - No working sandbox: the tests are skipped, status `unverified`, said
- in the compile output and in a `## rote verification` section rote
- appends to compile-report.md. `--trust-skill` runs them with the env
- allowlist only; `--no-verify-tests` never runs them.
- - Not isolated: file metadata on macOS (above), CPU and memory (only the
- 120 s timeout), writes inside the work dir (a test can modify the
- deliverable it tests).
- Regressions: `tests/test_sandbox.py` (each denial verified by removing
- it from the profile), and
- `tests/test_verify.py::test_generated_tests_cannot_see_credentials_or_the_users_home`.
- - **One stub predicate.** `rote.adapters._common.stub_raise` decides what
- a stub is for both `rote run`'s preflight and the compile's per-node
- status (whole body after the docstring is one `raise
- NotImplementedError`). Two predicates disagreed once.
- - **Outcomes cite the skill or are declared** (`rote.compiler.provenance`).
- Found by a real compile that chose block over review for a rule whose
- severity the skill never states, then wrote tests that encoded the
- choice. Every rule in it exists because a review got a guess past a
- looser version, so keep them: a citation fragment is 4+ words of prose
- (headings excluded), fragments match in order within one paragraph,
- the quote closes at the `"` ending a comment line (so it may contain
- quotes), every outcome literal in a pure_function module needs a
- marker on its statement or in the comment directly above it or above
- the `if` that decides it (an early-return `if` ladder counts as one;
- independent checks do not, because proximity let a real compile's
- cap severity pass under an unrelated citation), labels reached by
- reference (module constants, policy tables, Enum members) are checked at
- their definition, and Open questions must have one entry per distinct
- assumption. Presence guards against an accidental unmarked label, not a
- deliberate evasion: a slot whose name is not in `OUTCOME_WORDS`, or a
- label built at run time, is not seen.
- `tests/test_provenance.py` holds the gaming cases, each verified by a
- mutation.
-
- ### The claude driver repairs by resuming, and a resumed result is cumulative
-
- `ClaudeDriver` sends a repair as `claude -p <instruction> --resume
- <session_id>` with the same flags. Measured on claude 2.1.302: a resumed
- run's `total_cost_usd` and `modelUsage` cover the WHOLE session (the
- first run's plus its own), while `num_turns` and `duration_ms` are per
- invocation. That is observed, not documented with a version floor, and
- least likely to hold after an abnormal exit, so `_session_metadata`
- checks it: non-decreasing costs mean cumulative (use the last result),
- anything else is summed per round (`usage_basis` says which), and a
- round with no result marks `usage_complete` False. The usage gotcha
- above applies: a total that silently drops a round reads as a small
- number.
- `--resume` looks the session up by cwd, so every invocation of one
- compile runs with `cwd=work_dir`.
-
- ### The compiler's `claude -p` child is isolated from the user's setup
-
- `rote.inference.CLI_CONTEXT_ISOLATION_FLAGS` (`--setting-sources ""`,
- `--strict-mcp-config`, `--settings '{"autoMemoryEnabled": false}'`), then
- `--tools <allowlist>`, and `cwd=work_dir`. The context flags are the SAME
- tuple the emitted judge and agent-loop lanes use (they add `--tools ""`
- instead); the driver imports it from the helper, like
- `build_subscription_env`, so there is one copy. Each one was found by a real compile: a user's Vale
- prose hook (em dashes in `compile-report.md`) cost 12 of 48 turns in
- one-line edits. The same user settings granted Bash despite the
- allowlist. And run from this repo, the agent globbed into
- `examples/<skill>/runs/` and read an earlier compile of the same skill as
- a template, so the output depended on where the command was typed.
- `--tools` takes a VALUE here, as in the judge helper. When measuring the
- compiler, run it against a copy of the skill outside the repo anyway:
- an older installed rote has none of this.
-
- ---
-
- ## Canonical examples to imitate
-
- When adding new code, the fastest way to stay consistent is to
- copy the shape of an existing canonical example.
-
- | What you want to do | Imitate |
- | --- | --- |
- | Add a new node kind to the IR | `rote.ir.Node` — add an enum variant, a kind-specific required field, and a branch in `_validate_kind_specific_fields` |
- | Add a Python-emitting runtime adapter | `src/rote/adapters/temporal.py` (~570 lines) |
- | Add a TS-emitting runtime adapter | `src/rote/adapters/cloudflare.py` (~750 lines) — includes the JSON-Schema-to-Zod converter and signal-name validator |
- | Add a subprocess-based driver | `src/rote/compiler/drivers/claude.py` + `tests/test_claude_driver.py` |
- | Add an in-process driver | `src/rote/compiler/drivers/anthropic_api.py` + `tests/test_anthropic_driver.py` |
- | Add a new CLI subcommand | `_cmd_emit` in `src/rote/cli.py` (simple) or `_cmd_compile` (which calls into the Compiler orchestrator) |
- | Test a Python-emitting adapter end-to-end | `tests/test_temporal_e2e.py` (uses Temporal's `WorkflowEnvironment.start_time_skipping`) |
- | Test a TS-emitting adapter end-to-end | `tests/test_cloudflare_e2e.py` (real `npm install` + `tsc --noEmit`; `@pytest.mark.slow`) |
- | Test the CLI via subprocess | `tests/test_cli.py::test_subprocess_emit_bdr` |
- | Add a rubric file | Any file in `skills/rote-compile/references/` — keep them 200-300 lines, concrete, with BDR examples for every point |
-
- ---
-
- ## Invariants that must hold
-
- These are the things that, if violated, break the project's design.
- If a change of yours would violate any of them, stop and reconsider.
-
- 1. **The IR is runtime-agnostic.** No Temporal-specific or
- Inngest-specific concepts in `rote.ir`. If a field is meaningful
- only to one runtime, it belongs in that adapter's config, not
- the IR.
- 2. **Drivers contract on the filesystem only.** `work_dir/pipeline.yaml`
- is the deliverable. No stdout parsing, no side channels.
- 3. **Emitted code touches MCP only where the IR declares it.** There
- are exactly two declarations, and no third: a node's `mcp:` binding,
- and an `agent_loop`'s `tools:` (which the IR documents as MCP tool
- names — a loop that declares any pulls in the MCP helper even when
- no node in the pipeline carries a binding). Everything else never
- references MCP in any adapter or language, enforced by AST/text
- tests that **name** the legitimate sites rather than skipping files
- that happen to mention MCP — so an unexpected reference still fails,
- and an expected site that stops referencing MCP fails too
- (`tests/_helpers.py::assert_no_mcp_in_ts`'s `expected=`). Both
- declarations emit a working, *never-interactive* client call — auth
- problems raise/park, they never open a browser from workflow code.
- 4. **Mandatory nodes cannot become conditional.** The IR validator
- rejects `mandatory: true` on `agent_loop` nodes, and the
- Temporal adapter emits mandatory nodes as unconditional
- activities. Adding a conditional-skip mechanism would destroy
- the "MANDATORY prose check becomes impossible-to-skip code"
- value prop.
- 5. **The repo is PH-free and secret-free.** `scripts/sanity-check.sh`
- is the source of truth. Run it before any push. If you add a
- new example skill or a new test fixture, run the script
- afterward.
- 6. **Two adapters before declaring the IR generic.** *(Satisfied as
- of v0.2.0.)* The IR was stress-tested by the Cloudflare adapter,
- which emits TypeScript with a single-class programming model
- (vs. Temporal's workflow + activities split). Two pressure points
- surfaced:
-
- - The retry `backoff` enum already maps cleanly to Cloudflare's
- categorical enum — no IR change needed.
- - HITL signal names need a `[A-Za-z0-9_-]+` constraint for
- Cloudflare's `waitForEvent`, but this is enforced at adapter
- emission time, not in the IR.
- - LLM signatures needed a structured cross-language form — solved
- by adding `signature_spec` (JSON Schema + prompt) alongside the
- legacy Python-path `signature` field.
-
- The Inngest adapter added a third pressure point without requiring
- an IR change: Inngest v4 has **no per-step retry/timeout config**,
- so the per-node `RetryPolicy` degrades to a function-level budget
- (max across nodes) with per-node deltas emitted as comments. That's
- a lossy mapping documented at the adapter layer — acceptable
- because the IR stays the richer form; a runtime that can't express
- a policy documents the gap rather than forcing the IR down to the
- lowest common denominator.
-
- Future adapters (Restate, etc.) shouldn't require IR changes for
- their core programming model — but if one does, that's a real
- signal to revisit the IR shape rather than papering over it in the
- adapter.
-
- 7. **IR string fields that reach emitted code are charset-constrained
- at the IR boundary, not in the adapters.** A `pipeline.yaml` is
- untrusted input (it's LLM-generated from a possibly third-party
- `SKILL.md`, or shared directly), yet every adapter splices certain
- fields *verbatim* into emitted source and filenames. `Node.id`
- (emitted as `async def {id}` / `@activity.defn(name=…)` and as
- `signatures/{id}.py`), `signal` (a Temporal signal-handler method /
- DBOS topic), and the two halves of `impl`/`signature`
- (`from … import {symbol}` + call site, and the module path) are all
- validated to a safe identifier / relative-path shape by
- `field_validator`s in `rote.ir` — see `_IDENTIFIER_RE`,
- `_validate_impl_ref`, and the `Node._validate_id/_validate_signal/
- _validate_impl_signature` validators. Prose fields (`description`)
- can't be charset-restricted, so they're escaped at emission via
- `rote.adapters._common.safe_docstring_line` (Python docstrings) and
- `safe_block_comment_line` (TS JSDoc). Emitted-file writes also pass
- through `resolve_within` as defense-in-depth. **Do not add a new IR
- string field that an adapter emits verbatim without giving it an
- equivalent validator**, and don't move these checks down into a
- single adapter — the IR boundary is where all three (and every
- future) adapter inherit them. Regression coverage:
- `tests/test_ir.py::test_node_id_must_be_a_safe_identifier`,
- `::test_impl_symbol_injection_is_rejected`, and
- `tests/test_temporal_adapter.py::test_malicious_description_cannot_break_out_of_docstring`.
-
- ---
-
- ## Tests vs. evals: the dividing line is inference
-
- **Anything that does not require inference is a test.** Automated, run
- on every push, and kept fast.
-
- **Anything that costs tokens is an eval.** Run deliberately and
- periodically — `rote eval --run`, the compiler against a real skill,
- a live gateway judge. Never wired into CI.
-
- This is not a style preference. A token-spending check in an automated
- suite bills someone on every push, gets slower as it grows, and fails
- for reasons that have nothing to do with the change under test. Both
- halves of the split are enforced, not just documented:
-
- - `tests/conftest.py::_no_inference_in_tests` scrubs every vendor
- credential from the environment and makes a bare-name `shutil.which`
- lookup of `claude` / `codex` return None. The second half is the one
- that matters: `claude -p` authenticates from an OAuth session, so
- having no API key is *not* protection — not finding the binary is.
- Two DBOS e2e suites burned real subscription inference on every run
- until it was caught by wall-clock timing.
- - `tests/test_no_inference_in_tests.py` verifies the guard itself,
- because its failure mode is silent spending that no other assertion
- would notice.
- - Only *bare-name* lookups are blocked, so a test that writes a fake
- `claude` stub and passes its absolute path still works — a guard that
- forced tests to route around it would soon be disabled.
-
- Consequences worth keeping in mind:
-
- - **An e2e test is still a test.** The TypeScript suites run real
- `npm install`, real `wrangler dev`, a real Inngest dev server and a
- real Postgres — no inference, so they are automated (see the
- `TypeScript e2e` / `DBOS-TS e2e` CI jobs). `@pytest.mark.slow` says
- what *toolchain* a suite needs, never whether it should run.
- - **Emitted judges and agent loops get stubbed, not called.** The live
- suites point `ROTE_BASE_URL_<NODE_ID>` at a local Messages stub and
- set a fake key. That is what makes it possible to prove the agent-loop
- machinery end to end without paying per run.
- - **Speed is a feature of the test half only.** The fast suite is ~18s;
- keep it there. Evals are allowed to be slow and expensive because
- nothing blocks on them.
-
- ---
-
- ## Testing discipline: make assertions capable of failing
-
- Five real defects in this repo shipped past a green suite because a
- test observed something *adjacent* to the behavior. This is the single
- most common failure mode here, so it gets its own section.
-
- | Defect | What the test checked | What it should have checked |
- | --- | --- | --- |
- | bare `--tools` killed the whole subscription lane | `"--tools" in command` | the flag's **value** |
- | DBOS e2e mocked agent loops via an unreachable branch | that the overlay file was written | that the mock was actually *called* |
- | BDR's fan was a no-op in 4 live e2e suites | that the workflow completed | that the fanned node ran **N** times |
- | per-node `timeout:` ignored (temporal, cloudflare gates) | that *a* timeout was emitted | that the **declared** one was, and the default was not |
- | `1h` emitted as `"1 hours"` | nothing — no per-node timeout test existed | — |
-
- Three habits that catch these:
-
- - **Assert the value, not the presence.** A flag, key, or node id
- appearing in the output says nothing about what it was set to.
- - **Give a negative half.** Pair "the declared value is present" with
- "the default is absent" — otherwise emitting both still passes.
- - **Make fixtures non-degenerate.** An empty list makes a fan, a loop,
- or a retry a *correct* no-op that no assertion can catch. Fixture
- collections that drive iteration count carry >1 element
- (`tests/_helpers.py::FAN_OUT_ELEMENTS`), and their values are derived
- from the IR (`fan_out_source_keys`) so a mock cannot disagree with the
- emitted code about which input is which.
-
- **Verify a new test by breaking the code.** Not by inspection — apply
- the regression the test is meant to catch and confirm it fails. Any
- test written for a bug fix should be run against the *unfixed* code
- once. When the fix spans runtimes, check the split: the fan_out parity
- suite fails on the five broken runtimes and passes on `dbos`, which
- already worked, proving it recognizes correct behavior rather than just
- matching new strings.
-
- A periodic mutation sweep is the systematic version, and it works well
- here because emission is pure template substitution — mutate a module,
- run `pytest tests/`, and any mutation that survives is an untested
- behavior. Three sweeps so far, 64 mutations, 48 caught on the first pass:
-
- - **Adapters** (18): retry policy, timeouts, step naming, payload
- threading, MCP allowlist narrowing, the `ANTHROPIC_API_KEY` scrub,
- invariants #4 and #7. Gaps found: per-node `timeout:` was ignorable on
- both temporal and cloudflare gates with nothing failing.
- - **Drivers / inference / MCP / probe** (17): the flags that keep
- `claude -p` cheap, `is_error` inside a zero exit, fastmcp's
- `result.data`, probe's never-guess rule, and the cost-critical driver
- defaults.
- - **`rote.eval` / `rote.config` / `rote.cli`** (29): dollar arithmetic,
- tier detection, the read-only MCP gate, the four-layer config
- precedence, and the login-aware compile default. Config precedence and
- pricing tier detection were already airtight — every mutation died.
- Six gaps found, all in the two places where a wrong number or a wrong
- exit code looks like success (see below).
-
- Two lessons from the second sweep worth keeping:
-
- - **The agent-loop CLI path is the weak twin.** `_runtime_helper` carries
- two copies of the `claude -p` flag block (judge, agent loop), and
- mutations died on the judge copy while surviving on the loop copy —
- `--setting-sources` and `is_error` were covered on one and not the
- other, and `--system-prompt` on neither. That asymmetry is exactly how
- the bare `--tools` bug shipped. When you touch one copy, test both.
- - **A symbolic assertion cannot pin a constant.**
- `args[args.index("--model") + 1] == DEFAULT_MODEL` verifies the flag is
- wired up and says nothing about the constant, because both sides move
- together. `DEFAULT_MODEL` could be flipped to Opus (~5x cost) and
- `DEFAULT_MAX_TURNS` halved (the original `error_max_turns` failure)
- with the suite fully green. `tests/test_cost_defaults.py` pins the
- *property* — not-Opus, `>= 60` — so a legitimate model bump passes and
- the expensive regression fails.
-
- Three more from the third sweep:
-
- - **A self-comparison is the same tautology in disguise.**
- `pipeline_cost_usd` had `cost.high == pytest.approx(expected_high)`
- recomputed from the fixture rates and killed every mutation.
- `agent_run_cost_usd` was only ever compared against *itself*
- (`with_cache.mid < without_cache.mid`) — an ordering that holds under
- any transformation applied to both sides, so the divisor could become
- 1e3 instead of 1e6, or the cache-write premium could vanish, with the
- suite green. Cost numbers are what a user makes a spending decision
- on; assert the dollars.
- - **Asserting the loop bound leaves the per-call figures unpinned.** The
- `agent_loop` estimate asserted `calls.high` and nothing else, so both
- the token and the wall-time lines could drop
- `agent_loop_turns_per_iteration` (a 3x understatement) undetected —
- in the turn-dominated regime the loop-aware model exists to capture.
- - **Strictness is a property of a layer, not of one command.** Four
- commands call `load_layers()`; only `rote config` asserted exit 2, so
- compile's handler could `return 0` — reporting success while doing
- nothing. The replacement test parametrizes over all four, and the
- next command to read config fails there instead of shipping a silent
- fallback.
-
- Watch for **equivalent mutants** — the fan_out tier ordering in
- `fan_out_element_param`, and `args.no_deploy` in `_cmd_compile`'s
- cloud-vs-local branch, both survive because they are provably
- unobservable, not because they are untested. Both are commented in
- place; confirm the claim empirically before writing one off, since
- "probably equivalent" is exactly what a real gap looks like.
-
- ---
-
- ### The suite never writes into examples/
-
- The committed runtime snapshots under
- `examples/bdr-outreach/expected/runtimes/<runtime>/` are compared, not
- regenerated: adapter suites emit into `tmp_path`, and
- `tests/test_committed_snapshots.py` checks that a fresh emission (with
- the same `extracted/` sources `rote emit` reads) reproduces each one
- byte for byte. The suites used to emit straight into those dirs, so every
- pytest run rewrote committed files, and without the example's sources it
- replaced the real implementations with stubs. That dirtied the
- merge-queue worktree and blocked merges. A session hook in
- `tests/conftest.py` now fails the run if anything under `examples/`
- changed (`__pycache__` and `node_modules` excepted). When an adapter
- change is intended, regenerate with the `rote emit` command the failing
- snapshot test prints, and commit the result.
-
- ## Workflow expectations
-
- ### Running tests
-
- ```sh
- .venv/bin/pytest tests/ # full suite, ~1s
- .venv/bin/pytest tests/test_ir.py # fast iteration on IR changes
- ```
-
- The suite is intentionally fast (no real API calls, no real
- subprocess spawns, no real LLM invocations). Every slow thing is
- mocked. Keep it that way — if a new test needs to be slow, gate
- it behind a `@pytest.mark.slow` decorator and document why.
-
- ### Running the real compiler
-
- ```sh
- .venv/bin/rote compile examples/bdr-outreach/skill \
- --runtime temporal \
- --out /tmp/bdr-compiled
- ```
-
- ~13 minutes wall clock on BDR. Expect 30-40 turns with Sonnet 4.6.
- The output landing at `/tmp/bdr-compiled` (or wherever you point
- `--out`) is what you'd commit as a new regression snapshot if the
- rubric or IR changed in a way that requires re-compiling.
-
- ### Sanity-checking before commit
-
- ```sh
- ./scripts/sanity-check.sh
- ```
-
- Must exit 0. If it exits non-zero, either the scan found real leaks
- (fix them) or the pattern needs adjustment (be careful — err on
- the side of false positives over false negatives).
-
- ### Before a PR
-
- - [ ] `pytest tests/` — all tests pass
- - [ ] `./scripts/sanity-check.sh` — clean
- - [ ] `ruff check . && ruff format .`
- - [ ] `mypy src/rote` — strict, no ignores
- - [ ] If the rubric or IR changed materially: re-run the compiler
- on BDR and diff the snapshot
- - [ ] Commit message explains the *why*, not just the *what*
-
- ---
-
- ## What's stubbed vs. working
-
- Don't waste time debugging stubs. These are intentional.
-
- **Intentionally stubbed:**
-
- - The BDR example's `extracted/*.py` modules raise
- `NotImplementedError` — users fill them in with real API client
- code. The compiler leaves vendor calls (`external_call` impls) as
- contract-documented stubs unless an `mcp:` binding or probe ground
- truth makes them real. `pure_function` nodes are NOT in this column:
- the compiler implements and tests them (see the verification gotcha).
-
- **Working end-to-end:**
-
- - `fan_out` on ALL SIX runtimes: a node marked `fan_out: true` is
- invoked once per element of its bound list, and the result binds as a
- list in input order. Which input is the list is resolved by
- `rote.ir.fan_out_element_param` (fan_out edge marker >
- incoming-edge source > only node-bound param > with no node-bound
- param at all, the only pipeline-input field that can hold a list;
- ambiguity is an emit-time error, never a guess). It lives in the IR module on
- purpose: two adapters disagreeing about which input is the list would
- make one pipeline mean two different things, and the judge-binding
- validator must agree with them too. Per
- runtime: DBOS enqueues one durable step per element; python uses
- `pool.map`; temporal `asyncio.gather` over one activity execution
- each; Cloudflare and Inngest `Promise.all` over per-element steps;
- DBOS-TS `Promise.allSettled` + `unwrap`. See the fan_out gotchas
- below for the two traps.
-
- - `agent_loop` on ALL runtimes. Python got the real loop first; the
- three TypeScript runtimes now emit one too — MCP tools bound through
- whichever MCP helper that runtime already emits, `loop_body` sub-nodes
- bound as callables, and the whole thing handed to `runAgentLoop` in
- the verbatim `signatures/_roteInference.ts`
- (`rote.adapters._ts_common.ROTE_INFERENCE_HELPER_TS`). Proven live in
- `tests/test_ts_agent_loop_e2e.py`: real fastmcp server, real emitted
- module compiled by the emitted toolchain, real Anthropic SDK tool
- runner against a local Messages endpoint.
- - IR load + validate
- - `rote emit` (IR → runtime code; DBOS default), with hash-guarded
- re-emission: every adapter writes through
- `rote.adapters._common.EmitWriter`, which records what rote wrote in
- `.rote-manifest.json` and, on re-emit, preserves any file the user
- edited (fresh content lands in a `<name>.new` sibling and the CLI
- reports it). Never write emitted files with bare `write_text` in an
- adapter — route through the writer. Emission also sources the agent's
- REAL `extracted/<module>.py` implementations found beside the
- pipeline.yaml verbatim into the runtime dir (dbos, python and
- temporal adapters, `rote.adapters._py_common.resolve_extracted_source`);
- IR-derived stubs are only the fallback. Temporal writes its package
- through the shared `write_extracted_package` (python uses it too) and
- its activities call plain functions in a worker thread and await
- coroutine functions, since agent-written modules are plain. Only the
- BDR snapshot points temporal at a user-maintained package, through an
- explicit `TemporalAdapterConfig(extracted_module="expected.extracted")`.
- Known gap: TS runtimes can't consume Python modules at all (the
- compiler would have to emit TS implementations).
- - `rote compile` (SKILL.md → IR → Temporal code), stamping a
- `provenance.json` sidecar (section hashes; see `rote.skill_source`).
- Every run streams NDJSON progress to `<out>/progress.jsonl`
- (`--progress-file` just moves it) and prints the `tail -f` watch
- command as its first stderr line — agents driving a run relay that
- line to the human (AGENTS.md).
- - `rote compile --update` — incremental re-compilation: diffs the
- skill against the stamped provenance, re-derives only nodes whose
- source sections changed, enforces that unchanged nodes' ids survive,
- and merges file-level so user-filled stubs are kept. No section
- changes → no agent run at all.
- - Per-node inference overrides in emitted judges: `ROTE_MODEL_<ID>` /
- `ROTE_BASE_URL_<ID>` (plus `ROTE_TEMPERATURE_<ID>` when the IR sets
- one) env vars beat the IR defaults on all four runtimes, and
- `signature_spec.base_url` reaches any OpenAI-compatible endpoint.
- Node.source (provenance) is excluded from the pipeline hash on
- purpose — metadata must not re-version in-flight workflows.
- **Proven config — judges through a Cloudflare AI Gateway** (no
- Anthropic key at all; Unified Billing pays provider rates from CF
- credits and logs every call's tokens + cost):
- `ROTE_BASE_URL_<ID>=https://api.cloudflare.com/client/v4/accounts/<ACCT>/ai`,
- `ROTE_MODEL_<ID>=anthropic/claude-sonnet-5`, and the CF API token in
- `ANTHROPIC_AUTH_TOKEN` — the SDK sends it as `Authorization: Bearer`,
- which is what that endpoint wants; passing it as `ANTHROPIC_API_KEY`
- (→ `x-api-key`) gets a 401, contradicting Cloudflare's own JS
- example. Two more traps: only *current-generation* model ids resolve
- (`anthropic/claude-sonnet-4-5` from the CF docs 404s), and `@cf/*`
- Workers AI models are rejected by the `/v1/messages` route entirely —
- they only work on `/v1/chat/completions`. Routing to a *named*
- gateway needs a `cf-aig-gateway-id` header, which emitted clients
- can't inject; the account's default gateway needs no header.
- - All three compiler drivers: `ClaudeDriver` (`claude -p`),
- `CodexDriver` (`codex exec` — workspace-write sandbox, no env
- scrubbing since Codex login is only overridden by `CODEX_API_KEY`,
- smoke-tested against the real CLI), `AnthropicApiDriver`
- - `rote analyze` — the `plan` to `compile`'s `apply`: runs the
- compiler, then prints a structural report (node-kind breakdown,
- roteness matching `rote eval`, HITL gates, targetable runtimes)
- instead of emitting runtime code. `--json` for machines; `--out`
- keeps the IR for a later `rote emit`.
- - MCP requirements manifest: `Pipeline.required_mcp_servers` (derived,
- never stored — same rationale as `requires_durable_execution`) feeds
- an `mcp_servers` entry — `{server, nodes, tools, auth}` — in
- `analyze`/`emit`/`compile` output (human, `--json`, and the
- progress-file summary line), joined against the local registry +
- token store. Deliberately **advisory-only**: rote never hard-gates
- on auth (park-on-auth is the backstop); it prints `rote mcp login`
- recommendations for servers that would park a run.
- - Data-flow threading: nodes declare `inputs:` (param → source
- reference, grammar in `rote.ir.parse_input_ref`) and all three
- adapters — Temporal, Cloudflare, and DBOS — thread real payloads
- through the DAG, with HITL gate resume payloads participating as
- the gate's result — validated empirically in the runtime e2e tests
- - Park-on-auth (ALL MCP-capable runtimes — DBOS Python, DBOS-TS,
- Inngest, Cloudflare): MCP-backed steps with a missing/dead
- credential suspend the run durably; `rote mcp login <server>` (or
- `rote mcp release <server>`) releases every parked run across the
- apps recorded in `~/.local/share/rote/apps.json` (written at
- emit/compile time). Release channels differ per runtime: DBOS =
- discovery + per-workflow send (portable serialization); Inngest =
- one broadcast event per pipeline (events fan out; no discovery
- needed, but NO buffering for unstarted waits — retry-once covers the
- race); Cloudflare = REST blast to every non-terminal instance
- (no broadcast, but events buffer per-instance, so blasting is
- race-free; needs CLOUDFLARE_API_TOKEN/ACCOUNT_ID, local dev uses
- `wrangler … send-event --local`). Retry opt-outs also differ:
- should_retry (DBOS-py) / shouldRetry (DBOS-TS) / NonRetriableError
- (Inngest) / NonRetryableError (Cloudflare — and its waitForEvent
- THROWS on a 24h-default timeout, hence the explicit "30 days").
- Proven live per runtime: `tests/test_mcp_park_e2e.py`,
- `test_mcp_park_ts_e2e.py`, `test_mcp_park_inngest_e2e.py`,
- `test_mcp_park_cf_e2e.py` (the last also runs the TS MCP SDK live
- inside workerd).
- - `rote run` — one-off local execution of either side, built as a thin
- detection/dispatch layer (`rote.runners`) over the proven trial
- primitives: a skill dir runs via `rote.eval.baseline.run_baseline_trial`
- (registry MCP injection + read-only gate), an emitted dir via
- `rote.eval.empirical.run_pipeline_trial` (python in-process; dbos with
- cross-process `DBOSClient` gate delivery — payloads are collected up
- front, safe because DBOS notifications persist per-topic; cloudflare
- under `wrangler dev --local` via `rote.runners.cloudflare`, driving
- the emitted `src/index.ts` router with parked-then-send gate delivery
- in topological gate order — parking is inferred from
- `__LOCAL_DEV_STEP_OUTPUTS` stability because local-dev status lies,
- see the Cloudflare gotcha; inngest via `rote.runners.inngest` —
- emitted serve entrypoint + managed `inngest-cli dev`, forced PUT
- registration, and gate events RE-BROADCAST every poll tick because
- Inngest drops events with no active waitForEvent, with output read
- from the dev server's GraphQL RunComplete op; dbos-ts via
- `rote.runners.dbos_ts` — the emitted main.ts shares dbos-py's
- one-shot CLI contract, signals go Python→TS over the portable
- serialization channel, and an unreachable Postgres falls back to a
- throwaway Docker container; NOTE the TS SDK prints its startup
- banner to STDOUT ahead of the result JSON, hence the trailing-JSON
- extraction; temporal via `rote.runners.temporal` — no subprocess
- dance at all: temporalio's WorkflowEnvironment.start_local() manages
- a real dev server, the worker runs in-process on the emitted
- workflow.py/activities.py with UnsandboxedWorkflowRunner, and
- signals send up front because Temporal buffers them server-side).
- ALL SIX emitted runtimes are now locally runnable.
- Detection understands both the `compile --out` layout
- (`runtime/<target>/`) and bare emitted dirs (marker files — see
- `rote.runners._detect_runtime`).
- - `rote deploy` — push-deploy wrappers (`rote.deploy`): cloudflare via
- `npx wrangler deploy` (whoami-preflight surfaces the session account
- first — ambient wrangler state on this machine is the WORK account),
- dbos/dbos-ts via `npx dbos-cloud app deploy` (verified current: no
- `app register` step anymore; CI auth = `login --with-refresh-token`).
- Temporal/Inngest/python have NO push model — the command prints
- current hosting guidance with doc URLs instead of pretending
- (Temporal Cloud hosts only the server; Inngest syncs via
- authenticated POST to api.inngest.com/v2/apps/<id>/syncs, the old
- unauthenticated PUT-to-your-endpoint flow is deprecated). The
- `rote-cloud` target (`rote.deploy_rote_cloud`) bundles the emitted
- `src/workflow.ts` with `npx esbuild` (esm/neutral, worker-first
- conditions, `cloudflare:workers` + `node:*` external — must match the
- platform's reference bundler) and POSTs the manifest-derived
- DeployPayload to `/v1/pipelines` with a tenant bearer token; the
- emitted cloudflare `manifest.json` is REQUIRED (no TS-regex
- fallback). Proven live against the platform running under
- `vite dev`.
- - `rote login` / `logout` / `whoami` (`rote.cloud_auth`) — OAuth 2.0
- device flow (RFC 8628) against the platform's better-auth
- `deviceAuthorization` plugin. BOTH UXes ride the one grant path:
- browser available → auto-open `verification_uri_complete` (code
- pre-filled, one Approve click); `--device`/headless → printed code +
- URL. There is deliberately NO localhost-callback listener — no second
- server-side mechanism to secure. The polled session token is traded
- at `POST /v1/cli/keys` for a durable tenant `rote_…` API key; the CLI
- never persists a session token. Credential at
- `~/.local/share/rote/cloud.json` (0600, atomic;
- `ROTE_CLOUD_CRED_PATH` test override — conftest sets it suite-wide).
- Resolution everywhere: flag > `ROTE_CLOUD_URL`/`ROTE_CLOUD_TOKEN` >
- stored login. `logout` revokes server-side (`DELETE /v1/cli/keys`,
- the key revokes itself) but ALWAYS clears the local store; `whoami`
- verifies live via `GET /v1/me`.
- - Layered defaults (`rote.config`) + `rote init` + `rote config`:
- `runtime`/`deploy`/`agent`/`model` resolve flag > `ROTE_*` env >
- project `rote.yaml` (walk-up discovery, stops at `.git`;
- `ROTE_PROJECT_CONFIG_PATH` overrides) > user
- `~/.config/rote/config.yaml` (`ROTE_CONFIG_PATH` overrides —
- conftest isolates both suite-wide) > the command's built-in. Config
- files are strict: unknown key or invalid value = loud exit 2, never
- a silent fallback. `rote init` is the interactive wizard that writes
- them (the ONLY interactive command besides login; exits 2 without a
- TTY; I/O injected for tests — and its `input_fn` default resolves at
- call time, not def time, or monkeypatched `builtins.input` never
- lands). `rote config` prints each key's value + winning source.
- Compile deploy semantics: config `deploy: rote-cloud` + logged out
- fails fast BEFORE the agent spends money; an explicit `--runtime`
- flag downgrades a configured cloud deploy with a warning instead of
- blocking; contradictory config (non-cloudflare runtime + rote-cloud
- deploy, both from files) is exit 2.
- - `rote run` surfaces bundled dev UIs: the temporal runner starts its
- dev server with `ui=True` on an explicit `find_free_port()` port
- (`ui_port=None` picks one we couldn't print) and the inngest runner
- prints the dev-server dashboard URL — both live only while the run
- lasts.
- - Cloud-side compilation (`rote.cloud_compile` + `_cmd_compile_cloud`):
- logged in + flagless → the compilation runs ON rote cloud — bundle
- sync with per-file sha diff-skip, start-or-attach (an active
- compilation for the same skill attaches instead of double-spending;
- Ctrl-C cancels a run we started but only detaches from one we
- attached to), server events streamed through the same progress
- renderer/JSONL sink as local, artifacts downloaded into the standard
- `--out` layout. `--local`/`--cloud` flags (mutually exclusive);
- local-only flags (`--agent`, `--backend`, `--baseline*`, `--yes`,
- `--no-eval`) reject in cloud mode with a `--local` pointer. Config
- interplay: a config-pinned local runtime or `deploy: none` opts out
- of the cloud default (one-line reason printed); config `agent`/
- `model` NEVER leak into cloud mode — the server owns driver choice
- and validates models against its own lineup. Server contract lives
- in rote-cloud (`/v1/skills` + bundle endpoints, `/v1/skills/:id/
- compilations`, container job) — the server must be deployed before a
- CLI release that enables this. E2E-proven against local dev
- (including a drained-tenant "insufficient credits" path).
- - Login-aware LOCAL compile default (applies when the cloud path is
- opted out but a login exists): no explicit `--runtime` → emit
- `cloudflare` and auto-deploy to rote cloud after emission
- (`--no-deploy` opts out; a deploy failure exits 1 but keeps all
- local artifacts and prints the `rote deploy … --target rote-cloud`
- retry). Logged out → today's `dbos` default with a one-line hint,
- never a prompt (CI-safe). The parser's `--runtime` default is now
- `None` — resolution happens at the top of `_cmd_compile`; don't
- reintroduce a static default in the parser.
- - `rote register` + `rote serve` (compiled pipelines as MCP tools,
- FastMCP 4.x, stdio + Streamable HTTP — see
- [`docs/mcp-trigger.md`](docs/mcp-trigger.md))
- - `rote eval` + the auto-emitted `compiled/scorecard.md` (static
- before/after estimate of speed, cost, determinism; live-fetched
- prices — see the eval gotchas below)
- - `rote eval --run` (empirical mode: real trials of both sides —
- `claude -p` for the skill, the emitted python/DBOS app for the
- pipeline with cross-process gate signaling via `DBOSClient`; real
- judge usage captured through the emitted `$ROTE_USAGE_LOG` hook;
- measurements appended to `~/.local/share/rote/eval-corpus.jsonl`)
+ - **System-1 is the default compile path.** Before the agent runs,
+ [`src/rote/compiler/system1/`](src/rote/compiler/system1/) decides
+ the whole structure with rules (kinds, data flow, fan_out, mandatory,
+ gates, constants), and the agent gets the smaller
+ `skills/rote-compile/SYSTEM1.md` rubric and writes only text. rote
+ restores every decision after each agent stop. The full rubric above
+ runs under `--no-system1` and for `--update`. `Compiler(system1=None)`
+ means exactly that rule, and rote-cloud constructs `Compiler` with the
+ default, so flipping it there changes cloud compiles too. Who decides
+ kinds and gates is the layered `decider` setting: `rules` (built-in, no
+ call), or the `clef` / `jev` decision models through one Workers AI
+ client (`system1/decision_model.py`), each with thresholds fitted on the
+ dev skills and frozen before a held-out run. Steps still below 0.7 after
+ that go to Haiku in one schema-locked call (`system1/tiebreak.py`,
+ layered `tiebreak` setting, default `none` because it did not beat
+ rules-only fidelity). It bills through the
+ compile's own lane via `structured_call` in the inference helper. Don't
+ give it its own client or a temperature. See
+ [`docs/design/system1-compile.md`](docs/design/system1-compile.md).
+ - **System-1's text is written in parallel** when the driver has
+ `generate()` (`claude`, `api`): one schema-locked call per node's
+ interface entry, then one per `pure_function` (module + tests) and per
+ judge (prompt), all at once under `--max-parallel`, each checked by code
+ and re-asked alone ([`src/rote/compiler/system1/generate.py`](src/rote/compiler/system1/generate.py),
+ rubric `skills/rote-compile/GENERATE.md`). `codex` and `openai-api` keep
+ the single SYSTEM1.md session. See
+ [`docs/design/parallel-generation.md`](docs/design/parallel-generation.md).
+
+ **When you edit this layer:** you are changing how the agent
+ behaves. Changes are testable by running `rote compile` end-to-end
+ against `examples/bdr-outreach/skill` and diffing the output
+ against the previously committed snapshot in
+ `examples/bdr-outreach/runs/`.
+
+ ### Layer 2 — The driver layer (the "runtime for the agent")
+
+ - **Lives at:** [`src/rote/compiler/drivers/`](src/rote/compiler/drivers/)
+ - **Protocol:** every driver implements `CompilerDriver` with
+ `name`, `is_available() -> (bool, str)`, and
+ `async run(skill_dir, compiler_skill_dir, work_dir) -> DriverResult`
+ - **Three concrete drivers:** `ClaudeDriver` (subprocess to `claude -p`),
+ `CodexDriver` (subprocess to `codex exec`), `AnthropicApiDriver`
+ (in-process via the `anthropic` SDK)
+ - **The contract is the filesystem:** every driver writes
+ `work_dir/pipeline.yaml` and returns a `DriverResult` pointing at it.
+ Do not invent new return-shape fields or side-channel comms.
+
+ **When you edit this layer:** you are changing how the agent is
+ invoked, not what it does. Changes are testable by mocking
+ subprocess spawning (see
+ [`tests/test_claude_driver.py`](tests/test_claude_driver.py)) or
+ the Anthropic SDK client (see
+ [`tests/test_anthropic_driver.py`](tests/test_anthropic_driver.py)).
+
+ ### Layer 3 — The runtime adapters (the "code emitter")
+
+ - **Lives at:** [`src/rote/adapters/`](src/rote/adapters/)
+ - **Five adapters today:**
+ - `DbosAdapter`
+ ([`src/rote/adapters/dbos.py`](src/rote/adapters/dbos.py))
+ — the CLI default; emits a single durable Python app (`main.py`
+ with `@DBOS.workflow`/`@DBOS.step`, plus `signatures/*.py`,
+ `extracted/*.py`, `dbos-config.yaml`, `README.md`) — no
+ orchestrator process, state in SQLite/Postgres
+ - `TemporalAdapter`
+ ([`src/rote/adapters/temporal.py`](src/rote/adapters/temporal.py))
+ — emits Python (`workflow.py` + `activities.py`)
+ - `CloudflareAdapter`
+ ([`src/rote/adapters/cloudflare.py`](src/rote/adapters/cloudflare.py))
+ — emits TypeScript (`src/workflow.ts` extending
+ `WorkflowEntrypoint`, plus `signatures/*.ts`, `extracted/*.ts`,
+ and a `wrangler.jsonc` / `package.json` / `tsconfig.json`)
+ - `PythonAdapter`
+ ([`src/rote/adapters/python.py`](src/rote/adapters/python.py),
+ registered as `python`) — the maximum-legibility target: emits a
+ plain, orchestrator-free Python script (`main.py` with one function
+ per node, visible inline retry loops, stdlib `ThreadPoolExecutor`
+ for parallel waves, plus `extracted/`, `signatures/`,
+ `requirements.txt`). **Refuses pipelines with `hitl_gate` nodes at
+ emit time** (via the derived `Pipeline.requires_durable_execution`
+ property) — a plain script cannot durably park for human approval;
+ the error points at `--runtime dbos`. Shares the Pydantic
+ signature / extracted-stub emitters with the DBOS adapter via
+ [`src/rote/adapters/_py_common.py`](src/rote/adapters/_py_common.py).
+ - `DbosTsAdapter`
+ ([`src/rote/adapters/dbos_ts.py`](src/rote/adapters/dbos_ts.py),
+ registered as `dbos-ts`) — emits TypeScript for DBOS Transact
+ (`src/main.ts` with `DBOS.registerWorkflow` / `DBOS.registerStep`
+ function wrappers, plus `signatures/*.ts`, `extracted/*.ts`, and
+ `package.json` / `tsconfig.json` / `dbos-config.yaml`). Shares the
+ TS emission machinery in
+ [`src/rote/adapters/_ts_common.py`](src/rote/adapters/_ts_common.py)
+ with the Cloudflare adapter. Note: the DBOS **TS** SDK is
+ Postgres-only (no SQLite parity with DBOS Python), and its
+ `DBOS.recv` defaults to a 60s timeout — the emitter always passes
+ the IR timeout explicitly. Targets DBOS 3 (Python) / Transact v5
+ (TS); see the DBOS 3 queue gotcha below.
+ - `InngestAdapter`
+ ([`src/rote/adapters/inngest.py`](src/rote/adapters/inngest.py),
+ registered as `inngest`) — emits a TypeScript Inngest app (one
+ durable `inngest.createFunction` in `src/inngest/pipeline.ts`
+ running the DAG waves, `step.waitForEvent` per HITL gate on a
+ `<pipeline>/<signal>` event, plus a framework-neutral
+ `inngest/node` serve entrypoint and a README documenting the
+ Next.js mount). Shares `_ts_common.py` with the other TS
+ adapters. Note: Inngest v4 retries are **function-level only** —
+ the emitter maps max-across-nodes and comments the per-node
+ deltas (see the Inngest gotcha below).
+ - **What they do:** consume a validated `Pipeline` IR and write
+ runtime-native code into an output directory
+ - **Never run an agent loop.** Code emission is pure template
+ substitution. The agent's job ends when the IR is valid; the
+ adapter's job begins there.
+
+ **When you edit this layer:** you are changing the output code
+ format or adding a new runtime. Changes are testable with unit
+ tests covering emission (AST / textual invariants), plus a real
+ runtime smoke test — `tests/test_temporal_e2e.py` for Temporal
+ (time-skipping `WorkflowEnvironment`); `tests/test_cloudflare_e2e.py`
+ for Cloudflare (runs `npm install` + `tsc --noEmit` on the emitted
+ output, gated by `@pytest.mark.slow`); `tests/test_dbos_ts_e2e.py`
+ for DBOS TypeScript (`npm install` + `tsc --noEmit`, plus a live run
+ on the real DBOS TS runtime against a Docker Postgres);
+ `tests/test_inngest_e2e.py` for Inngest (`npm install` +
+ `tsc --noEmit`, plus a live run through both HITL gates against the
+ real Inngest dev server, `inngest-cli dev`).
+
+ ---
+
+ ## The IR is the source of truth
+
+ [`src/rote/ir.py`](src/rote/ir.py) defines the Pydantic models that
+ every layer agrees on. If you're uncertain about a field's semantics,
+ read that file — it's ~290 lines, fully typed, and the authoritative
+ reference (not the rubric's `ir-schema.md`, which is agent-facing
+ prose that can drift).
+
+ **The five node kinds:**
+
+ | Kind | Required field | Where the LLM lives |
+ | --- | --- | --- |
+ | `pure_function` | `impl` | Not involved |
+ | `external_call` | `impl` | Not involved |
+ | `llm_judge` | `signature` (legacy path) and/or `signature_spec` (structured) | Typed signature (DSPy / BAML in Python; Zod + vendor SDK in TS) |
+ | `agent_loop` | `tools` | Bounded agent loop |
+ | `hitl_gate` | `signal` | Durable suspend/resume |
+
+ **The `signature_spec` model** ([`src/rote/ir.py`](src/rote/ir.py))
+ carries a JSON Schema for input + output, a Jinja-style prompt
+ template, and the LLM client config (anthropic / openai). It's the
+ runtime-agnostic alternative to the legacy `signature: 'path/to/file.py:Class'`
+ form, which only works for Python adapters. Cloudflare requires
+ `signature_spec`; Temporal accepts either and prefers `signature_spec`
+ when present.
+
+ **The canonical skill that covers all 5 kinds:** the BDR outreach
+ example in `examples/bdr-outreach/skill/`. If you need an IR that
+ exercises every code path, its hand-drafted baseline is at
+ `examples/bdr-outreach/expected/pipeline.yaml`.
+
+ ---
+
+ ## Gotchas (things you will absolutely trip on if you don't read this)
+
+ ### `claude -p` and the `ANTHROPIC_API_KEY` trap
+
+ Claude Code's print mode (`claude -p`) has a documented behavior
+ where `ANTHROPIC_API_KEY` **always wins** over any active OAuth
+ session. If your system has the env var set, `claude -p` will use
+ API-key billing even though the user expected subscription auth.
+
+ **The fix:** `ClaudeDriver._build_child_env()` scrubs
+ `ANTHROPIC_API_KEY` and `ANTHROPIC_AUTH_TOKEN` from the child
+ environment before spawning. **Do not remove this.** If you need
+ API-key auth to Anthropic, use `AnthropicApiDriver` instead — it's
+ a separate driver precisely so the Claude driver can force
+ subscription auth.
+
+ ### YAML 1.1 parses `on:` as boolean `True`
+
+ PyYAML follows YAML 1.1, which has historical bool keys (`on`,
+ `off`, `yes`, `no`). A field literally named `on` in a YAML file
+ becomes the Python value `True`, not the string `"on"`. This
+ **broke the first IR test run** when I tried to use `on:` for
+ retry trigger conditions.
+
+ **The fix:** the IR uses `retry_on:` (with the `_on` suffix), not
+ `on:`. Propagate this convention if you add new fields.
+
+ ### Loop body sub-nodes are excluded from top-level waves
+
+ An `agent_loop` node can declare a `loop_body: [node_id, ...]`
+ listing sub-nodes invoked per iteration. Those sub-nodes also
+ exist as top-level entries in the `nodes:` list (so they can be
+ tested in isolation) but `Pipeline.execution_waves()` (in `rote.ir`;
+ the adapters' `_execution_waves` just calls it) filters them out of
+ the top-level DAG, otherwise the workflow would dispatch them twice.
+
+ ### Valid IR is emittable IR: reachability lives in `rote.ir`
+
+ A node's `inputs:` may only name a node in an earlier wave, never a
+ same-wave sibling and never a `loop_body` sub-node (it has no top-level
+ result). That rule used to be checked only inside each adapter at emit
+ time, after `rote compile` had verified the deliverable as clean: a real
+ bdr-outreach compile paid $4.46 for an IR whose `email_template_loop`
+ bound its own body sub-node `personalize_email` as an input, then exited
+ 1 at emit. It is now `Pipeline._validate_input_reachability`, so the
+ compile's repair loop sends it back like any validation error, and the
+ six adapters' copies are gone. Cycles are a validation error for the
+ same reason. Loop-body sub-nodes' own `inputs:` are exempt (they
+ describe, they do not bind).
+
+ For the rules no validator states (python refusing a `hitl_gate`, an
+ MCP-incapable adapter refusing an mcp-only node, an emitter crash),
+ verification dry-emits the deliverable for the compile's runtime
+ (`verify.EmitTarget` / `emit_findings`, wired from `--runtime`), and an
+ `emit_refused` finding goes back to the agent. Don't add an emit-time
+ check to an adapter when it can be IR validation; when it can't, the
+ dry emit already covers it.
+
+ ### Driver factories accept `**kwargs`
+
+ The driver registry's factories have signature `**kwargs ->
+ CompilerDriver` because `Compiler(model=...)` needs to pass
+ kwargs through at construction time. Drivers that don't care about
+ a particular kwarg should swallow it via `**kwargs` in their
+ `__init__`, not hard-fail. **Do not change the factory signature
+ to typed keyword args** — that breaks the registry's polymorphism.
+
+ ### Emitted code touches MCP only through an explicit `mcp:` binding
+
+ Two regimes, don't mix them up:
+
+ - **Nodes without an `mcp:` binding must never reference MCP** in any
+ adapter or language — no `mcp` imports, no `@mcp.tool()` decorators,
+ no MCP SDK dependencies. Enforced per adapter:
+ Temporal — `tests/test_temporal_adapter.py::test_emitted_activities_never_reference_mcp`
+ (Python AST walk); Cloudflare —
+ `tests/test_cloudflare_adapter.py::test_emitted_files_never_reference_mcp`
+ (every `.ts` file, comments + string literals stripped). Copy the
+ matching test when adding an adapter.
+ - **Nodes WITH an `mcp:` binding** (DBOS adapter, `--backend mcp`, the
+ default) emit a *working* FastMCP call: the step goes through the
+ emitted `extracted/_rote_mcp.py` helper — the verbatim source of
+ `rote.mcp._runtime_helper` (never hand-edit the emitted copy; fix
+ the module) — which resolves the endpoint (env > rote registry > IR)
+ and credentials (`rote mcp login` token store > registry static
+ headers > unauthenticated) at runtime. On DBOS Python AND DBOS-TS,
+ auth failures **park the workflow durably** instead of failing it
+ (`RoteMcpAuthNeeded` → `DBOS.recv` on `rote:auth:<server>`;
+ `rote mcp login` — or `rote mcp release` without a login — discovers
+ parked workflows via the app registry + the `rote_auth_status` event
+ and releases them, one Python code path for both runtimes). Traps if
+ you touch this: a parked DBOS workflow is just PENDING (no WAITING
+ status — discovery *requires* the advertised event); parallel-wave
+ siblings can surface stale auth failures after the release signal was
+ already consumed — hence the retry-once-before-parking shape in the
+ emitted wrappers; and the cross-language leg only works in DBOS's
+ *portable* serialization — the emitted TS `setEvent` passes
+ `{ serializationType: "portable" }` and the Python release `send`
+ passes `WorkflowSerializationFormat.PORTABLE` (the defaults,
+ superjson and pickle, are mutually unreadable). In TS, detect the
+ auth error by `name` string, never `instanceof` (serialized errors
+ reconstruct as plain `Error`s). See
+ [`docs/mcp-client.md`](docs/mcp-client.md).
+
+ ### Sync Python runtimes run async work on ONE background loop
+
+ python and dbos steps are plain functions, so every coroutine they need
+ (an `async def` extracted function, a dbos MCP step, a legacy async
+ judge) goes through the emitted `_run_async`, which submits it to one
+ daemon-thread event loop per process. Do not "simplify" it back to
+ `asyncio.run`: that makes and closes a loop per call, so a module-level
+ `httpx.AsyncClient` fails from its second call on ("Event loop is
+ closed"), which is every fan_out element and every retry; and it raises
+ inside a running loop, where `run_pipeline` lands when async code
+ imports it. `_call_impl` comes in two forms from one place
+ (`_py_common.call_impl_helper`, `is_async=True` for temporal), and both
+ check the *result* for awaitability, so a plain wrapper returning a
+ coroutine works on every runtime (`tests/test_call_impl_helper.py`).
+
+ ### A missing MCP endpoint is configuration: never retried, never parked
+
+ `resolve_url` / `resolveUrl` raise `RoteMcpConfigError` (Python and all
+ three TS helpers, detected by `name` in TS like the auth error) when no
+ endpoint resolves. Every runtime's retry opt-out excludes it next to the
+ auth error, so a step fails on its first attempt with a message naming
+ `ROTE_MCP_<SERVER>_URL`. Agent loops reach the same gap through tool
+ discovery, so `bindAgentTools` / `_mcp_config_for` raise the config error
+ when it explains a missing tool, and loop steps carry a name-based opt-out
+ (by name so `main` never imports MCP code, invariant #3). It must not join
+ the park path: a parked run
+ waits for a login, and no login supplies an endpoint. Before this, DBOS
+ spent 4 attempts and surfaced only `DBOSMaxStepRetriesExceeded`
+ (`tests/test_mcp_config_error_e2e.py`). On Cloudflare local dev the
+ instance status then reads only `WorkflowFatalError`; the real message is
+ in workerd's console output, which `rote.runners.cloudflare` reads back
+ from `wrangler-dev.log`.
+
+ ### Never return fastmcp's `result.data` from an MCP step
+
+ `_result_payload` in `rote.mcp._runtime_helper` must NOT be
+ "simplified" to `return result.data`. Re-verified on fastmcp 4.1.1
+ against real TS-SDK servers (v1 and v2) and five hosted servers, it
+ fails both ways:
+
+ - **None for text-only results.** fastmcp hydrates `.data` only from
+ structured content. FastMCP servers auto-generate output schemas
+ (which is why FastMCP fixtures never caught this), but real-world
+ servers, the TS SDK in particular, return JSON as text and `.data` is
+ silently None. Found live on the first real-server run: every MCP
+ step returned None.
+ - **Unpicklable for typed results.** With an output schema, `.data` is
+ a generated `types.Root` class: DBOS cannot pickle it, and keys that
+ are not identifiers come back renamed (`display-name` →
+ `display_name`). Found live against `docs.mcp.cloudflare.com`.
+
+ The payload is `structured_content` (the wire value, with fastmcp's
+ `{"result": ...}` wrapper removed only when the server marked it), else
+ the parsed text blocks. Two things to keep: the marker is
+ `_meta.fastmcp.wrap_result` on current fastmcp but only the
+ `x-fastmcp-wrap-result` output-schema flag on 2.x/3.0 servers, so the
+ schema is still needed; and the TS helpers apply the same rule through
+ `_FASTMCP_UNWRAP_TS` (one source spliced into the Node and Workers
+ helpers). The text fallback is shared the same way: `_MCP_TEXT_PAYLOAD_TS`
+ (spliced into the Node, Workers and binding helpers) applies
+ `_result_payload`'s exact text rules, which are every text block parsed,
+ non-JSON kept verbatim, several blocks as a list, and none as null. The
+ TS helpers used to read only the first block, throw on non-JSON text and
+ return `{}` for no text, so the same MCP step returned different values
+ per runtime (`tests/test_mcp_text_payload_parity_e2e.py` runs both
+ helpers against one real server). The helper's explicit `list_tools()` is not an extra round
+ trip, because the MCP SDK's `call_tool` lists tools itself on a fresh
+ session to validate the result (measured, and pinned by
+ `test_one_tools_list_per_step`). Regressions: `tests/test_mcp_client.py::
+ test_result_payload_is_plain_json_from_a_real_fastmcp_server` and
+ `tests/test_mcp_ts_server_e2e.py` (real TS-SDK servers, slow).
+
+ ### fastmcp 4 hides HTTP status codes; the helper hooks the transport
+
+ MCP SDK v2 turns any non-2xx answer into a bare `MCPError` (-32603,
+ "Server returned an error response") with no cause chain, and the HTTP
+ stack is httpx2, not httpx. So a 401 can no longer be found by walking
+ the exception for `httpx.HTTPStatusError`; that check silently matched
+ nothing and park-on-auth stopped working. `_HttpStatusWatch` builds the
+ transport's httpx2 client with a response hook and remembers whether
+ the latest answer was a 401. Regression: `tests/test_mcp_client.py::
+ test_call_mcp_tool_maps_a_real_401_to_auth_needed` (a real localhost
+ 401, with a 500 as the negative half). Any new `except httpx.` around a
+ fastmcp call is dead code for the same reason.
+
+ Two smaller fastmcp 4 traps: `client.ping()` raises "Method not found"
+ on the sessionless 2026-07-28 era, which `Client(url)` negotiates by
+ default (use an authenticated `list_tools()` to trigger OAuth), and an
+ SDK v2 `ServerSession` is per request, so anything that remembers a
+ client across requests must hold the `Connection` instead (see
+ `rote serve`'s list_changed delivery, which also has to publish on a
+ `subscriptions/listen` bus for modern clients).
+
+ ### MCP 2026-07-28: what keeps both eras and every registration mode working
+
+ Seven things, each with a test that fails when it is undone
+ (`tests/test_mcp_era_matrix_e2e.py`, `tests/test_mcp_oauth_registration_e2e.py`,
+ `tests/test_mcp_park_e2e.py`). See [`docs/mcp-client.md`](docs/mcp-client.md).
+
+ - **The TS helpers must pass `versionNegotiation: { mode: "auto" }`.**
+ `@modelcontextprotocol/client` 2.x defaults to `legacy`, which fails
+ against a modern-only server exactly like v1 (`-32022`). Checking the
+ import line proves nothing; the era matrix asserts the era each server
+ saw. A 401 is `SdkHttpError.status`; v1's `err.code === 401` reads an
+ SdkErrorCode string in v2 and never matches, which silently turns
+ park-on-auth into a failure.
+ - **The CIMD document's redirect URIs and `CALLBACK_PORTS` are one fact.**
+ A CIMD server compares redirect URIs by exact string match and no spec
+ defines a loopback port wildcard, so the login binds a published port.
+ `tests/test_mcp_client.py::test_cimd_document_matches_what_the_login_offers`
+ pins `docs/oauth/rote-cli.json` to the code, and the copy served by
+ rote-cloud's marketing site must match it too. Add ports to the
+ document before the CLI uses them, never after. Selecting a port must
+ bind it (`bind_callback_sockets`), and the callback server must run on
+ those sockets: a check now and a bind later lost the port to a
+ concurrent login in CI after the redirect URI already named it.
+ - **rote refuses registrations the SDK would send on its own.** Without
+ a `registration_endpoint` the SDK POSTs a guessed
+ `<resource-origin>/register` (GitHub: not its authorization server).
+ `RoteOAuth._refuse_registration` stops that, the emitted helper refuses
+ any registration (it parks instead), and an issuer-bound pre-registered
+ client that the SDK dropped is not silently replaced by CIMD or DCR.
+ The tests assert on the authorization server's hit log, because the
+ failure is a request that should never have been sent.
+ - **Refresh goes to the stored `token_endpoint`.** Before discovery the
+ SDK refreshes at `<resource-origin>/token`. For a path issuer or an AS
+ on another host that 404s, and an unattended run parks on a credential
+ that was fine. `retarget_token_request` is shared by the CLI and the
+ emitted helper. FastMCP fixtures cannot catch this: their AS is the
+ server's own origin root.
+ - **Never offer the CIMD document unverified.** The SDK does not fall back
+ to DCR when a CIMD server rejects it, so an unpublished or stale
+ document breaks logins that DCR serves. `_check_offered_document`
+ fetches it once per process when the AS advertises CIMD and withdraws
+ the URL unless it equals the packaged `client_metadata.json`.
+ - **A token file is bound to an endpoint and an issuer.** Files are keyed
+ by server name, so `stored_for_url` gates every read (else a repointed
+ name gets the old bearer token), and `_with_bound_issuer` fills
+ `client_info.issuer` from the file (the SDK treats a missing one as
+ matching every issuer).
+ - **`RoteMcpInputRequired` is not an auth error.** A server that wants
+ input mid-call (`input_required`, or `-32021` for an undeclared
+ capability; TS SDK servers send the latter because rote declares no
+ capabilities) fails the step on every runtime, and every retry opt-out
+ exempts it. Making it a `RoteMcpAuthNeeded` would park the run for 30
+ days waiting for a login that answers nothing.
+
+ ### `mcp.args` literals are values, never code
+
+ A real BDR compile wrote fixed tool arguments ('management-levels', 'BDR
+ do not contact') into `mcp.args`, which held only references, and the IR
+ rejected them. `{literal: <str|int|float|bool>}` (`rote.ir.MCPLiteral`)
+ is the fixed-value form. It is a mapping so it can never be confused with
+ a reference: a bare string is always a payload key or a constant name.
+ Strict types, so `"3"` stays a string. No `null` (omit the argument) and
+ no lists or objects (a constant, named by reference, already carries
+ those).
+
+ `mcp_tool_arguments` returns `(arg, "literal", value)` and every emitter
+ renders it exactly as it renders a constant: `json.dumps` in TS,
+ `_py_literal` in Python (invariant #7). Two traps the malicious-string
+ parity test (`tests/test_mcp_args_literal_parity*.py`, the four MCP
+ runtimes) found:
+
+ - `json.dumps` escapes a character outside the BMP as a surrogate pair,
+ which JS reads correctly and Python reads as two lone surrogates. Python
+ string literals go through `py_str_literal`: raw non-ASCII, but every
+ character in Unicode category Cc, Cf, Zl, Zp or Cs escaped
+ (`escape_invisible`). Raw, a bidi override, zero-width space or BOM is
+ the Trojan Source shape (CVE-2021-42574), and a lone surrogate (PyYAML
+ makes one from `"\ud800"`) cannot be written as UTF-8 at all. The same
+ rule covers docstrings (`safe_docstring_line`), TS comments
+ (`safe_block_comment_line`) and the Cloudflare README; TS literals are
+ ASCII `json.dumps`, already safe. `_pipeline_hash` falls back to an
+ escaped encoding only when the JSON encoder refuses a lone surrogate, so
+ no existing workflow hash changes. Regression:
+ `tests/test_emitted_source_is_visible.py`.
+ - `JSON.stringify` leaves those three raw, so the DBOS-TS runner's
+ trailing-JSON parser splits on `\n` only, or a valid result is cut
+ apart.
+
+ python and temporal have no MCP backend and refuse an MCP-bound node
+ without an `impl`, so `mcp.args` never reaches them.
+
+ ### A compiled contract is decided by data, not by the model's reading of prose
+
+ A skill run as an agent adapts to whatever input arrives. A compiled
+ pipeline has fixed field names. Measured with `rote bench --fidelity`:
+ every one of 36 expense-audit compiles failed on the skill's own input,
+ because the skill describes it in prose and each compile invented names
+ (`expense_lines`, `lines`, `reporting_period`), invented required inputs
+ the skill never takes (`fx_rates`, though the skill reads
+ references/fx-rates.md; `controller`), and indexed rows by
+ `line["expense_id"]` where the data says `id`.
+
+ `rote.compiler.grounding` decides the contract with code wherever data
+ exists: a sample input (`--sample-input` > `input.json` in the skill dir >
+ the input a baseline ran on) is reduced to a schema by
+ `rote.probe.infer_schema` and handed to the agent as an `INPUT
+ CONTRACT`. Verification then checks it, and `rote.compiler.prefix` runs
+ the pipeline's pure functions (and MCP nodes a baseline recorded, matched
+ on exact arguments) on the sample in the generated-test sandbox. A
+ `KeyError` there goes back to the agent with the file and line. MCP tool
+ arguments are grounded the same way, in each tool's own input schema:
+ a live `tools/list` of the registered servers the skill names, else the
+ skill's bundled snapshot (`mcp-tools.json` in the skill dir,
+ `rote.tool_schema.load_bundled_tools`). Live always wins per server; a
+ live schema that differs from the snapshot for a bound tool is reported
+ as a stale snapshot, and the summary and compile-report.md name what
+ grounded each server. Things to preserve:
+
+ - **The prefix runner is a dumb interpreter.** `_prefix_runner.py` is
+ stdlib-only and copied into the sandbox scratch dir. The semantics
+ (payload from `inputs:`, omitted optional fields left out, constants
+ merged last, `fan_out_element_param`, `mcp_tool_arguments`) live in
+ `prefix.build_plan`, which reuses the adapters' own helpers. If you
+ change how an emitted step builds its payload, the plan must follow,
+ or the prefix checks a different program than the one that runs.
+ - **Never guess a tool result.** An MCP node with no recorded call, or
+ a recorded call with other arguments, ends the prefix on its branch
+ and becomes an open question (`contract_questions`), not a fixture.
+ - **Generated tests cannot catch this.** The model that wrote
+ `line["expense_id"]` writes tests with `expense_id` rows. The sample is
+ the only input in the loop the agent did not write.
+ - **Tool argument names come from the tool, not the skill.**
+ `rote.compiler.tool_grounding` lists the tools (`tools/list`, read-only,
+ through the emitted pipelines' own non-interactive client, all servers
+ concurrently) of the registered servers the skill names, before the
+ agent runs. Only those: a third-party skill must not trigger
+ authenticated requests, and silent token refreshes, against the user's
+ other servers (`--tool-discovery all` opts in, `none` opts out on any
+ driver). Schemas are server-supplied text, so the agent sees only
+ identifier-shaped names and JSON type names, labeled as data.
+ Verification checks each `mcp:` binding's arguments against a declared
+ schema; one that is not a flat object schema (top-level `anyOf`,
+ non-local `$ref`, no `properties`) is an open question, never a
+ failure. The schema
+ is read one way everywhere (`rote.tool_schema`): the bench's fixture
+ servers refuse exactly what verification refuses, and the bench
+ registers those fixtures for each compile, so compile and fidelity see
+ one schema. A schema inferred from a baseline's recorded calls never
+ fails a compile: it only shows what someone sent.
+
+ ### `--backend api` is a compile mode, not only an emit flag
+
+ With `--backend api` every external_call runs its `impl:` module, so the
+ agent must write real vendor calls, and a call from training-data memory
+ fails at run time. The Compiler owns this (`rote.compiler.api_docs`), on
+ both the legacy and the System-1 path:
+
+ - **The backend reaches the Compiler as `emit_target.external_backend`.**
+ The Compiler, not the CLI, grants the driver `web_tools` and adds the
+ brief to the prompt, so any caller that compiles for `api` gets both.
+ Claude: WebSearch/WebFetch. api: the Messages API's server-side
+ `web_search_20250305` / `web_fetch_20250910`, and NOT the newer
+ versions: those provision code execution, and a real compile spent
+ about $14 of extra input tokens running code in it. Resend a
+ `pause_turn` as-is; it is not a stop. Codex: `-c web_search="live"`
+ plus `--json` (`codex exec` has no `--search`). openai-api: no web
+ tool exists on chat/completions, so it reports itself unavailable
+ rather than compiling from memory.
+ - **Citations are checked, and fetches are the evidence.** Each vendor
+ function carries `# docs: <url>` (or `# docs: skill` for an internal API
+ the skill itself documents). Verification sends back an implemented
+ external_call that cites none, and, when the driver reports fetches, a
+ cited page the agent never fetched. The report marks each cited page as
+ fetched in this compile or not. The fetched list comes from the drivers'
+ `tool` events (their `url` / `query` fields), so a driver that does not
+ put them there makes the report say "unknown".
+ - **TypeScript runtimes are refused at compile.** They emit a stub for
+ every external_call, so `compile --backend api --runtime cloudflare`
+ (including the logged-in default) exits 2 before the agent runs. The
+ Compiler refuses it too, for callers that skip the CLI (the cloud
+ container, the bench).
+ - **The backend is recorded beside the IR, never in it.** A compile writes
+ `backend.json` next to `pipeline.yaml` (`rote.compiler.backend`).
+ `rote emit` and `compile --update` use it when `--backend` is not
+ passed, and warn when the flag contradicts it, because emitting the
+ other backend drops code the compile wrote for this one. python and
+ temporal never call a binding, so they advertise no MCP server for one.
+
+ ### Newer Anthropic web tool versions start a code execution container
+
+ `web_search_20260209`+ and `web_fetch_20260209`+ turn on "dynamic
+ filtering": the API provisions code execution for the request on its own,
+ with no tool in your `tools` array asking for it. Inside the api driver's
+ compile loop the model used that container as a second workspace, 70+
+ `code_execution` / `bash_code_execution` calls inside two requests, billed
+ as 3.69M and 2.91M input tokens ($8.04 + $6.18, measured on the Cloudflare
+ AI Gateway logs). The same skill compiles for about $1 on the claude driver.
+
+ So the api driver pins `web_search_20250305` / `web_fetch_20250910` with
+ `max_uses` caps and `max_content_tokens`, refuses to send any other server
+ tool (`_check_server_tools`), stops the run when a response carries a
+ server tool it did not ask for, and stops when one request bills more than
+ `DEFAULT_MAX_INPUT_TOKENS_PER_REQUEST` input tokens. Do not "upgrade" the
+ versions without a measured run. The ceiling is checked on the response,
+ so it stops the next request, not the one that crossed it.
+
+ ### Fidelity for `--backend api` pipelines fakes the vendor, from its spec
+
+ `rote bench --fidelity` runs an api pipeline against a local fake of each
+ vendor API (`rote.vendor_specs.fake`), built from the vendor's spec and
+ answered from the case's `http:` fixtures (`rote.bench.http_fixtures`).
+ Things that will bite if you touch it:
+
+ - **Routing is a `sitecustomize`** (`rote/vendor_specs/intercept/`), put on
+ the pipeline process's `PYTHONPATH`. It patches httpx, requests,
+ httplib2, urllib and `google.auth.default`, rewrites the faked hosts to
+ the fake, and BLOCKS every other host of a vendor in play
+ (`ROTE_HTTP_FIXTURE_BLOCK`), so a bench run cannot reach a real API. It
+ must stay standard-library-only at module level: it runs in the app's
+ environment, not rote's.
+ - **A fixture is held to the spec at load** (`rote.vendor_specs.fixtures.
+ validate`): a field the response schema lacks is an error, so a fixture
+ can never teach a pipeline a field the real API does not return.
+ - **`reads: <server>.<tool>[i]` is what makes API and MCP reads
+ comparable.** Without it an API read keys by operation and counts as
+ unexpected against the skill's reference.
+ - **Fixture sets are stored keyed by `<vendor>/<api>/<version>/<name>.json`**
+ (`rote.vendor_specs.fixture_store`, `ROTE_FIXTURE_DIR`), validated and
+ stamped with the spec revision on save, re-validated on load. Only
+ authored data goes there: a live check compares shapes and never keeps
+ what it read.
+ - **`rote run --live-check` never runs the deliverable's code.** A live
+ run of generated code holds the user's real credentials, and no patch
+ over its HTTP clients can promise it only reads: review6's reproducers
+ wrote through an unextracted urllib call, a helper module, a host alias
+ and http.client while the intercept guarded the checked call. So
+ `rote.vendor_specs.live` builds each entry read itself from the static
+ extraction (the spec-resolved operation, the URL template, the
+ parameters and headers as `Expr` templates filled from the node's inputs
+ and the user's environment, Google calls as the client library would send
+ them with the application-default token) and sends only GET/HEAD through
+ one urllib client with redirects off. A read it cannot build exactly is
+ "not live-checked", never sent. A docs operation named like a write is
+ refused whatever its method. Three more rules, from review6's exfil
+ reproducer (an agent-written docs param plus `os.environ["AWS_SECRET_..."]`
+ sent the secret): trust comes from the vendor or the user, never the
+ agent. A credential is attached only from a vendor-published spec
+ (Google's OAuth token, which the check supplies itself, gated on the
+ spec's provenance, `source="google-discovery"` from rote's store, and a
+ googleapis.com host, never on the agent-writable `vendor`; a docs spec
+ claiming a vendor rote reads a spec for is refused at compile time); a docs spec's
+ `auth` is the agent's choice of secret, so it is withheld unless the
+ user passes both `--allow-host <host>` and `--allow-credential <ENV>`
+ (`--yes` never covers either), and any other `{$env:...}` refuses the
+ whole check. A docs-spec host is called at all only when it equals, or
+ shares the registrable domain of (`rote.vendor_specs.psl`: the bundled,
+ pinned Public Suffix List with its private section, so `a.github.io` and
+ `b.github.io` are two owners; docs on another registrable domain, such
+ as docs.slack.dev for slack.com, need `--allow-host`), a cited page in
+ `web-research.json`
+ (rote's own record, written by the Compiler over any file the agent
+ wrote), or `--allow-host` names it; "every cited page was fetched"
+ vouches for nothing. Every request is built first and the plan is
+ confirmed (`--yes`, or a TTY answer; neither exits 2 having sent
+ nothing). The report holds the operation's path
+ template and schema paths (a map's keys and any non-field-like live key
+ are `{key}`), and an error is its exception type and HTTP status, never
+ its text (a URL in it carries query credentials). For a docs spec, extra
+ live fields are not drift; for a vendor's own spec they are.
+ - **Vendor libraries are installed per requirement set** into
+ `<spec cache>/../bench-deps/<hash>` with `uv pip install --target`, from
+ a curated import-to-distribution map (`DISTRIBUTIONS`). An unmapped
+ import is an error, never a guessed package name.
+
+ ### Vendor requests are checked against the vendor's spec, in code
+
+ On `--backend api`, verification reads every request an external_call
+ module makes (`rote.vendor_specs.extract`: httpx/requests calls with URL
+ templates rebuilt from literals and constants, and Google client-library
+ `build(...)` chains, including helpers that wrap `build` and resources
+ bound to names) and checks it against the vendor's machine-readable spec
+ (`rote.vendor_specs.check`): operation, parameter names, types, enums,
+ required parameters, body fields, and the response fields the code reads.
+ Violations are errors that go back to the agent. Three rules:
+
+ - **The spec comes from the vendor, never from memory.** Google: the
+ Discovery directory (`https://discovery.googleapis.com/discovery/v1/apis`)
+ names each API version's document. Specs are cached under
+ `ROTE_SPEC_CACHE_DIR` (default `$XDG_CACHE_HOME/rote/specs`, else `~/.cache/rote/specs`) as
+ `<vendor>/<api>/<version>.json`, the `ApiSpec` model, which is also the
+ shareable format. An expired cache is refetched; a failed refetch is
+ `SpecUnavailable`, never a stale read.
+ - **What cannot be checked is listed, not failed.** A URL built at run
+ time, a host with no spec source yet, or a spec service that is down
+ shows in the report as "not checked: <why>". An unreachable spec must
+ never fail a paid compile.
+ - **OpenAPI comes from a curated registry of vendor-published documents**
+ (`rote.vendor_specs.openapi.REGISTRY`, host -> URL), never from a
+ community directory: APIs.guru's Slack entry is an archived 2021 spec.
+ A vendor's own spec outranks a docs spec the compile wrote for the same
+ host (`check.spec_for_key`, the one precedence rule for the check, the
+ live check and the bench), and a docs spec naming a registered vendor is
+ refused outright (`docs.PUBLISHED_VENDORS`). Add a vendor by fetching and parsing its
+ document first, and register every host its `servers` name at any level
+ (Stripe uploads at files.stripe.com, GitHub at uploads.github.com).
+ - **A false error is worse than a missed one**, because the agent rewrites
+ correct code to satisfy it. So a part of a request the spec does not
+ describe is `partial` ("no violations; not checked: <what>"), never
+ "ok" and never a finding: a field under an object that lists no
+ fields, `additionalProperties: true`, an `anyOf` branch that is open, a
+ `$ref` rote cannot follow, a body or response with no JSON schema. An
+ object that lists its fields is closed (the vendors leave
+ `additionalProperties` out), and a field is missing only when no branch
+ of an `anyOf` / `oneOf` has it. The correct-call corpus
+ (`tests/fixtures/vendor_specs/correct_calls/`) must stay at zero
+ findings; add a vendor's real calls there with its fixture.
+ - **Send parameters in the spec's own style.** Stripe's 282 `expand` and
+ 63 `created` parameters are `style: deepObject`: `expand[]=...`,
+ `created[gte]=...` (the spec has no bare `expand=`), and its form bodies use
+ the same brackets (`metadata[order_id]`). GitHub's `x-multi-segment`
+ path parameters become `{+name}` (a file path keeps its slashes). The
+ body's media type is checked too: Stripe takes form bodies, never JSON.
+ - **GraphQL is checked by graphql-core against the vendor's schema**
+ (`rote.vendor_specs.graphql.REGISTRY`, endpoint -> SDL file or the
+ endpoint's own introspection), plus the variables sent (names, and a
+ literal value's type) and the response fields read against the
+ document's selection, inline and named fragments followed. Four traps:
+ - **Build with `assume_valid=True`.** GitHub's public schema fails
+ graphql-core's schema validation (a deprecated field implementing one
+ its interface does not deprecate), and `validate()` raises a
+ `TypeError` on such a schema. The request is still validated in full.
+ - **The operation is chosen as GraphQL's GetOperation does**: one
+ operation, or the one `operationName` names; several and no name is a
+ real request error; a computed `operationName` is not checked.
+ - **A document is read only from a name bound once.** A name rebound,
+ or bound under an `if`, could send either document (a mutation where
+ a query was read), so it is "built at run time", never guessed. The
+ same goes for a body dict.
+ - **A read below a custom scalar is not checked** (AniList's `Json`);
+ below a built-in scalar or an enum it is a finding.
+ A GraphQL read is a POST: the live check sends only a document whose one
+ operation is a query, to a declared endpoint. A GET to an endpoint is not
+ checked.
+ - **Where no spec exists, the agent writes one, from fetched pages.**
+ `vendor_specs/<vendor>.<api>.<version>.json` (`rote.vendor_specs.docs`)
+ becomes an `ApiSpec` with `source="docs"`, checked the same way and
+ labeled lower trust everywhere. Each of its sources must be a page
+ fetched in this compile. A host with no spec of either kind is a
+ `spec_missing` error on the api backend, never a silent "not checked".
+ - **Tests never fetch a spec.** conftest replaces `http_get_json` and
+ points `ROTE_SPEC_CACHE_DIR` at the test's tmp dir. Tests use the trimmed
+ real documents in `tests/fixtures/vendor_specs/` through an injected fetch.
+
+ ### A tool result has one shape: what the step receives
+
+ Four places read an MCP result: the emitted helper at run time, and at
+ compile time the baseline's schema inference (`rote.probe`), the prefix
+ run's recorded stubs (`rote.compiler.prefix`) and System-1's interface
+ (`Context.known_result_schema`). All four go through
+ `_runtime_helper.returned_value` / `returned_schema`, so a fastmcp tool
+ that returns a string (wrapped on the wire as `{"result": ...}` under an
+ `x-fastmcp-wrap-result` output schema, DeepWiki's shape) is a string
+ everywhere. A recording is decoded before its schema is inferred; without
+ that, every text-recorded result was typed `string`. When nothing is
+ recorded, the outputSchema the server declares is the shape, never a
+ guess from the skill's words: a guessed `url` where Microsoft Learn says
+ `contentUrl` made every live run raise. The compile writes the declared
+ schemas to `compiled/declared-tools.json` itself, after the agent's work
+ dir is moved, because the post-compile enrichment and cross-check run in
+ the CLI and must not read an agent-written file.
+
+ ### An mcp-bound node is a BARE tool call — post-processing must split
+
+ The default backend emits an mcp-bound step as exactly "call the tool,
+ return the result". If a skill step is "call X, then filter/match/
+ reshape", the transformation half must be its own `pure_function` node
+ or it silently vanishes at emission (live failure: match-by-name logic
+ lost, workflow crashed dereferencing a declared-but-never-computed
+ key). The rubric (`node-kinds.md`, external_call) mandates the split;
+ `rote.probe.cross_check` flags violations as `output_mismatch` (node
+ declares `output` keys the observed tool result can't provide) and
+ `rote compile` prints them as WARNINGs.
+
+ ### `temperature` is rejected outright by current Claude models
+
+ A judge carrying `temperature` 400s on every call against a current
+ Anthropic model (`invalid_request_error: \`temperature\` is not
+ supported on this model`). The compiler used to write
+ `temperature: 0.0` on judges as a determinism instinct, which made
+ every compiled pipeline dead on arrival the moment `ROTE_MODEL_<ID>`
+ pointed at a current model — found live, first real gateway run.
+
+ Two halves of the fix, keep both: the rubric
+ (`llm-judge-extraction.md`) says never set `temperature` — determinism
+ comes from schema-locked decoding (`output_config.format` /
+ `json_schema`), not the sampler. When the IR *does* set one, emitted judges
+ render it as a `ROTE_TEMPERATURE_<NODE_ID>` env knob whose empty value
+ drops the parameter entirely (`_sampling_kwargs()` in Python,
+ `...(TEMPERATURE.trim() ? … : {})` in TS). Don't "simplify" it back to
+ a literal keyword argument: a baked constant cannot be escaped without
+ a re-emit, which defeats the model-override knob next to it.
+
+ ### First-run output: three contracts that live in one place each
+
+ Found by a fresh-install run of the published wheel, where every one of
+ these read fine to someone who already knew the answer:
+
+ - **Install hints come from `rote.extras.install_hint`.** The PyPI
+ distribution is `rote-cli`; `rote` is somebody else's package, so a
+ hand-typed `pip install rote[api]` installs the wrong thing.
+ `tests/test_first_run_output.py` fails on the bare form, and on a
+ hand-typed `rote-cli[` outside `rote/extras.py`.
+ - **A failed subprocess reports its tail** (`rote.eval.empirical.failure_excerpt`).
+ The exception is at the end; a `[:800]` head slice showed a newcomer
+ DBOS's startup banner cut off mid-word and nothing else.
+ - **`--json` always puts one object on stdout.** `main()` wraps every
+ `--json` handler; a nonzero exit with empty stdout gets
+ `{"ok": false, "exit_code", "error"}` built from the `error:` line the
+ handler printed. New error paths only need the usual
+ `print(f"error: ...", file=sys.stderr)`.
+
+ `rote run` also refuses stubs up front (`rote.runners.preflight_pipeline`,
+ reading the emit manifest via `rote.adapters._common.unimplemented_stubs`),
+ so a stub never burns a durable runtime's retry budget before failing.
+ A stub is a function raising with `STUB_SENTINEL` (`[rote stub]`, in
+ every stub rote emits) or, in Python, one whose whole body is `raise
+ NotImplementedError` (the shape compile agents write), at any depth,
+ methods included, `@abstractmethod` excepted. The marker counts only
+ where it is executable: in Python as a `raise` argument (AST), in TS
+ after `strip_ts_comments`. A raw-text match kept a filled-in module that
+ merely mentions the marker in a comment listed, and `rote run` refused
+ it. Never go back to a `NotImplementedError` substring match either: a
+ real module that raises it on an unsupported branch was refused with no
+ way past it. `--allow-stubs` is the escape hatch.
+
+ ### Current Sonnet rejects forced `tool_choice`
+
+ `claude-sonnet-5-5` answers every request with `tool_choice` type
+ `"tool"` or `"any"` with HTTP 400: `tool_choice: type "tool" and "any"
+ are not supported for this model.` (verified live 2026-10-20;
+ claude-haiku-5-5 and claude-sonnet-4-6 still accepted it, and the
+ Cloudflare AI Gateway `/ai/v1/messages` route passes the same 400
+ through). Every emitted judge used to force a tool call, so pointing
+ `ROTE_MODEL_<ID>` at the current Sonnet killed every judge call: the
+ same dead-on-arrival class as the `temperature` gotcha above.
+
+ Schema-locked decoding is now `output_config: {format: {type:
+ "json_schema", schema}}` on every Anthropic lane, for every model, with
+ no branch on model ids: the Python helper's `_call_anthropic`, the TS
+ judge emitter, and both agent loops' one extra turn (with `tool_choice:
+ none`, because the history holds `tool_use` blocks). Three things to
+ keep:
+
+ - **The request schema is rewritten, in one place.**
+ `structured_output_schema` in `rote.inference._runtime_helper` closes
+ every object (`additionalProperties` omitted or `true` is a 400),
+ turns `oneOf` into `anyOf`, and moves bounds, `pattern`, `maxItems`,
+ `minItems` above 1 and unknown `format`s into the description (all
+ 400s as written). The judge's own validator, built from the original
+ schema, still enforces them. Python runs it per request; TS runs the
+ same function at emit time. `tests/_messages_api.py` is an
+ independent copy of the measured rules, and the Messages stubs refuse
+ what it refuses, so the suite fails where the API would.
+ - **Two shapes cannot be expressed, and are never approximated.** An
+ object that admits undeclared keys (no declared properties, or
+ `additionalProperties` set to `true` or a schema, as in a fixed key
+ plus a typed map) and a value with no type. Only an object that *omits*
+ `additionalProperties` is closed. Closing a map would make the model
+ drop its keys with nothing noticing.
+ An anthropic judge with either fails at `rote emit`, naming the path.
+ An agent loop's output often has one (an IR type name the IR never
+ defines), so its extra turn falls back to asking for `submit_result`.
+ That fallback keys on `STRUCTURED_INEXPRESSIBLE` in the message, so any
+ other normalizer error still propagates.
+ - **The SDK floor is load-bearing.** `anthropic>=1.15` (has
+ `output_config`) reaches every Python runtime's install instructions
+ through `_py_common.vendor_sdk_requirements`; an older SDK fails every
+ judge with an unexpected-keyword error.
+ - **Only `end_turn` is an answer.** `refusal` and `max_tokens` raise a
+ plain RuntimeError naming the stop reason; a truncated JSON prefix
+ must never reach a downstream node.
+
+ The `claude-cli` lane (`claude -p --json-schema`) and the OpenAI lane
+ are separate mechanisms and were unaffected (the CLI handles
+ claude-sonnet-5-5 itself).
+
+ Effort rides in the same `output_config`. Unset, the API runs `high`.
+ `ROTE_EFFORT_<NODE_ID>` > `ROTE_JUDGE_EFFORT` set it for judges and agent
+ loops on every lane, and an empty value omits it (the temperature knob's
+ shape). On the `claude-cli` lane it is `claude -p --effort`, added by one
+ function (`_cli_effort_flags`) to both the judge and the agent-loop
+ command lines. The CLI does not fail on an unknown level: it warns on
+ stderr and runs at its default, exit 0. So before any paid call the
+ helper asks the CLI itself with `claude --effort <level> --version`
+ (free, ~50ms, validated before `--version` exits), once per process:
+ first with a level nothing accepts, which must draw a warning rote
+ recognizes (any line naming effort and saying unknown / invalid /
+ unsupported), then with the real level, which must not. Never hardcode
+ the level list in the helper; it goes stale. If a CLI release rewords
+ the warning past recognition, the self-check fails loudly instead of
+ letting every bad level through. Tested against a stub CLI
+ (`tests/test_cli_effort_check.py`). The default stays unset on
+ data: on 2026-10-20, 8 example judges x 6 inputs, `low` cut output tokens
+ 15% (sonnet-4-6) and 33% (sonnet-5-5) but changed decisions the default
+ made the same way twice, including dropping an ARR stated in the input.
+ Don't name the pipeline-wide knob `ROTE_EFFORT`: that is the compiler's,
+ and it would silently retune every judge a developer runs. On the Node
+ TS runtimes a judge only sees the env vars `judge_env_arg` threads, so a
+ new knob must be added there too (the temperature knob was missing until
+ this was found).
+
+ ### Vendor SDK errors do not survive a durable step boundary
+
+ `anthropic.APIError` / `openai.APIError` subclasses take keyword-only
+ `response`/`body` arguments, so `BaseException.__reduce__`'s
+ `(cls, args)` cannot rebuild them. A runtime that persists a failed
+ step by pickling the exception — DBOS does — therefore replaces the
+ real status and body with `TypeError: APIStatusError.__init__()
+ missing 2 required keyword-only arguments` at the join. That masked
+ the temperature 400 above completely.
+
+ Emitted judges wrap the SDK call and re-raise through
+ `_durable_error()`, a plain `RuntimeError` carrying type, HTTP status,
+ model, node id, and the original message (regression:
+ `tests/test_dbos_adapter.py::test_judge_translates_sdk_errors_so_they_survive_a_durable_step`,
+ which asserts the translated error actually pickle-round-trips). This
+ bites hardest on the fan_out enqueue path, where the failure is
+ observed in a *different* process from where it was raised — any
+ durability boundary is also an error-fidelity boundary.
+
+ ### The TS tool runner's `done()` deadlocks; use `runUntilDone()`
+
+ `client.beta.messages.toolRunner(...)` returns an async iterator plus two
+ ways to wait on it, and they are not interchangeable. `done()` *observes*
+ a loop somebody else drives — its own docs say it "works even if the async
+ iterator hasn't yet started, and will wait for an instance to start" — so
+ a single-consumer caller that never iterates hangs forever, silently, with
+ no output and no error. `runUntilDone()` consumes the iterator itself.
+ The Python twin's `until_done()` has no such split, which is exactly why
+ this is easy to port wrong. Found by running it (rc=124, zero output);
+ tsc and the SDK's types are both happy with the deadlocking version.
+
+ Related, same file: report the turns that actually ran, not the cap.
+ The count is assistant messages
+ (`runner.params.messages.filter(m => m.role === "assistant").length`),
+ `number | null`, and goes to stderr (TS) or the usage log's `iterations`
+ field (Python), since a loop's result is now its declared output. The
+ Workers AI lane reports null because `runWithTools` genuinely does not
+ report it. Echoing back `maxIterations` made every loop look saturated. The Python
+ `_run_agent_via_sdk` has no public `params` accessor, so it drives the
+ runner's iterator itself and counts each yielded message, summing usage
+ over every turn. Then `until_done()` on the drained runner returns the
+ final turn without another request.
+
+ ### An agent_loop's tools carry no server — `tool_servers` supplies it
+
+ `Node.tools` are bare names (`zoominfo_search_contacts`): one loop may
+ reach several servers, and an `MCPBinding` is a single server/tool pair.
+ That gap was not cosmetic. `Pipeline.required_mcp_servers` derives from
+ server *names*, so a pipeline whose only MCP usage is an agent loop
+ reported **zero** required servers — `analyze`/`emit`/`compile` and the
+ `rote mcp login` advisories all said there was nothing to authenticate.
+ BDR is exactly that shape, so the flagship example demonstrated the bug.
+
+ `Node.tool_servers` (tool name → server, agent_loop only, keys validated
+ ⊆ `tools`) closes it. Three things now depend on it:
+
+ - **The advisories are correct.** `required_mcp_servers` counts both
+ declarations — a node's `mcp:` binding and a loop's `tool_servers`. BDR
+ reports five servers instead of none.
+ - **A resolved tool binds from its own server and no other.** Two servers
+ exporting one tool name is precisely what an allowlist cannot
+ disambiguate; first-wins discovery would pick whichever answered first.
+ Python narrows its `--allowedTools` cross product the same way.
+ - **Workers works at all.** There is no registry in workerd —
+ `resolveUrl(env, server, …)` reads env bindings — so the server name
+ *must* be known at emit time.
+
+ Three sources feed the emitted loop, in descending authority: the IR's
+ `tool_servers` (knowledge), other nodes' `mcp:` bindings (a guess — the
+ loop probably reaches the servers the rest of the pipeline does), and at
+ run time the local registry plus `ROTE_MCP_SERVERS` (the escape hatch).
+ Only the first is trustworthy, which is why only it feeds the advisories.
+ A partial map is valid and an unresolved tool still works via discovery;
+ the emitted module names what it could not resolve rather than failing
+ later with a bare "no server provides this tool".
+
+ Two behaviors to preserve:
+
+ - **Nothing guesses.** `rote.probe.enrich_pipeline` writes `tool_servers`
+ from what the baseline actually *called* (`cross_check`'s
+ `resolved_agent_tools`), and leaves a tool served by two servers in one
+ trial unresolved — pinning either would silently send the loop to the
+ wrong endpoint. An authored value always beats inference.
+ - **Discovery is best-effort, binding is not.** A server that is down may
+ not be one this loop needed, so it is recorded and skipped; a declared
+ tool no reachable server provides is fatal. When an auth failure
+ plausibly explains the gap, that failure is rethrown so the workflow
+ parks and `rote mcp login` releases it, instead of dying with a
+ misleading "unknown tool".
+
+ ### `rote run` must see a park, and only the runner can release a local one
+
+ A parked run used to look like a hang under `rote run`. The run-side
+ watcher (`rote.mcp.parked`) reads each runtime's park signal: the DBOS
+ `rote_auth_status` event (now `{awaiting, node}`), and on Inngest and
+ Cloudflare a `rote-auth-park {json}` line the emitted park prints before
+ its `waitForEvent`, because neither local dev server reports what a run
+ waits on. Traps, all found by running it:
+
+ - **The prompt goes to stderr.** `input()` writes its prompt to stdout,
+ which is `rote run`'s result JSON. On a pty with stdout piped the user
+ is asked a question they never see and the run waits forever.
+ `_stderr_input` exists for this; don't "simplify" it back to `input`.
+ - **Replays reprint the marker.** Both TS runtimes re-execute workflow
+ code after a release, so the line prints again. Dedupe on its `wait`
+ field (the wait's step id), never on server or node.
+ - **`rote mcp login` cannot reach a `rote run` dev server.** The
+ registry-wide release targets Inngest's default port and the
+ Cloudflare REST API. So the watcher releases its own run when the
+ server's token file or registry entry changes. On DBOS it waits a few
+ seconds first, because login already released registered DBOS apps.
+ - **A login writes the token file twice.** Client registration lands
+ first with no token. Releasing on that write wakes the run into the
+ same failure, it parks again, and the finished login then looks like
+ "no change". `credentials_settled` waits for an access token.
+ - **A Worker never sees the local token store.** On Cloudflare the
+ watcher shows the export-to-`.dev.vars` fix and neither offers a login
+ nor self-releases, since a login there would "succeed" and change
+ nothing the Worker can read.
+ - **An auth park looks like a gate park to the Cloudflare stability
+ heuristic.** While the watcher reports an active park, the runner must
+ not spend a gate payload on it.
+ - Non-TTY never prompts and never opens a browser. The browser login is
+ offered only when stdin and stderr are both terminals.
+
+ ### An agent loop returns its declared output, through a submit tool
+
+ Downstream nodes index an agent loop's declared `output` keys
+ (`process_invoices_loop.output.invoice_results`), but the loop helper
+ returned `{result, provider, iterations}` on every runtime, so none of the
+ keys existed. Python raised KeyError in the next step, Cloudflare carried
+ `undefined` on silently (a measured run reported `"undefined / NaN"`), and
+ DBOS-TS and Inngest failed tsc with TS7053. No test had a downstream node
+ reading a loop key.
+
+ The contract now, from `rote.adapters._common.agent_loop_output` (one
+ derivation, all six runtimes):
+
+ - **A declared output is submitted, not parsed from prose.** The loop gets
+ a `submit_result` tool whose input schema is the declared output (keys
+ required, `additionalProperties: false`). The CLI lane uses
+ `--json-schema` instead, the same forced structured output a judge
+ uses. A single-type output is submitted as `{result: ...}` and returned
+ bare. A loop with no `output` still returns `{result: <final text>}`.
+ - **Submitting does not end the loop.** It runs until the model ends its
+ turn, and the last submission wins. A loop that ends (or hits its cap)
+ without one gets ONE more request, then fails. That request is
+ schema-locked the judge way (`tool_choice: none` plus
+ `output_config.format`, never a forced `tool_choice`; see the gotcha
+ on the current Sonnet) when the output schema fits structured outputs,
+ and otherwise only asks the model to call `submit_result`. At the cap the runner
+ has already run that turn's tools, so the nudge joins the trailing
+ user message rather than re-running anything (the Python runner does
+ not append a normally-ended final turn; the TS one does, hence the
+ check on both sides). The helper builds the assistant message param by
+ hand: `Message.to_param()` exists only in newer SDKs, and an emitted app
+ runs on whatever version it installed.
+ - **Every lane validates the same way.** The submit tool is `strict: true`
+ (Anthropic's grammar-constrained tool input, no beta header) when the
+ schema fits that subset: every leaf typed, every object closed, and at
+ most 16 `anyOf` / type-array parameters (each nullable field is one;
+ past that the request is a 400). A named type is description-only, so
+ its loop stays non-strict. Either way,
+ `_schema_errors` / `schemaErrors` check types, required keys and extra
+ keys on every lane, and every failure reads `agent loop <id>
+ (<provider>) did not hand back its declared output: <reason>`. Workers
+ AI cannot force a final call, and says so.
+ - **Types:** scalars, lists and `X | null` map exactly, and
+ `output_schema` (evidence) beats the type names. A named type the IR
+ does not define (`InvoiceResult`) accepts any JSON value and, on TS, is
+ an alias in the user-editable `src/types.ts` (`export type InvoiceResult
+ = JsonValue`). The loop's return type, and so every downstream read,
+ uses the alias, so a typed reader compiles once the user narrows it.
+ EmitWriter keeps the edit. Cloudflare types the parked loop's result as
+ `Awaited<ReturnType<typeof loop>>`, not `Record<string, unknown>`, or the
+ alias would be erased at the read. `rote emit` prints one `note:` line
+ per unchecked type.
+ - **Metadata leaves the result.** Provider and turns go to the usage log
+ (Python) or stderr (TS). Putting them next to the declared keys would
+ break a single-type output and pollute any whole-output consumer.
+ - **TS `JsonValue` is spelled out five levels deep, not recursive.**
+ Cloudflare's `step.do` checks results against `Rpc.Serializable`, which
+ fails TS2589 ("excessively deep") on a recursive type.
+ - **Python agent loops honor `ROTE_BASE_URL_<ID>`** like the TS ones and
+ the judges. The Python loop used to ignore it, so a test pointing it at
+ a stub reached the real API.
+
+ Regression: `tests/test_agent_loop_output_parity*.py` runs all six
+ runtimes live against a stub Messages endpoint, with a downstream node
+ reading two declared keys, and typechecks invoice-push on the three TS
+ runtimes. All of it fails on the old helper.
+
+ ### A Cloudflare agent_loop is parkable, so it leaves its parallel wave
+
+ Its tools are MCP tools, so `bindAgentTools` and every in-loop tool call