experiment-loop git:20260913.fb3aa4eA

Immutable. This exact content is served forever at /api/v1/blob/02d126698053b8f8.

---
model_tier: inherit
name: experiment-loop
description: "Use to drive a scalar metric down or up across bounded iterations — keep on strict improvement, revert otherwise, state on disk. Triggers 'minimize X', 'optimize until it stops improving'."
domain: process
workspaces:
  - agent-config-maintainer
packs:
  - meta
---

# experiment-loop

> A bounded **change → commit → evaluate → keep-or-revert** cycle against a
> **scalar metric**, where the decision input is an
> [evaluator-output](../../../docs/contracts/evaluator-output.md) verdict and the
> loop's entire state lives in an append-only register on disk.

This file **routes**. The protocol, the register format, and the pivot ladder are
reference files loaded on demand — a loop doctrine read in full on every session
that merely mentions optimization is a payload nobody asked for.

| Need | Load |
|---|---|
| The iteration protocol and its two exit conditions | [`references/protocol.md`](references/protocol.md) |
| The register's row shape and write ordering | [`references/register-format.md`](references/register-format.md) |
| What to do when the metric stops moving | [`references/pivot-ladder.md`](references/pivot-ladder.md) |

## When to use

* The goal is expressible as **"minimize X"** or **"maximize X"** against a
  mechanical verifier — bundle size, violation count, query count, runtime.
* A verifier already emits, or can be wrapped to emit, the evaluator-output
  contract. Wrapping is usually free: three of this repo's verifiers were wrapped
  with zero changes to any of them.

Do **NOT** use when:

* The target is a **known end state**, not a number — "make the tests pass" has a
  destination, so use [`verify-repair-loop`](../verify-repair-loop/SKILL.md),
  which converges toward it. This skill has no destination; it has a direction.
* The verdict is **subjective craft** — no scalar, nothing to optimize.
* The metric is a **proxy nobody agreed on**. Optimizing an unowned proxy is how
  a loop makes a system worse while its number improves.

## The Iron Law

```
THE METRIC IS NEVER SOVEREIGN. `pass` IS.
KEEP ONLY ON `pass && score > baseline`. A CHANGE THAT IMPROVES THE NUMBER
AND BREAKS BEHAVIOUR IS REVERTED — MEASURED, NOT ASSUMED.
COMMIT BEFORE YOU EVALUATE, SO EVERY REVERT IS A GIT OPERATION.
THE REGISTER IS WRITTEN BEFORE THE ACTION IT RECORDS, NEVER AFTER.
STATE LIVES ON DISK AND IS RE-READ EACH CYCLE — NEVER IN THE CONVERSATION.
```

Each line is a measured failure, not a preference. Spike s01 ran a change that
improved its metric by 67 % and broke the behaviour it measured; only `pass`
reverted it. The same run crashed between its commit and its register append,
twice, leaving the branch one iteration ahead of its own record.

## Procedure

1. **Inspect the verifier before trusting it.** Run the evaluator once on the
   unchanged tree and check its verdict: `pass` must be true and `metric_state`
   must be `present`. A baseline taken from an evaluator that is already red, or
   whose metric is unreadable, is not a baseline — every later comparison
   inherits the defect silently.
2. **Fix the metric and the verifier** before the first change. Name the
   evaluator, its direction, and the baseline score.
3. **Declare the bound** — a maximum iteration count, written into the register's
   first row. An unbounded loop is not this skill.
4. **Iterate** per [`references/protocol.md`](references/protocol.md): one focused
   change, commit, evaluate, keep or revert, append the outcome.
5. **Exit** on either condition (both machine-checkable, never prose): the bound
   is reached, or the explicit exit signal fires.
6. **Report** from the register, not from memory. If the register cannot
   reconstruct the run without the transcript, the run is not reportable.

## Validation

* The register reconstructs the run with the transcript discarded — every kept
  change traceable to a row, every row to a commit.
* Both branches ran at least once. **A loop whose revert branch never executed
  is a one-branch loop with a comment**; spike s01 found its revert broken on
  first use after four green iterations.
* The final tree passes the evaluator with `pass: true`.

## Output format

1. **The register path**, and the iteration count actually run against the bound.
2. **A metric line**: baseline → final, and the number of keeps versus reverts.
3. **The exit condition that fired**, named — bound reached, or exit signal.
4. **A `run_terminal` value** — the exit in the vocabulary every other loop in
   this tree reports in, so a reader can count this run's stop alongside the
   continuation hook's and the self-fix lanes' without a per-surface
   translation table.

   | exit condition | `run_terminal` |
   |---|---|
   | `score >= target` — the run had a destination and reached it | `success` |
   | `iterations_run >= max_iterations` — the declared bound | `exhausted` |
   | `consecutive_reverts >= N` — the exit signal fired | `stagnated` |
   | the evaluator was red on the unchanged tree, so no iteration ran | `clean-no-op` |
   | the run stopped on a missing precondition (metric unreadable, verifier absent) | `blocked` |

   The value set is `RunTerminalState` — `success`, `clean-no-op`, `blocked`,
   `approval-required`, `exhausted`, `stagnated`, `premise-invalidated` —
   declared once at `src/scripts/_lib/outcome_vocabularies.ts`
   (`RUN_TERMINAL_STATES`) and described in
   [`terminal-states`](../../agent-src/contexts/execution/terminal-states.md).
   A new exit condition maps onto an existing member or it is not a terminal
   state; never invent a value here.

   `exhausted` and `stagnated` stay distinct on purpose. This skill's own pivot
   ladder already treats them differently — a spent bound may deserve a larger
   one, while consecutive reverts mean the hypothesis class is done and route to
   the ladder instead. Reporting both as "stopped" throws away the part that
   decides what happens next.

   Reaching the bound is a normal exit, not a failure, and `exhausted` does not
   say otherwise — it says where the run stopped, never whether it was worth
   running. A published null is still a result.
5. **Every reverted iteration listed** with its reason. A revert is a result the
   next run needs, not noise to summarise away.

## Gotcha

* **A metric that improves while the system degrades.** The reason `pass` is
  sovereign. Never widen a verifier to make an iteration keep.
* **A revert path that has never run.** Exercise it deliberately once.
* **A register appended after the mutation.** A crash in between leaves the
  branch ahead of the record; write the row first, then update its outcome.
* **Loop state in the conversation.** It decays silently and the loop keeps
  comparing against a baseline it no longer holds.
* **Iterating past the bound because the number is still moving.** The bound is
  the contract; extending it is a new run with a new baseline.

## Do NOT

* Do NOT keep a change on `score` alone.
* Do NOT edit the verifier to move the metric.
* Do NOT batch several changes into one iteration — a keep you cannot attribute
  is a keep you cannot revert.
* Do NOT delete a register row. Rows are append-only, including the embarrassing
  ones.