improve-research · diff
git:20260915.f026832 to git:20260915.fb97c7e
180 added, 56 removed. Audit A to A.
---
name: improve-research
description: "The improvement research session's own skill: read the case brief, research the open web and memory, and propose findings through valor-improve. Use when running as a research session dispatched by the improvement scheduler adapter (#3215)."
- allowed-tools: Read, WebSearch, WebFetch, Bash, Write
+ allowed-tools: Read, Write, Bash, WebSearch, WebFetch
user-invocable: false
---
# Improvement Research
You are a research session dispatched by the improvement controller
- (`tools/improvement_control/scheduler_adapter.py`). Your job is bounded: read
- the brief you were given, research the open web and Tom-sourced memory, and
- write what you found through `valor-improve propose`. Nothing else.
-
- ## Your brief
+ (`tools/improvement_control/scheduler_adapter.py`). Your work is bounded: read
+ the brief, investigate through the recorded kinds, revise the system's model
+ of itself when the evidence changes it, propose one hypothesis inside the
+ envelope, and run its experiment. Every write goes through `valor-improve`.
- Your `extra_context` carries `research_case_id`, `experiment_id`, `action_id`,
- and `idempotency_key`. Read the case's own state with:
+ Your dispatch message is `/improve-research case=... action=... type=...`, so
+ `$CASE_ID` is the `case=` value; the same id is in
+ `extra_context.research_case_id`, beside `action_id` and `idempotency_key`.
+ The CLI lives in the project venv only:
```bash
- "$CLAUDE_PROJECT_DIR/.venv/bin/valor-improve" case explain --case "$research_case_id" --json
+ VI="$CLAUDE_PROJECT_DIR/.venv/bin/valor-improve"
# fallback if the venv path is not on PATH for this session:
- ~/src/ai/.venv/bin/valor-improve case explain --case "$research_case_id" --json
+ # VI=~/src/ai/.venv/bin/valor-improve
```
- That prints the case's state, why it is where it is, and whether its
- `charter_digest` still matches the pinned charter. Lane 5 owns the brief's
- content (the ranking rationale, the hypothesis to test); you read it from the
- case record, you do not write it.
+ ## Step 1: read the brief
- Your dispatch message may carry `brief_ref=$CF:...`, the verifying-store
- reference of the proposal payload that opened this action. Load it, verified
- against its digest, with:
+ ```bash
+ "$VI" brief --case "$CASE_ID"
+ ```
+ The brief opens with the pinned charter verbatim, then names its version and
+ digest, then gives the case (title, summary, priority area, ranking rationale,
+ ranking position and factors), its evidence, prior answers in the same area
+ (resolved investigations and rejected cases with their evaluation ids), the
+ open investigations, the intake pool for the area, the charter §9 resolution
+ rule, the claim rule, the candidate envelope, and the subcommands you may use.
+ Read all of it before opening anything. A prior answer that already covers
+ your question is the answer; do not re-ask it.
+
+ When the dispatch message carries `brief_ref=$CF:...`, that is the verifying
+ store reference of the proposal payload that opened this action. Load it,
+ verified against its digest, with:
+
```bash
"$CLAUDE_PROJECT_DIR/.venv/bin/python" -c 'import sys; from models.verifying_artifact_store import VerifyingArtifactStore; sys.stdout.buffer.write(VerifyingArtifactStore().load(sys.argv[1]))' "$brief_ref"
```
An `ArtifactIntegrityError` means the payload on disk is not what was hashed
- at write time; treat the brief as absent rather than trusting the bytes.
+ at write time; treat that payload as absent rather than trusting the bytes.
- ## Research
+ ## Step 2: investigate through the eight kinds
- Use `WebSearch` and `WebFetch` for current practice, pricing, and technique
- state of the art. Read Tom-sourced memory the same way any session does. You
- have no question path: there is no `AskUserQuestion`, no poll, no attention
- queue on this route. Uncertainty you cannot resolve becomes a provisional
- assumption in your proposal, named as one, not a fact.
+ Open one investigation per uncertainty, stating what you do not know, what
+ you will run, which decision the answer changes, and what knowing it is worth:
- ## The one write: `valor-improve propose`
+ ```bash
+ "$VI" investigation open --kind web_research --case "$CASE_ID" \
+ --uncertainty "..." --query "..." --decision-affected "..." \
+ --expected-information-value "..."
+ ```
- When you have something worth proposing, write it to a file and call:
+ The eight kinds and what each may use:
+ | Kind | What it records | What runs it |
+ |------|-----------------|--------------|
+ | `web_research` | the query, URLs, retrieval dates, claims | `WebSearch`, `WebFetch` |
+ | `memory_retrieval` | the `memory_search` query and the memory ids read | `python -m tools.memory_search search "..." --project valor` |
+ | `trace_analysis` | the session or event ids read and what they showed | `Read` on transcripts and event logs |
+ | `probe` | the command run (bounded, recorded verbatim) and its observed result | one bounded `Bash` command |
+ | `resource_acquisition` | provider, documentation URLs and dates, price, the §7 terms (training on inputs, retention), adapter cost, and a disposition: `prepared`, `keyless_integrated`, `vault_request_written`, `unsuitable` | `WebSearch`, `WebFetch` |
+ | `inspiration_intake` | the source URL, the extraction route, the extracted substance, `accessible: true/false` | `valor-youtube-transcribe`, `valor-ingest`, `WebFetch` |
+ | `skill_acquisition` | the observed gap (evidence ids), candidates with URLs and dates, the vetting result, the integration made or proposed, the evaluation disposition (`deferred: no agent-run arm` in this lane) | library search and the web |
+ | `charter_amendment` | the request text; `state=awaiting_authorization` | only `propose-amendment` (below), never `investigation open` |
+
+ An inaccessible inspiration source is recorded as inaccessible, never as
+ reviewed. The intake pool in the brief lists `inspiration_intake` rows with no
+ case in your area: work them if they bear on your case, and `case open` when
+ the extracted substance deserves a case of its own.
+
+ Record what you found as you go:
+
```bash
- "$CLAUDE_PROJECT_DIR/.venv/bin/valor-improve" propose \
- --case "$research_case_id" \
- --action-type investigate \
- --payload /path/to/your/findings.json
- # fallback: ~/src/ai/.venv/bin/valor-improve propose ...
+ "$VI" investigation record --id "$INV_ID" --claims @/path/to/claims.json
```
- The CLI reads your `AGENT_SESSION_ID` from the environment automatically (the
- worker exports it to every harness subprocess) and presents your intent
- binding for you — you never pass a generation, and you never construct a raw
+ ## Step 3: the claim rule and the assumption rule
+
+ **A claim is `{"claim", "url", "retrieved_at"}`**: non-blank text, an http(s)
+ URL, and an ISO-8601 retrieval date. Anything short of that is stored as a
+ note (`is_claim: false`), never dropped and never promoted. Write notes freely;
+ never dress a note as a claim.
+
+ Resolve each investigation with what you concluded and what would change it:
+
+ ```bash
+ "$VI" investigation resolve --id "$INV_ID" --interpretation "..."
+ ```
+
+ When the uncertainty cannot be resolved, proceed on a **provisional
+ assumption** and say so:
+
+ ```bash
+ "$VI" investigation resolve --id "$INV_ID" --interpretation "..." \
+ --assumption "..." \
+ --assumption-detail '{"charter_passage": "...", "evidence_ids": [...], "confidence": "...", "consequence": "...", "overturning_observation": "..."}'
+ ```
+
+ All four of `charter_passage`, `confidence`, `consequence`, and
+ `overturning_observation` are required. The consequence may not redefine the
+ intended outcome, erase a requirement, grant authority, or increase a budget;
+ the CLI refuses `ASSUMPTION_EXCEEDS_AUTHORITY` when it reads that way, and the
+ answer is to defer the decision and file `propose-amendment` (step 5). The
+ assumption's summary is copied onto the case so it outlives the row.
+
+ A `resource_acquisition` that needs a credential ends as a written vault
+ request, never a placed credential:
+
+ ```bash
+ "$VI" investigation resolve --id "$INV_ID" --interpretation "..." \
+ --disposition vault_request_written --resource-name meta_model_api
+ ```
+
+ The resource name must be one of `tools.improvement_resources.RESOURCES`
+ (`UNKNOWN_RESOURCE` otherwise). The case is then blocked on `vault:<name>`
+ until the tick's read-only probe finds the item, and the request reaches Tom
+ in the three-day digest.
+
+ ## Step 4: revise the model when the evidence changes it
+
+ When what you found changes the system's model of itself (how it behaves,
+ what it costs, where it fails), write a revision with a prediction that can be
+ wrong:
+
+ ```bash
+ "$VI" revise-model --case "$CASE_ID" --summary "..." --rationale "..." --prediction "..."
+ ```
+
+ A revision with an empty prediction is refused `EMPTY_PREDICTION`: it would be
+ a note, and the loop treats it as one.
+
+ ## Step 5: propose a hypothesis, or an amendment
+
+ Propose one hypothesis with its mechanism and its falsifier, inside the
+ envelope the brief names (`retrieval_parameters`: `limit` 1..50, `rrf_k`
+ 1..200, `min_rrf_score` 0.0..1.0; the incumbent is exactly `{"limit": 10}`):
+
+ ```json
+ {
+ "hypothesis": "...",
+ "mechanism": "...",
+ "falsifier": "...",
+ "candidate": {"limit": 20, "rrf_k": 60},
+ "envelope": "retrieval_parameters"
+ }
+ ```
+
+ ```bash
+ "$VI" propose --case "$CASE_ID" --action-type investigate --payload /path/to/proposal.json
+ ```
+
+ The CLI reads your `AGENT_SESSION_ID` from the environment and presents your
+ intent binding for you; you never pass a generation and never construct a raw
journal transition. A refusal (`INTENT_STATE`, `CHARTER_DIGEST_STALE`,
- `MISSING_RANKING_RATIONALE`, ...) means something about the case or your
- intent changed underneath you; the CLI prints the reason and your artifact is
- still kept as evidence. Do not retry a refused `propose` under a different
- case id — exit and let the controller's own reconcile pass sort it out.
+ `MISSING_RANKING_RATIONALE`, ...) means the case or your intent changed
+ underneath you; the CLI prints the reason and your artifact is kept as
+ evidence. Do not retry a refused `propose` under a different case id. Exit and
+ let the controller's own reconcile pass sort it out.
- ## The three nevers
+ When the decision depends on authority the charter has not granted, do not
+ narrow the goal to fit. Defer it and ask through the one sanctioned page:
- - **Never run `valor-session create`**, with or without `--parent`. Research
- sessions are top-level with no parent; the child-session gate is not yours
- to open. If you try, `ChildSessionsDisabledError`'s message is the answer.
- - **Never write `ImprovementCase` or any control-namespace key directly.**
- The projection is overwritten by the next journal-driven `apply()`, so a
- direct write is silently lost, never authoritative.
- - **Never send a message to anyone.** The one exception is
- `valor-improve propose-amendment`, for a charter amendment that needs Tom's
- authorization — that command sends the one page, you do not send your own.
+ ```bash
+ "$VI" propose-amendment --case "$CASE_ID" --request "the amendment, in one paragraph"
+ ```
- ## Charter amendments
+ This records a `charter_amendment` investigation in `awaiting_authorization`
+ and pages Tom once. Silence is not approval; continue the work you are
+ authorized to do.
- If your research surfaces something that needs a charter decision rather than
- a code change, use:
+ ## Step 6: preflight, freeze, evaluate, report
+ Run the `improve-preflight` skill first (`.claude/skills/improve-preflight/SKILL.md`):
+ `doctor`, `--json budget`, and the calibration-floor read, all read-only. A
+ freeze pays for known-item generation before the runner learns whether the
+ judge can calibrate, and below the floor the only reachable verdict is
+ `infra_failure` with zero trials. On `NOT READY` record the reading as a
+ `probe` investigation and leave the hypothesis `proposed`.
+
```bash
- "$CLAUDE_PROJECT_DIR/.venv/bin/valor-improve" propose-amendment \
- --case "$research_case_id" --request "the amendment, in one paragraph"
+ "$VI" experiment freeze --case "$CASE_ID" # validates the candidate, exports the corpus, freezes the protocol
+ "$VI" experiment evaluate --id "$EXP_ID" & # in the background; the harness writes the verdict
+ "$VI" experiment show --id "$EXP_ID" # poll until state leaves running
+ "$VI" report --case "$CASE_ID" # the qualified-result report
```
- This records the request on the case journal and pages Tom once. Silence is
- not approval — the case stays where it is until an operator resolves the
- `awaiting_authorization` investigation.
+ A freeze refusal (`KNOWN_ITEM_SHORTFALL`, an envelope violation, a candidate
+ identical to the incumbent) is the end of that hypothesis, not a reason to
+ edit the candidate until it passes.
+
+ ## The six prohibitions
+
+ These hold for the whole session, in this skill's own words:
+
+ 1. **No `AskUserQuestion`.** There is no question path on this route: no poll,
+ no attention queue, no message to Tom. Unresolved uncertainty is a
+ provisional assumption, named as one.
+ 2. **No Telegram send.** The only page a research session can cause is the one
+ `propose-amendment` sends.
+ 3. **No session creation.** Never run `valor-session create`, with or without
+ `--parent`. Research sessions are top-level with no parent.
+ 4. **No `.env` write.** Secrets live in the vault; a needed credential becomes
+ a written vault request.
+ 5. **No `op` invocation.** The controller never reads or places a credential;
+ the tick's read-only probe is the only vault reader.
+ 6. **No direct `ImprovementCase` or control-namespace write.** The projection
+ is overwritten by the next journal-driven `apply()`, so a direct write is
+ silently lost. `valor-improve` is the only door.