AGENTS.md · git:20261016.35f87d0 · 2026-10-16 · sha256 63677fb2d014fe2a
AGENTS.md git:20261016.35f87d0B
Immutable. This exact content never changes and is served at /api/v1/blob/63677fb2d014fe2a.
# AGENTS.md
**The agent-context file for this repository.** `CLAUDE.md` and `QWEN.md` are pointers to this
one — there is a single copy of this guidance, and it lives here.
That is deliberate. `QWEN.md` used to be a second, independently-maintained context file, and by
the time anyone checked it, five of its claims were false about the present code — a stale test
count, a "Known Bug" about dependencies that had been fixed, and "no async code anywhere" against
a package this document elsewhere calls required. Two documents describing one codebase is the
same defect as any other hand-maintained duplicate, which is most of what this file warns about
below. One copy, several filenames.
## Commands
```bash
# Python 3.11+ (mcp_client/client.py uses asyncio.timeout). pyproject.toml
# declares requires-python and reads its dependency list FROM requirements.txt
# via `dynamic`, so there is one list, not two. markitdown and protobuf are in
# the `documents` extra, not the default install (#144).
pip install -r requirements.txt
# The SUITE's packages are a separate file since batch 136 (S18-07): they were
# in requirements.txt, which pyproject reads, so `pip install` of the
# distribution resolved a test runner and the npm launcher put one in every
# end user's venv. requirements.txt alone RUNS the harness; this is what makes
# `pytest` work. (`pip install -e ".[test]"` is the same four packages for
# anyone who never clones.)
pip install -r requirements-test.txt
python main.py # chat mode, new thread
python main.py --thread <uuid> # resume a thread
python main.py --mode research "some query" # ten-pass deep-research pipeline
python main.py --provider OPENAI --model gpt-5.1 # override provider/model
python main.py --tui # Textual TUI (§16); the CLI stays the default (D12)
python main.py --tui --provider OPENAI --model gpt-4o # or switch in-session with /model
python main.py --trust-project # grant D17 workspace trust non-interactively
python main.py --mode research --grant "q" # §25: pick gated tools to authorise for this run
python main.py --mode research --attended "q" # §25: approve every gated call as it happens
python main.py --mode research --review "q" # §20: review the finished run, consent to each correction
# §21a compaction runs automatically in every shell; /compact triggers it by hand in the TUI
# §26: /claims [run id] or ctrl+l opens a run's claims;
# /copy [last|report|claims|conversation|all] [--file path] -- batch 48
# adds `conversation` (the exchange, no harness lines); `all` stays the
# superset
# §44: /window <tokens|off> is REMEMBERED per (provider, model) now and sets
# the number the compaction trigger is a share of; /trigger stays ephemeral
python main.py --summary <thread> # §21c: print a thread's distilled summary
python main.py --ref <thread> --ref <thread> # §21c: attach other threads' summaries as context
# §21c in the TUI: /summary distils this conversation; /ref [--list|--clear] references another
# §27/batch 18 in the TUI: /threads lists conversations only (most recently active
# first, capped at 200 with a notice); /resume <thread-id> opens any thread by id
python main.py --init # §24/§44: scaffold AGENTS.md and the stubs it links
python main.py --init --research-project # §24: skip the type question on a piped run
python main.py --init --project-config # §24 I17: .venastine/settings.json + mcp.json, no model call
# §24 in the TUI: /init [--software|--research] [--config]
# §23 slice 2: the model asks with `ask_user` and keeps a checklist with
# `todo_write`; the TUI panel's placement is the `tui.todo_position` setting
pytest # 7376 tests, offline. 1:26 on 16 cores, 9:06 serial (+~5s first run: matplotlib font cache)
pytest tests/test_orchestrator.py # one file
pytest tests/test_orchestrator.py::test_name # one test
pytest -k "grounding" -x # by keyword, stop on first failure
pytest -m integration # opt-in, 4 tests; three spawn a real stdio MCP server (§17 AC8) and one the fallback shell (batch 130)
# PARALLEL BY DEFAULT since batch 109 -- pytest.ini's addopts carries
# `-n auto --dist loadscope`, and nothing below needs typing normally.
pytest -n0 # SERIAL. Debuggers, output parsing, the mutation harness
pytest -n 4 # exactly four workers
pytest -n logical # every core -- and MEASURABLY NOT FASTER, see below
```
Test dependencies (`pytest`, `pytest-mock`, `pytest-asyncio` and `pytest-xdist`) are listed in `requirements-test.txt`, not `requirements.txt` — the comment above its `pip install` line says why. **`pytest-xdist` is not optional**: `pytest.ini` puts `-n auto` in `addopts`, so without it `pytest` fails at argument parsing rather than quietly running serially, which is the intended trade — a suite that silently takes eight times longer on a machine missing a package is worse than one that names the package.
**`--dist loadscope` is a coverage decision, not a performance one, and it is the one thing here not to "optimise".** xdist's default `load` hands out individual tests, and three places depend on one test running immediately after another in the same process: `test_probe_isolation.py::TestWhatOneTestLeavesForTheNext` (two ordered pairs, and its docstring says so), `test_config_edit.py::TestTheLoaderCacheDoesNotLeakBetweenTests`, and `test_storage_e2e.py`'s module-scoped `real_storage` plus the tests that read what earlier ones wrote to its database. Split across workers **none of them fails** — the autouse fixtures in `tests/conftest.py` reinstall the clean state for every test, so "and the next one does not inherit it" passes whoever ran before it, and the file goes silently vacuous. `loadscope` keeps all three **structurally**: it groups by CLASS for a test inside one and by MODULE for a test that is not, and each of the three is a single class or a module of plain functions.
**And a file is no longer one unit, which is the trap loadscope adds.** A file holding both module-level tests and a class is TWO groups, so a non-function-scoped fixture in it is set up twice on any worker that takes both — found by measuring, not by reading, when a `-n logical` run put `test_storage_e2e.py`'s two groups on one worker and the second `real_storage` raised `Table 'conversationthread' is already defined`. Five `-n auto` runs had been green, so it is a scheduling lottery. **If you add a module- or session-scoped fixture, its file must hold no test class** -- and nothing here states how many such fixtures exist, because a count written beside a rule that does not need one goes stale (A34). The rule is enforced for the file you are writing rather than described for the file that happens to hold the fixtures today: `test_every_file_with_a_module_scoped_fixture_declares_no_test_class` reads every test module's fixture DECORATORS and fails on any file that has both, and `test_this_file_declares_no_test_class` still pins the one file that has them now.
**The suite was FLOOR-BOUND and is not any more, and batch 114 is why.** The slowest single GROUP sets the wall time whatever the worker count, and under loadscope that was `tests/test_tui.py`'s 183 module-level tests — one group, because a test outside a class is grouped by MODULE — at 292 node ids and about a fifth of the suite's measured time. They were cut into 41 classes along the section banners the file already had (TECHNICAL_DEBT 30) and are **73** now, which is the rule working rather than drifting. That number is read back out of this sentence by `test_agents_md_states_the_real_test_tui_class_count` (batch 128), so a batch that adds a class here also edits this digit. Measured properly: three runs a side, ALTERNATED on one box so drift cancels instead of landing on whichever side ran second — **2:17.9 / 2:07.9 / 2:28.6 before against 1:10.7 / 1:40.6 / 1:26.3 after, a mean of 138.1s against 85.9s, 1.61×**, and the two ranges do not overlap, which is the part that makes it a result rather than a pair of averages. Against a serial run that read 8:41 the same evening, 6.1× where it was 3.8×.
**What is left as the floor is one PARAMETRISED function**, and it cannot be split by any class boundary: loadscope groups every node id of a parametrised test together, and `test_every_modal_keeps_its_decision_on_screen` is 3 sizes × 19 cases = **57** of them, pinned against the collected node ids by `test_agents_md_states_the_real_modal_parametrisation` (batch 128) because `_DECISION_CASES` is computed and cannot be counted by reading. Splitting it further was measured and declined — `-n auto` is 11 workers here and the serial suite is ~520s, so the ideal worker share is ~47s and that group already sits under it. Below a bound that has stopped binding there is nothing to win.
**Two things not to re-derive from the above.** CI gains nothing: `-n auto` is 70% of the cores, so a 4-core runner takes 3 workers and its ideal share was already above the old floor — the payoff is on developer machines, which is where the suite runs many times a batch. And extra workers still buy nothing: alternated on the same commit, `-n auto` (11 workers) ran 2:09 / 2:10 and `-n logical` (16) ran 2:13 / 2:13 / 2:14, about 3% SLOWER, which is the same answer batch 109 got under loadfile.
**A new test in `tests/test_tui.py` goes inside a class.** Pinned by `TestThisFileIsGroupedForLoadscope::test_every_test_here_is_inside_a_class`, which parses the file with `ast` and sits inside a class itself, because a module-level test there would be the thing it forbids. Without it the module group rebuilds one commit at a time and nothing says so until the floor is back. `tests/test_themes.py` is the larger remaining classless population — 319 node ids, no classes — and was left alone on a measurement: ~12.5s, far under the floor, so classing it would lower nothing.
**A fresh clone needs `cp providers.json.example providers.json` before `pytest`.** The file is gitignored, and **12 tests across four files** fail with `ValueError: No providers configured: ...` without it — 7 in `test_loop_tool_dispatch`, 3 in `test_loop_stop_conditions`, 1 in `test_agents` and 1 in `test_thread_legibility` (measured by moving the file aside and running the suite) — `api_initialization()` needs the provider ENTRY to exist, even though the key inside it stays empty. Since #24 the message names the file and its remedy instead of reporting an unknown provider, and `AGENT_PROVIDERS_FILE` redirects it like every sibling path. "Offline, no API keys" is true; "no config file" is not. Also note `python3 -m pytest`: a `pytest` on PATH from a separate tool install runs in its own environment and sees none of the project's dependencies.
MCP servers are a *third* config file again — `mcp.json`, user-level or `.venastine/`-level. Not `.env`, not `providers.json`, not `settings.json`.
**Credentials are two separate mechanisms, deliberately.** LLM provider keys → `providers.json` (managed by `credentials.py`, gitignored, template at `providers.json.example`). Misc tool API keys (GitHub, NVD) → `.env` (managed by `env_secrets.py`, template at `.env.example`). Never mix them.
## Distribution (npm, batch 35)
The project ships through two channels: `git clone`, and npm as **a folder of source** — no binary,
no bundle. `bin/venastine.mjs` is the only thing npm can execute; its whole job is to find a Python
≥3.11, make sure the dependencies exist, and spawn `python <root>/main.py` with the user's argv,
cwd and stdio. The harness itself did not have to change to support this, because every shipped
asset already resolves from `__file__` (`HARNESS_ROOT`, `_package_root()`, `prompts/system_prompts.py`)
and every piece of state from cwd with an env override.
**`npm update` replaces the package directory wholesale, `config.yaml` included** (batch 86, measured with `npm pack` and two installs into a scratch prefix). `config_update.py` is what makes that survivable: the launcher runs it before `main.py`, and it re-applies the user's own values onto the freshly shipped file from a pristine copy and a mirror kept in `~/.config/venastine/config-state/`. Two rules there are load-bearing rather than tidy. **The mirror is refreshed only after the version check** -- copying `config.yaml` over it first overwrites it with the file npm just replaced, and every other test still passes. Batch 87 moved the post-session copy out of this file for that reason: it was four lines of `fs.copyFileSync` guarded only by a version file EXISTING, so a merge that failed was followed at session end by the newly shipped file being copied over the mirror, and the user's settings were gone from every copy. The launcher now spawns `config_update.py --mirror` and owns no decision at all; the state is keyed per install, and the launcher READS `installs.json` rather than recomputing a path. **Nothing in that path may fail a launch**: the module always exits 0 and the launcher does not check its status, which is why the cheap path imports no first-party code at all (`import config_schema` alone costs 294ms of pydantic and ruamel, on a path that runs at every launch and does nothing on almost all of them).
**The launcher must never set `PYTHONPATH` or `AGENT_WORKSPACE`** (test-pinned,
`test_the_launcher_never_sets_the_two_variables_that_would_break_it`). `tools/isolation.py` builds
its child's `PYTHONPATH` from the *parent's resolved `sys.path`* on purpose, so an injected value is
inherited into that computation and poisons it. `WORKSPACE_DIR` is a **permission boundary** —
`file_ops` auto-approves writes inside it — so pointing it at a shared home directory would silently
auto-approve writes there from every project. That second one is the trap: it is the obvious next
move after seeing `APP_DB_PATH` redirected two lines above, and it is wrong.
**Since §44 it is also the PROJECT PATH, which makes that rule stricter rather than looser** (WS7).
`main()` sets `project_path` to the resolved workspace when the workspace is *named as the project*, so
trust, `.venastine/`, `/init`'s destination and `UserMemory`'s project scope all follow it. **Two ways to
name it since batch 111** (TECHNICAL_DEBT 28): the presence of `AGENT_WORKSPACE`, or `workspace_is_project:
true` in `config.yaml`, OR'd inside `config_schema.derived_values(cfg)` so `main()` still reads one
boolean. **And when NEITHER names it, `main()` falls back to `os.getcwd()`, which is refused if that is
the install tree**
(`protected_paths.check_project`, TECHNICAL_DEBT 26, batch 107) -- naming the harness was always
refused, and until then the implicit route reached the same state by saying nothing. PRESENCE, never
value: the default `./workspace` is a subdirectory of the launch directory, so a value test would
move the project one level down for everyone who set nothing. An injected value from the launcher
would now relocate a user's whole project, not merely widen a write boundary.
**State is global, artifacts are local.** The launcher sets `APP_DB_PATH` and `AGENT_LOG_FILE` to
`~/.config/venastine/` — together, on one decision, because a global database writing to a local log
is worse than either. `main.py` sets `project_path = os.getcwd()` and `UserMemory` carries `scope`
*and* `project_path` (M12/D25) precisely so ONE database can tell projects apart; per-directory
databases would stack an accidental scoping axis on that deliberate one. `AGENT_OUTPUT_DIR` follows the
workspace since §44: `output/` is write-only (nothing reads it back — the report and claims live in
the `PipelineRunRecord` row), so artifacts belong beside the work, and "beside the work" is the
workspace once one is named. Its default is one expression with no branch —
`os.path.join(os.environ.get("AGENT_WORKSPACE", "."), "output")` — so an unset variable still means
`./output`. An existing `./app.db` always wins.
**`~/.config/venastine` on every platform, Windows included.** The Python half already resolves its
user tier with `os.path.expanduser` in three modules; a Node-idiomatic `%APPDATA%` would split one
config directory across two locations.
**`.env` resolution was genuinely broken outside a checkout** and is now fixed at
`env_secrets.py`. A bare `load_dotenv()` walks up from *that module's own directory*, not from cwd —
identical in a checkout, wrong everywhere else. It is `find_dotenv(usecwd=True)` now, with
`AGENT_ENV_FILE` as the override. Do not simplify it back. And since batch 160 (A70) the file is
READ by `get_env_secret` at the call and never loaded into `os.environ`, and a `.env` cannot supply
an `AGENT_*` or `VENASTINE_*` name -- the module used to `load_dotenv()` at import, which would have
let a cloned repository set `VENASTINE_REDACT_OFF` for the whole process.
**Three lists exclude local secrets from three channels, and they must agree**: `.gitignore` for
`git push`, `.gitattributes export-ignore` for `git archive`, and `files` in `package.json`
for npm. npm's is the one that fails *open* — with no `files` key it falls back to `.gitignore`, so
`providers.json` would be excluded by accident rather than decision. It is an allowlist for that
reason, and it needs its `!**/__pycache__` / `!**/*.pyc` / `!**/.ipynb_checkpoints` guards because a
directory entry sweeps in whatever is inside it. The three-channel rule itself is stated in `docs/PRIVACY.md` and recorded in `docs/DEVLOG.md`, and **it has no decision id** -- batch 128 removed a cited `H3/C2`, which named §31's `RLIMIT_CPU`/`RLIMIT_AS` decision and an id the record does not define at all.
**Never vendor wheels into the tarball.** The package is first-party source and the launcher has
zero npm dependencies, which is what lets `THIRD_PARTY_NOTICES.md` say the npm channel adds no
licence surface at all. Bundling `openai`, `google-genai` or `pytest-asyncio` would trigger
Apache-2.0 §4(d) and oblige shipping their `LICENSE`/`NOTICE` files as received. Publishing *is*
distribution under §4 where running from a checkout was not, which is why `LICENSE` and `NOTICE` are
in the allowlist — npm auto-includes the first and **not** the second.
**The publish workflow is armed (2026-09-04).** `.github/workflows/publish.yml` runs on
`workflow_dispatch` and on pushed `v*` tags. `dry_run` still defaults to true on a manual
dispatch, and until the `NPM_TOKEN` repository secret exists every run — tags included —
downgrades itself to a dry run, so arming the trigger while the secret is missing only
produces rehearsals. A tag push carries no input: with the secret set it publishes for real.
Publishing makes the source public regardless of the repository's visibility, and a
published npm version can never be replaced — only deprecated, so the repository goes
public and the secret is added deliberately, in that order.
## Documentation map
Read these before changing anything non-trivial — they carry design decisions that are locked, not defaults to re-derive.
**The record lives in `docs/` and six documents stayed at the root, each for a reason that is not
tidiness** (batch 89). `README.md` is the landing preview; `LICENSE` and `NOTICE` are what GitHub
detects and what `pyproject.toml`'s `license-files` names; `CONFIG_ARCHITECTURE.md` ships in the npm
tarball and `config.py` points at it. `AGENTS.md` is the load-bearing one: `core/config_loader.py`
reads it through `workspace_trust.PROJECT_CONTEXT_FILENAME`, and WS9 puts the ROOT copy inside D17's
trust content hash -- moving it would break a runtime path and silently change every project's hash.
`CLAUDE.md` / `QWEN.md` follow it, since the whole point of them is that a harness looking for one
filename finds it without searching. Nothing under `docs/` is read by production code, which is what
made the move free: no import changed, and `tests/test_docs_consistency.py` is the only module that
opens any of these files (its `DOCS` constant is where the path now comes from).
- **docs/ARCHITECTURE.md** — what's built, file-by-file contracts ("what belongs here / what does NOT"), known gotchas (§11).
- **docs/ROADMAP.md** (§1–§12, all built — but see §10's revisit note) and **docs/ROADMAP_v2.md** (§13–§48, all built) and **docs/ROADMAP_v3.md** (§49–§52, §55, §61, §62, §63, §64, §65, §66, §67, §68, §69, §70, §71, §72, §73, §74, §75 and §76 built; §53, §54 and §56–§60 recorded from the owner's requests in batches 122 and 123, specified and not yet designed) — full implementation specs with a locked Design Decisions Record (D1–D31, plus S1–S4 from the §14–§18 review, R1–R16 from §25, K1–K7 from §19, V1–V9 from §20, M1–M21 from §21a/§21b/§21c, P1–P4 from §22, L1–L6 from §26, T1–T9 from §27, I1–I17 from §24, J1–J14 from §23, E1–E14 from §10's revisit, C1/C3/C6/C8/C10 from Rev. 1's review, G1–G7 from §28, N1–N8 from §29, B1–B11 from §30, H1–H10 from §31, A1–A76 from §32 (A16 in ROADMAP_v3 §62, A17, A18 and A19 in §63, A20 and A21 in §64, A22, A23 and A24 in §65, A25, A26 and A27 in §66, A28, A29, A30, A31, A32 and A33 in §67, A34, A35, A36, A37, A38 and A39 in §68, A40, A41, A42, A43, A44 and A45 in §69, A46, A47, A48 and A49 in §70, A50, A51, A52 and A53 in §71, A54, A55, A56 and A57 in §72, A58, A59, A60, A61 and A62 in §73, A63, A64 and A65 in §74, A66 and A67 in §75, A68–A76 in §76), W1–W9 from §33 U1–U9 from §34, Y1–Y5 from §35, Z1–Z8 from §36, F1–F8 from §37, O1–O8 from §38 Q1–Q6 from §39, UN1–UN6 from §40, X1–X7 from §41, RA1–RA6 from §42, RM1–RM6 from §43, WS1–WS10 from §44, SQ1–SQ10 from §45 EP1–EP8 from §46, NA1–NA18 from §47, CE1–CE7 from §48 SS1–SS72 from §49, NW1–NW5 from §50, PG1–PG9 from §51, RB1–RB7 from §52, IO1–IO7 from §53, AT1–AT8 from §54, TR1–TR15 from §55, JV1–JV10 from §56, SM1–SM4 from §57, PR1–PR5 from §58 UI1–UI4 from §59, UH1–UH12 from §60 and CN1–CN12 from §61). Section and D-numbers are stable and cross-referenced everywhere.
**Six namespaces use the same `LETTER+NUMBER` shape, and only the first is the
record.** An id that resolves to two places is a cross-reference that fails
silently — you read the wrong one and it makes sense. So every citation outside
the record says which namespace it means, and
`tests/test_docs_consistency.py::test_every_decision_id_cited_in_production_code_resolves`
fails on one that does not.
| namespace | ids | written as | where defined |
|---|---|---|---|
| **design decisions** | the families above | bare — `D14 forbids widening` | `docs/ROADMAP.md` / `docs/ROADMAP_v2.md` / `docs/ROADMAP_v3.md` |
| pipeline gates | `D0`, `D1`, `D2` | `gate D0` | §4/§26 — pure-code routing between passes, no LLM call |
| acceptance criteria | `AC1`… | `§21 AC6` — **section-scoped**, so the section is required | each section's *Acceptance criteria* block |
| Rev. 1 open questions | `S11`, `S12`, `S16` | `Rev. 1 S12` | resolved inline in `ROADMAP_v2.md` |
| audit severities | `S1`–`S4` | `severity S1`, or an issue title | `#15`, the audit tracker |
| review findings / mutations | `F2`, `P10` | `review finding F2`, `mutation P10` | `DEVLOG.md`, the audit's mutation passes |
`D1` and `D2` are the unavoidable pair: both a decision and a gate. They resolve,
so nothing can force the distinction mechanically — write `gate D1` when you mean
the gate. A seventh use of `C1`–`C10` is the research pipeline's **claim ids**,
which are run data rather than references; see the C table in `ROADMAP_v2.md`.
- **CONFIG_ARCHITECTURE.md** — why every value in `config.yaml` is what it is. `config.py`'s 788 lines of rationale, carried over verbatim when the values moved into YAML, keyed by the YAML's own names. Read the entry for a key before changing it; several of these numbers have already been "corrected" back to a value they were deliberately moved away from.
- **docs/DEVLOG.md** — per-section implementation notes: what was followed verbatim, what was deviated from (every deviation was an explicit user decision — do not silently override).
- **docs/TECHNICAL_DEBT.md** — the register: what a review found and what was deliberately left, each item open or closed with the batch that closed it. Cite an item as `TECHNICAL_DEBT 42`, bare — one form, since three were in use. An entry is evidence, not instruction: batches 126 and 127 each found the ENTRY itself wrong about where its own defect came from, so measure the claim before building the fix it proposes.
- **docs/REVIEW_MARATHON.md** — the pre-merge review marathon: 19 independent review scopes covering every production module, the finding format every report uses, and the rules of engagement for a read-only reviewer. **Open while it is open** — the run log at the bottom says which scopes have been reviewed. The per-scope reports are NOT in the repository: they go to a gitignored `review/`, because a tracked `.md` moves the gated readable-doc count and because `read_project_doc` would advertise a list of open defects to every run.
- **tests/BREAKING_CHANGES.md** — what breaks each test when production code changes, the symptom, and the fix.
- **CLAUDE.md / QWEN.md** — pointers to this file, nothing more. They exist so a harness that looks for one filename finds it without searching; the content has one copy.
## Architecture
A from-scratch agentic harness (no LangChain). Two modes — regular tool-using chat, and a ten-pass self-critiquing research pipeline — share the *same* primitives: `core/loop.py` (control flow), `core/client.py` (one model call), `tools/registry.py` (dispatch). The pipeline is not a separate codebase; it is the same loop configured differently.
**Synchronous throughout, except the MCP bridge and the Textual event loop.**
That qualifier is load-bearing, not pedantry (audit #128). This file used to say "No async anywhere", and `mcp_client/client.py` has four `async def`s, imports `anyio`, runs a dedicated event-loop thread and bridges into it with `run_coroutine_threadsafe` — which the MCP section below documents at length and calls **required**. An absolute stated here plus four `async def`s in one file reads as a violation to clean up, and that section exists precisely because someone already tried moving those enter/exit calls across a task boundary.
### The layers, and the boundaries that matter
| File | Owns | Does NOT own |
|---|---|---|
| `config.yaml` | Plain values only, harness tier, ONE location | Anything derived; a user or project tier; an override variable |
| `config_schema.py` | The schema, the env-override table, the loader, `HARNESS_AUTHORITY_KEYS` | Any first-party import — that is a cycle back through `core/config_loader.py` |
| `config_edit.py` | `/config`'s catalogue, the value grammar, the comment-preserving writer, `RestartRequest` | Anything about a terminal; applying a change to a running process |
| `config_update.py` | The three-way merge that carries an edited `config.yaml` across an `npm update`; `bin/venastine.mjs` runs it before `main.py` | Anything at runtime; a user tier or a redirectable config path -- it puts values back into the ONE location, it does not add a second |
| `config.py` | Publishing those values as plain mutable module globals | Any logic, any `if`/`else`, any `os.environ` read, any decision-making function |
| `credentials.py` / `env_secrets.py` | LLM provider keys / misc tool keys | Each other's domain |
| `database.py` | The engine/connection only | Table classes, CRUD, awareness of what data exists |
| `storage.py` | Schema (table classes) + CRUD | Active conversation state, provider-specific shapes |
| `core/memory.py` | Active in-run state, provider-neutral, write-through | SQL, table classes, provider shapes |
| `core/session.py` | The ephemeral `/trigger` override, and what clears it | Resolving that number; persisting anything, ever |
| `core/model_windows.py` | The remembered context window per `(provider, model)` | Deciding whether a number is a sensible window |
| `core/client.py` | One call + provider format translation | Looping, retry, tool dispatch, bookkeeping |
| `core/provider_errors.py` | Which provider failures are transient, the retry backoff, the one-line description of a failure | The attempt itself (`core/loop.py`); importing any SDK |
| `core/loop.py` | Call-dispatch-repeat control flow | Provider formats, tool policy, pipeline state |
| `security/permissions.py` | Policy (`is_tool_allowed`, `requires_approval`) | Dispatch mechanics |
| `safety/policy_enforcement.py` | Post-call content policy (secret redaction, blocked domains) | Access control |
| `tools/registry.py` | Dispatch mechanics; the only file importing both permissions and tool modules | Policy definitions |
`database.py` and `storage.py` live at the **project root**, not under `core/` — an import expecting `core.storage` was a real bug once.
### The loop (`core/loop.py`)
`RunAgentLoop._run()` is a **generator** yielding `LoopEvent`s (`core/events.py`). Two public entry points — `run_agent_conversation` and `continue_conversation` — return a `ModelResponse`; a research pass is `stream_deep_research_mode()`, a generator its caller drains. Never reimplement the loop elsewhere.
`run_agent_conversation` and `continue_conversation` drain `_run()` directly via `run_to_completion()`. A pass is one step further out since §26: `stream_deep_research_mode()` forwards `_run()`'s events and *returns* the response, and `_run_pass` takes that return value with `yield from _translate(...)`.
**The two questions are still different, and only one of them is production's.** `run_to_completion` reads the response off the terminal event — right for a bare `_run()`, which returns nothing. A generator that returns its own value needs the other question asked, because `thread_id` is attached after the loop finishes and the terminal event carries the copy without it. §26 answered it with a second drainer, `return_value_of`, beside `run_to_completion`; its only caller was the synchronous `run_deep_research_mode` wrapper, and when that retired (see §26 below) the helper went with it. `tests/conftest.drain` had been the byte-identical function on the test side the whole time, which is why nothing needed writing to replace it.
Three stop conditions, all in `_run()`: no tool calls (`complete`), `max_steps` exhausted (`max_steps_reached`), cumulative input+output tokens ≥ `max_total_tokens` (`token_budget_exceeded`). `stop_reason` and `thread_id` are set by `_run()`/the wrappers, never by `call_model_stream()`.
**D20 — persist before branch.** `memory.add_assistant_message(response)` runs immediately after the call returns, *before* any stop-condition branching. Moving it into a conditional silently drops plain-text final answers and budget-truncated responses from the persisted thread, and breaks the §3 JSON-retry path. This has been reintroduced once already; `test_streaming_loop.py` AC6 fails if it happens again.
**Persist before EMIT, too (#42).** `memory.add_tool_result()` runs *before* the `tool_result` event is yielded — the same rule §22 gives `_Progress.checkpoint()`, and for the same reason: a generator only advances while someone iterates it, so emitting first makes durability depend on a UI continuing to read. It shipped the other way round. Because D20 has already written the assistant turn carrying the `tool_use` block, an abandoned generator left a `tool_use` with no matching `tool_result` — M4's pairing broken from the other side, and a thread that cannot be resumed. The tool *ran*; only the record was lost. `tool_call_start` stays before its dispatch, correctly: nothing is persisted there.
**A model call is retried, and since batch 113 the screen is no longer the boundary** (batch 91, amended by batch 113 / TECHNICAL_DEBT 25). The attempt loop in `_run_steps` re-sends one step's call when `core/provider_errors.is_transient` says the failure can pass: a connection error, a 408/409/429/5xx, or an error event inside a stream that opened with 200 — which both the openai and anthropic SDKs raise unretried, and which is how OpenRouter reports an overloaded upstream. Up to `model_call_max_retries` more attempts, `model_call_retry_base_delay_s` doubling with jitter; the SDKs' own pre-stream retries stay on inside each attempt. **One rule for every caller**: batch 91 split them, retrying a *drained* run whatever it had streamed and refusing a *drawn* one after its first delta, because a RichLog row could not be taken back. Batch 112 made one retractable, so the loop now yields `LoopEvent(retract=...)` before the backoff wait — only when the attempt had drawn something — and the shell deletes its own rows. **`drained` is gone**; do not reintroduce a two-speed retry, which `test_one_rule_for_every_caller` guards by reading the source. The loop sits below the wrappers that write the user row, so a retry never writes it twice, and above the accounting, so only the attempt that succeeded is billed or persisted (D20).
**A quit STOPS the turn it says it stops** (batch 145, S14-06). `exit()` printed
"the active work is abandoned at its next event" on any exit while `_busy`, and
`_busy` covers a chat turn, a research run and `/compact` alike — but only the
research path read the flag (`_forwarding` breaks at the next event, #105) and
`_consume` is recorded in `tests/BREAKING_CHANGES.md` as deliberately untouched. So
on a chat turn the sentence asserted an abandonment that did not happen while the
worker went on draining the generator: up to `max_steps` further model calls and
every tool call inside them, against an app that has gone. **The fix is SS61's
cancel token, not a break in the drain**, and the difference is measured: a break
lands on whatever event is next, and one of those is `tool_call_start`, where D20
has already persisted the assistant turn carrying the `tool_use` and nothing has
written the matching `tool_result` — M4's pairing broken and an unresumable thread.
Driven on the real generator three ways, abandoning on `tool_result` orphaned
nothing (that is what #42 fixed), abandoning on `tool_call_start` orphaned one, and
the token orphaned nothing and ended the turn `cancelled`. The token is tested
BETWEEN tool groups and `_finish_cancelled` writes a row for every call it will not
run, which is "#42's lesson from the other side" in that function's own words. The
flag is set BEFORE the cancel, so a worker that can see the stop can already see the
teardown — `_wait_for_sessions` reads them in that order and the message it posts
depends on which it finds.
**A retraction is bounded by a MARK, not by the open span** (batch 113). `Transcript._retract_mark` is armed by the first delta of a span and cleared by `flush_stream`; `retract_to_mark(note)` truncates `_entries` back to it, clears both pending buffers, replays and writes the note. A second retraction keeps the first line for free, because `write_system` flushes — which clears the mark — and the next delta re-arms it after the note. Not the open span, because `_write_thinking_chunk` closes an open ANSWER before it draws — one attempt with interleaved reasoning is three entries with only the last open. Not taken at the flush either, though that was tried: every `write_*` opens with `flush_stream()`, so a line written mid-span moved the mark PAST the attempt's own text and the retraction then deleted the foreign line and kept the half-answer. `Transcript` is the only surface in this repository that draws model text — the research orchestrator translates a character COUNT and the CLI shell (`main.py`, never a `cli.py` -- there has never been one) ignores `retract` entirely, which `core/events.py` states is still correct about failures — so it is the only consumer of this event.
**`_run` is a thin frame; the loop is `_run_steps`** (batch 91). On an `Exception` escaping it, the frame stores `redact_diagnostic_text(describe(e))` on the thread's newest user row (`redact_secrets` until batch 144, which is the vendor tokens alone -- and `API_URL` is user-supplied, so an httpx error names a URL that may carry userinfo, into the sink `docs/PRIVACY.md` says is deleted by wiping the database rather than one thread) (`MessageLog.error`, via `mark_turn_failed`) and re-raises unchanged — `core/events.py` still has no error variant. Not on `GeneratorExit`: an abandoned generator (#42) is not a failed run. **A column, not an `error` ROLE, is the decision:** provider translation and the compactor's `_as_text` read a user row's `content` alone, so the failure reaches `replay_entries` (an `error` entry right after its user entry) and nothing that sends or summarizes the conversation — a role would have been M8's mistake. `test_grants.py` walks `core.loop`'s AST for exactly one function named `_run` with no `authorized_call` in it; the frame satisfies both, so do not "simplify" it back into the body without re-reading that test.
**"Headless" means unable to ask, not "not a TUI."** `_run()` hides approval-gated tools when there is neither a `permission_channel` nor an `approval_provider` (§25 broadened this from the channel alone). A research pass in attended mode can ask, so its gated tools are advertised.
**Approval single source of truth:** `ToolRegistry.approval_needed(name, params, context)` — the tool's own `approval_check` **OR'd with** the config/context lookup (§15; it used to *replace* it, which made agent overrides inert for `read`/`write`/`edit`/`shell`). Both `dispatch()` and `_run()`'s permission bridge call it, so they cannot diverge. `_run()` checks reachability (`registry.is_allowed`) *before* asking, so a context-excluded tool reports the context denial instead of prompting and then being denied anyway. A tool already in `run_info.granted_tools` skips the prompt entirely — that is §18's per-turn subagent sign-off, keyed off `ToolSpec.grant_scope`, so no tool name appears in the loop. §15's spec sketch inlines this into `dispatch()` because it predates §13 — don't follow it there.
**ONE `registry.dispatch()` call site in `_run()`, and it carries the loop's own decision (batch 60).** There were two — identical but for the arguments that carry authorization — and the one reached by an ALREADY-ANSWERED call was the one that omitted them, so `dispatch()`'s re-check denied every call the loop had just waved through: a second spawn of one agent in a turn, a subagent using a tool it was ticked for at the sign-off, any `--grant-tools` name in an unattended run. `dispatch()` still re-checks and must — it is a public entry point and the fail-closed backstop; what changed is that the loop answers it. The `approval_callback` is passed as `(lambda n, p: True) if authorized_call == call.id else None`: **an ID, not a flag**, so a variable hoisted out of `for call in response.tool_calls:` holds the PREVIOUS call's id and fails CLOSED instead of covering every later call in the turn. The bool version was measured — hoisting it survived the whole suite, because nothing can currently reach that line gated and un-authorized. `test_grants.py::test_the_loops_authorization_cannot_outlive_one_tool_call` reads this source, since no behaviour can see either mutation.
**Permissions are "stricter wins" (§15/D14).** Allow/deny ANDs across global config and every active `ToolContext` (`tools/context.py`); the global check runs first and unconditionally, so no agent or skill can re-enable a globally disabled tool. Approval ORs across all layers, making `approval_overrides` a one-way ratchet — a `False` entry is indistinguishable from an absent one, and that asymmetry is the design, not a gap. `_run()` takes a `context`, not an `allowed_tools` list, and performs no membership check of its own. `registry.schemas(context)` advertises only what is actually callable.
**`.venastine/` is the one path the file tools HARD-deny, and the one the shell always asks about.** `PROTECTED_SEGMENTS` (`security/protected_paths.py`) is one list with three consumers, because three copies of one segment drift. `read`/`write`/`edit` refuse any path resolving into the segment — via a `refusal_check` (§32 A7: pre-approval, so nobody is asked a question the tool would refuse) **and** re-tested in all three handlers, `write_run`'s before its `makedirs` so a refused write cannot conjure the directory. `_shell_approval_check` returns True for any command whose tokens name the segment, placed BEFORE the fallback opt-in and the capability rule but AFTER the mode gate — `SHELL_APPROVAL_MODE = "never"` stays the opt-out it always was (G3). The shell side is an always-ASK trigger and never a deny, because a token check that learned shell syntax would be a parser whose bugs auto-approve (G2); it is sound on the auto-approved INERT tier because EP5 executes argv with no shell, so the tokens ARE the arguments. The approval layer is deliberately untouched — approval ORs can only tighten, and there is no answer in it that DENIES, which is why the deny does not live in `_file_approval_check`. `project_docs._DENIED_SEGMENTS` unions the same list rather than re-spelling the segment.
### The TUI (`tui/`, §16)
A **shell**, not a feature home. D12 makes the CLI a permanent fallback, so anything implemented in `tui/` is invisible to the CLI *and* the research pipeline — which is why the question tool, todo list, goal mode, `/init`, session summaries and cross-thread referencing are all specified in §18/§21/§23/§24 instead. `tui/commands.py` is a registry other sections register into.
**`/config` is the exception that proves the rule** (batch 84). The command lives here; the catalogue, the value grammar and the comment-preserving writer live in `config_edit.py` at the root, so a command-line surface can be added over the same module without moving a line. What is in `tui/` is the two-stage completer, the authority confirmation and the queued restart -- drawing and asking, which is what a shell is for.
§19 follows the same split: `skills/manager.py` holds **no session state** (K3), so the TUI owning `active_skills` is a call-site fact rather than a design one. §25 is the cautionary tale — the pipeline could not reach a whole capability, and undoing that was a section of work.
`/model [[PROVIDER] name]` switches provider/model for the session and is **remembered for the next launch** beside `settings.json`, never in it (§43, RM3 — see the paragraph below). It refuses while `_busy`, re-runs the mount-time effort validation because effort is per-model, and warns rather than refuses on an empty `API_KEY` so a local OpenAI-compatible endpoint stays usable. `/effort` and `/thinking` still persist nothing. `/window` DOES persist, since §44 — but to
`core/model_windows.py`, not here, because `core/compaction.py` reads it and D12 forbids core
importing the shell (WS6).
**Nothing writes to `settings.json`; the theme AND the model are remembered BESIDE it**
(batch 36, extended by §43). `/theme` used to persist nothing either, and a colour scheme
chosen in one session was gone in the next. The rule that stopped it is about the FILE, not
about the preference: a project's `settings.json` beats the user's (D29), and it is
hand-authored, so a session rewriting it is a decision rather than a convenience. So both are
remembered in `tui/preferences.py`'s `~/.config/venastine/ui_preferences.json` — user tier
only, which no project can reach — following `trusted_projects.json` and
`known_mcp_servers.json`: `expanduser` at call time, versioned, failing soft on anything
unreadable. Since batch 45 the four stores share their MECHANICS through root
`json_store.py` (atomic write, versioned fail-soft read) and nothing else: each keeps its
own module, path, `STORE_VERSION` and failure posture, because those were decided
separately. **Batch 108 added the two that hold credentials** -- `credentials.py` and
`security/secrets.py` -- which were the last stores in the project still writing `O_TRUNC`
over the live file, so a crash mid-write destroyed every stored API key or secret; the
helper's own docstring had argued that exact case since batch 45. That extraction is also what made `known_mcp_servers.json` atomic — it was
the one of the four writing a CONSENT record through a bare `open(path, "w")`, while its
three siblings each explained at length why that is unacceptable. `/effort` and `/thinking` are unchanged.
**The model has a second reason not to be written into that file** (RM3): a project's
`.venastine/settings.json` is inside D17's trust content hash, so a session writing
`default_model` there would re-trigger the trust prompt at the next launch — and the
user-tier copy would be outranked by any project that sets the key, so the switch would
silently fail to stick in exactly those projects. The pair is restored WHOLE or not at all (a
model name is meaningless against the wrong provider), a `--provider`/`--model` flag pins the
launch (`cli_pinned`, passed from `main()` because the resolved pair can no longer say whether
a flag spoke), a remembered provider absent from `providers.json` falls back, and the mount
banner says `(remembered)` when the restore fired. Both records share one file, so both
writers read-modify-write and `STORE_VERSION` stays 1 — the model keys are optional, and
bumping would have discarded every user's theme once (RM4).
Three things about the THEME half are load-bearing:
- **The store records the theme AND the `tui.theme` value it was chosen against.** A remembered
choice outranks `settings.json` only while that value still says what it said; edit it, add it
or remove it and the file re-asserts itself. Same staleness rule as `workspace_trust`'s content
hash and the MCP store's entry digest. The key is the RAW settings value, so `None` (no theme
named) stays distinguishable from an explicit `"dark-plain"` — collapse them and *adding*
`tui.theme` becomes a silent no-op.
- **The hook is `watch_theme`, not `_cmd_theme`.** ctrl+p's command palette is enabled and sets
`App.theme` directly, offering Textual's own built-ins beside the fourteen. Textual calls
`_watch_theme` (its restyle) and then a subclass's `watch_theme`, so one hook covers both
routes — and the restore therefore validates against `self.available_themes`, not
`themes.resolve()`'s whitelist, which knows only this project's names.
- **`_theme_persisting` arms at the END of `on_mount`.** `App.theme` is a `Reactive` with `init`
on, so its watcher fires during mount as well: without the guard every launch records the theme
it merely restored, and a default becomes indistinguishable from a decision. `App.theme` also
RAISES `InvalidThemeError` on an unregistered name, which is why a stored theme this build no
longer has falls back rather than crashing at mount.
`tui/themes.py` is untouched and stays "pure presentation -- no import of core/, no harness
state"; the store is its own module. `tests/conftest.py`'s autouse `isolate_ui_preferences` is
not optional hygiene: the mount path READS the store, so without it
`test_theme_preference_from_settings_is_applied_at_mount` passes or fails depending on whether
the person running the suite had ever typed `/theme`.
**Logging and Textual cannot share the terminal.** `main.py` calls `configure_logging(stderr=False)` immediately before `run_tui()` — not at the top of `main()`, because pre-mount messages (db creation, the trust prompt, MCP connections) genuinely want stderr. `TranscriptLogHandler` routes WARNING+ into the transcript so dropping stderr does not just make warnings invisible; it is attached in `on_mount` and **removed in `on_unmount`**, or it keeps a dead app alive and stacks one handler per instance across the test suite. **`on_unmount` is also where the permission channel is released and `_shutting_down` is set** (batch 145, S14-01): `exit()` is every DELIBERATE route out and not the involuntary one — textual's `_handle_exception` reaches `_fatal_error()` -> `_close_messages_no_wait()` and never calls `App.exit()` — so a crash tore the app down with a worker parked on `channel.get(timeout=600)`, on a non-daemon executor thread that holds `App.run()`. `exit()` keeps its own copy of both lines, because removing them would leave the window between a quit and the unmount with no flag set, and the release reads-and-clears so the second call is free. The ordering this rests on is pinned rather than assumed: `App._shutdown` dispatches `Unmount` THIRD, after `_close_all()` and `_close_messages()`, and measured on the installed textual neither waits for the thread workers — so the release at unmount is early enough for the parked worker to see it. Anything written to stderr while Textual is up paints over the rendered screen and disappears on the next repaint.
**The routed line carries the exception's reason and is redacted** (batch 91). `emit` appends `describe(exc)` when a record has `exc_info` — `logger.exception("Tool %s raised; ...")` otherwise showed a failure with no cause — and passes the whole line through `redact_diagnostic_text`, the log file's rule. Before this the handler redacted nothing: the file had been redacted at its formatter since #132 while the screen got `record.getMessage()` raw. **And "the log file's rule" was two rules until batch 144** (S07-01): this line called `redact_secrets` with no value redaction at all, so the harness's OWN stored value -- the one thing SS61 calls unconditional -- was drawn on screen verbatim while app.log redacted it. All five diagnostic sinks now call one function.
**The transcript renders progressively, and it computes its own line boundaries** (§38, O7).
`Transcript.stream_delta` used to do nothing but `self._pending += delta`, so a plain answer
with no tool calls showed nothing until `TurnFinished` — the widget's own docstring had said
so since §26, and nobody read it as a bug report. `RichLog` appends and cannot rewrite a drawn
row, so a chunk is committed only when it ends where a rendered row ends: everything through
the last newline, and past that a tail longer than one row up to its last space. That second
rule is why the wrap has to be ours — Rich would soft-wrap the row still being appended to —
and it is what makes a single long paragraph stream instead of arriving whole.
An open ``` fence is held until it closes, checked against `_stream_text + chunk` rather than
the whole buffer: a block whose closing fence has no newline after it yet puts the last
newline *inside* the fence, so an otherwise-committable prefix would render an unterminated
opener as plain text and leave the block splitter counting from the wrong place for the rest
of the answer. **`_entries` still holds ONE entry per span, updated in place** — §26's "every
write path goes through `_emit()`" obligation kept by a different route, because appending per
chunk splits a copied answer across `as_text()`'s joins and draws a `venastine ›` label per
fragment.
**That fence hold is a CAP over FIVE constructs now, and the reason was never about fences**
(batch 53, extended by 58). `RichLog` appends and cannot rewrite a drawn row, so *anything*
committed in halves renders as its own source and can never be put right — which is one
sentence about a fence, a table, a heading, a list item and an unclosed `**`.
`markdown.commit_span(committed, pending)` returns the offset past which drawing would split
one of them, and `_commit_ready` cuts there. A cap is strictly better than the rejection it replaces: a paragraph sharing a
buffer with a fence used to wait for the fence to close, and now streams. §38's two pins still
say what they always said, because in both of them the fence starts the buffer and the cap is
therefore 0.
- **A table is held from its header row and released by a line that is not a row** — or by
`flush_stream`, since a table has no terminator. That is the same trade the fence hold
already makes.
- **A line carrying a `|` cannot be classified until the line under it is seen**, so it is
held for one line. `_delimiter_possible` is what bounds that: one character outside a
delimiter row's four-character alphabet settles the line above as prose, so an ordinary
sentence releases on its first letter rather than on its newline.
- **Detection is GFM-strict — a delimiter row is REQUIRED**, and that is a decision rather
than tidiness. A false table is not a cosmetic slip: the cap would hold the stream waiting
for one that is never coming, so a shell pipeline or a `|` in prose would stall the answer.
- **Only the LAST line is capped for a heading or an open mark.** On a line that has ended, an
unmatched `**` is literal on both paths, so holding it would be waiting for something that
already happened.
**And the width rule measures RENDERED cells, which is the half that is useless alone** (batch
53). `**bold**` draws four cells narrower than it is written, so a cut taken in source columns
lands where Rich would not have wrapped — and the streamed rows then differ from the ones
`rerender()` draws from the same text, which is exactly the divergence
`TestAStreamedAnswerRendersLikeAWrittenOne` exists to catch (it has caught three already). The
cap is what makes the measurement well-defined, because no cut is ever offered while a mark is
still open and its drawn width therefore still unknown. `_split_committable`'s `marks=` is
**off by default**, and the default is the right answer for `thinking_delta`: reasoning renders
as prose, marks and all, so measuring it as anything else would describe a rendering that does
not happen.
**`rich.markdown.Markdown` is the obvious next reach and is the one thing that must not be
reached for casually** (batch 53): it re-flows text itself, which is precisely what the cell
measurement above exists to keep under our control, and it takes over the fence handling §26
and §38 pinned. Textual's own `Markdown` is a **Widget, not a renderable**, so it cannot go
inside a `RichLog` at all — adopting it means replacing the transcript, not extending it.
`rich.table.Table` *is* a renderable, which is why a table could be added by the route `Syntax`
already takes.
**Lists, links and two more inline marks render; block quotes and rules still do not** (batch
58). `tui/markdown.py` grew the rest of the grammar it was going to grow, curated rather than
completed. What renders now: a table, a heading, `**strong**`, `` `code` ``, `*emphasis*`,
`~~strike~~`, a list item with a hanging indent, and a bare URL. What does not, deliberately:
block quotes (a bar down the left collides with the thinking span's own bar — TECHNICAL_DEBT 17), horizontal rules (drawing one is inventing furniture nobody typed), heading LEVELS (`#` and
`######` still draw identically, because a ladder that reads as arbitrary is worse than none),
setext headings, indented code as a *highlighted* block, nested marks composing, and anything at
all inside a reasoning span. `rich.markdown.Markdown` is still the thing not to reach for, for
the reasons above.
**A list item is the third construct this widget wraps itself**, after the diff gutter and the
thinking bar, and for the same reason: every rendered row needs a prefix. Rich has no hanging
indent — `Text` has none and `Padding` indents the first row too — so the item's body is
pre-wrapped by `markdown.wrap_display` at the width left after the marker, and continuation rows
are padded to the column the marker's own text starts at. That is why a list item joins the
commit cap: a fragment committed without its marker could never be indented afterwards.
`wrap_display` is ONE pass over ONE `_cut_points` list, shared with `width_split` — calling
`width_split` in a loop rescans from the start of the remainder each time, which is quadratic on
a long line and paid again on every `/theme`.
**The thinking bar is wrapped in `_write_thinking_lines`, so both paths get it** (batch 90). It used
to be cut only by `thinking_delta`'s commit rule, so a replay — `rerender()`, a resumed thread, the
thread viewer — handed a whole entry to one `Text` and Rich soft-wrapped it with the bar on the first
row alone; the newline rule did the same live, whenever one delta carried a long line and its
newline. `markdown.plain_wrap` is `plain_split`'s rule walked once over the line (the rule has one
copy, `_plain_cut`, for `wrap_display`'s quadratic reason), and **an empty remainder after a cut at
the very end of a line is not a row**: `plain_split` hands one back when a trailing space overflows
the width by its own cell, Rich folds that space into the row it ends, and drawing the remainder put
a bare bar under every chunk committed that way.
**And the COMMIT side of that same shape is batch 112** (TECHNICAL_DEBT 24). `plain_wrap`
refuses to DRAW the empty remainder; `_commit_ready` and `thinking_delta` now refuse to COMMIT a
cut that keeps nothing back, which is the only way the width rule can drain the buffer -- a row
that filled exactly, then a space that overflows by its own cell. Committing there buys nothing,
and if the next delta opens with the newline, the newline rule commits a chunk that is only
`"\n"` and both draw paths render it as a blank row the replay does not have (measured: 5 rows
live against 4, and 7 against 6 for reasoning). Holding for one delta puts the newline in the
same chunk, where it ends the line. It is deliberately NOT in `_split_committable`, and the
`limit >= len(self._pending)` clause beside it is why: that function is handed the CAPPED prefix,
so a drained prefix is not a drained buffer, and holding there would make a paragraph in front of
a held table wait for the table to close.
**`line_start=False` says a chunk begins mid-line, and it fixed a shipped bug.** With a prefix of
exactly 77 characters the wrap boundary falls immediately before a `# ` token; the committed
fragment read as the start of a line, rendered as a heading and **ate the hash**, which a
`/theme` replay then put back. Invisible to every assertion over `_entries` and over
`safe_commit_limit` — only the streamed-versus-written row equality can see it. The flag is
derived from `self._stream_text.endswith("\n")`, so it costs no state, and it is what the
HOLD_LIMIT release rides on too.
**`HOLD_LIMIT` is enforced in two places and only works as a pair** (1000 characters).
`commit_span` abandons a line-scoped hold past it and returns `forced`, which reaches the
renderer as `line_start=False`; `list_item()`, `heading()` and the mark scan refuse the same line
at the same limit. Enforce it in the cap alone and the released chunk draws as a list item while
the replay draws prose. Enforce it in the grammar alone and a model that never sends a newline
holds the screen. **A fence and a table are exempt and stay exempt** — both must be drawn whole
to be drawn correctly, and `flush_stream` is their bound. `safe_commit_limit` is now
`commit_span(...)[0]`: one scan, two answers.
**A line indented four columns or more is drawn exactly as written** — no list marker, no table
row, no inline marks. It closes a batch-53 leak (` x = a ** b ** c` drew ` b ` in bold and
lost four characters that were never markup) and it is STATELESS on purpose: GFM tells indented
code from a nested list item by the surrounding list context, and tracking that would put state
across the commit boundary. The price, named and tested: an item nested with four spaces renders
flat, two-space nesting renders. `verbatim()` is asked in exactly three places — `_scan`,
`_is_row`, `list_item` — and **fences are exempt**, which is TECHNICAL_DEBT 14 rather than a
silence.
**A terminal resize re-lays the transcript, and the route is `reflow()` rather than `rerender()`**
(batch 112, closing TECHNICAL_DEBT 23 and 15). RichLog renders at write time and stores
Strips, so nothing re-wraps itself and every row drawn before a drag kept the width it was drawn
at. `rerender()` cannot be the handler: it opens with `flush_stream()`, which ENDS an open answer
span, so a resize mid-turn would split the live answer into two entries under two `venastine ›`
labels. `reflow()` hands the open span's half-drawn line back to `_pending`, moves `_entries[-1]`
with it, replays, and then **commits again** at the new width — that last step because the
handed-back line is usually the whole of an open span, so without it a half-written answer blanks
until the next delta. `_render_entry(closed=False)` withholds the one piece of furniture a
replayed open span must not draw, reasoning's `╰`. `on_resize` is keyed on `_wrap_width()`
CHANGING — a height drag moves no wrap — and debounced on `REFLOW_DEBOUNCE_S`. **RichLog defers
its auto-scroll to after the next refresh**, which is why the reflow turns `auto_scroll` off and
restores the reader's place with `immediate=True`: a position set during the redraw was
overwritten a frame later, and the reader landed at the bottom.
**A URL is its own label, and that is the security rule as a data shape.** `[text](url)` is not a
construct: a bare `http(s)://` URL is detected wherever it appears — catching the one inside the
parentheses for free — and only the URL is marked, so there is no name to hide a destination
behind. The span's TEXT *is* the target, so there is nowhere for a target to differ from what the
reader sees. Ctrl+click opens it (never a plain click), `http`/`https` only, and **ASCII only**:
"visible equals target" is no defence against a homograph, so a non-ASCII URL renders, copies and
stays unarmed. The URL rides in a plain style METADATA key read by `on_click` — never textual's
`@click` action string, which is PARSED, so building one from model output would be an injection
grammar fed by the model. `clickable()` is re-asked in `open_url` because a style in a `RichLog`
outlives the text that made it. Textual dispatches on `style.meta`, not on Rich's `link`; OSC 8
was left off, since a terminal that honours it would double-open what we already handle.
**A tool line arms its URLs too, and takes a DIFFERENT scanner** (batch 65, closing
TECHNICAL_DEBT 16). `LINKED_ROLES` — `tool`, `pipeline_tool`, `tool_error` — go through
`markdown.link_spans`, which recognises URLs and nothing else. Not the prose grammar: a
digest is a shell command or a JSON argument, and `inline_spans` does not merely
over-decorate one, it **eats characters** — `# \`date\`` loses its backticks to a code span
and `name~=*test* ~~old~~` renders as `name~=test old`. A digest that no longer shows
what ran is worse than an inert URL. The harness's own voice (`system`, `error`) is
deliberately not in the set.
**`param_digest` truncates at 60 characters, so the target rides beside the text.**
Most real URLs are longer, and an elided one is refused by the ASCII rule (the ellipsis
is not ASCII), so batch 65 carries `registry.call_links` — `redacted_values`, untruncated —
alongside the line. **That means the visible text is no longer the target, and the rule
it is replaced by is narrower rather than absent: _the visible text tells you the origin,
and the origin is where it goes._** A span resolves only when it is actually elided, when
exactly ONE candidate has the visible part as its prefix, and when the whole authority is
visible — a run cut off inside its host stays literal. Targets are stored in `_links`
beside `_entries` (a rendering fact, not part of what was said), which is why `rerender()`
and `reset()` both have to touch it, and why `ReplayEntry` grew a third element.
**A value carrying a URL is capped at 200, not 60 (batch 96), because the mismatch was
never ours to keep.** The resolution above works — measured end to end, the click opens the
whole URL — but the drawn text `https://host/a/b…` minus its ellipsis is *itself* a valid
URL, and a terminal's own link detector reads exactly that. So ctrl+click offered a
**shorter address than the harness had resolved**, reported as the link going somewhere
wrong. Nothing in the harness could fix it, since the terminal never sees `_links`: the fix
is to stop producing a visible-text/target mismatch for ordinary URLs at all. `_DIGEST_URL_CHARS`
and `_DIGEST_URL_TOTAL` move **together** — a value kept whole at 200 and then cut by the
140-character *total* is the same mismatch one line down — and the total widens only for a
digest that actually holds a URL, so a `write` digest keeps the width it always had. Past
200 the batch 65 machinery above is unchanged and is still what arms the span.
**A URL the harness REWROTE is not a link.** Two refusals added in batch 65, both found
by measurement rather than by reasoning. `clickable` now rejects **userinfo**:
`https://accounts.google.com@phish.example/x` was armed and opened `phish.example` — the
same hidden-destination attack `[label](url)` is refused for, arriving through the URL's
own syntax. And it rejects the **redaction mark**: `https://x/?api_key=[REDACTED]` is ASCII,
https and userinfo-free, and clicking it would send the literal string `[REDACTED]` to a
real host. `REDACTION_MARKER` lives in `safety/policy_enforcement.py`; `tui/markdown.py` is
pure and keeps its own copy, and `tests/test_markdown_render.py` holds the two against
each other. `redacted_values` closes the same door at the producer by dropping any value
the redactor touched, so no rewritten URL is ever offered as a target in the first place.
**`*emphasis*` is narrowed, not reversed, and underscores stay refused.** Measured against a real
CommonMark parser: full GFM renders `call __init__ on it` as a bold `init`, and `2*3*4` and
`x*y*z` with emphasis. So an asterisk WEDGED between two word characters is refused, and `_`/`__`
are not recognised at all — no flanking rule saves `__init__` and it is everywhere in this
domain. The other half of `_em_edge` (an opener needs a non-space after it) is about STREAMING:
without it `a * b` opens a mark that never closes and the cap holds the rest of the paragraph.
`~~strike~~` has no collisions and is the `**` branch again.
**Every table cell is a `Text`, header cells included**, and both halves of why were measured
rather than assumed. A bare `str` handed to a `Table` is markup-parsed by the console that
renders it, and the `RichLog`'s own `markup=False` does not reach inside a renderable: `[bold]x`
renders as `x` — swallowed, nothing raised — and `a[/]b` raises `MarkupError`, which is batch
42's RA1 arriving in the middle of an answer instead of in a modal. `[1, 2]` survives, so the
rule has to be about the TYPE and not about scanning for brackets; the shapes that fail are not
the ones a reader expects to be dangerous.
**The transcript's own vocabulary is three things now** (§41, X1-X7).
- **Colour separates KINDS of line, and the pins measure that rather than "
contrast.** `role_styles`
fills `MESSAGE_ROLES` with pairwise-distinct style strings on all fourteen themes; the accent and
secondary slots are held >= 120 redmean units apart (dark-plain's were 98.5, the tightest pair
of any theme, on the shipped default); and `dim` appears in `system` alone, because it is an
attribute no floor in `tests/test_themes.py` can see. `TranscriptLogHandler` carries a palette
ROLE, not an `is_error` bool -- it had no warning case at all, so every WARNING it routed
rendered as `system`.
- **One checklist vocabulary** (`tui/widgets.py`'s `MARK_*`), shared by `TodoPanel` and
`ResearchProgress`, which had been spelling it two different ways -- `.` meant *pending* in one
and *an already-finished code stage* in the other. Running (`>`) and stage (`.`) are
deliberately not boxes; the reasons are beside the constants.
- **`write` and `edit` render an inline diff**, chat mode only. That is a property of the data,
not a shortcut: research-mode tool lines come from `PipelineEvent`, which carries a pre-digested
string and no params, and both tools are `GRANT_NEVER` so a headless pass cannot call them. The
pre-image is snapshotted at `tool_call_start` and the block drawn at a SUCCESSFUL `tool_result`
-- a call can still be denied, and a diff drawn at the call would show a change that never
happened. **Every rendered row is padded and written with an explicit `width=`**: RichLog
otherwise raises a short row to `min_width` and `Strip.adjust_cell_length` pads the tail with an
UNSTYLED segment, so the background stops at the last character of the source line. The wrap is
ours for the same reason -- Rich's soft wrap returns rows of 78, 75, 78 and 66 cells and the
tint stops wherever the text broke.
**The sidebar SCROLLS, and two panels used to clip instead** (batch 59). Textual's `Vertical`
is `overflow: hidden hidden`, so `#sidebar` silently dropped whatever did not fit: measured at
the 24-row floor with a checklist and a research run both live, `max_scroll_y` was 11 — rows the
container had computed and clipped anyway — and `#research-progress` was drawn ENTIRELY off
screen with its header on the last visible row. It is `overflow-y: auto` with
`scrollbar-size-vertical: 1` now.
- **`overflow-y` on the container, NOT a swap to `VerticalScroll`.** That class is
`can_focus=True` and binds up/down/home/end/pageup/pagedown, so it would add a tab stop and
take the arrows batch 55 gave the suggestion panel. The named trade: the sidebar scrolls by
wheel, not by keyboard. EP3's "a consent surface must not be mouse-only" does not reach here —
nothing is decided in the sidebar.
- **`Static` + `max-height` + `overflow-y: auto` CLIPS, it does not scroll**, and this was EP3's
trap a second time. A `Static`'s `virtual_size` follows its clamped box rather than its text,
so `allow_vertical_scroll` is False and the rows past the bound are gone, not below the fold.
§46 hit it in a permission modal and answered with `ScrollBox`; `#todo-panel` and
`#research-progress` still carried the pairing, and `ResearchProgress.ROWS`' comment claimed
the panel scrolled. Both caps are gone; the widgets' own `ROWS` windows are the bound.
- **The usable width is 19, not 20** — `border-left: solid` takes a column on top of
`padding: 1` — and **18 while the scrollbar is up**. Every "22 columns" comment in
`tui/widgets.py` is about the box, not the budget. `AgentPanel.WIDTH` lays out against 18,
since a row that fits the narrow case fits the wide one.
**Three live figures ride the prompt's BORDER SUBTITLE, and the silence they fill was
built on purpose** (batch 61). `commit_span` holds a table from its header row, and a table
-- like an open fence -- is EXEMPT from `HOLD_LIMIT`, so the gap has no upper bound; beside
that, `on_loop_event_message` pauses the raven on every token delta, because a redraw loop
competing with deltas is the one place animation costs responsiveness. Both are right, and
together they leave the screen most static when the harness is busiest. The meter refills
that spot; it does not reverse the pause.
- **The subtitle, because `border_title` is already the placeholder** (deliberately, not for lack of options -- `TextArea`'s native in-box one drew the hint twice, so it is never assigned), and because it costs ZERO ROWS against the 24-row floor. `RichLog`
cannot rewrite a drawn row, so the live half could never have lived in the transcript --
only the finished figure goes there, under the answer, in the `system` role that is
already in `META_ROLES` so `/copy conversation` keeps excluding it.
- **`_busy` is a PROPERTY now, and that is the whole leak prevention.** It is written at
eleven sites with FOUR distinct turn exits (`on_turn_finished`, `on_one_shot_finished`,
`on_research_finished`, and `_cmd_compact`'s worker `finally`), so a clock started at one
and stopped at another leaks -- batch 60's defect one layer up. The setter is the funnel;
`test_the_busy_flag_is_the_only_way_to_move_the_clock` reads the source and fails if
`_busy_state` is ever assigned outside it, because no behaviour can see the fifth exit
that forgets. A property rather than a rename for batch 54's reason: 89 references across
11 test files, all plain reads and assignments.
- **The rate is an ESTIMATE and says so with a tilde.** `StreamToken` carries no incremental
usage and only THREE of the fifteen configured providers set `supports_stream_usage`, so
an exact live figure would read `0 tok/s` forever on the other twelve -- D21's own
failure mode, correct-looking output, one layer up. The estimate also keeps MOVING during
a table hold, because the deltas arrive whether or not the renderer draws them. The
completion line carries the exact count from `turn_output_tokens` and makes NO TOKEN CLAIM
where the provider reports none; a zero there would say the model wrote nothing.
- **`turn_output_tokens` is a THIRD instrument, not a convenience.** `turn_billed_tokens` is
a spend meter (the prompt is counted again every step) and `turn_new_tokens` a size meter
(it adds the input deltas a tool-using turn brings in). Divide either by a duration and
the result looks like tok/s and is not -- which is TECHNICAL_DEBT 9's misreading
exactly.
- **Updates are EVENT-DRIVEN first and timed second**, which is what makes
`tui.animations: false` cheap to honour: a token delta arrives per chunk even while the cap
withholds, so the figures stay live through the hold with or without the tick. The
`set_interval` timer -- `ANIMATION_INTERVAL`, created paused, resumed only while `_busy` --
covers only a long tool call and pre-first-token latency. The named price of pausing it
when idle: uptime is refreshed at each turn boundary and on each loop event, so a session
left alone shows the uptime it had when the last turn ended.
- **The modal pause is read off `len(screen_stack)`, not off a screen event.** Measured:
`on_screen_suspend` does not reach the App, but the stack depth moves 1 -> 2 -> 1
reliably, so every modal counts including ones added later. `_blocking_modal` -- where all
four asks already funnel -- also calls `set_blocked` DIRECTLY, never through
`call_from_thread`: a quitting app has no message pump left, and marshalling there parks
the worker forever and breaks the shutdown release (caught by
`test_quitting_during_an_attended_research_prompt_releases_the_worker`).
- **`from time import monotonic`, never `import time`.** `mocker.patch("tui.app.time.monotonic")`
resolves to the GLOBAL time module and freezes it for textual's event loop and conftest's
`settle` as well -- measured, the suite hangs rather than fails. The module-local name is a
patch point that reaches nothing else, and it is why `tui/meters.py` can stay clockless.
**The prompt box is a `TextArea`, and `priority=True` is what makes it one** (batch 54).
`PromptInput` (`tui/widgets.py`) wraps and grows to four rows as the user types, then
collapses on submit. It replaced a plain `Input`, which is single-line by construction —
`height: 3`, no wrap, horizontal scroll — so a paragraph showed its last ~70 columns and
nothing before them. `TextArea` is the only multi-line widget in the pinned textual, so
this is a swap rather than a setting; the layout needed nothing, because `#prompt` was
already `dock: bottom` over a `1fr` transcript.
Four things are decisions rather than defaults, all measured against the installed
package (D22's rule) rather than read from docs — and re-measured against `8.2.8`
when the pin moved. Three of the four came back unchanged; the `shift+enter` one
did not, and says so below.
- **`priority=True` on the `enter` binding is load-bearing and reads like caution.**
`TextArea._on_key` maps `enter` to a newline insert and calls `event.stop()` /
`event.prevent_default()`, which beats an ordinary binding — measured both ways: without
the flag `enter` inserts and never submits, so no turn in this shell would ever start.
Dropping it fails three tests in `TestEnterSubmitsAndCtrlJDoesNot` and
`TestThePromptBoxGrowsWithWhatIsTyped`.
- **`ctrl+j` carries the newline, and `shift+enter` is still not claimed — for a
different reason than it used to be.** Textual enabled the kitty keyboard protocol in
its LINUX drivers alone through 6.5, so on the Windows driver a terminal sent a bare
CR for shift+enter and the parser yielded plain `enter` — the box would submit.
**6.6.0 added the protocol to the Windows driver** (`drivers/windows_driver.py` writes
`\x1b[>1u`), so that reason expired, and **batch 81 went looking for what replaced
it**. Every half the CODE owns works: the driver asks for the protocol, the parser
turns `\x1b[13;2u` into `shift+enter`, and `PromptInput` already binds it. **The
obstacle is the TERMINAL now.** That driver reads console records and keeps only the
translated character, discarding the modifier state, so the escape encoding is the
only route a modified Enter has, and a terminal without the protocol still sends a
bare CR. Windows Terminal shipped support in Preview 1.25 and this is developed on
1.24, so the end-to-end press has never been observed here and the claim stays unmade.
`TestShiftEnterIsStillNotTheKey` pins those three dependency facts, so the next reason
cannot expire in silence the way this one did for twenty months. `ctrl+j` is byte
0x0a, parses to
its own key, is bound by neither `Input`, `TextArea`, `App` nor `Footer`, and is
exactly what iTerm2 / VS Code / Windows Terminal emit once configured to send a
newline on shift+enter. **`alt+enter` is the obvious third guess and is a dead
end**: fed `ESC CR` the parser yields no key at all, and a second one behind it
degrades to `escape`, `enter`.
- **`value` is an alias over `.text`, and it is why batch 54 edited no tests.** Roughly
sixty sites across five files say `app.query_one("#prompt").value = …`; none queries it
by class. A one-line property is cheaper than sixty edits made to land a rendering
change — see `tests/BREAKING_CHANGES.md` for what deleting it looks like.
- **`PromptInput.Submitted` is its own message type, and that fixed a live bug.**
`Input.Submitted` BUBBLES past a modal's own handler to the app's (measured: the screen
handler runs and then the app's), so `enter` in `ReviewScreen`'s note box — a screen with
no submit handler at all — reached the prompt's, cleared the note, and dispatched the text
as a slash command if it began with one. A distinct type ends that by construction.
`tab_behavior` stays at its `"focus"` default deliberately: under `"indent"`,
`TextArea._on_key` also swallows `escape`. `ctrl+k` is still bound by the prompt (both
widgets bind it), so §26's `ctrl+l` note above is unchanged. `ctrl+c` was ALREADY shadowed
before batch 54 — `Input` and `TextArea` both bind it to `copy`, and measured, an app-level
`ctrl+c → quit` fired under neither while the box had focus. Batch 57 is what fixed that; see
**ctrl+c is one key with two bindings** below.
**A prompt that "got taller" is not a prompt that wrapped** (batch 54). A `TextArea` with
`soft_wrap` off grows a HORIZONTAL SCROLLBAR, so a 186-character line drew *two* rows at
`max_scroll_x == 133` — one text row and one bar — against four rows at `max_scroll_x == 0`
wrapped. The first draft of `test_a_long_line_wraps_with_no_newline_in_it` asserted only
that the box got taller and passed with wrapping switched off, which is the whole defect the
batch exists to remove. `max_scroll_x` is the assertion that tells them apart, and the row
count is pinned EXACTLY and below the cap, so "wrapped" cannot be confused with "hit
`max-height`".
**Assignment is the API, typing is the user, and `load_text` is where they part** (batch 55).
This is the seam the whole suggestion panel hangs off, and it is not obvious: setting
`prompt.value` posts `TextArea.Changed` *exactly* as a keystroke does (measured). Since `enter`
always completes while the panel is open, a panel driven straight off `Changed` would open in
the ~73 `query_one("#prompt").value = "/..."` sites across four test files and turn each one's
single `press("enter")` into a completion instead of a dispatch — every bare assignment in the
suite is an exact command name. Textual draws the same line itself: `_replace_via_keyboard`'s
docstring says "as opposed to the API". That method is the obvious seam and the WRONG one —
measured, backspace does not go through it, and a panel that ignores deletions fails the
feature outright. `PromptInput.load_text` is the right one: public, and the single funnel
behind both `.text =` and `.value =`. Delete the override and `test_assigning_the_value_does_
not_open_it` goes red first, before the four files that follow.
**The suggestion panel owns the highlight because it owns the width** (batch 55). How many
entries fit is a function of the rendered width — at 80 columns the main column is 54, most
entries wrap to two rows, and a bare `/` therefore shows four of twenty-seven under an eight-row
budget. Split the selection from the budget and the prompt can highlight a sixth entry the
panel had no room for: an invisible selection that `enter` would then complete. `SlashSuggest`
exposes `chosen`, `move()` and `shown`, and that is the whole interface. The budget stops at
the first entry that would overflow rather than skipping it, so the list stays contiguous and
alphabetical, and the border title counts what was DRAWN against what MATCHED (`4 of 4` under
`/t`, not `4 of 27`, which would claim candidates that do not exist).
**`up`/`down` are ordinary bindings on `TextArea`; `enter` is not** (batch 55). `enter` needs
batch 54's `priority=True` because `TextArea._on_key` intercepts it. The arrows do not: they
are plain `BINDINGS`, so a subclass claims them by overriding `action_cursor_up` /
`action_cursor_down` — which is strictly better than re-binding the keys, because `super()` is
still there to hand them back the moment the panel is closed. Batch 54 made this box
multi-line, so a prompt four rows tall that cannot move its own cursor is the regression this
shape prevents. `select=True` (shift+up) is excluded: that is a text selection and stays one.
`tab` and `escape` are gated by `check_action` returning **`None`, not `False`** — `False`
disables the binding, `None` declines it and lets the press carry on to the focus system.
**Two rendering traps, and both were found by drawing rather than by reasoning** (batch 55).
A `Static` RE-WRAPS a row you already wrapped: entries wrapped to `width - 2` and then drawn
under a four-space continuation gutter rendered as THREE rows, so the height was wrong by one
per entry. Both gutters must fit inside the wrap width, and the outer `Text` carries
`no_wrap` / `overflow="crop"` so a miscalculation clips visibly instead of reflowing
invisibly. And `Text.truncate()` on a WRAPPED line does nothing — the overflow is in the lines
that were dropped, not in the line that was kept — so the ellipsis is appended deliberately.
The pin for the first is swept over every registered command WITHOUT a pilot, and that detail
is the lesson: the pilot version of that test **survived the mutation it was written for**,
because only `/research` overflows at 80 columns and a bare slash does not show it.
**A trailing space is invisible to `strip()`, and that is why `matching()` lstrips** (batch
55). The predicate first read the line the way `on_prompt_input_submitted` does — `strip()`,
then reject internal whitespace — which is right for dispatch and wrong here: `"/copy "`
stripped back to `"/copy"`, so the panel stayed open over a line that had already moved on to
its arguments, and `enter` would have completed `/copy` on top of itself rather than sending.
Once a space is typed there is nothing left to complete, whichever end of the token it is on.
The leading half stays tolerant, because `dispatch` tolerates it and `" /help"` genuinely runs.
**Past the first space, the command answers** (batch 84). `SlashCommand` grew an optional
`complete(argument) -> [SlashCommand]`, and a command without one behaves exactly as every
command did before the field existed — which is what keeps `"/copy last"` offering nothing.
The rows a completer returns are named for the WHOLE line, so `/config max_tokens` draws and
completes with no change to the panel at all. Two details are load-bearing. The split is on
a SPACE and nothing else, because `dispatch` partitions on a space and nothing else — a line
with a newline in it is not a command line there and must not look like one here. And
`_complete` sets `_completing` rather than posting the new list itself: the assignment it just
made queues a `TextArea.Changed`, so anything posted from inside `_complete` is handled
BEFORE that Changed arrives and is then overwritten by the empty list its api-edit branch
posts. Measured: the panel came back with 133 rows and was cleared a tick later.
**The suggestion window is DERIVED from the selection, never stored beside it** (batch 56).
Batch 55 wrapped the highlight at the last *drawn* entry, so a bare `/` matched twenty-six
commands, showed four, and left twenty-two unreachable by keyboard under a title that said
twenty-six. The fix keeps the same split one step further: `move()` picks a command out of all
the matches and `_budget()` scrolls the window the least it can to keep that command visible,
because where the window has to sit is a function of the WIDTH and only `_budget` knows it.
`_first` is therefore recomputed on every measurement rather than maintained — two numbers that
must agree are the shape §22 spent a section removing. `visible` is the accessor for what is on
screen; `_matches[:shown]` silently assumes a window starting at zero and stopped being true
here.
Minimal in both directions: `first = min(self._first, self._selected)` and then advance while
the selection falls outside. Drop the `min` and a reader who arrows back up is stranded at the
bottom of a list they scrolled into; recompute from zero instead of from where the window
already is and the list jitters under them, scrolling for a selection it was already showing.
Both are pinned.
**A panel that changes SIZE cannot refresh itself, and cannot move itself** (batch 56, and this
is the half that is easy to get wrong). The window slides by whole entries of one or two rows,
so the panel is four entries over seven rows at the top of the list, four over eight one step
later, five over seven in the middle — measured. Two consequences, neither optional:
- `move()` must `refresh(layout=True)`. Measured: with eight rows queued after a plain
`refresh()`, the widget stayed *measured* at seven and the extra row was clipped where
nothing could see it — no exception, no visible error, just a missing entry.
- `move()` must be reached through `app.py`, not called from the prompt. A resize takes rows
from the transcript, and textual does not re-pin a scroll on shrink; measured, a
seven-to-eight-row step unpins it exactly as opening the panel does. Batch 55's re-pin
therefore became `_change_suggestions(panel, apply)`, and the arrow keys post
`SuggestionsMoved` so they land inside it. Called directly, the conversation creeps upward on
most keypresses and no test about suggestions would notice.
**The title carries a range only when the list can scroll** (batch 56): `1-4 of 26` when there
is somewhere to be, and `4 of 4` / `1 of 1` when everything already fits. A range on a list
with nowhere to go is noise, and the rule keeps every case shipped in batch 55 reading exactly
as it did.
**Prompt recall is on `ctrl+↑`/`ctrl+↓`, and the pair was chosen by elimination** (batch 63).
Until batch 63 a submitted prompt existed nowhere a person could reach — `on_prompt_input_submitted`
cleared the box and that was that — so a misfire on a long `/research --attended --grant …` cost the
typing. The keys were picked by enumerating `screen.active_bindings` with the prompt focused, which
is App, Screen and `TextArea`'s forty-odd editing bindings in one list rather than a docs page.
**`ctrl+p` is not available and must not be taken.** It is textual's `COMMAND_PALETTE_BINDING`,
bound `priority=True`, and the palette is enabled here deliberately — `watch_theme` exists *because*
it sets `App.theme` directly, bypassing `/theme`. The ClassVar is overridable, but `ctrl+shift+p` is
indistinguishable from `ctrl+p` without the kitty protocol — which textual turned on in its Linux
drivers alone until **6.6.0 added the Windows driver**. So relocating it would have deleted the
palette on Windows outright, and now merely makes it depend on whether the terminal implements the
protocol, which is a worse trade than leaving the binding where textual put it rather than a
different one. The plain arrows were the other
candidate and already mean two things — the cursor in a box batch 54 made multi-line, and the
suggestion panel's highlight since batch 55 — so the shell convention (recall only when the cursor is
already on line 1) would have been a third meaning gated on a position.
**Both movers take the box's current text, and that parameter is the whole design.** `tui/history.py`
records what it last handed out; a `current` that differs means the user has typed, so the move ends
the old walk and starts a new one with the edit saved as the draft. Recall, edit, recall again: the
edit comes back under `ctrl+↓` instead of being stepped over. The obvious alternative — hooking
`PromptInput.on_text_area_changed`, where batch 55's `_api_edit` flag already separates assignment
from typing — is the wrong seam, because that message is POSTED, so the app would hold a flag across
an async hop, and the panel's own `SuggestionsChanged` fires on both halves of the split anyway. The
stated limit: a reader who retypes an entry character for character continues the old walk, because
position is not knowable from text.
**The walk STOPS at the oldest rather than wrapping**, which is the opposite of the suggestion panel
one widget away, and deliberately: that list is a menu, where wrapping is how the far end is reached
quickly, and this is a walk backwards through time under a key that gets held down.
**`check_action` returns `None` here, where ctrl+c needs `False`** — the same distinction used the
other way round. `active_bindings` KEEPS a `None`-refused binding marked disabled, so the footer greys
the entry rather than dropping it and reflowing, while `_check_bindings` still declines the press. One
return value buys both halves. And `remember()` must be followed by **`refresh_bindings()`** on the
empty-to-non-empty transition: the `Footer` recomposes off the screen's bindings signal and nothing in
a submit raises it, so without it the entries stay grey for the session while the keys work — a state
only the drawn row can see. Measured: five footer entries plus the palette fit at 80 columns with 15
to spare and clip at about 68.
**The bottom border says which keys the panel spends** (batch 62). Four keys change meaning
while the suggestion panel is open — the arrows move the highlight, `tab` and `enter` both
complete, `escape` dismisses — and none of them was written anywhere in the TUI. `enter` is the
one that costs something: it completes rather than sends, so a fully typed `/help` takes two
presses and the first changes nothing a reader can see except a trailing space.
Three placements were measured and two are closed. Not the border TITLE, which the count holds:
at 80 columns the label budget is 52 cells, the count is 25 and the hint is 42. **Not the
FOOTER, and that one is worth recording** — `Screen.active_bindings` drops a binding only on
`check_action` returning `is False`, and batch 55 needs `None` for `tab`/`escape` so they still
reach the focus system with the panel shut, so `show=True` would park two greyed entries in the
footer permanently. It is batch 57's asymmetry from the other side: `False` and `None` differ,
and which one is right depends on whether the key has a life outside the panel.
The hint shortens by STATE first and by width second. The state half is the title's own rule
applied to a control: with one match `move()` wraps to the command it is already on, so
`↑↓ move` would advertise a key that does nothing — the same claim the title stopped making
when it dropped the range from `1 of 1`. The width half drops whole parts from the LEFT rather
than truncating, because an ellipsised `esc dism…` is furniture and not help; what survives the
narrowest terminal is therefore how to get rid of the panel, which is the useful key at a width
where the entries are unreadable anyway.
Two numbers in that are measured rather than reasoned. **A border label is truncated at the
widget's outer width minus six**, so with `border: solid` and `padding: 0 1` the budget is two
cells narrower than the width `_budget` is handed — change either in `app.tcss` and the
arithmetic is wrong, which is why the pilot test asserts the drawn row carries no ellipsis. And
**an unstyled border SUBTITLE renders in `$primary`, not `$text-muted`**: without a rule the
hint would be brighter than the count on the same box, and the less important of the two labels
would be the louder one. `border-subtitle-align` needs no rule — textual already defaults a
subtitle to the right, which is where `#prompt`'s meter sits one box down, so titles read left
and status reads right throughout.
**ctrl+c is one key with two bindings, and `check_action` picks which is live** (batch 57).
A `Binding`'s description is fixed at class definition, so relabelling the footer entry —
`Quit` at rest, `Press again to quit` while armed — is not a matter of editing anything: it
is two bindings on `ctrl+c` with different descriptions and different actions, and
`check_action` enabling exactly one. **It must return `False`, never `None`.**
`Screen.active_bindings` skips a binding only on `is False`; `None` leaves it in the map
marked disabled, and the map is keyed by KEY, so the first-listed binding keeps the slot
either way. Under `None` the DISPATCH is still correct — `_check_bindings` walks past a
refused action to the next binding for the same key — and the footer shows the wrong label
for the rest of the session. Measured, and the whole reason the footer test asserts on the
drawn row rather than on the binding object, where the bug cannot be seen.
`refresh_bindings()` is what repaints; the `Footer` recomposes off the screen's bindings
signal and nothing else in the gesture raises it.
**`priority=True` is what makes `ctrl+c` reachable at all, and it is why the APP does the
copy** (batch 57). The key meant four different things depending on focus: `copy` in the
prompt (and nothing whatsoever there without a selection), an immediate quit one `tab` away,
nothing at all under a modal — `ModalScreen` blocks non-priority app bindings — and `copy`
again in a modal's `Input`. Only a priority binding reaches past all four. Measured, it also
pre-empts `TextArea.copy` completely: the clipboard stayed empty. So `_copy_selection()`
hands the copy back, and the rule the gesture rests on is **ctrl+c copies when there is
something to copy and starts a quit when there is not** — a press that copies never arms and
never quits, in either state, so no sequence of copies can end the session. `ctrl+q` is
textual's own priority binding and already quit from everywhere, modals included; it is
untouched and now documented.
**escape is the SECOND key with two app bindings, and it is also the key the prompt may want**
(ROADMAP_v3 §61, CN8, batch 124). `close_thread_view` and `cancel_turn` share it, and the pair
works exactly as the ctrl+c pair above does — including **`False`, never `None`**, for the reason
spelled out there. What is different is the third claimant: `PromptInput` binds escape to
dismissing its suggestion panel and gates that with `check_action`, whose own comment is the
licence this took — *"with no panel open they are the keys they have always been: tab moves focus,
escape reaches whoever wants it."* So **no `priority=True`**, which is the opposite of ctrl+c and
for a reason that is not a preference: priority would reach past the prompt and break the panel
instead of cooperating with it. Precedence is therefore suggestion panel, then thread view, then
cancel, and it needed no arbitration written anywhere. The one-sentence rule §47 set for this key
survives: escape means back to the conversation, and stopping a turn is that.
`cancel_turn` is also the one binding here that **cannot show a footer entry**. Greying rather than
reflowing needs `None` from `check_action`, `None` keeps the key slot, and keeping the slot would
shadow `close_thread_view` — so `show=False` is forced, and a wrong label on a live key would be
worse than no label.
**An alias is a field on the command, and the LISTS stay canonical** (batch 57).
`SlashCommand.aliases` plus a second index in the registry; `get()` falls back to it so
`dispatch` needed no change. `all()` and `names()` are canonical, and `matching()` returns
alias rows **only once a prefix is typed** — a bare slash is the MENU, which names each thing
once, and `/ex` is a guess, whose answer is whether it works. So `matching("/") == all()`
still holds exactly, and no window position batch 56 measured moves. Registering `/exit` and
`/bye` outright instead would have put `bye` second in that menu, ahead of `/claims`. An
alias row is a synthesized `SlashCommand` carrying the alias as its NAME, which is what makes
completion fill in what was being typed rather than correcting it to `/quit` — and the panel
needs to know nothing about aliases at all. Collisions raise at registration in both
directions, because `get()` prefers real names: the alias would otherwise just stop working,
silently.
**Thinking has two forms and one closing path** (§38, O6/O8). `tui.show_thinking` (default
`True`, defaulted in `tui/app.py` beside `animations` rather than in `config.yaml`) renders
reasoning inline as a bar-prefixed block; off, it collapses to `ThinkingIndicator`, a `Static`
under the transcript with an animated ellipsis — an ellipsis cannot animate inside a `RichLog`.
Both end through `VenastineApp._end_thinking()`, which every non-thinking event calls, so a
flipped setting cannot leave one of them running. `_write_thinking_chunk` closes an open answer
through `_close_answer()` and **not** `flush_stream()`: the latter re-enters `end_thinking()`
at a point where `_thinking_pending` already holds the rest of the chunk being committed, which
emits that remainder as a second entry and then drops it. `/thinking [on|off]` is the session
toggle and persists nothing, matching `/effort`.
**The `venastine ›` label belongs to the TURN, not to an answer span** (§43, RM1). §38 made
reasoning visible and left the label where it had always been — drawn by
`_write_stream_chunk` when the first committable chunk of ANSWER text arrived — so on the
shape §38 itself made normal the whole thinking block rendered above it, under `you ›`, and
read as the user having done the thinking. `_open_label()` now draws it once above the
turn's first model output, reasoning, text or a tool call included; a tool line, a diff or
a notice inside the turn does not re-open one, and the next `you ›` retires it. The tool
call half is not decoration on the shape MCP made normal: a call made before any prose IS
the turn's first output, and the pre-amendment opener set left such a turn unlabelled
until the prose followed, so the call read as if the user had made it. The research
pipeline's tool lines carry their own role (`pipeline_tool`, styled as `tool` by alias and
classified CONVERSATION so `/copy` is unchanged) precisely so they cannot open one — the
run's label belongs to the report. Placement is a pure function of the role sequence in
`_entries`, which is what lets `rerender()` re-derive it after a `/theme` instead of
remembering where the labels went — a replay that moves a label is the same defect as one
that reflows a paragraph.
Two things in `tui/app.py` are load-bearing and easy to break silently:
- **`run_worker(..., exit_on_error=False)` + `on_worker_state_changed`.** Textual's default tears the whole app down on a transient worker exception.
- **Every permission dismissal path must put a boolean on the channel.** The worker blocks inside `_run()` on `permission_channel.get()`; a dismissal carrying `None`, or one that puts nothing, hangs it forever. That is why §16's AC2 tests assert on the *dispatched tool*, not on the modal rendering.
**A modal's LAYOUT is part of its contract, and asserting on its text is not testing it**
(§46, EP1/EP3). This cost four consent surfaces at once. `#permission-dialog` had
`grid-rows: auto 1fr auto` with `height: auto`, and a fractional row inside an auto-height
container resolves to **zero** — so `#permission-params` drew at `region.height == 0` and
`virtual_size.height == 0`, and every tool approval, the subagent sign-off, `/init`'s
confirmation and `/init`'s project-kind question rendered their titles, their buttons and
none of their substance. It shipped because every assertion about these screens reads
`.visual._renderable.plain`, the string a widget was BUILT from, which a widget of zero
height still reports in full.
Three rules follow, and the first two are the ones a reasonable person gets wrong:
- **A bound the PARENT cannot see is not a bound** (batch 49, amending EP3). The sentence
that used to be here — "`height: auto` on a scrollable container is also zero, so a bound
that should scroll needs a DEFINITE height" — is false, and it is what caused the next
defect. The zero was EP1's `1fr` ROW; the box had been measured inside one. Measured again
with the row `auto`: `height: auto` draws the content, `height: N` draws N **and reports
the content height upward anyway**, and `height: auto; max-height: N` draws N and reports
N. So a grid row or a vertical stack sized itself to the full payload, the dialog hit its
`max-height`, and the LAST children — the buttons, always — were clipped off. Nine of
fourteen modal shapes lost at least one, `shell`'s approval losing both. Bound a scroll box
with `height: auto; max-height: N`, which also shrinks for a short payload.
- **`max-height` on a `Static` truncates without scrolling.** A Static's `virtual_size`
follows its clamped box rather than its content, so `max_scroll_y` is 0 and the hidden rows
are gone, not below the fold. Bound a `ScrollBox` and put the text in an auto-height child.
- **Assert on the rendered REGION.** `test_every_modal_actually_draws_its_body` walks every
modal in `_modal_cases()` and fails any widget that carries text and was allocated no rows;
`test_every_modal_is_centred_bounded_and_styled` still pins the 80×24 floor beside it.
- **And assert on the BUTTONS, which neither of those can see** (batch 49). One pins where
the dialog sits and the other that its body drew rows; a modal can satisfy both while
showing no way to answer it. `test_every_modal_keeps_its_decision_on_screen` requires every
decision button inside its dialog and every option button at least reachable in its scroll
box. It runs over a case table that now includes content long enough to OVERFLOW — the old
one was every screen at its smallest ("Body text.", one claim, two short options), which is
why three parametrised tests were green while nine shapes were broken. A domain too small
to fail is the same trap batch 15's citation check hit from the other side.
- **The consent surfaces open focused on the DECLINING button** (batch 49), because focus is
what Enter fires and `ConfirmScreen` opened on its affirmative one. `GrantPickerScreen` and
`ProjectKindScreen` have no declining button — escape declines on both — so they must at
least not open on an answer. `Button:focus` also drops textual's `reverse`, which inverts
the label alone and reads as a highlighted selection rather than as a cursor.
### Agents (`agents/`, §18/§32)
An agent is a `.md` file with YAML frontmatter: a system prompt body plus a tool policy. `/agent
<name>` makes one active for the rest of the session, `spawn_subagent` delegates one task to one,
and `agents/manager.py` composes the scope for both. Five things a definition may say are
load-bearing, and only one of them can widen anything.
- **`spawnable` is mandatory at the harness tier** (§32 A3). `assert_spawnable_declared` raises at
discovery, naming every offender, if one of our own files omits it; a user's or a project's file
that omits it reads as `false`, because a stranger's silence must get the safe answer. **It is a
REFUSAL since A16** (`docs/ROADMAP_v3.md` §62), and this bullet said "what the model is TOLD
exists and nothing else" for as long as that was true: `refusal_reason` never read the field, so
`spawn_subagent` would spawn any agent the loader had discovered, by name — including one A3 had
deliberately left off the catalog. Silence therefore bought INVISIBILITY rather than
unreachability, which ranks an adversarial author above a careless one. Now
`agent_catalog_text()` and `refusal_reason` give the same answer: `agent_catalog_text()` lists
only spawnable agents, a spawn of anything else is refused, `/agent` still lists them all (it
goes through `manager.names()`, which consults neither), and C6 still caps what any of them may
do.
- **It answers a question about the INPUT, not about trust** (§32 A4, #69). `spawn_subagent` passes
exactly one thing: `params["task"]`, as the first message of a FRESH thread. An agent whose body
opens "read the thread so far" cannot be fed that way, and spawning it does not fail — it returns
a confident answer about the task description, and the parent has no way to tell. Degraded rather
than broken is what makes it worth suppressing.
- **`spawn_targets` names WHICH agents a definition may spawn**, and it is a REFUSAL rather than a
narrowing — which is the entire reason it exists, because C6 below already narrows and that is
the wrong answer here. A parent that should not be delegating a kind of work at all gets, under
C6 alone, a child that runs anyway with pieces missing: `plan` spawning `build` returned a
`build` that could not `write` or `edit`, degraded the way A4 describes and invisible from both
ends. Undeclared means no opinion (every spawnable agent), `[]` means none, and a declared list
composes by intersection like everything else here, so it cannot be widened by nesting. Read in
two places for one reason: `refusal_reason` refuses the spawn, and `agent_catalog_text()` leaves
the agent off the list in the first place, since advertising what will be refused is D24's defect.
- **A child's tools are its own ∩ the parent's, never a union** (C6), and `approval_overrides` union
the same way (decision S2). §18's spec sketch built the child's overrides from the agent definition
alone, which drops the parent's and reopens the escalation C6 exists to prevent, one field over.
Only `True` entries carry, so unioning can only tighten. **BOTH sides are filtered, and until batch
150 only one was** (A28): the parent's dict was filtered and the agent's merged raw, so a `False` in
a child's frontmatter overwrote the parent's `True` and this paragraph was false of the code it
describes. Three of the four quadrants had tests. The cost was wider than the gate --
`candidate_approvals` reads the same dict and is both the sign-off notice and the grant, so the
loosened tool left the candidate list and the user was neither asked nor told. `active_context`
composes nothing and keeps its unfiltered copy deliberately, pinned as a control: at depth 0 there
is no parent to overwrite and the next hop's filter drops a `False` anyway. **The VALUES are
validated at parse time too**, beside `use_memory` and `use_project_context`, for the reason stated
there -- the mapping was checked and its contents were not, so `web_search: "false"` parsed to the
string. No value ever widened a gate (approval ORs), so that half is a type boundary, not an
escalation.
- **`max_steps` is the one frontmatter field that REPLACES rather than narrows.** Both call sites
are `agent.max_steps or config.MAX_ITERATIONS` (`tui/app.py`, `agents/subagent_tool.py`) with no
clamp, so a declaration above the ceiling RAISES it. Every other field in a definition can only
tighten, which is exactly why this one is written down. An unusable value is repaired to `None`
with a warning naming what it read (#45); a valid large one is honoured.
- **`max_steps` is also the one frontmatter field that REPAIRS rather than skips** (A55, locked,
batch 155). Every other typed field in `_parse_md_file` warns and skips the whole file, including
the three the function's own "ELEMENT TYPES TOO, not just the container" comment was stated over
and never applied to: `model`, `provider` and the ELEMENTS of `allowed_tools`, which sat under
that comment with the container checked and the contents not. The asymmetry is deliberate and the
reason is which way `None` reads. `max_steps` has no honest reading for a bad value, so a repair
invents nothing; `model` and `provider` DO -- `None` means "inherit the session's" -- so repairing
one would leave the agent available and running on a model nobody chose for it, and an agent file
names a model when being a *different* model is the point (§11's critic rationale is the sharp
case). `allowed_tools` skips for `approval_overrides`' reason: half a whitelist is worse than no
file at all. Measured before the guard: `model: [a, b]` loaded clean and raised `TypeError:
unhashable type: 'list'` out of the effort cache's `(provider_name, model)` key; a hashable
non-string in `allowed_tools` survived `set()` and matched nothing, while an unhashable one raised
out of `AgentManager.active_context` and `child_context` -- the single place that `Optional[list]`
becomes `ToolContext.allowed_tools`'s declared `Optional[set[str]]`.
- **`use_project_context` opts into the project's root `AGENTS.md`** (§44 moved that file out of
`.venastine/`); **`use_memory` defaults to `True`** and is §21b's opt-out. Both are VALIDATED as
booleans rather than coerced — `bool("false")` is `True`, and these are the two fields a
restrictive definition uses to opt out, so a coerced string would invert a deliberate choice
silently.
**Eleven ship, six of them spawnable** (§32 A12, batch 51; `build`/`test`/`writer`/`general` added after).
| agent | reached by | what a spawn would have to carry |
|---|---|---|
| `compactor` | `core/compaction.py` | a stretch of transcript |
| `initializer` | `project_init/generator.py` | a document manifest |
| `pipeline-reviewer` | `core/reasoning/review.py` | a finished `PipelineRun` |
| `grill-me` | `/grill-me`, one shot in the live thread | the thread |
| `plan` | `/agent plan` | the conversation |
| **`explore`** | **`spawn_subagent`**, `/agent` | — a task string IS its input |
| **`review`** | **`spawn_subagent`**, `/agent` | — a task string IS its input |
| **`build`** | **`spawn_subagent`**, `/agent` | — a task string IS its input |
| **`test`** | **`spawn_subagent`**, `/agent` | — a task string IS its input |
| **`writer`** | **`spawn_subagent`**, `/agent` | — a task string IS its input |
| **`general`** | **`spawn_subagent`**, `/agent` | — a task string IS its input |
Until batch 51 the spawnable column was empty, so `agent_catalog_text()` returned `""` and
`## Available agents` had never appeared in a prompt this harness produced — while
`spawn_subagent` was advertised in every attended run telling the model to name an agent
"exactly as listed in the Available agents catalog". The research passes are unaffected either
way: `spawn_subagent` is unreachable headless (R16), so `with_catalogs` suppresses that section
in all ten. The SKILL catalog is the opposite and deliberately so — `load_skill` is callable
headless, so all fourteen skills ride every pass.
**Leaves, one branch, and the superset that is now scoped rather than total.** `build`, `test`, `writer`, `explore` and `review` are C6 leaves with no `spawn_subagent` — the spawning discipline lives in `general`, the only spawnable branch, so it is taught once rather than repeated in every leaf. `plan` keeps its own `spawn_subagent` for interactive `/agent` use; it is not spawnable, so the two branches never compete for one route. A13 still holds — an agent's whitelist must be a strict SUPERSET of every agent it may spawn, because C6 intersects a child's tools with its parent's and a spawn of `build` without `write`/`edit` hands back a degraded agent with neither side able to tell — but **`plan` satisfies it by not being able to reach `build` at all**, not by holding the two tools. That is the decision and not a technicality: the plan is the deliverable, a person reads it and iterates before anything is built, and an agent that could pass the work straight to `build` has skipped the step it exists to create. So `plan` declares `spawn_targets: [explore, review, writer, test]` — investigation, which is what planning needs — and `general`, which holds `write`/`edit` for PASS-THROUGH rather than its own use, is off that list too, or the restriction would have been one hop deep. `general` itself declares no targets and is the superset of every leaf. The invariant is swept over the whole roster rather than asserted of two agents by name, which is how it broke: batch 51's expansion added `build`, and `plan`'s comment was updated to claim a superset the whitelist under it never gained. And the leaves name `read`/`write`/`edit`/`shell` although
`config.ToolPermissions` denies all four to everything (A15) — on a stock install they work from `read_project_doc` (plus the network tools where declared) and become code agents only where the operator has enabled file access. That is deliberate: `registry.schemas(context)` filters by the same
predicate, so nothing uncallable is advertised, while an agent that omitted them would stay
crippled on an install where the operator had enabled them.
**AGENTS RUN AS A TREE, AND THE PANEL THAT DRAWS IT STILL CARRIES LIFECYCLE ONLY**
(batch 59, changed by §47 slice 8). This entry used to read "ONE AGENT RUNS AT A TIME. What
nests is a STACK", and it was measured rather than assumed, because the question comes up
whenever someone reaches for concurrency: `core/loop.py` dispatched tool calls in a plain
`for` loop, `spawn_subagent` BLOCKED on the child run, every TUI worker path is `_busy`-guarded,
and even ensemble Pass 1 is a loop with a comment saying why. Two `spawn_subagent` blocks in one
model turn ran one *after* the other.
NA9 changed the first clause and nothing else. `core/loop.py` now PARTITIONS a response:
consecutive calls whose `ToolSpec` declares `parallel` run together in a pool bounded by
`SUBAGENT_MAX_PARALLEL` (3), and every other call stays strictly sequential in its original
position. So the maximum simultaneously-live set is root + `SUBAGENT_MAX_PARALLEL` peers, each
still able to nest to `SUBAGENT_MAX_DEPTH`. **`_busy` is unchanged and still correct**: it guards
the TUI's TURN, not the spawns inside it. **A response with nothing parallel in it takes the same
path it always did**, and that is structural rather than incidental -- a group of one skips the
pool and the gate entirely, which is what makes every pre-slice-8 test of that loop a test of the
sequential branch.
**SEVEN things were correct only because nothing ran alongside anything else**, and each is now
guarded rather than lucky: `GrantBudget.take()` (a read-modify-write on an object shared by
reference), §23's sign-off memo (a check-then-act window, not a racy dict), the loop's
`authorized_call` (now a per-FUNCTION local, which is stronger than the per-iteration one it
replaces), every shell's single pending-answer slot, `TuiActivity`'s stack, and the panel's row
ORDER. The one that was measured and needed nothing is the busy timeout on SQLite; WAL was added
anyway, because the ceiling is a tunable constant. See ROADMAP_v2 §47 NA9-NA18.
The seventh is `core/compaction.py`'s re-entrancy flag, found by the review of §47 (batch 75) and
worth the sentence because of HOW it was found: its own comment said "a thread-local would be more
precise, but the loop is synchronous and a compaction runs to completion before its caller
resumes", which was measured and true until NA9 removed the premise. A module flag then made a
sibling's compaction skip silently and let whichever child finished first clear the guard under
the other. **Per thread now, and still a flag rather than a lock** -- the property that comment
argued for (impossible to be half-set) is the one that matters, and two conversations folding at
once is correct rather than tolerated. The bare `_compacting` name is GONE rather than aliased:
`if _compacting:` against a callable is always true, so a stale reader has to fail at import
instead of quietly disabling every compaction in the process. **The lesson is the comment, not the
flag** -- a justification that names the condition it depends on is what made this findable at
all, and this file is full of them.
**A batch's results are ordered by POSITION, never by the provider's call id** (NA11, corrected in
batch 76). `_dispatch_parallel` collected outcomes into a dict keyed by `call.id` and rebuilt the
list by looking each one back up -- and that id is the provider's, not ours. `core/client.py`'s
v1-compatible branch starts a tool-call fragment's id at `""` and fills it only if some delta
carries one, so a provider that streams parallel calls without ids produced two requests EQUAL on
the key, and the dict then handed every call in the batch the last-finishing worker's outcome: one
child's answer reported for another child's call, which is the guarantee NA11 exists to make. It is
a list of `(call, outcome, failure)` in submission order now, so ordering is structural. **And the
adapter defaults a missing id** the way its Google sibling always has, because the same id also
pairs `app._spawn_threads` to a spawn line and `parent_call_id` to a stored thread. Two halves on
purpose: the loop should not need a provider to keep a promise for its own bookkeeping to hold.
`core/agent_activity.py` is how a shell sees that stack while it exists. Four things about it
are decisions:
- **It is not a `LoopEvent`, and that is structural.** A generator cannot yield from inside a
nested call: `spawn_subagent`'s handler runs inside `registry.dispatch()`, inside `_run()`'s
`for call in response.tool_calls:` body, where there is no yield point. And
`run_agent_conversation` drains its own `_run()` through `run_to_completion()`, so a child's
events never reach the parent's stream. The channel therefore rides `response_channel`'s
route — an out-of-band object is what crosses a boundary a generator cannot.
- **It carries a NAME and a DEPTH, never content.** §18/D6 returns the subagent's distilled
answer on the "don't share raw history" principle; forwarding the child's stream would undo
that one field over. Anything richer than lifecycle is the thing D6 declines to give.
- **`span()` is a context manager, and the `finally` is the point.** The exit has to run when
the child RAISES — `core/events.py` is explicit that exceptions propagate rather than becoming
events — or a row describing a finished run stays on screen for the session. Sink failures are
contained both ways: display machinery must not fail the run it describes.
- **`activity=activity` on the child's `run_agent_conversation` is what makes depth 2 visible.**
Drop that one argument and the panel reports exactly what `tool_call_start` already implied.
Depth comes from `ToolContext.subagent_depth`, never a second count.
- **§47 gave it an IDENTITY, and an identifier is not content.** A span carries `id` and
`parent_id` now, and `bind()` says which thread the run is writing. That does not reopen the
bullet above it: everything a shell then displays it reads back out of the ARCHIVE, through
`core/replay.py` — the same function and the same policy `/resume` uses. The channel gained
an address, not a payload.
- **The parent comes from a `ContextVar`, and the reset lives in the SAME `finally` as `exit`.**
Five call sites open spans and none knows what is above it, so threading a parent through
them would put the answer in the hands of whoever remembers to pass it. A token that outlived
its frame is worse than absent lineage — the next sibling would be recorded as a child of a
run that has finished, which is wrong rather than missing. It is also the shape that survives
concurrency: a run submitted to an executor takes `contextvars.copy_context()` with it.
- **`bind()` is a THIRD call, not a field on the span, because the thread does not exist yet
when the span opens.** The handler brackets the call and the thread is created inside it, so
the address arrives later — and it has to arrive DURING the run, or a sidebar row could only
be opened after the run it describes had finished. `core/loop.py` binds immediately after
`ConversationMemory` is constructed, the earliest moment the id exists. A no-op when no span
is open, so the call site needs no branch.
**A panel row is armed with METADATA, never with arithmetic** (§47). `AgentPanel` rows
carry the thread they stand for as a Rich style, and a click reads it back through
`Static.get_style_at` — the mechanism batch 58 built for URLs. Computing a row index from
the click's y would need a second copy of the widget's layout, and the two would be free
to disagree: `#agent-panel` has `padding-top: 1`, so **the Nth content line is at y = N+1**,
which a test scanning `renderable.plain.splitlines()` silently gets wrong. Scan
`panel.region.height`. The root row is armed too, so "back to the main agent" is a click
like any other rather than a special case someone has to remember.
**`refresh_agent_panel` reads `self._memory`, never `self.memory`.** The property CREATES a
thread on first use, and the panel is redrawn at mount and on every span — so the property
would leave a phantom empty conversation in the picker for every launch. Same rule as
`refresh_goal_banner` and `refresh_todo_panel`: painting a panel must not be what starts a
conversation.
**`TuiActivity.exit` removes by SPAN ID, and the case that proves it is not reachable
through `span()`.** Nested spans always unwind innermost-first, so removing the last row
matching `(name, depth)` happened to be right and the old code was never wrong in
practice. It is wrong the moment two runs of one agent are open as PEERS and either may
finish first — which slice 8 made ordinary. The test drives `enter`/`exit`
directly, because a `with` block cannot express "the outer one ended first".
**And all three of its mutators are LOCKED, with the stack posted as a SNAPSHOT** (NA16).
This class's docstring used to finish "`self._stack` is read and written only here, under the
worker's own serialisation", which was true until there were two workers. `exit` and `bind` are
read-rebuild-assign, so two at once each filter the list they read and the second assignment
RESURRECTS the row the first removed — a finished run left on screen for the session, which is
what `span()`'s `finally` exists to prevent, reached from the other side. `enter`'s append races
them the other way and is simply lost. The posted list is a copy because the UI thread stores and
iterates it, and handing over the object the worker keeps appending to makes the worker a second
writer of what the UI thread is reading.
**The panel is ordered by LINEAGE, not by arrival** (NA17). Peers at one depth now legitimately
share a column. What had to change is the order, because arrival order and lineage order are the
same thing for a stack and are not for a tree: two children of one turn each spawning a
grandchild can arrive A, B, A's child, B's child, and drawing that in order puts A's child one
level in from B. The panel would be INVENTING a lineage, which is worse than omitting one.
`lineage_rows` keys its cycle guard on row IDENTITY rather than on `span_id`, because `span_id`
defaults to `""` — so a set of ids drops the second hand-built row, invisibly in the app and
wrongly in every test that builds rows directly.
**And the INDENT is the walk, not `AgentRow.depth`** (batch 75, review of §47). This entry used
to say "rows still indent by depth", and that was one number doing two jobs. `depth` is
`ToolContext.subagent_depth`, the number C3 bounds, about how many SPAWNS deep a run is — and a
research pass adds a display level without adding a spawn level, because it opens its span at
`context_depth + 1` and hands the same context down. So a pass and the subagent it spawned both
reported depth 1 and the child drew level with its own parent; `/init`'s initializer is the same
shape. `lineage_rows` returns `(row, level)` pairs and the level is the DEEPER of the walk's
column and the row's own depth: each is a lower bound on nesting, and the second is what places a
row whose parent is not in the stack (`parent_id` is None both for a top-level span and for a row
that never recorded one, and a depth above 1 tells them apart). The alternative considered and
refused was bumping the pass's context, which would have spent one level of what a pass may spawn
on a rendering fix.
**ctrl+g is the keyboard route to the sidebar's rows** (§47). The panel is deliberately not
focusable, so making its rows clickable made navigation mouse-only; the picker mirrors ctrl+t and
draws from the same two facts the panel does, **through the same `lineage_rows` walk** — it
iterated the stack raw until batch 75, so it indented by a column in arrival order and drew A's
child under B, which is NA17's defect in the surface NA17 did not touch. One walk is what makes
"the two cannot come to offer different things" structural rather than intended. Below the live
rows it lists the stored descendants the stack has already forgotten (muted): a finished run
stays openable, and the picker answers what can be READ while the panel keeps saying what is
RUNNING — the two differ on purpose past the live half.
**The letter was chosen by elimination and the test MEASURES it** against
`screen.active_bindings` with the prompt focused, which is the only state where the answer means
anything: `TextArea` claims a/c/d/e/k/u/v/w/x/y/z, `App` claims ctrl+c and ctrl+q, textual claims
ctrl+p, and this app already held ctrl+t, ctrl+l and ctrl+up/down. (That list said **f** as well
until batch 97, which re-measured it across the whole MRO on the current pin: `ctrl+f` is not bound
by `TextArea` on 8.2.8. The letter is still not worth taking — "find" is what a reader expects of
it — but a list presented as a measurement should be one.)
**ctrl+b stops a background session** (ROADMAP_v3 §49, SS2, batch 97), by the same elimination and
re-measured the same way. What narrows it past the free letters is the TERMINAL rather than
textual: ctrl+h/i/j/m ARE backspace, tab, LF and CR, and ctrl+s/ctrl+q are software flow control,
which leaves b, n, o and r. It is the one control SS2 keeps usable while sessions are blocking
input, which is why it is a key at all — `/kill` is typed into the prompt, and the prompt is where
the block is felt — and why `check_action` returns **None** rather than False while nothing is
running: False drops the footer entry and reflows, None greys it, and a control a blocked user
needs has to stay visible.
**ctrl+b stops the SESSIONS and escape stops the CONVERSATION** (ROADMAP_v3 §61, CN9, batch 124),
and the two must not be collapsed. `_busy` is true through the post-turn session wait as well as
through the turn, so a cancel binding reading `_busy` would arm escape in the one place ctrl+b
already means something adjacent. It reads the live `CancelToken` instead — present only while
there is a turn to stop. A cancel landing during the wait ends the WAIT and leaves the sessions
running, with their results held for the user's next message (SS19), which is the suspension path
arriving by a second route rather than a new behaviour.
**The permission modal names the asking run, and the harness fact is written LAST.** A subagent's
approvals have always surfaced on the parent's screen — the channel is inherited — and the modal
said only which tool was asked for. It reads the open span through §47's ContextVar, says nothing
at depth 0 (the asking run is then the conversation on screen), and uses `asking_agent` rather
than `agent`, which `spawn_subagent`'s own `request_payload` already uses for the agent about to
be SPAWNED. **It is written after `payload.update(request_payload)`**, so a tool cannot declare
that key and overwrite it: an agent-supplied claim replacing a harness fact is the inversion §42's
RA6 orders this screen to prevent. Nothing declares it today; the point is that nothing can.
**All three surfaces that can be asked from inside a run, in BOTH shells.** §47 shipped the line on
the permission modal alone; batch 75's review added the sign-off and the model's own question
(`tui/screens.py`'s `asker_label` is one function and `#permission-asker` one id, because ids are
per-screen and three copies drift), and batch 76's second round added all three to `main.py`. The
CLI mattered for the reason its own handlers state: *a kind one shell renders and the other drops is
D12's wired-up-but-invisible gap*, and the terminal is where an unattended-looking run is watched
from. It is worth most on the **sign-off**, because that screen's subject is the run about to be
SPAWNED -- three children of one response each spawning a grandchild produce three prompts that are
otherwise identical. `ask_user` is the one that writes the keys itself rather than having them added
by `_obtain_approval`: `core/interaction.py` is a stdlib-only leaf pinned by
`tests/test_interaction.py`, so the tool is the producer and the channel stays a leaf.
**A `spawn_subagent` line is armed with its CALL, and resolved at PRESS time** (§47). The line
is drawn when the call starts, before the child has a thread, so a target baked in at draw time
would have to be nothing. `app._spawn_threads` maps call id to thread id and has **three
sources**, because the line outlives each of them: the agent stack while the child RUNS (its span
carries the call, `bind` gave it a thread), the tool result once it finishes (`subagent_thread_id`,
returned since §18 and never read until now), and `storage.child_threads()` on a resumed
conversation, which is the only one that survives a restart. Only the agent's NAME is armed — the
digest on a spawn line is the task, prose a reader wants rather than a control.
**`opens_thread` is declared on the `ToolSpec`, not matched by name.** Two readers ask which calls
open a run: `core/replay.py`, deciding what a stored tool line carries, and the TUI, deciding
which live line to arm. A tool name written in both is how they come to disagree about which lines
are clickable — R13's argument, one field over. Defaulted to False, unlike `grant_policy`, because
the safe answer is "this line opens nothing".
**A SESSION line is armed the same way, and the slot carries WHICH KIND** (ROADMAP_v3 §49, SS14,
batch 98). `shell_background` and `shell_monitor` declare `opens_session`, and consumers ask one
question — `registry.opens(name)` → `("thread",)`, `("session",)` or, since §55's TR7 in batch 125,
`("call",)`. The fourth slot of a
`ReplayEntry`, and the transcript's side table, carry `(kind, call id)` rather than a bare id, and
that is load-bearing rather than tidy: a drawn line is text and metadata by the time anyone can
click it, so the tool name that decided the kind is gone. Before batch 98 the slot was armed only
for threads, which meant a `▸ shell_background` line could not be armed in a REPLAY at all — a
session opened until you restarted and then went quiet with no error. **`app._session_calls` has
ONE source**, not a spawn's three, and the asymmetry is the point: the tool result names the session
in the same step the call was made (D20), so there is no window where the line exists and the target
does not. It is an optimisation for the common case — the manager answers a live session and
`core/session_view.stored_entries` rebuilds a finished one from the archive, both keyed by the call
id the line already carries, so `open_session_call` works with the map empty.
**`registry.opens()` HAS NO EMPTY ANSWER SINCE §55 (TR7, batch 125), AND THE GATE IS NOW THE CALL
ID.** It returned `()` for a line that opened nothing, and both consumers paired that falsy tuple
with an id as their test — `opens and call_id`. Every call now opens ITSELF: its whole command,
un-collapsed, and the result that nothing rendered before. So an unarmed tool line is one whose call
the archive cannot be asked about, never one whose tool declared nothing, and `core/replay.py`'s
surviving `if opens` is a guard with nothing left to guard.
Two things about that are deliberate and are the parts worth not re-deriving. **`OPENS_CALL` is
answered without a registry lookup**, unlike its two siblings: `opens_thread` and `opens_session`
are False for a tool this registry has never seen, which is what a runtime-registered MCP tool is
and what a tool removed between a thread being written and being replayed is for ever — and neither
is a reason to hide a stored result. The fallback is a property of the ARCHIVE, not of the registry.
And **there is no exception list**, including the two `DIFFED_TOOLS`: §41's X4 kept `content` off the
transcript line and the diff that replaced it shows the CHANGE rather than the call, so the pane is
the only place the untruncated call can be read. The cost is stated rather than discovered — arming
no longer means "this line started something".
**Two side tables on the transcript, one rule.** `_links` (batch 65) and `_opens` (§47, renamed and
widened in batch 98) are both keyed by entry index, both dropped by `reset()`, both replayed by
`rerender()`. They stay separate because a link opens a BROWSER and a call opens a PANE, and the
click handler tells them apart anyway — but a thread and a session share `_opens`, because they
are one fact about a line and a third dict would be a third lifetime to keep in step. **Testing a `reset()` on either needs two things that are easy to get wrong**: the test
has to REDRAW before asserting, because on the live path a write is handed its targets as an
argument and a stale table is invisible; and the new thread's lines have to reach the same INDEX
the old entry held, or the assertion passes against a `reset()` that clears nothing. Both mistakes
have now been made once each, in batches 65 and 70.
**The read-only thread view is a SWITCHER in `#main`, and the sidebar is why** (§47).
`#sidebar` lives outside `#main`, so it stays visible, live and interactive while a stored run is
on screen — which is what lets a reader move between runs without going back to the conversation
first. A modal would have to compose its own copy of the agent list to offer that, and two
independent writers of one piece of display state is the bug shape this project already fixed
once. Both panes stay mounted; only `current` changes, so a running turn goes on writing to the
live one with nothing to buffer and nothing to flush on the way back. **That last part is batch
66's held transcript doing the work** — a per-access `query_one` would put the answer in the pane
being read.
**A hidden pane measures 0, so both panes write at the width of the one on screen.** The switcher
hides with `display = False`, a hidden widget has no region, and RichLog floors a write to
`min_width` (78) and stores the Strips it drew — so a turn that went on writing behind the viewer
left its answer down the left half of the panel after escape, healed only by a restart's replay.
The viewer is painted hidden as well, which is the same bug from the other side (663be29 patched
that side alone with a width set before the paint and cleared after). `Transcript._region_width`
borrows the on-screen sibling's width, measured per write, and the `write` funnel hands RichLog the
width RichLog ITSELF would have chosen with that region. Two traps, both measured. Passing the
borrowed region as `width=` is the obvious shortcut and switches off RichLog's shrink, so a code
block under a syntax theme with a background paints edge to edge — no shipped theme has one, so it
cannot be seen, which is why the pins run under monokai. And a pilot's `app.console` stays 80 wide
whatever `run_test(size=)` says, which clamps a `width=None` write's measurement to where it cannot
be told from the floor: set `app.console.size` in any width pin.
**`_viewing` is the one fact; everything else is derived from it.** The switcher's `current`, the
crumb's rows, the prompt's `disabled` flag and whether escape is bound all follow from it, so they
cannot drift into disagreeing about whether the viewer is open.
It **widened in batch 98** (ROADMAP_v3 §49, SS14) to hold a background session as well as a stored
thread: a UUID is a thread, a tuple is a session. A second flag beside it was the alternative and is
the failure the paragraph above describes, reached by ADDING a fact rather than by letting one go
stale — two flags can both be true. `_viewing_thread` and `_viewing_live_session` are the two
accessors, so nothing else type-sniffs, and the session gets its own pane (`#session-view`) in the
same switcher: the thread poll repaints `#thread-view` from the archive, so a session drawn there
would be wiped by the next tick of a timer meant for something else.
**A session view polls the BUFFER, and by character count** (SS14). There is no archive to read: a
session's output lives in memory and nowhere else. And the sink cannot stand in for the poll — it
fires when a session MOVES (start, each monitor match, finish, the wake-limit suspension) and never
when it prints, so a background session's view would sit frozen between its first line and its last.
The comparison is the buffer's own `total`, not the length of the tail drawn: a session past the tail
bound has a tail that stops changing LENGTH while its content goes on moving, so the drawn length
freezes the view exactly when it gets interesting. The handler repaints once more when the session
finishes, because the poll stops the tick it stops being live.
**A rebuilt view is keyed by CALL id, and takes its command and rationale from the stored call**
(SS18). Two measurements behind one sentence. A session id is a per-process counter (`s1`, `s2`), so
looking one up across a restart draws a different session with complete confidence. And the saved
wake record is `{id, call_id, shape}` — the row's text carries the command but never the rationale —
so the tool CALL, which has the exact params the model sent, is the only honest source for both and
nothing had to be duplicated into what is persisted. The same route covers a session pruned from
memory in this process (SS12 keeps twenty), so `app._session_calls` is an optimisation rather than a
dependency.
**The trail is read from STORAGE, never from where the reader walked.** Open a sub-subagent
directly and the crumb still says `chat › explore › review`, because the parent link is a column.
A history-derived trail would record what the reader did rather than what spawned what. Every step
but the last is armed; the last is where you already are. The walk carries a `seen` set, because a
hand-edited database must not hang the UI — and note that the mutation removing it does not fail,
it HANGS, so a mutation harness covering it needs a subprocess timeout.
**The poll reads the archive, and only while the run is live.** A child's events are drained
internally (§18/D6), so the only honest source for "what has it written since" is the store it is
writing to. `on_agent_stack_changed` starts and stops the timer, because a span closing is the
ONLY signal a shell gets that a child has finished. Redraws are gated on the entry count: a
repaint every second would fight the reader's scroll position, and **a test comparing the pane's
TEXT cannot see that** — identical entries repaint identically. Count `reset()` calls.
**View-only is enforced by disabling the prompt, and that decides which paths can reach the
viewer's teardown.** `/new` cannot: no slash command can be typed while the prompt is disabled, so
`_cmd_new` has no close of its own. The thread picker can, because ctrl+t is an app binding, so
`switch_to_thread` closes the view. One test pins that ctrl+t stays live, which is what keeps the
first half of this from becoming untrue silently.
**Escape means one thing: back to the conversation.** Going up one level is the crumb's job.
`check_action` keeps the binding off at every other moment — `PromptInput` binds escape to
dismissing the slash panel, and a live-but-inert app binding is exactly what would shadow it.
**The child's thread records its parent; the tool result is not a link** (§47).
`spawn_subagent` has always returned `subagent_thread_id` to the model, so the edge existed —
as a repr inside a JSON blob in one message row, which nothing can query and replay skips
outright (T4). It is stored on the CHILD's row now, in three nullable columns added the way
`pinned` and `kind` were. **The call id is what makes the edge specific**: one turn can spawn
`explore` three times, so the agent name identifies the roster entry and not the run.
`"call_id"` joins `_INJECTABLE_PARAMS` rather than riding in params, twice over — params are
the MODEL'S, and a model that could write its own call id could claim a line it did not make.
**The tempting wrong source is the sign-off.** A depth-2 `SUBAGENT_SIGNOFF` request does reach
the TUI naming the grandchild, and there is no matching completion signal — a row pushed from it
would never clear.
### Skills (`skills/`, §19)
- **Activation pins the body into the system prompt** (K1). `load_skill` already returns any body on request, so activation is not about *reading* one — a tool result scrolls away and is subject to §21 compaction; an activated skill governs the session.
- **`additional_tools` is a declaration of need, never a grant** (K2). It cannot be a grant: `allowed_tools` is a whitelist that narrows and D14 forbids widening, so there is no configuration in which it adds anything. `SkillManager.missing_tools()` reports, and activation proceeds anyway.
- **Category folders; category is metadata, not identity** (K4). `_discover` recurses for skills and stays flat for agents. The name stays global, so `load_skill`, `/skill` and D18's collision rule are untouched by where a file sits — and two same-named skills in different categories still collide.
- **Active bodies are appended outside `with_catalogs()`** (K6) — see the progressive-disclosure note in ARCHITECTURE §4. This is the one cross-shell leak the design can have.
- **Schemas need no mid-loop refresh** (K7, settling ROADMAP_v2's Rev. 3 verification item). Activation happens between turns, and `_run()` recomputes `registry.schemas()` at the top of every call.
### MCP (`mcp_client/`, §17)
**The package is `mcp_client/`, never `mcp/`.** The project root is on `sys.path`, so a top-level `mcp/` shadows the installed SDK — the `logging.py` incident again. §17's spec sketches `mcp/client.py`; don't follow it there. Every SDK symbol lives inside this package so a version migration touches one directory.
**Pinned to `mcp==2.0.0`, EXACTLY, and the range is what it was pinned away FROM.** `requirements.txt` carries the reason: under `>=2.0,<3` a minor release went red on main while every local run stayed green, so CI's result depended on WHEN it ran and CI was not testing what anyone runs locally. Moving it is deliberate: read 2.1.0's split first. v2 is not v1: fields are snake_case (`.content` / `.structured_content` / `.is_error`), `Tool.input_schema`, timeouts are `float`, the exception is `MCPError`, and `Client(server)` replaces `ClientSession` + a separate transport CM. Reading the v1 spellings against v2 doesn't crash — `getattr` returns `None` and every failed call silently looks successful.
**The manager task is required.** `AsyncExitStack` cannot be entered in `connect_all()` and closed in `disconnect_all()`: anyio binds cancel scopes to the **task**, and `run_coroutine_threadsafe()` makes a new task per call, so the right *loop* is necessary but not sufficient. One long-lived `_manager()` task enters every context, signals ready, and parks until shutdown. If `RuntimeError: Attempted to exit cancel scope in a different task` appears, something moved an enter/exit across a task boundary.
**Two timeouts, both needed.** `read_timeout_seconds` cancels the request protocol-side; `future.result(timeout=)` only stops waiting. The margin makes the protocol one fire first.
**§37's edges (F1–F8).** SSE is its own transport, selected only by an explicit `"type": "sse"` (F5) — a bare `url` stays streamable, and a bare URL STRING means streamable to `Client()`, so `_transport_for` returning a string vs a context manager is load-bearing and asserted on type. Teardown shares ONE wall-clock budget (`TEARDOWN_BUDGET_S`, F6); servers that never confirm a clean close by expiry are NAMED in a WARNING — `_TrackClose` records only unexceptional exits, so force-cancelled sessions still count as stragglers, deliberately. The acknowledgement discloses autoApprove/cwd/disabled/env-key-names/header-key-names, hashes nothing of it (`entry_digest` reads the raw entry; text changes never re-ask), and the store is versioned: legacy v1 stores read as unknown-everything exactly once, and `remember_server` starts EMPTY on legacy so one answer never converts unasked siblings (F3). The auto-approve notice is a WARNING (F4) because at INFO it reached nothing but app.log on the TUI path. Registration serves the connect-time catalogue (F8) — MCP's one timeout-less bridge call is gone — and `teardown_mcp(client, registry)` now calls `unregister_all` after disconnecting (F7), which is the "disconnect handling" ARCHITECTURE.md always claimed.
**MCP tools are allowed by default but require approval by default** (D28). The per-server `autoApprove` opt-out is a *base default* (`permissions.set_dynamic_approval_default`), never a `ToolSpec.approval_check` — `approval_needed()` ORs those and OR can only tighten, so an opt-out there is inert.
**Tier precedence is harness > user > project** (D29) for MCP servers, agents and skills — inverted from the usual "more specific wins", because "project" means "arrived with a repo you cloned". `settings.json` deliberately keeps project-over-user, since the trust prompt shows its values verbatim. Non-colliding definitions from all tiers still load; precedence only breaks same-name ties.
**`mcp.json` accepts unknown keys** (unlike `settings.json`, which raises) so configs shared with Claude Desktop/Cursor work. Validation is by *connecting*; per-server failures are named and skipped, never fatal.
**A flag that is not a JSON boolean is refused, and the two flags are refused DIFFERENTLY** (A29, batch 150). `bool("false")` is True, so `_flag` has never coerced -- but it used to answer `False` for both keys, and for `disabled` that means NOT disabled: `"disabled": "true"` connected the server and the consent line, which reads `self.disabled`, showed it as live. Neither constant is safe there, because `"disabled": "false"` is one hand edit away and wants the opposite answer. So `_flag` reports that it could not parse and each caller states its own direction: `autoApprove` falls to `False` (the safe answer either way, and pinned as a control), while an unparseable `disabled` makes the entry **UNUSABLE by name** -- transport `unknown`, `describe()` printing the reason on the acknowledgement screen, the server in `failures`. Nothing runs and nothing is silent, which is decision G's own remedy rather than an exception to it.
**A composed tool name is validated and an invalid one is refused** (A30, batch 150). Both halves of `mcp__<server>__<tool>` come from outside this repo and the installed SDK constrains neither -- `Tool.model_fields["name"]` is a bare `str`. A dotted `mcp.json` key, a space or newline from a server, or two ordinary long words (70 characters) all used to register AND advertise, and `schemas()` is rebuilt every call (K7), so one such name made every provider request malformed for the rest of the session from a server that was working. Refused rather than mangled, because a renamed tool is one the model cannot name back and the prefix is not a parser. One warning per SERVER when the server half is at fault (it poisons every name, and the operator can rename it), one per TOOL otherwise. This is not the up-front schema validation decision G rules out: nothing in `register_all` reads `mcp.json`, and a named per-server failure is what G asks for.
**A tool's description, title and input_schema are third-party text in every request, and all three are bounded at the producer** (A31, batch 150). A5's rule, which the other two sources of model-facing third-party text already followed: whitespace collapsed, capped at `registration.MAX_TOOL_DESCRIPTION_CHARS` (2000 -- NOT `MAX_CATALOG_TEXT_CHARS`, whose 300 is sized for a one-line bullet), applied after the `[MCP: ]` prefix and the server's `title` are composed in so one call covers all three strings. `input_schema` is bounded by a different mechanism because capping strings inside a schema turns a load-time failure into a call-time one: the serialised whole schema is capped at `MAX_TOOL_SCHEMA_CHARS` (16384) and an oversized tool is refused. The success path's `result` is still unbounded on purpose -- that is TECHNICAL_DEBT 48, ruled DEFER.
**Redaction recurses now.** `check_output_policy()` used to scan top-level strings only; `structured_content` arrives under `result` as a dict, so third-party MCP output was the one unscanned path. Fixed in `safety/policy_enforcement.py`, not at the MCP boundary — any tool returning nested data had the same hole.
**`_normalize` obeys that rule too, and until batch 150 it obeyed none of it** (A32). The non-text placeholder was guarded by `if not text and blocks`, so it fired only when NO text block produced anything: a result carrying text AND an image dropped the image with no placeholder and no log line, and "Chart generated:" with no chart is a result the model will answer from. The placeholder is now APPENDED to whatever text survived, the dropped kinds come from the blocks the join actually skipped (an empty text block has nothing to drop, and reporting it as non-text was the placeholder contradicting itself), and one WARNING names the tool -- which required making the tool name a PARAMETER, because `_normalize` took the result alone and the rule below was therefore unsatisfiable in the function it applies to. A structured error payload rides in the error STRING rather than a second key, because D23 promises one error shape.
**Nothing an output-policy pass alters is silent, and the redaction has an off switch (#49/#167, batch 20).** Every altered dispatch result logs ONE WARNING naming the tool and what changed — before #49 the depth cap replaced a whole container with a sentence and logged nothing. Credential *shapes* (`password "…"`, `<password>…</password>`, `scheme://user:pass@host`) join the vendor tokens, applied to `param_digest` too (a credentialed URL had been printing its password into both transcripts since §26). The shapes are OUTPUT-ONLY by pinned decision: `_SECRET_PATTERNS` stay `_input_leaf`'s whole vocabulary, because refusing a write whose content is a build file breaks the call the user asked for. `VENASTINE_REDACT_OFF` / `config.REDACT_TOOL_OUTPUTS` disable pattern substitution — never input refusals, never the depth-cap sentinel, never logging_setup's formatter; deliberately not a settings.json key (G7's reason).
### Reasoning effort
`effort=None` sends nothing, on every provider. `reasoning_effort` is rejected by OpenAI-compatible non-reasoning models and adaptive thinking by pre-4.6 Anthropic models, which is why None was the shipped default for its whole life — **until batch 25 changed it** (#139, owner decision): `config.DEFAULT_EFFORT` is now `"high"`, because the research pipeline consumed no effort at all and the mode this project exists for ran at the provider's own default invisibly. The old note's risk is managed in `MODEL_EFFORT_LEVELS` rather than by silence: an entry mapped to an EMPTY list (the gpt-4o/gpt-4.1/gpt-3.5 family ships listed) is authoritative "takes no effort parameter", so `effort_for()` drops the default cleanly instead of 400ing; an unlisted non-reasoning endpoint still fails loud — same posture as `MODELS_REJECTING_SAMPLING_PARAMS`, and `--effort auto` is the off switch. Precedence: CLI flag > settings.json top-level `effort` > `DEFAULT_EFFORT`, with `tui.effort` still winning inside the TUI; a TUI level validates against the mounted model only when a human named it (`_effort_named`) — probing a default-derived one fired fallback WARNINGs on every launch and broke #138's healthy-mount silence.
Anthropic's valid levels are **queried** (`capabilities.effort` on the Models API) so new Anthropic models need no table entry; everything else falls back to `config.MODEL_EFFORT_LEVELS`, where an unknown model ASSUMES `["low","medium","high"]` — optimistic, cached, and the reason the empty-list entries above matter. Google reports no levels at all on the pinned `google-genai==1.0.0`, which has no `thinking_budget` field, so effort drops there regardless of the default.
The pipeline carries effort like authorization (#139): every pass, every JSON retry ("a retry is the same pass continuing"), §20's reviewer and its re-synthesis inherit it, and `effort_for` validating per RECEIVING model is what makes ensemble rosters and critic routing safe by construction. **§47's `activity` sink is threaded the same way and along the same hops**, which is worth naming because two of them dropped it until batch 75: `json_retry.retry_until_json` and the review consent walk down to `_refine`. Both reach the model through `continue_conversation`, which drains its own loop — so on those two paths the sink is the ONLY way a subagent spawned inside them is visible at all, which is the reverse of how "just a retry" reads.
`config.MAX_TOKENS` caps thinking **plus** response on current Anthropic models — that is why §16 raised it from 4096.
### Sampling parameters
Current Anthropic models reject `temperature`/`top_p`/`top_k` (`config.MODELS_REJECTING_SAMPLING_PARAMS`). `_sampling_kwargs()` drops them with a WARNING.
**Nothing in the harness passes a `temperature` any more, and the WARNING is what keeps that safe.** Ensemble mode was the one caller; §10's revisit replaced sampling variation with a roster of different models, so `_sampling_kwargs` is now a **backstop** rather than a guard on a live path — the boundary the next caller wanting sampling variation will meet, and the warning is how that caller learns it cannot have it. The threading through `core/loop.py` stays for the same reason. Do not delete either as dead code: silence here is exactly how §10 came to be built, documented as working, and incapable of running against this harness's own default model.
### Ensemble mode (ROADMAP §10, as revisited)
**Diversity comes from different MODELS, not different sampling** (E1). `config.ENSEMBLE_MODELS` is a roster of `{provider_name, model}`; Pass 1 runs once per entry, each on its own provider/model. §11's stated reason is the argument — "a model checking its own output for errors shares that model's blind spots" is the identical objection to self-consistency, since N samples of one model agree most confidently on that model's systematic errors. §10's own revisit note proposed *prompt-framing* variation; that was rejected, because framings partition what each candidate looks for, so an omission becomes a systematic bias rather than noise.
- **No new plumbing was needed.** `_run()` calls `api_initialization(provider_name)` itself and `effort_for` validates against the receiving model, so a heterogeneous roster is correct by construction — the same way §11 already routes 3a/3b/6c to `critic_provider`/`critic_model`.
- **`config.yaml` only** (E2), following `critic_model`. `ensemble_models` is rejected from `settings.json` **by name**, R12's rule applied to a second kind of authority: turning the mode on can only spend more of the provider the user already chose, but a *roster* chooses providers, and a project's `settings.json` beats the user's.
- **N is `len(ENSEMBLE_MODELS)`** (E3). `ensemble_n` survives as a vestigial parameter and settings key with a WARNING — removing the key would make every existing `settings.json` that sets it raise.
- **Fewer than two DISTINCT `(provider, model)` pairs is refused** (E5). Repeating one entry N times recreates the original defect through the new config: no sampling variation remains to make the candidates differ, so they arrive near-identical and every claim scores maximal consistency. Counting entries instead of distinct pairs is a pinned mutation.
- **A failed candidate is named, traced and skipped** (E7) — §20's containment rule. One survivor **degrades to the single-candidate path** (`ensemble_n=0`), which is the correct formula when there is nothing to compare against, not a degraded one; scoring one candidate as an ensemble of one would report unanimous corroboration from a single source.
- **Candidate labels follow the SURVIVORS, never roster position** (E11). A gap ("Candidate 1, Candidate 3") makes Pass 2 tag a claim both survivors asserted as `[1, 3]` against a denominator of 2, so unanimous claims read as dissented-against.
- **Consistency enters as a DISAGREEMENT PENALTY** (E8): `raw -= 0.15 * (1 - consistency)`, factual claims only, subtracted like `ASSUMPTION_FLAG_PENALTY`. §10's shipped formula redistributed `0.5/0.35` → `0.4/0.3`, which made grounding count for *less* whenever ensemble was on and — since the maximum stayed `0.85` — meant consistency could only ever demote a well-grounded claim. As a penalty, unanimity is free and the non-ensemble formula is *literally* unchanged, so §10's byte-for-byte regression holds by construction. Non-factual claims are untouched (E9).
- **`score_claim` rounds ONCE** (E10). The tier used to compare the unrounded float while the breakdown stored `round(raw, 4)`, so a claim computing `0.7999999999999999` was persisted as `raw_score: 0.8` and tiered MEDIUM against a table saying `≥0.8` is HIGH. Reachable from ordinary weights, and it is what §10's own acceptance criterion produces.
- **The record keeps every surviving candidate** (E13, #77). `run.candidates` carries metadata per survivor (text in memory only; `candidates_json` strips it at persistence — the bodies are artifacts and pass threads), Pass 3b sees all labelled candidates when ≥2 survive, and the artifacts are `01_candidate_N.md`, numbered to match the trace and `asserted_by_candidates`. A degraded single survivor keeps `01_raw_response.md` byte for byte.
- **A pass failure is `pass_complete(ok=False)`, never a new kind** (E14, #115). Both pass wrappers emit it before re-raising; the widget's `pass_completed(pass_id, *, ok)` makes the answer REQUIRED so no caller can tick a failed row done. Panel-only display: the trace line already narrates the cause, and no shell prints it twice.
### Provider translation (`core/client.py`)
Provider wire formats exist in **exactly two functions**: `_tools_for_provider()` and `_messages_for_provider()`. Adding a provider means extending those, plus the branch in `call_model_stream` and the client construction in `api_initialization`, and nothing else. There is **one** call path: `call_model()` was deleted in batch 13 (#38) after audit unit 3 found it had no production caller and had already diverged from the streaming path on `raw`, while this sentence told every new-provider author to maintain both. Tool schemas are always authored in Anthropic's native shape (`TOOL_SCHEMA` per tool file); translation happens at the boundary.
Anthropic, OpenAI-compatible (any `is_v1_compatible` provider), and Google are all fully implemented. The load-bearing differences (system prompt placement, tool-result batching, `assistant` vs `model` role, `max_tokens` vs `max_completion_tokens` vs `max_output_tokens`, response parsing) are enumerated in ARCHITECTURE.md §4.7.
**D21 — streaming must not silently degrade token accounting.** `supports_stream_usage` is read from `providers.json`; all three streaming branches *raise* if the flag is set but the stream ends with zero usage (audit #40 brought Anthropic in line). A silently-zero budget disables the budget stop condition. Do not soften this to a `getattr(..., 0)` default. **The flag has nothing to do with whether text streams** — deltas are yielded on every branch regardless of it, and §38 records that misreading because it is where anyone chasing streaming latency looks first.
**Thinking is captured, persisted, and echoed back to the model that wrote it** (§38 O1–O3, as amended by §44 WS3/WS4). `StreamToken`/`LoopEvent` still carry a `thinking_delta` beside `token_delta` for DISPLAY; the Anthropic branch iterates the `MessageStream` rather than `stream.text_stream` — the two share one underlying iterator, so a caller gets text or both, never text plus thinking — matching the SDK's synthetic `TextEvent`/`ThinkingEvent`, and the OpenAI-compatible branch reads `reasoning_content` **or** `reasoning`, because there is no standard name and `ChoiceDelta` is `extra="allow"`.
**`display: "summarized"` is sent EXPLICITLY, and §38 shipped without it.** On every Anthropic model from Opus 4.7 onward — the Fable, Opus 5.x and Sonnet 5.x lines and Haiku 5.5 included — the default is `"omitted"`: thinking blocks still stream, with EMPTY text. So `ev.thinking` was `""`, `core/loop.py`'s `if token.thinking_delta` never fired, and §38 rendered nothing at all on Anthropic for its whole life — invisible because the OpenAI-compatible branch is unaffected and that is where it was being used. The raw chain of thought is never exposed on any model; `"summarized"` is the readable form.
**What is PERSISTED is the provider's own blocks, not the streamed prose** (WS3). `ModelResponse.thinking` is `{provider, model, blocks}`, read off `get_final_message()` and stored in a nullable `MessageLog.thinking` column (additive, the `pinned` precedent). An Anthropic block carries a `signature` the model verifies against its own state, so a record rebuilt from the deltas would be unsigned and useless. `_to_neutral` adds the key ONLY when a row carried reasoning, so every pre-§44 message reconstructs to exactly the shape it always did. `redacted_thinking` is stored and never rendered; `core/replay.py` pulls the prose back out by field name for the two shells.
**And it goes back on the wire, gated on the `(provider, model)` pair** (WS4). Blocks return unchanged to the model that produced them and to nothing else — after a `/model` switch the thread is full of blocks the new model never wrote. Anthropic's lead the assistant content array, because that is the order they were emitted in. The OpenAI-compatible side is opt-in per provider (`echoes_reasoning` in `providers.json`): DeepSeek 400s on `reasoning_content` as input, OpenRouter varies by model. Google is unchanged. Before §44 the model had never seen its own reasoning at all — not across a restart, and not between two turns of one live session.
**Two providers show no thinking, by construction rather than by bug** (O2). `OPENAI`'s Chat Completions endpoint returns no reasoning text at all, and `GOOGLE`'s request deliberately does not ask for thought summaries — `ThinkingConfig(include_thoughts=True)` exists on the pin, and setting it is an owner decision (it changes a verified provider's request and bills the summaries), not an omission to fix. Both SDK shapes are asserted against the REAL packages in `tests/test_sdk_conformance.py`: a field rename here fails *silently*, since `getattr` returns `None` and every stream then looks like it carried nothing.
**The empty assistant turn is dropped ABOVE the branch split (#13, #36).** A turn with neither text nor tool calls — a safety filter, a truncation, an empty pass response — is persisted by D20 and so reaches translation on every resume. It used to produce three different results from one neutral input: `content: null` on OpenAI (#13, which killed a live Pass 2), `content: []` on Anthropic (#36, on the *default* provider), and a skip on Google — guarded only because Google was the provider being debugged when it surfaced. `_is_empty_assistant_turn` now filters once at the top of `_messages_for_provider`, so a fourth branch inherits the fix instead of re-deriving it, and Google's local `if parts:` is gone. **Both halves must be empty**: a tool-call-only turn legitimately has no text, and a guard keyed on text alone drops the `tool_use` and orphans the `tool_result` after it. Dropping the turn can leave two consecutive `user` messages, which is the shape Google's branch has always produced here.
### Research pipeline (`core/reasoning/`)
`orchestrator.py` sequences the passes; `base.py` is data only (`Claim`, `PipelineRun`); `confidence_scoring.py` is Pass 5; `pipeline_storage.py` persists runs; `output_writer.py` writes `/output/<run_id>/`. Prompts live as `.md` files in `prompts/`, loaded by `prompts/system_prompts.py`.
Invariants that look like simplification opportunities but are not:
- **Each pass gets its own fresh thread.** Passes share distilled JSON, never raw history. Do not merge into one thread.
- **Pass 5, the gates D0, D1 and D2, and Pass 6c make zero LLM calls.** Thresholds, routing, retry caps, dedup, and annotation are pure Python. Adding a model call to any of them is a rearchitecture, not a fix.
- **The scoring formula caps before subtracting the assumption penalty**, not after. Cap-after was a real bug, caught by a test comparing a flagged and unflagged claim landing on the same tier.
- **Pass 6b does not re-run Pass 4.** Assumption-flagged claims exhaust retries and fall to UNVERIFIED. Known design gap, not a bug.
- **The payload boundary declares the type of a container's MEMBERS, not just the container** (§66, A25). `payload_validation.py` types `entities` and `asserted_by_candidates` as arrays AND their members as strings and numbers, and every `per_claim_flags` VALUE as an array, because the pure code downstream does not merely iterate what it is given: Pass 3a builds `sorted({e for c in factual_claims for e in c.entities})` and Pass 5 computes `len(set(claim.asserted_by_candidates))`, so `["madrid", 1947]` raises on the sort and `[["madrid"]]` on the hash. A container type alone left three of six measured crash shapes open. **A boolean is not a number here**: `isinstance(True, int)` is True, so `"severity": true` passed and Pass 5 scored `1.0 - True` = 0.0, the MAXIMUM critic penalty, with no trace and no retry. Do not extend this to members nothing reads -- flag STRINGS and Pass 3c's `gaps` members are deliberately untyped, because `len()` and a dict wrapper are all that touch them, and B4's rule is that a check earns its place by having a demonstrated consequence.
- **The 6a/6b loop counts the ROUND, not the revision** (#74). `retry_count` is incremented for every claim still in `flagged` after Pass 6a returns; the per-revision write sets `revision_text` only. Incrementing inside the loop over 6a's response — which is how it shipped — makes D2's cap reachable only with the model's cooperation, and 6a is one batched call across every flagged claim, exactly the shape a model drops an item from. A claim it omits then never exhausts and `while flagged:` has no bound at all. Do not *add* a per-round increment beside the per-revision one; it must replace it, or a named claim counts twice. And because `flagged` only ever shrinks, every claim in it exhausts on the same round — which is what keeps a fallback from being undone by a later one.
- **The loop's re-tiering is scoped to its own batch** (`run_confidence_tiering(..., claims=flagged)`). Pass 5 re-tiers everything; a retry round must not, or a claim already finished at D1 or D2 is recomputed from unchanged score data while `_apply_fallback`'s annotation stays — so it ships as LOW beside a note saying it could not be verified, and Pass 6c leaves both alone because `if claim.annotation is None` is false. Reproduced in batch 149 while ruling against an unconditional global re-tier, which is the fix that suggests itself for the paragraph below.
- **And so is everything else a 6b round does** (§66, A26). The narrow re-tier is only half a rule: 6b's payload used to be validated against every id in the run and applied over every claim in it, so an entry naming a finished claim changed its grounding and left its tier alone — a HIGH claim with `grounding_status='ungrounded'`, `sources=[]`, and a breakdown still recording `source_count: 1`. The ids, the apply and the RESCORING are all scoped to `flagged` now; the last one matters because `score_grounding_sources` rewrites every claim's sources by default and its `claims` argument had existed since §45 with no caller. An entry outside the batch is dropped and TRACED, which is `review.py`'s rule for the same shape. **Narrowing `claim_ids` alone does not fix it**: B5 makes a total id mismatch the error and a partial one deliberately not, so the batch-plus-one-extra payload a model actually sends passes either way.
- **§20's review runs AFTER final synthesis, and the report it regenerates is NOT re-reviewed.** The regress is cut deliberately — reviewing the corrected report would need its own consent pass, and so on. Do not "fix" this. The stage is inside the pipeline's `try`, but a transient REVIEWER failure (provider error, unrecoverable JSON) is contained like the missing-agent branch: traced skip, run completes — an optional stage must not flip a finished ten-pass run to `status='failed'`. A failed re-synthesis after accepted corrections is contained the same way and for the same reason: the deferred commit restores the pre-review claims (so the record never carries corrected claims beside a stale report), the trace says the corrections did *not* land, and **the run still completes**. Only a failure that escapes the stage itself — a bug, not a transient provider error — reaches the pipeline's `except` and records `status='failed'`. Both halves are test-pinned; changing one without the other puts the code and this paragraph back into disagreement, which is what the §19–§20 review found here.
- **`_synthesis_input` is a function, not a string built once.** §20 re-runs synthesis after an accepted correction; reusing the earlier string regenerates the report from the UNCORRECTED claims, so the report changes for no reason while the correction silently goes nowhere.
- **`vars(c)` serializes `Claim` at every site** — find them with `grep -rn 'vars(c' --include='*.py' .`, not from a list. Adding a nested-dataclass field to `Claim` requires migrating *all* of them to `dataclasses.asdict(c)` in one change. This bullet used to enumerate three (orchestrator's passes, `pipeline_storage.update_pipeline_run`, `output_writer.write_run_artifacts`) and §26 added two more in `tui/app.py` that fell out of both this list and `base.py`'s (audit #108) — an enumeration that says *all* has to be complete to mean anything.
- **§45's source scoring MULTIPLIES the grounding component; it is not a fourth weighted term.** `grounding_component = GROUNDING_WEIGHTS[status] * max(SOURCE_QUALITY_FLOOR, q_max)`, where `q` per source is `authority × similarity`. A fourth term is the exact shape §10 shipped for consistency and then reversed (E8): redistributing 0.5/0.35 to make room makes grounding count for less whenever the feature is on. Scaling an existing component leaves the unscored formula LITERALLY unchanged, which is what keeps the byte-for-byte regression guard true by construction.
- **`sources_scored` is PASSED to `score_claim`, never inferred.** An empty source list means "the model claimed grounded and cited nothing" on a scored run and "this claim was built without a scoring stage" everywhere else — a test, a §20 correction, a resume. Reading 0.0 from the second re-tiers claims nothing measured. The `weights` parameter is the same shape and the same reason: it defaults to the module's own constants, so every pre-§45 caller scores exactly what it did.
- **`AUTHORITY_FULL_CREDIT` is not redundant, and `SOURCE_QUALITY_FLOOR` is not 0.4.** The domain table is a RANKING and the formula needs an EVIDENTIAL scale; feeding authority in raw multiplies two rank-scales and collapses the tier — swept, Nature and a Pinterest pin both landed MEDIUM. And `0.5*0.4 + 0.35` IS the MEDIUM threshold, so a floor of 0.4 guarantees MEDIUM for every grounded claim however bad its sources. Both are pinned by a sweep in `test_confidence_scoring.py`; check the arithmetic against `TIER_THRESHOLDS` before moving either.
- **`_ground_and_score` is the ONE seam both grounding sites cross**, and the only writer of `run.source_documents`. Pass 3a and Pass 6b both ground, and a source scored only on the first would carry a number computed for text that no longer exists. Deriving beside each call is what "fix at the producer" exists to prevent. Its `claims` argument narrows the apply AND the rescoring together, and only the 6b caller passes it (§66, A26): Pass 3a keeps the whole run, because it is the first grounding and every claim is in play. `run.source_documents` is refreshed either way — it is the evidence the RUN gathered, not a per-claim quantity.
- **The source corpus is a SECOND SINK on `_translate`, not a change to it.** §26's rule — a successful tool result body never reaches the UI — is unchanged. Scoring is silent on a clean payload, which `test_pipeline_trace_contains_expected_lines_in_order` depends on; a Pass 3a fixture that drifts from `source_grounding.md` makes it talk, and that test then fails about trace ordering.
- **`confidence` and `source_scoring` are TOP-LEVEL settings sections, and an out-of-range value warns rather than raising.** `_NESTED_SETTINGS` supports exactly one level, so `research.confidence` would silently fail to merge across tiers (review finding F2's defect). And the warn-vs-raise split from `effective_compaction` is argued in §45: a bad trigger burns model calls on every turn, a bad weight costs one number in one report. Unknown KEYS still raise.
- **`fetch_url` pages from a HELD body, and the policy check runs anyway** (ROADMAP_v3 §51, PG5/PG6). `offset=0` always re-fetches and refreshes a 300 s per-URL entry; `offset>0` reads it, so every page of one document comes from one fetch. This is not an optimisation and reverting it to a re-fetch is not a simplification: measured, `news.ycombinator.com` changed inside page 1 across 45 seconds, so page 2 of a re-fetch is a splice of two documents that no key in the result can disclose. `is_url_permitted` runs on EVERY call including a cache hit -- a cache consulted before the blocklist is #54 one layer up, reached by a second call instead of a redirect. And `MAX_CONTENT_BYTES` is **not** a tuning knob: PG6 declined to raise it, so the last reachable page says the document was cut at the read limit rather than naming an offset that returns nothing.
- **`fetch_url` REDACTS BEFORE IT CUTS, and that ordering is the whole of it** (§52, RB1). `redact_output_text` runs once on the decoded body, above both `_hold` and the slice, so the held copy and every page cut out of it are slices of already-clean text. Moving it below either line is not a refactor: measured end to end, a `ghp_` token at characters 4,980–5,020 came back clean in both pages, was rejoined by PG7's append, and reached `sources/<sha256>.txt` and the window sent to the embedding provider. The repo had locked this rule twice before — `param_digest` "redacts BEFORE truncating" and `tui/app.py`'s diff block runs on whole texts — and §51 is the first place it was applied in the wrong order. Two consequences follow and both are deliberate: the tool's `offset` and `chars_available` count REDACTED characters (RB3), and the tool logs its own one-line warning (RB2) because `check_output_policy` now finds nothing to alter and #49's no-silent-alteration rule would otherwise stop covering it.
- **`TTLCache` sweeps on write and caps its entries at `MAX_ENTRIES = 128`** (§52, RB5/RB6). Expiry on READ is unchanged and still load-bearing — it is what makes a post-TTL query a real request again — but it bounded staleness and nothing bounded memory: measured, 100 fetches held 6.3 MB and released none of it, because nothing read those keys a second time. The cap is derived, not chosen: `MAX_ITERATIONS` is 50 and a redirect stores two keys for one body, so 100 is one pass's reach and the cap sits above it. Do not tighten it to "save memory" — an eviction inside a pass turns a page-2 request into a re-fetch, which is the spliced document PG5 exists to prevent. `set` pops before it assigns, so a refreshed entry is the newest rather than keeping its original position and being evicted next.
- **The corpus CONCATENATES pages of one URL; it does not take the longest** (§51, PG7). `_store`'s `offset` parameter distinguishes `None` ("this tool does not page", the old behaviour exactly) from `0` ("page one of something that may have more"), and they must stay different values. `SourceDocument.next_offset` is in the SOURCE's character positions, never in held characters — `.strip()` and `MAX_DOCUMENT_CHARS` change how much is HELD and never where a page sat in the document it came from, and the two numbers coincide at offset 0, which is why a test starting there cannot tell them apart. Redaction used to be a third reason they differ and no longer is for `fetch_url`, which redacts at its producer since §52 (RB1); the entry-side pass here STAYS, because `arxiv_search` and `web_search` do not. Before this, page 2 was silently DROPPED (`5000 >= 5000`) and the grounding pass scored page 1 however far the model read.
- **All three network tools' caches are cleared by an autouse fixture, and their keys carry the page** (§51, PG9). `arxiv._cache` and `web_search._cache` had been uncleared since they were written — `clear_scholar_cache` next door had the same problem and the same fix. A key without the page in it serves page 1 to every request for page 2, silently.
- **A truncated abstract or snippet says so, and the caps do not move** (§51, PG8). `MAX_SUMMARY_CHARS` and `MAX_SNIPPET_CHARS` bound model-facing text from a source this project did not write (#131). The fix for "the model cannot tell a short abstract from a cut one" is the flag, not a larger cap and not a knob.
- **"The caps do not move" forbids RAISING one; it never meant "add no more"** (batch 126, closing TECHNICAL_DEBT 41). #131 is the reason the caps exist, so it *requires* bounding model-facing third-party text wherever it is unbounded, and `arxiv_search` had three such fields plus an unbounded wire read while the abstract — the one field anyone had capped — was not the field that grew. Measured: `str(result)` of 223,191 characters for an ordinary physics query, **91.8% of it the author list**, one entry carrying 2,832 authors; everything the tool already bounded ceilinged near 16,900 at `max_results=20`, so the capped fields could never have accounted for the 116,996-char row the register recorded. `_MAX_AUTHORS = 25` takes PG8's full shape (flag, `author_count` with the REAL total, `full_authors_url`). `MAX_TITLE_CHARS = 300` and `MAX_CATEGORIES = 20` take the flag and **no route**, and that asymmetry is the decision: #49 says nothing is altered silently so the flag is owed, but PG4 exists so the model can reach what it *wanted*, and a title past 300 characters is a malformed feed rather than a long paper (measured: the longest of sixty real titles was 188). Do not "complete the pattern" by adding a route there.
- **`MAX_FEED_BYTES` is `arxiv_search`'s half of §31 (H7), and it is not a tuning knob either** (batch 126). `_call_arxiv_api` was `httpx.get(...).text` — the last `httpx.get` in production code, and the exact shape `fetch_url.py`'s own comment calls out: httpx buffers the whole body before anything slices it, and a Content-Length check is not the fix because a hostile server can omit it or lie about it. Streamed now, dropped inside the `with` the moment the cap is passed. **2 MiB is derived and the first derivation was wrong** — it guessed ~100 KB per hyperauthorship entry; measured, the heaviest *legitimate* feed is 838,560 bytes (`max_results=20`, 13,859 `<author>` elements), so the bound clears the real worst case by 2.5x. That number is the one that matters: a cap under the largest real feed is a FALSE REFUSAL, the tool reporting it read nothing for a query arXiv answered correctly. An oversized feed is **refused and not retried** — a cut Atom document does not parse, so unlike `fetch_url` there is no degraded body to keep, and retrying reaches the same place twice more before reporting a network failure that is not there.
- **`scholar.py`'s OpenAlex lookup is OFF by default and is the one network call outside the tool layer**, so it never appears in `granted_calls.json`. Its `fetch` is injectable because the suite runs offline, and its module-level cache is cleared by an autouse fixture — without it a test injecting a FAILING fetch is served a payload a passing test cached, and goes quiet rather than red.
- **`update_pipeline_run(status=...)` defaults to `None`, not `"running"`** — so a data-only checkpoint can't reset a `complete`/`failed` row.
- **The pipeline is a GENERATOR** (§22). `stream_deep_research_pipeline()` yields `PipelineEvent`s; `run_pipeline_to_completion()` drains. Same shape as `_run()` / `run_to_completion()`, for the same reason. **P3's third name is retired.** It kept a synchronous `run_deep_research_pipeline()` so §22 would change no caller — and then both shells moved to the generator anyway, to render progress, each with a comment saying it deliberately did not call the wrapper. What was left was a public function with no production caller, fifty call sites in `tests/`, and three docstrings still calling it the live entry point. The convenience is `tests/conftest.run_pipeline` now, which is what it had become. Its pass-level twin `run_deep_research_mode` retired with it and for the same reason; `tests/conftest.run_pass` is that one.
- **`_Progress.checkpoint()` is the only place a trace line is recorded** (§22). It persists, then yields every line appended since it last ran — so §5's per-pass persistence and the events are DERIVED from `run.trace` rather than emitted beside it. This is what finally made "a checkpoint after every trace line" true: `review.py` writes fifteen lines and checkpointed after none, and `json_retry.py` appends straight to the list. Both are now carried without either module changing, which is also why a JSON-parse retry needs no event kind. Persist-before-emit is deliberate: a generator only advances while someone iterates it, so emitting first would make durability depend on a UI continuing to read.
- **An abandoned pipeline generator leaves `status='running'`** (§22). `GeneratorExit` is not an `Exception`, so a consumer that stops iterating is recorded as abandoned rather than failed — and the persist-before-emit ordering is what keeps the checkpoints it already took.
- **A pass's `LoopEvent`s are TRANSLATED, not propagated** (§22 P2 as amended by §26), and `PipelineEvent` is a separate, kind-discriminated type from `LoopEvent` (§22 P1) — adding §22's or §26's kinds to that flat bag is the thing the decision rejected, and `test_pipeline_events.py` fails if `LoopEvent` grows a TENTH field. (It has nine: eight payload fields plus `stop_reason`. The test asserts the exact eight-name set, so any addition goes red, and a new field is argued for in that test the way `retract` and `thinking_delta` were.) §22 kept a pass opaque on the premise that its internals were not worth seeing; the first real ten-pass run disproved that, so `_run_pass` now iterates `RunAgentLoop.stream_deep_research_mode()` and converts a chosen subset into pipeline kinds. A consumer still sees exactly one event type — that is what P1/P2 were protecting, and it still holds.
- **A truncated pass is detected AT the pass.** `_check_not_truncated()` reads
`stop_reason`: truncated with text traces and continues, truncated with no text
raises naming the pass and the reason. A stop condition returns the last
response as it stands, so a pass cut off mid-tool-call returns `""` — which was
stored as `raw_response`, made Pass 2 report it had nothing to extract, and
surfaced three passes later as `Claim.__init__() missing 3 required positional
arguments`. The check runs BEFORE the JSON retry: a cut-off pass did not write
malformed JSON, so nudging it spends more of an exhausted budget for nothing.
- **There is no default token budget anywhere** (batch 27 closed item 9). The
cumulative counter is a spend meter — correct as billing, useless as a size
limit — and both 1M envelopes died with the misreading that created them. A cap
exists only if settings.json sets `max_token_budget`; wrapper defaults are a
sentinel resolving it per call, and explicit `max_total_tokens=None` stays
genuinely uncapped. Size reasoning reads `turn_new_tokens`/`last_input_tokens`,
never the spend figure.
- **The orchestrator CARRIES a `RunAuthorization`, it does not interpret one** (§25). Which tools may be granted is `core/reasoning/authorization.py`; enforcement is `core/loop.py`; building it from a human's answer is the shell's job. `authorization=None` is the default and means the pre-§25 behaviour: no grants, nobody to ask, every gated tool hidden from every pass.
- **`run.granted_calls` IS the bundle's list, not a copy.** Sharing one object is what puts the audit trail on the failure path and every §5 checkpoint; copying at the end would lose it on exactly the runs most worth auditing.
### Authorized tool use in the pipeline (§25)
The pipeline could not call any approval-gated tool in any shell — the pass entry point had no way to receive a `permission_channel`, so every pass ran headless. Authorization is now **two independent axes**, not a mode: a grant set (possibly empty) and an `ApprovalProvider` (possibly absent). Both absent is the default and changes nothing.
- **A grant covers only tools with no `approval_check`** (R2, `registry.grantable`). A per-call gate was never consented to by name. This *narrowed* §18's shipped sign-off at the shared mechanism — a signed-off subagent calling `shell` now prompts its parent.
- **`spawn_subagent` is excluded from *every* grant path** (R4, `grant_policy=GRANT_NEVER`): approving a spawn *is* the §18 sign-off, so pre-granting it compounds one yes into unbounded delegated authority. "Every" is R13's correction — the exclusion used to live in a frozenset in the pipeline's module, so the sign-off R4's argument is *about* went on offering it (#67).
- **Grant policy is declared on the tool, and both callers read it** (R13). `GRANT_ANYWHERE` / `GRANT_SIGNOFF_ONLY` / `GRANT_NEVER`, required on every static registration and enforced at import by `assert_grant_policy_declared`. The pipeline accepts only `ANYWHERE`; the sign-off accepts anything but `NEVER`, so it is strictly looser and `write_project_doc` sits in the gap — **as a policy value with no shipped instance**, which is `docs/TECHNICAL_DEBT.md` 83: `GRANT_SIGNOFF_ONLY` has exactly one member in the registry, and no agent this project ships declares it. A denylist defaulted an unknown tool to *grantable*, which is how `write_project_doc` became the entire `--grant` menu (#133).
- **`candidates()` is empty on a default install, by design.** Every built-in is ungated, param-dependent, or excluded by policy; the population both grant paths serve is MCP tools, which are allowed-but-gated by default and carry no `approval_check`. R1's "the description is the whole of informed consent" was written about exactly them.
- **The SIGN-OFF path is empty on *every* install, for the shipped roster** (batch 139's owner ruling, measured; `docs/TECHNICAL_DEBT.md` 83). Same population, one extra filter: `candidate_approvals` intersects it with the CHILD's reachable tools, and all six spawnable shipped agents narrow `allowed_tools` to built-ins. Configuring an MCP server does not help, because an agent file cannot name `mcp__server__tool` for a server that did not exist when it was written. So `SubagentSignoffScreen`'s picker branch is reachable only for an agent that declines to narrow or names a gated tool outright — a user- or project-tier file. **The subset picker is for operator-configured installs; the shipped roster deliberately declares nothing to pick**, and every shipped spawn shows the "needs no approval-gated tools" branch instead.
- **The ADVERTISEMENT filter reads the grant too** (R15, `schemas(…, granted=)` / `headless_hidden(…, granted=)` sharing `_answered_by_grant`). `headless = response_channel is None` asks "can anything ask?"; since §25 the question is "can anything *answer for this tool*?", because a grant is an answer given early. #156: `--grant-tools` without `--attended` sent the model byte-identical tools to an unflagged run while the CLI printed that it had authorised something — **§25's own founding defect, still standing on the path §25 was written for**, since `--attended` cleared `headless` and grant-only never did. The filter re-derives *both* `grantable()` and `grant_policy == GRANT_ANYWHERE` — `ANYWHERE`, not `!= NEVER`, because it is reached only when the run is headless, which is the unattended case `SIGNOFF_ONLY` excludes. **Three readers of `grant_policy` now, and they do not all want the same predicate:** `candidates()` and this filter ask *may an unattended run have this?* (`== ANYWHERE`); the loop's grant branch asks *does a name grant apply at all?* (`!= NEVER`, so a signed-off subagent keeps its `SIGNOFF_ONLY` tool). The branch must not apply the policy to a subject-keyed memo — `spawn_subagent` is `GRANT_NEVER`, so that stops §23's memo working; `recall_signoff` separates them by returning a subset for a memo and `None` for a name grant.
- **`GrantBudget` exhaustion degrades to *asking*, not to failing.** One instance shared by reference across all ten passes; rebuilding per pass would multiply the ceiling by ten while reading as if it enforced one. It is deliberately **not** consulted when deciding what to advertise — R6's fallback is a *call-time* answer, and retracting a tool mid-run would read to the model as the tool ceasing to exist.
- **A headless denial names its CAUSE** (#158). `callable_only` probes with empty params, so for a params-dependent tool it guarantees *some* call shape is callable, not this one — `write` is advertised in a pass and denied per out-of-workspace path. The denial therefore distinguishes *gated by name, nothing can ask* (stop) from *gated by these arguments* (vary them). The old wording told the model to retry with approval, which headless is the one thing it cannot get. No tool name and no notion of a path is in `core/loop.py` — `ToolSpec.grant_scope` and `request_kind` exist to keep it that way.
- **A headless run cannot spawn at all** (R16). `spawn_subagent` is gated with no `approval_check`, so it never reaches a headless schema list; every run that *can* spawn has a channel, and `subagent_tool` forwards it. That is the half of r3-1 nobody wrote down, and one config flag used to remove it with the suite still green (#159).
- **`ApprovalProvider` exists because `run_to_completion()` discards the `permission_request` event.** A bare queue would block with nothing displayed. §23 should absorb it into the general response channel, not add a third mechanism.
- **The grant list is never persistable; the mode is** (R12). A persisted mode can only add prompts; a persisted grant list could only remove them, and `settings.json` is the one config file where project tier beats user tier. `research.granted_tools` is rejected *by name* so nobody "fixes" the omission.
The eight JSON-emitting passes go through `_run_pass_with_json_retry()`, which re-enters the *same* pass thread via `continue_conversation()` so the model sees its own malformed output. Passes 1 and final synthesis are plain text and use bare `_run_pass()`.
### Post-pipeline review (§20)
The research pipeline can now review its own finished output and correct it, one consented change at a time. Off by default (D9: `config.SUBAGENT_REVIEW`, `settings.json` `research.subagent_review`, `--review` / `--no-review`, `/research --review`).
- **The orchestrator invokes it; the shell supplies consent as data** (V3). `core/reasoning/review.py` owns what may be corrected, the pipeline decides when. The alternative — a post-pipeline stage each shell calls, mirroring `write_run_artifacts` — is exactly the shape that left `/research` in the TUI writing no output directory at all.
- **"Requested" and "can be asked" are two parameters** (`subagent_review` and `review`). `build_review_consent()` returns `None` on a non-tty stdin, so collapsing them would make `--review` on a piped run skip the review entirely and report nothing.
- **No consent route means nothing is applied** (V6). The review still runs and still records. Same rule §25 applies to gated tools: the inability to ask is not permission to proceed.
- **The reviewer inherits the run's `RunAuthorization` unchanged** (V7) — same grants, same provider, same `GrantBudget` **instance**. No new security axis.
- **A refinement re-enters the reviewer's own thread** (V5, via `continue_conversation`) and touches only its own finding. A note about #3 must not redraft #7.
- **The reviewer opens a §47 span, like a pass** (batch 75). It forwarded the sink and opened nothing until then, and forwarding is not being visible: `run_agent_conversation` only ever calls `bind()`, which is a no-op with no span open. So the reviewer drew no sidebar row, was absent from ctrl+g, could not be opened while it ran, and asked for its gated tools with no "asked by" line — a nested run's question presented as the conversation's own. ONE span over the whole call, retries included, which is `compact()`'s rule for the same shape; the depth is derived from the context `run_review` already builds, never written as a 1. A refinement gets none: it continues a thread whose run has finished, in front of a reader who triggered it.
- **Four consent outcomes, and only one of them can ever accept.** `reject_all` is the escape a long review needs precisely because it only declines; an accept-all shortcut is the affordance that must not exist, since an injected "correction" needs one reflexive yes.
- **Every unclear answer is a rejection** — an unrecognised string, a callback that raises, a timeout, a TUI modal dismissed with the bare `False` its shutdown path puts. This is the fifth place the "every dismissal carries a value" invariant applies and the first where the unsafe failure is silently applying an edit rather than hanging a worker.
### Compaction and pinning (`core/compaction.py`, §21a)
A long thread is now condensed rather than left to hit a wall. Compaction adds a
`CompactionCheckpoint` row and **never edits the archive** — `storage.archive_history()`
returns every message ever written however many times this has run (§21 AC1), which is
what makes an early, frequent trigger safe.
- **The trigger is a SHARE of the model's window** (WS5, reversing M1). M1 made it a
flat 40k and argued that a window-derived threshold "sits at a size a working
thread never reaches" — arithmetic about `MAX_TOKEN_BUDGET`, then a default spend
ceiling of 250k. **Batch 27 deleted that ceiling**, so nothing competes with a
large trigger and the constant outlived its own argument: a 1M-window thread was
folding at 4% of its window. Still not `window - margin` — that shape is the
BACKSTOP's and cannot be right at both ends of the roster, since 40k below 1M is a
rounding error and 40k below 64k leaves 24k.
- **What the fraction NAMES is the headroom, and batch 119 clamped it at both ends**
(TECHNICAL_DEBT 39). `compaction.derived_trigger()` is now `context_limit()` minus
`headroom_for()`, where the headroom is `COMPACTION_TRIGGER_FRACTION` (0.90)
applied to the window, held at or above `COMPACTION_MIN_HEADROOM_TOKENS` (16,000)
and at or below `COMPACTION_MAX_HEADROOM_FRACTION` (0.25) of it. **Exactly the
fraction at every window at or above ~160k**, so a 262k or 1M thread gets what the
number says; a floor below that. The reason is the mirror image of batch 44's
against a flat margin: a bare SHARE holds the percentage constant while the thing
the headroom must contain does not scale with the window at all — one
`max_read_chars` result is 12,500 proxy tokens whether the model holds 64k or a
million, and one measured `arxiv_search` result was 29k. **There is no
context-overflow handler anywhere in `core/`** (grep for one): the provider raises,
the loop retries the identical oversized prompt three times, the turn dies. This
number is the only thing between a thread and that, which is why it is the thing
with the floor on it rather than the trigger.
The derived value is a DEFAULT and sits LAST: an explicit `/compact` override, a
session `/trigger` bound to its `(provider, model)`, and settings.json's
`trigger_tokens` all outrank it, arriving at `thresholds()` through D27's existing
per-invocation slot rather than as a fifth tier. `COMPACTION_TRIGGER_TOKENS`
survives as the trigger for a caller with no model in scope, which is
`pin_measurements` when nothing passed it a pair.
**AND THAT SENTENCE USED TO END "so #89's cap is unchanged", which was true
of the cap's VALUE and false of the property it bought** (batch 148,
S11-06). #89's cap is a fraction of the trigger, batch 44 moved the trigger
off this constant, and the cap kept reading the constant — so it was
20,000 for every model: 250% of the trigger that fires on an 8,192-token
window, where one pin could protect more than the whole trigger and floor
the thread permanently above it, and 8% of it on a 262,144-token one,
where `pin` refused what the policy allows. `pin_measurements` now takes an
optional `(model, provider_name)` and resolves the trigger the way
`thresholds()` does; `tools/builtin/pin.py` declares `parent_run`, which
already carried both. The bare read is the FALLBACK, for a call from
outside a run (A22).
- **And there is a THIRD clamp, below the two above: the dependent floor**
(TECHNICAL_DEBT 42, batch 127). `effective_compaction()` validates the trigger
against two settings of identical form — `warning_margin_tokens` and
`keep_recent_tokens`, each of which must be STRICTLY below it — so
`derived_trigger()` floors at `max(margin, keep_recent) + 1`. Without it the cap
above held the trigger at `0.75*window` at the small end, which clears
`keep_recent_tokens` from a window of 5,334 up but `warning_margin_tokens` only
from **10,667** up: a provider reporting **8,192** derived 6,144, and
`should_compact()` raised `ValueError` at the top of every step with both
`_maybe_compact` call sites bare, so **every turn on that model died before the
first model call**. Reachable by what the PROVIDER reports, never by `/window`,
which refuses anything at or below 20,000.
**The floor is READ, through `effective_compaction()` itself, and a literal here is
a defect** — it must be the same resolution the validator uses or it drifts from the
number it exists to satisfy the moment someone edits `config.yaml`. **Below ~10,667
the returned value is the floor rather than the share, so the fraction stops
describing it**; that is stated rather than discovered. Do not "fix" this by
catching the `ValueError` at the call sites: compaction would then silently never
run on that model.
- **`context_limit()` now decides when every thread folds, so its sources matter**
(WS6, WS10). Order: a REMEMBERED window (`core/model_windows.py`, written by
`/window`, kept per `(provider, model)` across launches) → the provider's own
answer (`client.context_window_for`, primed before every send) → the normalized
`MODEL_CONTEXT_WINDOWS` lookup — a dated or `vendor/`-prefixed name used to miss
the table and take the default silently — → `DEFAULT_CONTEXT_WINDOW` (256k),
warned. The remembered value outranking the QUERY is a deliberate inversion of
this project's usual direction: a query can be confidently wrong where a typed
number is a statement about the deployment being run. The query itself falls back
from `models.retrieve` to `models.list` because a `vendor/model` id has no per-id
route on OpenRouter or NVIDIA — the reader was always right and the ROUTE was the
bug. A missing window still also sets `_input_budget()`, which gates the lossy
truncation in `summarize_thread()`.
- **The spend meter is optional; the size instrument is not.** Batch 27 deleted the
constants: no default ceiling anywhere, and the only cap is the user's
`max_token_budget` setting. What every caller gains unconditionally is the size
instrument — `ModelResponse.turn_billed_tokens` (running spend) and
`.turn_new_tokens` (what this turn added to the thread: outputs plus positive
input deltas, inherited first call excluded, compaction shrinks clamped), both
unpriced counts — plus the TUI usage line (billed-since-resume · ctx) and CLI
early-stop figures. Do not read EITHER number as a price or the spend figure as
a size.
- **The fold boundary never splits a step** (M4, as restated by batch 120). A summary
that swallows an assistant message and leaves its `tool_result` rows is an HTTP 400
on Anthropic — a hard wire error, not a degraded answer. This bullet said "always a
turn start" until batch 120, which was the IMPLEMENTATION rather than the rule: a
turn start satisfies M4, and so does the row before any **assistant** row inside a
turn, because the archive there is `user, (assistant, tool+)*` — D20 persists the
assistant row before dispatch and every answering row after it returns, a parallel
dispatch included. So the rule is unchanged and the set of positions satisfying it
grew. A turn-aligned boundary is still PREFERRED and a sub-turn one is reached only
when no turn-aligned fold exists at all; the second boundary `pinned_through()` has
always owned is the other place M4 reaches (#88).
- **Three floors bound what may be folded, earliest wins** (M5): the current turn
(structural — `_run()` records how many turns were complete when it began), the
last 3 completed turns, and `keep_recent_tokens`. The current-turn floor is what
makes the mid-turn valve safe by construction rather than by hoping the threshold
is far enough away.
- **AMENDED BY BATCH 119: the turn COUNT is a preference, the other two are
absolute** (TECHNICAL_DEBT 39). Only two of M5's floors can measure what they hold.
The current turn is structural. `keep_recent_tokens` is a SIZE, so it holds exactly
what it says and can never be what holds a thread that is over the trigger.
`keep_recent_turns` is a COUNT, and a count cannot tell three small turns from
three enormous ones — **measured at 230k tokens held behind `3 - 3 = 0` on a
thread 92.5% of which was foldable**, with no way out from inside the app: this
function takes no model, so a 1M-context model recomputes the same zero; it reads
the archive, so a restart re-reads the same rows; `/compact` parses only
`--strength`. So the count applies while it still leaves something foldable and
otherwise yields — ALL the way to the size-aware floors, not one turn at a time,
because one step of relaxation on that thread folds 253 tokens, writes a
checkpoint, and then answers `no-progress` forever with `advances()` satisfied
against a watermark that freed 0.1%. The guarantee is size-based now: the current
turn verbatim, plus at least `keep_recent_tokens` of completed turns verbatim.
- **AMENDED AGAIN BY BATCH 120: the current turn's MIDDLE may fold** (TECHNICAL_DEBT
40). The structural floor was the one batch 119 left absolute, and a thread whose
only turn is the one still running then has no legal fold at all — the reported
thread was 142 rows and 168,311 proxy tokens inside ONE turn, where the best
turn-aligned boundary freed **253**. M5's reason is that folding the current turn
"would rewrite the question mid-answer", and that reason does not cover the whole
turn: after forty-six steps the model is not reasoning about step three's tool
output. So the request and `keep_recent_tokens` of recent steps stay VERBATIM and
the middle folds — the question is carried, not summarized. The request comes back
through `pinned_through()`, which already owned exactly this question, as ONE index
into its existing `keep` list so its existing sort places it. 97.2% of that thread
becomes foldable; measured, with M4 checked by walking both sides of the cut.
- **Chain is the default; rederive survives as the fidelity option** (M2, as
amended by batch 16 — an explicit owner decision recorded in DEVLOG). M2's
original choice was rederive: exactly one summarization step always sits
between an original message and what the model sees. Cost flipped it —
rederive's input is the WHOLE covered span, making every compaction the
most expensive call of the turn on exactly the threads that trigger most.
Whichever strategy is configured, a span that outgrows one call falls back
to chain with a WARNING (#90's fallback, built after three documents
promised it), and one that outgrows even chain truncates its oldest
material with the cut stated in both the instruction and the stored
summary.
- **The summary is a synthesized leading `user` message and is NEVER persisted**
(M8). Persisting it would put derived content in the archive, and the next
compaction would fold a summary into a summary on the path that means to re-derive.
- **Turn counting is ARCHIVE-space, never view-space** (M11), and this is M8's
other half. Because the summary is a `user` message, the derived view holds a
user message that is not a turn — so any turn count taken from
`memory.messages` is off by one AND in a different space from the archive
indices `compactable_span()` compares it against. §21a shipped that way: the
watermark went backwards and the second automatic compaction of a thread
UN-COMPACTED it, invisibly to 849 tests, because the two spaces coincide until
a checkpoint exists. `ConversationMemory.completed_turns()` is the only source.
- **The watermark only ever moves forward, and that is enforced twice** —
`compact()` skips before spending a model call, `save_checkpoint()` refuses the
write. `latest_checkpoint` resolves by timestamp, so a backwards watermark does
not merely fail to help: it becomes the live one.
- **Pinned rows inside the covered span are re-added verbatim** (M9). One watermark
cannot say "everything through K except these", so without this the message a user
explicitly asked to keep is the one that disappears.
- **M4's rule reaches the PINNED boundary too, and `pinned_through` is where** (#88).
The derived view has two span boundaries and only the watermark is turn-aligned. The
pinned set never can be: `pin` is a tool, D20 persists the assistant turn carrying its
own `tool_use` before dispatch, and the answering row is written after dispatch returns
— so it is never pinned. Once a watermark passes the pinned region the view carries an
unanswered `tool_use`, which is the same HTTP 400 M4 exists to prevent, arriving through
the boundary M4 does not mention. Fixed at the producer; `_derived_view` dropping
unanswered calls would be consumer-side and would hide from the model a call it made.
M4 and M9 are each implemented exactly and neither names the other — the composition is
what nobody wrote down.
- **A research pass compacts only at a hard backstop** (M6), near the real context
window. Routine compaction in an unattended pass would spend on a judgment call
nobody is watching.
**"Only at a hard backstop" is an ORDERING claim, and until batch 148 the
arithmetic broke it** (S11-04): the backstop was a flat 20,000-token margin
while batch 44 made the ordinary trigger a clamped share, so the backstop
fired EARLIER than the trigger it sits behind at every window below
200,000 — 4,000 tokens earlier at 64,000, 11,808 at 32,768, and at or
below 20,001 its `max(1, ...)` clamped it to 1, so every evaluation of
every step of every pass asked `compact()`. It is now
`max(derived_trigger(...) + 1, window - margin)`, which makes the order
structural rather than arithmetical. The shipped table is why nothing
caught it: every entry is at or above 200,000, where the two numbers were
equal or already in the right order, and the way in is a window a provider
REPORTS.
- **`context_limit()` warns once per MODEL, not once per call.** `thresholds()`
calls it on every evaluation and `_maybe_compact` runs at the top of every step
of every turn — so an unknown model produced a WARNING twice a step across ten
research passes, which on the TUI was also painting over the screen. Keyed on
the model rather than a single boolean for the same reason
`_headless_notices_shown` keys on the hidden set: a run that switches model, or
a pipeline whose critic model differs (§11), must still hear about the second
one. `tests/test_compaction.py` clears the set in an autouse fixture — it is
module state, and a warmed set makes the fallback test pass or fail depending
on which tests ran first.
- **The ratio is a character proxy and says so** (M10). No tokenizer offline; it is
self-consistent and never compared against a provider's token count.
- **Notices travel on the ModelResponse AND as a `LoopEvent`.**
`run_to_completion()` discards non-final events, so the event route alone is
invisible to the CLI and the pipeline. Third time this shape has come up (§20,
§25), so it is a rule now.
- **A failed compaction is contained**, like §20's review stage: the turn completes,
a `compaction_failed` notice says so, the thread stays uncompacted. And since #136
it LATCHES: one failed attempt per turn, not one per step — an empty compactor
response rides the same kind (status `"failed"`), and `_maybe_compact` returns before
calling `compact()` again once this turn has seen it. The cheap outcomes (blocked,
no-progress, reentrant) are deliberately not latched — they exit before any model
call — and a test pins them at three evaluations so that boundary cannot drift silently.
- **`pin` is ungated deliberately** (D26). §13 does not merely deny an approval-gated
tool where nothing can ask — it stops advertising it, so gating `pin` would make it
invisible on the CLI and in every research pass. Its interface is ordinal
(`last_n` turns), because `MessageLog.id` is deliberately absent from the neutral
message shape.
**`database.ensure_columns()` exists because `create_all()` never `ALTER`s** (M7).
Adding a field to a table that is already on disk breaks every existing database at
read time. Additive only — it warns rather than raises on a type mismatch, since
SQLite's type affinity makes most mismatches harmless and a raise here would brick
every launch. If it ever grows a `DROP`, the project has outgrown it.
**Nullability rides along where SQLite allows it (#26).** A not-null column WITH a
scalar default migrates as `NOT NULL DEFAULT x`, matching what create_all permits;
a factory-default column cannot be added NOT NULL at all and lands nullable under
a WARNING. The two columns migrated before this existed (`pinned`, `kind`) remain
nullable on pre-#26 databases — accepted, because flipping them needs a table rebuild
and no writer can produce a NULL for them. One inherent residue, documented in the e2e
test: the migrated column carries a DEFAULT literal and create_all emits none, because
ALTER cannot backfill without one.
**The three public reads return plain dicts (#31), and `save_message` stamps
`last_activity_at` (#32).** `get_thread` / `latest_checkpoint` /
`latest_thread_summary` copying columns is the same rule `_ordered_rows` states —
two of the three sit on the compaction path — and the stamp rides the single
MessageLog writer so a message cannot persist without its thread moving to the top
of an activity-ordered picker. `advances()` checks containment even when no
checkpoint exists yet (#27): "first compaction" is exactly when nothing else catches
a foreign watermark. The TUI picker caps at 200 and says so; `/resume <thread-id>`
is the by-id path that keeps older conversations reachable (#30).
### Durable memory (`memories/`, §21b)
`remember(content, scope)` writes a row that outlives its thread; an opted-in run gets
those rows injected as a third context tier beside the project hub. `use_memory` had
been parsed since §14 with no consumer at all — this is its first.
- **Turn counting's sibling trap: the fragment goes OUTSIDE `with_catalogs()`** (M13).
That function feeds `pass_prompt()`, so a memory appended inside it lands in all ten
research passes — §19's K6, second instance. The fragment is appended by its three
callers: `system_prompt_for` for agents, and the no-agent path in each shell.
- **Plain chat counts as opted in** (M13). `system_prompt_for` is unreachable without
an agent, so the literal "an agent opts in" reading would make the whole feature
invisible in the default shell.
- **An unresolved project means NO memories, not global-only.**
`config_loader.get_project_path()` is `None` exactly when `initialize()` has not run,
and that is "there is no session". Same convention as `get_agents()`.
- **`UserMemory.project_path` is M12's addition to §21's schema.** D25 keys project
memories to the resolved path; with only a `scope` string, §21's AC6 is unwriteable.
- **Injection is capped and says so** (M14). `MAX_INJECTED_MEMORIES`, newest first,
with "showing the 50 most recent of 73" in the fragment the model reads — not only
in a log, which reaches nobody mid-conversation. **A cap of `0` is OFF, in both
halves, and off is silent** (A51, batch 154). `remember`'s `available_check` has
always hidden the tool at 0; `prompt_fragment` returned `""` for nobody, so
`rows[:0]` plus `total > cap` announced "showing the 0 most recent of 2" with a
WARNING behind it on every turn — telling the model that memories exist and are
being withheld, while the tool that could have earned them was not on the menu.
The two halves are asserted together, in one test, because the defect was that
they disagreed while each was separately defensible.
- **`remember` declares `grant_policy=GRANT_NEVER`** (M17, carried by R13). It has no
`approval_check`, so §25's R2 would make it grantable and one `--grant remember` would
let ten unattended passes write durable memories — exactly what D26's consequence 1
forbids. It used to sit in a `PIPELINE_UNGRANTABLE` frozenset, which is where #67 found
the gap: D26's consequence 1 is about what a *subagent* can do and not only a pass, and
a set scoped to the pipeline's module never reached the §18 sign-off. `NEVER`, not
`SIGNOFF_ONLY` — a durable cross-session write is precisely the authority that outlives
the turn somebody was watching.
- **Removal is by id, never by substring** (M15). A substring matching two memories
would have to pick one, and picking silently deletes what the user did not name.
- **CLI chat now has an `ApprovalProvider`** (M16), so `spawn_subagent` and every MCP
tool are advertised and promptable there, not just `remember`. `None` on a
non-tty, so piped runs stay headless. **Not `shell`** — this sentence used to list
it (audit #21), and `shell` is globally denied like `read`/`write`/`edit`:
`is_tool_allowed` reads `config.ToolPermissions()` directly and returns before any
context or approval logic, D14 forbids widening, and `settings.json` has no
`permissions` section. Its whole approval apparatus is unreachable by
construction, and so is `security/sandbox.py` plus `config.yaml`'s sandbox block,
until someone edits `config.yaml` — which is the harness's own file, has exactly
one location and no override variable, and is outside the workspace, so writing
it needs a human exactly as editing `config.py` did.
### §29 — the CLI shell (N1–N8)
`main.py` reads stdin through **one** `_StdinReader`. There is no live `input()` in that module and a
test enforces it: `input()` is a second reader on the descriptor the approval pump already owns, and
that is audit #100 — after the first gated call the pump takes every line and chat blocks forever.
Two methods, and the difference is a safety property rather than a preference:
- `ask(prompt, timeout=None)` — the gated prompts. **Drains** stale input first, because a line typed
after a question timed out was answering *that* question.
- `readline(prompt)` — the chat prompt and the one-shot startup prompts. **Keeps** type-ahead, blocks,
raises `EOFError` on Ctrl+D. Do not add a `drain=` flag; that puts the rule one keyword argument
away from being switched off at an approval.
**THE RULE IS "ONE READER AT A TIME", AND BATCH 121 IS WHAT IT COST TO LEARN THAT.** N1 above is
scoped to **this module**, and the biggest reader of this process's stdin is not in it: Textual reads
the same console from its own thread (`win32.EventMonitor` on Windows, a selector on fd 0 on POSIX).
So the trust prompt — which fires whenever anything under `.venastine/`, or the root `AGENTS.md`,
changes — left `_pump` parked in a blocking read, and the TUI that started two lines later lost every
keystroke to it. Measured with the real app, one flag apart, and the same on both platforms:
| state | the app's `on_key` | shutdown |
|---|---|---|
| a reader parked | never fired | **`App.run()` never returned**; the process had to be killed |
| batch 121 | all five injected keys | exit 0 |
The second column is why the owner's only way out was Alt+F4: `disable_input()` joins an input thread
stuck in a read whose records the pump had already taken, so even a quit cannot complete. And Ctrl+C
is no escape because `enable_application_mode` clears `ENABLE_PROCESSED_INPUT` — Ctrl+C is just
another key event in the queue the wrong reader is draining.
Two changes, and the first is the fix:
- **`readline()` reads on the CALLING THREAD when no pump exists.** It has no deadline (N2), so there
is nothing a thread could be abandoned *for*. Once a pump exists the queue path is taken unchanged,
which is what keeps #100 fixed — the pump is authoritative the moment it exists, and before that
there is nothing to contend with. Type-ahead is unaffected: the **terminal** buffers an early line,
and always did; the queue only ever held what the pump had already taken out.
- **`main.py` hands the terminal over, by name, immediately before `run_tui`.** `_release_terminal()`
latches the reader closed, so `_start()` refuses afterwards and every later read answers the way it
answers a closed stdin. It returns `False` if a pump is somehow still parked — nothing can produce
that today, and it is reported rather than asserted because the alternative symptom is a UI that
animates and ignores the keyboard with nothing anywhere saying why.
`ask()` is untouched. A deadline cannot be imposed on a read that cannot be abandoned, which is the
entire reason the thread exists.
The same release covers `replace_process`: on Windows the waiter keeps this process alive with the
console inherited, so a parked reader of ours would have read the **replacement's** keystrokes.
**Pre-mount notices travel into the app** (`tui.app.run(startup_notices=...)`). Anything printed
before Textual renders is erased by it and `TranscriptLogHandler` attaches only at mount — #138 and
batch 87 each solved that once, by recomputing and by leaving a file. Neither works for a fact
`main()` established and the app cannot rediscover, which is what these are. This is **not** batch
44's `startup_warnings` returning: a provider finding is recomputable, and computing it in `main()`
froze it against the config pair while the app restored a remembered one.
**The deadline belongs to the channel, not the kind.** `build_attended_provider(timeout=…)` decides
it: run-backed builders pass `ATTENDED_APPROVAL_TIMEOUT_S`, `--init` passes `None`. The default is a
`_DEFAULT_TIMEOUT` sentinel resolved in the body — a `timeout=config.ATTENDED_APPROVAL_TIMEOUT_S`
default is evaluated at import and silently defeats `conftest`'s `shrink_approval_timeout`.
**Every kind in `interaction.SAFE_DEFAULTS` must have a handler in BOTH shells**, and a test
enforces it. The CLI's is the `handlers` table at the end of `build_attended_provider`, published
for that test as `channel.rendered_kinds`; the TUI's is the dispatch in
`VenastineApp._ask_blocking`. `tests/test_cli.py::TestEveryKindIsRendered` is the enforcement: it
holds the CLI's table against `SAFE_DEFAULTS`, drives every kind to see that a human was actually
asked, and reads the TUI's source for the matching branch. A missing branch is not a missing
feature — it falls through to the kind's declining default, silently, while looking wired up (#7). The name this
sentence used to give the CLI's table existed nowhere in the tree, for 24 batches;
`test_every_private_symbol_agents_md_names_exists_in_the_tree` is why the next one cannot.
**Nothing runs before `parse_args()`** except building the parser. `--help` and a typo'd flag must
leave the filesystem untouched (#101). `_classify_legacy_threads()` runs below the early-exit
commands: it exists for the thread picker, and none of them shows one.
**Tests type at the CLI through `cli_stdin`**, not by patching `builtins.input` — that stopped being
the seam, which is the fix working. `FakeStdinReader` records `prompts` and `timeouts`, so a test can
assert what the user was asked and whether it carried a deadline.
A mutation that strands a reader **hangs** the suite rather than failing it, since two read paths now
have no deadline. The mutation harness reports HANG as its own outcome.
### The shell gate (`§28`, G1–G7; routing and disclosure amended by `§46`, EP4–EP8; what `tiered` approves amended by `§48`, CE1–CE7; the container runtime amended by ROADMAP_v3 `§49`, SS22–SS24; declared network added by ROADMAP_v3 `§50`, NW1–NW5; a third backend added by ROADMAP_v3 `§49` slice 3, SS35–SS45; a fourth by slice 4, SS46–SS55; the harness-held secret store and SSH credentials by slice 5a, SS56–SS64; root commands by slice 5b, SS65–SS72)
Read §28's record before touching `security/sandbox.py`, `security/capability.py` or
`_shell_approval_check`, and §46's before touching where a command RUNS. The four things most
likely to be re-derived wrongly:
- **INERT runs in the CONTAINER; only HOST_READ takes the host path** (EP5). This inverts what
§28 shipped and what the older comments in this file describe, so it is first. The argument is
the tier's own definition: INERT means every argument resolves inside the workspace, which is
the directory the container mounts, so the tier the gate AUTO-APPROVES can run against a pinned
image's coreutils. HOST_READ is defined by an argument the container *cannot see*, so it has
exactly one backend — and `auto_approved` never covers it, because `escapes_workspace` is True.
Docker being down falls INERT back to the host; that changes nothing about the approval answer
(`containment_for` explains why, and a test pins it) and is reported to the model as
`ran_on: "host"`.
- **`containment_for` and `run_sandboxed` are ONE decision written twice** (EP6). Drift between
them is what #157 was. Change the routing in one and you change it in the other, in the same
commit, or a command is auto-approved as one thing and executed as another.
- **Every result says where it ran, and `shell.run()`'s error paths must not** (EP4). `ran_on` and
`tier` are added at `run_sandboxed`'s routing sites. The gate had told the *human* "Runs on the
HOST, with your own file access" since §28 and told the *model* nothing, so a host read and a
contained run were indistinguishable to the party that writes the user an answer about the
output. An `{"error": ...}` never executed anything and annotating one would claim a run that
did not happen.
- **On Windows the host path resolves its binary off the user's PATH, and this is disclosed, not
fixed** (EP8). `ls`, `cat` and `grep` have no native Windows binary, so whichever POSIX
toolchain is first on PATH is what runs — on the machine where this was found, one shipped by
unrelated software. `_is_inert` rejects a path-qualified binary so a planted one cannot be
NAMED; PATH lookup reaches the same place from the other side. After EP5 it applies to one tier
that always asks a human, and the notice reports the resolved absolute path. **Do not "fix" this
by resolving from `AGENT_WORKSPACE`** — that directory is model-writable, so it would let the
model plant a binary and have the harness execute it on the host under an auto-approval. §46's
Rejected section has the rest.
- **`SANDBOX_DOCKER_IMAGE` is a tag, not an identity** (EP7). The resolved image ID is logged once
per process and containers carry `--label venastine.sandbox=1`. Digest-pinning was considered
and deferred.
And from §48, which amended what `tiered` approves (CE1–CE7):
- **`tiered` asks about anything that may run code, contained or not** (CE1, amending G4). G4
approved every contained command without network, so `python -c`, a script the auto-approved
`write` tool had just made, `pytest` (a `conftest.py`) and `make` all ran unasked — and code read a
workspace's `.venastine/` straight past the token check, which three documents said could not
happen. The capability is `CommandProfile.runs_code` (CE7), True for every non-INERT profile.
**Do not add an interpreter list.** `"python"`, `pyth?n`, `python3.13`, `echo x | python` and
`find -exec` are the same call to a shell; a list is G2's parser whose misses auto-approve.
- **`contained` is the old rule under its own name** (CE2), applied in `_shell_approval_check` as a
visible override below the protected-segment check, beside the fallback opt-in. Its constant is
`capability.CONTAINED_MODE`, **never `CONTAINED`** — that name is a *containment* value with the
same spelling, so a mode compared against it matches and reads as correct.
- **The fallback opt-in is honoured under `tiered` and `contained`; `always` beats it** (CE5, CE6).
`unsafe_reasons()` has to follow the gate rather than the flags: under `always` the pair is not
"no ask", and the badge said it was. A test holds the badge against the gate in every mode.
`contained` itself is on the badge (CE4).
And from §49 (`docs/ROADMAP_v3.md`), slice 0 -- Podman (SS22–SS24):
- **The container route has two runtimes and one probe** (SS22). `security/sandbox._probe()` asks
`docker info` first and Podman only when Docker is missing or its daemon does not answer, once per
process under a lock. `is_docker_available`, `_run_docker`, `docker_available` and
`sandbox_docker_image` keep their names and now mean the container route: renaming the key would drop
a user's own value in `config_update.py`'s merge. **The CLI a route invokes is `known_runtime()`, which
never probes.** A test that patches availability gets Docker's CLI, and GitHub's runners ship Podman,
so a probing read would change the argv under CI.
- **A Podman that cannot enforce the limits is not a runtime** (SS24). Rootless on cgroups other than
v2, or without cpu/memory/pids delegated, accepts `--memory` and ignores it.
`_why_podman_cannot_limit` reads `info --format {{json .}}` and fails CLOSED on anything unreadable. A
`docker` that is really Podman is recognised by `buildahVersion` in its `info` output and checked the
same way -- otherwise the podman-docker shim skips the check by name.
- **The shipped image is fully qualified** (SS23). Podman resolves a short name through
`registries.conf`, and `python:3.13-slim` resolved on the machine it was measured on only because that
distro's `shortnames.conf` aliases `python`.
And from §50 (batch 99) -- the agent declares that a command needs the network (NW1–NW5):
- **`requires_network` is OR'd into the one `network` fact, never an override** (NW2). It can only ADD
egress, so a model cannot clear it on `curl` to dodge a prompt, and the single fact keeps its two
consumers -- the gate and the runner -- reading the same value. **Nothing new gates it** (NW3): the
existing ladder does the work, and `never`/`always` still decide for themselves.
- **`security.sandbox.declared_network(params)` is THE coercion, and only literal `True` counts.** The
gate reads the model's tool-call input BEFORE Pydantic validates it, so the value can be any JSON at
all. The param models use `StrictBool`, so `"true"` is False at the gate and a `ValidationError` at run
time -- nothing executes, and the two readings never disagree. **Do not "helpfully" widen either half**:
a lenient runner with a strict gate is a command approved as no-network and executed with it, which is
#157's shape in the module written to close it.
- **`contained` is the mode where the flag changes the approval answer** (measured). Under `tiered` a
non-inert command already asks because `runs_code` is true (CE1), so a test asserting the flag's effect
through `tiered` passes without the flag doing anything.
- **The three tools share ONE description string** for the field (NW4, NW5): `shell.ShellParams`' is the
copy `shell_background` and `shell_monitor` read.
And slice 2 (batch 100) -- an interactive shell the agent types into (SS25-SS34):
- **The pty is made INSIDE the container**, by `script -qfec`, not by `docker run -t`. The client then
never needs a real terminal and docker never mangles the stream. `-i` keeps stdin a pipe, and
`_popen_session` takes its stdin rather than asserting `DEVNULL` -- N1's reason survives the constant,
because what it forbids is a child INHERITING THE CONSOLE and a pipe only the harness writes to is not
a second reader of anything.
- **The echo is OFF and the pty is sized** (SS33). Measured: the pty is 0x0, so readline echoes a
200-character command back as `\r<xxxx`, its horizontal-scroll marker, which no honest rule could
strip. Turning the echo off makes the problem not exist at any length instead of parsing it back out.
- **The prompt is a token this process minted** (SS34), so "the shell is ready" is READ rather than
inferred from silence. Its consequence is the thing to hold on to: with the echo off, NOTHING appears
on the stream when a command starts, so a prompt still standing from the last command is
indistinguishable from an idle shell. A send therefore waits for the NEXT prompt and requires it to
settle for the quiet interval. **Do not "optimise" that settle away** -- it is what stops a prompt
merely passed through on the way to a queued line being reported as the answer.
- **`send` always returns** (SS26), on whichever of four comes first -- `ready`, `matched`, `quiet`,
`bound` -- and the result NAMES which, because only `ready` says a command finished. `quiet` never
pre-empts a `wait_for`: a caller that said what it was waiting for did not ask about silence.
- **A line over 4000 bytes is REFUSED, not sent** (measured: a pty in canonical mode takes 4095 bytes on
one line and silently drops the rest, so the shell runs a truncated command and reports success). A
command a human approved must not be edited by a terminal driver on its way in.
- **Opening is approved as a SESSION, not as its first line** (SS29, SS32). `shell_interactive` carries
`shell._interactive_approval_check`, not shell's own, and the profile that routes it is STATED
(`interactive_profile`: runs_code, writes, measured) rather than read off text. Reading the first line
would mean opening with `pwd` was auto-approved as a read and handed back a shell that can run
anything. `_SHELL_MODE_GATES` holds each exempt tool to ITS gate by identity.
- **Two questions, one list** (SS25): the 4-live cap counts an interactive session, and the prompt block
does not. `_live_locked` takes the kinds, and `wait_for_wake` passes the blocking ones -- load bearing,
since that loop keeps a turn alive and an idle shell would hold it open exactly as a hung read would.
- **Its end is HELD, never a wake** (SS31), whoever ended it.
And slice 4 first, because it is the one with a rule the others do not have -- see below for
slice 3.
Slice 5b (batch 104) -- a command the harness runs as root (SS65-SS72):
* **A root command ALWAYS asks**, above both opt-ins, in every mode but `never` (SS65). This is a
CORRECTION to the gate and not a restatement of it: measured, `sudo apt-get install` on
`backend: "wsl"` answered `asks=False` whenever `AUTO_APPROVE_SANDBOX_FALLBACK` was on, because WSL
is UNCONTAINED and that opt-in sits above the capability rule. The backend where root is most
reachable was the one where nothing asked.
* **The step fires on the DECLARATION or on the command TEXT** (SS67). `needs_root` is
`declared_network`'s sibling -- literal `True` only -- and it is OR'd with a raw token check for
`sudo`, `doas`, `pkexec`, `su`. Without the OR a model steps around the step by typing `sudo` and
leaving the flag off, which is exactly the call the measurement found running unasked. It is a
token check and never a parser (G2); `echo sudo` asks, and that is the trade.
* **The token half never ARMS anything.** A password is supplied for a DECLARED call and for nothing
else, so a spelling that slips past the check reaches a `sudo` with no password and fails. That
asymmetry is also why the no-secret REFUSAL reads `declared_root` and not the token check --
refusing an undeclared `sudo` for want of a stored password would break a passwordless setup that
works today.
* **`-k` IS NOT WHAT KEEPS THE PASSWORD AWAY FROM THE COMMAND.** SS66 said it was and SS66 was wrong.
`sudo -S` reads stdin only when it must AUTHENTICATE; under a NOPASSWD rule it reads nothing and the
password becomes the command's own input. Measured on sudo-rs 0.2.13 AND classic Sudo 1.9.17p2,
**with `-k` and without it**. The control is SS72's `exec 0</dev/null; ` in front of the command.
`-k` stays for what it does do: one approval buys exactly one authentication.
* **Sudo wraps the SHELL, not the first word** (SS69), so the whole command line is root -- otherwise
`sudo echo x > /etc/f` writes as the user and the approval notice is a lie. Argv mode is dropped on
this path and nothing is lost: argv mode exists for AUTO-APPROVED inert commands, and a root command
is never auto-approved.
* **No stored password is a REFUSAL, not a prompt** (SS68), naming `wsl.<distro>.sudo` or
`ssh.<host>.sudo` and the `/secrets set` line. The store's contents ARE the opt-in -- there is no
`allow_root_commands` key, because a second switch over one decision is the ratchet G3 removed.
`sudo_refusal()` is the ONE copy, read by the gate, by `run_sandboxed` and by `start_sandboxed`.
* **A WSL one-shot used to inherit this process's stdin** (SS70) -- measured, a command in the distro
read a string written to the harness's own. That is §29 N1's second reader on a route written before
the rule. It is DEVNULL now, or a PIPE carrying one line. The container, inert and host-fallback
one-shots have the same gap and are recorded, not changed.
* **With `needs_root` false, both argvs are byte-identical to `9048427`'s**, WSL and SSH, proved
against the pre-patch module rather than against a literal typed twice.
Slice 5a (batch 103) -- the harness-held secret store (SS56-SS64):
* **`security/secrets.py` is the ONE place a plaintext lives.** scrypt -> AES-256-GCM at
`~/.config/venastine/secrets.json`, opened by a master passphrase the user types once a session.
`AGENT_SECRETS_FILE` is the test seam; the path resolves at CALL time, so redirecting HOME works.
* **`Secret` has no `__str__`** (SS62). `str`, `repr` and `__format__` are all `[REDACTED]`, it is
unhashable, unpicklable, uncopyable and not JSON-serialisable, and `reveal()` is the only way to
the bytes -- so `grep -rn 'reveal()'` IS the audit. Cheapest control in the batch, largest class
of leak prevented, because every other control assumes the value stays where it was put.
* **`maxmem` must be passed to `hashlib.scrypt`.** Measured: the DEFAULT limit refuses N=2**15 with
"memory limit exceeded". Without the argument this module raises on the first unlock on every
machine, in a message that reads as a bad parameter rather than a library default -- which is how
it gets "fixed" by lowering N. The floor in `_scrypt_maxmem` matters too: for a tiny `n` the
derived figure is BELOW OpenSSL's working set and produces the identical error.
* **Everything that is not "still working normally" locks** (SS60), through one `lock()`: the
command, quit, `atexit`, an exception escaping `guarded()` (BaseException, so Ctrl+C counts), any
askpass refusal, and the store file changing on disk. A crash is a lock because the plaintext is
nowhere else -- which is only true because four tests say it is in no file, no environment, no
argv and no log. **A refused AUTHENTICATION does NOT lock**: that would cost the master passphrase
over a typo. It quarantines that host's credentials instead (SS63).
* **Value redaction is UNCONDITIONAL** (SS61), above `redaction_enabled()` in `redact_output_text`
and first inside `redact_diagnostic_text`, which is the one composition all five diagnostic sinks
call since batch 144 -- the log formatter, the TUI's log handler and its tool-error line,
`mark_turn_failed`, and the orchestrator's pipeline event. `redact_tool_outputs` governs guesses
about other people's credentials; this is the harness leaking its own, and no switch asks for that.
The values never leave `security/secrets.py` -- the redactor calls IT rather than asking for a list.
* **The order is live values, then vendor tokens, then shapes, in that order and one place**
(S07-05, batch 144). `redact_live_values` matches by exact substring, so a pattern pass that has
already rewritten part of a stored value leaves the remainder standing: measured, a stored value
containing a `ghp_` token came back as `PREFIX-[REDACTED]-SUFFIX` from the log's old order and
`[REDACTED]` from the other. Two sites composed the same two passes in opposite orders and nothing
could say which was right. Same rule as `fetch_url` redacting before it cuts, with a substitution
as the destroyer instead of a slice.
* **A new §23 AC1 request kind needs a branch in BOTH shells** (SS59), and the parity test in `test_cli.py`
enforces it. The TUI got this project's first masked modal; the CLI got `readline_masked` on the
ONE stdin reader -- `getpass` is refused because it reads stdin itself, which is the second reader
audit #100 exists to prevent. `SAFE_DEFAULTS[SECRET]` is None, not `""`: an empty password is an
authentication ATTEMPT, and a dismissed modal must not make one.
* **The askpass listener is NOT one-shot** (SS64), and that correction came from measuring: a wrong
password produces TWO asks despite `NumberOfPasswordPrompts=1`, because keyboard-interactive and
password are separate methods with separate budgets. It bounds serves instead.
* **The prompt strings are measured, and both carry a trap.** OpenSSH TRUNCATES
`Enter passphrase for key '<path>': ` mid-path, so matching the identity file's path fails for
most paths -- match the prefix. And the password prompt has two forms, `user@host's password: `
and `(user@host) Password: `, of which a server offering both serves the SECOND.
* **With nothing stored, the SSH argv is slice 4's, option for option and in order**, pinned
against the literal `84c886b` shipped. `BatchMode=yes` comes off only when this process actually
holds a credential for that host.
And slice 4 (batch 102) -- SSH, a command on another machine (SS46-SS55):
* `backend: "ssh"` on `shell`, `shell_background` and `shell_monitor` runs the command on the one
host configured in `config.yaml`, behind `allow_ssh_backend` (frozen posture, ships false).
* **NOTHING on this route is auto-approved (SS48)** -- the one thing to remember about it. Every
other backend runs a read-only in-workspace command unasked; this one cannot, because `_within`
resolves against the LOCAL filesystem and would be answering about the wrong machine. SS45 is
what that costs: not a loud failure, a confident approval. It is ONE explicit step in
`_shell_approval_check`, above both opt-ins, because `auto_approve_fallback` and `contained`
each answer before the capability rule does.
* The host key is PINNED (`ssh_host_key`, the `ssh-keyscan` line) and a changed key is refused.
No trust prompt: a first-use question cannot be answered in a headless run.
* Auth is a key file or a running `ssh-agent`, and **this harness never holds a secret** (SS47).
`BatchMode=yes` is load-bearing -- measured, without it an encrypted key with no agent hangs
forever waiting to prompt on a terminal this process does not own.
* There is NO argv mode (SS51): `ssh` joins its trailing argv with spaces and the remote LOGIN
SHELL re-parses, so `shlex.quote` owns the one boundary and `-F none` stops the local `ssh`
reading a user config this harness cannot even redirect.
* **A kill does not reach the remote command** (SS53), measured. The in-guest `timeout -k` is the
bound. `-tt` would fix it and merge stderr into stdout and CRLF every line, so it is not used.
* `uncontained_refusal` (renamed from `wsl_refusal` in batch 101) is the one funnel for the gate,
`run` and the session start handler, and now answers for both uncontained backends.
And slice 3 (batch 101) -- WSL, a backend with no isolation at all (SS35-SS45):
- **`_within` answers False for a link THIS platform cannot follow** (SS45), and this one is not
about WSL being a backend -- it was already open. A symlink the distro writes into the workspace
is an LX reparse point Windows does not understand: `os.path.exists` False, `os.path.islink`
False, and **`os.path.realpath` returns the path UNCHANGED** rather than following it or raising.
So `_within` vouched for `escape`, `cat escape` classified INERT and was auto-approved under the
shipped `tiered`, and in the distro it read the host's real `/etc/passwd` (measured). The rule is
the platform's own admission -- a component that `lexists` and does not `exist` is one whose
lstat succeeded and whose stat did not -- and it is deliberately **not** "is a reparse point",
because a OneDrive placeholder is one and its path means what it says. It applies on **every**
route: `_within` answers one question, and a predicate that answered it per backend would be the
drift it exists to prevent.
- **The user enables the backend, the agent picks it per call** (SS35). `allow_wsl_backend` is a
frozen-posture key shipped `false`; `backend: "container" | "wsl"` is a field on `shell`,
`shell_background` and `shell_monitor`. **`shell_interactive` deliberately has no such field** --
its stream shape in WSL is unmeasured, and batch 100 is the evidence that measuring it is a batch
rather than a step in one. `security.sandbox.declared_backend` is THE coercion, `declared_network`'s
sibling and for its reason: only the exact string `"wsl"` selects WSL, and the `Literal` on the
param model refuses anything else at run time, so the gate and the runner never disagree.
- **The WSL branch is FIRST in `_route` and in `containment_for`, above `HOST_READ`** (SS38). `cat
/etc/passwd` asked for on WSL classifies HOST_READ, and the old ladder would have run it through
`_run_inert` on **Windows** against `C:\etc\passwd` -- with whichever `cat` is first on the
user's PATH, which on this machine is a GnuWin32 one, so it would have run rather than failed. The
tier keeps its whole meaning; what it must not do is pick a machine the call did not ask for.
- **A call that asked for WSL and cannot have it is REFUSED** (SS37), never served by the container
or the host, and `_unavailable_message` answers it about WSL alone -- a model told to install
Docker when it asked for WSL retries the same call.
- **A protected segment is REFUSED on this route, not asked about** (SS40), and the gate returns
"do not ask" for it beside the `UNAVAILABLE` branch, for that branch's reason: a call that will be
refused must not spend a human decision first. In the container `.venastine/` is mounted `:ro`, so
a write fails with EROFS and a read is documented risk; here a write SUCCEEDS and D17's hash
notices at the next launch.
- **The classifier learns NOTHING about WSL path forms** (SS39), measured rather than assumed: over a
generated corpus resolved both ways -- `_within` on Windows against `realpath -m` in the distro --
no token reads as inside the workspace on Windows and outside it in Linux. Every disagreement is
the safe way round and costs a prompt. **It rests on INERT running as a bare argv** (SS43): under a
shell, `~` and `$HOME` would expand after classification, and `_SHELL_METACHARACTERS` rejecting
both is the second lock rather than the first.
- **`env=_scrubbed_env()` is load-bearing here, not hygiene** (SS42). Windows variables do not cross
into the distro by themselves, but the ones named in `WSLENV` do and `WSLENV` is itself inherited
-- measured, the unscrubbed run printed a probe secret inside the distro and the scrubbed one
printed nothing.
- **The workspace goes through `wslpath`, never through `wsl.exe --cd`'s own translation** (SS44).
`--cd` with an untranslatable Windows path does not fail: it exits 0, warns on stderr, and runs in
the user's Linux home -- reporting success from the wrong directory. `wslpath` fails with an exit
code instead. **Do not "simplify" this back to passing the Windows path.**
- **No resource limits, no process group, and both are facts rather than omissions** (SS44). The
limits belong to the whole WSL VM and a `ulimit` inside the shell is removable by the command it
bounds; killing the Windows process was measured to kill the Linux children, a backgrounded
grandchild included, so `WslSessionProcess` adds nothing to the base class and a test asserts the
absence.
- **An enabled backend is on the badge** (SS41), through `unsafe_reasons()` like everything else
(UN3), and reported when it is ENABLED rather than when it is used -- a badge describes what the
process may do, and a user who reads it afterwards has read it too late.
And slice 1's foundations (batch 94) -- the pieces under the session tools, not yet reachable from any tool:
- **A wake is a harness-written USER row, marked by `MessageLog.harness`** (SS5). A column, never a role
(batch 91's rule): nothing that builds a request reads it, replay shows its first line as a `wake`
entry, and the compactor labels it `harness:`. **It must never carry a `tool_call_id`**:
`pinned_through` reads any kept row's id as an answered call, so a wake row carrying its session's
start call would hide the real result -- M4's HTTP 400. Consecutive user messages were already normal
on every compacted thread (M8's summary followed by the tail's first turn), so no merge was added.
- **`re2` is imported by `core/line_pattern.py` and nothing else** (SS8, AST-pinned). `re` holds the GIL
for a whole backtracking match, and every thread froze (measured). That module refuses a `re2` from any
distribution but google-re2 and keeps RE2's compile errors off stderr, where they paint over Textual.
- **`_route` is both `run_sandboxed`'s ladder and `start_sandboxed`'s**, held against `containment_for`
over every tier, runtime answer and fallback posture (EP6). **`_docker_argv` is the one container
argv**: a session adds only `--sig-proxy=false` and an in-container `timeout -k`, and every container
carries `venastine.process=<PROCESS_TOKEN>` so exit cleanup cannot touch another instance's. A session
process gets `stdin=DEVNULL` (§29 N1 -- a second stdin reader), merged stderr, and its own signal
group, so a console Ctrl+C reaches it only through `kill()`.
- **The manager lives in `core/shell_sessions.py`** (D12) and refuses a start outside `consuming()`
(SS17): under `never` a research pass is offered shell tools, and would otherwise start a session no
shell ever reports. **The wake text is built in `core/session_wake.py` through the real
`check_output_policy`**, because that text never passes through dispatch — **and
before every cut, which is the other half** (batch 148, A23): a session's output is
cut by the producer at `max_read_chars`, by the wake's per-session division, by the
view's display budget and by `shell_output`'s paging, and the redactor ran after
each of them. A pattern only matches what it can see whole. The producer now
redacts what it STORES, which covers the wake, the rebuilt view and
`_kill_report` at once, and `OutputBuffer.page` takes an optional `clean` so the
model-facing routes page over redacted text while the monitor's own search still
matches on raw output. Measured before the fix, at every interior cut position:
the three credential shapes leak at each tail cut, and the PAGING route leaks all
seven — including the fixed-length vendor tokens — because the model asks for
the adjacent page itself and holds both halves.
- **What a session's VIEW shows lives in `core/session_view.py`** (SS14, SS18, batch 98), for
`core/replay.py`'s reason: D12 makes the CLI a permanent fallback, so what may be shown is a policy
decision neither shell owns, and the entries are the same `(role, text)` shape a replay produces.
Both routes -- the manager's live row and the archive's rebuilt one -- go through the real
`check_output_policy` under the originating tool's name, which is the adopted default that the
session view is redacted for display. `live_entries` takes EVERYTHING the buffer
holds and applies `tail_chars()` after the redaction (A23); the rebuilt route draws
stored rows, which were redacted when they were written.
- **`RunAgentLoop.wake_conversation` is a wrapper, not a second `_run`** (`test_grants.py` walks for
exactly one), and it shares `_conversation_prompt` with `run_agent_conversation`, so a wake turn cannot
run under a prompt missing the goal, the references or the checklist. It takes the run's
`authorization` too (batch 95), mutually exclusive with `granted_tools` like its sibling: a wake turn
without the route to a human would deny every gated call inside it for a reason about the harness
rather than about the request.
And slice 1's tools, the sleeping subagent and the CLI (batch 95):
- **The five session tools are `shell` with a lifetime** (SS11, SS16). `shell_background` and
`shell_monitor` are registered with `shell._shell_approval_check` ITSELF, and
`security/permissions.py`'s `APPROVAL_BY_SHELL_MODE` is the set exempt from `tool_approvals`;
`_assert_shell_mode_exemption` holds those two facts against each other at import, so the exemption
cannot outlive the gate that justifies it. Listing, reading and stopping need no approval -- they act
only on this conversation's own sessions, each already approved as a command.
- **All five are hidden wherever nothing would wake the run** (SS17), through `available_check`, and a
start is refused there as well. Under `shell_approval_mode: never` the gate answers "no approval", so
a research pass would otherwise be offered a tool whose sessions nobody could ever report.
- **The owning thread is the injected `memory`'s, never a parameter.** A model that could name a thread
could reach another conversation's sessions.
- **A kill the AGENT makes is its own report and wakes nobody.** `SessionManager.kill` returns what the
wake would have said -- final state, the end of the output, matches nobody had taken yet -- because a
turn spent telling the agent about a session it just stopped is a model call nobody needed. The claim
belongs to the call that SET the reason, and is given back if the session outlives the wait, so a slow
death is reported the ordinary way rather than dropped.
- **A subagent SLEEPS INSIDE ITS SPAWN** (SS9). `agents/subagent_tool.py` marks the child as a consumer,
runs it, then stays in `_sleep_for_sessions` until nothing is live: each batch is a wake turn in the
child's own thread under the SAME prompt, context, channel and grant. D6 is untouched -- one distilled
answer still crosses to the parent. Past the wake limit the remaining sessions are killed and one final
wake carries them (SS20), because a subagent has no user whose message could reset the count. A child
that RAISES has its sessions killed by SPAN id: a span is frozen and carries no thread, so a run that
failed before returning one is identifiable anyway.
- **The CLI waits between turns** (`main._await_sessions`) and runs a wake turn per batch. Ctrl+C there is
aimed at the SESSIONS, not at the conversation: they are stopped and the prompt comes back, with their
results held for the next message (SS19). Held results are written into the thread before the turn that
answers that message, so the model reads them beside it. Quitting with sessions live confirms (SS6),
and nobody to ask counts as yes. `main._close_sessions` kills, records a row in each owning thread and
sweeps this process's containers -- before `teardown_mcp`, because that row needs the database.
And the four from §28 itself:
- **`SHELL_APPROVAL_MODE` is the gate, and the only one** (G3, ratchet removed). `tool_approvals` deliberately has no `shell` key: the field used to be the ratchet (`True` forced `always`), and every outcome it could produce is an outcome the mode already names -- so the second switch could only agree with the gate or surprise someone who set the two differently. The mode check reads the frozen posture; an agent's `approval_overrides` keeps its one-way tightening power through the context layer, so D14's ratchet is intact where it was never redundant. **Do not re-add the field.**
- **The argument rule must never parse** (G2). Every token after the first is read as a
path; a flag passes only because it is relative. `_is_inert` is sound *because* it
rejects metacharacters rather than understanding them, and a classifier that learns
shell syntax is a shell parser whose bugs auto-approve. False positives are the
designed error direction. **This is also why `&&` is not a hole** — the chained
command is rejected wholesale, so the first-word allowlist match never decides
anything. Splitting on separators to classify each segment would *undo* that.
- **`_SHELL_METACHARACTERS` is closed under shlex, and that is the whole point** (Q1,
Q5). The class exists because this module tokenises twice — the classifier reads raw
`.split()` tokens, `_run_inert` execs `shlex.split()` tokens — and shlex consumes
exactly whitespace, `'`, `"` and `\`. Whitespace splits both the same way; the other
three are all rejected, so the raw tokens ARE the executed tokens. #157 arrived
through this gap twice: quotes in §37, the backslash in §39, both auto-approving a
host read of an arbitrary file. **Removing a character from that class re-opens it**,
and `TestTheTwoTokenisersCannotDisagree` is what says so.
- **The security posture is bound ONCE, at import, and frozen** (UN1). `security/posture.py`
reads `config` and the environment exactly once; `config.yaml` is where a human writes
the value and is no longer what is read when a decision is made. Before §40 every
flag was read live, and `tools/builtin/shell.py` said so -- "a test or a runtime edit
can change it after this line has run" -- which describes a test seam and an attack
in one sentence. **Do not reintroduce a call-time read**, and do not call
`override_for_tests` from anything that ships.
- **No module may read the posture at IMPORT time** (UN2). `apply_cli()` runs inside
`main()`, after every import, so a module-level `posture.current()` binds it before
the command line exists. `shell.py`'s import-time mode validation reads `config`
directly for that reason. **Pinned by the source scan in `test_posture.py`, and by
nothing else** -- this bullet used to say "every launch dies in the ordering
guard", and S02-09 measured that it does not: `main.py` calls `apply_cli()` with no
overrides, so the guard has no change to detect and cannot fire on a shipped
launch. It goes live the day the first posture flag exists. The scan covers import
time only, so a posture read placed inside `main()` ABOVE the `apply_cli()` call is
caught by neither -- keep that call first among readers.
- **`unsafe_reasons()` is the ONLY description of the posture** (UN3). Banner, launch
WARNING and TUI badge all read it. A fourth surface that re-derives the question is
the bug.
- - **`_within` fails closed** (Q3). `realpath` raises on a malformed UNC path, and this
runs inside the approval gate on model-supplied input with no handler above it.
Unresolvable means "outside", which means "ask".
- **`profile.measured` gates every containment branch** (G5). A profile that could not be
characterised answers `False` to every capability question, so without it the
CONTAINED branch approves an unmeasurable command *for having no network*. It is also
where batch 9's "the model is not sure" has to land.
- **`classify_command` must stay total.** `registry.approval_needed` is handed the
model's tool input verbatim; `ShellParams` does not validate until `run()`, which is
after approval. A non-string `command` used to be an `AttributeError` escaping the
approval check and then the loop. **Totality is deeper than the isinstance guard**
(Q3): `ls \\;` is a perfectly good string whose token made `realpath` raise, so it
walked past the guard written for exactly this and crashed the gate anyway.
`shell_approval_mode` is rejected in `settings.json` **by name** (G7), the way R12
rejects `research.granted_tools`: a project's settings beat the user's, and this key
decides whether shell commands are asked about at all.
### Thread summaries and cross-thread references (`§21c`)
`compaction.summarize_thread()` distils a whole thread and stores the result;
`/summary` shows one, `/ref` attaches another thread's to the current conversation as a
prompt tier. Built almost entirely out of §21a's parts — the compactor agent,
`_summarize`'s length-checked retry, `_as_text`, `advances()` — so the load-bearing
decisions are about the seams.
- **A `ThreadSummary` is NOT a `CompactionCheckpoint`, and that is why it has its own
table** (M18). They carry nearly the same columns and mean opposite things: a
checkpoint *changes what the model sees* (`latest_checkpoint` resolves by timestamp and
feeds `_derived_view`), while a summary describes a thread and changes nothing about it.
A summary row in the checkpoint table would silently compact the thread it was only
meant to describe, and would win, being newest. Separate tables make that impossible
rather than warned about.
- **`save_thread_summary` has no advance guard, and `save_checkpoint`'s is not cargo.** A
backwards compaction watermark un-compacts a live thread; a redundant summary row can
only describe a thread as it was a moment ago, which the next `advances()` check
notices. Do not copy the guard, and do not route a summary through `save_checkpoint`.
- **The summary target is ABSOLUTE (`config.SUMMARY_TARGET_CHARS`), not a strength
ratio.** A ratio is right for a fold, where the summary replaces the span it came from
and scales with it. §21c's consumer is a prompt tier present on *every* turn of the
referencing thread, so a 500 KB source at strength 3 would inject 75 KB into every call
indefinitely.
- **A thread already shorter than the target is stored verbatim, with no model call.** Its
own text is its best summary. Same instinct as `compact()`'s no-progress-no-call guard.
- **Summaries read the ARCHIVE** (T3's reasoning, one section on). A view-based summary of
a compacted thread describes the *summary* of the conversation, degrading on every
compaction — the chaining loss M2 arranges the fold to avoid, arriving through another
door. The test for this needs a REAL compaction: the two spaces coincide until a
checkpoint exists.
- **Staleness is detected, not assumed.** The stored watermark plus `advances()` means an
unchanged thread costs nothing however many times it is referenced, and a grown one is
re-summarized rather than served stale.
- **`with_refs` goes OUTSIDE `with_catalogs()`** (M19) — §19's K6, **third** instance
after §21b's M13. A conversation referenced in a chat session would otherwise land in
all ten research passes.
- **Refs are appended UNCONDITIONALLY, unlike memories.** Copying the
`if system_prompt is None` guard from the line above would make an active agent lose
every attached reference: a ref is thread state, and `system_prompt_for()` has no memory
object to read one from. It has its own test because the edit looks right.
- **The cap REFUSES rather than dropping the oldest** (`MAX_INJECTED_REFS`). A reference is
something a person chose by name; making room silently is what M15's "removal is by id,
never by substring" rule exists to prevent. The fragment states the cap when it
truncates (M14) and says outright that the summaries are **not** this conversation's
history — the risk `SUMMARY_PREFIX` addresses for a compaction summary.
- **`/summary` describes; `/compact` folds** (M20). Two different requests that happen to
share a summariser.
- **The CLI gets flags, not slash commands** (M21): `--summary [thread]`, `--ref <thread>`
(repeatable). The CLI has no slash-command layer and adding one is §23's business. Both
go through the same `attach_ref` / `summarize_thread` the TUI uses, so the shells cannot
disagree about the cap or the stored shape.
### Live research view (§22)
The research pipeline reports as it runs. `tui/app.py` forwards each `PipelineEvent`
through `PipelineEventMessage` to `ResearchProgress` (sidebar: passes as they start,
tier tally keyed by claim, retry rounds) and the transcript; `main.run_research`
prints pass lines and trace lines as they arrive.
- **Neither shell may re-print `run.trace` at the end.** Every line already arrived as
a `trace_line` event. The CLI's `--- Trace ---` block and the TUI's
`on_research_finished` loop were both removed for this; re-adding either prints the
whole run twice.
- **The panel keys tiers by claim id, not by counting events.** 6c re-tiers the same
claim once per retry round, so counting would report more claims than the run has
and the number would keep climbing.
- **A widget method named `_render` shadows Textual's `Widget._render()`** and makes
the whole app fail to lay out with `'NoneType' object has no attribute 'get_height'`.
`ResearchProgress._redraw` is named that way on purpose.
- **Repointing the shells broke twelve test doubles, silently.** A patch on
the synchronous wrapper no longer intercepts anything, so the REAL pipeline
runs. `test_e2e.py`, `test_review.py` and `test_tui.py` now patch
`stream_deep_research_pipeline` with generator fakes routed through one helper each.
Same shape for `_run_pass` / `_run_pass_with_json_retry` / `_review_stage`: a
plain-function double is not an error under `yield from`, it iterates the string one
character at a time and returns `None`.
### Research legibility (§26)
A run is now readable while it happens. Five gaps, all found by watching a real ten-pass
run rather than by reading the code.
- **`stream_deep_research_mode()` is the third generator/drainer pair.** A callback
parameter was rejected: §23 AC1 exists to stop a third bespoke channel. Its
synchronous drainer `run_deep_research_mode()` is retired — see the §22 bullet
above; `_run_pass` iterates the generator, and the only remaining callers were tests.
- **The response is RETURNED, never read off the terminal event.** `thread_id` is
attached after the loop finishes, so the caller must drain with something that keeps
a generator's return value and `_translate` uses `next()` rather than `for` — a `for` loop
swallows a generator's return value, and the only symptom would be §3's JSON retry
opening a new thread instead of correcting the failed one.
- **One translation site** (`_translate`), consumed by both pass entry points. Two sites
for one translation is how they drift; this file has the twenty-checkpoint version of
that mistake in its own history.
- **The param digest is redacted BEFORE truncation.** Tool arguments were the one
unscanned path — `check_input_policy()` refuses a call rather than redacting one, and
`check_output_policy` only ever saw results. Truncating first cuts a credential below
the 20 characters its pattern needs. **Three surfaces now hold this rule** — this one,
`tui/app.py`'s diff block (whole texts, because a secret is as readable across two
wrapped rows as on one), and `fetch_url`'s paging since §52 (RB1). The third was
written in the wrong order with the first two already in the record, so treat a new
place that CUTS model-facing text as owing this check rather than inheriting it.
- **Streamed pass text never escapes, only its volume.** Seven of the ten passes emit raw
JSON; `pass_activity` carries a throttled running character total instead.
- **There is no step event, deliberately.** `_run()` has no step marker to read, so a
consumer counts `tool_call` events itself. Inferring one from the start/result
interleaving would be a guess presented as a fact.
- **`D2` emits once per retry ROUND**, outside the per-claim loop — inside it, a round
exhausting six claims reads as six separate stages.
- **Colour resolves from the `Theme` object** (`themes.role_styles`), because a `RichLog`
renders Rich `Text` and cannot use `app.tcss`'s variables. Severity uses
`warning`/`error`/`success` and identity uses the per-variant hues. Across the eight
grid themes the severity trio is shared; a standalone tinted theme (batch 29) may
override a severity slot when its own panel tint would swallow the shared value,
keeping the semantic hue family — each override carries its reason in `themes.py`,
and the contrast floors (foreground ≥ 7:1, severity ≥ 4:1, identity ≥ 3.5 dark / 3.0
light) are pinned in `tests/test_themes.py`, over **our fourteen** — thirteen of
textual's twenty-one fail one of them and the two ANSI ones cannot be measured at
all, so integrity is checked for all thirty-five and quality for ours. A widget with
no running app renders
**unstyled rather than raising** — `self.app` raises `NoActiveAppError`, and widgets
are built bare throughout the suite.
- **A `Theme` slot can be `None`, and `role_styles` must go through `_palette`**
(batch 64). Every slot but `primary` is `Optional[str]` on textual's own `Theme`, and
the command palette offers **thirty-five** themes, not fourteen — textual registers
its twenty-one in `App.__init__`. Four of them break a naive read: `textual-dark` has
no `background`, `textual-light` no `foreground`, and `ansi-dark` / `ansi-light` fill
every slot with `ansi_*` names, which are **textual's vocabulary and not Rich's**.
(There was ONE such ANSI theme, `textual-ansi`, until 8.2.5 split it in two —
`SELECTABLE_THEMES` being DERIVED is what carried this across the pin move, and what
put the nine new built-ins under test on the run that noticed.) Selecting the first
killed the harness *at mount*, and kept killing it, because `watch_theme` had already
written the name to a user-tier store the installer cannot reach. A blank slot falls
back to `theme.to_color_system().generate()` — textual's own derivation, so the colour
agrees with what `app.tcss` paints — and **only** where a slot is blank: that
derivation returns `color.lighten(0).hex`, an HSL round trip that moves `#d9a441` to
`#D8A441`, and it is 48× slower than `role_styles` itself on a function called once per
drawn line. `_tint` answers `None` where there is no RGB to blend toward, and the diff
row takes the severity colour as its foreground instead — git's convention in a
sixteen-colour terminal.
- **A bad style string is SILENT, which is why the test parses them** (batch 64). Rich's
`Text.render` resolves a style through `console.get_style(style, default=Style.null())`,
so an unparseable one renders plain and raises nothing — `textual-light` lost six roles
and the ANSI theme twenty-five without a single test noticing. A test that merely draws
a transcript cannot see this; `Style.parse` can. Same shape as batch 41's `dim`, one
layer along. `styles_for` also contains anything `role_styles` still manages to raise
(§27) and warns **once per theme**, since it runs once per line.
- **`Transcript._entries` serves the replay and `/copy`.** `RichLog` stores rendered
segments, so `/theme` needs `rerender()`. Every write path must go through `_emit()`,
or a line reaches the screen and neither the replay nor the copy.
- **Nothing the transcript draws keeps a terminal control sequence** (batch 145,
S15-01). Every ingress stored and drew model- and third-party text byte-for-byte:
`core/ansi.py` existed and was called in exactly one place, the pty stream, and
nothing in `tui/` was it. Neither layer below removes the byte — Rich keeps it
through `export_text()`, and textual's `Content` strips only codepoints 7, 8, 11,
12 and 13, so ESC and the whole C1 range survived both the `RichLog` path and the
`as_content(Text(...))` one. **THREE chokepoints, not one**, because `_emit` is the
single write path for complete strings and the two delta entry points are not on
it: `_write_stream_chunk` and `_write_thinking_chunk` append to `_entries`
themselves. **THREE stripper objects**, and each count is measured: the delta
streams need one held ACROSS calls (a fresh one per delta turns `hi ESC [` +
`2Jthere` into `hi 2Jthere`, and one that flushes per delta emits the ESC
outright), and the answer and reasoning streams need separate ones because they
interleave — with one shared, a CSI left open by the answer terminated on the
reasoning chunk's first character and ate it. The strip also CORRECTS the wrap:
`display_len` read 10 for text six cells wide, so an escape counted as cells was a
row that broke early. Nothing is drawn in its place and the widget logs nothing —
`tui/app.py` reads a count at `on_turn_finished` and records one INFO line per
turn, because `Transcript.write*` doing no logging is what stops
`on_log_record_message` closing a loop.
- **`/copy last` is READ from that log, never tracked beside it** (batch 48).
`_last_response` was the second buffer, updated from whichever flush site kept
`flush_stream()`'s return — and the one that mattered did not. `on_loop_event_message`
flushes at the terminal `final_response` event and discards, so `on_turn_finished`'s
flush found the span already closed and got `""`; the field then held a replayed
thread's newest answer, or nothing. **Two symptoms, one cause**: a resumed thread
copied the answer from before the close (in a one-exchange thread, the first message),
and a fresh one said `Nothing to copy` after a perfectly good answer. `last_answer()`
scans `_entries`; `reset()` is what clears it, which is the per-thread reset §27 AC4
and §43 RM2 already call.
- **Two role sets say what `/copy conversation` is** (batch 48). `CONVERSATION_ROLES`
(`user`, `assistant`, `thinking`, `tool`, `tool_error`, `diff`) against `META_ROLES`
(`system`, `warning`, `error`, `pass`, `pass_done`, `success`) — an ALLOWLIST, so an
unclassified role goes missing from the copy rather than leaking a harness line into
it, and `tests/test_themes.py` holds both against `MESSAGE_ROLES` so nothing stays
unclassified. `pass`/`pass_done` are meta deliberately: a ten-pass run's boundaries
are progress reporting, so a research run copies as the query, the tool lines and the
report. **`/copy all` is unchanged** — the superset, harness lines included (#140).
A failed tool with no known call id now writes `tool_error` rather than `error`, which
is what made the split sayable: one role cannot mean both a tool and the harness.
- **`/copy`'s clipboard route cannot be confirmed.** OSC 52 is fire-and-forget, so the
message says what was sent, and `--file` is the route that provably worked.
- **The claims view is `ctrl+l`.** `Input` binds `ctrl+k` to `delete_right_all` and holds
focus, so that key would silently eat the typed line. `ClaimsScreen` takes plain dicts
— the shape §5 already persists — so the finished-run, stored-run and mid-run sources
need no branch in the renderer.
### Thread legibility (§27)
Resuming a thread now shows the thread, and a thread records what it is. Two
independent bugs, both found by using the app.
- **`core/replay.py` is the only place that decides what a stored thread looks
like**, and both shells replay through it (T5, §26's L6 again). A per-shell
renderer would let the CLI and the TUI disagree about whether tool results are
shown — which is a policy question, not a painting one.
- **Replay reads the ARCHIVE, never the derived view** (T3). `memory.messages` on a
compacted thread BEGINS with the synthesized summary, so replaying it renders
harness-generated text under a `you ›` label — precisely what M8 says that message
must never be mistaken for. The same reasoning makes `list_threads`' preview read
the archive.
- **Tool calls replay as one redacted line; tool RESULTS are skipped** (T4). A
grounding-heavy thread carries hundreds of kilobytes of fetched page text in those
rows.
- **`ConversationThread.kind` is a column, not an `extra_data` key** (T1), because
`list_threads()` is the one query that must stay cheap on a database gaining ten
rows per research run. It is deliberately **not indexed**: `ensure_columns()` ALTERs
in a column and builds no index, and `create_all()` adds none to an existing table,
so `index=True` would mean an index on fresh databases and none on migrated ones.
- **The fresh thread per pass is the INVARIANT; the missing label was the bug.** Passes
share distilled JSON and never raw history. §27 hides those threads and records them
on the run (`PipelineRun.pass_threads`, T2) so hiding one does not orphan it — the
list is the run's own object, shared by reference like §25's `granted_calls`, which
is what puts it on the failure path.
- **The compactor is a FIFTH thread source, and §27's spec missed it.** Every automatic
compaction creates one, so the picker filled up fastest on exactly the long threads
§21a exists to serve. It is labelled `subagent` with `spawn_subagent` and §20's
reviewer; the reviewer gets no kind of its own, since §20 already calls it a subagent.
- **Classification is separate from `ensure_columns()`, and idempotent by construction**
(§27 AC3). M7's migration is additive and never backfills, so the column arrives with
every existing row defaulted to `chat`.
`pipeline_storage.classify_legacy_pass_threads()` runs at every launch over a DBAPI
connection (the `ensure_columns` seam) and works in TWO TIERS (#82): every thread id
named in a populated `pass_threads_json` is relabelled outright — the run's own
record, authoritative over any clock comparison and bound-free, which is what makes
a chat thread another session creates during a modern run safe; only runs with an
absent/NULL/empty/corrupted column fall back to the time window, a pre-T2 set that
cannot grow. The window tier keeps the **finished-run** guard: a run with
`finished_at IS NULL` (§22's abandoned generator) contributes no window, because an
open one would swallow real conversations — under-classifying leaves a straggler in
the picker, over-classifying hides a real conversation, and only one of those is
recoverable by the user. Corrupted JSON falls back for that one run AND warns naming
it. Under the old correlated subquery, dropping the `IS NOT NULL` guard was invisible
to the whole suite (`x <= NULL` was already not true); as ordinary Python it now has
teeth, and both tiers are pinned.
**Tier 1 compares UUIDs, not strings** (A50, batch 154), and that is locked. The two
sides are spelt differently — `pass_threads_json` carries `str(thread_id)` (dashed,
36 characters, also what `output_writer.py` ships and what an operator reads) and
`conversationthread.id` comes out of the raw DBAPI as the 32 hex characters sqlmodel
stores — so a literal `in` was False for every id there has ever been and the
authoritative tier had never relabelled one row. `_as_uuid_key` renders every RECORDED
id as hex and answers `None` for a value that is not a UUID, never the value itself.
**One side, and the other is pinned, not branched on** — the symmetric version was
written first and the controls found it unearned, since the column already holds hex;
`test_a_thread_id_is_stored_in_a_spelling_str_does_not_produce` asserts that against a
real database and fails naming the function if a library upgrade ever changes it.
A recorded id it cannot read costs that entry's membership and **must not** drop
its run to the window tier: a populated `pass_threads_json` is what "post-T2" means,
and consulting a modern run's clock is how the concurrent chat thread gets hidden.
Both halves have their own test in `TestTwoTierClassification`, whose ids are real
UUIDs spelt the two ways on purpose — they used to be one hand-written string on
both sides, which is why four tests passed against a dead tier.
- **`param_digest` lives in `safety/policy_enforcement.py`**, beside `redact_secrets`,
since §27's replay renderer is its second caller. It redacts BEFORE truncating, and
a second copy is how that ordering regresses in one place and not the other.
- **`/new` redraws, through the same reset** (§43, RM2). `switch_to_thread` cleared the
transcript and the per-thread state; `_cmd_new` did neither, so a new thread inherited
the previous conversation's screen, its `/copy all` and its `/claims` — §27's bug 1 and
§27's AC4, surviving on the path that section did not touch. The `_busy` refusal stays first, so a
refused `/new` clears nothing, and the banner is REPRINTED (`_write_session_banner`)
rather than replayed: a `/model` switch survives `/new`, so the launch pair would name a
model the next turn will not call.
- **Resuming resets per-thread state.** `_last_run`, `_live_claims` and
`_last_response` survived a thread switch, so `/claims` after a resume showed the
previous thread's run — the same class of bug the goal banner already fixed in that
callback. `Transcript.reset()` (not `rerender()`'s `clear()`) drops `_entries` too,
or `/copy all` keeps handing back a thread the session has left.
- **The transcript is HELD, not queried, and that is what makes a modal survivable**
(batch 66, amending #104). `app.query_one` searches the ACTIVE screen, so any pushed
modal shadowed `#transcript` and `_transcript` raised `NoMatches`. The rule used to be
"catch it at each site", and it was applied at four of them — but the callers that
CANNOT catch it are the ones whose occasion IS an open modal: `on_log_record_message`
for any routed WARNING+, the startup workers' report handlers, and `_timed_out_ask`.
An exception out of a Textual message handler takes the app down, so a diagnostic
could kill the session it was diagnosing — the failure `TranscriptLogHandler.emit`
already refuses one hop earlier. It reached CI as an unrelated permission test dying
on `NoMatches`. `on_mount` now holds the widget; nothing ever remounts it, and the
query stays as the fallback for an app that was never mounted, which is most of the
suite. **A line written under a modal now LANDS** rather than being dropped, which for
a routed warning is the whole point of a handler that exists because the TUI detached
stderr. In a pilot test, holding the widget before opening a modal still works and is
still the clearer spelling; `#thinking-indicator` and `#prompt` are still queries, so
their `NoMatches` guards are still load-bearing.
- **The held set is twelve now, and the two that forced it were on the crash
path** (batch 76; §47's thread viewer added three more: the read-only
`Transcript`, the `ThreadCrumb` and the `#pane` switcher, and ROADMAP_v3
§49's SS14 two more in batch 98: the `#session-view` pane and the
`SessionPanel`. Both follow the same pattern and both have the occasion
this bullet is about -- the panel is redrawn from a sink called on the
session manager's own threads, which can speak while a modal is up.
ROADMAP_v3 §55's TR10 added the twelfth in batch 125, the `#call-view`
pane, which has NO such occasion -- nothing polls it and nothing writes
to it from another thread -- and is held anyway, because a pane that is
queried on one route and held on the three beside it is the asymmetry a
reader has to check rather than know).
A `permission_request` event is posted BEFORE the worker
pushes the modal it announces (`post_message` only enqueues for a later
pump; the push is a direct loop callback), so the first gated call of a
session was handled with a modal already active -- and died in
`refresh_usage_line` with `NoMatches`, taking the app down over a
usage-line repaint whose own docstring claimed containment. The
`permission_request` branch touches the raven three lines later through
the same unguarded query, so `UsageLine` and `RavenPanel` moved to
held-at-mount refs together; `GoalBanner`, `TodoPanel` and
`ResearchProgress` followed (notice events and pipeline events are
postable mid-modal too -- an attended review modal with passes still
reporting). Same preconditions as the transcript: unconditionally
composed (the checklist in exactly one of its three slots), never
remounted, query kept as the never-mounted fallback.
- **Narration is best-effort; the ANSWER is not.** `_blocking_modal` returns the raw
dismissal value, and `None` on timeout is what `interaction.decode` turns into the
declining default. A failure while merely SAYING a request timed out used to escape as
an exception, reach `interaction.ask`, and be logged as "response channel raised" —
naming itself as the cause of a refusal it did not cause, and masking whatever really
broke. That is exactly how batch 66's crash presented.
### The response channel (`core/interaction.py`, §23)
**One way to ask a human anything.** There were five: `permission_channel`
(§13), `ApprovalProvider` (§25), `ReviewConsent` (§20), and §24's
`ask_confirm_blocking` / `ask_project_kind_blocking`. Each re-derived what a
dismissal carries, what a timeout means, how shutdown releases a parked
worker, and how a malformed answer decodes.
- **What is shared is `decode`, not the dataclasses** (J2). A `Request` /
`ResponseChannel` pair is nearly free to write badly — five call sites can
each build one and still each invent their own failure handling, which is
how there came to be five. One function owning the declining default per
kind is the part that pays, and §25's "the inability to ask is not
permission to proceed" becomes a property of the module rather than a habit
five places remember.
- **`decode(request, raw)` takes the REQUEST** (J3). Two kinds cannot be
validated without knowing what was asked: a `CHOICE` only against the
options offered, a `SUBAGENT_SIGNOFF` subset only against the candidates
listed. With the kind alone, a shell could return a tool nobody offered and
thereby grant it.
- **`decode(Request(kind), None) == SAFE_DEFAULTS[kind]` for every kind**
(J4), pinned by a test. That is what lets `ask()` route its no-channel and
callback-raised paths through `decode` rather than reading the table, so a
kind added later cannot carry a default its decoder disagrees with.
- **An unknown KIND raises; an unanswerable question declines** (J5). An
unanswerable question is a runtime condition to fail safe on; an undefined
one is a bug, and inventing "no" would hide the typo that built the Request.
- **Review decoding is STRICT** (J6). The two decoders it replaced disagreed —
`review.py` accepted a bare `"accept"` string, `tui/app.py` did not — and
neither behaviour was tested. Strict wins because review is the only kind
whose permissive failure *applies* an edit rather than declining one.
- **`ToolSpec.request_kind` / `request_payload`** (J7) carry the shape of the
question and its tool-specific fields, for `grant_scope`'s reason: the loop
asks the registry, so no tool name appears in `core/loop.py`.
- **The sign-off memo is keyed by (tool, SUBJECT)** (J8). `grant_scope="run"`
meant a second spawn was not asked at all — and the test pinning it spawned
agent `a` then agent `b`, so approving `a` authorised `b`'s whole gated set.
Per-tool sign-off makes the answer agent-specific, so the memo is too; the
same agent twice is still one prompt.
- **The two parameters stay two.** §23 unified the mechanism, not the
arguments: `review` stays separate from `authorization.provider` because
`--review` without `--attended` is legitimate and one object would make it
unexpressible.
- **`_obtain_approval` returns (approved, grant).** A sign-off's answer is
three-way — `None` refuses the spawn, `set()` spawns with nothing granted, a
non-empty set grants those — and `TUI`/`CLI` both offer all three. Collapsing
the middle one is the easy mistake; `interaction.decode` tests `isinstance`
rather than truthiness for exactly that reason.
- **`tui/app.py` has one `_blocking_modal`.** Four asks did the same six lines,
**Slice 2 — the two tools the channel existed for.**
- **A tool that asks uses the INJECTED `response_channel`, not the approval
bridge.** `_obtain_approval` fires only under `if needs_approval:`, always
builds a `{"tool_name", "params"}` payload, and coerces the answer to
`(bool, grant)` — a multi-select or free-text answer has nowhere to go in it.
`response_channel` is injected into any handler naming it, on both dispatch
branches, so `ask_user` calls `interaction.ask` itself and `core/loop.py` is
untouched. `_INJECTABLE_PARAMS`' comment predicted this.
- **`QUESTION`'s answer is three-way** (J10's sibling shape): `None` is nobody
answering; `defer=True` is the "chat about this" escape and a REAL answer;
anything else is options-plus-text with the options intersected against what
was offered. A blank submission is someone who was present — `isinstance`,
never truthiness.
- **Adding a kind needs a branch in BOTH shells.** A kind a shell does not
render falls through to `None` and decodes as "nobody answered" on every run
there while looking wired up. `CONFIRM` and `CHOICE` are in that state on the
CLI today, masked because `/init` supplies its own callbacks.
- **Both tools are UNGATED** (J12/J9), and for `ask_user` the deciding reason is
not "asking is harmless" — it is that §13 stops *advertising* a gated tool
where nothing can ask, so gated it would be invisible rather than deniable and
§23's AC2 -- "an error result it can work around" -- would be unreachable.
- **A tool result's `notice` is forwarded as a `LoopEvent` and STRIPPED** (J10),
generically — the alternative was the loop recognising `todo_write` by name.
This is also how P1's deferred event-shape decision was discharged with
neither a new field nor a new type: `notice` was already kind-discriminated.
- **The todo panel reads its CONTENT from thread state**, using the event only
as a trigger, so the list never exists in two places that can disagree.
- **`with_todos` is K6's FOURTH instance** (after M13 and M19), and converging
the four `with_*` tiers is now a recorded deferral (J11) rather than a
docstring aside — they differ in signature *and* in conditionality.
- **Whole-list write** (J13): no item is ever addressed by name or index, which
is the problem `clear_refs` refused to solve.
and the `_permission_channel` field each published is what
`_release_permission_channel` reaches for when the app exits with a modal
open. It is no longer the loop's channel — it is this app's parked-worker
queue.
### Project scaffolding (`project_init/`, §24)
`/init` reads a project and writes its documentation set: a root `AGENTS.md`
as the hub, plus the documents it links. `--init` on the CLI (M21's precedent —
the CLI has no command layer).
**The hub is `AGENTS.md` at the ROOT, and `.venastine/` is configuration only**
(§44 WS8, reversing I9). I9 put it inside `.venastine/` so the one file every
agent reads sat inside the trust boundary; §44 keeps the boundary and moves the
file. A project shared WITHOUT `.venastine/` — the ordinary thing to do when you
do not want to ship your configuration — used to arrive as a set of root
documents pointing at a hub that was not in the archive; and `AGENTS.md` is the
name other harnesses already look for, so an initialized project is legible to
them without a second copy of one meaning under a second filename. `CONTEXT.md`
is gone outright, not kept as a fallback. `doc_path()` is one join now.
- **`write` and `read` are globally denied and cannot be re-enabled at runtime.**
Not a default: `is_tool_allowed()` reads `config.ToolPermissions()` directly —
whose field defaults are `config.yaml`'s `tool_permissions` booleans, read once at
import — `settings.json` has no `permissions` section (and `_KNOWN_SETTINGS`
*raises* on an unknown key), and D14 forbids a `ToolContext` widening anything — the global
check runs first and unconditionally. `dispatch("write", …)` therefore raises
`ToolCallDenied` **before** the `approval_callback` is consulted. §24's spec
preferred routing through `write`; that was not available.
- **Two narrowly-scoped tools instead of flipping either global** (I1/I2).
`read_project_doc` is confined by realpath to the project and by an allowlist to
documentation and manifests; `write_project_doc` takes a document **name** from
a fixed allowlist and has **no path parameter at all**, so the destination is
derived and it cannot be aimed elsewhere. Flipping the globals would hand every
agent and all ten research passes standing file access to serve one command.
**"Cannot be aimed elsewhere" needed the write path to RESOLVE, which it did not
until batch 146**: a document name that is a link aimed it elsewhere, and the read
path in the same module had resolved before deciding since the day it was written.
A hardlink is not caught by resolving anything -- it is a second name for the file,
not a link to it -- so the write also replaces the name instead of writing through
it (`json_store.write_text_atomic`).
- **The readable set is documentation, not "text files", and that is
load-bearing.** A registered tool with permission `True` is advertised to EVERY
run — the initializer's `allowed_tools` narrows *that agent*, not plain chat or
a research pass. `.venastine/`, `.env*` and `providers.json` are denied outright,
and source files are out of scope entirely — widening to those turns it into
`read` with no gate. But the set is not small: the tool takes a PATH, resolves
anywhere under the project, and admits `.md`/`.markdown`/`.rst`/`.txt` plus ten
named build files, which on a clean checkout of this repository is **62 files
and under 5 MB at depths 0-3**, including every agent and skill body under
`agents/builtin/` and `skills/builtin/` and the whole of `docs/`. The 61st is
`docs/REVIEW_MARATHON.md` (batch 129), and the per-scope review reports it
governs are deliberately NOT tracked, so they do not move this figure at
all -- that decision is recorded there and this is the number it was taken
against. **It said 40
files and ~1.1 MB until batch 128**, when it was re-measured through the
tool's own `is_readable_doc` at 60 files and 3.58 MB -- a 3.3x
understatement, and almost all of the growth is this record: `DEVLOG.md`
alone is 1.2 MB, and `BREAKING_CHANGES.md`, `ROADMAP_v2.md`,
`ARCHITECTURE.md` and this file are another 1.6 MB between them. Both
figures are read back out of this sentence by
`test_the_documented_doc_read_bound_is_not_understated`, which fails only in
the understating direction, because that is the direction the next sentence
says costs something. **The ceiling was 4 MB until batch 144**, when the
real figure reached 4.01 MB and this record was again almost all of it --
that batch's own DEVLOG entry is what crossed the line. Raised to 5 MB,
which is roughly thirty more batches at the measured rate (3.58 MB at batch
128, 4.01 MB at batch 144, so about 27 KB a batch), because the ceiling is
a ceiling on purpose: an equality would go red on the next commit and be
retyped without being read, and a round number going red is the moment
noticing is the correct outcome. The FILE COUNT is pinned exactly instead,
and it did not move. ARCHITECTURE.md used to say the most a prompt-injected pass
gains is the README (audit #97); understating a security bound is the one
direction that costs something.
- **The hub is the map of the documents it links** (I9, as relocated by WS8). All
of them are ordinary committed root files now, so the one file every agent reads
sits beside the ones it does not.
- **The index is generated, and is a pure function of the kind** (I11). It first
marked which documents already existed — which rewrote `CONTEXT.md` on every
later `/init` (the index changed because the previous run had created the
files) and labelled documents `/init` had authored moments earlier as
pre-existing work it had preserved. Which files a run left alone is a fact
about that run; it belongs in its report, and is there.
- **Stubs are templates; only the hub (`AGENTS.md`) is generated** (I10). A DEVLOG
written for a project with no history is confident fiction in a file that then
gets committed, linked from the hub and read as established. Stubs carry
headings, a what-belongs/what-does-not pair, and only facts the manifest
established deterministically.
- **Discovery is a directory walk** — no model call, no tool call — so two runs on
an unchanged project produce byte-identical input, which is what makes the
output diffable. The manifest carries **sizes** because choosing what to read
needs them: this repo's own root markdown is 721 KB against a 20 KB per-read cap.
- **The manifest is the TASK, not a system-prompt tier.** §19's K6 does not apply
only because of that — the same way `_summarize` passes `segment_text` as
`user_goal`. In the system prompt it would reach all ten research passes.
- **The agent produces text; the shell writes** (I8). `initializer.md` gets
`read_project_doc` and nothing else. Diffing, asking and dispatching happen in
`project_init/generator.py`, because consent belongs to the shell (§20's V3) and
a tool writing from inside the loop could not show a human a diff first.
- **One consent, covering a named list.** `write_project_doc` is gated, so the
naive version prompts eight times per command — the shape of consent that gets
clicked through. The user sees the diff and the file list once, and that answer
becomes the `approval_callback` for every dispatch, the move `core/loop.py`
already makes after a channel approval.
- **No confirm route means nothing is written** (I5) — §25's V6, sixth instance.
- **Trust is re-granted only for a project that was ALREADY trusted** (I6), and
the state is captured **before** the write, since the hash has moved afterwards —
which is still true with the hub at the root, because WS9 puts the root
`AGENTS.md` inside the hash.
In an untrusted project `.venastine/` may already hold agents, skills and an
`mcp.json` that arrived with a cloned repo, and granting as a side effect of
`/init` would wave all of it through. The root `AGENTS.md` is never
special-cased out of the hash — that is the same hole with a smaller mouth.
(It named `CONTEXT.md` until batch 153, which promised this about a file that
is not in the hash because it does not exist, and left the guarantee that
matters unstated. `test_the_file_agents_md_promises_is_inside_the_trust_hash`
reads the name out of this sentence and checks it against
`workspace_trust.content_files()`, so the promise cannot outlive the file
again.)
**Both trust calls are TOTAL** (A46, A47, batch 153), and that is locked
rather than defensive habit. `_trust_before` wraps the capture because it
READS every file the grant covers, so one file an editor holds open used
to kill `/init` before it wrote anything; an unanswerable question reads as
UNTRUSTED, which is this module's fail-closed posture and the only answer
that cannot launder a cloned `.venastine/` in. `_settle_trust` returns
`(note, settled_as_intended)` because `grant_trust` raises from both halves,
and it sat outside the handler on both paths — so a trust-store failure took
the success path's whole notice and the failure path's file list with it. A
re-grant that could not persist is **reported**, exit code unchanged, with
the warning FIRST; a settled run keeps the note last. Deleting either `try`
restores a defect, and `TestTheTrustSettleIsTotal` is where it fails.
- **`/init` runs bounded** (I7). Its separate 1M budget died in batch 27 with the
misreading that created it — the wrapper resolves the configured spend cap like
every other path. `INIT_READ_CHARS` (well below `MAX_READ_CHARS`: each read is
re-billed every following step) and `INIT_MAX_STEPS` remain, because neither
spend nor read size bounds the *number* of reads.
I7 states three config bounds and two of them are read from `config`; the third
is a FALLBACK. `generator._run_initializer` passes
`agent.max_steps or config.INIT_MAX_STEPS`, and `agents/builtin/initializer.md`
declares `max_steps: 12`, so the agent file wins and the constant is never
consulted. Both are 12, which is why nothing behaves differently and why the
claim read as true (audit #97). That `or` is the general rule that an agent may
narrow its own run — `compactor.md` relies on it too with `max_steps: 1` — so
the doc is what was wrong, not the code. Changing `config.INIT_MAX_STEPS` alone
does nothing.
- **The initializer is a SIXTH thread source.** §27 found the compactor as the
fifth; this run is labelled `thread_kind=THREAD_KIND_SUBAGENT` for the same
reason.
- **A project-tier config file is not inert because its values are defaults** (batch 50, I16).
`settings.json` merges with `merged.update(project)`, so a project key wins by PRESENCE and
not by difference — and six keys are read by presence somewhere further downstream, which
makes writing their own default a decision rather than a restatement. `_trigger_is_configured()`
is the clearest: it asks `"trigger_tokens" in ...`, so scaffolding the default disables the
window-derived trigger and the file gives no hint that it did. `/init --config` omits all six,
each with its reason in `config_files.OMITTED`, and the test asserts the scaffold is inert
over the whole observable set — **including the log**, which is how the last one turned up:
`grounding_weights` carries a `None` key, JSON has no null key, and it round-trips as the
string `"null"` that `_apply_grounding_weights` then warns about on every launch.
- **`--config` writes into `.venastine/` directly** (I14), because `write_project_doc` denies that
directory by path segment and names `mcp.json` as the reason. Do not "fix" that by widening
the allowlist: the tool is advertised to every run and all ten research passes, and `mcp.json`
names a local command to execute. The single consent is what replaces the tool's gate.
- **Detect-then-confirm, but ask outright for a blank folder** (I13). With no
manifests, no source and no docs there is nothing to infer from, so a proposal
would be a guess wearing a finding's clothes. The proposal itself is a Python
heuristic, not a model call, and the user confirms it either way.
### Config loading and workspace trust (ROADMAP_v2 §14)
`core/config_loader.py` discovers `.md` agents/skills across three tiers — harness (`<root>/{agents,skills}/builtin/`) → project (`.venastine/`, trust-gated) → user (`~/.config/venastine/`) — parses line-anchored YAML frontmatter, and merges `settings.json` (unknown keys **raise**). `core/workspace_trust.py` owns only the D17 trust store (resolved path + sorted-walk content hash); `main.py` owns the prompting UX.
**THE PROJECT DOES NOT DECIDE HOW LONG ITS OWN CONSENT SCREEN IS** (A54, locked, batch 155). Every
section of `describe_project_content()` goes through one `_bounded()` helper with a LINE bound and a
CHAR bound — the file listing, the hub, `settings.json` + `mcp.json`, and the agent/skill catalog —
and the marker names what was withheld in both currencies. **A bound on one axis is not a bound**,
which is the whole reason §44's twenty-line hub cap was replaced rather than copied: twenty lines
is no bound at all on a 2 MB `AGENTS.md` written as one line, or on twenty 100 KB lines from inside
the cap. Measured before this: the two JSON files were pasted whole, so 2 × 2.2 MB rendered as
**4.5 MB in 30 lines** with the `mcp.json` command the section exists to show at 99.9994% of the way
in; the file listing had no bound (20,000 padding files = 20,001 lines); and the catalog bounded each
description at `max_catalog_text_chars` and not the number of them (3,000 agents = 1 MB). The caps
are sized from the largest LEGITIMATE shapes, not picked — a five-server pretty-printed `mcp.json`
is 54 lines and 923 characters — because a cap that trims ordinary files turns its own marker into
noise. **`content_files()` is NOT bounded and must not be**: the hash and this module's "the loader
must read no file the trust listing omits" invariant are about that function, and what is bounded is
only what the prompt PRINTS of its answer.
**Trust is keyed to the PROJECT PATH, which is the workspace when something names it as the project -- `AGENT_WORKSPACE`, or `workspace_is_project` in `config.yaml` since batch 111 -- and `os.getcwd()` otherwise** (§44 WS7, reversing RM6; TECHNICAL_DEBT 28). RM6 recorded the opposite and the report that reopened it is what the split looked like in use: the file tools confined to the workspace while `/init` scaffolded the harness.
**What a grant covers is `content_files()`, and that is now the root `AGENTS.md` plus everything under `.venastine/`** (WS9). The hub reaching a system prompt from outside the boundary would be D17's own stated threat arriving through the door WS8 opened, so the listing carries it — with PROJECT-relative paths, because `AGENTS.md` and `.venastine/AGENTS.md` are different files. `is_trusted()` asks that listing whether there is anything to trust rather than asking whether the directory exists, so a cloned repo shipping an `AGENTS.md` and no `.venastine/` now prompts, and a project with neither is silent as before. Every grant made before §44 re-prompts exactly once, because the set being consented to genuinely changed. `_content_hash()` iterates `content_files()` instead of walking a second time, because two traversals that must agree by construction is how #18 drifted.
Load-bearing: untrusted project content is **absent**, not loaded-and-disabled. Trust-store and user-config paths resolve at call time, not import time (tests redirect them). Provider/model precedence is CLI > `settings.json` > `config.yaml`, which only works because argparse defaults are `None` (`main.resolve_runtime_defaults`). Since §43 the TUI adds one tier of its own between the first two — a remembered `/model` pair, applied only when no flag spoke and only while `default_provider`/`default_model` still say what they said when it was chosen. The CLI is unchanged and does not read that store.
**A symlinked PROJECT tier root is treated as absent (#18).** `os.walk` follows the path handed to it as `top` but not symlinked subdirectories, and the two sides of the trust boundary start from different places: `workspace_trust` walks from `.venastine`, so `skills/` is a subdirectory it will not descend, while `_md_files` walks from `.venastine/skills`, so it *is* the top and gets followed. Directory names never enter the hash either, so the link contributed nothing — not even its own name. Behind it, definitions loaded as project tier while being absent from the trust prompt's listing **and** from the hash, so their bodies could be rewritten freely after one grant with `is_trusted()` still returning `True`. The payload is instructions, not data: a project-tier body becomes system-prompt content. Git stores symlinks natively, so this arrives on clone — D17's own stated threat. The guard is scoped to the **project** tier deliberately; nothing hashes the harness or user directories, and symlinking `~/.config/venastine/` into a dotfiles repo is legitimate. The invariant to keep however this is later refined: **the loader must read no file the trust listing omits** (`test_workspace_trust.py`).
Progressive disclosure: system prompts list skills as name + one-line description only; full bodies enter context solely via the `load_skill` tool result. Assembly lives in one place — `prompts/system_prompts.py`'s `with_skill_catalog()` / `pass_prompt()` — so the catalog can't diverge between an attempt and its JSON retry.
## Conventions
**Ask before implementing.** Every built ROADMAP section followed the same cycle: read referenced files → verify claims against actual code → surface ambiguities as explicit questions with options and a recommendation *before* writing code → lock decisions → implement. New sections follow it too.
**Closing a `docs/TECHNICAL_DEBT.md` item is a DEVLOG entry and NOT a new ROADMAP section.** Batches 115–121, 126 and 127 are the precedent, and the reason is mechanical rather than stylistic: a section needs a decision family in this file's map, an index row and a `(BUILT)` status marker, and all three are gated — `test_the_map_names_every_family_the_record_defines`, `test_the_roadmap_v3_index_covers_every_section` and `test_an_index_status_marker_agrees_with_its_own_section`. So a closure that adds no config key and moves no documented threshold is an APPLICATION of ids that already resolve, and inventing a family for it would put three gated claims in the record to describe a fix that needed none. **The register entry is evidence and not instruction**: batch 126 found the entry naming the wrong field, and 127 found one naming a route that was already covered and a fix shape that could not bound the function. Measure the entry's claim before building what it proposes, and correct the entry in the same batch.
**Fix at the producer, not the consumer.** The project's canonical bug: `add_assistant_message()` persisted a redundant `"role"` key, was fixed with a read-side special case for `role == "assistant"`, and the identical bug in `add_tool_result()` survived because the fix lived one layer above where the divergence was produced. When you fix a shared root cause, grep every other call site sharing it. A per-case branch added to a normalizer/dispatcher usually means the fix belongs one layer down.
**Verify against production code, not the test double.** `tests/conftest.py`'s `FakeStorage` is a simplified model of `storage.py`, not the real thing — its `save_checkpoint` has none of the real one's watermark-advance guard, so a test that arranged a backwards checkpoint through the fake would see it applied, and `latest_checkpoint` is a dict lookup there rather than "newest row wins by timestamp". A passing test proves the fake and the code agree — not that either is right. If you change `storage.py`'s reconstruction, change the fake identically.
That example is deliberately a *current* divergence, because a rule whose only evidence is stale gets read as boilerplate. The checkpoint divergence costs nothing today, because the guard is pinned against real SQLite in `test_storage_e2e.py` and the fake's `save_checkpoint` is only ever used to *arrange* a compacted view. `test_fake_storage_mirror.py` (#123) now compares the reconstruction halves directly.
**Never name a top-level file after a stdlib or installed package.** A root `logging.py` once silently shadowed stdlib `logging`; hence `logging_setup.py`.
**Registering a tool is four steps**: import the module, `registry.register(ToolSpec(...))`, add a boolean to `config.ToolPermissions` and -- with one deliberate exception -- to `config.ToolApprovals`, and `assert_permissions_declared()` at the bottom of `tools/registry.py` enforces the third at import time (D24). The exception is `security/permissions.py`'s `APPROVAL_BY_SHELL_MODE` -- `shell` and the three session tools that START one -- whose approval lives solely in `shell_approval_mode`; a test pins that set as the sole exemption, and `_assert_shell_mode_exemption` requires each of them to carry THE GATE `_SHELL_MODE_GATES` NAMES FOR IT, by identity, so the hole cannot outlive the gate that justifies it (§49, SS16). Three of them carry `shell._shell_approval_check`; `shell_interactive` carries `shell._interactive_approval_check`, because its yes covers every line typed into the session afterwards and so cannot be a question about one command (batch 100, SS29/SS32). Skipping the declaration used to fail silently — `fetch_url` was registered, documented as working, and denied on every call for its entire life, with the schema still advertised so the model kept choosing it. It now raises `RuntimeError` on import instead. `mcp__*` names are exempt (they get `_default_for_unknown_tool`'s named default).
**A raising tool must not kill the run.** `dispatch()` wraps the handler call and turns any exception into `{"error": ...}`, logged at ERROR with the traceback so a real bug stays findable. `ToolCallDenied` and the unknown-tool `ValueError` are raised *above* the handler and deliberately still propagate. The error result goes through `check_output_policy` like any other — an exception message often carries the request that produced it, and for an HTTP client that means a URL with an API key in it. This is a backstop: `web_search` and `arxiv_search` return their own error dicts after exhausting retries (`fetch_url` always did), so the model gets something specific. The bug that forced this: `arxiv_search` requested `http://export.arxiv.org`, arXiv now 301-redirects it, httpx does **not** follow redirects by default, `raise_for_status()` ignores 3xx, and `ET.fromstring("")` on the empty redirect body raised — three retries, then an exception that flipped a finished ten-pass research run to `status='failed'`.
A tool that should not always be *offered* (as opposed to not allowed) supplies a `ToolSpec.available_check` predicate — `load_skill` does this so it disappears from `schemas()` when no skills are catalogued. Don't special-case it inside `schemas()`; the registry owns mechanism, not policy.
**Log detail must be in the message, not in `extra={}`.** The default formatter renders `%(message)s` only, so every field passed via `extra=` was silently dropped — fifteen consecutive `fetch_url failed` lines with no URL and no reason, and an `arxiv_search attempt failed` that hid a scheme change for three retries. Interpolate instead (`logger.warning("fetch_url failed for %s: %s", url, e)`). This matters more since the TUI routes WARNING+ into the transcript.
**Debug breadcrumbs name things, never quote them (batch 78).** `on_loop_event_message` logs event-kind flags plus screen-stack depth, and `_blocking_modal` logs screen type plus outcome class — params, answers and notes stay out of the file. A breadcrumb quoting a value is one log rotation away from a durable secret beside the redacted sinks, and the ordering story never needs the content. `logger.isEnabledFor(logging.DEBUG)` guards each site so the per-token hot path pays one predicate; a test asserts silence at INFO.
**Math tools** all share `_math_common.py`'s `safe_parse()`. Never use raw `eval`/`sympify` in a new one. Their tests assert **symbolic equivalence** (`sympy.simplify(actual - expected) == 0`), not string equality — SymPy's `str()` drifts across versions.
**`safe_parse` is an ALLOWLIST at the token boundary, not a blanked namespace (#52).** This file used to say "`__builtins__` blanked, injection-tested"; both halves were false. Blanking cannot work — `_SAFE_GLOBALS` is built from `dir(sympy)`, so SymPy's own `sympify`/`parse_expr`/`var`/`S`/`lambdify` are in scope by construction, and `sympify`'s inner parse gets **real** builtins back because Python inserts them into any globals dict handed to `exec`/`eval` that lacks them; separately, `().__class__.__bases__[0].__subclasses__()` needs no name lookup at all, so no namespace pruning reaches it. And "injection-tested" was one payload that was blocked by `auto_symbol` Symbol-izing the bare name `__import__`, for a reason unrelated to the defence it was named for. **A token filter was the first fix and it was wrong.** It rejected dunder and non-allowlisted NAME tokens, and was bypassed within the hour by `f"{sympify('__import__(chr(111)+chr(115)).getpid()')}"` — on Python 3.11 an f-string is a **single `STRING` token** (PEP 701 split it up only in 3.12), so a NAME filter is structurally blind to it while the code inside evaluates with `_SAFE_GLOBALS` in scope. `str.format`'s mini-language reaches dunder attributes the same way. The general lesson: **a token stream is not the language**, and which constructs the tokenizer hides is a property of the Python version, not of this code.
The guard is now `_validate_ast`: `parse_expr` is split into its two halves (`stringify_expr` → `eval_expr`) and the AST of **exactly the string that will be evaluated** is checked against a closed allowlist of **39** node types out of the ~130 Python defines. **`ast.Attribute` is absent, and that is the load-bearing omission** — attribute traversal *is* mechanism 2, and it is also how `str.format`, `.mro()` and every bound-method route are reached; no transformation emits an attribute for a legitimate expression. `JoinedStr`, `Lambda`, the comprehensions, `NamedExpr`, `Starred` and `Dict`/`Set` are all absent too, by being unlisted rather than by being recognised. The one exception is RECOGNISED: `f(**mapping)` is spelled `ast.keyword(arg=None)`, `keyword` is listed for ordinary keyword arguments, and `_validate_ast` refuses that one shape by name since batch 160 (A68). Names are checked against `_ALLOWED_NAMES` (**230** of the ~926 `sympy` exposes) or the caller's `symbols=`. Widening either list is a decision about a *kind* of thing; anything that parses, evaluates, compiles, renders or executes stays out, and no module object is ever permitted (`evalf` is a module, not the method — it was allowlisted by mistake once).
**Both of those figures are gated, and the denominator is a tilde for a reason.** `test_agents_md_states_the_real_sympy_allowlist_sizes` reads 230 and 39 back out of this paragraph. The 926 is approximate because **`dir()` on a PACKAGE is not a fixed number**: `tools/builtin/probability_stats.py` imports `sympy.stats`, which binds `stats` onto the package, so the count moves by a name or two with import order and reads 926 or 927 in one checkout. **And `_SAFE_GLOBALS` is built from that same `dir()` at import time, which is how a module object gets into it** -- against the rule stated just above. Inert, and measured so: `stats` is not in `_ALLOWED_NAMES`, so the name is never resolved, the bare word splits into `a*s**2*t**2`, and `stats.Die` is refused with every other attribute. The NAME gate is what makes the globals dict's contents not matter.
**117 of the 213 allowlisted callables evaluate a string handed to them** — measured by blackbox probe, not by reading source, because of the **229 allowlisted names that resolve** only 15 had retrievable source and the other 214 were C extensions. **The two figures count two different populations**, which is why 214 reads as larger than 213 and is not: 117/213 is over the CALLABLES, 15+214=229 is over the RESOLVABLE NAMES. Batch 128 named the populations rather than changing a number, having re-taken the measurement in WSL -- the probe needs `SIGALRM` and is skipped on Windows -- and got 117 of 213 back, with the control that proves the probe still detects a known evaluator run first. So the string-licensing rule is load-bearing rather than defence in depth, and `test_the_licensed_string_constructors_do_not_evaluate_their_argument` is the obligation it rests on. There is also one check on the OUTPUT: `_reject_escaped_value` refuses a result that is a module, a plain function/method, or a non-SymPy class, recursing through returned containers. It is a DENY of the escape signature rather than an allowlist of return types, deliberately — an allowlist can only catch escapes whose shape was already imagined, and four of this issue's five mechanisms were not. SymPy classes are values (`sin` is a `FunctionClass`); `Function('f')` is refused by the string rule, which is a recorded cost.
**String literals are licensed, not filtered.** Auditing the allowlist's own source found that seven permitted functions — `factor`, `cancel`, `together`, `apart`, `roots`, `degree`, `nsolve` — call `sympify()` on their arguments, so handing one a string literal re-enters the parser with SymPy's default globals and real builtins: `factor(" __import__(chr(111)+chr(115)).getpid() ")` returned the pid. A dunder tripwire on the string did not help, because it matched prefix/suffix and one leading space walked past it — the same category of mistake as `chr(111)+chr(115)` defeating a search for `"os"`. `stringify_expr` emits a string argument in exactly two places (`Symbol('x')`, `Float('1.5')`), so `_STRING_ARG_CONSTRUCTORS` licenses those two positions and every other string literal is refused. That closes the class without needing to know which functions sympify — which is the difference between a closure and a filter.
### Testing
All tests run offline: zero network, zero real API keys. The **root** `conftest.py` stubs 6 SDK packages into `sys.modules` at import time (`openai`, `anthropic`, `google`/`google.genai`, `sqlmodel`, `httpx`, `ddgs`) — plus `markitdown`, which is not an SDK but is an optional extra since #144, so a default install does not have it and `test_file_ops.py` patches it by string target — fixtures run too late, since `core/client.py`'s imports fire during collection. `pydantic` and `sympy` are deliberately *not* faked. Tests needing real SQLite monkeypatch the fake `sqlmodel` away (pattern documented in `test_memory_write_through.py`'s docstring).
**Waiting in a Textual pilot test: `settle`, never a pump count** (`tests/conftest.py`, TECHNICAL_DEBT 8). `await settle(pilot, predicate)` waits on a wall-clock deadline and quiesces once the predicate holds; `await pump(pilot, n)` gives the loop `n` turns and is for *negative* assertions only ("prove X did not happen"), where there is no predicate and a deadline would make every such site pay its full timeout. Do not hand-roll either — there were five copies with five budgets and two orderings, and the two bugs in them were invisible to 1246 passing tests, so `test_pilot_wait.py` now fails on a sixth. Two facts behind this: `pilot.pause()` is **not** a fixed unit of time (21ms with nothing burning CPU in-process, 1021ms with one busy sibling thread — its exit condition is a process-wide CPU heuristic), and a predicate like `isinstance(app.screen, SomeModal)` goes true *before* the worker reaches `channel.get()`, so joining that thread from the event-loop thread deadlocks both ways.
**A test that mocks `registry.dispatch` cannot see an authorization defect** (batch 60). The approval gate lives INSIDE `dispatch()`, so mocking it removes the thing under test. Every test pinning the §23 memo and the §25 grants did exactly that and stayed green through it for four weeks: they counted `permission_request` events and never asked whether the call that skipped the prompt actually ran. A test asserting an approval, grant or sign-off is HONOURED must drive the real `dispatch` and stub the tool's HANDLER instead — `mock.patch.object(registry._tools[name], "handler", ...)` — which removes the side effect and leaves every gate live. Two traps come with it: `shell` and `write` are DISABLED in the shipped config and are refused by `is_tool_allowed()` long before the gate, so such a test must raise `ToolPermissions` or it passes while proving nothing; and `subagent_tool.run` builds the child's system prompt, which reaches the memories subsystem and therefore storage.
### Before calling a change done
- Did you run (or `py_compile`) every file you touched, and run `pytest`?
- Did you run `ruff check .`, bandit with the CI job's own flags, and `gitleaks detect --source . --no-git -c .github/gitleaks.toml`? None of the three runs under `pytest` and all three gate every push, so a batch that skips them arrives red — which is how the push carrying batch 94 landed six lint errors and two new bandit findings, every one of them mechanical.
- **The bandit baseline is in LINUX path form, so locally every entry misses and every baselined issue reads as new.** It is generated in CI, where filenames are `./x.py`; bandit on Windows reports `.\x.py`. Rewrite a THROWAWAY copy, changing only the leading `./`, and run against that — **never regenerate the committed file**, which would relocate the baseline onto one machine's paths and make CI the thing that disagrees. The regeneration command in `.github/workflows/bandit.yml` is for CI, not for a developer box. Second trap in the same command: piping bandit to `tail` makes `$?` TAIL's exit code, so a run with new findings reads as green — capture the output and read bandit's own status.
- **Gitleaks is the one that can sit red for weeks, so run it even when the batch touched no production code.** Lint and bandit are caught the same day because something else usually runs them; gitleaks fires only `on: push`, so a branch held locally banks the surprise for as long as it stays there. `tests/test_session_tools.py`'s `SECRET` fixture was added 2026-09-17 and first went red 2026-10-01, twenty commits and several batches later (batch 115). **A new fake credential in a tracked test needs its fingerprint in `.gitleaksignore`; a real one needs rotation, never a fingerprint.** Scan a clean export rather than the worktree — `git archive HEAD` into a temp dir — or local untracked residue answers for CI, which checks out tracked files only. **`cd` into that export and scan `.`**: a `.gitleaksignore` fingerprint is `file:rule:line`, so an ABSOLUTE `--source` makes every one of them unmatchable and a clean tree reports all of its own deliberate test credentials as leaks (measured in batch 128: 7 findings absolute, 0 relative, same bytes).
- Did you grep for every other call site sharing the root cause of what you just fixed?
- Did you trace the fix against the real module, not the fake?
- Does the new test fail if the fix is reverted, asserting the field/shape that was actually wrong?
- Did anything surface a design decision not already settled in ARCHITECTURE.md / ROADMAP*.md / DEVLOG.md? Ask, don't assume.