/`, the kind of file), with organic opens, suggested, suggested-then-opened, unread, missed and negative per row and exposures per area, each decaying with `harness.skillSuggestHalfLifeDays` (30). The session's opens and edits are credited once, idempotently, from the transcript the harness parses (`harness.js:sessionHarness`, `harness-codex.js:codexSessionHarness`; `recordSessionSkills` from `commands/learn.js:distillFile`), so `learn.sessions`, `THINKER_NO_LEARN` and the quiet skip apply as to the distillation. Scoring (`candidates`) counts a (area, skill) only on the sessions where the skill was not read (not opened; a suggestion the agent did not follow leaves the session unread): Beta(0.5 + misses + organic opens, 4 + (unread − misses) + negatives), the best of the file's areas, Thompson-sampled, top two above a sample of 0.35. A session that read the skill says nothing about whether it prevents a break; counting those as failures, as the first version did, held a skill that prevents most breaks at a 0.4 suggestion rate the more it was followed (a simulation of the loop, 2026-10-19). A row made late counts the area's earlier sessions as unread unless the skill was `added` to the registry since (`refreshRegistry`), so a skill opened once in a busy area does not score as if that were its only session. Organic opens are weak evidence of relevance (weight `organic`, 1); a skill whose description shares a word of the path gets a small prior (0.25) only when the file's areas have no evidence-backed candidate at all: in a replay of 8 PostHog eval tasks none of its 24 picks was a skill the agent went on to open. Misses come from a separate, sampled outcome check (`skill-outcome.js:skillOutcomeCheck`, its own switch `harness: { skillOutcome: true }` or `THINKER_HARNESS_SKILL_OUTCOME=on`, off by default, never in test mode unless forced by the env), modelled on the Artemis conventions grader: once a session, on `harness.skillOutcomeShare` (0.25) of sessions by a hash of the id, when the session edited files with candidates, within the daily token cap, one model call through `llm.js:complete` (the session queue in a desktop app). It is shown the `git diff` of the files the session edited from the commit before its start (plus files it created), the full text of the best three candidates and of up to two skills a matching `.claude/rules` file names, those rule files, and the AGENTS.md text that applies (process sections left out), and returns violations quoting the rule and its source; a skill's break unread is a miss, read a negative, logged as `op: "skill-outcome"`. Asking the end-of-session distillation for broken rules instead (the first version) matched the grader on 0 of 30 graded sessions. On 39 graded PostHog sessions (Sonnet, claude CLI, $0.14 a session) the first prompt found 7 of the grader's 15 skill-sourced violations by rule, and 3 of 15 clean sessions got a skill violation; the prompt shipped found 13 of 15 on the 13 sessions with one (16 found, 3 on a rule the grader did not cite); its rate on clean sessions was not measured again. A held-out turn (`ops.js:holdoutTurn`) is told nothing and logs `withheld`; the cap of six lines a session counts withheld lines too. First use seeds the store from the machine's recent sessions of the repository; `thinker skills seed ` seeds from chosen ones (an eval: its training tasks only), `thinker skills suggest ` previews. The offline checks are in `bench/worktrees/eval-support/skill-protocol/` (local). No agent session has been run with it. **Built-in tools in the harness question (off until an experiment, 2026-10-20).** With `harness: { builtinTools: true }` in `.thinker/config.json` (or `THINKER_HARNESS_BUILTIN_TOOLS=on`; `=off` wins over the config), the harness counts Claude Code's own tools as sources beside plugins, skills and MCP servers: their definitions from the transcript's `prompt_snapshot` attachment, a call by name as a use (`harness.js:sessionHarness`, `builtinKey`). One called in under a tenth of the sessions is offered like any source and goes off as a bare name in `permissions.deny` of `.claude/settings.local.json`; `--undo` takes it out. Checked with `claude -p` 2.1.304 (2026-10-20): `permissions.deny` and `--disallowedTools` both take the definition out of the prompt, Workflow, ScheduleWakeup and ReportFindings together ~4k tokens a call. Never offered (`CORE_TOOLS`): the file and shell tools, Skill, AskUserQuestion (the question goes through it), and ToolSearch, whose absence puts every deferred definition inline (+29k tokens a call); an allow-list (`--tools`) does the same, so the switch is a deny list. Why: on this machine's sessions (Artemis Tokens page, waste view) Artifact (15k tokens) was called in 1 of 65 sessions that carried it, Workflow in 0 of 535. Not known: whether the desktop app's own tools (Artifact, SendFeedback, absent from `-p`) obey the deny list. ## Desired behaviors Every other kind of note is a claim about the code, and when code and note disagree the note yields: verification rewrites or retires it. A `behavior` note (`behavior.js`) is the other way round: a person writes down what the system must do and where that is upheld, and from then on a change must conform, or be called out for not conforming. The rule since 2026-10-04: **before a merge the behavior is the truth, after it the code is.** A pull request that stops upholding a behavior is told so, in full and first (a fixed behavior loudly, see Review below); once the change is merged on the default branch the behavior is revised to match the code, with the old text in its history. Behaviors are never evaluated against a gold set; the review's callout and the person's decision to merge are the check. `thinker system` is the view of them; the MCP `lookup` with `kind: "behavior"` returns them to an agent (all of them with no query); `thinker system md` writes them as `.thinker/SYSTEM.md` for people who read the repository without the tool. Legacy committed behaviors remain readable; new behaviors stay local. - **Fields.** A behavior is an ordinary note (title, body with `file:Symbol` pointers, answers, deps) plus `mutability`: `fixed` (a review treats a change that stops upholding it as an error and requests changes, with the behavior quoted in full) or `mutable` (the default; a warning, and a pull request may revise it by editing the note itself). Both follow the code once a change is merged. `mutability` and `revised` ({at, commit, reason}, set when the merged code rewrote it) are shared content; `violated` is this checkout's state (`store.js:LOCAL_FIELDS`). - **Who writes one.** A person: `thinker system add file.json [--fixed]`, or `thinker system promote [--fixed]` for a note that already states a rule (`thinker system propose` lists the rule notes, the ones the sessions acted on first). An agent may save one with `remember` (kind `behavior`); it is listed as *proposed* until `thinker system accept `. Distillation and PR mining are not offered the kind (`distill.js:distillSpec`, `prs.js`), archiving never takes one (`ops.js:archiveReason`). - **Build proposals.** `thinker setup --build` drafts up to twelve high-signal PR/document rule notes into desired behavior candidates in `.thinker/local/behavior-proposals.json`. The model sees each note's source and current symbol code; outputs are restricted to the note's existing symbol deps. `thinker system propose` shows drafts and their provenance; `thinker system accept proposal-` creates an active behavior only after a person reviews it. The draft is not served or reviewed as a requirement. Acceptance refuses a changed source note or code. The stage runs on every cache build; it returns no drafts if no suitable source notes exist. - **Verification** (`ops.js:verifyBehavior`, through `verifyNote`): when a dep changes the question is whether the code still upholds the behavior. `holds` re-baselines; `moved` re-points the deps at where the behavior is upheld now (history kept). `broken` depends on where the code is (`ops.js:onDefaultBranch`: the deps' files committed, and `HEAD` reached by origin's default branch, or a local `main`/`master` without a remote). Not merged (a working tree, a branch, unpushed commits): `status: violated` with the commit and the reason, deps re-hashed so the note reads as violated rather than stale until the code changes again, text untouched, named at the end of the next turn (`ops.js:noteUnreported`, `maintain.js:maintenanceNotice`) and served with a `⚠ VIOLATED` banner. Merged: the model is asked for the behavior as the code upholds it now, the body is replaced (pointers of the new text join the deps), `revised` records the commit and reason, the old body goes to `history`, the verdict is `revised`, the notice says so and `.thinker/SYSTEM.md` is rewritten. A behavior is never retired by verification, and never rewritten for code that is not merged. On the server the clone sits at the default branch, so the behaviors there follow every merge and sync out. - **Review** (`review.js:assessBehavior`, and the holistic prompt): the verdicts are `violation`, `consistent`, `unrelated` and `revised`; `revised` only for a mutable behavior whose note file the change edits (`review.js:noteFileChanged`, in the scope under review), otherwise it is a violation. A violation without a placed finding gets one at the first definition the change touched. A finding on a fixed behavior is an error, on a mutable one at least a warning, so `--strict` exits 2 in CI. The posted pull request review (`review-post.js:buildReview`) opens with a violated fixed behavior before anything else: its title, its full text under "What it requires", what the change does instead (the model's reason), and what merging means (the code becomes the truth and the behavior is revised to match; restore the enforcement if that is not meant). A violated mutable behavior gets the same block under a quieter heading. `renderReview` says the same in one line for the CLI. The holistic call may not list a behavior under `outdated`: one it does is turned into a finding. The report carries `behaviors`, one line per behavior in play (`upheld`, `violated`, `revised`, `unrelated`, or `consulted` on a dry run) with whether it was already violated before the change, rendered under "Desired behaviors" and in `--json`. - **This repository's own** (`thinker system`, `.thinker/SYSTEM.md`), written 2026-10-03: a review never writes to the cache; verification never rewrites or retires a behavior; serving and assessment never rewrite a shared note's committed file; `THINKER_TELEMETRY=off` and test runs send no telemetry (all fixed); the prompt hooks serve no stale note (mutable). - **Serving.** A behavior is served like an invariant: at orientation with a small kind prior, by the edit hook when a file it rests on is edited (`ops.js:lateNotes`, first among the rules), and in `thinker review`'s consulted set with the highest kind weight. ## Review was removed (2026-10-11) `thinker review` (a single model call over the diff, the code it touched and the notes resting on it; later with `usedContext`, the code the change uses) was removed, with task verification (`--run`), the pull request action, the server's review worker and the pre-commit option. On held-out bugs, fixes withheld from the cache, it caught no more with notes than without (mitmproxy 10-11 of 15 either way, PostHog 5 of 8), and the user's own coding agent reviewing the same change caught 14 of 15 with or without thinker alongside. Its one distinctive number, recall of fixes already in the cache, is not a claim this project makes. The measurements are in `research/regression-heldout` on the `agent/heldout-regression` branch. What survives: `src/change.js` (the model-free change helpers) for the diff-keyed serving of notes to a reviewing agent, and two findings for a review built around the agent: showing the code the change uses lifted the single call from 7 to 11 of 15, and an input-class checklist to 12 of 15. ## Supported agents | agent | notes for the request | notes about files being edited (installed by default; `--no-late` leaves it out) | MCP tools | written to (user scope) | |---|---|---|---|---| | Claude Code, and the Code tab of the Claude desktop app | added to each prompt | yes | yes | `~/.claude/settings.json`, `~/.claude.json` | | Codex CLI and the Codex desktop app | added to each prompt | yes | yes | `~/.codex/hooks.json`, `~/.codex/config.toml` | | Gemini CLI | added to each prompt | yes | yes | `~/.gemini/settings.json` | | Cursor | added to each prompt (`additional_context`), with first-tool fallback for `agent -p` and the `orient` tool | yes | yes (approved by setup) | `~/.cursor/hooks.json`, `~/.cursor/mcp.json`, and `.cursor/rules/thinker.mdc` in the checkout | | Pi, Windsurf, Copilot CLI (and Copilot in VS Code), OpenCode, Factory Droid, Qwen Code, Kiro CLI, Cline, Kilo Code, Auggie, Goose | see `docs/agent-integrations.md` (Auggie has no prompt hook, Kiro's V3 engine alone reads global hooks, Goose's hooks only record) | | | each agent's own files, machine-wide | - The desktop apps run the same engines as the CLIs and read the same user settings. The Claude desktop app's Code tab reads hooks and MCP servers from the user's and the project's files alike; the Codex desktop app reads MCP servers from the user's `config.toml` only (openai/codex#13025, open since 2026-02), which is why the wiring is machine-wide. Whether it starts the MCP server in the thread's directory is not verified: if not, the server asks for MCP roots, and failing that offers nothing there. - One copy of thinker per checkout (`clients.js:pruneInstalls`). A hook names the copy it runs (`node "/src/cli.js" hook …`), and so does an MCP entry; two copies wired into one checkout (an install left behind, a smoke-test copy, hooks in both of Claude Code's settings files) both fire on every prompt, serve the notes twice and bring the old copy's notice and state format back. `setup` takes every other copy's hooks out of the files of the client they install and point its MCP entry at the new copy; for Claude Code the hooks live in one settings file, so installing `--local` empties the shared one and the other way round. The prompt hook does the same on every prompt for copies that are gone or older by their `package.json` than the one running (`olderOnly`): a copy of the same version is left, or two copies would take each other out, and a newer copy is left to do the cleaning. What was removed is logged (`op: "prune"`) and said once at the end of the turn through the maintenance notice. Only the entries go; the old copy's files are not deleted. - The wiring follows the installed version (`clients.js:refreshWiring`). The hook and MCP entries are written in the shape the version that ran `setup` knew; a later version that adds an event (`SessionEnd` for the final distill of a session) or changes a command rewrites them from what is there: `inferWiring` reads from each client's files whether hooks, late notes, learning and the MCP entry were installed and which Claude settings file holds them, and `installClient` is run again with those options. Only entries that run this copy (by the script's install root) are rewritten; entries of another copy, alive (a development checkout) or gone (a deleted worktree: the benchmark checkouts pointed at one, and rewriting them would have given them live hooks), and hand-tuned commands (an env prefix, a `--budget`: a benchmark arm's) are left alone and named. Checkouts under the temp directory are not visited. Git hooks of this copy are rewritten the same way. `thinker update` runs `thinker rewire` from the new copy after an update, for every checkout the machine's log names that is still set up; the prompt hook runs it for its own checkout on every prompt (`mergeJson` and `installGitHooks` leave an unchanged file untouched, so a client watching its settings file sees nothing); `thinker rewire [--here] [--dry]` is the command by hand. Before 2026-10-04 nothing did this, and this repository ran without `SessionEnd` and `PostToolUse` hooks for weeks after both were added. - Cursor's `beforeSubmitPrompt` hook returns `{"continue": true, "additional_context": "..."}` at prompt time (since 2026-10-15); in headless modes (`agent -p`) where `beforeSubmitPrompt` does not fire, `postToolUse` orients from the transcript on the first tool call. An always-applied rule also tells the agent to call `orient`. - Codex reads a project's `.codex/` only once the project is trusted, runs a hook only once it is reviewed, and asks before each MCP tool call. `setup` takes care of all three: the MCP server is registered with `default_tools_approval_mode = "approve"`, and the project and thinker's hooks are marked as trusted in Codex's own `config.toml` (`CODEX_HOME`, `~/.codex`). In a terminal they ask first; `--yes` skips the question, `--no-trust` leaves it to you, and without a terminal nothing is marked unless `--yes` is given. The hook entry is the hash Codex 0.157 stores (see [What Codex stores as trust](#what-codex-stores-as-trust)); if a later Codex changes it, Codex asks for the review as before. `uninstall` takes the hook entries out again. - Cursor loads an MCP server only once it is approved; `setup` does that through Cursor's CLI when it is installed. - Gemini CLI also drives agent mode in Gemini Code Assist, which reads the same MCP configuration. Google AI Studio is a web app and cannot run local hooks or MCP servers, so it is not supported. ### What Codex stores as trust Codex has no command for this and does not document the format. What follows was read from the `config.toml` that Codex 0.157 writes after a review in its own interface, and checked by running `codex exec` 0.157.1 on entries that thinker wrote. All of it is in Codex's own config (`$CODEX_HOME/config.toml`, by default `~/.codex/config.toml`), never in the repository. A trusted project is a table named by the real path of the repository: ```toml [projects."/Users/me/src/repo"] trust_level = "trusted" ``` A reviewed hook is a table per handler: ```toml [hooks.state."/Users/me/src/repo/.codex/hooks.json:user_prompt_submit:0:0"] trusted_hash = "sha256:6118…1471" ``` - The key is `:::`: the event in snake case (`UserPromptSubmit` is `user_prompt_submit`), then the index of the group in the event's list and of the handler in the group's `hooks`, both from 0. A hook that moves in the file needs a new entry. - The hash is SHA-256, in hex, of this JSON with no whitespace, keys in alphabetical order, and `async` written out as `false` when the hook does not set it: ```json {"event_name":"user_prompt_submit","hooks":[{"async":false,"command":"…","timeout":15,"type":"command"}]} ``` So any change to the command or the timeout makes the entry void, and Codex asks for a review again; `setup` writes new entries when it is rerun. - Checked: handlers of type `command` with a `timeout` and no `matcher`, which is what thinker writes, for `UserPromptSubmit`, `PostToolUse` and `Stop`. Not known: how a `matcher`, a `statusMessage` or a missing `timeout` enter the hash. `trustCodex` leaves a group with a `matcher` alone, and hooks that are not thinker's are never marked. - `codex --dangerously-bypass-hook-trust` runs hooks without these entries for one invocation; thinker does not use it. The code is `clients.js:codexHookHash` and `clients.js:trustCodex`; a hash that Codex stored is held in `test/clients.test.js`. ### Learning from any agent Learning works with all four agents, and with others through the generic path below. Nothing in it is tied to one vendor: - **Sessions.** `thinker distill` reads Claude Code, Codex, Cursor and Gemini CLI transcripts (and the streamed output of their headless modes) and maps them to one event form: prompt, tool call with result, agent message. Tool names are mapped to one vocabulary, so `read_file`, `Read` and a shell `cat` are all seen as reading. - **When.** Once per session, not per turn: each call carries the prompt, schema and related notes whatever the size of the trace, and on 2026-10-03 one session was distilled 13 times. At the end of a turn (`Stop`, `AfterAgent`, `stop`) the hook distills only a backlog near the trace limit (`distill --batch`, `distill.js:batchDue`, 45,000 condensed chars); the end of the session (Claude Code's `SessionEnd`, Cursor's `sessionEnd`) distills the rest. Catch-up covers agents that fire no end: `thinker learn` finds every session any of these agents ran in the repository and distills what is new in those quiet for 20 minutes (`cli.js:LEARN_IDLE_MIN`). Hooks start it in the background at most every ten minutes. - **What the agent's record leaves out.** Where an agent gives no transcript, the hooks record the session themselves (`.thinker/state/trace-*.jsonl`). Cursor records tool calls without their output; reads and searches are repeated against the working tree when distilling. - **Model.** Distilling, verifying and PR mining run through whichever agent is installed, with the login it already has: `claude -p`, `codex exec`, `agent -p` (Cursor) or `gemini -p`. A session is distilled by the agent that ran it when possible. `THINKER_LLM` picks one (`claude`, `codex`, `cursor`, `gemini`, `anthropic`, `openai`); `THINKER_LLM_MODEL` names a model for the non-Claude ones; `THINKER_LLM_BASE_URL` is any OpenAI-compatible `/chat/completions` endpoint (token from the variable `THINKER_LLM_API_KEY_ENV` names), used before the agents; `THINKER_LLM_CMD` is any command that reads a prompt on stdin and prints the answer, told the tier, model and schema in `THINKER_TIER`, `THINKER_MODEL` and `THINKER_SCHEMA` (docs/usage.md). - **Any other agent.** Pipe events to `thinker record ` as JSON lines (`{"t":"prompt","text":…}`, `{"t":"tool","name":…,"input":…,"result":…}`, `{"t":"say","text":…}`), then run `thinker learn`. Agents that speak MCP can also save notes with the `remember` tool. ### What was run live, on 2026-09-27 | | Claude Code | Codex CLI 0.157 | Cursor agent CLI | Gemini CLI | |---|---|---|---|---| | notes served in a session | yes | yes | yes | not installed here | | session turned into notes | yes | yes (hooks, trace and distilling through Codex ran; the short test session produced no notes) | yes (4 notes from 2 sessions, distilled through Cursor) | not installed here | Gemini support follows its documentation and is covered by unit tests on constructed input only. Seen in the live runs: - Codex did not run hooks from a project's `.codex/hooks.json` in `codex exec` with only the project marked trusted, but ran the same hooks from the user-level `hooks.json`. With the hooks marked as reviewed too (what `setup` now writes), `codex exec` 0.157.1 loaded the project's MCP server, called `orient` without asking and ran the project's hooks. - Cursor's CLI (`agent -p`) fires `sessionStart`, `postToolUse`, `afterShellExecution` and `sessionEnd`, but not `beforeSubmitPrompt` or `stop`, and `postToolUse` carries a summary of the tool's output, not the output. So thinker installs both the editor's and the CLI's events, takes shell output from `afterShellExecution`, re-reads files when distilling, and `setup` approves the MCP server (`agent mcp enable thinker`) so the agent can call `orient` at the start and `remember` at the end. In the test session it did both unprompted. ## Jev, tried and removed TypeSafe's Jev (System One: calibrated probabilities for typed questions, no generated text) was wired into thinker between 2026-10-06 and 2026-10-08 and taken out entirely on 2026-10-09, at the user's decision. It had ranked the servings (a hosted proxy on AWS, a personal-key path, `thinker ranker --jev-key`), narrowed the related notes and gated the optional steps of a review, typed the drift surface of every note for derived verify verdicts, chosen the passages a distillation saw, and gated what learning wrote: a catalog scan, a relation judgment against existing notes, grounding of every claim against the source, and a fidelity check of each search description, with a deferral queue under `.thinker/state/learning-pending/` for what the gates held back. The measurements are in `research/bench-history/RESULTS.md` ("Serving: Jev", "Derived verify verdicts", "Jev capture selection", "Current Jev versus historical cached runs") and under `research/jev-*`. What they showed: a strong offline serving result (2.5x the important notes reached at the same precision on 54 labelled tasks) that did not pay in live paired runs (costlier in 8 of 8 Click pairs); a smaller review prompt with the same catches and, with verification, 86% more tokens; archive triage and every intrinsic note facet at AUC ~0.50; capture selection that kept 0 of 13 reference facts at three times the tokens; learning gates whose thresholds discarded most proposals until retuned, with no paired measurement of the gated cache against a plain one. Only the external-tool re-baseline in verification was sound, and it covered 5% of stale notes. What stands in its place is what was there before: the hooks rank with the cross-encoder and the agent's `orient` and `lookup` by words; learning saves what the distiller proposes through `saveNotes` (lexical near-duplicate merging, named `extends` targets honoured), the distiller shown the notes resting on the files a session or pull request touched; `phraseNotes` keeps the description the writer returns; every stale note goes to the verify model. The deferral queue is still written for assessments of notes a session was shown but did not discuss (`learning-pending.js`). One daily token cap (`dailyTokens`) covers learning and maintenance. Git holds the removed code at `0cb765c`. Do not bring Jev back without a live paired measurement on held-out tasks; the offline labels were not predictive of session cost.