git:20260916.7a55f38 to git:20261002.3e13332

---

name: mcp-tool-surface

description: How to change the whiteboard MCP tool table — rename, re-describe, reshape, add, consolidate or retire a tool — with the two scoreboards and the LLM-driven lane that ADR-0031 makes the evidence. Use before touching a tool's name, description, input schema, annotations or existence under packages/server-core/src/tools/** or packages/mcp-server/src/server/mcp/**, when a consolidation is proposed, when someone says "too many tools", or when adding an eval task. Not for the daemon's HTTP routes or the web app.

---

# The tool surface is judged by an instrument

A tool definition is read by a model on every turn of every conversation

that has this server attached, and a model asked to do something reads the

table before it reads the document. So a change to the table is not judged

by whether the tests pass — they pass either way — but by four numbers

nobody sees in the diff: what the table costs to read, how much of it

explains itself, what an errand costs in calls, and whether a real model

still completes the errands. The criteria are

[ADR-0031](../../../docs/contributing/adr/0031-tool-surface-criteria.md)

§1 (C1–C14, each tagged with the rung that checks it); this skill is the

procedure.

The count is not a criterion. The last consolidation would have steered

by it, and the count cannot see the two largest debts the instrument found

on its first run: one tool being 45% of everything the model reads, and

299 of 324 parameters undescribed.

## The ladder, and which rung a change runs

| rung | what | command | runs |

|---|---|---|---|

| 1 | tool-surface scoreboard: per tool, model-visible bytes, wire bytes, description words, parameters and how many are undescribed, what a stray key gets, which neighbours the description names; plus totals, and C9 for every tool | `pnpm test --project mcp-node packages/mcp-server/src/server/mcp/tool-surface-quality.test.ts` | every push; pinned exactly |

| 2 | errand scoreboard: calls and bytes per errand | `pnpm test --project mcp-node packages/mcp-server/src/server/mcp/tool-call-count-quality.test.ts` | every push; pinned exactly |

| 3 | LLM-driven lane: a real model, only this server's tools, a seeded fixture, graded by outcome | `pnpm eval:tool-surface` (`--trials=3`, `--only=<substring>`, `--model=<alias>`, `--out=<file>`, `--dry-run`) | on demand; spends quota (~$0.06 a task, ~$0.70 a run); skips without the `claude` CLI |

| the change | rungs |

|---|---|

| a description or a `.describe()` on a parameter | 1, then 3 for the tasks that touch the tool — the one study that measured it found description fixes regress a sixth of cases, so a rewrite is not assumed to help |

| an input schema reshaped | 1, 2 |

| a tool added | 1, 2 (write its errand), 3 (write its task), plus the four-place list `mcp-smoke-coverage.ts` documents and a `smoke:e2e` call |

| a rename, a consolidation, a retirement | 1, 2, and 3 with `--trials=3` BEFORE and AFTER, both readings in the PR |

| a change to the server `instructions` | 3 |

## Procedure for a rename, consolidation or retirement

1. **Take the before reading.** On the base commit: rung 1 and 2 are the

pinned values already in the tests; rung 3 is

`pnpm eval:tool-surface --trials=3 --out=tmp/notes/tool-surface-before.json`.

Three trials, because one is a reading and the pass column moves

between trials on a tool that is reachable but not reliably reached

(the tag-write task took three different routes in three trials).

2. **Make the change.** Rung 1 and 2 go red — that is the design. Re-pin

each moved row, and write beside it WHY it moved, in the currency it

moved in: bytes the model no longer reads, calls an errand no longer

makes, parameters that now say what they are for. A row re-pinned with

no reason is the regression the exact pin exists to refuse.

3. **Take the after reading** with the same command and trials.

4. **Decide by the columns, in this order.** C14 (rung-3 pass^k) may not

fall. Then at least one of C1 (visible bytes), C13 (calls) or C3

(undescribed) must have moved down, and the PR says which and by how

much. A change that cut the count and moved none of those has changed

the table's shape and nothing a model pays for.

5. **Read the calls column as a diagnostic, never as the gate.** A task

that passes in seven calls where the errand scoreboard's cheapest path

is two has not failed; it says C4 or C5 is unmet on the tools it

wandered through, and the `tools` column says which. Grading by the

path an author expected is what the sources call brittle, and the

lane deliberately does not do it.

6. **Put both readings in the PR body**, the way `visual-evidence` puts

two panels: the rung-1 totals before and after, the rung-2 rows that

moved, and the rung-3 summary (pass^k, mean calls, cost) before and

after. Retirements are breaking for MCP clients under the 0.0.x

no-compat policy; say so in the title (`feat(mcp)!:`) and update the

product skills under `./skills/*` — `skills-tool-surface.test.ts`

catches a skill still naming a retired tool.

## In the development flow

- **Inline (the default for a surface change):** load this skill, take the

before reading, change, re-pin, take the after reading, then run

`review.workflow.mjs` over the diff with `dimensions` including

`{ name: 'tool-surface', content: <resources/tool-surface.md> }` beside

the defaults — the criteria file is

`.claude/skills/review-gate/resources/tool-surface.md`, and the

`review-gate` skill says how a caller globs and passes it.

- **Through `dev-loop`:** its design prompt asks for the rows the change

will move and, for a rename or retirement, the lane's before/after in

`testScenarios`; pass `reviewArgs: { dimensions: [...] }` so the

composed review carries the dimension. The `developer` agent has no

`claude` CLI and no quota, so the rung-3 readings are the main

session's: take them before launching and after folding.

- **In the PR:** both readings, the way `visual-evidence` ships two

panels. A reviewer reads C14 first, then which of C1/C13/C3 moved.

## Reading a rung-3 result

The lane prints one line per task and a summary; `--out` keeps the runs

with every tool call, tool-error text, token and cost figure.

- **A failure is probed against the store, not read off the transcript.**

Two of the baseline's failures looked like model mistakes and were the

product's: a tag-filtered search that cannot stand alone, and a comment

thread that does not survive a restart. Reproduce what the model saw

against a seeded directory before concluding anything about the model.

- **`toolErrorTexts` is the C11 evidence, and sometimes the C5 evidence.**

It is what the model was told when a call was refused. A refusal that

names the wrong tool's parameter (`wb_document_list` telling the caller

to pass `createWorkspace`) is a finding on the message and the fix is

cheap; a refusal that says the tool cannot take what the task needs

(`facets.tags` must match `{namespace}.{name}/v{n}`) is a gap in the

surface, and the fix is a parameter.

- **An errand that keeps counting past a refusal is a scoreboard that

lies.** The errand corpus's `call` throws on `isError` for this reason:

two of its four rows had measured a refused write and a crashed seam

as one cheap call each. A harness that drives real tools supplies every

seam the tools reach (`InMemoryVersionHistory`), or the tool answers an

error and the count reads as success.

- **`tools` says what the surface failed to offer — read the refusal

texts before blaming the description.** Three trials of "tag this note"

wandered through four tools, and the first reading was "the description

never says tags". The refusal text said the tool could not take a tag at

all: an errand step with no tool behind it (C5), which no description

fixes. When a task wanders, read what each refusal told the model, then

the description of the tool it should have reached, in that order.

- **A batch tool's line names the ARM the model picked.** `tools` prints

`wb_canvas_edit[region.set]` or `wb_canvas_edit[node.add]`, because the

tool name alone cannot say whether a declarative op was reached or the

model got the same outcome with the imperative one — and that is the

whole question a change to one arm's shape is judged on. The `--out`

JSON keeps each call's `inputs` beside it, so whether the model declared

geometry or left it to placement can be read rather than guessed.

- **A call AFTER the write is the lane grading the write's result.** The

group-contents task passed every trial, and after a description got the

model to leave geometry to placement, every trial then spent one more

`node.patch` moving the placed box — placement had been putting it on

top of what the group already held, since the op shipped. Read

`inputs` for what the extra call undid; a pass with a repair call in it

is a product finding, not a model one.

- **A description is a promise the refusal text has to keep.** `within`

said "the group grows to fit"; a node given `within` and a position past

the edge was still refused, and the lane hit that refusal three trials

of three — the model had done what the description said it could. When

a parameter's description names a behaviour, grep the refusals on that

path for the case the description covers.

- **Look at the boards, not only the verdicts.** A write task that

declares `boards: ['boards/x']` gets each named board rendered through

`wb_scene_render` into `<out>-boards/` beside `--out`; rasterise with

resvg from `packages/mcp-server` (it owns the binding) and read the

PNGs. The first look found two rendering bugs — a cropped group label

and a coloured group painted over its members — on drawings the grader

had passed, because the grader reads the store and a person reads the

picture. A layout task without its picture has been half-graded.

- **Read the `drawing` column before the picture.** Each board a write

task names is also scored by canvas-render's `scoreDrawing` (the

instrument `package-canvas-render.md` describes), and the run's line

prints every non-zero DEBT column by name — `overlaps 1 straddles 1

nearMisses 5` — with the full score under `drawing` in `--out`. A pass

with debt is a drawing the verifier accepted and a reader would not;

the column says which mistake, and `drawing-quality.test.ts`'s drafted

rows show what each looks like on a board. `--dry-run --out=<file>`

scores the fixture's own boards with no model call, which is how the

wiring is checked.

- **Every read costs ~69k input tokens even at two calls.** The table is

~8.7k of that; the rest is the CLI's own system prompt. That is why the

lane reports cost beside tokens, and why a change is judged on C1

rather than on the lane's token column.

- **pass^k, not pass@1, is the reliability number.** A task that passes

one trial in three is answerable and not reliably answered — the case

the pair exists to separate.

## Adding a task to the lane

Anthropic's mcp-builder rules:

- **Read tasks** carry a single stable `answer` compared as a string.

Phrase the question so it does not carry the target document's words

or any tool's name — a task that can be answered by matching a word in

a description measures the description, not the surface. The fixture

construction; add what the task needs there.

- **Write tasks** carry a `verify(wb, ids)` that reads the state back

through the real tools. Run `--dry-run` after adding one: the verifier

must FAIL on the untouched fixture, or it would pass a model that did

nothing. That is the mutation check, and the dry-run refuses to exit 0

without it.

- **One task per question the surface should answer**, held out from the

descriptions: refreshing tasks when descriptions are rewritten is part

of the description increment, and a description may not quote a task.

- **A task that fails every trial on a frontier model is first suspected

of being a broken task**, then of being a surface gap — the tag count

was the second, and probing the store is what told them apart.

## Adding a tool

Beside the four-place list and the smoke call that

`mcp-smoke-coverage.ts` documents, a new tool lands with:

- every input parameter `.describe()`d — C3 counts down, never up;

- a `TOOL_PROFILES` entry with the honest worst-op annotation (C6, C7);

- the Zod OBJECT registered, not its `.shape`, so `.strict()` reaches

the boundary and a stray key is refused by name (C10) — the way

`wb_body_edit` and `wb_workspace_edit` already do;

- a description that says when to reach for it over its neighbours (C4);

- its rung-1 row, an errand in `mcp-errand-corpus.ts` if it changes what

an errand costs, and a rung-3 task.

## Where the bytes go, and which cuts were free

Measured on the current table (ADR-0031 §3b): descriptions are ~30% of

what a model reads and are the useful bytes; the rest is schema, and the

schema carries three kinds of dead weight a `z.toJSONSchema` emits by

default. Check these before designing a bigger cut:

- **Safe-integer bounds.** `z.number().int()` emits `minimum:

-9007199254740991, maximum: 9007199254740991` on every field — sixty

times in one tool. Use the model's `integerSchema` /

`nonnegativeIntegerSchema` (`.int()` validation, `.meta()` strips the

bounds). `multipleOf(1)` is not the same integer: it accepts `1.4e-45`.

- **A stored union inlined per op arm.** The SDK round-trips a raw JSON

Schema through `z.fromJSONSchema` and emits `$ref`s inlined, so dedup is

impossible on the wire; it has to be the SOURCE schema. A flat

write-side object narrowed on parse (`nodeExtensionWriteSchema`) cut

the node extension from 487 bytes x8 to 290 x8, and made a broken embed

a refusal instead of a silent drop.

- **`$schema` and `additionalProperties: false`.** 55 and 28 bytes a tool.

The first is the SDK's and cannot be removed; the second is C10's price

and is kept.

Two cuts that look available and are NOT, each refuted by measurement

rather than by argument. Both would pass every gate this repo has.

- **Registering a DESCRIBED LEAF deletes its description.** Naming a

schema in zod's registry emits it into `$defs` once and `$ref`s it at

each site, which is where the 5,177 bytes the five existing

registrations save come from (measured by removing each one: `LineEnd`

-2,540 visible, `EdgeEnd` -1,382, `Bends` -609, `CanvasColor` -595,

`CanvasPoint` -51, `NodeEmbed` 0 — output schemas only). But a

description INSIDE a registered object survives (`EdgeEnd` keeps all

three of its) while a description ON the registered schema is dropped:

`$defs.BoxWidth` came back `{"type":"number","minimum":0}` with the

sentence gone. Registering the described `width`/`height` reads as -837

bytes and 4 arms x (60 + 150) is the whole of it — the bytes ARE the

descriptions. **Only a composite may be registered.**

- **`allOf` + `$ref` inheritance: CLOSED, and not on the axis first claimed.**

The four node arms spend 424 bytes each on the same shared base, 1,696

together, so extracting it looks worth ~1,100. Four things were measured,

in the order they were found, and each on its own settles it:

1. **It never reaches the table.** The published node draft is derived

`spatialNodeSchema.options` -> `.omit()` -> `.partial()` -> `.extend()`

-> `z.discriminatedUnion`. Every step requires a ZodObject and every

step FLATTENS, so making the stored schema an intersection changes the

table by nothing. Reaching it means building the write-side drafts as

intersections directly, which costs `discriminatedUnion` (refusal

quality), `NODE_DRAFT_KEYS`, and the free derivation the code comment

calls "the point".

2. **It does not compile or run.** An intersection is not an object:

`.omit()` is gone (four sites) and `z.infer` degrades to `{}` /

`unknown` across `canvas-edit.ts`. The server does not start.

3. **Spec-conformance and C10 cannot both hold.** ajv 8, 2020-12, four

shapes: flat + strict accepts valid input and refuses a stray key;

`allOf` with both halves strict REFUSES valid input; `allOf` with

neither strict accepts a stray key (C10 lost); only `allOf` +

`unevaluatedProperties` is correct, and zod 4.4.3 reads that keyword

without ever emitting it.

4. **The model axis was never the blocker.** The table already publishes

five `$ref`s and the round-14 reading was pass^3 1.0 / debtFree^3 1.0,

so `$ref` comprehension is established. Only `allOf` specifically is

unmeasured — and a rung-3 run would not move any of 1-3, which is why

it was not spent.

The first record of this said "publishes a schema that refuses every valid

call" and stopped there. That is true of the artifact and was the wrong

thing to lead with: ajv answers spec-conformance (defensibly — both MCP

SDKs bundle it, and `fromJsonSchema` uses it for raw-JSON-Schema tools),

and says nothing about C14. It also asserted "only a client validating

against the published schema breaks" with no client checked. Checked

since: this server validates with zod, and the MCP client SDK does not

validate outgoing arguments against `inputSchema` at all.

**Nothing else in the table is worth registering.** Re-scanned after the

node extension landed: every remaining repeated fragment is a described

leaf, which is the refuted shape above. The one exception is a

description-free record type (`{type: object, propertyNames, additional

Properties}`, four sites, ~99 bytes) — safe by the rule and 0.26% of the

table, so not worth an increment on its own.

**The same multiplication prices an ADDITION, and usually refuses it.**

Registration is attractive because a fragment repeats per arm; accommodation

is expensive for exactly that reason, and the arithmetic runs the other way.

Worked case: a lane trial wrote `stencil` INSIDE `node` — where every other

property of the box goes, and where a person thinks it belongs ("this box is

a database") — and lost the batch to the refusal. Accepting it where the

model already reaches looks like a one-field change. Measured on the

four-arm node draft, `node.add` alone:

| shape | visibleBytes | parameters | undescribed |

|---|---|---|---|

| refuse + redirect (ships today) | 38,300 | 345 | 221 |

| accept in `node`, described | 39,320 (**+1,020**) | 349 | 221 |

| accept in `node`, undescribed | 38,604 (**+304**) | 349 | 225 (**+4**) |

Neither shape moves C1, C13 or C3 DOWN, so neither clears §1's rule on its

own; the only column that could justify either is C13. **Its ceiling is

already known from the before-reading, which is why no rung-3 run was

spent**: the misplacement hit one task, one trial in three, and the trial

still PASSED — the refusal names the key, says it is not a node field, and

says where it goes, so the model repaired it in one call. The table is read

on every turn of every conversation; a ten-turn conversation pays the cheap

shape's +304 bytes ten times to save at most that one call.

The reason the trade is this lopsided is that **the expensive half of the

problem was already fixed by a change that cost nothing**. When the draft

STRIPPED, the identical mistake was accepted silently, the boxes were drawn

undressed, and one trial spent seventeen further calls rebuilding the

vocabulary by hand. Making the draft `.strict()` with a redirect turned that

into one refused call. What remained was a retry, and a retry is not worth

1,020 bytes a turn — nor 304.

**And when a field IS bought, its PLACE is most of its price.** The

classification write path (`facets` on node.add / node.patch, ADR-0036 §6)

was priced as two candidates before either was built: a generic

`extensionFacetsSchema` record at +600 visible / undescribed +0, a dedicated

`{axis, value}` field at +750 / +4 — both beside `op`, so emitted once per

op; the same one field inside the draft was +1,020. The cheaper on both C1

and C3 was taken and landed at +590 — and was WITHDRAWN one reading later

(round 18: 0 of 3, the field unused in fifty-five node ops), under a rule

written before the run. A field's cost is decided by how many times the

emitter repeats it, before a word of description is written; and a field

bought on a hypothesis is bought with its withdrawal rule attached.

**So: price the refusal before buying the accommodation.** A good error

message is the cheapest thing on this table — it costs bytes only when it

fires, while a field costs bytes on every turn forever. Reach for the schema

only when the refusal cannot name the repair.

## Traps, each paid for once

- **Run the lane from an EMPTY directory.** The `claude` CLI loads the

working directory's `CLAUDE.md`, which in this repo names the tools the

model is being tested on. The runner makes its own empty `cwd`; do not

point it at a checkout to "save a copy".

- **The seeding server and the model's server are different processes.**

Anything that survives only in memory is invisible to the model, and

the first run found exactly that: comment threads and proposals were

flattened to values by the workspace record's fold and came back

unreadable after a restart (fixed; `comment-threads.durability.test.ts`

is the shape of the test that crosses a reopen). A task that fails

every trial on a read the fixture plainly supports is first checked

for this: seed, close, reopen with `connectWhiteboard`, read.

- **`--only` is a substring over task names** — `--only=tag` matched two

tasks. Quote the whole name for one.

- **Bytes, not tokens, on rung 1.** A tokenizer would make the pin depend

on which model reads it. Divide by ~4 when quoting tokens, and say so.

- **Register the Zod OBJECT, never its `.shape`.** Handed a shape, the SDK

rebuilds a non-strict object around it and a misspelt optional parameter

is dropped with the call reporting success; `wb_facet_list`, every

parameter of which is optional, answered the unfiltered list to any

input at all. Every registration is the object now (C10), the rung-1

`strays` column is `refused` for all 18, and a test holds it by name.

The price was 29 visible bytes a tool for `additionalProperties: false`.