decide · diff
git:20260920.cdfe831 to git:20261011.5cc7b5e
101 added, 125 removed. Audit A to A.
---
name: decide
- description: "Use when the user wants to try BlockRun's free typed-judgment endpoint (POST api.blockrun.ai/v1/decide, served by OpenJev) on their own data from Claude Code — yes/no, labelled choice, or scored-rung questions over a text or JSON state, up to 64 per call. Not an MCP tool: call it with curl from the shell. Covers the request shape, what the confidence number does and does not mean, and why OpenJev is not Jev."
+ description: "Use when the user wants typed judgments over their own data from Claude Code — yes/no probabilities, a labelled choice, or a rubric score over text or images, many items at a time. BlockRun's Decisions endpoint (OpenAI Decisions-compatible, gpt-6-luna): served only at POST api.blockrun.ai/v1/decisions, free with a BlockRun key (no x402 rail). Not an MCP tool: call it with curl. Replaces the OpenJev-backed /v1/decide, which is retired."
triggers:
- "decide"
+ - "decisions"
- "v1/decide"
+ - "v1/decisions"
+ - "decisions api"
- "typed judgment"
- "typed judgments"
- "openjev"
- - "open jev"
- - "jev"
- - "nli"
- - "natural language inference"
- - "entailment"
+ - "gpt-6-luna"
- "classify with a small model"
- "cheap classifier"
- "judgment model"
- "score these"
- "label these"
- "triage tickets"
---
- # Decide — free typed judgments from Claude Code
+ # Decisions — typed judgments from Claude Code
- `POST https://api.blockrun.ai/v1/decide` takes a **state** (a message, a ticket, a
- diff, a tool result, a row of data) and up to **64 questions**, and returns a typed
- answer with a number beside each one. No prose to parse. The backend is
- **OpenJev**, an open-source natural-language-inference model BlockRun hosts.
+ `POST https://api.blockrun.ai/v1/decisions` takes an **input** (a message, a ticket,
+ a diff, a tool result, an image) and a list of **questions** with a fixed answer
+ space, and returns one typed answer per question — no prose to parse. It is
+ compatible with OpenAI's Decisions API and served by `gpt-6-luna`.
It is **not a BlockRun MCP tool**, on purpose: you are already a frontier model,
and for a one-off "is this urgent?" you are the better judge. The endpoint earns
its place when the user wants the *same fixed ruler over many items*, or is
prototyping a judgment they will later run from a pipeline **without** a model.
From Claude Code, call it with `curl` from the shell.
- ## What it is not
+ ## Where to call it
- - **Not paid, not x402.** It is free behind a registered API key. There is no
- payment header, no wallet path, and it is served by `api.blockrun.ai`, not the
- x402 gateway at `blockrun.ai` (a POST to `blockrun.ai/v1/decide` is a 404).
- - **OpenJev is not Jev.** Jev is TypeSafe's model. OpenJev is an unaffiliated
- open-source NLI cross-encoder (MIT, published by AlexWortega on Hugging Face,
- Qwen3.5 4B base). It is not made by the people who make Jev and it is not a
- smaller or free tier of it. BlockRun does not sell or resell Jev. There is one
- backend, and the `x-blockrun-backend` response header names it on every call.
- - **No quality comparison exists.** BlockRun expects OpenJev to be materially
- weaker than Jev, has not benchmarked the two, and will not put a number on the
- gap. Do not invent one. The model authors' own zero-shot NLI figures, with
- attribution and a read date, are at <https://blockrun.ai/openjev>.
+ | Endpoint | Auth | Price |
+ |---|---|---|
+ | `https://api.blockrun.ai/v1/decisions` | `Authorization: Bearer brk_live_…` | Free for any registered key, per-key hourly limit |
+ That is the only rail. Decisions is **not sold over x402**: the x402 per-call
+ floor is above what a Decisions call costs, so a paid route would bill nearly
+ every call at the floor. Do not look for it on `blockrun.ai` or
+ `sol.blockrun.ai`, and do not offer to pay for it from the wallet.
+ Batch the questions anyway: many questions in one call share one input and one
+ round trip.
+
+ **OpenJev is retired** (2026-10-07). The old `POST api.blockrun.ai/v1/decide`
+ shape (`state`, `noul`/`choice`/`score` with `criteria`) is still accepted and
+ answered by the same model, but write new calls against `/v1/decisions`.
+
## Getting a key without changing how the MCP pays
Keys are minted at <https://user.blockrun.ai/dashboard/keys> (`brk_live_…`,
shown once; registration, not a card).
For experiments, **export the key in the shell** and leave the MCP server alone:
```bash
export BLOCKRUN_API_KEY=brk_live_…
```
- Do **not** write it to `~/.blockrun/.api-key` just to try `decide`. The MCP
- server reads that file at startup and a present key moves **every** paid tool
- from wallet mode to account billing — the same switch `BLOCKRUN_API_KEY` in the
- MCP server's own config makes. That is fine if the user wants account billing
- (see the `blockrun-setup` skill); it is a surprise if they only wanted a free
+ Do **not** write it to `~/.blockrun/.api-key` just to try this. The MCP server
+ reads that file at startup and a present key moves **every** paid tool from
+ wallet mode to account billing — the same switch `BLOCKRUN_API_KEY` in the MCP
+ server's own config makes. That is fine if the user wants account billing (see
+ the `blockrun-setup` skill); it is a surprise if they only wanted a free
judgment. If the MCP is already on account billing, the same key works for both.
## Request
```bash
- curl -sS -X POST https://api.blockrun.ai/v1/decide \
+ curl -sS https://api.blockrun.ai/v1/decisions \
-H "authorization: Bearer $BLOCKRUN_API_KEY" \
-H "content-type: application/json" \
-D /dev/stderr \
-d '{
- "state": "Help! My payouts have been failing for 3 days.",
- "questions": {
- "is_urgent": { "type": "noul", "instructions": "Does this convey urgency?" },
- "department": { "type": "choice", "instructions": "Which team should handle this?",
- "criteria": { "billing": "Payments, refunds",
- "technical": "Bugs, outages" } },
- "frustration": { "type": "score", "instructions": "How frustrated is the customer?",
- "criteria": ["Calm", "Frustrated", "Very angry"] }
- }
+ "model": "gpt-6-luna",
+ "input": "Help! My payouts have been failing for 3 days.",
+ "questions": [
+ { "type": "predicate", "name": "is_urgent",
+ "instructions": "Does the message convey urgency?" },
+ { "type": "choice", "name": "department",
+ "instructions": "Which team should handle this?",
+ "choices": [ { "value": "billing", "description": "Payments, payouts, refunds" },
+ { "value": "technical", "description": "Bugs and outages" } ] },
+ { "type": "score", "name": "frustration",
+ "instructions": "How frustrated is the customer?",
+ "levels": [ { "label": "calm", "description": "Neutral tone" },
+ { "label": "frustrated", "description": "Clearly unhappy" },
+ { "label": "angry", "description": "Hostile or threatening to leave" } ] }
+ ]
}'
```
- `-D /dev/stderr` shows the response headers (backend, rate-limit) without mixing
- them into the JSON on stdout.
+ `-D /dev/stderr` shows the response headers (rate limit) without mixing them
+ into the JSON on stdout.
| Field | Type | Required | Notes |
|---|---|---|---|
- | `state` | string \| object \| array | yes | What to judge. Text and JSON both work. If a fact matters, put it in the state — the model looks nothing up. |
- | `questions` | object | yes | Your own id → question. **1 to 64** per call. |
- | `model` | string | no | Defaults to `openjev`, the only backend. Leave it out. |
-
- Three question types, one operation underneath (state = premise, question =
- hypothesis, answer = how strongly the premise entails it):
+ | `model` | string | no | `gpt-6-luna`, the only Decisions model. |
+ | `input` | string \| array | yes | A string, or messages whose `content` holds text and inline base64 images (no hosted URLs). If a fact matters, put it in — the model looks nothing up. |
+ | `questions` | array | yes | Each `{ type, name, instructions }`, plus `choices` or `levels`. |
- | Type | `criteria` | You get back |
+ | Type | You add | You get back |
|---|---|---|
- | `noul` | none — it is true or false | one number for how strongly the state supports the claim |
- | `choice` | map of label → its meaning, **2 to 255** entries | the top label, plus a share per option |
- | `score` | array of rung descriptions in words, **2 to 255** | a weighted position across the rungs, plus the distribution |
+ | `predicate` | nothing | `probability` 0–1 |
+ | `choice` | `choices: [{ value, description }]` | `choice`, `probabilities` per value, `confidence` |
+ | `score` | `levels: [{ label, description }]`, lowest first | `score` (weighted level index from 0), `probabilities` per level, `confidence` |
- `instructions` — the question itself, in plain language — is required on all three.
+ A question the model declines comes back as `type: "refusal"`.
- ## Response
+ ## Response (measured 2026-10-07 for the request above)
```json
{
- "model": "openjev",
- "answers": {
- "is_urgent": { "type": "noul", "noul": 0.986 },
- "department": { "type": "choice", "choice": "billing",
- "probabilities": { "billing": 0.71, "technical": 0.29 },
- "confidence": 0.71 },
- "frustration": { "type": "score", "score": 1.42,
- "legend": { "0": "Calm", "1": "Frustrated", "2": "Very angry" },
- "probabilities": { "0": 0.11, "1": 0.36, "2": 0.53 },
- "confidence": 0.53 }
- }
+ "model": "gpt-6-luna",
+ "answers": [
+ { "type": "predicate", "name": "is_urgent", "probability": 0.95 },
+ { "type": "choice", "name": "department", "choice": "billing",
+ "probabilities": [ { "value": "billing", "probability": 1.0 },
+ { "value": "technical", "probability": 0.0 } ],
+ "confidence": 1.0 },
+ { "type": "score", "name": "frustration", "score": 0.94,
+ "probabilities": [ { "value": 0, "label": "calm", "probability": 0.06 },
+ { "value": 1, "label": "frustrated", "probability": 0.94 },
+ { "value": 2, "label": "angry", "probability": 0.0 } ],
+ "confidence": 0.91 }
+ ],
+ "usage": { "input_tokens": 399, "output_tokens": 0, "total_tokens": 399 }
}
```
- `score` is the probability-weighted rung index (`1.42` sits between
- "Frustrated" and "Very angry"); `legend` maps index → rung text.
-
- ## Read this before you build a threshold
-
- **The `confidence` on a `choice` is not the probability the answer is correct.**
-
- Every option is scored against the state, and the scores are then **divided by
- their total** so they sum to one. That step throws away how strong the scores
- were:
-
- - three options each scoring 0.1 — the model supporting **none** of them — come
- back as `0.33 / 0.33 / 0.33`;
- - two options scoring 0.9 and 0.85 — **both** strongly supported — come back as
- `0.51 / 0.49`.
-
- Opposite situations, identical response. The number is a **share of the
- agreement found** — an ordering with a margin. Wide means the model clearly
- preferred one option; narrow means it did not. It says nothing about whether
- *any* option fits, and nothing has been fitted against real outcomes the way a
- calibrated model's probabilities are. The same applies to `score`'s
- `probabilities` and `confidence`. A `noul` is the raw entailment score and is
- not normalised — but it is not calibrated either.
+ ## Before you build a threshold
Set thresholds by running the user's **own labelled examples** through the
- endpoint and looking at where the errors land, never by reading the number as a
- percentage. An agent that gates on `confidence > 0.8` as if it were accuracy
- will be wrong with no error to tell it so.
+ endpoint and looking at where the errors land — not by reading a number as an
+ accuracy. An agent that gates on `confidence > 0.8` without that check will be
+ wrong with no error to tell it so.
+ The model judges only the input. A question whose answer is not in it ("does
+ this need a reply today?" over a message that says nothing about timing) still
+ gets an answer that looks like a verdict. That one is policy — answer it in code.
+
## Trying it on the user's data — the shape that works
- 1. **Build the state** with everything the judgment needs: the message, the
- records it refers to, the policy that governs it. Fetch with `blockrun_exa`
- or `blockrun_search` first if a fact is missing; `decide` cannot look it up.
- 2. **Ask the independent questions together** in one call — one round trip,
- one rate-limit hit, up to 64 answers.
- 3. **Combine with deterministic checks** the code already has (amount, account
- age, "was a refund already issued").
+ 1. **Build the input** with everything the judgment needs. Fetch with
+ `blockrun_exa` or `blockrun_search` first if a fact is missing.
+ 2. **Ask the independent questions together** in one call.
+ 3. **Combine with deterministic checks** the code already has.
4. **Route by margin.** Act automatically where the margin is wide *and* the
- checks agree; hand narrow cases to a reasoning model — `blockrun_chat` with
- a `claude-*`, `o-`series or DeepSeek model — or to a person.
+ checks agree; hand narrow cases to a reasoning model via `blockrun_chat`, or
+ to a person.
- A quick evaluation loop from Claude Code: put 20–50 labelled rows in a JSON
- file, loop `curl` over them with `jq`, and tabulate agreement per question.
- That is the only number that should decide whether the endpoint is good enough
- for the user's task.
+ A quick evaluation loop: put 20–50 labelled rows in a JSON file, loop `curl`
+ over them with `jq`, and tabulate agreement.
```bash
jq -c '.[]' examples.json | while read -r row; do
- state=$(jq -c '.state' <<<"$row"); want=$(jq -r '.label' <<<"$row")
- got=$(curl -sS -X POST https://api.blockrun.ai/v1/decide \
+ input=$(jq -c '.input' <<<"$row"); want=$(jq -r '.label' <<<"$row")
+ got=$(curl -sS https://api.blockrun.ai/v1/decisions \
-H "authorization: Bearer $BLOCKRUN_API_KEY" -H "content-type: application/json" \
- -d "{\"state\": $state, \"questions\": {\"q\": {\"type\": \"choice\",
- \"instructions\": \"Which team should handle this?\",
- \"criteria\": {\"billing\": \"Payments, refunds\", \"technical\": \"Bugs, outages\"}}}}" \
- | jq -r '.answers.q.choice')
+ -d "{\"model\": \"gpt-6-luna\", \"input\": $input, \"questions\": [{\"type\": \"choice\",
+ \"name\": \"q\", \"instructions\": \"Which team should handle this?\",
+ \"choices\": [{\"value\": \"billing\", \"description\": \"Payments, refunds\"},
+ {\"value\": \"technical\", \"description\": \"Bugs, outages\"}]}]}" \
+ | jq -r '.answers[0].choice')
echo "$want,$got"
done | sort | uniq -c
```
- ## What it does not do
-
- It does not write, summarise, or explain — there is no free-text field in the
- response. It has no memory between calls. Image input exists in the model's v2
- checkpoint but the endpoint does not expose it.
-
## Limits and errors
- | Status | Meaning | What to do |
+ | Status | Meaning | Charged? |
|---|---|---|
- | 400 | Body did not match the schema; the response names the field | Fix the field. Common: a `choice` with `criteria` as an array, a `score` with a map, fewer than 2 criteria, more than 64 questions. |
- | 401 | Missing or invalid key | `BLOCKRUN_API_KEY` is unset in this shell, or wrong. Check <https://user.blockrun.ai/dashboard/keys>. |
- | 429 | Per-key hourly limit reached | Wait for the interval the response carries (`retry-after`). The limit is enforced server-side and reported in the response — read it there; do not assume a number. |
-
- None of these cost anything; nothing here ever does.
+ | 400 | Body did not match the schema; the error names the field | No |
+ | 401 | Missing or invalid key | No |
+ | 429 | Per-key hourly limit; wait for `retry-after` | No |
+ | 502 / 504 | Model unreachable or timed out | No |
## Related
- - API reference: <https://blockrun.ai/docs/api-reference/decide>
- - OpenJev, with the authors' published figures: <https://blockrun.ai/openjev>
- - Account billing for the whole MCP (if the user wants the key to pay for the
- paid tools too): `blockrun-setup` skill, "API key" section.
+ - API reference: <https://blockrun.ai/docs/api-reference/decisions>
+ - Account billing for the whole MCP: `blockrun-setup` skill, "API key" section.