read-validation · git:20261117.82b2487 · 2026-11-17 · sha256 e19bd7b00a78dc8e
read-validation git:20261117.82b2487A
Immutable. This exact content never changes and is served at /api/v1/blob/e19bd7b00a78dc8e.
No tool to install first
claude plugin marketplace add https://markdownregistry.com/mp/a/art_3h2ixuv2doe7cjvj/e19bd7b00a78dc8e22bee8bdb3ed535fcf4b8424fb118a8aa802d0a01b528789.json
Then /plugin install read-validation@markdownregistry-3h2ixuv2doe7cjvj in Claude Code. It installs this exact version, the one graded A here, pinned to the commit it was graded at, and leaves it there.
---
name: read-validation
description: Read a AKLR validation result, local or from CI, and decide what it means for a kernel - which Points pass, which fail and why, whether a failure is the kernel or the measurement, and which Points to keep. Use after `python -m aklr validate` or a failed CI `aiter-test`, `sglang-test` or `vllm-test` job.
---
# Read a AKLR Validation Result
A verdict is per Point. Before changing a kernel or its workloads, read the rounds behind each
failing Point in the report: the log names the failing schema, and only the report names the Point.
## Required Context
Read [references/ruler.md](references/ruler.md) (how a Point is timed and judged),
[references/ci.md](references/ci.md) (what CI validates and where its report is) and
`aklr/ops/README.md`.
## Procedure
1. **Get the report.** Locally, `python -m aklr validate --base origin/main --stages audit,bench
<schemas> --out <dir>` writes `<dir>/report.md` and `report.json`. From CI, download the run's
artifact `validation-report-aiter-test-<attempt>` (`aklr/ops`), `validation-report-sglang-test-<attempt>`
(sglang's plugins) or `validation-report-vllm-test-<attempt>` (vLLM's); the `aiter-test`, `sglang-test` and
`vllm-test` jobs' `Validate` step logs end with the findings, and `Enforce the overall CI result` only names the job that failed.
2. **Separate findings from demotions.** A run fails on its `findings` (`not_faster`, `regressed`,
accuracy, build, crash). `demotions` are committed kernels' Points that measured too slow and stop
being served; they fail nothing. A committed Point that newly computes wrong is a demotion and a
finding at once.
3. **Read the failing Point's rounds.** In `report.json`, the bench stage's `workload_records` entry
with that `schema_path` and `axes` holds `timings[]`: per-round `candidate_milliseconds` and
`versus_milliseconds`, `ratio_lower_bound`, `log_ratio_mean`, `min_ratio`. Compare the medians, then
look for rounds far above either side's median.
4. **Classify the failure.**
- **Spikes**: a few rounds of one side well above its median, the medians themselves apart, and
spikes on either side elsewhere in the run: the node, not the kernel. The median ratio is the
kernel's.
- **Floor**: both medians near the graph-node floor (~4 µs): the Point's gain is a fixed latency
difference that does not carry across nodes (ruler.md).
- **Real**: the medians are close or inverted.
5. **Decide per Point** (thresholds in ruler.md). Keep a Point whose lower bound is at least the
dispatch threshold, 1.05, with margin; one under 1.05 can merge when another Point of the file
qualifies, but is never served. Remove only Points the report shows failing or under 1.05, never others on a guess. Rerun a spike failure
before removing a Point its medians support.
6. **Change and rerun on the new head.** Removing a workload renames the schema (its uuid hashes the
workloads); the kernel's sources and content hash stay.
## Boundary
Read-only unless the invoking user asks for the change. Do not request GPU nodes or rerun CI on
their behalf.
## Completion
For each failing Point: the finding, its lower bound, both medians, the spike rounds if any, the
classification, and the change proposed with its evidence.