# Driftproof

> A public instrument and public record measuring whether agent skills still
> deliver their claimed lift as models, providers and releases move underneath
> them. Every published number carries a dated, hash-verified receipt.

Driftproof runs a skill's own eval suite twice, with the skill and without it,
scores each answer several times so every result is a range rather than one
fragile number, and claims a verdict only when the two ranges do not overlap and
the move clears the effect floor. When it cannot stand behind a number it
publishes a refusal instead of a guess.

## Reports

- [Report 015: Skills.sh snapshot 2026-10-17: text tasks](https://driftproofhq.com/reports/015/): For the Skills.sh snapshot 2026-10-17: text tasks by Driftproof; Haiku 5.5 judge; eval account checked before/after runs, not during: Sonnet 5.5 12 helped, 4 hurt, 4 too few answers; Opus 5.5 12 helped, 2 hurt, 6 too few answers. Models: Claude Sonnet 5.5, Claude Opus 5.5. Driftproof wrote the text-task suites from the pinned skill descriptions and READMEs. Both model sets use Haiku 5.5 as judge. The eval account’s configuration was read and found clean before and after each run, not during it.
- [Report 014: Claude Haiku 5.5 on release day, three skills](https://driftproofhq.com/reports/014/): On Claude Haiku 5.5 no skill clearly helped in two or more of three repeats. Models: Claude Haiku 4.5, Claude Haiku 5.5.
- [Report 013: Claude Sonnet 5.5, the day after its release](https://driftproofhq.com/reports/013/): With the skill, Sonnet 5.5 was never clearly higher or lower than Opus 5.5 in nine results (three tasks, three repeats); five of the nine had too few answers to tell. Models: Claude Opus 5.5, Claude Sonnet 5, Claude Sonnet 5.5.
- [Report 011: Claude Opus 5.5, the day after its release, three skills](https://driftproofhq.com/reports/011/): With the skill, the models showed no clear difference on code review, too few answers to tell on git and documentation. Models: Claude Opus 5, Claude Opus 5.5.
- [Report 010: repeated ADR evaluations across two Claude Code configurations](https://driftproofhq.com/reports/010/): Eight passed and two failed, both under the first setup, where one command gave both results. Models: Claude Opus 5.
- [Report 009: three skills under two eval harnesses](https://driftproofhq.com/reports/009/): Driftproof scored answers with the skill clearly higher on all three tasks. Models: Claude Opus 5.
- [Report 008: the skill stabilises the floor, not the ceiling](https://driftproofhq.com/reports/008/): With the skill, none of fourteen tasks scored clearly higher or clearly lower on the new model. Models: Claude Fable 5.1.
- [Report 007: the baseline is the noisy arm](https://driftproofhq.com/reports/007/): No pair was clearly higher or lower on average. Models: Claude Fable 5, Claude Sonnet 5.
- [Report 006: the skill text moves, the substrate holds still](https://driftproofhq.com/reports/006/): All three pairs came out not measured: on some tasks, answers without any skill scored clearly higher or clearly lower than in Report 005, so old and new text could not be compared. Models: Claude Fable 5, Claude Sonnet 5.
- [Report 005: what does a skill cost to run?](https://driftproofhq.com/reports/005/): In twenty of thirty skill and model pairs, at least one task scored clearly higher or lower with the skill. Models: Claude Fable 5, Claude Sonnet 5, GPT-5.6 Sol.
- [Report 004: does encoded expertise still lift output on the frontier tier?](https://driftproofhq.com/reports/004/): For three skills, each model had at least one task clearly higher with the skill and none clearly lower. Models: Claude Fable 5, Claude Opus 5.
- [Report 003: do skill verdicts hold across a model release?](https://driftproofhq.com/reports/003/): With the skill, four skills had at least one task clearly higher on the newer model, two had one task clearly lower, and four showed no clear difference. Models: Claude Opus 4.8, Claude Opus 5.
- [Report 002: does a skill's benefit hold across substrates?](https://driftproofhq.com/reports/002/): On three skills, both models had at least one task clearly higher with the skill, none clearly lower. Models: Claude Sonnet 5, GPT-5.6 Sol.
- [Report 001: do skill verdicts hold across a model release?](https://driftproofhq.com/reports/001/): With the skill, nine of ten skills had at least one task scoring clearly higher or clearly lower on the newer model. Models: Claude Sonnet 4.6, Claude Sonnet 5.

## Method and definitions

- [What is Driftproof?](https://driftproofhq.com/what-is-driftproof/): Driftproof (driftproofhq): open-source agent skill evaluation and regression testing across model releases.
- [How to tell whether a skill improved output](https://driftproofhq.com/agent-skill-evaluation/): How can I tell whether an agent skill actually improved output? How should I handle variance in agent evaluations?
- [Does a skill still help after a model upgrade?](https://driftproofhq.com/agent-skill-regression-testing/): Does a SKILL.md still help after upgrading Claude? How do I compare an agent skill across model releases?
- [What each evaluation tool measures](https://driftproofhq.com/compare/): What each tool measures, cell by cell, read from that tool's own docs or code and linked to where it says so.
- [The paper](https://driftproofhq.com/paper/): Reported, Not Measured: An Empirical Study of Measurement Defects in LLM and Agent Evaluation Tools. Version 1, 30 September 2026. Preprint, not peer reviewed.
- [Is this benchmark gap bigger than sampling noise?](https://driftproofhq.com/benchmark-gap/): type two benchmark scores and the question count, and see whether the gap clears the benchmark's own sampling noise.
- [Skill evaluation across model releases](https://driftproofhq.com/agent-skill-regression-testing/): whether an agent skill still helps after the model under it changes, answered from the published reports.
- [Evaluation receipts](https://driftproofhq.com/r/): every published receipt, dated and hash-verified, listed by the report it belongs to.
- [LLM eval measurement defects](https://driftproofhq.com/paper/): the paper on measurement defects in LLM and agent evaluation tools, with its data and its citation.
- [Maintainer](https://driftproofhq.com/maintainer/): who maintains Driftproof: Maverick (mavericksea-ai on GitHub), who also writes its reports and the paper.
- [Fixes merged into other evaluation tools](https://driftproofhq.com/what-is-driftproof/#fixes-merged-upstream): the fixes the maintainer contributed upstream, each credited in its project's own release notes.
- [AI benchmark gaps vs sampling noise](https://driftproofhq.com/leaderboard-noise/): a piece by Maverick that checks the gaps quoted in frontier launch posts and on public leaderboards against each benchmark's own sampling noise, with its code and data linked.
- [Methodology](https://driftproofhq.com/methodology/): how a run is scored, what a band is, and what the effect floor does.
- [Neutrality](https://driftproofhq.com/neutrality/): what Driftproof will not claim, and the limitations it discloses.
- [Glossary](https://driftproofhq.com/glossary/): drift report, receipt, band, effect floor, substrate, surface, refusal, cell, arm, and the verification lattice.
- [Report types](https://driftproofhq.com/report-types/): the seven kinds of report and what moves underneath the skill in each.
- [Receipt specification](https://github.com/driftproofhq/driftproof/blob/main/spec/RECEIPT.md): the receipt format, its schema versions, and the UNVERIFIED / DECLARED / TESTED / FORMAL levels.

## Optional

- [Interop](https://driftproofhq.com/interop/): importing results from other tools as DECLARED, and exporting a summary.
- [Authoring](https://driftproofhq.com/authoring/): writing a skill and an eval suite worth measuring.
- [Judge policy](https://driftproofhq.com/judge-policy/): which model judges, and why it is held fixed.
- [Findings](https://driftproofhq.com/findings/): what three external audits of 0.10.0 and 0.10.1 found, what is fixed, and what is still open.
- [Atom feed](https://driftproofhq.com/feed.xml)