llms.txt@docs git:20261009.c6ace67A

Immutable. This exact content never changes and is served at /api/v1/blob/fac306ee6b836011.

# Driftproof

> A public instrument and public record measuring whether agent skills still
> deliver their claimed lift as models, providers and releases move underneath
> them. Every published number carries a dated, hash-verified receipt.

Driftproof runs a skill's own eval suite twice, with the skill and without it,
scores each answer several times so every result is a range rather than one
fragile number, and claims a verdict only when the two ranges do not overlap and
the move clears the effect floor. When it cannot stand behind a number it
publishes a refusal instead of a guess.

## Reports

- [Report 014: Claude Haiku 5.5 on release day, three skills](https://driftproofhq.com/reports/014/): On Claude Haiku 5.5 no skill clearly helped in two or more of three repeats. Models: Claude Haiku 4.5, Claude Haiku 5.5.
- [Report 013: Claude Sonnet 5.5, the day after its release](https://driftproofhq.com/reports/013/): With the skill, Sonnet 5.5 was never clearly higher or lower than Opus 5.5 in nine results (three tasks, three repeats); five of the nine had too few answers to tell. Models: Claude Opus 5.5, Claude Sonnet 5, Claude Sonnet 5.5.
- [Report 011: Claude Opus 5.5 on release day, three skills](https://driftproofhq.com/reports/011/): With the skill, the models showed no clear difference on code review, too few answers to tell on git and documentation. Models: Claude Opus 5, Claude Opus 5.5.
- [Report 010: repeated ADR evaluations across two Claude Code configurations](https://driftproofhq.com/reports/010/): Eight passed and two failed, both under the first setup, where one command gave both results. Models: Claude Opus 5.
- [Report 009: three skills under two eval harnesses](https://driftproofhq.com/reports/009/): Driftproof scored answers with the skill clearly higher on all three tasks. Models: Claude Opus 5.
- [Report 008: the skill stabilises the floor, not the ceiling](https://driftproofhq.com/reports/008/): With the skill, none of fourteen tasks scored clearly higher or clearly lower on the new model. Models: Claude Fable 5.1.
- [Report 007: the baseline is the noisy arm](https://driftproofhq.com/reports/007/): No pair was clearly higher or lower on average. Models: Claude Fable 5, Claude Sonnet 5.
- [Report 006: the skill text moves, the substrate holds still](https://driftproofhq.com/reports/006/): All three pairs came out not measured: on some tasks, answers without any skill scored clearly higher or clearly lower than in Report 005, so old and new text could not be compared. Models: Claude Fable 5, Claude Sonnet 5.
- [Report 005: what does a skill cost to run?](https://driftproofhq.com/reports/005/): In twenty of thirty skill and model pairs, at least one task scored clearly higher or lower with the skill. Models: Claude Fable 5, Claude Sonnet 5, GPT-5.6 Sol.
- [Report 004: does encoded expertise still lift output on the frontier tier?](https://driftproofhq.com/reports/004/): For three skills, each model had at least one task clearly higher with the skill and none clearly lower. Models: Claude Fable 5, Claude Opus 5.
- [Report 003: do skill verdicts hold across a model release?](https://driftproofhq.com/reports/003/): With the skill, four skills had at least one task clearly higher on the newer model, two had one task clearly lower, and four showed no clear difference. Models: Claude Opus 4.8, Claude Opus 5.
- [Report 002: does a skill's benefit hold across substrates?](https://driftproofhq.com/reports/002/): On three skills, both models had at least one task clearly higher with the skill, none clearly lower. Models: Claude Sonnet 5, GPT-5.6 Sol.
- [Report 001: do skill verdicts hold across a model release?](https://driftproofhq.com/reports/001/): With the skill, nine of ten skills had at least one task scoring clearly higher or clearly lower on the newer model. Models: Claude Sonnet 4.6, Claude Sonnet 5.

## Method and definitions

- [What is Driftproof?](https://driftproofhq.com/what-is-driftproof/): Driftproof (driftproofhq): open-source agent skill evaluation and regression testing across model releases.
- [How to tell whether a skill improved output](https://driftproofhq.com/agent-skill-evaluation/): How can I tell whether an agent skill actually improved output? How should I handle variance in agent evaluations?
- [Does a skill still help after a model upgrade?](https://driftproofhq.com/agent-skill-regression-testing/): Does a SKILL.md still help after upgrading Claude? How do I compare an agent skill across model releases?
- [What each evaluation tool measures](https://driftproofhq.com/compare/): What each tool measures, cell by cell, read from that tool's own docs or code and linked to where it says so.
- [The paper](https://driftproofhq.com/paper/): Reported, Not Measured: An Empirical Study of Measurement Defects in LLM and Agent Evaluation Tools. Version 1, 30 September 2026. Preprint, not peer reviewed.
- [Is this benchmark gap bigger than sampling noise?](https://driftproofhq.com/benchmark-gap/): type two benchmark scores and the question count, and see whether the gap clears the benchmark's own sampling noise.
- [Skill evaluation across model releases](https://driftproofhq.com/agent-skill-regression-testing/): whether an agent skill still helps after the model under it changes, answered from the published reports.
- [Evaluation receipts](https://driftproofhq.com/r/): every published receipt, dated and hash-verified, listed by the report it belongs to.
- [LLM eval measurement defects](https://driftproofhq.com/paper/): the paper on measurement defects in LLM and agent evaluation tools, with its data and its citation.
- [Maintainer](https://driftproofhq.com/maintainer/): who maintains Driftproof: Maverick (mavericksea-ai on GitHub), who also writes its reports and the paper.
- [Fixes merged into other evaluation tools](https://driftproofhq.com/what-is-driftproof/#fixes-merged-upstream): the fixes the maintainer contributed upstream, each credited in its project's own release notes.
- [AI benchmark gaps vs sampling noise](https://driftproofhq.com/leaderboard-noise/): a piece by Maverick that checks the gaps quoted in frontier launch posts and on public leaderboards against each benchmark's own sampling noise, with its code and data linked.
- [Methodology](https://driftproofhq.com/methodology/): how a run is scored, what a band is, and what the effect floor does.
- [Neutrality](https://driftproofhq.com/neutrality/): what Driftproof will not claim, and the limitations it discloses.
- [Glossary](https://driftproofhq.com/glossary/): drift report, receipt, band, effect floor, substrate, surface, refusal, cell, arm, and the verification lattice.
- [Report types](https://driftproofhq.com/report-types/): the seven kinds of report and what moves underneath the skill in each.
- [Receipt specification](https://github.com/driftproofhq/driftproof/blob/main/spec/RECEIPT.md): the receipt format, its schema versions, and the UNVERIFIED / DECLARED / TESTED / FORMAL levels.

## Optional

- [Interop](https://driftproofhq.com/interop/): importing results from other tools as DECLARED, and exporting a summary.
- [Authoring](https://driftproofhq.com/authoring/): writing a skill and an eval suite worth measuring.
- [Judge policy](https://driftproofhq.com/judge-policy/): which model judges, and why it is held fixed.
- [Findings](https://driftproofhq.com/findings/): what three external audits of 0.10.0 and 0.10.1 found, what is fixed, and what is still open.
- [Atom feed](https://driftproofhq.com/feed.xml)