aperivue/medsci-skills · skills/design-ai-benchmarking/SKILL.md
design-ai-benchmarking skillA
Design and validity review for studies that benchmark one or more AI systems against a human-expert panel as the reference. Covers the evaluation question and arm definition, decoupled multi-dimensional rubrics with anchors, planted calibration probes, reviewer-panel construction, inter-rater reliability targets, LLM-as-judge versus human-as-judge adjudication, construct-independence guards, and a structured rating-export schema. Use before data collection on an AI-vs-expert evaluation.
Latest version
mdr add aperivue/medsci-skills/design-ai-benchmarking@git:20260625.92ee046Exact content
mdr add aperivue/medsci-skills/design-ai-benchmarking@sha256:b8f794a1f6c800d8Badge
[](https://markdownregistry.com/a/art_6jwjb34n77rpx3ix)
0 badge views in 30 days
Versions
| version | committed | commit | size | audit | |
|---|---|---|---|---|---|
| git:20260625.92ee046 latest | 2026-06-25 | 92ee046 | 12,094 B | A | view · diff |
| git:20260605.041597a | 2026-06-05 | 041597a | 10,820 B | A | view |
Audit of the latest version
A 17 of 17 checks passed. Deterministic, no model, same answer every run.
- pass: Frontmatter block present
- pass: Frontmatter declares a name
- pass: Frontmatter declares a description
- pass: Size between 200 bytes and 200 KB (12094 bytes)
- pass: No zero-width or bidi control characters
- pass: No instruction hidden inside an HTML comment
- pass: No link to an exfiltration or paste host
- pass: No credential-shaped string
- pass: No instruction to send local credentials anywhere
- pass: No text hidden with inline styles
- pass: No prompt-injection phrasing
- pass: No curl or wget piped into a shell
- pass: No recursive delete of root, home or parent
- pass: No instruction to read or print local credentials
- pass: No base64 blob over 200 characters
- pass: No link to a raw IP address
- pass: No script tag
Source
GitHub
aperivue/medsci-skills · 283 stars · license MIT · pushed 2026-09-05 · branch main
API
GET https://markdownregistry.com/api/v1/artifacts/art_6jwjb34n77rpx3ix GET https://markdownregistry.com/api/v1/resolve?ref=aperivue/medsci-skills/design-ai-benchmarking GET https://markdownregistry.com/api/v1/blob/b8f794a1f6c800d821305a4df8a797bea61cf34a602e0dc0dbea8f2c0c458ca5