golden-dataset-builder SKILL.md skillA
golden-dataset-builder from pramoddutta/aitesterblueprint4x is an agent skill: a SKILL.md folder an AI agent loads when a task calls for it.
mdr add pramoddutta/aitesterblueprint4x/golden-dataset-builder@v1.0.0Pins v1.0.0 in mdr.lock. Pin the fingerprint instead.
- Audit grade
- A, 17 of 17 checks passed
- Kind
- SKILL.md skill
- Repository
- pramoddutta/aitesterblueprint4x
- Path
chapter_14_MCP_Create_VIBE/testskill-finder-mcp/skills/framework-packs/llm-eval-pack/golden-dataset-builder/SKILL.md- Versions recorded
- 1
- Latest version
- v1.0.0, committed Oct 10, 2026
- Size
- 3,711 bytes
- Repository stars
- 14
- License
- none declared
Builds and maintains a versioned golden evaluation dataset for an LLM feature. Use when an SDET or QA lead says "build a golden dataset", "we need eval data for the chatbot", "how many test questions do we need", or shares production logs or FAQs to turn into eval cases. Produces a draft record schema, labeling guidelines, coverage matrix, contamination checks, and a dataset card, for human labelers to review.
Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.
What the file says
# Golden Dataset Builder You build the dataset every eval depends on: **small, labeled with care, versioned, and honest about what it does not cover**. Bad golden data makes every metric lie. ## When to use - An LLM feature has no eval data, or only a handful of examples someone typed once. - Eval scores look great but users still hit failures the dataset never covered. - The dataset has grown ad hoc and nobody knows which version a score came from. ## Workflow 1. **Scope it.** Feature, intent taxonomy, and target cases per intent. Derive intents from logs with the team; ask rather than invent the taxonomy. 2. **Source cases.** Sampled production traffic (PII scrubbed, usage approved), SME-written cases, past incidents and bugs, and synthetic cases only to fill gaps (tagged as synthetic and human-reviewed). 3. **Write labeling guidelines.** What counts as correct, required facts, acceptable variants, and how to label unanswerable items. Double-label a subset, measure agreement, and adjudicate disagreements. 4. **Check coverage.** Build an intents x difficulty matrix (happy, edge, adversarial, …
Read the whole file at its exact version.
How to install
mdr add pramoddutta/aitesterblueprint4x/golden-dataset-builder@sha256:d4157ab807f60d23The line at the top of this page pins the label v1.0.0, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.
[](https://markdownregistry.com/a/art_774b2uwglsesyx3f)
2 badge image fetches in 30 days. What the badge shows.
Versions
The registry has recorded 1 version of this file, committed on Oct 10, 2026.
Audit of the latest version
- pass: Frontmatter block present
- pass: Frontmatter declares a name
- pass: Frontmatter declares a description
- pass: Size between 200 bytes and 200 KB (3711 bytes)
- pass: No zero-width or bidi control characters
- pass: No instruction hidden inside an HTML comment
- pass: No link to an exfiltration or paste host
- pass: No credential-shaped string
- pass: No instruction to send local credentials anywhere
- pass: No text hidden with inline styles
- pass: No prompt-injection phrasing
- pass: No curl or wget piped into a shell
- pass: No recursive delete of root, home or parent
- pass: No instruction to read or print local credentials
- pass: No base64 blob over 200 characters
- pass: No link to a raw IP address
- pass: No script tag
Source
pramoddutta/aitesterblueprint4x · 14 stars · license none declared · pushed 2026-10-11 · branch main
API
GET https://markdownregistry.com/api/v1/artifacts/art_774b2uwglsesyx3f GET https://markdownregistry.com/api/v1/resolve?ref=pramoddutta/aitesterblueprint4x/golden-dataset-builder GET https://markdownregistry.com/api/v1/blob/d4157ab807f60d235e3f26580b7459881f656e62b02ce191b724e837abbe49b1
Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.
Guides: the SKILL.md format and its required fields · what to check before you install a skill · how to pin a skill to an exact version · where to find agent skills · keeping a team on the same skills · which agent file does what.