eval-design SKILL.md skillB
eval-design from agentscope-ai/openjudge is an agent skill: a SKILL.md folder an AI agent loads when a task calls for it.
mdr add agentscope-ai/openjudge/eval-design@git:20260708.2151defPins git:20260708.2151def in mdr.lock. Pin the fingerprint instead.
- Audit grade
- B, 16 of 17 checks passed
- Kind
- SKILL.md skill
- Repository
- agentscope-ai/openjudge
- Path
skills/eval_pipeline/01-eval-design/SKILL.md- Versions recorded
- 1
- Latest version
- git:20260708.2151def, committed Jul 8, 2026
- Size
- 11,173 bytes
- Repository stars
- 1,020
- License
- Apache-2.0
Use when the user needs to design evaluation datasets, create test cases, stratify samples, generate adversarial examples, extract eval dimensions from traces/specs, or build a labeled evaluation set. Also use when the user mentions test data design, eval coverage, difficulty stratification, synthetic data generation for eval, or "how to create good evaluation data." Outputs datasets in OpenJudge-compatible format.
Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.
What the file says
# Eval Design Design high-quality evaluation datasets that measure what actually matters for your application. You extract evaluation dimensions from business context, structure them into stratified test cases, and output datasets ready for OpenJudge `GradingRunner`. ## When to Activate - User has agent traces / production logs and wants to build an eval set from them - User has evaluation principles but needs properly stratified test data - User wants to generate adversarial examples that stress-test their system - User needs coverage analysis — are they testing all the right things? - User wants a labeling guide for human annotators ## Checklist You MUST create a task for each item and complete them in order: 1. **Extract eval dimensions** — from traces, spec, or user interview 2. **Design stratified sampling** — 60/30/10 split with difficulty strata 3. **Generate test data** — synthetic inputs + adversarial examples 4. **Output OpenJudge dataset** — structured format ready for GradingRunner ## Coverage check: run the bundled script After you have a dataset, validate coverage with the bundled, tested script …
Read the whole file at its exact version.
How to install
mdr add agentscope-ai/openjudge/eval-design@sha256:f3ee57d53d1ae30dThe line at the top of this page pins the label git:20260708.2151def, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.
[](https://markdownregistry.com/a/art_pwbq64ncqrny6b5q)
5 badge image fetches in 30 days. What the badge shows.
Versions
The registry has recorded 1 version of this file, committed on Jul 8, 2026.
Audit of the latest version
- fail: No prompt-injection phrasing (matched: Ignore previous instructions)
- pass: Frontmatter block present
- pass: Frontmatter declares a name
- pass: Frontmatter declares a description
- pass: Size between 200 bytes and 200 KB (11173 bytes)
- pass: No zero-width or bidi control characters
- pass: No instruction hidden inside an HTML comment
- pass: No link to an exfiltration or paste host
- pass: No credential-shaped string
- pass: No instruction to send local credentials anywhere
- pass: No text hidden with inline styles
- pass: No curl or wget piped into a shell
- pass: No recursive delete of root, home or parent
- pass: No instruction to read or print local credentials
- pass: No base64 blob over 200 characters
- pass: No link to a raw IP address
- pass: No script tag
Starting a new one? Use a template for SKILL.md on mdpantry, which passes every one of these checks and says what to change in it.
Source
agentscope-ai/openjudge · 1,020 stars · license Apache-2.0 · pushed 2026-10-30 · branch main
API
GET https://markdownregistry.com/api/v1/artifacts/art_pwbq64ncqrny6b5q GET https://markdownregistry.com/api/v1/resolve?ref=agentscope-ai/openjudge/eval-design GET https://markdownregistry.com/api/v1/blob/f3ee57d53d1ae30d7234d7f89280fafab0306592e38bbbb127715b32596904be
Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.
Guides: the SKILL.md format and its required fields · what to check before you install a skill · how to pin a skill to an exact version · where to find agent skills · keeping a team on the same skills · which agent file does what.