eval-authoring SKILL.md skillA
eval-authoring from sigistry/marketplace is an agent skill: a SKILL.md folder an AI agent loads when a task calls for it.
mdr add sigistry/marketplace/eval-authoring@git:20260826.571bc2fPins git:20260826.571bc2f in mdr.lock. Pin the fingerprint instead.
- Audit grade
- A, 17 of 17 checks passed
- Kind
- SKILL.md skill
- Repository
- sigistry/marketplace
- Path
plugins/llm-app-hardener/skills/eval-authoring/SKILL.md- Versions recorded
- 2
- Latest version
- git:20260826.571bc2f, committed Aug 26, 2026
- First version recorded
- Jul 15, 2026
- Size
- 4,141 bytes
- Repository stars
- 3
- License
- MIT
This skill should be used when the user mentions "llm eval", "evaluation", "promptfoo", "deepeval", "regression test", "llm judge", "golden dataset", "eval suite", "test a prompt", or wants to prove a prompt/model change improved rather than regressed behavior. It provides a standardized methodology for authoring assertion-based, LLM-as-judge, and golden-dataset eval suites that gate CI.
Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.
What the file says
# Eval Authoring ## Purpose Give LLM apps a repeatable way to escape "vibes-based" development, where a prompt is changed, a few outputs are eyeballed, and the change ships with no evidence it helped. An eval suite turns that into a measured, versioned, CI-gated check: every prompt or model change is scored against fixed cases, and a regression fails the build. This skill standardizes what to test, how to grade it, and how to wire it in, riding an existing framework (promptfoo, DeepEval, Ragas) when one is present, and falling back to plain unit tests plus an LLM judge when none is. ## The three grader types | Grader | Use for | Cost / stability | Example | |---|---|---|---| | **Assertion** | Deterministic, checkable facts about the output | Cheap, stable, run every commit | `is-json`, `json-schema`, `contains`, `regex`, `not-contains` (banned strings), latency/cost budget | | **LLM-as-judge** | Qualities assertions can't capture | Costs a call, needs a threshold | Faithfulness, helpfulness, tone, "answers the question", pairwise vs. baseline | …
Read the whole file at its exact version.
How to install
mdr add sigistry/marketplace/eval-authoring@sha256:9c2cff8ed35894f6The line at the top of this page pins the label git:20260826.571bc2f, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.
[](https://markdownregistry.com/a/art_op433r4u67zun6xs)
3 badge image fetches in 30 days. What the badge shows.
Versions
The registry has recorded 2 versions of this file, between Jul 15, 2026 and Aug 26, 2026, about one every 42 days.
| version | committed | commit | size | audit | |
|---|---|---|---|---|---|
| git:20260826.571bc2f latest | 2026-08-26 | 571bc2f | 4,141 B | A | view · diff |
| git:20260715.6169119 | 2026-07-15 | 6169119 | 4,141 B | A | view |
Audit of the latest version
- pass: Frontmatter block present
- pass: Frontmatter declares a name
- pass: Frontmatter declares a description
- pass: Size between 200 bytes and 200 KB (4141 bytes)
- pass: No zero-width or bidi control characters
- pass: No instruction hidden inside an HTML comment
- pass: No link to an exfiltration or paste host
- pass: No credential-shaped string
- pass: No instruction to send local credentials anywhere
- pass: No text hidden with inline styles
- pass: No prompt-injection phrasing
- pass: No curl or wget piped into a shell
- pass: No recursive delete of root, home or parent
- pass: No instruction to read or print local credentials
- pass: No base64 blob over 200 characters
- pass: No link to a raw IP address
- pass: No script tag
Source
sigistry/marketplace · 3 stars · license MIT · pushed 2026-11-06 · branch main
API
GET https://markdownregistry.com/api/v1/artifacts/art_op433r4u67zun6xs GET https://markdownregistry.com/api/v1/resolve?ref=sigistry/marketplace/eval-authoring GET https://markdownregistry.com/api/v1/blob/9c2cff8ed35894f6bda353e75cf21d8064704bf01b0f64b3c1e02c7e790813a0
Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.
Guides: the SKILL.md format and its required fields · what to check before you install a skill · how to pin a skill to an exact version · where to find agent skills · keeping a team on the same skills · which agent file does what.