ai-product-eval-ops SKILL.md skillA
ai-product-eval-ops from gastonchevarria/god-mode is an agent skill: a SKILL.md folder an AI agent loads when a task calls for it.
mdr add gastonchevarria/god-mode/ai-product-eval-ops@git:20261014.522e28aPins git:20261014.522e28a in mdr.lock. Pin the fingerprint instead.
- Audit grade
- A, 17 of 17 checks passed
- Kind
- SKILL.md skill
- Repository
- gastonchevarria/god-mode
- Path
plugins/god-mode-dev/skills/ai-product-eval-ops/SKILL.md- Versions recorded
- 2
- Latest version
- git:20261014.522e28a, committed Oct 14, 2026
- First version recorded
- Sep 26, 2026
- Size
- 1,488 bytes
- Repository stars
- 3
- License
- MIT
Sets up continuous LLM evals: ground-truth datasets, hallucination metrics, latency and cost profiling, regression tests. Use for 'set up evals' or 'did this prompt change make it worse'.
Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.
What the file says
# AI Product & Evaluation Ops ## Overview Software with non-deterministic LLMs requires automated evaluation (Evals) to measure accuracy, catch regressions when updating prompts/models, and protect against hallucinations. `ai-product-eval-ops` builds rigorous testing pipelines for AI features. ## Evaluation Framework ### 1. Golden Dataset Creation - Maintain a curated suite of 50-200 representative input-output test cases in JSONL format. - Include edge cases: malformed user inputs, adversarial prompts, multi-language requests, empty documents. ### 2. Evaluation Metrics Matrix - **Exact Match / Schema Validity**: 100% required for structured JSON extraction. - **Semantic Similarity / Cosine Distance**: Measure fidelity against ground-truth answers (> 0.88 threshold). - **LLM-as-a-Judge Evaluation**: Use a frontier model to score outputs on: 1. *Faithfulness / Groundedness* (Is every claim backed by retrieved context?) 2. *Answer Relevance* (Did it answer the actual question?) 3. *Completeness* (Did it omit critical details?) ### 3. CI/CD Integration - Run evaluation suites on pull requests when prompts, model parameters, or retrieval logic are modified. …
Read the whole file at its exact version.
How to install
mdr add gastonchevarria/god-mode/ai-product-eval-ops@sha256:f1ff7ca54a7a3202The line at the top of this page pins the label git:20261014.522e28a, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.
[](https://markdownregistry.com/a/art_7d7e2nt4khj7s5ai)
3 badge image fetches in 30 days. What the badge shows.
Versions
The registry has recorded 2 versions of this file, between Sep 26, 2026 and Oct 14, 2026, about one every 19 days.
| version | committed | commit | size | audit | |
|---|---|---|---|---|---|
| git:20261014.522e28a latest | 2026-10-14 | 522e28a | 1,488 B | A | view · diff |
| git:20260926.941aa31 | 2026-09-26 | 941aa31 | 1,486 B | A | view |
Audit of the latest version
- pass: Frontmatter block present
- pass: Frontmatter declares a name
- pass: Frontmatter declares a description
- pass: Size between 200 bytes and 200 KB (1488 bytes)
- pass: No zero-width or bidi control characters
- pass: No instruction hidden inside an HTML comment
- pass: No link to an exfiltration or paste host
- pass: No credential-shaped string
- pass: No instruction to send local credentials anywhere
- pass: No text hidden with inline styles
- pass: No prompt-injection phrasing
- pass: No curl or wget piped into a shell
- pass: No recursive delete of root, home or parent
- pass: No instruction to read or print local credentials
- pass: No base64 blob over 200 characters
- pass: No link to a raw IP address
- pass: No script tag
Source
gastonchevarria/god-mode · 3 stars · license MIT · pushed 2026-10-16 · branch main
API
GET https://markdownregistry.com/api/v1/artifacts/art_7d7e2nt4khj7s5ai GET https://markdownregistry.com/api/v1/resolve?ref=gastonchevarria/god-mode/ai-product-eval-ops GET https://markdownregistry.com/api/v1/blob/f1ff7ca54a7a3202234cf9e242345b8698cd89b89716687642f00152daec3256
Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.
Guides: the SKILL.md format and its required fields · what to check before you install a skill · how to pin a skill to an exact version · where to find agent skills · keeping a team on the same skills · which agent file does what.