agent-evals SKILL.md skillB

agent-evals from sickn33/agentic-awesome-skills is an agent skill: a SKILL.md folder an AI agent loads when a task calls for it.

mdr add sickn33/agentic-awesome-skills/agent-evals@v1.0

Pins v1.0 in mdr.lock. Pin the fingerprint instead.

Audit grade

B, 16 of 17 checks passed

Kind

SKILL.md skill

Repository

sickn33/agentic-awesome-skills

Path

plugins/agentic-awesome-skills-claude/skills/agent-evals/SKILL.md

Versions recorded

2

Latest version

v1.0, committed Oct 14, 2026

First version recorded

Sep 20, 2026

Size

13,389 bytes

Repository stars

49,308

License

MIT

Build automated evaluation suites for AI agents using golden datasets,.

Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.

What the file says

# Agent Evals

Create repeatable checks so agent behavior improves safely over time.

## When to Use This Skill

Use this skill when:
- Shipping new agent features or changing prompts
- Adding CI gates for agent quality and safety
- Building regression suites for tool-calling agents
- Measuring LLM output quality at scale
- Validating RAG retrieval accuracy

## Prerequisites

- Python 3.10+
- An LLM API key (OpenAI, Anthropic, etc.)
- pytest or a custom eval harness; `pytest-timeout` is required for the shown `--timeout` options
- Optional: Braintrust, Promptfoo, or LangSmith account

## Evaluation contract

The following are application templates, not bundled executable tests. Supply the `agent` adapter, fixtures, supported provider/model IDs, reviewed dependencies and test configuration. Approve provider cost and data transmission; keep fixtures synthetic and keys private. Sandbox tool effects and inspect tool/network traces, not only response words. Keyword refusals and numerical judge scores do not prove safety or factual accuracy. Define thresholds and human review before evaluating.

## Evaluation Layers

### Unit Evals — Prompt-Level Correctness
…

Read the whole file at its exact version.

How to install

mdr add sickn33/agentic-awesome-skills/agent-evals@sha256:bc03648fc0d4d70f

[![mdr](https://markdownregistry.com/badge/art_gdpryqxxvzj5pgw5.svg)](https://markdownregistry.com/a/art_gdpryqxxvzj5pgw5)

Versions

The registry has recorded 2 versions of this file, between Sep 20, 2026 and Oct 14, 2026, about one every 24 days.

Audit of the latest version

Source

API

GET https://markdownregistry.com/api/v1/artifacts/art_gdpryqxxvzj5pgw5
GET https://markdownregistry.com/api/v1/resolve?ref=sickn33/agentic-awesome-skills/agent-evals
GET https://markdownregistry.com/api/v1/blob/bc03648fc0d4d70f5a10150397f7ecf60644b28dff7aace22e36f3128532cb4f

Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.

Guides: the SKILL.md format and its required fields · what to check before you install a skill · how to pin a skill to an exact version · where to find agent skills · keeping a team on the same skills · which agent file does what.

More from sickn33/agentic-awesome-skills

Every file in sickn33/agentic-awesome-skills

Other files named agent-evals

Browse by kind, by grade B, or by owner.