agent-evals skillB
agent-evals is agent-read markdown (skill) from sickn33/agentic-awesome-skills: Build automated evaluation suites for AI agents using golden datasets,.
Indexed from public GitHub and served as immutable, content-addressed versions. Install it pinned to an exact SHA-256 with the mdr CLI, and every file is verified against the hash recorded here before it reaches your agent. The deterministic audit below grades the latest version, and the same file always earns the same grade.
What the file says
# Agent Evals
Create repeatable checks so agent behavior improves safely over time.
## When to Use This Skill
Use this skill when:
- Shipping new agent features or changing prompts
- Adding CI gates for agent quality and safety
- Building regression suites for tool-calling agents
- Measuring LLM output quality at scale
- Validating RAG retrieval accuracy
## Prerequisites
- Python 3.10+
- An LLM API key (OpenAI, Anthropic, etc.)
- pytest or a custom eval harness
- Optional: Braintrust, Promptfoo, or LangSmith account
## Evaluation Layers
### Unit Evals — Prompt-Level Correctness
Test individual prompt → response quality:
```python
# evals/test_unit.py
import json
import pytest
from agent import generate_response
CASES = json.load(open("evals/fixtures/unit_cases.json"))
@pytest.mark.parametrize("case", CASES, ids=lambda c: c["id"])
def test_prompt_correctness(case):
result = generate_response(case["prompt"], model=case.get("model", "default"))
# Exact match for structured output
if case.get("expected_json"):
assert json.loads(result) == case["expected_json"]
# Substring match for free-text
for keyword in case.get("must_contain", []):
…Read the whole file at its exact version.
How to install
mdr add sickn33/agentic-awesome-skills/agent-evals@v1.0mdr add sickn33/agentic-awesome-skills/agent-evals@sha256:f7b1131e7bd0c482Pin to a label to follow the author's releases, or to a sha256 to freeze the exact bytes forever. Either way the resolved hash is written to mdr.lock, and mdr install reproduces it on any machine.
[](https://markdownregistry.com/a/art_gdpryqxxvzj5pgw5)
1 badge views in 30 days
Versions
Audit of the latest version
- fail: No prompt-injection phrasing (matched: Ignore all previous instructions)
- pass: Frontmatter block present
- pass: Frontmatter declares a name
- pass: Frontmatter declares a description
- pass: Size between 200 bytes and 200 KB (12537 bytes)
- pass: No zero-width or bidi control characters
- pass: No instruction hidden inside an HTML comment
- pass: No link to an exfiltration or paste host
- pass: No credential-shaped string
- pass: No instruction to send local credentials anywhere
- pass: No text hidden with inline styles
- pass: No curl or wget piped into a shell
- pass: No recursive delete of root, home or parent
- pass: No instruction to read or print local credentials
- pass: No base64 blob over 200 characters
- pass: No link to a raw IP address
- pass: No script tag
Source
sickn33/agentic-awesome-skills · 46,848 stars · license MIT · pushed 2026-09-24 · branch main
API
GET https://markdownregistry.com/api/v1/artifacts/art_gdpryqxxvzj5pgw5 GET https://markdownregistry.com/api/v1/resolve?ref=sickn33/agentic-awesome-skills/agent-evals GET https://markdownregistry.com/api/v1/blob/f7b1131e7bd0c482eee1edb16da79d9a8b9a3c49fa5a71af4e0737f26e0b4914
Your agent does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.
More from sickn33/agentic-awesome-skills
Every file in sickn33/agentic-awesome-skills