Home / hashgraph-online / awesome-codex-plugins · GitHub

eval-harness SKILL.md skillA

eval-harness from hashgraph-online/awesome-codex-plugins is an agent skill: a SKILL.md folder an AI agent loads when a task calls for it.

Install it, pinned

mdr add hashgraph-online/awesome-codex-plugins/eval-harness@git:20260512.6523792

Pins git:20260512.6523792 in mdr.lock. Pin the fingerprint instead.

Audit grade: A, 17 of 17 checks passed
Kind: SKILL.md skill
Repository: hashgraph-online/awesome-codex-plugins
Path: plugins/Colin4k1024/tsp/skills/eval-harness/SKILL.md
Versions recorded: 1
Latest version: git:20260512.6523792, committed May 12, 2026
Size: 4,285 bytes
Repository stars: 1,537
License: Apache-2.0

正式评估框架，实现 eval-driven development (EDD) 原则。用于定义 pass/fail 标准、测量 pass@k 指标、创建回归测试套件。.

Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.

What the file says

# Eval Harness Skill

一个用于 Claude Code 会话的正式评估框架，实现 eval-driven development (EDD) 原则。

## 何时激活

- 为 AI 辅助工作流设置 EDD
- 定义 Claude Code 任务完成的 pass/fail 标准
- 使用 pass@k 指标测量 agent 可靠性
- 为 prompt 或 agent 更改创建回归测试套件
- 跨模型版本基准测试 agent 性能

## 理念

Eval-Driven Development 将评估视为"AI 开发的单元测试":
- **在实现前**定义预期行为
- 在开发期间**持续运行**评估
- 用每次变更**追踪回归**
- 使用 pass@k 指标**测量可靠性**

## 评估类型

### Capability Evals
测试 Claude 能否做以前不能做的事:
```markdown
[CAPABILITY EVAL: points-calculation]
Task: 计算用户积分并确定等级
Success Criteria:
  - [ ] 积分正确累加
  - [ ] 等级边界正确
  - [ ] 权益解锁逻辑正确
Expected Output: 用户总积分 = 1500，等级 = L3
```

### Regression Evals
确保变更不破坏现有功能:
```markdown
[REGRESSION EVAL: login-flow]
Baseline: sha-abc123
Tests:
  - existing-login: PASS
  - session-management: PASS
  - logout-flow: PASS
Result: 3/3 passed (previously 3/3)
```

## Grader 类型

### 1. Code-Based Grader
使用代码的确定性检查:
```bash
# 检查文件是否包含预期模式
grep -q "export function handlePoints" src/points.ts && echo "PASS" || echo "FAIL"

# 检查测试是否通过
npm test -- --testPathPattern="points" && echo "PASS" || echo "FAIL"
```

### 2. Model-Based Grader
使用 Claude 评估开放式输出:
```markdown
[MODEL GRADER PROMPT]
评估以下代码变更：
1. 它是否解决了陈述的问题？
2. 结构是否良好？
3. 边界情况是否处理？
4. 错误处理是否适当？
…

Read the whole file at its exact version.

How to install

The exact contents, by fingerprint

mdr add hashgraph-online/awesome-codex-plugins/eval-harness@sha256:bdec481c9fef36b4

The line at the top of this page pins the label git:20260512.6523792, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.

Badge

[![mdr](https://markdownregistry.com/badge/art_62yh4ynv5eaix7vn.svg)](https://markdownregistry.com/a/art_62yh4ynv5eaix7vn)

5 badge image fetches in 30 days. What the badge shows.

Versions

The registry has recorded 1 version of this file, committed on May 12, 2026.

version	committed	commit	size	audit
git:20260512.6523792 latest	2026-05-12	6523792	4,285 B	A	view

Audit of the latest version

A 17 of 17 checks passed. Fixed automatic checks with no AI involved, so the same checks give the same answer every run.

pass: Frontmatter block present
pass: Frontmatter declares a name
pass: Frontmatter declares a description
pass: Size between 200 bytes and 200 KB (4285 bytes)
pass: No zero-width or bidi control characters
pass: No instruction hidden inside an HTML comment
pass: No link to an exfiltration or paste host
pass: No credential-shaped string
pass: No instruction to send local credentials anywhere
pass: No text hidden with inline styles
pass: No prompt-injection phrasing
pass: No curl or wget piped into a shell
pass: No recursive delete of root, home or parent
pass: No instruction to read or print local credentials
pass: No base64 blob over 200 characters
pass: No link to a raw IP address
pass: No script tag

Source

GitHub

hashgraph-online/awesome-codex-plugins · 1,537 stars · license Apache-2.0 · pushed 2026-11-08 · branch main

API

GET https://markdownregistry.com/api/v1/artifacts/art_62yh4ynv5eaix7vn
GET https://markdownregistry.com/api/v1/resolve?ref=hashgraph-online/awesome-codex-plugins/eval-harness
GET https://markdownregistry.com/api/v1/blob/bdec481c9fef36b46239c43d49d3e47b480511c861e18e49dc4cdb56d55415ab

Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.

Guides: the SKILL.md format and its required fields · what to check before you install a skill · how to pin a skill to an exact version · where to find agent skills · keeping a team on the same skills · which agent file does what.

More from hashgraph-online/awesome-codex-plugins

list-my-plugin SKILL.md skill

hashgraph-online/awesome-codex-plugins · .agents/skills/list-my-plugin/SKILL.md · Add a Codex plugin to the awesome-codex-plugins README as one correctly placed line, check it locally, and finish with…

git:20261105.cfef970 · audit A · 1,537 stars

AGENTS.md AGENTS.md file

hashgraph-online/awesome-codex-plugins · AGENTS.md

git:20261105.dcc7950 · audit A · 1,537 stars

ontoly-software-graph SKILL.md skill

hashgraph-online/awesome-codex-plugins · plugins/0xsarwagya/ontoly-codex-plugin/skills/ontoly-software-graph/SKILL.md · Use Ontoly Software Graph evidence, CLI reports, MCP, and official Ontoly Agent Skills before searching source files…

git:20260714.2040a1b · audit A · 1,537 stars

agent-debug-fixer SKILL.md skill

hashgraph-online/awesome-codex-plugins · plugins/1139030773-cmd/agent-workflow-system/skills/agent-debug-fixer/SKILL.md · 中文调试修复技能。用于报错、测试失败、页面异常、功能不符合预期、需要定位根因并做最小修复时。触发语包括"进入调试模式""帮我修问题""报错了""测试失败""页面坏了""找根因"。