eval-forge SKILL.md skillA
eval-forge from this-hw/hiway-kit is an agent skill: a SKILL.md folder an AI agent loads when a task calls for it.
mdr add this-hw/hiway-kit/eval-forge@git:20261019.c7710ecPins git:20261019.c7710ec in mdr.lock. Pin the fingerprint instead.
- Audit grade
- A, 17 of 17 checks passed
- Kind
- SKILL.md skill
- Repository
- this-hw/hiway-kit
- Path
.claude/skills/eval-forge/SKILL.md- Versions recorded
- 2
- Latest version
- git:20261019.c7710ec, committed Oct 19, 2026
- First version recorded
- Sep 28, 2026
- Size
- 5,936 bytes
- Repository stars
- 3
- License
- MIT
Turn an observed agent defect into a machine-checkable eval scenario, generated and self-validated in one step. Use after a defect lands in the feedback ledger, or before changing an agent definition that has no eval coverage. Trigger with /eval-forge.
Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.
What the file says
# Eval Forge — 결함을 회귀 게이트로 굳히기 `/self-improve`의 HARD-GATE는 **eval 커버리지가 있는 대상에서만** 이중 게이트다. 커버리지가 없는 에이전트의 정의 파일 수정은 "사용자 승인 단일 게이트"로 퇴화한다. 스킬이 그 한계를 정직하게 고지하고는 있지만, **고지는 해결이 아니다.** 이 스킬은 커버리지를 늘리는 비용을 낮춰 그 구멍을 메운다. > **적용 범위**: **이 킷 레포 전용 프로젝트 스킬**이다(`.claude/skills/` — 플러그인에 실리지 > 않는다, v5.0.0). 아래 경로(`scripts/`·`evals/`)는 전부 레포 루트 기준이다. > `/self-improve`·`/native-watch`와 같은 킷 개발용 스킬 계열이다. ## 사용 시점 | 상황 | 왜 | | --- | --- | | ledger에 에이전트 행동 결함이 올라왔다 | 같은 결함이 다시 나면 기계가 잡게 만든다 | | eval 커버리지 없는 에이전트의 정의를 고치려 한다 | 고치기 **전에** 안전망을 깐다 (없으면 self-improve가 게이트 없이 돈다) | | 리뷰형 에이전트가 결함을 놓치는 걸 목격했다 | 거짓 음성은 eval의 1순위 표적이다 | ## 절차 ### 1. 대상과 실패 형태를 확정한다 [건너뛰기 금지] "무엇을 놓쳤는가"가 아니라 **"통과/실패를 무엇으로 판정할 것인가"**를 먼저 정한다. 판정 기준 없이 시나리오를 만들면 채점 불가능한 자산이 트리에 남는다. - 리뷰/스캔형 → 출력에 **반드시 등장할 표현**(OR 묶음)과 **등장하면 실패인 표현** - 수정/구현형 → `pytest_green` + 필요하면 `file_contains` ### 2. 픽스처를 준비한다 결함을 심은 최소 코드. 아래는 **강제 규칙**이다 (러너가 거부한다): - `conftest.py` 금지 — 채점 시 임의 코드 실행 통로 (있으면 생성기가 제거한다) - **모듈 스코프(=import 시 실행되는 위치)에서 위험 호출 금지.** 러너가 AST로 판정하며, 대상은 최상위 문장뿐 아니라 **클래스 본문·데코레이터 표현식·함수 기본 인자 값**까지다 (전부 import 시 실행된다). 함수/메서드 **본문**은 실행되지 않으므로 자유롭다 — 결함 코드는 함수 안에 두면 된다. …
Read the whole file at its exact version.
How to install
mdr add this-hw/hiway-kit/eval-forge@sha256:5e58de503e411290The line at the top of this page pins the label git:20261019.c7710ec, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.
[](https://markdownregistry.com/a/art_ljnpitpfusanwlpu)
2 badge image fetches in 30 days. What the badge shows.
Versions
The registry has recorded 2 versions of this file, between Sep 28, 2026 and Oct 19, 2026, about one every 21 days.
| version | committed | commit | size | audit | |
|---|---|---|---|---|---|
| git:20261019.c7710ec latest | 2026-10-19 | c7710ec | 5,936 B | A | view · diff |
| git:20260928.4c05332 | 2026-09-28 | 4c05332 | 5,909 B | A | view |
Audit of the latest version
- pass: Frontmatter block present
- pass: Frontmatter declares a name
- pass: Frontmatter declares a description
- pass: Size between 200 bytes and 200 KB (5936 bytes)
- pass: No zero-width or bidi control characters
- pass: No instruction hidden inside an HTML comment
- pass: No link to an exfiltration or paste host
- pass: No credential-shaped string
- pass: No instruction to send local credentials anywhere
- pass: No text hidden with inline styles
- pass: No prompt-injection phrasing
- pass: No curl or wget piped into a shell
- pass: No recursive delete of root, home or parent
- pass: No instruction to read or print local credentials
- pass: No base64 blob over 200 characters
- pass: No link to a raw IP address
- pass: No script tag
Source
this-hw/hiway-kit · 3 stars · license MIT · pushed 2026-10-20 · branch main
API
GET https://markdownregistry.com/api/v1/artifacts/art_ljnpitpfusanwlpu GET https://markdownregistry.com/api/v1/resolve?ref=this-hw/hiway-kit/eval-forge GET https://markdownregistry.com/api/v1/blob/5e58de503e4112900df58c5432887e352eae005f3231016e26ad1eac45fa4798
Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.
Guides: the SKILL.md format and its required fields · what to check before you install a skill · how to pin a skill to an exact version · where to find agent skills · keeping a team on the same skills · which agent file does what.
More from this-hw/hiway-kit
Every file in this-hw/hiway-kit