Home / reidemeister94 / development-skills · skills/ai-agent-bench/SKILL.md · GitHub

ai-agent-bench skillA

Name: ai-agent-bench
Author: reidemeister94

ai-agent-bench is agent-read markdown (skill) from reidemeister94/development-skills: Compare Claude Code and Codex on the same code task, with isolated worktrees, common checks, time, and cost..

Indexed from public GitHub and served as immutable, content-addressed versions. Install it pinned to an exact SHA-256 with the mdr CLI, and every file is verified against the hash recorded here before it reaches your agent. The deterministic audit below grades the latest version, and the same file always earns the same grade.

What the file says

# AI agent bench

Compare agents only with the same task, starting commit, and outcome check. The harness preserves result branches and removes temporary worktrees.

Create `<repo>/.agent-bench.toml`:

```toml
prompt = "prompts/task.md"
start_branch = "main"              # or start_commit
agents = ["claude", "codex"]
outer_check = "./scripts/full_check.sh"
inner_check = "uv run pytest tests/integration/test_x.py -q"
```

`outer_check` proves the real outcome before and after, and measures wall time. `inner_check` gives agents fast feedback.

Require a clean repo, available CLIs, and a passing `outer_check`. Confirm agents and run ID, then run trials sequentially to avoid load-biased timing:

```bash
uv run python <skill>/scripts/run_trial.py --repo "$REPO" --config "$REPO/.agent-bench.toml" --agent "$AGENT" --run "$RUN_ID"
```

Results go to `eval-results/<task>/<agent>/run-<id>-<timestamp>/`. Record unexpected behavior in `ai-agent-bench-anomalies.md` per [anomalies](references/anomalies.md).
…

Read the whole file at its exact version.

How to install

Latest version

mdr add reidemeister94/development-skills/ai-agent-bench@git:20260919.a9f7da5

Exact content

mdr add reidemeister94/development-skills/ai-agent-bench@sha256:000f291fe9127b2b

Pin to a label to follow the author's releases, or to a sha256 to freeze the exact bytes forever. Either way the resolved hash is written to mdr.lock, and mdr install reproduces it on any machine.

Badge

[![mdr](https://markdownregistry.com/badge/art_aotgrvuskyue3wli.svg)](https://markdownregistry.com/a/art_aotgrvuskyue3wli)

1 badge views in 30 days

Versions

version	committed	commit	size	audit
git:20260919.a9f7da5 latest	2026-09-19	a9f7da5	1,666 B	A	view · diff
git:20260724.ba2c70f	2026-07-24	ba2c70f	1,710 B	A	view · diff
git:20260713.d210701	2026-07-13	d210701	1,888 B	A	view · diff
git:20260617.174aad8	2026-06-17	174aad8	4,507 B	A	view · diff
git:20260429.0d36b6b	2026-04-29	0d36b6b	6,383 B	A	view

Audit of the latest version

A 17 of 17 checks passed. Deterministic, no model, same answer every run.

pass: Frontmatter block present
pass: Frontmatter declares a name
pass: Frontmatter declares a description
pass: Size between 200 bytes and 200 KB (1666 bytes)
pass: No zero-width or bidi control characters
pass: No instruction hidden inside an HTML comment
pass: No link to an exfiltration or paste host
pass: No credential-shaped string
pass: No instruction to send local credentials anywhere
pass: No text hidden with inline styles
pass: No prompt-injection phrasing
pass: No curl or wget piped into a shell
pass: No recursive delete of root, home or parent
pass: No instruction to read or print local credentials
pass: No base64 blob over 200 characters
pass: No link to a raw IP address
pass: No script tag

Source

GitHub

reidemeister94/development-skills · 12 stars · license MIT · pushed 2026-09-20 · branch main

API

GET https://markdownregistry.com/api/v1/artifacts/art_aotgrvuskyue3wli
GET https://markdownregistry.com/api/v1/resolve?ref=reidemeister94/development-skills/ai-agent-bench
GET https://markdownregistry.com/api/v1/blob/000f291fe9127b2b1c67310660212b7e1cef089cc32b2ec3bf7e1db4c768f190

Your agent does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.