--- name: agent-benchmark description: "Self-benchmark: YOU write the code, adversarial reviews it (multi-provider), you fix, you write tests, adversarial reviews tests, you fix. Measures YOUR quality as an agent. Run in different models (Opus, Sonnet, Haiku) and compare results." category: Utility --- # zuvo:agent-benchmark — Self-Benchmark You are the subject of this benchmark. YOU write the code and tests. Adversarial review (multi-provider) critiques your work between rounds. You fix based on findings. Run this skill in different models (Opus, Sonnet, Haiku) to compare agent quality. ## Argument Parsing | Flag | Effect | |------|--------| | `--quick` | Skip adversarial rounds (R1 code + R3 tests only, no fixes) | | `--no-tests` | Skip test rounds (R1 + R2 only) | | `--dry-run` | Print what would happen, don't execute | | _(no flags)_ | Full 4-round benchmark with adversarial | ## Mandatory File Loading Read these files before starting: ``` CORE FILES LOADED: 1. ../../shared/includes/benchmark-corpus/task-code.md -- READ/MISSING 2. ../../shared/includes/benchmark-corpus/task-tests.md -- READ/MISSING 3. ../../shared/includes/benchmark-scoring-rubric.md -- READ/MISSING 4. ../../shared/includes/run-logger.md -- READ/MISSING 5. ../../shared/includes/retrospective.md -- READ/MISSING ``` If any file is missing, stop. --- ## Setup 1. Detect current model: check `$CLAUDE_MODEL` or infer from context. Record as `agent_model`. Build a short slug using this table: | Model string contains | Slug | |----------------------|------| | `opus` | `opus` | | `sonnet` | `sonnet` | | `haiku` | `haiku` | | `gpt-5.4` or `codex` + `5.4` | `codex-5.4` | | `gpt-5.3` or `codex` + `5.3` | `codex-5.3` | | `gpt-5` or `codex` (no version) | `codex` | | `gemini` | `gemini` | | `composer` | `composer` | | `cursor` | `cursor` | | anything else | use the model name as-is | **Do NOT use `agent` as a slug.** The slug must identify which model ran the benchmark. 2. Create output directory with agent name in folder: ```bash AGENT_SLUG="" RUN_ID="agent-bm-${AGENT_SLUG}-$(date -u +%Y%m%dT%H%M%SZ)" OUT_DIR="zuvo/reports/${RUN_ID}" mkdir -p "$OUT_DIR" ``` Dispatch follows `../../shared/includes/execution-policy.md` through env-compat. Reuse existing authorization within that policy; session restrictions take precedence. Run each required gate and report its actual independence or an unmet requirement. 3. Find adversarial-review.sh: - `scripts/adversarial-review.sh` (if in zuvo-plugin repo) - `~/.codex/scripts/adversarial-review.sh` (Codex install) - `~/.cursor/scripts/adversarial-review.sh` (Cursor install) - `~/.zuvo/adversarial-review` (stable; the versioned cache path breaks after any release) (Claude Code) 4. Record start time. --- ## Round 1 — Write Code Read `../../shared/includes/benchmark-corpus/task-code.md`. This is your task. **Write the two files yourself.** Do NOT dispatch to external providers. YOU are the agent being tested. Write each file using the Write tool: - `$OUT_DIR/r1-OrderService.ts` - `$OUT_DIR/r1-useSearchProducts.ts` Follow the spec exactly. Do your best work — this is measuring YOUR quality. Record R1 time: `r1_time_s = end - start`. Print: ``` ── Round 1: Code ── r1-OrderService.ts — written (N lines) r1-useSearchProducts.ts — written (N lines) Time: Xs ``` --- ## Round 2 — Adversarial Review + Fix Run adversarial-review.sh on your R1 files. **Use all available providers** (default multi-provider mode): ```bash adversarial-review.sh --files "$OUT_DIR/r1-OrderService.ts $OUT_DIR/r1-useSearchProducts.ts" --json ``` Read the findings. Print summary: ``` ── Round 2: Adversarial Review ── Providers: [list of adversarial reviewers] Findings: N critical, M warning, K info ``` Now **fix your code** based on the adversarial findings. **Do NOT edit R1 files — they are the baseline.** Write corrected code as NEW files: - `$OUT_DIR/r2-OrderService.ts` (new file — corrected version of r1) - `$OUT_DIR/r2-useSearchProducts.ts` (new file — corrected version of r1) Save adversarial findings: - `$OUT_DIR/r2-adversarial-findings.txt` If adversarial found zero issues, copy R1 files as R2: ```bash cp $OUT_DIR/r1-OrderService.ts $OUT_DIR/r2-OrderService.ts cp $OUT_DIR/r1-useSearchProducts.ts $OUT_DIR/r2-useSearchProducts.ts ``` Record R2 time (review + fix). Print: ``` Fix applied: [brief description of what you changed] Time: Xs ``` --- ## Round 3 — Write Tests Read `../../shared/includes/benchmark-corpus/task-tests.md`. Replace `{{ROUND_1_CODE}}` with the contents of your R2 files (the fixed versions). **Write tests yourself** for both files: - `$OUT_DIR/r3-OrderService.test.ts` - `$OUT_DIR/r3-useSearchProducts.test.ts` Record R3 time. Print: ``` ── Round 3: Tests ── r3-OrderService.test.ts — written (N lines, M test cases) r3-useSearchProducts.test.ts — written (N lines, M test cases) Time: Xs ``` --- ## Round 4 — Adversarial Review on Tests + Fix Run adversarial-review.sh on your test files. **Multi-provider mode**: ```bash adversarial-review.sh --files "$OUT_DIR/r3-OrderService.test.ts $OUT_DIR/r3-useSearchProducts.test.ts" --json --mode test ``` Read findings. **Do NOT edit R3 files — they are the baseline.** Read R3, apply fixes, and write corrected tests as NEW files: - `$OUT_DIR/r4-OrderService.test.ts` (new file — corrected version of r3) - `$OUT_DIR/r4-useSearchProducts.test.ts` (new file — corrected version of r3) **Critical: R3 and R4 must be different files showing before/after.** If you edit R3 directly, the benchmark loses the ability to compare pre/post adversarial quality. Save findings: - `$OUT_DIR/r4-adversarial-findings.txt` Record R4 time. Print: ``` ── Round 4: Adversarial on Tests ── Findings: N critical, M warning, K info Fix applied: [brief description] Time: Xs ``` --- ## Self-Scoring After all 4 rounds, score your own output using `../../shared/includes/benchmark-scoring-rubric.md`. 1. Read your R2 files. Score C1-C7 (code quality, max 35). 2. Read your R4 files. Score T1-T5 (test quality, max 25). 3. Compare R1 vs R2 + adversarial findings. Score A1 (code fix, max 5). 4. Compare R3 vs R4 + adversarial findings. Score A2 (test fix, max 5). 5. Record all scores in `agent-benchmark.json` under `scores` key. **Be honest.** This is self-evaluation — inflated scores will be caught when comparing across models. --- ## Output — Summary + Artifacts ### File Inventory Print: ``` ── Artifacts: $OUT_DIR/ ── r1-OrderService.ts (N lines) — original code r1-useSearchProducts.ts (N lines) — original code r2-OrderService.ts (N lines) — after adversarial fix r2-useSearchProducts.ts (N lines) — after adversarial fix r2-adversarial-findings.txt — adversarial review output r3-OrderService.test.ts (N lines, M tests) — original tests r3-useSearchProducts.test.ts (N lines, M tests) — original tests r4-OrderService.test.ts (N lines, M tests) — after adversarial fix r4-useSearchProducts.test.ts (N lines, M tests) — after adversarial fix r4-adversarial-findings.txt — adversarial review output agent-benchmark.json — machine-readable results ``` ### JSON Report Write `$OUT_DIR/agent-benchmark.json`: ```json { "version": "1.0", "skill": "agent-benchmark", "run_id": "", "timestamp": "", "agent_model": "", "agent_slug": "", "project": "", "r1_time_s": 0, "r2_time_s": 0, "r3_time_s": 0, "r4_time_s": 0, "total_time_s": 0, "tokens": { "r1_input": 0, "r1_output": 0, "r2_input": 0, "r2_output": 0, "r3_input": 0, "r3_output": 0, "r4_input": 0, "r4_output": 0, "total_input": 0, "total_output": 0 }, "cost_usd": { "note": "Estimated API cost at current pricing. $0 if running from subscription.", "r1": 0.0, "r2": 0.0, "r3": 0.0, "r4": 0.0, "total": 0.0, "price_per_1m_input": 0.0, "price_per_1m_output": 0.0 }, "r2_adversarial": { "providers": ["gemini", "codex-fast"], "critical": 0, "warning": 0, "info": 0 }, "r4_adversarial": { "providers": ["gemini", "codex-fast"], "critical": 0, "warning": 0, "info": 0 }, "files": { "r1_code_lines": 0, "r2_code_lines": 0, "r3_test_lines": 0, "r3_test_count": 0, "r4_test_lines": 0, "r4_test_count": 0 } } ``` ### Token Estimation + API Cost For each round, estimate tokens: - **Input tokens**: count words in the prompt × 1.3 - **Output tokens**: count words in the written files × 1.3 Use these API prices (USD per 1M tokens) to compute cost: | Model | Input $/1M | Output $/1M | |-------|-----------|------------| | opus | 15.00 | 75.00 | | sonnet | 3.00 | 15.00 | | haiku | 0.80 | 4.00 | | composer | 0.00 | 0.00 | | cursor-composer | 0.00 | 0.00 | | unknown | 3.00 | 15.00 | Formula per round: `cost = (input_tokens × input_price / 1_000_000) + (output_tokens × output_price / 1_000_000)` ### Completion Block ``` AGENT BENCHMARK COMPLETE Model: [agent_model] Run ID: [run_id] Artifacts: [OUT_DIR]/ | Round | Time | Tokens (in/out) | API Cost | |-------|------|-----------------|----------| | R1 Code | Xs | ~Nk/~Mk | $X.XX | | R2 Adversarial+Fix | Xs | ~Nk/~Mk | $X.XX | | R3 Tests | Xs | ~Nk/~Mk | $X.XX | | R4 Adversarial+Fix | Xs | ~Nk/~Mk | $X.XX | | **Total** | **Xs** | **~Nk/~Mk** | **$X.XX** | Adversarial Impact: Code: N findings → [what changed] Tests: N findings → [what changed] Files: 8 artifacts + 2 findings + 1 JSON = 11 files ``` ### Run Log ``` Run: agent-benchmark - - PASS 4-round ``` After printing, append to the log file path resolved per `../../shared/includes/run-logger.md`.