performance-goal ยท diff

git:20261101.81c7b21 to git:20261105.f70265c

8 added, 1 removed. Audit A to A.

What this update changes in the checks

No check changed: every fixed check gives the same answer before and after this update.

A tripped check is a label, not a verdict: read the matched text and the lines below. This week's updates that newly trip a check.

---
name: performance-goal
description: "[oh-my-agy] Evaluator-gated performance optimization loop with durable artifacts: define an evaluator command and pass/fail contract first, then optimize in small reversible patches with recorded checkpoints until the evaluator passes. Use when the user wants measurable speed, latency, throughput, memory, or benchmark improvement as a goal-oriented loop."
---
# Performance Goal Workflow
Use this skill when the user asks to optimize performance and wants a goal-oriented loop rather than a one-off review.
## Contract
- Durable workflow state lives under `.omagy/goals/performance/<slug>/` (`state.json`, `checkpoints.jsonl`).
- No optimization work starts until an **evaluator command** and a **pass/fail contract** exist.
- Completion requires a passing evaluator checkpoint plus a completion audit. Passing ordinary tests alone is not enough unless they are the declared evaluator.
- Codex's hidden goal tools do not exist in Antigravity. The artifacts are the source of truth.
## Create
If no workflow exists, establish (asking with `ask_question` only for values the repo cannot answer):
- objective (e.g. "Reduce CLI startup latency by 20%")
- evaluator command (e.g. `npm run perf:startup`)
- evaluator contract (e.g. "PASS when p95 latency improves by >= 20% vs baseline and regression tests pass")
- slug
Write `.omagy/goals/performance/<slug>/state.json`:
```json
{"slug":"startup-latency","objective":"...","evaluator_command":"...","evaluator_contract":"...",
"baseline":null,"lastValidation":null,"status":"active","created_at":"<ISO>","updated_at":"<ISO>"}
```
Also write `.omagy/state/performance-goal-state.json` (`mode:"performance-goal"`, `active:true`, `current_phase:"baseline"`, `state.slug`). The Stop hook uses it to keep the loop going.
## Agent loop
1. **Baseline**: run the evaluator (several runs if the metric is noisy) and record `baseline` with the raw numbers.
2. **Optimize** in small, reversible patches, one hypothesis per patch. Use the `debugger` or `architect` subagent for profiling analysis when useful.
3. **Evaluate**: run the evaluator and related regression tests after each patch.
4. **Checkpoint** every result by appending to `checkpoints.jsonl` (using newline-isolated appends via `sh <plugin>/hooks/run.sh append-jsonl .omagy/goals/performance/<slug>/checkpoints.jsonl '<json>'` or ensuring a leading newline if the file tail was torn):
`{"at":"<ISO>","status":"pass|fail|blocked","evidence":"<numbers, command, exit code>","patch":"<summary>"}`
Then update `lastValidation` in `state.json` and `updated_at`/`iteration` in the mode state. Revert patches that regress or do not help.
5. **Complete** only when `lastValidation.status == "pass"` and the audit confirms no required work remains. Set `state.json` `status:"complete"` and write the mode state terminal (`active:false`, `current_phase:"complete"`, `completed_at`).
6. **Blocked** (e.g. a missing fixture or benchmark infra): checkpoint `blocked`, set the mode state to `current_phase:"blocked"`, `active:false`, and ask the user.
+ ## Evaluation Integrity Audit (`eval-integrity`, ported from OMO `v5.1.59` `#12040`)
+
+ Before declaring a benchmark or evaluator result trustworthy, audit the evaluation setup across:
+ 1. **7 Evaluation Roles**: map `Claim`, `System under test`, `Dataset and split`, `Designer`, `Evaluator or judge`, `Scorer`, and `Subjects`, and confirm an independent ground-truth signal exists.
+ 2. **8 Independence Failure Probes**: check for `Contamination` (benchmark leakage), `Invalid controls` (unfair baseline flags/warmup), `Circular judging` (system judging its own output), `Scorer-label coupling`, `Evaluator-designer coupling` (tuning the evaluator to make the patch pass), `Split/annotator dependence`, `Hypothesis priming`, and `Observer/demand effects`.
+ 3. **Verdict Line**: record a single `**Verdict: PASS | FAIL | INCONCLUSIVE**` line in `.omagy/goals/performance/<slug>/summary.md` (and `evalIntegrityVerdict` in `state.json` if needed), which the HUD `Overview` tab surfaces on the benchmark card.
+
## Completion gate
- A performance goal is incomplete unless `state.json` holds `lastValidation.status: "pass"` produced by the declared evaluator command, with the evidence recorded in `checkpoints.jsonl`. Report baseline vs final numbers, the patches kept, the patches reverted, and the residual risks, and surface `[oh-my-agy HUD](sidecar://oh-my-agy/hud/)` once so the user can inspect the `Benchmarks` tab in the Auxiliary Pane.
+ A performance goal is incomplete unless `state.json` holds `lastValidation.status: "pass"` produced by the declared evaluator command, with the evidence recorded in `checkpoints.jsonl` and an `eval-integrity` `**Verdict: PASS**` in `summary.md`. Report baseline vs final numbers, the patches kept, the patches reverted, and the residual risks, and surface `[oh-my-agy HUD](sidecar://oh-my-agy/hud/)` once so the user can inspect the benchmark card in the Auxiliary Pane.