aeon-skill-repair · diff
git:20260512.9addfe3 to git:20260512.2331923
78 added, 79 removed. Audit A to A.
---
name: aeon-skill-repair
description: |
- Auto-diagnose and fix failing or degraded skills. Triages systemic first (one shared fix for N
- skills hitting the same root cause), then per-category playbook (api-change, rate-limit, timeout,
- sandbox-limitation, prompt-bug, output-format, missing-secret, config). Every PR includes a
- verification block the operator can execute. Use when one or more skills are failing consecutively
- or producing low-quality output.
- Triggers: "fix skill X", "repair the failing skills", "auto-fix the catalog", "skill X is broken".
+ Auto-diagnose and fix a failing or degraded installed skill. Reads the SKILL.md plus recent error
+ output, classifies the failure category (api-change / rate-limit / timeout / sandbox-limitation /
+ prompt-bug / output-format / missing-secret / config), and applies the matching playbook. Every
+ fix includes a verification block the operator can execute. Use when a previously-working skill
+ has started failing or producing low-quality output.
+ Triggers: "fix this skill", "skill X is broken", "diagnose this failure", "auto-repair my
+ failing skill", "the output of X looks wrong".
---
# aeon-skill-repair
- Self-healing fleet. Reads skill-run state, builds a diagnostic dossier (including a regression-hunter pass over git history), classifies the failure category, and applies the matching playbook. Every PR ships with a `## Verification` section the operator can execute manually.
+ Targeted repair for a single failing skill. Builds a diagnostic dossier, classifies the failure into a known category, and applies the corresponding playbook with a minimum-edit principle. Every repair ships with a verification step.
## Phases
- `PREFLIGHT → TRIAGE → DIAGNOSE → REPAIR → VERIFY → LOG`
-
- The skill stops early at the appropriate exit code if any phase finds nothing actionable.
-
- ## Systemic-first triage
-
- Before treating any single skill, the skill checks for clusters: if 2+ skills fail with the same normalized error signature or category, one shared issue is filed and one shared fix opened — instead of N redundant per-skill patches.
+ `PREFLIGHT → DIAGNOSE → REPAIR → VERIFY`
- Categories that trigger systemic mode: `api-change`, `rate-limit`, `missing-secret`, `sandbox-limitation`.
+ The skill stops at the appropriate exit code if any phase finds nothing actionable.
## Inputs
| Param | Description |
|---|---|
- | `target` | Optional. Skill name to repair. If empty, runs auto-selection. |
- | `mode` | `repair` (default) or `dry-run` (diagnose only). |
-
- ## Auto-selection rules
-
- Reads `memory/cron-state.json` and `memory/issues/INDEX.md`. Candidates: any skill where
-
- - `consecutive_failures >= 2`, **or**
- - `success_rate < 0.5` and `total_runs >= 3`, **or**
- - `last_status == failed` and `last_failed` within 48h, **or**
- - `last_quality_score <= 2` (degraded output).
-
- Sort: critical issue > high issue > consecutive_failures desc > lowest success_rate > stalest `last_success`. Skip `permanent-limitation` and any target under cooldown.
+ | `target` | Skill name or SKILL.md path. Required. |
+ | `error_output` | Optional. The skill's recent failed output (paste from run log). If absent, the skill will ask for it before proceeding. |
+ | `mode` | `repair` (default) — apply the fix. `dry-run` — diagnose and propose without writing. |
## Diagnostic dossier
- Built before touching any file. Six sources, each tagged `ok` / `empty` / `fail`:
+ Before touching any file, the skill assembles:
- 1. **Skill file** — frontmatter, declared data sources, env-var references.
- 2. **Cron state** — `last_error`, `consecutive_failures`, `success_rate`, `last_quality_score`.
- 3. **Regression hunter** — `git log --since=$LAST_SUCCESS` against the skill file, `aeon.yml`, `scripts/`. A single suspect commit becomes the prime root-cause hypothesis.
- 4. **Recent failed runs** — `gh run view --log-failed` for the last 5 + check-run annotations for cleaner error rows.
- 5. **Logs** — last 3 days of run logs for the skill name.
- 6. **Eval assertions** — if the skill has an `aeon-skill-evals` manifest, the failing assertions become the fix target.
+ 1. **Skill file** — reads the target SKILL.md. Identifies frontmatter, declared data sources, env-var references.
+ 2. **Error analysis** — parses `error_output` for known signatures (HTTP status codes, common API error strings, rate-limit hits, timeout patterns, refusal markers).
+ 3. **Source liveness check** — if the skill references URLs / APIs, WebFetch each to check for 404s, redirects, or schema changes.
+ 4. **Frontmatter integrity** — verifies the skill's frontmatter is valid YAML.
+ 5. **Reference parity** — if the skill links references files, verifies they exist.
- Distinguish **consistent** (same signature 4–5/5 runs → deterministic bug) from **intermittent** (1–2/5 → rate limit, flaky upstream).
+ ## Categories and playbooks
- ## Per-category playbooks
+ | Category | Detection signal | Playbook |
+ |---|---|---|
+ | **api-change** | 404, 410, schema mismatch, deprecated endpoint warning | WebFetch the live API spec / status page / release notes. Update endpoints, payload shape, headers, error codes. Cite the spec URL in the fix notes. |
+ | **rate-limit** | 429, "too many requests", rolling-window quota error | Add backoff or fallback endpoint. Never raise the limit from the skill side. If the skill's invocation cadence is too aggressive, recommend reducing it but don't change it unilaterally. |
+ | **timeout** | Skill takes too long, partial output, killed mid-run | Stage the work, add early-return on partial success, downgrade the model if it doesn't need the most capable tier. |
+ | **sandbox-limitation** | Outbound curl fails with auth headers; secrets not expanded in shell | Convert curls to a prefetch pattern (write to a state file before the agent runs) or postprocess pattern (write requests to a queue, process after). |
+ | **prompt-bug** | Hallucination, refusal ("as an AI..."), missing required output section | Minimum-edit specificity insertion. Add the missing constraint, a forbidden phrase, a required output structure. Don't rewrite — < 30 lines diff. |
+ | **output-format / quality-regression** | Output passes execution but fails downstream parser; eval assertions fail | Cross-reference the skill's eval manifest if one exists; edit until the next run satisfies the failing assertion. |
+ | **missing-secret** | `not configured`, `API key missing`, env var unset | **Do not modify the skill.** Identify the missing env var by name, write a short note for the operator. Exit `REPAIR_DIAGNOSED_NO_FIX`. |
+ | **config** | Bad input config (watchlist, distribution list, RSS feeds) | Validate the config file shape, fix obvious errors (trailing commas, malformed YAML), but never invent entries. |
+ | **unknown** | None of the above | Don't edit blindly. Append the full dossier to a `repair-notes.md` file next to the skill, exit `REPAIR_DIAGNOSED_NO_FIX`. Operator triages. |
- | Category | Playbook |
- |---|---|
- | `api-change` | WebFetch the live spec/status page. Update endpoints, payload, headers. Cite the spec URL in the PR. Never guess — if WebFetch fails, drop to `REPAIR_DIAGNOSED_NO_FIX`. |
- | `rate-limit` | Add backoff or fallback endpoint. Never raise the limit from the skill side. |
- | `timeout` | Stage the work, add early-return on partial success, downgrade model if not Opus-critical. |
- | `sandbox-limitation` | Convert auth-bearing curls to the prefetch (`scripts/prefetch-*.sh`) or postprocess (`.pending-*/` + `scripts/postprocess-*.sh`) pattern. |
- | `prompt-bug` | Minimum-edit specificity insertion. Don't rewrite — add the missing constraint, a forbidden phrase, a required output structure. < 30 lines diff. |
- | `output-format` / `quality-regression` | Cross-reference `aeon-skill-evals` assertions. Edit until the next run satisfies the failing assertion. |
- | `missing-secret` | **Do not modify the workflow.** File an issue naming the secret, notify operator, exit `REPAIR_DIAGNOSED_NO_FIX`. |
- | `config` | Reversible `aeon.yml` edits only — `schedule`, `var`, `model`, `enabled: false`. Keep diff < 5 lines. |
- | `unknown` | Don't edit blindly. Append the dossier to the issue file as `## Diagnosis Notes`, exit `REPAIR_DIAGNOSED_NO_FIX`. |
+ ## Risk classes
- ## Risk classes (gate the PR)
+ Every proposed fix is labeled:
- | Class | Scope | Auto-merge? |
+ | Class | Scope | Auto-apply? |
|---|---|---|
- | **LOW** | Clarifying prompt, fallback, comment-only, < 30 lines diff. | Yes |
- | **MED** | Data source change, new env-var reference (already in workflow), output format edit. | Yes if tests pass |
- | **HIGH** | Touches workflows, removes features, disables a skill, modifies `scripts/*.sh`. | **No — `manual-review` label, human only.** |
+ | **LOW** | Clarifying prompt edit, fallback added, comment-only change, < 30 lines diff. | Yes |
+ | **MED** | Data source change, new env-var reference (must already be available), output format edit. | Yes with verification |
+ | **HIGH** | Touches behavior fundamentally, removes features, changes default config. | **No — operator review required.** |
## Verification block
- Every PR (except `REPAIR_DIAGNOSED_NO_FIX`) includes:
+ Every repair (except `REPAIR_DIAGNOSED_NO_FIX`) emits a verification recipe:
- ```markdown
+ ```
## Verification
- **Manual trigger:** dispatch the skill with `skill=${target}` and `var=${var}`.
+ To verify this repair worked:
- **Expected:**
- - Workflow conclusion: `success`
- - Output matches: `${eval_pattern or "memory/logs/${today}.md mentions ${target}"}`
- - ${category-specific signal}
+ 1. Re-run the skill: `<one-line invocation>`
+ 2. Expected behavior: <category-specific signal — e.g. "no rate-limit errors in output" /
+ "produces ≥ 200 words" / "output matches required pattern X">
+ 3. If still failing: <fallback path — e.g. "the error category may have been mis-classified;
+ delete the repair-history entry and re-run repair">
+ ```
- **If still failing after this PR:** delete `memory/state/skill-repair-history.json[${target}]` to clear cooldown, then re-dispatch for a second pass.
+ ## Output
+
```
+ *Skill Repair — token-movers — 2026-05-12*
- ## Cooldown
+ Diagnosis
+ Category: api-change
+ Signal: HTTP 410 from /v1/markets endpoint (consistent across 5 recent runs)
+ Root cause: CoinGecko deprecated v1 in favor of /api/v3/coins/markets
- A target cannot be repaired twice within 24h without operator clearing `memory/state/skill-repair-history.json`. Prevents repair loops on fixes that didn't stick.
+ Fix applied
+ Updated 3 endpoint references in skills/token-movers/SKILL.md
+ Risk: MED (data source change, no new env vars)
+ Diff: +12 -9 lines
- Also rate-limits: max 3 skill-repair PRs per UTC day.
+ Verification
+ Re-run: bankr-run token-movers
+ Expected: output contains "Top movers" header, lists ≥ 5 coins, no HTTP 410 errors in trace
- ## Exit taxonomy
+ Notes
+ CoinGecko v1 deprecation announced 2026-02-15. v3 schema is broadly compatible — field names
+ preserved. If field-level differences surface in production, re-run skill-repair against the
+ new error signature.
+ ```
- | Code | Meaning |
- |---|---|
- | `REPAIR_OK_FIXED` | Per-skill fix applied, PR opened. |
- | `REPAIR_OK_SYSTEMIC` | Shared root cause across N skills — single shared fix or shared issue filed. |
- | `REPAIR_DIAGNOSED_NO_FIX` | Root cause known, needs operator action. |
- | `REPAIR_NO_TARGETS` | Fleet healthy. |
- | `REPAIR_DRY_RUN` | Diagnostic only. |
- | `REPAIR_BLOCKED` | Preflight failed or cooldown active. |
+ ## Cooldown
+ A skill cannot be auto-repaired more than once in a 24h window without operator confirmation. Prevents repair loops on fixes that didn't stick. The cooldown state is stored in a local `repair-history.json`.
+
## Guardrails
- - One target per run (or one systemic cluster).
- - Minimum-edit principle — diffs are small. The fix is rarely "rewrite the skill".
- - Never modify workflow files, secrets, or the messages spec.
- - Never push to main — branch + PR only.
- - HIGH-risk PRs are human-merge only.
+ - One target per run. Never bundles unrelated repairs.
+ - Minimum-edit principle. Diffs are small.
+ - Never modifies env-var configuration; missing secrets are flagged for the operator.
+ - HIGH-risk fixes are proposed, not applied.
+ - For repairs inside a git repo, changes land in a branch (`repair/${target}`), never directly on main.
+ ## Required keys
+
+ None directly. The `api-change` playbook may use WebFetch to look up live API specs.
+
## Pairs with
- `aeon-skill-evals` upstream (surfaces what's failing).
- - `aeon-autoresearch` for quality-lift evolution (this skill handles deterministic bugs).
+ - `aeon-autoresearch` for quality lifts vs deterministic bugs.