aeon-autoresearch · diff
git:20260512.9addfe3 to git:20260512.2331923
73 added, 48 removed. Audit A to A.
---
name: aeon-autoresearch
description: |
- Evolve an existing skill by generating four distinct improved variations, scoring them against a
- rubric, and shipping the winning version as a PR. Genetic-algorithm-style self-improvement for
- agent skills. Use when an existing skill is producing low-signal output, hitting deprecated APIs,
- or just feels stale and you want to harden it without rewriting by hand.
- Triggers: "improve this skill", "evolve token-movers", "auto-research $skill_name",
- "regenerate variations", "tune the output of $skill".
+ Evolve any installed Bankr skill by generating four distinct improved variations along separate
+ theses (better inputs / sharper output / more robust / rethink), scoring them on a weighted
+ rubric, and applying the winning version. Genetic-algorithm-style self-improvement. Never
+ downgrades a working skill. Use when an installed skill is producing low-signal output, hitting
+ deprecated APIs, or just feels stale.
+ Triggers: "improve this skill", "evolve $skill_name", "auto-research my $skill", "regenerate
+ variations for X", "tune the output of skill Y".
---
# aeon-autoresearch
- Self-improvement loop for skills. Given a target SKILL.md, the agent generates four parallel improved variations along distinct theses, scores each on a weighted rubric, and applies the winner.
-
- The point isn't to replace human-authored skills. It's to harden them — fix deprecated endpoints, sharpen output formats, add fallbacks — between maintenance windows.
-
- ## When to use this
+ Self-improvement loop for any installed skill. Given a target SKILL.md, generates four parallel improved variations along distinct theses, scores each on a weighted rubric, and applies the winner.
- - A skill's quality score (per `aeon-skill-evals`) has decayed.
- - Run logs show empty data, dropped sources, or low-signal output.
- - You're about to manually rewrite a skill — try evolving it first.
+ Not a replacement for human authorship — a way to harden working skills between maintenance windows. Fix deprecated endpoints, sharpen output formats, add fallbacks.
## Inputs
- | Param | Required | Description |
- |---|---|---|
- | `target` | yes | Skill name to evolve (e.g. `token-movers`). |
- | `mode` | no | `evolve` (default) writes a PR. `dry-run` scores variations and prints, writes nothing. |
+ | Param | Description |
+ |---|---|
+ | `target` | Skill name (e.g. `token-movers`) or path to a SKILL.md. Required. |
+ | `mode` | `evolve` (default) — score, apply winner, write diff. `dry-run` — score and print, write nothing. |
## The four theses
+ The skill generates exactly four variations, each with a different focus:
+
| Variation | Focus |
|---|---|
| **A — Better inputs** | Improve data sources. Replace deprecated APIs, add fallbacks, fix broken endpoints. |
| **B — Sharper output** | Tighter format, signal over noise, explicit verdicts, banned filler phrases. |
| **C — More robust** | Empty-data handling, retries, dedup state, rate-limit awareness. |
| **D — Rethink** | Fundamentally different methodology for the same goal. |
- Each variation must be a complete, runnable SKILL.md. Frontmatter shape is preserved (name, description, var, tags).
-
- ## Scoring rubric (1–5 per axis, weighted)
+ Each must be a complete, runnable SKILL.md. Frontmatter shape preserved (name, description, tags, any custom fields).
- | Axis | Weight | What it measures |
- |---|---|---|
- | Improvement | 3× | How much better than the original. |
- | Output value | 2× | Actionable, low noise, worth reading. |
- | Clarity | 1.5× | Will an LLM execute this correctly? |
- | Data quality | 1.5× | Sources reliable, diverse, likely to return useful data. |
- | Robustness | 1.5× | Handles failures, empty data, edge cases. |
- | Conventions | 1× | Frontmatter, logging, notification patterns. |
+ ## Scoring rubric
- Tie-break (within 2 points): prefer the variation making the single biggest improvement over many small ones.
+ Each variation scored 1–5 on:
- ## Output
+ | Axis | Weight |
+ |---|---|
+ | Improvement vs original | 3× |
+ | Output value | 2× |
+ | Clarity (LLM-executable) | 1.5× |
+ | Data quality | 1.5× |
+ | Robustness | 1.5× |
+ | Conventions (frontmatter, structure) | 1× |
- A PR titled `improve(${target}): autoresearch evolution` containing:
- - The full scoring table for all four variations.
- - The winning thesis and a paragraph rationale.
- - Diff vs the original.
- - One-paragraph summaries of the three runners-up.
+ Max weighted total: 50. Tie-break within 2 points: prefer the variation making the biggest single improvement over many small ones.
## Safety guarantee
- If every variation scores at or below the original on the **Improvement** axis, the run aborts with `AUTORESEARCH_NO_IMPROVEMENT` — no PR, no notify, no change. Never downgrade a working skill.
+ If every variation scores ≤ original on the **Improvement** axis, the skill aborts with `AUTORESEARCH_NO_IMPROVEMENT`. No file written, no change applied. Working skills are never downgraded.
The skill also preserves the original's:
- Core purpose (evolution, not replacement).
- - Tags and var semantics.
- - Required env vars (so the workflow's secret manifest stays compatible).
+ - Frontmatter shape and required fields.
+ - Any env vars the skill already declares (no new dependencies introduced silently).
- ## Example flow
+ ## Output
+ A diff against the target SKILL.md, plus a report:
+
```
- target: token-movers
+ *Autoresearch — token-movers — 2026-05-12*
Variation A (Better inputs): 43/50 — added GeckoTerminal fallback, dropped deprecated v1 endpoint
- Variation B (Sharper output): 47/50 — added pump-risk flags, banned hedge phrases, structured top-5
+ Variation B (Sharper output): 47/50 — added pump-risk flags, banned filler phrases, structured top-5
Variation C (Robustness): 41/50 — added retry + empty-data path
Variation D (Rethink): 38/50 — narrative-clustered instead of raw ranking
- Winner: B (47/50). Diff: +84 / -52 lines. PR opened: aeonframework/aeon#412
+ Winner: B (47/50)
+ Reason: largest improvement axis (5) — output value bump is the biggest single gain.
+
+ Diff:
+ SKILL.md +84 -52
+ References to deprecated /v1/markets removed (3 places)
+ Added pump-risk flag table (8 lines)
+ Banned filler phrase list added to constraints (4 lines)
+
+ Runners-up summary:
+ A: improvement was real but smaller — same output shape with better sources.
+ C: tightened reliability without changing the core output. Strong second.
+ D: interesting reframe but data-quality unproven on the new methodology.
```
+ In `evolve` mode, the target SKILL.md is replaced with the winner. In `dry-run` mode, the report is emitted and no files change.
+
+ ## Integration with version control
+
+ If the target skill is inside a git repo, the skill creates a branch (`autoresearch/${target}`), writes the changes, and offers a `git diff` for review. It never pushes — the operator commits and pushes manually after reading the diff.
+
+ If not inside a repo, the original SKILL.md is preserved at `${target}/SKILL.md.before-autoresearch` so the operator can roll back.
+
## When this fails
- - All four variations scored ≤ original → no change, log only.
- - Target skill not found → abort.
- - Pattern: chronic `AUTORESEARCH_NO_IMPROVEMENT` on the same target → the skill genuinely needs human authorship, escalate to operator.
+ | Condition | Exit |
+ |---|---|
+ | All variations ≤ original on improvement | `AUTORESEARCH_NO_IMPROVEMENT` — no change. |
+ | Target skill not found | `AUTORESEARCH_TARGET_NOT_FOUND`. |
+ | Repeated `AUTORESEARCH_NO_IMPROVEMENT` on the same target | Indicates the skill genuinely needs human authorship; flag for operator. |
+ ## Guidelines
+
+ - Improvement is the headline. A variation that scores high on clarity but doesn't improve over the original is not the winner.
+ - Tie-breaker rewards bold improvement over incremental polish.
+ - Preserves the skill's core purpose. The skill's name and stated description are inviolate; evolution refines how it accomplishes that purpose.
+
+ ## Required keys
+
+ None directly. The skill may invoke WebFetch / WebSearch during the "Better inputs" variation to research current APIs.
+
## Pairs with
- - `aeon-skill-evals` upstream (surfaces which skills need evolution).
- - `aeon-skill-repair` downstream (handles deterministic bug fixes; autoresearch handles quality lifts).
+ - `aeon-skill-evals` upstream (surfaces which skills need evolution — low scores on the eval manifest are the trigger).
+ - `aeon-skill-repair` for deterministic bug fixes; autoresearch handles quality lifts.