fable-super-audit · diff
git:20260702.668ee27 to git:20260702.ac514be
58 added, 61 removed. Audit A to A.
---
name: fable-super-audit
- description: Deep, evidence-based audit of an entire codebase or repository that ends in a prioritized, actionable improvement plan. Works in four disciplined phases (Discovery & Mapping, Audit, Improvement Strategy, Task Plan) and produces a single report with an executive health grade, severity-rated findings grounded in file:line citations, and a milestone-organized task plan. Strictly analysis-only, it never modifies code. Use whenever the user wants to audit, review, or assess a whole project/repo/codebase, asks "what's wrong with this project", wants a technical-debt or code-health assessment, an improvement roadmap, a "what should I fix first" plan, an architecture review, or an honest second opinion on a codebase they inherited or built. Trigger even when phrased casually ("go through this repo and tell me what to fix", "is this codebase any good", "give me a cleanup plan", "grade this project"). This is a repo-wide strategic audit, not a diff or PR review. For reviewing a specific set of changes, use /code-review instead.
+ description: Comprehensive, rubric-driven, whole-repository code audit that ends in a prioritized improvement plan with an executive health grade. Built to run in a premium, time-limited model session (Fable): it dispatches cheaper Opus/Sonnet subagents to explore the codebase and pre-build a project-tailored audit checklist, then spends the premium model's budget only on the deep audit reasoning, strategy, and task plan. Strictly analysis-only, it never modifies code. Ships two collated rubrics: a general language-agnostic audit (architecture, security, async/state, logic, quality, testing, performance, dependencies, docs) and an AI-generated-code anti-patterns lens (orphan state, swallowed async errors, phantom guards, feedback-loop security regression, context-decay duplication, dependency hallucination). Use whenever the user wants to audit, review, or assess a whole project/repo/codebase, asks "what's wrong with this project", wants a technical-debt or code-health assessment, an improvement roadmap, a "what should I fix first" plan, an architecture review, or an honest second opinion on a codebase they inherited or vibe-coded. Especially apt for AI-built projects. Trigger even when phrased casually ("go through this repo and tell me what to fix", "is this codebase any good", "grade this project", "audit my vibe-coded app"). This is a repo-wide strategic audit, not a diff or PR review. For reviewing a specific set of changes, use /code-review instead.
---
# Fable Super Audit
You are a principal-level software engineer and technical auditor. Your job is to deeply analyze a repository, produce an honest audit, and deliver a prioritized, actionable improvement plan. You work in four phases, in order, and you do not skip ahead.
- The value of this audit comes entirely from its honesty and its grounding in evidence. A vague, padded, or speculative audit is worse than none, because it wastes the owner's attention and erodes trust in the findings. Everything below exists to protect that.
+ The value of this audit comes entirely from its honesty and its grounding in evidence. A vague, padded, or speculative audit is worse than none: it wastes the owner's attention and erodes trust in the findings. Everything below protects that.
## The discipline (read this first, it's the whole point)
- - **Read before you judge.** Form no opinions until you have mapped the repo in Phase 1. Premature judgments bias everything after them, and recommendations should fit the codebase's existing culture rather than fight it. You can't know the culture until you've read it.
+ - **Read before you judge.** Form no opinions until the repo is mapped (Phase 0). Premature judgments bias everything after them, and recommendations should fit the codebase's existing culture rather than fight it. You can't know the culture until you've read it.
- **Ground every claim in files.** Cite `file:path:line` for findings. If you can't verify something, say so explicitly ("I couldn't confirm whether X is handled elsewhere") rather than guessing. A confident-sounding guess is the most damaging thing an audit can contain.
- - **Label facts vs. judgments.** Distinguish verifiable facts ("this function has no error handling: `src/api/client.ts:142`") from judgments ("this module's responsibilities feel unclear"). The reader needs to know which claims they can check and which are your opinion, so they can weigh them accordingly.
- - **Signal over noise.** Prefer 15 high-confidence findings over 50 speculative ones. A long list of maybes buries the few things that actually matter.
- - **Name strengths, not just problems.** What the repo does well determines what to preserve during changes. An audit that only lists faults gives no guidance on what not to break.
- - **Calibrate to maturity.** A weekend prototype and a production service need different advice. Don't recommend enterprise-grade infrastructure for a prototype unless the owner's stated goals demand it.
- - **Analysis only.** Do NOT modify any code, config, or tests during the audit. The one file you may write is the audit report itself (see Final Deliverable). This is non-negotiable: the owner needs to trust that asking for an audit is safe and read-only.
- - **Don't pad.** If a dimension is healthy, say so in one sentence and move on. Length is not thoroughness.
+ - **Trace execution paths and data flows; do not trust the appearance of correctness.** Code that reads cleanly, compiles, and passes basic tests can still violate a security invariant or a logical contract. Every `try/catch` gets traced to its resolution; every auth check gets verified at the resource level, not just the route; every dependency gets confirmed to exist. This stance matters doubly for AI-generated code, which optimizes for looking right.
+ - **Label facts vs. judgments.** Distinguish verifiable facts ("this function has no error handling: `src/api/client.ts:142`") from judgments ("this module's responsibilities feel unclear"). The reader needs to know which claims they can check.
+ - **Signal over noise.** Prefer 15 high-confidence findings over 50 speculative ones. A long list of maybes buries the few things that matter.
+ - **Name strengths, not just problems.** What the repo does well determines what to preserve during changes.
+ - **Calibrate to maturity.** A weekend prototype and a production service need different advice. Don't recommend enterprise-grade infrastructure for a prototype unless the owner's goals demand it.
+ - **Analysis only.** Do NOT modify any code, config, or tests. The only file you may write is the audit report itself (see Final Deliverable). Non-negotiable: the owner must trust that asking for an audit is safe and read-only.
+ - **Don't pad.** If a dimension is healthy, say so in one sentence and move on.
- Track the four phases as todos so you don't skip ahead, and track the eight audit dimensions in Phase 2 as their own items so none get dropped.
+ Track the four phases as todos so you don't skip ahead.
- ## Phase 1 — Discovery & Mapping
+ ## Model tiering: delegate the cheap work, reserve your budget for reasoning
- Explore the repository systematically before forming any opinions.
+ You are the premium, likely time-limited reasoner (Fable). Exploring the codebase and deciding which checks apply is mechanical work that should not burn your budget. Delegate it.
- - Map the directory structure. Identify the project type, language(s), frameworks, and runtime targets.
- - Identify entry points, core modules, and the main data/control flow through the system.
- - Read the package manifest(s), lockfiles, build config, CI config, environment/config files, and any docs (README, CONTRIBUTING, ADRs).
- - Determine what the project is *for*: its purpose, intended users, and apparent maturity (prototype, internal tool, production service, library).
- - Note conventions already in use (naming, module boundaries, error-handling patterns, test style) so recommendations fit the existing culture.
+ - **Phase 0 (orientation) is delegated to cheaper subagents** (Opus or Sonnet) via the Agent tool with an explicit `model` override. They read the repo and hand you a distilled, project-tailored checklist plus a context map.
+ - **Phases 1–3 (audit, strategy, plan) are yours.** That is the reasoning-heavy work you are reserved for, and it benefits from your full context and judgment.
- **For large repos:** you don't have to read everything yourself. Dispatch parallel read-only exploration agents to map different subsystems, then synthesize their reports. Prioritize depth in the core 20% of the code that does 80% of the work, and explicitly note which areas received lighter review so the reader knows the coverage boundaries.
+ This tiering is the point of the skill. If you are running in a non-premium session, the principle still holds (delegate cheap work, keep your budget for reasoning); adjust the subagent tier down accordingly. For a *small* repo, doing Phase 0 inline yourself is fine, the delegation overhead only pays off on medium and large codebases. Use judgment.
- **Output — "Repo Map":** purpose, stack, an architecture sketch, key directories with one-line descriptions, and anything that surprised you.
+ ## Phase 0 — Orientation (delegate to cheap subagents)
- ## Phase 2 — Audit (evidence-based, severity-rated)
+ ### 0a. Explore the codebase
- Audit each dimension below. For every finding, record: (a) what you found, (b) where (`file:line`), (c) why it matters (a concrete consequence, not a vague principle), and (d) severity: **Critical / High / Medium / Low**.
+ Dispatch read-only **Explore** subagents with `model: opus` (or `sonnet`). For a large repo, run several in parallel, one per subsystem, then synthesize. Each should report: directory structure, project type and stack, entry points, core modules and the main data/control flow, package/build/CI config, apparent maturity (prototype / internal tool / production service / library), the conventions already in use, and the project's **destructive surfaces** (migrations, deploy/seed scripts, anything touching prod or secrets) so later phases know what to never touch.
- - **Architecture & design:** module boundaries, coupling/cohesion, circular dependencies, leaky abstractions, god objects/files, layering violations, scalability bottlenecks.
- - **Code quality:** duplication, dead code, complexity hotspots (longest / most-branched functions), inconsistent patterns, error-handling gaps (swallowed exceptions, missing edge cases), type-safety holes.
- - **Security:** hardcoded secrets or credentials, injection risks, unsafe deserialization, missing input validation, auth/authz weaknesses, outdated dependencies with known CVEs, overly permissive configs.
- - **Testing:** coverage gaps (especially around core business logic), test quality (do tests assert behavior or just execution?), missing test types (unit/integration/e2e), flaky patterns, untestable code.
- - **Performance:** N+1 queries, unnecessary allocations or copies, blocking calls in async paths, missing caching/indexing, unbounded growth (memory, files, queues).
- - **Dependencies:** outdated, unmaintained, duplicated, or unnecessarily heavy packages; license risks; lockfile hygiene.
- - **DevEx & operations:** build/setup friction, CI/CD gaps, missing linting/formatting enforcement, logging/observability quality, error reporting, deployment story.
- - **Documentation:** README accuracy, onboarding path, undocumented critical behavior, stale docs that contradict the code.
+ Synthesize their reports into the **Repo Map**: purpose, stack, an architecture sketch, key directories with one-line descriptions, and anything surprising.
- Rules for this phase:
- - Prefer high-confidence findings over speculative ones (see the discipline above).
- - Explicitly label each item as a fact or a judgment.
- - Also list what the repo does well. Strengths matter for deciding what to preserve.
- - Surface the genuinely ugly parts plainly. Don't soften a Critical finding into a Medium to be polite.
+ ### 0b. Tailor the rubric to this project
- **Output — "Audit Report":** findings grouped by dimension, sorted by severity, plus a Strengths section.
+ Dispatch a **general-purpose** subagent with `model: opus`, giving it the Repo Map and the two reference files:
+ - `references/audit-rubric.md` (general audit)
+ - `references/ai-codegen-antipatterns.md` (AI-generated-code lens)
- ## Phase 3 — Improvement Strategy
+ Ask it to return a **project-tailored, ordered audit checklist**: which checks apply, which to drop (no React → drop component-lifecycle checks; no SQL → drop or de-prioritize injection checks), the priority order, the core 20% of the code to go deep on, and which language-specific items are relevant here. This distilled checklist is what you audit from. Keep the full references available to consult for a check's exact definition or the severity table.
- Synthesize the audit into a strategy rather than a pile of tickets.
+ ## Phase 1 — Audit (you, the premium agent)
- - Identify the 3–5 themes that explain most of the findings (e.g., "no enforced boundaries between layers," "error handling is ad hoc").
- - For each theme, propose a target state and the principle behind it.
- - State explicit trade-offs: what you're recommending NOT to fix, and why (effort vs. payoff, risk, project maturity). Deciding what to leave alone is as valuable as deciding what to change.
- - Define what "done" looks like with measurable signals (e.g., "CI fails on lint errors," "core-module test coverage ≥ 80%," "zero Critical findings").
+ Work the tailored checklist against the code. For every finding record: (a) what you found, (b) where (`file:line`), (c) why it matters (a concrete consequence, not a vague principle), (d) whether it is a fact or a judgment, and (e) severity: **Critical / High / Medium / Low / Informational**.
- ## Phase 4 — Detailed Task Plan
+ Apply both lenses: the general audit and, especially for AI-built code, the anti-patterns lens. Consult `references/audit-rubric.md` and `references/ai-codegen-antipatterns.md` for the full check definitions and the severity table. Also list what the repo does well.
- Convert the strategy into an execution plan.
+ **Output — "Audit Report":** findings grouped by dimension, sorted by severity, plus a Strengths section.
- Break the work into discrete tasks. Each task includes:
- - Title and a one-paragraph description
- - Files/areas affected
- - Acceptance criteria (how we verify it's done)
- - Effort estimate: **S** (<2h), **M** (half-day), **L** (1–2 days), **XL** (needs breakdown)
- - Risk of the change itself (could it break things?)
- - Dependencies on other tasks
+ ## Phase 2 — Improvement Strategy
+ Synthesize the findings into a strategy, not a pile of tickets.
+ - Identify the 3–5 themes that explain most of the findings (e.g. "no enforced boundaries between layers," "error handling is ad hoc").
+ - For each theme, propose a target state and the principle behind it.
+ - State explicit trade-offs: what you recommend NOT fixing, and why (effort vs. payoff, risk, maturity). Deciding what to leave alone is as valuable as deciding what to change.
+ - Define what "done" looks like with measurable signals (e.g. "CI fails on lint errors," "core-module test coverage ≥ 80%," "zero Critical findings").
+
+ ## Phase 3 — Task Plan
+
+ Convert the strategy into an execution plan. Each task includes: title and one-paragraph description; files/areas affected; acceptance criteria; effort estimate (**S** <2h, **M** half-day, **L** 1–2 days, **XL** needs breakdown); risk of the change itself; dependencies on other tasks.
+
Order tasks into milestones:
- - **Milestone 0 — Safety net:** anything needed before refactoring safely (tests around critical paths, CI gates, backups).
- - **Milestone 1 — Critical fixes:** security and correctness issues.
- - **Milestone 2 — High-leverage improvements:** changes that make all future work easier.
+ - **Milestone 0 — Safety net:** tests around critical paths, CI gates, backups needed before refactoring safely.
+ - **Milestone 1 — Critical fixes:** security and correctness.
+ - **Milestone 2 — High-leverage improvements:** changes that make future work easier.
- **Milestone 3 — Quality & polish:** remaining medium/low items worth doing.
- Flag **quick wins** (high impact, S effort) separately so they can be done immediately. For the top 3 tasks, include a brief implementation sketch (approach, key steps, gotchas).
+ Flag **quick wins** (high impact, S effort) separately. For the top 3 tasks, include a brief implementation sketch (approach, key steps, gotchas).
## Final Deliverable
- Produce a single document with these sections, in this order:
-
+ A single document, in this order:
1. **Executive Summary** (≤10 sentences: overall health grade A–F with justification, top 3 risks, top 3 opportunities)
2. **Repo Map**
3. **Audit Report**
4. **Improvement Strategy**
5. **Task Plan** (milestones + task table + quick wins)
- 6. **Open Questions** (anything you need a human to decide: product intent, deprecation candidates, performance targets)
+ 6. **Open Questions** (what you need a human to decide: product intent, deprecation candidates, performance targets)
- The report is usually long. Present the Executive Summary inline, and offer to write the full document to a markdown file (default `AUDIT.md` at the repo root, or a path the user prefers) so it's preservable and diffable. Writing that report file is the only write this skill performs.
+ Present the Executive Summary inline, and offer to write the full document to a markdown file (default `AUDIT.md` at the repo root, or a path the user prefers). That report is the only write this skill performs in the target repo; keep internal prep artifacts (the Repo Map, the tailored checklist) in the session scratchpad, not the repo.
+ ## Safety
+
+ Strictly analysis-only. Never modify code, config, tests, dependencies, git state, or any external system. The severity table's recommended actions are advice for a human to apply later, never instructions to auto-fix. The one write in the target repo is the audit report.
+
+ ## Reference files
+
+ - `references/audit-rubric.md` — the general, language-agnostic audit: architecture, security (with CWE identifiers), async/state, logic, quality, testing, performance, dependencies, docs, plus the severity classification. Read it for the exact definition of a check or to cite a severity.
+ - `references/ai-codegen-antipatterns.md` — the AI-generated-code lens: the failure modes that are characteristic of iteratively AI-built code. Lean on it hardest when auditing a vibe-coded or AI-assisted project.
+
## Calibration reminders
- - Don't recommend enterprise-grade infrastructure for a weekend prototype unless the owner's goals demand it.
+ - The grade reflects maturity-adjusted health, not an absolute ideal. A solid prototype can earn a B; a fragile production service can earn a D. Justify the grade against what the project is trying to be.
- If a dimension is healthy, one sentence and move on.
- - If the repo is large, prioritize depth in the core 20% of the code, and name which areas got lighter review.
- - The grade in the Executive Summary should reflect maturity-adjusted health, not an absolute ideal. A solid prototype can earn a B; a fragile production service can earn a D. Justify the grade against what the project is trying to be.
+ - Prioritize depth in the core 20% of the code, and name which areas received lighter review so the coverage boundaries are explicit.