codebase-audit · diff
git:20260612.95a036c to git:20260703.eebadde
97 added, 157 removed. Audit A to A.
---
name: codebase-audit
- description: "全面代码库审计 — 自适应并行深度分析（前后端契约、数据完整性、异常处理/安全、架构/技术债、配置/缓存），输出按严重程度排序的统一报告和修复路线图。Use when user asks to audit, analyze, or review an entire codebase for design issues, find hidden bugs, check architecture health, or asks '全面审查', '代码库审计', '分析设计问题', 'audit codebase', 'health check', '有哪些问题'. Also trigger when user asks to find silent degradation, data flow breakpoints, type mismatches between frontend and backend, or wants to understand technical debt across a project."
+ description: "全面代码库审计 — 自适应并行深度分析（前后端契约、数据完整性、异常处理/安全、架构/技术债、配置/缓存），结构化 findings + 对抗验证 + 基线对比，输出按严重程度排序的统一报告和修复路线图。支持 quick 快速体检模式。Use when user asks to audit, analyze, or review an entire codebase for design issues, find hidden bugs, check architecture health, or asks '全面审查', '代码库审计', '分析设计问题', 'audit codebase', 'health check', '有哪些问题', '快速体检'. Also trigger when user asks to find silent degradation, data flow breakpoints, type mismatches between frontend and backend, or wants to understand technical debt across a project."
---
# Codebase Audit — Adaptive Deep Analysis
- A comprehensive codebase audit that adapts its agent configuration to the project's tech stack. Each agent uses the highest-tier model for maximum thoroughness. Findings are verified before compilation, then compiled into a unified report sorted by severity with a phased repair roadmap.
+ Comprehensive codebase audit that adapts its agent configuration to the project's tech stack, forces structured findings, adversarially verifies Critical/High findings before they enter the report, diffs against the previous audit's ledger (resolved / still-open / new), and outputs a severity-sorted report plus a phased repair roadmap.
## Core Principles
- 1. **Highest-tier model** — All audit agents MUST use the highest-capability model available in the current environment (e.g. `fable`; `opus` acceptable as fallback). Never use lightweight tiers (haiku) for audit agents. If the highest tier is unavailable, fall back one tier and note the downgrade in the report header.
- 2. **Depth over breadth** — Fewer agents with broader scope and deeper analysis beats many shallow agents. Each agent should trace issues across file boundaries.
- 3. **Adaptive** — Agent count and focus areas vary by project type. Don't waste an agent on "frontend rendering" for a backend-only project.
- 4. **Verify before report** — No Critical/High finding enters the report unverified. LLM audit agents misreport intentional patterns (deliberate fallbacks, feature gates) as defects; Phase 1.5 exists to catch that.
-
- ## When to Use
-
- - User asks to audit/review/analyze an entire codebase
- - User wants to find hidden bugs, silent degradation, or design inconsistencies
- - User asks about technical debt, architecture health, or "what's broken"
- - Before a major refactor or after inheriting an unfamiliar codebase
- - Periodic health check (monthly/quarterly) — see Phase 3 for baseline diffing
-
- ## Workflow
-
- ### Phase 0: Tech Stack Detection & Deterministic Scans
-
- Detect the project's tech stack to determine the agent configuration:
-
- ```
- Detection checklist:
- - package.json / tsconfig.json → TypeScript/JavaScript (React, Next.js, Vue, etc.)
- - pyproject.toml / requirements.txt / setup.py → Python (FastAPI, Django, Pydantic, etc.)
- - Cargo.toml → Rust (serde, axum, actix, etc.)
- - go.mod → Go (gin, echo, gorm, etc.)
- - Multiple stacks → Full-stack project (frontend + backend)
- - Async runtime signals: Cargo.toml contains tokio/async-std, go.mod present,
- asyncio imports → enable the Concurrency & Async Hygiene section for the
- Data Integrity agent (see agent-prompts.md)
- ```
+ 1. **READ-ONLY** — Audit agents must never create, modify, or delete files in the target. Every agent prompt starts with the read-only preamble in `references/agent-prompts.md`.
+ 2. **Inherit the session model** — Omit the `model` param on all agents so they inherit the session model (usually the strongest available). Only override *upward* if the session model is clearly weak for cross-file reasoning. Never hardcode a specific model name in this skill.
+ 3. **Depth over breadth** — Fewer agents with broader merged scopes beat many shallow agents. Each agent traces issues across file boundaries.
+ 4. **Adaptive** — Agent count and dimensions vary by stack and mode.
+ 5. **Verified findings** — Critical/High findings must survive an adversarial verify pass. Medium findings pass through but are labeled `unverified` in the report.
- After stack detection, run the matching dependency audit from the target project root, never from the assistant's incidental current working directory. If the user supplied an explicit target path, use it as `{TARGET_DIR}` before invoking the tool. For Python projects, run each matching audit input that exists and feed all outputs to the security agent (deterministic, zero LLM cost):
+ ## Operating Contract
- ```
- - Rust → cd "{TARGET_DIR}" && cargo audit
- - Node → cd "{TARGET_DIR}" && npm audit
- - Python project metadata (`pyproject.toml` / `setup.py`) → cd "{TARGET_DIR}" && pip-audit .
- - Python requirements (`requirements.txt`) → cd "{TARGET_DIR}" && pip-audit -r requirements.txt
- - Python environment fallback (no project files found) → cd "{TARGET_DIR}" && pip-audit .
- - Go → cd "{TARGET_DIR}" && govulncheck ./...
- ```
+ - Direct actions: read-only inspection, local dependency audits, report writing under the target, and ledger updates under `<target>/.audit/` after the user invokes this skill.
+ - Escalate before: editing audited project source files, dependency manifests, `.gitignore`, CI config, remote issues, PR state, or anything outside the requested audit/report scope.
+ - Evidence-backed pushback: challenge "all clear" or "resolved" only with file evidence, dependency-audit output, verifier results, or ledger spot-checks.
+ - Feedback loop: promote repeated misses into prompt updates, ledger matching rules, or fixture eval cases rather than leaving them as session-only notes.
- Feed the raw output to the Error Handling & Security agent (frontend-only: to Agent 2) for classification: Critical = RCE-grade CVE with PoC on a reachable path; High = known vuln on a reachable path. If the tool is unavailable, the report MUST state "依赖审计降级跳过" — never omit silently.
+ ## Gotchas
- ### Phase 1: Launch Agents (Adaptive)
+ - Dependency-audit commands must run from `{TARGET_DIR}`, not the assistant's incidental cwd.
+ - Finder agents must not read `evals/expected-findings.json` or eval README files when auditing the planted-bug fixture.
+ - A previous ledger miss is not proof that a finding was resolved; spot-check the file before marking an old finding `resolved`.
- Based on the detected stack, choose the appropriate agent configuration below. Launch ALL agents in a SINGLE message, each with the model tier defined in Core Principles #1.
+ ## Modes
- Read `references/agent-prompts.md` for complete prompt templates. Every prompt MUST be self-contained: target path, stack info, and the Unified Output Contract (defined at the top of agent-prompts.md) — sub-agents cannot see this file.
+ | Mode | Trigger | Agents | Verify pass | Ledger |
+ |------|---------|--------|-------------|--------|
+ | **full** (default) | plain invocation, "全面审查" | 3–5 by stack (+ optional dims) | yes | yes |
+ | **quick** | "quick" in args, "快速体检" | 2 (Silent Degradation & Security; Data Integrity & Registry) | no — all findings labeled `unverified` | yes |
- Before launching, verify every agent Type against the available subagent registry. If a type is not registered, STOP and report the missing type — do NOT substitute a similar one.
+ **Optional dimensions** (full mode only, enable when user asks or the repo obviously needs them):
+ - `tests` — test quality: assertion strength, skip markers, coverage of critical paths (Agent 6)
+ - `concurrency` — races, blocking calls in async, leaked tasks/goroutines (Agent 7)
- ---
+ ## Workflow
- #### Full-Stack Projects (5 agents)
+ ### Phase 0: Detect & Prepare
- When both frontend and backend exist (e.g., React + FastAPI, Next.js + Go).
+ 1. **Stack detection**: `package.json`/`tsconfig.json` → TS/JS; `pyproject.toml`/`requirements.txt` → Python; `Cargo.toml` → Rust; `go.mod` → Go; multiple → full-stack.
+ 2. **Size estimate**: `tokei <target>` (fallback: `find <target> -name '*.<ext>' | xargs wc -l`), excluding vendored/generated code. If effective size ≥ 400K LOC, split each agent's scope by top-level directory and note the split in the report.
+ 3. **Exclusions** (always, in every agent prompt): `node_modules/`, `vendor/`, `target/`, `dist/`, `build/`, `.git/`, lockfiles, generated code.
+ 4. **Ledger**: read `<target>/.audit/findings.json` if it exists — this is the previous audit baseline (format: `references/ledger-format.md`).
+ 5. **Deterministic dependency audit**: from `{TARGET_DIR}` (never the assistant's incidental cwd), run each matching tool and feed raw output to the Error Handling & Security prompt:
+ - Rust: `cargo audit`
+ - Node: `npm audit`
+ - Python metadata (`pyproject.toml` / `setup.py`): `pip-audit .`
+ - Python requirements (`requirements.txt`): `pip-audit -r requirements.txt`
+ - Python fallback with no project files: `pip-audit .`
+ - Go: `govulncheck ./...`
+ If a required tool is unavailable, the report must state `依赖审计降级跳过: <tool>`; never omit the degradation silently.
- | # | Agent | Type | Scope (merged dimensions) |
- |---|-------|------|---------------------------|
- | 1 | **Frontend-Backend Contract** | `senior-code-reviewer` | Type consistency (field names, types, missing fields) + Rendering pipeline (layout/block/card routing completeness, dead slots, unrendered fields) + Serialization boundaries (models that silently drop fields). This agent reads BOTH sides and traces data across the API boundary. |
- | 2 | **Data Integrity & Flow** | `senior-code-reviewer` | Data pipeline end-to-end: from input through every transformation layer to output. Covers: field resolver filters, serialization/deserialization, model_validate/model_dump, cache read/write symmetry, registry key-set alignment (sole owner — Agent 4 must not re-check). Also covers: declaration-execution gaps (registered but unwired handlers, enum without config), and Concurrency & Async Hygiene when an async runtime is detected. |
- | 3 | **Error Handling & Security** | `security-auditor` | Exception patterns (bare except, debug-level errors, warning+fallback), security (hardcoded secrets, injection, unsafe deserialization), silent degradation (error paths that produce user-visible wrong output instead of failing), classification of Phase 0 dependency-audit output. |
- | 4 | **Architecture & Code Quality** | `code-archaeologist` | Layer violations, circular dependencies, god objects (files >800 lines), code duplication (parallel systems, scattered mapping tables), extension cost analysis (how many files to add a new type), DI pattern consistency, test quality (coverage gaps, weakened assertions, skip markers, stale tests). |
- | 5 | **Config & Persistence** | `code-archaeologist` | Config completeness (template/schema vs code expectations, conflicting defaults), cache key completeness (missing code version dimension), DB schema consistency, temp file cleanup, state persistence across restarts. |
+ ### Phase 1: Assemble Dimensions
- ---
+ Pick the configuration by detected stack. Full prompt templates in `references/agent-prompts.md`; prepend the read-only preamble and inject `{TARGET_DIR}` / `{STACK_INFO}` into each.
- #### Backend-Only Projects (4 agents)
+ **Full-Stack (5 agents)** — frontend + backend both present:
- When only backend exists (Python API, Rust service, Go microservice, etc.)
+ | # | Dimension | Scope (merged) |
+ |---|-----------|----------------|
+ | 1 | Frontend-Backend Contract | Type consistency + rendering pipeline + serialization boundaries. Reads BOTH sides. |
+ | 2 | Data Integrity & Flow | End-to-end pipeline tracing, field dropping, declaration-execution gaps, registry coverage alignment. |
+ | 3 | Error Handling & Security | Silent degradation, exception patterns, secrets, injection, unsafe deserialization. |
+ | 4 | Architecture & Code Quality | Layer violations, god objects, duplication/drift, extension cost, registry cross-reference. |
+ | 5 | Config & Persistence | Config completeness, cache key/integrity, DB schema, temp files, state persistence. |
- | # | Agent | Type | Scope |
- |---|-------|------|-------|
- | 1 | **API Contract & Data Integrity** | `senior-code-reviewer` | API schema vs internal models, serialization boundaries, data pipeline tracing, field dropping, registry key-set alignment, declaration-execution gaps, concurrency hygiene when async runtime detected. |
- | 2 | **Error Handling & Security** | `security-auditor` | Same as full-stack Agent 3. |
- | 3 | **Architecture & Code Quality** | `code-archaeologist` | Same as full-stack Agent 4. |
- | 4 | **Config & Persistence** | `code-archaeologist` | Same as full-stack Agent 5. |
+ **Backend-Only (4 agents)**: replace #1 with "API Contract & Data Integrity" (which absorbs #2's data-flow/registry scope — do NOT also dispatch #2); keep #3–#5.
+ **Frontend-Only (3 agents)**: Component Architecture & Rendering; Error Handling & Code Quality; Config & Build.
+ **Quick mode (2 agents)**: Silent Degradation & Security (= #3); Data Integrity & Registry (= #2 core).
- ---
+ Fallback-path agent types (when using the Agent tool instead of Workflow): agent availability is environment-specific — check the subagent registry visible in the current session and use only type names that appear there. Never invent aliases (there is no generic `reviewer` type). If no specialized type matches, use `general-purpose` (or the environment's default catch-all) for every dimension; the prompts are self-contained. See the example mapping in `references/agent-prompts.md`.
- #### Frontend-Only Projects (3 agents)
+ ### Phase 2: Orchestrate
- When only frontend exists (React SPA, Vue app, etc.)
+ **Preferred — Workflow tool** (skill invocation is the user's opt-in): use the script in `references/workflow-template.md`. It schema-forces every finder's output into structured findings, then pipelines each dimension's Critical/High findings straight into adversarial verify agents (no barrier — verification starts while other dimensions are still scanning).
- | # | Agent | Type | Scope |
- |---|-------|------|-------|
- | 1 | **Component Architecture & Rendering** | `senior-code-reviewer` | Type routing completeness, component registration gaps, dead props/slots, state management consistency, API consumption patterns, minimal accessibility checks (alt/label/keyboard reachability). |
- | 2 | **Error Handling & Code Quality** | `senior-code-reviewer` | Unhandled promise rejections, error boundaries, catch-and-ignore patterns, god components, code duplication, test quality, dependency-audit classification. |
- | 3 | **Config & Build** | `code-archaeologist` | Build config consistency, env variable management, bundle analysis, dead dependencies. |
+ **Fallback — Agent tool** (if Workflow is unavailable): launch all finder agents in a SINGLE message; prompts already demand the same JSON output. After they return, launch one verify agent per Critical/High finding (also batched in one message), using the verify prompt from `references/workflow-template.md`.
- ---
+ ### Phase 3: Dedup & Ledger Diff
- ### Phase 1.5: Verify Findings
+ Dedup:
+ - Same file + same line → merge.
+ - Same root cause found by multiple agents → keep the most detailed, note cross-agent confirmation (raises confidence).
+ - Severity conflicts → use the highest.
- For every Critical and High finding:
+ Verification results:
+ - `confirmed=false` findings do NOT enter the main report; list them in an appendix "Refuted by verification" with the refutation reason (keeps the work auditable).
+ - Verifier failed/absent → keep the finding, label `unverified`.
- 1. Spawn 1-2 verifier agents (same model tier). Input = ONLY the finding's claim + involved file paths (minimal context).
- 2. The verifier must actively search for counter-evidence: test coverage, feature flags, intentional-design comments, call sites that handle the "missing" case.
- 3. Verdict: confirm / refute / uncertain, with reasons.
- 4. Refuted findings move to a report appendix "已排除项" with the refutation reason (do NOT delete).
- 5. Medium findings: spot-check ≥30%.
+ Ledger diff (skip if no previous ledger — everything is `new`):
+ - Match previous ↔ current findings by **(category, file, root-cause summary)** — never by line number (lines drift).
+ - Previous finding with no current match → open the file and spot-check before marking `resolved`; if still present but missed, re-add as `still-open`.
+ - Classify every current finding: `new` / `still-open`.
+ - Write the updated ledger to `<target>/.audit/findings.json`. If the repo is tracked and `.audit/` isn't ignored, suggest adding it to `.gitignore` (don't edit `.gitignore` yourself).
- Critical findings MUST be 已复核 before entering "Fix Immediately".
+ ### Phase 4: Report
- ### Phase 2: Compile Unified Report
+ Write the full report to `<target>/audit-report-YYYY-MM-DD.md`, then post a chat summary: counts per severity, top Criticals, ledger delta (N resolved / N still-open / N new), dependency-audit status, and the roadmap.
- After verification completes, compile findings into a single report.
+ Report body requirements:
+ - Use Chinese for problem descriptions, impact analysis, and repair advice; keep code identifiers, paths, and error messages in their original form.
+ - Separate each finding into fact / inference / recommendation: the finding itself is a fact with `file:line`; impact is an inference with confidence; repair advice is a recommendation with stated assumptions.
+ - Inferred-only findings cannot be higher than Medium unless a verifier confirms user-visible or security impact.
- > 报告正文（问题描述、影响分析、修复建议）一律使用中文；代码片段、文件路径、标识符、错误消息保留原文。
- >
- > 每条发现按三层呈现：发现本体 = 事实（带 file:line）；影响分析 = 推断（带置信度，evidence_type 为 inferred 的不得高于 medium）；修复建议 = 建议（带前提假设）。
+ Report structure:
```markdown
- # [Project Name] Codebase Audit Report
-
- > Audit date: YYYY-MM-DD
- > Target: path
- > Tech stack: detected stack
- > Agents: N (list agent names) | Model tier: (note any downgrade)
- > Dependency audit: ran / 依赖审计降级跳过
+ # [Project] Codebase Audit Report
+ > Date / Target / Stack / Mode / Agents / Dependency audit / Previous audit: date or "none"
## Summary
- | Level | Count | Key Areas |
- |-------|-------|-----------|
- | Critical | N | ... |
- | High | N | ... |
- | Medium | N | ... |
- | Low | N | ... |
-
- ## Critical (Fix Immediately)
- | # | Problem | Agent | Impact | evidence_type | confidence | 验证状态 |
- |---|---------|-------|--------|---------------|------------|----------|
- For each: file:line, code snippet, risk description (推断需标注置信度), fix suggestion (标注前提).
+ | Level | Count | Verified | Key Areas |
- ## High (Fix This Week)
- ### [Category]
- [Same columns as Critical]
+ ## Delta vs Previous Audit (omit if first audit)
+ Resolved: N (list) | Still-open: N | New: N
- ## Medium (Plan to Fix)
- [Same structure]
+ ## Critical (Fix Immediately)
+ Per finding: file:line, code snippet, risk, fix suggestion, verify status.
- ## Low (Informational)
- [Same structure; include only if Low findings exist]
+ ## High / P1 (Fix This Week) — grouped by category
+ ## Medium / P2 (Plan to Fix) — labeled unverified where applicable
- ## 已排除项 (Refuted in Phase 1.5)
- | # | Original claim | Refutation reason |
- |---|----------------|-------------------|
+ ## Refuted by Verification — appendix: finding + refutation reason
## Repair Roadmap
| Phase | Scope | Est. Files |
- |-------|-------|------------|
- | Phase 0 (urgent) | Critical fixes | ~N files |
- | Phase 1 (this week) | High priority | ~N files |
- | Phase 2 (next week) | Medium priority | ~N files |
- | Phase 3 (ongoing) | Low / Architecture | ~N files |
-
- Est. Files = 该级别所有发现 files 字段去重并集大小；若修复涉及发现位置之外的文件，可追加估算但必须标注「推断，置信度低」。
```
- ### Deduplication
-
- All agents emit the Unified Output Contract table (see agent-prompts.md), so the severity scale and the `files` dedup key are consistent across agents:
-
- - Same file + same line → merge
- - Same root cause found by multiple agents → keep the most detailed one, note cross-agent confirmation (this actually increases confidence)
- - Severity conflicts → use the highest (scales are unified, so this is mechanical)
-
- ### Severity Classification
-
- The authoritative severity rubric lives in the **Unified Output Contract** at the top of `references/agent-prompts.md` — it must be inside the prompts because sub-agents cannot see this file. Do not redefine it here; Phase 2 uses the same four levels (Critical / High / Medium / Low).
-
- ### Phase 3: Baseline Diff (periodic health checks)
-
- Persist artifacts after every audit:
-
- - `<target>/.audit/report-YYYY-MM-DD.md`
- - `<target>/.audit/findings-YYYY-MM-DD.json` — the structured finding table
- - Stable finding ID: `hash(category + 归一化文件路径 + title 关键词)` (robust to line drift)
- - `<target>/.audit/false-positives.json` — user-marked false positives; filter or downweight matching findings during the next compile step
-
- When a previous `findings-*.json` exists, add a baseline diff section:
-
- | 状态 | 定义 |
- |---|---|
- | 已修复 | 上次有、本次无（需复核确认，不默认已修） |
- | 仍存在 | 两次都有（标注存活轮数） |
- | 新增 | 本次新出现 |
-
- Summary 表增加环比列。
-
- ## Stack-Specific Patterns
+ ## Severity Classification
- Read `references/stack-patterns.md` for technology-specific search patterns.
+ | Level | Criteria |
+ |-------|----------|
+ | **Critical** | Data loss, rendering failure, security vulnerability, complete feature breakage affecting users NOW |
+ | **High/P1** | Silent degradation (user sees wrong/incomplete output), type mismatches causing data truncation, missing config causing empty output, architectural violations blocking development |
+ | **Medium/P2** | Code duplication, inconsistent patterns, suboptimal error handling, tech debt that slows development but doesn't break features |
- ## Evals
+ ## References
- `evals/evals.json` contains trigger/output test cases — keep `expected_output` in sync whenever the agent configuration or report format changes (a past 10-agent → 5-agent refactor left them stale and permanently failing).
+ - `references/agent-prompts.md` — read-only preamble + prompt templates (Agents 1–7)
+ - `references/stack-patterns.md` — per-stack search patterns
+ - `references/workflow-template.md` — Workflow script, finding/verdict schemas, verify prompt
+ - `references/ledger-format.md` — ledger JSON schema and matching rules
+ - `evals/` — planted-bug fixture; evals measure recall against `evals/expected-findings.json`