doc-impact · diff
git:20260409.df814ae to git:20260511.125144f
41 added, 47 removed. Audit A to A.
What this update changes in the checks
No check changed: every fixed check gives the same answer before and after this update.
A tripped check is a label, not a verdict: read the matched text and the lines below. This week's updates that newly trip a check.
---
name: doc-impact
description: Context health analysis — freshness, domain coverage, hub surfacing, drift detection, LLM-powered interpretation, and auto-repair for generated agent context
+ triggers:
+ files:
+ - src/commands/doc-impact.js
+ - src/lib/impact.js
+ - src/prompts/impact-analyze.md
+ - tests/impact.test.js
+ - tests/doc-impact.test.js
+ keywords:
+ - impact
+ - freshness
+ - coverage
+ - drift
+ - health score
+ - context health
+ - hook health
+ - usefulness
---
- ## Activation
-
- This skill triggers when editing doc-impact files:
- - `src/commands/doc-impact.js`
- - `src/lib/impact.js`
- - `src/prompts/impact-analyze.md`
- - `tests/impact.test.js`
- - `tests/doc-impact.test.js`
-
- Keywords: impact, freshness, coverage, drift, health score, context health, hook health, usefulness
+ You are working on **doc impact** — the command that shows whether generated agent context is keeping up with the codebase, optionally interprets results via LLM, and can interactively apply recommended repairs.
- ---
+ ## Domain purpose
+ Audits the agent context generated by aspens against the live source tree, reports per-target health (Claude, Codex), and offers an interactive `--apply` flow that re-runs the right `aspens doc init/sync` variant to repair gaps. The LLM interpretation is a thin layer on top of deterministic metrics — the metrics are the contract.
- You are working on **doc impact** — the command that shows whether generated agent context is keeping up with the codebase, optionally interprets results via LLM, and can interactively apply recommended repairs.
+ ## Business rules / invariants
+ - **Target inference:** If `.aspens.json` is absent, targets are inferred from scan results (`.claude/` → claude, `.agents/` → codex). Falls back to `['claude']`.
+ - **LLM is optional and tool-less.** Runs only when a CLI backend is detected. `runLLM` is invoked with `disableTools: true`; prompt must return pure JSON (`bottom_line`, `improves`, `risks`, `next_step`). Failure is caught and reported as "Analysis unavailable" — never fatal.
+ - **Graph failure is non-fatal.** If `buildRepoGraph` throws (or `--no-graph` is passed), `graph` is `null` and hub coverage is skipped/`n/a`.
+ - **Hub coverage haystack is the code-map, not AGENTS.md.** Post-Phase 1, `## Key Files` no longer lives in root instructions. `computeHubCoverage` reads `.claude/code-map.md` (claude) or `.agents/skills/architecture/references/code-map.md` (codex). If the code-map file is missing, it reports `codeMapMissing: true` instead of spurious "missing hub" warnings. Older callers without `repoPath` fall back to the legacy `contextText` haystack.
+ - **Health score deductions** (start 100): missing instructions −35; no skills −25; domain gaps proportional up to −25; missed hubs −4 each; drift −3 per file (cap −20); unhealthy hooks −10 (hook-capable targets only); broken save-tokens −5 (claude only).
+ - **`LOW_SIGNAL_DOMAIN_NAMES`** (`config`, `test`, `tests`, `__tests__`, `spec`, `e2e`) are excluded from coverage scoring but tracked in `excluded`.
+ - **`SOURCE_EXTS`** extends the scanner set with `.scala`, `.clj`, `.elm`, `.vue`, `.svelte` for drift detection. Adding a language for drift requires updating this set.
+ - **Walk depth capped at 5** — deep nested source files won't appear in drift analysis.
- ## Key Files
- - `src/commands/doc-impact.js` — CLI command: calls `analyzeImpact()`, renders per-target report with health scores, coverage, drift, usefulness, hook health, save-tokens health, LLM interpretation, opportunities, and interactive apply confirmation
- - `src/lib/impact.js` — Core analysis: `analyzeImpact()` orchestrates scan + config + graph + per-target summarization; exports `evaluateHookHealth()`, `evaluateSaveTokensHealth()`, `summarizeOpportunities()`, `summarizeValueComparison()`, `summarizeMissing()`
- - `src/prompts/impact-analyze.md` — System prompt for LLM-powered impact interpretation (returns JSON with `bottom_line`, `improves`, `risks`, `next_step`)
- - `tests/impact.test.js` — Unit tests for coverage, drift, health score, status, report summarization, value comparison, missing rollup, hook health, save-tokens health, opportunities
- - `tests/doc-impact.test.js` — Unit tests for `buildApplyPlan()` and `buildApplyConfirmationMessage()`
+ ## Non-obvious behaviors
+ - **Agent skill-ref check (Phase 6):** `checkAgentSkillReferences()` scans `.claude/agents/*.md` frontmatter for `skills: [a, b]` and verifies each `.claude/skills/<name>/skill.md` exists. Broken refs surface as `agent-skill-refs` opportunities.
+ - **Save-tokens health is Claude-only** and only activates when `config.saveTokens.enabled` is true and `claude.enabled !== false`. Required hook/command files vary by sub-config (`saveHandoff`, `warnAtTokens`/`compactAtTokens` thresholds). Legacy `.mjs` siblings of the `.sh` hooks must be cleaned up — their presence is an issue.
+ - **`buildApplyPlan` dedupes** with `aspens doc sync` as a target-agnostic key; everything else is keyed `${target.id}:${action}`.
+ - **`applyRecommendedAction` is a hand-maintained dispatch table** mapping action strings to `docInitCommand`/`docSyncCommand` option shapes. Adding a new recommendation in `recommendActions()` requires a matching branch here, otherwise it warns "Cannot apply automatically".
- ## Key Concepts
- - **`analyzeImpact(repoPath, options)`** — Main entry point. Runs `scanRepo()`, loads config from `.aspens.json`, infers targets if not configured, collects source file state, optionally builds import graph, then produces per-target reports. Now also computes `summary.opportunities`.
- - **Target inference:** If no `.aspens.json` config, infers targets from scan results (`.claude/` → claude, `.agents/` → codex). Falls back to `['claude']`.
- - **`summarizeTarget()`** — Per-target analysis: finds skills, evaluates hook health, evaluates save-tokens health (Claude only), checks instruction file existence, computes domain coverage, hub coverage, drift, usefulness, status, health score, and recommended actions.
- - **Domain coverage:** `computeDomainCoverage()` matches scan-detected domains against installed skills. Filters out `LOW_SIGNAL_DOMAIN_NAMES` (config, test, tests, __tests__, spec, e2e) from scoring — tracked in `excluded` field.
- - **Hub coverage:** `computeHubCoverage()` checks if top 5 graph hub file paths appear in the instruction file + base skill text.
- - **Drift detection:** `computeDrift()` finds source files modified after the latest generated context mtime. Maps changed files to affected domains via directory matching.
- - **Health score:** `computeHealthScore()` starts at 100, deducts for: missing instructions (-35), no skills (-25), domain gaps (up to -25), missed hubs (-4 each), drift (-3 per file, max -20), unhealthy hooks (-10 for Claude), broken save-tokens (-5 for Claude).
- - **Hook health:** `evaluateHookHealth(repoPath)` checks for required hook scripts, validates `settings.json` hook commands resolve to existing files.
- - **Save-tokens health:** `evaluateSaveTokensHealth(repoPath, saveTokensConfig)` checks if configured save-tokens installation is complete — validates required hook files, command files, legacy file cleanup, and settings.json entries. Returns `{ configured, healthy, issues, missingHookFiles, missingCommandFiles, invalidCommands, installedLegacyHookFiles }`.
- - **Opportunities:** `summarizeOpportunities(repoPath, targets, config)` identifies optional aspens features not yet installed: save-tokens, agents, agent customization, doc-sync hook. Each returns `{ kind, message, command }`. Displayed in the "Missing Aspens Setup" section.
- - **Usefulness summary:** `summarizeUsefulness()` produces `{ strengths, blindSpots, activationExamples }` per target.
- - **Value comparison:** `summarizeValueComparison(targets)` computes before/after metrics for the report header.
- - **Missing rollup:** `summarizeMissing(targets)` aggregates cross-target gaps including broken save-tokens installations with severity levels.
- - **LLM interpretation:** If CLI backend is available, sends report + comparison as JSON to `impact-analyze` prompt. `saveTokensHealth` included in the analysis payload.
- - **Interactive apply:** `buildApplyPlan(targets)` collects all recommended actions across targets with interactive confirmation.
+ ## Critical files (purpose, not inventory)
+ - `src/commands/doc-impact.js` — CLI rendering, LLM prompt assembly, `buildApplyPlan`, and the action dispatch into `docInitCommand`/`docSyncCommand`.
+ - `src/lib/impact.js` — All deterministic analysis (`analyzeImpact`, target summarization, scoring, drift, hub/code-map coverage, hook/save-tokens health, opportunities, missing rollup, value comparison).
+ - `src/prompts/impact-analyze.md` — Strict JSON contract the LLM must honor; do not change shape without updating `parseAnalysis`.
## Critical Rules
- - **LLM interpretation is optional** — runs only if a CLI backend is detected. Failure is caught and reported as "Analysis unavailable".
- - **LLM gets no tools** — `disableTools: true` passed to `runLLM()`. The prompt expects pure JSON output.
- - **`--no-graph` flag** — skips import graph build; hub coverage section shows `n/a`.
- - **Graph failure is non-fatal** — if `buildRepoGraph` throws, graph is set to null and analysis continues without hub data.
- - **`SOURCE_EXTS` set** — only these extensions count as source files for drift detection. Adding a language requires updating this set.
- - **Walk depth capped at 5** — deep nested source files won't appear in drift analysis.
- - **`LOW_SIGNAL_DOMAIN_NAMES`** — `config`, `test`, `tests`, `__tests__`, `spec`, `e2e` are excluded from domain coverage scoring but tracked in `excluded` array.
- - **Exported functions** — `computeDomainCoverage`, `computeHubCoverage`, `computeDrift`, `evaluateHookHealth`, `evaluateSaveTokensHealth`, `computeHealthScore`, `computeTargetStatus`, `recommendActions`, `summarizeReport`, `summarizeMissing`, `summarizeOpportunities`, `summarizeValueComparison` from `impact.js`; `buildApplyPlan`, `buildApplyConfirmationMessage` from `doc-impact.js`.
-
- ## References
- - **Patterns:** `src/lib/skill-reader.js` — `findSkillFiles()` used for skill discovery per target
- - **Prompt:** `src/prompts/impact-analyze.md`
+ - Skills/`activationPatterns` are matched via `findMatchingSkill` (substring or `/domain/` path hit). Renaming the skill-reader contract breaks coverage scoring.
+ - Don't surface root-context hub warnings when `codeMapMissing` is true — emit the `code-map-missing` item instead and tell the user to run `aspens doc graph`.
+ - Exported surface used by tests/consumers: `computeDomainCoverage`, `computeHubCoverage`, `computeDrift`, `evaluateHookHealth`, `evaluateSaveTokensHealth`, `computeHealthScore`, `computeTargetStatus`, `recommendActions`, `summarizeReport`, `summarizeMissing`, `summarizeOpportunities`, `summarizeValueComparison`, `checkAgentSkillReferences` from `impact.js`; `buildApplyPlan`, `buildApplyConfirmationMessage` from `doc-impact.js`.
---
- **Last Updated:** 2026-04-09
+ **Last Updated:** 2026-05-11