chat-mining · diff

git:20261010.c8614d2 to git:20261010.9ca2554

3 added, 3 removed. Audit A to A.

---
name: chat-mining
description: Use when asked to mine or analyze past agent session chats/transcripts (e.g. "what can we learn from workspace X's chats", "audit this workspace's history for process improvements", "what went wrong in that run") for defects, lessons, or process improvements — including friction: runs that reached a correct outcome through unnecessary exploration, self-corrections, or rework. Inventories session history, runs bounded themed searches, extracts candidate claims, then verifies each against ground truth (code, docs, git) before proposing changes — a transcript's narrative is a claim, not evidence. Files verified items via track-work only when track-work's own trigger applies. Not for reading one specific session's log to resume work.
---
# Chat mining — retrospective session analysis
## Why
Session transcripts accumulate hard-won lessons that die with the transcript. Mining recovers them — but every statement in a transcript is a **claim by the agent that wrote it**: optimistic, partial, or hallucinated. Verify against ground truth before acting.
A session's PATH carries as much signal as its outcome. A run that ended correctly can still have taken a wasteful route — unnecessary exploration, self-caused corrections, rework that an encoded rule, template, or test would have prevented. Friction is a finding even when nothing is broken and nothing is pending.
## Procedure
- 1. **Inventory** — `history list_sessions` for the workspace; note durations, turn counts, session names, and files-touched. **Watermark first (S-003):** read the workspace's `.agents/mining-ledger.md` for its mined-through marker — only sessions newer than the marker are candidates; a thematic sweep records its scope, never claiming exhaustive coverage. Scope: one workspace when the question is about that workspace; ALL workspaces when the question is about a cross-cutting process (billing, tooling, a skill's behavior) — the same lesson often surfaces independently in several. The listing pages at ~100 sessions — treat it as orientation, not completeness; the themed sweep is the coverage mechanism. Triage signals: long many-file sessions are usually implementation runs (their recommendations have often already landed — verify before proposing), short single-turn sessions are usually advice/decision runs ( their recommendations are often still pending). Stale-index self-heal: if `history` reports skipped stale session indexes for a target workspace, rebuild by opening it once in the app — `open "repoprompt-ce://open/<repo-path>?workspace=<name>&focus=false"` (the workspace-restore path rewrites `AgentSessionIndex.json` from the authoritative session files; never hand-edit indexes) — then re-run the inventory (the cross-workspace cache lives ~90s).
+ 1. **Inventory** — `history list_sessions` for the workspace; note durations, turn counts, session names, and files-touched. **Watermark first (S-003):** read the workspace's `.agents/mining-ledger.md` for its mined-through marker — only sessions newer than the marker are candidates; no ledger yet means the workspace has never been mined: treat every session as a candidate and create the ledger at close.; a thematic sweep records its scope, never claiming exhaustive coverage. Scope: one workspace when the question is about that workspace; ALL workspaces when the question is about a cross-cutting process (billing, tooling, a skill's behavior) — the same lesson often surfaces independently in several. The listing pages at ~100 sessions — treat it as orientation, not completeness; the themed sweep is the coverage mechanism. Triage signals: long many-file sessions are usually implementation runs (their recommendations have often already landed — verify before proposing), short single-turn sessions are usually advice/decision runs ( their recommendations are often still pending). Stale-index self-heal: if `history` reports skipped stale session indexes for a target workspace, rebuild by opening it once in the app — `open "repoprompt-ce://open/<repo-path>?workspace=<name>&focus=false"` (the workspace-restore path rewrites `AgentSessionIndex.json` from the authoritative session files; never hand-edit indexes) — then re-run the inventory (the cross-workspace cache lives ~90s).
2. **Deterministic pre-pass, then themed sweep before sequential reads** — rank the candidates WITHOUT transcript reads first (S-004): themed-search hit count, turn-count vs files-touched ratio (high turns over few files = exploration-heavy), duration, and 1-turn decision-shaped sessions lead; LLM reads begin with the top-ranked. Then search ALL candidate sessions for retrospective language ("lesson", "takeaway", "next time", "improve", "hook", "gate", "blocked", "decision", "your call", "say the word", "want me to", plus domain terms), then read the hit sessions in bounded windows. The decision-language terms matter: a session that ENDS on an unanswered decision request or offer is where recoverable pending work lives — the improvement existed, was never approved, and dies with the transcript unless mined. Also sweep for FRICTION language — "no-op", "construction error", "re-read", "accidental", "re-run", "regenerat", "mis-read", "fixing my own", "again" — and remember friction is often NOT narrated at all: read the transcript's structure — repeated failed calls, the same file read multiple times, suite re-runs after self-introduced defects, multiple regenerations of the same artifact, high turn counts over small landed diffs. Read the FINAL turn of each hit session first (`around_turn` = last, minimal context) before deciding whether to read deeper. Mechanics: search results cap at ~50 snippets over a bounded scan budget (raise `max_sessions_scanned` for full-workspace coverage); snippets locate sessions, bounded `get_session` windows read them. Prioritization tactics (not rules): incident/review/orchestrator sessions and final-turn summaries are often high-yield; themed searching usually converges faster than sequential reading.
- 3. **Verify every claim** — re-check each claimed defect/lesson against current code, docs, and git before proposing fixes; a transcript's "done/fixed/verified" is never fact by itself (the inbound form of the Verifying Delegated Work rule). This includes re-validating that a recommended change is still needed — code may have moved since the session. For friction findings, judge the PATH, not the outcome: a correct final state does not excuse the detour. But separate friction from irreducible discovery — exploration is waste only when a durable artifact (rule, template, test, tooling, encoded procedure) would have prevented it; the deliverable is that artifact, and the discriminator keeps the finding honest.
+ 3. **Verify every claim** — re-check each claimed defect/lesson against current code, docs, and git before proposing fixes; a transcript's "done/fixed/verified" is never fact by itself (the inbound form of the Verifying Delegated Work rule). This includes re-validating that a recommended change is still needed — code may have moved since the session. For friction findings, judge the PATH, not the outcome: a correct final state does not excuse the detour. But separate friction from irreducible discovery — exploration is waste only when a durable artifact (rule, template, test, tooling, encoded procedure) would have prevented it; the deliverable is that artifact, and the discriminator keeps the finding honest. When a subagent or scripted probe performs the validation (S-011), it must include a negative control — the check must fail on the un-fixed state or stay silent on an adjacent no-trigger case, so it can discriminate rather than rubber-stamp — its report is verified independently (never accepted as-is), and destructive testing requires explicit user permission for that specific action; read-only routes (dry-run, read-back, isolated fixtures) are always preferred.
4. **Map each lesson to its durable home**, cheapest-first with per-request cost in mind — `global.md` loads on EVERY agent request, so it is the most expensive home: prefer a line in an existing skill or a documentation file whenever the guidance can be applied consistently there (an existing skill's line → a doc → a repo issue via track-work when its trigger applies → a new skill → only then a `global.md` corollary on an existing rule, minimal text, never a new section when a corollary suffices). Respect the Minimalism bar and global.md's cheapest-sufficient-home rule. Classify each artifact per your organization's repo-placement policy — full match (exists only for one project) homes in that project's repo; partial match (shared beyond one project) homes in one of two shared repos: the public machinery repo for generic, public-appropriate artifacts — non-public content never goes there — or the organization's own repo for org-specific skills and docs. Resolve both at runtime from symlink targets: a global/shared link (`readlink ~/.claude/rules/global.md`, `~/.claude/skills/track-work`) points at the public machinery repo; the private rules-overlay link beside it points at the org repo. The scope ladder, cheapest first: the affected repo's own memory/contributing docs (repo-local convention) → a shared skill or workflow in the matching shared repo → `global.md` last (per-request cost; corollary on an existing rule, never a new section when a corollary suffices).
- 5. **Retention map at close** — record where each finding landed or is recommended to land (commit / issue / doc / rule / skill) with its full/partial classification. Nothing of substance should remain only in a chat; a lesson left only in the transcript is lost. File or update a tracking issue for anything not immediately actionable (with user authorization) so no validated finding dies in the report. **Watermark write (S-003/S-012):** advance the workspace's mined-through marker in `.agents/mining-ledger.md` only on a completed sweep (last fully-processed session + timestamp + scope), set next-due (quarterly or ~150 new sessions, whichever first), and report cadence status — current, or overdue with the accumulated-session count.
+ 5. **Retention map at close** — record where each finding landed or is recommended to land (commit / issue / doc / rule / skill) with its full/partial classification. Nothing of substance should remain only in a chat; a lesson left only in the transcript is lost. File or update a tracking issue for anything not immediately actionable (with user authorization) so no validated finding dies in the report. **Watermark write (S-003/S-012):** advance the workspace's mined-through marker in `.agents/mining-ledger.md` only on a completed sweep (last fully-processed session + timestamp + scope), creating the ledger on the workspace's first completed sweep, set next-due (quarterly or ~150 new sessions, whichever first), and report cadence status — current, or overdue with the accumulated-session count.
6. **Report shape** — rank validated improvements by impact × effort, quick wins called out as a tier; label each with its validation status — still-pending, already-landed, or user-gated — and include a short "already made, skip these" section so nobody re-proposes landed work; keep user-gated items visible but separate from agent-actionable ones.
## Hard rules
- Bound the read: history windows and max_chars per call; stop when themed searches stop surfacing new sessions.
- Deliver findings and retention recommendations; create tracking items or edit files only under the applicable skill/trigger and user authorization.
- Repo-independence: the mined workspace is provenance, never content — its name, session IDs, and incidents do not enter a resulting rule's or skill's own text except as explicitly labeled provenance citations. Generalize each lesson before recording it; if it cannot be stated without the source workspace, it is a full match and belongs in that workspace's repo, not in global machinery.