cf-sys-debug · git:20260827.debd979 · 2026-08-27 · sha256 93dcf21ec1b20b6f
cf-sys-debug git:20260827.debd979A
Immutable. This exact content is served forever at /api/v1/blob/93dcf21ec1b20b6f.
---
name: cf-sys-debug
description: >
Systematic 4-phase debugging — root cause, hypothesis testing, regression-guarded fix,
mandatory bug doc. Auto-invoke for non-trivial or recurring bugs — signals: "still
broken", "same error again", "came back", flaky/intermittent/race, "used to work, now
broken", hard-to-reproduce, works locally fails in CI, "find the root cause",
"investigate", "diagnose", "why is this happening". Prefer over cf-fix for hard bugs. Do
NOT auto-invoke for trivial typos, one-line fixes, or obvious config errors.
created: 2026-02-17
updated: 2026-08-27
disable-slash-command: true
---
# Systematic Debugging
> **CLI Requirement:** OPTIONAL — Uses the memory MCP from `coding-friend-cli` for fast indexed search and storage. Without the CLI: falls back to grep over `docs/memory/` and direct file writes. Full functionality preserved, slower memory recall. See [CLI requirements](../../../docs/cli-requirements.md).
## Custom Guide
Custom guide — auto-loaded below (if the raw command shows instead of its output, run it yourself):
```!
bash "<plugin-root>/lib/load-custom-guide.sh" cf-sys-debug
```
If output is not empty, integrate returned sections: `## Before` → before first step, `## Rules` → apply throughout, `## After` → after final step.
## Core Constraint
**Do not touch code until you can state the root cause in one sentence:**
> "I believe the root cause is [X] because [evidence]."
Name a specific file, function, and line. "A state management issue" is not testable. "Stale cache in `useUser` at `src/hooks/user.ts:42` because the dependency array is missing `userId`" is testable. If you cannot be that specific, you do not have a hypothesis yet.
**Same symptom after a fix = hard stop.** Both a recurrence and "let me just try this" mean the hypothesis is unfinished. Re-read the execution path from scratch before touching code again.
**After 3 failed hypotheses, stop.** Use the Handoff Format below to surface what was checked, ruled out, and unknown. Ask how to proceed.
## Rationalization Watch
When these surface, stop and re-examine:
| Thought | What it means | Rule |
| ----------------------------------- | ----------------------------------------- | --------------------------------------------------------------------------------- |
| "I'll just try this one thing" | No hypothesis, random-walking | Stop. Write the hypothesis first. |
| "I'm confident it's X" | Confidence is not evidence | Run an instrument that proves it. |
| "Probably the same issue as before" | Treating a new symptom as a known pattern | Re-read the execution path from scratch. |
| "It works on my machine" | Environment difference IS the bug | Enumerate every env difference before dismissing. |
| "One more restart should fix it" | Avoiding the error message | Read the last error verbatim. Never restart more than twice without new evidence. |
## Progress Signals
When these appear, the diagnosis is moving in the right direction:
| Thought | What it means | Next step |
| ---------------------------------------------------- | ----------------------------------- | ------------------------------------------------------------- |
| "This log line matches the hypothesis" | Positive evidence found | Find one more independent piece of evidence to cross-validate |
| "I can predict what the next error will be" | Mental model is forming | Run the prediction; if it matches, the model is correct |
| "Root cause is in A but symptoms appear in B" | Propagation path understood | Trace the call chain from A to B and confirm each link |
| "I can write a test that would fail on the old code" | Hypothesis is specific and testable | Write the test before applying the fix |
Do not claim progress without observable evidence matching at least one of these signals.
## 4-Phase Process + Documentation
### Phase 1: Root Cause Investigation
**1a. Check existing bug docs** (memory recall):
Before investigating, search for related past bugs. Extract 2-3 keywords from the bug description.
**Primary — Memory MCP** (if `memory_search` tool is available):
Call `memory_search` with: `{ "query": "<bug keywords>", "type": "episode", "limit": 3 }`
**Fallback — grep** (if memory MCP unavailable):
Check `{docsDir}` from `.coding-friend/config.json` (default: `docs`).
1. Grep `^description:` lines across `{docsDir}/memory/bugs/**/*.md` — match against bug keywords
2. If no match, grep `^tags:` lines across `{docsDir}/memory/bugs/**/*.md`
If matches found, read the top 1-2 matched files — they may reveal known root causes or patterns that save investigation time.
**1b. Investigate:**
1. **Read the actual error.** Do not guess. Read the full stack trace, error message, and logs.
2. **Reproduce the bug.** Write a test or command that triggers the failure reliably.
3. **Trace backward.** Follow the call chain from the error to its origin. The bug is usually NOT where the error appears.
### Phase 2: Pattern Analysis
1. **When did it start?** Check recent changes: `git log --oneline -20`, `git diff HEAD~5`
2. **Is it consistent?** Does it fail every time, or intermittently? Intermittent = timing/state issue.
3. **What's the minimal reproduction?** Strip away everything unrelated until you have the smallest case.
4. **Pay attention to deflection.** When someone says "that part doesn't matter," treat it as a signal. The area someone avoids examining is often where the problem lives.
### Bisect Mode
Activate when the symptom is "used to work, now broken" or "broke after an update". Random-walking forward from the current state wastes context and produces random fixes.
**Flow:**
1. Find `last-known-good` using the most recent tag where the behavior was correct: `git tag --sort=-version:refname | head -5`. Do not use a date or raw SHA as the anchor.
2. Define a pass/fail test command before starting. It must be runnable non-interactively and produce an unambiguous exit code. Write it once; reuse it at every step.
3. Run `git bisect start`, `git bisect bad` (current), `git bisect good <tag>`. Let bisect drive; do not jump ahead.
4. Context conservation: do not re-read large files at each step. Read once, note the key function or line, reference from notes.
5. When bisect names the culprit commit: read only that commit's diff, not surrounding history. Identify the specific line that introduced the regression.
### Phase 3: Hypothesis Testing
1. **Form one hypothesis** using the template: "The bug is caused by [X] because [evidence]." Name file and line.
2. **Add one targeted instrument:** a log line, a failing assertion, or the smallest test that would fail if the hypothesis is correct. Run it.
3. **If the evidence contradicts the hypothesis, discard it completely.** Re-orient with what was just learned. Do not preserve a hypothesis the evidence disproves.
4. **External tool failure: diagnose before switching.** When an MCP tool or API fails, determine why first (server running? API key valid? config correct?) before trying an alternative.
5. **Stack trace points into a library?** Walk back 3 frames into your own code. The bug is almost always there, not in the dependency.
### Phase 4: Implementation
1. **Fix the root cause**, not the symptom. If the fix touches more than 5 files, pause and confirm scope with the user.
2. **Regression Guard** — for any bug that recurred or was previously "fixed", the fix is not done until:
- A regression test exists that fails on the unfixed code and passes on the fixed code
- The test lives in the project's test suite, not a temporary file
- The commit message states why the bug recurred and why this fix prevents it
3. **Write a regression test** that would have caught this bug
4. **Run the full test suite** — your fix must not break anything else
5. **Verify the original error is gone** — reproduce the original failure and confirm it's fixed
#### Capturing out-of-scope side-effects
While investigating or fixing, if you uncover a problem **unrelated to the root cause under investigation** that is non-trivial (addressing it inline would derail this fix), do NOT fix it now. Record it for later, then continue:
```bash
bash "<plugin-root>/lib/capture-later.sh" \
--name "<short title>" --description "<what & where — enough to act on cold>" \
--source cf-sys-debug [--slug <bug-doc/task slug, if one exists>] [--problem "<the bug under investigation>"]
```
This writes `<docsDir>/later/YYYY-MM-DD-<name>.md` with frontmatter (slug, problem, conversation_id). This is an in-repo audit trail, independent of the `spawn_task` tool.
### Phase 5: Document the Bug
cf-sys-debug is only invoked for hard bugs — always document the findings.
1. Read `language` config (local `.coding-friend/config.json` overrides global, default: `en`)
2. Use `MAIN_REPO_ROOT` from the SessionStart bootstrap context (injected via session-init.sh). If absent, fall back to running `pwd` for `$CWD` and use `$CWD` as `MAIN_REPO_ROOT`. Read config from `CF_CONFIG_FILE` (= `$MAIN_REPO_ROOT/.coding-friend/config.json`) for `docsDir` (default: `docs`) — do NOT search sub-folders. Use `CF_DOCS_ROOT` as the docs base dir.
3. Construct a write spec and delegate to **cf-writer agent** by calling `invoke_subagent` with agent `cf-writer` (use absolute `file_path`):
```
WRITE SPEC
----------
task: create
file_path: {CF_DOCS_ROOT}/memory/bugs/YYYY-MM-DD-{name}.md
language: {language from config}
content: |
---
title: "<Short bug title>"
description: "<One-line summary of the bug and fix, under 100 chars>"
tags: [tag1, tag2, tag3]
created: YYYY-MM-DD
updated: YYYY-MM-DD
type: episode
importance: 4
source: conversation
---
# <Bug Title>
## Overview
<What went wrong — symptom and why it was hard to diagnose>
## Investigation
<What was tried and ruled out — hypotheses that didn't pan out>
## Root Cause
<What was actually wrong — the real cause>
## Fix
<What was changed to fix it>
## Prevention
<How to avoid this bug in the future>
## Related Files
- `path/to/file1`
- `path/to/file2`
readme_update: false
auto_commit: false
existing_file_action: skip
```
> **Backward compat:** Existing bug memory files without a date prefix are still readable — do not rename them.
**Frontmatter rules:**
- `description`: factual summary for grep recall. Good: `"Circular dependency in plugin loader causing silent init failure"`. Bad: `"Hard bug fixed"`.
- `tags`: include error type, affected module, root cause category (e.g., `[circular-dependency, plugin-loader, initialization]`)
### Index in CF Memory (MANDATORY)
**This step is REQUIRED — do NOT skip it.**
After the cf-writer saves the bug doc, you MUST call the `memory_store` MCP tool to index it in the database. This is a separate action from writing the file — the cf-writer agent does NOT do this.
Call `memory_store` with:
- `title`: from the frontmatter title
- `description`: from the frontmatter description
- `type`: `episode`
- `tags`: from the frontmatter tags
- `content`: the full markdown content (including frontmatter)
- `importance`: 4
- `source`: "auto-capture"
- `index_only`: true
If the MCP tool is unavailable, log a warning to the user but do NOT fail silently.
Show the user a 2-line summary:
- **Markdown file:** `{docsDir}/memory/bugs/...md` (created or updated)
- **Memory DB:** indexed ✓ — or: MCP unavailable, file only
## Gotchas
| What happened | Rule |
| ------------------------------------------------------------------ | -------------------------------------------------------------------------------------- |
| Patched symptom file instead of origin | Trace the execution path backward before touching any file |
| MCP not loading, switched tools instead of diagnosing | Check server status, API key, config before switching |
| Orchestrator said RUNNING but a downstream stage was misconfigured | In multi-stage pipelines, test each stage in isolation |
| Race condition diagnosed as stale-state bug | For timing-sensitive issues, inspect event timestamps and ordering before state |
| Reproduced locally but failed in CI | Align the environment first (runtime version, env vars, timezone), then chase the code |
| Stack trace points deep into a library | Walk back 3 frames into your own code; the bug is almost always there |
| Adding a `try/catch` to suppress the error | Hides the bug, doesn't fix it — find and fix the root cause |
| Multiple changes at once | Can't tell which one fixed it — one change at a time, test after each |
## Debugging Tools
- `git bisect` — Find the exact commit that introduced a bug
- `git stash` — Isolate your changes to test clean state
- Print/log tracing — Add strategic logging at decision points
- Minimal reproduction — Smallest possible code that triggers the bug
## Outcome
### Success Format
```
Root cause: [what was wrong, file:line]
Fix: [what changed, file:line]
Confirmed: [evidence or test that proves the fix]
Tests: [pass/fail count, regression test location]
Regression guard: [test file:line] or [none, reason]
```
Status: **resolved**, **resolved with caveats** (state them), or **blocked** (state what is unknown).
### Handoff Format (after 3 failed hypotheses)
```
Symptom:
[Original error description, one sentence]
Hypotheses Tested:
1. [Hypothesis 1] → [Test method] → [Result: ruled out because...]
2. [Hypothesis 2] → [Test method] → [Result: ruled out because...]
3. [Hypothesis 3] → [Test method] → [Result: ruled out because...]
Evidence Collected:
- [Log snippets / stack traces / file content]
- [Reproduction steps]
- [Environment info: versions, config, runtime]
Ruled Out:
- [Root causes that have been eliminated]
Unknowns:
- [What is still unclear]
- [What information is missing]
Suggested Next Steps:
1. [Next investigation direction]
2. [External tools or permissions that may be needed]
3. [Additional context the user should provide]
```
Status: **blocked**
## Review Reminder
After the fix is verified, ask the user if they want to run `/cf-review` or `/cf-commit`. Do NOT auto-run — wait for their choice.