Immutable. This exact content is served forever at /api/v1/blob/864522471185ed98.
---
name: backfill
description: >-
Reconstruct an OKF bundle by event-sourcing a repository's history (git log
and Claude session transcripts). Use when creating an `.okf/` bundle for an
existing repository that predates this skill, or when resuming an interrupted
backfill session. Triggers on: "reconstruct the OKF bundle", "backfill the
knowledge bundle", "event-source the history".
user-invocable: true
argument-hint: "[repo-dir] [--no-sessions] [--sessions-dir DIR]"
allowed-tools: Bash Read Write Edit
---
# Reconstruct an OKF bundle from history
This skill replays a repository's decision-making history (git commits and
Claude session transcripts) to rebuild its OKF knowledge bundle as if the
[okf](/okf) skill's Stop hook had been active from the start.
The extraction layer is **deterministic** (same repo → byte-identical events);
the replay layer is an LLM loop, **replayable and auditable but not
byte-identical** (timestamps, summaries change per run). See the event schema
(§1) and skip rules (§3).
## 1. Event schema
Git commits and session turns become events (JSONL, one per line, sorted by
timestamp then source):
```json
{"id":"git:<sha>","source":"git","ts":"<ISO8601 Z>","sha":"...","author":"...","subject":"...","body":"...","files":[{"path":"...","add":N,"del":N}]}
{"id":"session:<file>:<lineno>","source":"session","ts":"<ISO8601 Z>","user":"...","outcome":"...","title":"...","branch":"...","skip":"..."}
```
- **Git events** come from `git log --first-parent --numstat`.
- **Session events** pair each user message with the **last** assistant text
block of that turn (the wrap-up), extracted from
`~/.claude/projects/<repo-slug>/*.jsonl` and worktree subdirs.
- **Timestamps** are normalized to UTC ISO 8601 strings ending in `Z`.
- **Skip field** (optional, added by extraction): marks low-signal events —
see §3, never overridden by replay.
## 2. Protocol: Preflight → Extract → Bootstrap → Replay (Map+Reduce) → Finalize
### Preflight: check if bundle can be rebuilt
```bash
if [ -d <repo>/.okf ] && ! [ -f <repo>/.okf/.backfill-state.json ]; then
echo "ERROR: .okf/ exists but is incomplete. Delete it to rebuild from scratch, or pass --resume to continue from the last checkpoint."
exit 1
fi
```
If `.backfill-state.json` exists, resume from the cursor; otherwise, fresh bootstrap.
### Extract: generate event stream
```bash
uv run "${CLAUDE_SKILL_DIR}/scripts/okf_backfill_events.py" <repo-dir> \
--out events.jsonl \
[--no-sessions] [--sessions-dir ~/.claude/projects] \
[--max-text 2000] [--skip-globs "vendor/**"]
```
Write `events.jsonl` to the scratchpad (never committed). Report:
- Total events extracted, per-source counts, and per-rule skip counts.
### Bootstrap: initialize bundle (fresh only)
Create `.okf/` with:
```bash
mkdir -p <repo>/.okf
echo 'okf_version: "0.2"' > <repo>/.okf/index.md
echo '# Update Log' > <repo>/.okf/log.md
```
Cursor (`.okf/.backfill-state.json`):
```json
{"last_id": null, "done": 0}
```
### Replay: two-phase map/reduce protocol
The replay is now structured in two phases to enforce semantic depth and anti-degeneration rules:
#### Phase 1: Map (parallel analysis, per-event)
Launch one `okf:event-analyzer` agent per live event (those without `skip` field), orchestrated
in waves via Claude Code's Workflow `agentType` parameter. Each analyzer:
- Reads the event (git diff or session transcript)
- Outputs `analyses/<event-id>.md` (event id with `:` sanitized to `-`) to scratchpad
- Never touches `.okf/`
- Is resumable: skip events already in `analyses/`
**Host-agnostic fallback** (if no Workflow support): spawn generic subagents with the same
system prompt as `agents/event-analyzer.md`, one per event, collecting analyses to scratchpad.
#### Phase 2: Reduce (sequential folding, one agent)
Run a single `okf:bundle-weaver` agent that:
- Reads all analyses in chronological order
- Folds them into `.okf/`, updating or creating concepts
- Enforces anti-degeneration rules (§3)
- Updates `log.md` with dated bullets (the "why" from each analysis)
- Manages cursor (`.okf/.backfill-state.json`) for resume capability
- Is the only actor that writes `.okf/`
**Resume behavior:** both phases support resumption. Phase 1 skips already-analyzed event ids;
Phase 2 restarts from `last_id` in the cursor.
### Domain priming (advisory)
Before starting the map phase, read the repository's README and directory structure to sketch
a candidate taxonomy of concepts (skills, integrations, decisions, etc.). This priming is
*advisory only* — it helps the analyzer emit better candidate names. The *guarantee* that
no events are lost comes from the deterministic coverage check in Finalize (§5), not from
priming; analyses without candidate names still flow to the weaver.
### Finalize: validate and clean up
1. **Write `index.md` per directory by hand** (do NOT run `okf_init.py` — it
would scaffold placeholder files): one `# Section` per directory, one
bullet per concept, `* [Title](file.md) - description` taken from each
concept's frontmatter (SPEC §8). The root `index.md` keeps its
`okf_version: "0.2"` frontmatter and links every subdirectory.
2. **Add final log entry** (today's date, appended to existing dated section if present):
```
## 2026-09-01
- **Backfill**: reconstructed from N events by okf-backfill/0.9.3
```
3. **Delete cursor:**
```bash
rm -f .okf/.backfill-state.json
```
4. **Run self-checks for anti-degeneration rules:**
```bash
# No concept filenames derived from change type
! find .okf -name "*.md" | grep -E "merge-pull-request|(^|/)(feat|fix|chore|docs)[:-]"
# No non-kebab-case characters in filenames
! find .okf -name "*.md" | grep -E "[^a-z0-9/.-]"
# No identical consecutive log bullets
awk 'p==$0 && /^- / {exit 1} {p=$0}' .okf/log.md
```
5. **Run coverage check to guarantee every event is mapped:**
```bash
uv run "${CLAUDE_SKILL_DIR}/scripts/okf_backfill_events.py" \
--check-coverage events.jsonl .okf
```
Exit code 0 means all live events (non-skipped) appear in the bundle's `sources` or log.
Exit code 1 with a list of unmapped ids means the weaver skipped some events; fix and re-run.
6. **Validate bundle schema:**
```bash
uv run "${CLAUDE_SKILL_DIR}/../validate/scripts/okf_validate.py" .okf --strict
```
Fix every error before finishing.
7. **Report** the resulting bundle:
- Concept count per directory
- Log entry samples (first and last)
- Coverage check result (all events mapped)
- Validation result (pass/fail, warnings)
## 3. Skip rules
Events matching a rule below are marked `skip: <rule-id>` and skipped during
replay (cursor advances, no log entry):
| Rule | Condition | Rationale |
|------|-----------|-----------|
| `paths-only-generated` | Git commit touches ONLY lockfiles or generated code (vendor/, node_modules/, dist/, *.min.*, *.lock, etc.) | Noise: build byproducts, no knowledge content |
| `merge-no-files` | Git merge commit with no file changes (first-parent only) | Housekeeping |
| `session-command-noise` | Session user message is a slash command or <20 chars and has no useful outcome | Ephemeral UI/chatter |
Extend the first rule with `--skip-globs "extra/**"` for repo-specific patterns.
## 4. Anti-degeneration rules
The analyzer and weaver enforce these rules to prevent the bundle from devolving into
a mechanical listing of commits or a taxonomy-by-accident:
1. **Concept names describe domain entities, not changes.** Forbidden:
- Filenames derived from git subject: `merge-pull-request-#2-....md`, `feat:-add-docs-skill.md`, `fix:-handle-edge-case.md`
- Bare action words: `update.md`, `fix.md`, `add-feature.md`
- Non-kebab-case: spaces, underscores, capitals, special characters (allowed: `[a-z0-9-]` only)
Example: a commit with subject "feat: add presales pipeline" touches a domain concept.
The *concept* is named `presales-pipeline.md` (the entity), not `feat:-add-presales-pipeline.md`
(the change). The change history lives in frontmatter `sources` and log bullets.
2. **Prefer update over create.** Every event is analyzed for which concepts it touches; if
a concept already exists and the analysis fits, update its `sources` and body. Only create
a new concept if the analysis reveals a distinct domain entity not yet captured.
3. **Log bullets explain intent, not restate subjects.** A bullet should answer "why did this
change happen?" from the commit body or session outcome, never just re-read the subject:
- Bad: `- Added presales-pipeline.md feature`
- Good: `- **Presales pipeline**: formalized the sales workflow to clarify handoff points`
4. **No identical consecutive bullets.** Group same-date updates by concept to avoid:
```
- Feature X update
- Feature X update
- Feature X update
```
Instead: one bullet per concept, or combine into "Feature X: multiple updates".
## 5. Implementation notes
- **Determinism**: extraction is byte-identical; replay is not (time, LLM variance).
Both are auditable: events.jsonl is deterministic, map analyses are stored and resumable,
reduce loop is human-readable and cursor-backed.
- **Privacy**: events.jsonl goes to scratchpad, never committed. Session turn text is
truncated (head+tail) before extraction.
- **Cost note**: deep replay reads ~1 diff per git commit (full `git show` output) to
extract rationale. For histories >500 events, consider splitting into sub-ranges and
replaying sequentially, or use `--skip-globs` to exclude low-signal paths (e.g.,
vendored dependencies, generated code). Map phase is massively parallel (64 analyzers
can run concurrently); reduce is sequential but much cheaper (concepts already analyzed).
- **Future sources** (not implemented): GitHub PR/issue text, release notes, CI/deploy logs
— all optional post-MVP. **Lore protocol** (arXiv 2603.15566): on repos that adopt git
trailers (structured decision metadata), the extracted event's `body` already contains
trailers in a machine-readable format. The analyzer and weaver should use trailers as
the primary source of "why" (overriding generic inference from diff content).
- **Trust metadata**: `generated.by` is the backfill agent (`okf-backfill/0.9.3`), not
claimed as human-reviewed (`human:...`); concepts are correctly `unverified` (SPEC §5.3).
Map-phase analyses are working artifacts (stored for auditability during reduce); the
weaver's output is the canonical bundle.