Immutable. This exact content is served forever at /api/v1/blob/8655ee459f938acb.
# CLAUDE.md — csift operating manual
Project-specific operating manual for any AI agent (Claude Code, Codex, Cursor) working in this repo. Read this first when a conversation opens.
> **Companion doc.** [`SPEC.md`](./SPEC.md) is the product/behaviour spec (Phase 2 finalizes it). This file is authoritative for _how to work in the repo_; SPEC.md for _what to build_.
---
## 1. What csift is
**csift — "ripgrep for Claude Code session transcripts".** A fast Rust CLI that **lists** and **regex-searches** Claude Code session `.jsonl` files.
- **Primary consumer is an LLM** — a Claude Code agent searching/recovering its own or a peer session. Output must be clean, token-efficient, and regex-driven. Default output is human/LLM-readable with clear session/turn/category/timestamp headers; `--json` is the machine format.
- **Explicitly NO BM25 / embeddings / semantic search.** Pure regex/ripgrep only. Chinese tokenisation is intractable for lexical scoring; regex is the strength and the whole point.
- **Subcommands:** `list`, `search`, `whoami`.
---
## 2. LOCAL git only — never pushed
This repo is **local git only**: `git init` + local commits, nothing else.
- **NEVER add a git remote. NEVER push.** There is no origin and there must not be one.
- No CI service runs here; the pre-commit hook (below) is the entire quality gate.
---
## 3. Stack
| Layer | Choice | Why |
| --- | --- | --- |
| Language | Rust 2021, `rust-version = 1.89` | Fast byte/IO, strong types over dense jsonl |
| CLI | `clap` (derive) | Subcommands + example-rich `--help` |
| Regex | `regex` | ripgrep-like matching, smart-case |
| JSON | `serde` + `serde_json` | Lazy parse only on candidate lines |
| Scan | `memchr` (SIMD newline) + `memmap2` (mmap) | 200MB+ files without full-buffer reads |
| Parallel | `rayon` | Fan-out across many session files |
| Errors | `anyhow` | Error chains surfaced on stderr; no `unwrap` in lib paths |
| Date/TZ | `jiff` | ISO8601 parse + Australia/Sydney local render alongside raw UTC |
| Hooks | `cargo-husky` (dev-dep, `user-hooks`) | Installs the pre-commit gate |
Versions are pinned by `^`-range in `Cargo.toml` + `Cargo.lock`. **Do not bump majors without an explicit reason.**
---
## 4. Conventions — read before changing code
- **No `unwrap`/`expect` in library/hot paths.** Propagate with `anyhow::Result` and `?`. Tests may `unwrap`. The `main` shim is the only place that turns an error into an exit code.
- **No silent truncation.** If a result set is capped (`--max-count`), the output MUST state how many were dropped. A skipped malformed line must be counted, never hidden.
- **Tolerant parsing.** Real jsonl carries far more fields than any doc lists (`attachment`, `file-history-snapshot`, `queue-operation`, `isMeta`, `toolUseResult`, `slug`, …) and some records have no `timestamp`. Deserialize only what's used, ignore the rest, never crash on a new field or block type. The `Block` enum has a `#[serde(other)] Unknown` arm for exactly this.
- **Performance is a contract, not a nicety.** `list`/`search` must stay fast on 200MB+ files: mmap + `memchr` line scan + a cheap byte/regex prefilter, with full `serde_json` only on candidate lines; tail reads SEEK from EOF backward (never parse the whole file); `rayon` parallelizes across files.
- **`PascalCase` types, `snake_case` items, one module per concern** (`cli`, `path`, `model`, `parse`, `session`, `search`, `whoami`).
- **Comments capture _why_ / a non-obvious constraint**, not what the code already says.
- **Scaffold note:** `src/main.rs` currently has a crate-level `#![allow(dead_code)]` because Phase-1 stubs declare the full surface before handlers are wired. **Remove it in Phase 2** once every public item is referenced — a leftover allow then masks real dead code.
---
## 5. Commands
```bash
cargo build # debug build (GATE: must succeed)
cargo build --release # optimised (thin-LTO, 1 cgu) for real scans
cargo run -- list [PATH...] # list sessions
cargo run -- search PATTERN [flags] # regex search
cargo run -- whoami [--path] # identify the calling CC session
cargo fmt --all # format
cargo fmt --all -- --check # format gate
cargo clippy --all-targets -- -D warnings # lint gate (warnings-as-errors)
cargo test # unit tests (also installs the hook)
```
**Pre-commit gate (cargo-husky).** On the first `cargo test`/`cargo build` after checkout, cargo-husky installs `.git/hooks/pre-commit` from `.cargo-husky/hooks/pre-commit`. It runs, in order: `cargo fmt --all -- --check` → `cargo clippy --all-targets -- -D warnings` → `cargo test`. A failure blocks the commit. Edit the **source** hook (`.cargo-husky/hooks/pre-commit`), not the installed copy, then re-run `cargo test` to reinstall. Genuine-WIP bypass: `git commit --no-verify` (use sparingly).
---
## 6. The Claude Code jsonl knowledge (verified empirically 2026-06-07)
This is the load-bearing domain knowledge. Verified against real `~/.claude/projects/**/*.jsonl`.
### 6.1 Data location
```
~/.claude/projects/<ENCODED_PROJECT_DIR>/<session-uuid>.jsonl # a session transcript
~/.claude/projects/<ENCODED>/<session-uuid>/subagents/*.jsonl # subagent transcripts
~/.claude/projects/<ENCODED>/<session-uuid>/subagents/*.meta.json
~/.claude/projects/<ENCODED>/<session-uuid>/tool-results/<id>.txt # externalised tool output
```
### 6.2 Path encoding (verified, deterministic forward / lossy reverse)
Claude Code encodes a project's absolute cwd into a dir name by replacing **every** non-`[A-Za-z0-9]` byte with a single `-`. **No** consecutive-dash collapsing; `.`, `/`, `_`, space all map to `-`. Confirmed:
- `/Users/testuser/Projects/widget_app_prototype` → `-Users-testuser-Projects-widget-app-prototype` (both `/` and `_` → `-`).
- `/Users/testuser/Projects/Acme/widget_factory-worktrees/main` → `-Users-testuser-Projects-Acme-widget-factory-worktrees-main`.
- A `/.claude/` segment → `--claude-` (a literal `--` double-dash — proves no collapse, and `.` → `-`).
Forward is deterministic; **reverse is lossy** (a `-` could have been `/`, `_`, `.`, …) so we never reverse. The tool ACCEPTS either (a) an actual filesystem path (encode it, locate the matching dir) or (b) a direct `~/.claude/projects/<encoded>` path (use as-is). Detect which by whether the arg resolves under the projects root.
### 6.3 Record model (one JSON object per line)
**Top-level fields used:** `type`, `uuid`, `parentUuid`, `timestamp` (ISO8601 UTC, e.g. `2026-06-07T05:43:00.000Z`), `sessionId`, `cwd`, `version`, `gitBranch`, `isSidechain`, `userType`, `message`, plus `subtype`/`content` on system records and `isCompactSummary` on compaction summaries. **Many more fields exist and are ignored** (`attachment`, `file-history-snapshot`, `queue-operation`, `isMeta`, `toolUseResult`, `sourceToolAssistantUUID`, `slug`, `entrypoint`, `promptId`, …).
**`type` values seen:** `user`, `assistant`, `system`, plus metadata-only records `last-prompt`, `ai-title`, `agent-name`, `mode`, `permission-mode`, `attachment`, `file-history-snapshot`, `queue-operation` (the metadata-only ones often have **no `timestamp`** — skip in time logic, never crash).
**`type:"user"`** — `message.role="user"`; `message.content` is EITHER a string (genuine user text, older format) OR an array of blocks. **CRUCIAL: a "user" record is NOT always a human turn** — `tool_result` blocks are carried on `role:user` records too. In one real session: 332 genuine string-content + 61 text-block users vs **1619 tool_result-carriers**. The genuine-user classification is load-bearing.
**`type:"assistant"`** — `message.role="assistant"`; `message.content` = array of blocks.
**Block types:** `{type:text,text}`, `{type:thinking,thinking,signature?}`, `{type:tool_use,id,name,input}`, `{type:tool_result,tool_use_id,content,is_error?}`, `{type:image,source}`. `tool_result.content` may be a string OR an array of `{type:text,text}` / `{type:image}`.
**AskUserQuestion** = a `tool_use` block with `name="AskUserQuestion"`. **HARD-WON:** a PENDING/unanswered AskUserQuestion is **not** flushed to jsonl — only answered ones appear (the answer returns as a later `tool_result`/user record).
**`type:"system"`** — `{subtype, content?, level?, toolUseID?}`. Subtypes seen: `stop_hook_summary`, `turn_duration`, `away_summary` (a short auto-summary of what the session was doing when it went idle), `compact_boundary`.
**Compaction (verified shape):** the summary is a `type:"user"` record with `isCompactSummary:true` + `isVisibleInTranscriptOnly:true` carrying **string** content (NOT a `type:"summary"` record). A separate `type:"system"` `subtype:"compact_boundary"` record carries `compactMetadata:{trigger,preTokens,postTokens,durationMs}`. **A compaction summary must be excluded from "genuine user".**
**Externalised output:** large tool outputs may be moved to a sibling `tool-results/<id>.txt`, with an inline `<persisted-output>` pointer (carrying an absolute path + a preview). Optionally resolved with `--resolve-persisted`.
### 6.4 Genuine-user vs tool-result-carrier (the filter that everything hinges on)
A GENUINE user turn (for the `user` category and for turn-delimiting):
1. `type:"user"` with `message.role == "user"`, AND
2. `isCompactSummary` is falsey, AND
3. content is a string, OR content blocks contain a `text` block and NO `tool_result` block.
A `tool_result`-carrier record does **not** count as genuine user and does **not** start a turn. See `model::Record::is_genuine_user`.
### 6.5 Categories (`search -t/--category`, repeatable)
- `thinking` = assistant thinking blocks.
- `user` = genuine user input + user answers to AskUserQuestion (NOT tool_result-carriers).
- `tool` = `tool_use` blocks (AskUserQuestion is a tool_use).
- `tool-response` = `tool_result` blocks.
- `agent` = assistant visible end-of-turn text (the agent message; "agent includes AskUserQuestion").
### 6.6 Complete x (round-trip)
On a match, return the COMPLETE exchange, not a fragment: a matched `tool_use` WITH its `tool_result`; a matched user turn WITH the agent response. Reconstruct via `uuid`/`parentUuid` linking. A **turn** is delimited by genuine-user messages.
### 6.7 whoami detection (verified)
Claude Code exports **`CLAUDE_CODE_SESSION_ID`** into its Bash tool env, equal to the session's own jsonl basename (verified: env value `0a1b2c3d-…` matched `…/0a1b2c3d-….jsonl` in this project dir). This is definitive — per-session, version-independent, survives bash nesting, zero false positives. Use it and nothing else. When absent/empty, **DO NOT GUESS** (concurrent sessions, different binaries; most-recent-mtime is a false-positive trap) — error with guidance to pass `--session`. It is acceptable for whoami to often say "ambiguous". (`CODEX_COMPANION_SESSION_ID` mirrors it but is Codex-plugin-specific; prefer the canonical var.)
---
## 7. Module map
```
src/main.rs # binary entrypoint: parse args, dispatch, error→exit code
src/cli.rs # clap derive: Cli/Command + ListArgs/SearchArgs/WhoamiArgs + Category
src/path.rs # encode_cwd + projects-root + target resolution (real-path vs encoded)
src/model.rs # serde Record/Message/Content/Block + is_genuine_user
src/parse.rs # mmap + memchr head/tail/stream readers + lazy parse_line
src/session.rs # `list`: head+tail read → SessionSummary
src/search.rs # `search`: regex + filters → complete x Exchange
src/whoami.rs # `whoami`: CLAUDE_CODE_SESSION_ID detection, false-positive-safe
```
Phase-1 handler bodies are `todo!()`; the type/flag/test surface is real. Phase 2 fills the bodies per SPEC.md.
---
## 8. What NOT to do
- **Don't add a remote / don't push.** Local git only (§2).
- **Don't introduce BM25 / embeddings / semantic search.** Regex only (§1).
- **Don't `unwrap`/`expect` in library paths**, and **don't silently truncate** (§4).
- **Don't parse a whole 200MB file** when a head or tail read answers the question (§4, §6).
- **Don't trust most-recent-mtime for `whoami`** (§6.7).
- **Don't blindly trust this doc's field list** — real jsonl evolves; re-verify against `~/.claude/projects` and EXTEND the model tolerantly rather than tightening it.
- **Don't bump a dependency major** without an explicit reason.
- **Don't edit `.git/hooks/pre-commit` directly** — edit `.cargo-husky/hooks/pre-commit` and re-run `cargo test`.