git:20260507.dec1192 to git:20260507.7e99d8a

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Build & Dev Commands

```bash

# Or start the daemon as a managed launchd service:

cargo build -p origin-server

./target/debug/origin-server status

# Workspace-level builds

cargo check --workspace

cargo build --workspace

cargo test --workspace

# Per-crate builds (faster for iteration)

cargo check -p origin-types

cargo check -p origin-core

cargo check -p origin-server

cargo build -p origin --release

./target/release/origin --help

# Run tests for a single crate

cargo test -p origin-core

cargo test -p origin-core --lib <module>::tests

cargo test -p origin-core <test_name>

# Generate coverage reports (opens in browser)

bash scripts/coverage.sh

bash scripts/setup-hooks.sh

# Eval benchmarks (require GPU + model files, run manually)

# Unit tests for eval modules (fast, no GPU):

cargo test -p origin-core --lib locomo::tests

cargo test -p origin-core --lib longmemeval::tests

# Generate eval baselines (slow, needs Qwen 3.5-9B on Metal GPU):

cargo test -p origin-core --test eval_harness save_locomo_baseline -- --ignored --nocapture

cargo test -p origin-core --test eval_harness save_locomo_reranked_baseline -- --ignored --nocapture

cargo test -p origin-core --test eval_harness save_locomo_expanded_baseline -- --ignored --nocapture

cargo test -p origin-core --test eval_harness save_longmemeval_baseline -- --ignored --nocapture

cargo test -p origin-core --test eval_harness save_longmemeval_reranked_baseline -- --ignored --nocapture

cargo test -p origin-core --test eval_harness save_longmemeval_expanded_baseline -- --ignored --nocapture

```

## Local vs CI test responsibilities

Origin runs across several layers. The split is driven by three questions: **(1) Can a hosted runner do this?** (no GPU, no API keys, no cost). **(2) Is it under 60s on cold cache?** **(3) Does it gate correctness or measure quality?** Quality measures never gate.

| Layer | What runs | Where | When | Time | Blocks? |

|---|---|---|---|---|---|

| **L1 dev loop** | rust-analyzer / IDE | Local | Every save | <1s | No |

| **L6 main canary** | Embedding-only eval (`cargo test -p origin-core --lib eval::token_efficiency -- --ignored`) | GitHub (`ci.yml`) | Push to `main` | ~10min | No (post-merge) |

| **L7 manual local** | `bash scripts/coverage.sh` (HTML coverage), GPU eval suite (`cargo test -- --ignored`), Anthropic batch judge (`ANTHROPIC_API_KEY=... cargo test ...`) | Your laptop | On demand | minutes-hours | No |

### What does NOT run in CI and why

- **GPU evals (LongMemEval / LoCoMo runner functions, Qwen3.5-9B inference)** — GitHub macOS runners have no Metal acceleration. The tests are `#[ignore]`d so they don't accidentally run.

- **Anthropic API batch judge** — costs $0.35/run and requires `ANTHROPIC_API_KEY` which we don't expose to PR runs from forks.

### Why pre-push doesn't run coverage

Earlier versions of `.githooks/pre-push` enforced a 90% `cargo llvm-cov` gate. That violated the principles above:

- **Not mirrored in CI:** the main `ci.yml` lane doesn't run coverage at all, so the gate added local friction without upstream protection.

The current pre-push runs only clippy + non-instrumented tests. Coverage is L5 (informational on PR) or L7 (manual command on laptop).

### Eval baselines cache

```bash

export EVAL_BASELINES_DIR=$HOME/.cache/origin-eval

```

Path must be writable and local (network mounts not recommended). When set, also chains `EVAL_ENRICHMENT_CACHE_DIR` default to the same dir unless explicitly overridden. Migration: `bash scripts/migrate-eval-cache.sh <source-baselines>`.

## Releasing (release-please)

Releases are automated via [release-please](https://github.com/googleapis/release-please). The workflow runs on every push to `main`.

**How it works:**

1. Every push to `main`, release-please scans new commits and maintains an open "release PR" that accumulates changes and updates `CHANGELOG.md`.

2. When you're ready to ship, merge the release PR. That triggers a GitHub release (draft) + git tag.

**Commit messages control version bumps.** Pre-1.0:

| Commit prefix | Version bump | Example |

|---|---|---|

| `fix:` | patch | 0.1.1 -> 0.1.2 |

| `feat:` | **minor** | 0.1.1 -> 0.2.0 |

| `BREAKING CHANGE` | minor (capped) | 0.1.1 -> 0.2.0 |

| `chore:`, `ci:`, `docs:`, `refactor:`, `test:` | no bump | (hidden in changelog) |

After 1.0, standard semver: `feat:` bumps minor, `BREAKING CHANGE` bumps major.

**IMPORTANT: `feat:` bumps minor, not patch.** Use `fix:` for small features, improvements, and bug fixes. Only use `feat:` when you intentionally want 0.x.0 -> 0.(x+1).0. The `bump-patch-for-minor-pre-major` and `release-as` config flags do NOT work with release-please v17 + simple release type. If a `feat:` commit lands on main, the only way to prevent the minor bump is to rewrite the commit message via `git filter-branch` and force-push.

**Config files:**

```bash

# To release: just merge the open release-please PR on GitHub.

# To check what version is pending:

cat .release-please-manifest.json

```

### Release pipeline gotchas (learned the hard way)

**release-please determines "last version" from merged PR commit messages**, not tags or manifest. It scans for commits matching `chore(main): release X.Y.Z`. Deleting a tag or GitHub Release is NOT enough to reset the version. You must also ensure no commit message in the history matches that pattern, or use `release-as` to force-override.

**Never delete a release tag without also cleaning up the commit history.** If you need to undo a release version, you must rewrite the commit message that release-please created (`git filter-branch --msg-filter`), delete the tag, delete the GitHub Release, and rename the merged PR title via API. Otherwise release-please will keep bumping from the old version.

### Branch protection

Main branch has: required CI ("Test & Lint") before merge, no force pushes, no deletion. `enforce_admins: false` so the repo owner can push directly for hotfixes. Force push requires temporarily enabling it via API (remember to re-disable after).

### Git hooks (auto-activated)

## Architecture

### Workspace Layout

The repo is a Cargo workspace with 4 crates:

| Crate | Role | Key dependencies |

|---|---|---|

| `crates/origin-server` | Headless HTTP daemon on `127.0.0.1:7878`. Depends on `origin-core`. Provides `install/uninstall/status` subcommands for launchd management. | axum, tower, clap |

### Stack

- **Daemon**: Rust, Axum 0.8 (HTTP), libSQL (Turso's SQLite fork — vectors, knowledge graph, documents), Tokio, FastEmbed (BGE-Base-EN-v1.5-Q, 768-dim, 512-token max), llama-cpp-2 (Qwen3-4B-Instruct-2507 via Metal GPU; Qwen3.5-9B optional), launchd for process management

### Database: libSQL (owned by origin-core)

One libSQL database at `~/Library/Application Support/origin/memorydb/origin_memory.db`, owned by `MemoryDB` in `crates/origin-core/src/db.rs`:

- **Document chunks**: `chunks` table with `F32_BLOB(768)` vector column, DiskANN indexing (768-dim, BGE-Base-EN-v1.5-Q)

- **Knowledge graph**: `entities`, `relations`, `observations` tables with FK cascades

- **Full-text search**: FTS5 virtual table (`chunks_fts`) auto-synced via triggers

- **Hybrid search**: Vector similarity + FTS combined with Reciprocal Rank Fusion (RRF)

**Connection pattern**: `tokio::sync::Mutex<libsql::Connection>` — `libsql::Connection` is `Send` but not `Sync`, so it's wrapped in an async `Mutex` inside `MemoryDB`.

**Sharing pattern**: `MemoryDB` is wrapped in `Arc<MemoryDB>` at the state layer (`ServerState.db: Option<Arc<MemoryDB>>`). This lets handlers clone the `Arc` out of the `RwLock<ServerState>` guard and drop the guard before performing long-running operations.

### Events: EventEmitter trait (no tauri::Emitter in core)

Instead of passing `tauri::AppHandle` into business logic, `origin-core` defines an `EventEmitter` trait:

```rust

// crates/origin-core/src/events.rs

pub trait EventEmitter: Send + Sync {

fn emit(&self, event: &str, payload: &str) -> Result<()>;

}

pub struct NoopEmitter;

```

- `MemoryDB::new(db_path, emitter: Arc<dyn EventEmitter>)` takes the trait object

This keeps `origin-core` framework-agnostic and testable with `NoopEmitter` in unit tests.

### IPC Surface

- General: `/api/health`, `/api/status`, `/api/search`, `/api/context`, `/api/chat-context`, `/api/ping`

- Ingest: `/api/ingest/text`, `/api/ingest/webpage`, `/api/ingest/memory`

- Memory CRUD: `/api/memory/store`, `/api/memory/search`, `/api/memory/confirm/{id}`, `/api/memory/list`, `/api/memory/delete/{id}`

- Knowledge graph: `/api/memory/entities`, `/api/memory/relations`, `/api/memory/observations`

- Profile & Agents: `/api/profile`, `/api/agents`, `/api/agents/{name}`

- WebSocket: `/ws/updates`

## Key Modules — origin-core (`crates/origin-core/src/`)

All business logic lives here. No tauri, no axum. Framework-agnostic.

| Module | Purpose |

|---|---|

| `db.rs` | `MemoryDB` — libSQL storage, vectors, chunks, hybrid search, embeddings, knowledge graph, migrations. Three search methods: `search_memory` (embedding+FTS+RRF), `search_memory_reranked` (+ LLM reranking after), `search_memory_expanded` (+ LLM query expansion before). Uses `EventEmitter` trait for UI notifications (no tauri). |

| `events.rs` | `EventEmitter` trait and `NoopEmitter` |

| `engine.rs` | `LlmEngine` — llama-cpp-2 wrapper, model download, inference loop, format helpers |

| `classify.rs` | Memory/profile classification via `LlmEngine` |

| `extract.rs` | Knowledge-graph extraction (entities, relations) via `LlmEngine` |

| `rerank.rs` | LLM reranker |

| `merge.rs` | Memory merging, pattern extraction, contradiction detection |

| `llm_provider.rs` | `LlmProvider` trait + `ApiProvider` (Anthropic API) + `OnDeviceProvider` shim |

| `llm_classifier.rs` | Higher-level classification orchestration |

| `refinery.rs` | Background refinement queue — dedup, auto-linking, consolidation |

| `post_ingest.rs` | Post-ingest enrichment (dedup check, entity linking, title enrich, recap, page growth) |

| `pages.rs` | Type definitions for the `Page` struct (synthesized wiki entries distilled from memory clusters). Actual clustering + distillation live in `db.rs` + `refinery.rs`. SQL tables remain `concepts`/`concept_sources` historically. |

| `spaces.rs` | Spaces / tag store |

| `narrative.rs` | Profile narrative assembly (editorial prose) |

| `briefing.rs` | Daily briefing assembly |

| `working_memory.rs` | Working memory builder |

| `access_tracker.rs` | Memory access counts + time decay |

| `contradiction.rs` | Contradiction detection |

| `context_packager.rs` | Context bundle → prompt packaging |

| `importer.rs` | File importer pipeline |

| `quality_gate.rs` | Pre-store quality gate |

| `tuning.rs` | Tuning config (refinery, distillation, weights) |

| `schema.rs` | Memory schema definitions (formerly `memory_schema.rs`) |

| `prompts/` | Prompt registry (defaults + override dir loader) |

| `chunker/` | Code-aware, Markdown-aware, fixed-size chunking |

| `sources/` | `RawDocument`, file watchers, Obsidian importer. `RawDocument` and related types re-exported from `origin-types`. |

| `privacy.rs` | PII redaction |

| `router/classify.rs`, `content_score.rs` | Smart router scoring helpers (non-tauri parts) |

| `config.rs` | Persistent config at `~/Library/Application Support/origin/config.json` |

| `export/` | Markdown/JSON/zip/PDF exporters |

| `state.rs` | `CoreState` — shared state struct used by origin-server |

## Key Modules — origin-server (`crates/origin-server/src/`)

HTTP daemon — owns the Axum router + all routes. All handlers operate on `Arc<RwLock<ServerState>>` where `ServerState.db: Option<Arc<MemoryDB>>`.

| Module | Purpose |

|---|---|

| `main.rs` | Binary entry — clap subcommands (`install`/`uninstall`/`status`/daemon), tracing init, port binding with existing-daemon fallback, `MemoryDB::new`, LLM provider init, background tasks, `axum::serve` |

| `state.rs` | `ServerState` struct with `db: Option<Arc<MemoryDB>>`, `llm`, `prompts`, `tuning`, `quality_gate`, `space_store`, `access_tracker`, `llm_processing_ids`, `watch_paths`. `SharedState = Arc<RwLock<ServerState>>` |

| `router.rs` | `build_router(state) -> axum::Router` — all route registrations |

| `routes.rs` | General endpoints: health, search, context, chat-context, status, profile/agents |

| `memory_routes.rs` | Memory CRUD, knowledge graph, classification, entities, pages |

| `ingest_routes.rs` | `/api/ingest/*` — text, webpage, memory |

| `ingest_batcher.rs` | Request-level coalescer for concurrent `/api/memory/store` — folds QualityGate in-line; async classify/extract; passes enrichment + hint through in the response |

| `knowledge_routes.rs` | Entity/relation/observation read paths + knowledge-graph queries |

| `source_routes.rs` | Source registry endpoints |

| `import_routes.rs` | Bulk import endpoints |

| `config_routes.rs` | Config read/write endpoints |

| `onboarding_routes.rs` | First-run wizard / milestone state |

| `scheduler.rs` | Background periodic tasks (refinery ticks, distillation, etc.) |

| `websocket.rs` | `/ws/updates` |

| `error.rs` | `ServerError` + axum `IntoResponse` impl |

| `resources/com.origin.server.plist` | launchd plist template (embedded via `include_str!`) |

## Conventions

### Crate boundaries

- **origin-core must have NO tauri or axum dependencies.** Verify with `grep -rn "use tauri\|use axum" crates/origin-core/src/` — expect zero hits. Any event emission goes through the `EventEmitter` trait.

- **Don't add business logic to origin-server.** Route handlers should call `origin-core` functions with state snapshots — the server's job is HTTP framing, not logic.

### Async and locking

- **Never hold a `tokio::sync::RwLock` read or write guard across `.await`.** Holding a read guard during an LLM call (which can take seconds) blocks all writers. Pattern: snapshot what you need from the guard into a scoped block that ends before the await, then call the async function with the cloned values. See `crates/origin-server/src/memory_routes.rs` `handle_store_memory` for an example of the post-ingest enrichment pattern.

- **`Arc<MemoryDB>` is the sharing primitive.** `ServerState.db` is `Option<Arc<MemoryDB>>`. Clone the Arc out of the guard rather than borrowing through the guard.

- **libSQL connection pattern**: `MemoryDB` holds `tokio::sync::Mutex<libsql::Connection>` internally. Never try to share a `libsql::Connection` across tasks directly (`Send` but not `Sync`).

### SQL, strings, data

- **SQL safety**: Always use parameterized queries — never interpolate user input into SQL strings

- **NULL semantics**: Store `Option<T>` as SQL NULL, not empty string — so IS NULL filters work correctly

- **UTF-8 safety**: Never byte-index Rust strings (`&s[..n]`) — use `chars().take(n)` or `strip_prefix`/`strip_suffix`. Exception: byte-slicing after a verified ASCII prefix check is safe (the boundary is guaranteed valid), but prefer the char-safe version anyway for consistency.

- **Batch SQL**: Wrap multi-row insert/delete loops in BEGIN/COMMIT transactions

- **LIKE patterns against JSON**: Quote the match target to avoid substring false positives — `%"{id}"%` not `%{id}%` (e.g., `mem_1` would otherwise match `mem_10`). See the fix at `crates/origin-core/src/db.rs:~9895` and the regression test.

### Dev environment gotchas

**Daemon lifecycle:**

- **Worktree daemon mismatch**: The daemon on port 7878 can be from launchd, main branch, a stale worktree, or a previous session. Always verify which binary is running: `lsof -i :7878` to get the PID, then `lsof -p <PID> | grep "txt.*origin-server"` to see the binary path and size. Kill and restart from the current working tree.

- **Stale binary after merge/pull**: `cargo build -p origin-server` may report "0.64s Finished" without recompiling if the source timestamps haven't changed (e.g., after `git pull` fast-forward). Touch a source file to force recompilation: `touch crates/origin-server/src/router.rs && cargo build -p origin-server`. Verify the binary timestamp matches: `ls -la target/debug/origin-server`.

- **kill vs kill -9**: `kill <PID>` may not terminate the daemon cleanly. Always use `kill -9 <PID>` and verify with `lsof -ti :7878` afterward. If the port is still in use, another process took over.

- **Worktree target directories are shared**: All worktrees share the same `target/` directory in the main repo. Building in one worktree overwrites binaries from another. After switching worktrees, always rebuild to ensure the binary matches the checked-out source.

**Other:**

- **Metal/ggml on macOS Tahoe 26.x**: `ggml_metal_init` may fail even though native Metal works. The daemon auto-degrades and continues without LLM. Not a code bug. Check for competing GPU processes: `pgrep -la origin`.

### Misc

- Log filter default is `warn` — add modules explicitly for `info` logs (e.g., `origin_core::db=info`, `origin_server=info`)

- All local data stored in `~/Library/Application Support/origin/` — MemoryDB, config, activities, tags