CLAUDE.md git:20260901.699d9aeB

Immutable. This exact content is served forever at /api/v1/blob/61d048b1896d6398.

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What this is

**Alchemy** — a local-first, macOS-focused research notebook inspired by NotebookLM (Tauri 2 + React 19 front-end, Rust backend, LanceDB embedded storage). Import sources, chat grounded in citations, generate documents; everything runs on-device by default. Package name is `alchemy`; the directory name `notebooklm-local` is historical.

## Commands

```bash
pnpm install            # postinstall fetches PDFium + builds the Swift fm sidecar
pnpm tauri dev          # run the full app
pnpm dev                # Vite front-end only
pnpm build              # tsc typecheck + vite build (this is the frontend "lint")
```

Rust (run from `src-tauri/`) — CI enforces all three, run before committing:

```bash
cargo fmt -- --check && cargo clippy --all-targets -- -D warnings && cargo test
```

Tests and evals:

```bash
cargo test --lib <name> -- --nocapture           # single test
cargo test --lib evals -- --ignored --nocapture  # distill/rerank evals — need live Ollama
```

**Slow work is opt-in.** A plain `cargo test` runs the 367 correctness tests
and nothing else: ~12s, no model calls, no corpus embedding. Two env flags buy
back the rest.

| flag | what it turns on | cost |
| --- | --- | --- |
| `ALCHEMY_EVALS=1` | the corpus evals (`evals::`, `retrieval_eval::`) — fixture corpora embedded through the built-in embedder | +23s, CPU-bound |
| `ALCHEMY_OLLAMA_TESTS=1` | anything that calls a live model (`rag_round_trip`, the LLM half of `eval_deep_rerank`) | model-speed |

```bash
ALCHEMY_EVALS=1 cargo test --lib -- --nocapture                         # retrieval quality
ALCHEMY_OLLAMA_TESTS=1 cargo test --lib rag_round_trip -- --nocapture   # e2e data path
```

Both default off because both used to fire on their own: the evals on every
run, and the Ollama tests whenever the port happened to answer — which on a
developer machine is always. Reachability isn't consent, and measurement
isn't correctness. CI sets `ALCHEMY_EVALS=1` (see `.github/workflows/ci.yml`),
so the retrieval numbers are still watched where it matters.

To exercise anything the OS has to know about — the `alchemy://` scheme,
file associations, the Dock menu, Services — build a real bundle, not
`--no-bundle`:

```bash
pnpm tauri build --debug --bundles app    # -> target/debug/bundle/macos/Alchemy.app
```

A bare executable has no `Info.plist`, so those integrations are never
registered and silently do nothing. Set `APPLE_SIGNING_IDENTITY` (a
`Developer ID Application: ...` name) in the **shell environment** first, or
macOS re-prompts for file access on every rebuild: privacy permissions are
keyed on the signing identity, and an ad-hoc bundle draws a fresh random one
each build. The Tauri CLI reads the process environment and does not load
`.env`. Never commit an identity — it belongs in the developer's env.

Releases go through `scripts/release.sh` (see `RELEASE.md`). pnpm 11 quirks (`allowBuilds`, `verifyDepsBeforeRun: false`) are deliberate — don't "fix" `pnpm-workspace.yaml`.

## Architecture

`docs/ARCHITECTURE.md` is the authoritative deep-dive; `docs/RFC-*.md` documents each major feature's design (this repo is RFC-driven — write/update the RFC before implementing complex features). The short version:

**Data flow:** import → extract (`ingest.rs`, per-filetype) → structure-aware chunking → embed → LanceDB `chunks` table (vector + BM25 FTS). Chat embeds the question, runs hybrid search (vector + BM25 merged by reciprocal rank fusion), builds a numbered-excerpt grounded prompt (`rag.rs`), streams the answer as `chat://token` events, persists the turn with citations. Every retrieval appends a trace line to `<app-data>/traces/retrieval.jsonl`.

**Backend (`src-tauri/src`):**
- `db.rs` — one embedded LanceDB, one table per entity, filtered by `notebook_id` (not relational). `chunks`/`routes` tables are created lazily once embedding dimensionality is known. Updates/deletes use Lance predicate strings with single-quote escaping.
- `commands.rs` + `commands/` — the `#[tauri::command]` IPC surface. Errors are flattened to strings to cross IPC; serde structs in `models.rs` are `camelCase` for the TS side.
- `inference/` — provider abstraction: Ollama, OpenAI-compatible gateways, Apple Foundation Models (via the Swift sidecar in `sidecar/alchemy-fm`), headless agent CLIs (Claude Code, Codex, …), and a built-in local embedder. Model roles (chat/small/embed) route through `AiConfig`.
- `router.rs` / `gist.rs` — semantic router (per-source embedded routes, self-healing diff) and per-source distilled gists; both power "ask everything" meta-chat across notebooks.
- `mcp/` — embedded MCP server (rmcp, streamable HTTP on `127.0.0.1:41414`) exposing notebook/source/note CRUD + hybrid search to agents. Same process owns LanceDB, so no cross-process write conflicts; mutations emit `mcp://changed`. `connectors.rs` registers it (plus `skills/alchemy`) with installed agent clients. **Dev builds bind `mcp_port + 1` (41415) and write their own discovery file (`mcp.dev.json`)** so a dev instance and the installed app never collide on the port or on `mcp.json` — agent configs written by Connect point at the configured port (the installed app); to aim an agent or the CLI at a dev build, temporarily edit its config to 41415 (CLI: `ALCHEMY_MCP_DISCOVERY="$HOME/Library/Application Support/com.thrashr888.alchemy/mcp.dev.json"`).
- `integrations.rs` / `mac.rs` — Apple Notes/Reminders/Calendar/Stocks sources via the `cider` CLI (Paul's repo — fix bugs upstream there, don't work around them here).
- `diagnostics.rs` — error and crash capture (docs/RFC-diagnostics.md). Panic hook, JSONL log at `~/Library/Logs/com.thrashr888.alchemy/alchemy.log`, an `os_log` mirror on the `com.thrashr888.alchemy` subsystem, and `recent_errors` over IPC + MCP. **Print with `crate::note!`, never `eprintln!`** — `eprintln!` panics on a broken stderr and has aborted the app in the field. Anything that leaves the app unusable records at `fatal`, which raises the front-end's restart screen.

**Frontend (`src`):** `lib/types.ts` mirrors the Rust models, `lib/api.ts` is a typed `invoke` wrapper, `lib/store.ts` is the Zustand store (optimistic messages, streaming buffer). Components subscribe to Tauri events for streaming and cross-window refresh. In multi-window scenarios, JS `Any` event listeners are NOT filtered by target — self-filter by payload label.

## Design system

`DESIGN.md` is the source of truth for all visual/interaction decisions. Key rules: 27 themes (dark + light) driven by semantic CSS tokens in `src/index.css` and `src/lib/themes.ts` — **never hardcode a hex in a component**. Linear-inspired: hairline borders instead of tonal fills, color only when it means something, no colored left-border accents. Shared primitives live in `src/components/ui.tsx`.

**Shaders.** The backdrop (`src/components/DitherBackground.tsx`, one GLSL ES 1.0
program with 17 theme-driven modes) and the Activity tile washes
(`src/components/settings/TileShader.tsx`) are WebGL1 on purpose — WKWebView
everywhere, no WebGPU. Never edit a `FRAG` blind: shader quality is aesthetic,
not just correct math, and one GLSL error kills the backdrop for every theme.
Run the harness, look at the pixels next to the reference, iterate:

```bash
python3 scripts/shader-harness.py --serve   # http://127.0.0.1:8791/ — contact sheet of every mode
```

Also a `shaders` entry in `.claude/launch.json` for the Browser pane. The page
sets `<html data-status="ok|fail">` and prints compile logs, so it doubles as
the compile gate. See `.claude/skills/shaders/SKILL.md` for the workflow.

`WRITING.md` is the source of truth for all user-facing words (website, release notes, in-app copy). Register scales with the surface: Apple-terse headlines, Google-plain body prose, HashiCorp-sober methodology, Vercel-clipped table cells. Translate internal vocabulary before publishing, claim only measured numbers, and run the tell check before shipping copy.

## Conventions

- Intelligent behavior ships default-ON; settings toggles are cost control, not opt-in gates.
- New user-facing features should be agent-reachable too (MCP tools / commands), not UI-only.
- Keep test notebooks/fixtures after verifying — they double as examples.


<!-- BEGIN BEADS INTEGRATION v:1 profile:minimal hash:7510c1e2 -->
## Beads Issue Tracker

This project uses **bd (beads)** for issue tracking. Run `bd prime` to see full workflow context and commands.

### Quick Reference

```bash
bd ready              # Find available work
bd show <id>          # View issue details
bd update <id> --claim  # Claim work
bd close <id>         # Complete work
```

### Rules

- Use `bd` for ALL task tracking — do NOT use TodoWrite, TaskCreate, or markdown TODO lists
- Run `bd prime` for detailed command reference and session close protocol
- Use `bd remember` for persistent knowledge — do NOT use MEMORY.md files

**Architecture in one line:** issues live in a local Dolt DB; sync uses `refs/dolt/data` on your git remote; `.beads/issues.jsonl` is a passive export. See https://github.com/gastownhall/beads/blob/main/docs/SYNC_CONCEPTS.md for details and anti-patterns.

## Session Completion

**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds.

**MANDATORY WORKFLOW:**

1. **File issues for remaining work** - Create issues for anything that needs follow-up
2. **Run quality gates** (if code changed) - Tests, linters, builds
3. **Update issue status** - Close finished work, update in-progress items
4. **PUSH TO REMOTE** - This is MANDATORY:
   ```bash
   git pull --rebase
   git push
   git status  # MUST show "up to date with origin"
   ```
5. **Clean up** - Clear stashes, prune remote branches
6. **Verify** - All changes committed AND pushed
7. **Hand off** - Provide context for next session

**CRITICAL RULES:**
- Work is NOT complete until `git push` succeeds
- NEVER stop before pushing - that leaves work stranded locally
- NEVER say "ready to push when you are" - YOU must push
- If push fails, resolve and retry until it succeeds
<!-- END BEADS INTEGRATION -->