# Memex - CLI and Agent Plugins

Public code for a local knowledge base of collaborative memos and Claude Code transcripts.
Install the CLI from PyPI with `uv tool install memex-plugin`; the executable and
Python import remain `memex`. Claude Code supplies the lifecycle hooks and slash
commands; Codex and Kimi Code use portable skills. The vault layout below describes
user data separately from this public source checkout. See [README.md](README.md)
for installation and [DEVELOPMENT.md](DEVELOPMENT.md) for packaging and tests.

## Quick Start

The `memex` CLI and dreamer resolve the configured vault without changing directories. Run the source-only Obsidian wrapper by its checkout path; it selects a vault by name (`--vault`, default `memex`).

```bash
# Check vault status
memex status

# Search for something (RRF scoring is default)
memex search "authentication"

# Search recent docs only (7d, 2w, 3m)
memex search "plugin" --since=7d

# Rebuild index after changes
memex index rebuild --incremental
```

Semantic search uses a configured Gemini API key or a local LM Studio embedding server. For Gemini, use `op run --env-file ~/.secrets.op -- memex search "query"` only for an explicitly authorized 1Password-backed command, or `memex auth set-key` to opt into automatic loading from a local owner-only key file. Environment keys still work; keyword search remains available without a key. See [credential setup](docs/gemini-credentials.md).

## Your Role

When asked to act as the **memex curator**, condense project knowledge into `_project.md` overviews, maintain `[[wikilinks]]`, and cultivate the vault's knowledge topology. Search the vault when you need context for that work — don't rely on pre-loaded summaries.

## Folder Structure

```
memex-plugin/                  # Public source checkout
├── src/memex/scripts/          # Core scripts (search, embeddings, etc.)
├── scripts/                    # Compatibility shims and source-only helpers
├── hooks/                      # Claude Code hooks (SessionStart, PreCompact, etc.)
├── commands/                   # Slash commands (/memex:*)
├── skills/                     # Intent-based skills
├── _templates/                 # Bundled note templates
├── .claude-plugin/             # Claude Code plugin manifest + marketplace
├── plugins/memex/              # Codex root + generated portable skills (scripts/portable_plugin.py)
├── .agents/plugins/            # Codex marketplace → ./plugins/memex
└── kimi.plugin.json            # Kimi skills manifest; no Claude commands/hooks

<vault>/                       # Configured separately from the source checkout
├── projects/<name>/memos/      # Session memos and <memo>.obs.jsonl sidecars
├── projects/<name>/auto-memory/ # Synced Claude Code auto-memory files
├── projects/<name>/transcripts/ # Full conversation logs
├── topics/                    # Cross-project concept notes + trails
├── _meta/                     # Curator infrastructure, when configured
└── _views/                    # Obsidian Base views, when configured

~/.memex/_index.sqlite         # Default index location; see memex path --index
```

## Knowledge Artifacts

The vault stores three kinds of knowledge, each with a different temperature:

| Artifact | Temperature | Maintenance | Purpose |
|----------|------------|-------------|---------|
| **Memos** | Cold (append-only) | None — write once | "What happened" — session records |
| **Topics** | Warm (periodically updated) | Rewrite during tending | "What is X" — encyclopedic reference |
| **Trails** (`type: trail`) | Warm (periodically extended) | Append during tending | "How X evolved" — narrative across projects |

Topics and trails both live in `topics/`. Trails are distinguished by `type: trail` in frontmatter. Topics are rewritten (synthesized); trails are extended (new chapters appended). Trails are for concepts with genuine cross-project temporal evolution.

## Plugin Commands

- `/memex:save [title]` - Save current context as memo
- `/memex:status` - Show index stats and pending memos
- `/memex:open` - Open vault in Finder/Obsidian

Retrieval is skill-based — Claude searches automatically when you ask about past work. Vault maintenance is handled by the garden-tending skill. Use `memex search` or `memex timeline` CLI for direct access.

## CLI Commands

Shell commands — work from any terminal, any agent:

```bash
memex search <query>        # Hybrid search (FTS + vector)
memex ask <question>        # Deep retrieval with observations
memex timeline <date>       # Browse by date (yesterday, 7d, last week)
memex read <path>           # Read vault document to stdout
memex path                  # Print resolved vault path
memex check                 # Vault health — crystallization readiness
memex check --folders       # Detect project-folder drift (cwd-fragment names, duplicate/split folders)
memex check --validate      # Lint frontmatter (merged keys, missing title, dangling delimiter, no-frontmatter)
memex check --signals       # Open Recent-signals per topic (closed-section-aware) — the fold backlog
memex check --condense      # Project overviews lagging their memos (NEW after condensed:, ARR = git-added after condensed: but older-dated, GAP vs memos_digested)
memex status                # Document count, chunks, last rebuild
memex context               # Project detection and pending memo status
memex auth set-key          # Save a key locally for automatic loading (hidden prompt)
memex auth status           # Show credential source without exposing the key or calling Gemini
memex auth clear-key        # Remove the saved local key
memex cloud login --url U   # Save a memex cloud Worker URL + SYNC_TOKEN (hidden prompt)
memex cloud push            # Mirror the index to the cloud (incremental; --include-transcripts, --dry-run)
memex cloud pull            # Write memos saved by remote agents into the vault
memex cloud status          # What the cloud holds; pending inbox memos
memex similarity            # Detect near-duplicate topics (--threshold, --json)
memex scrub <path>          # Detect API keys / secrets (--apply redacts in place)
memex mark-saved            # Mark memo saved (prevents duplicate generation)
memex sync                  # Sync auto-memory into vault
memex graph <subcmd>        # Backlinks, orphans, tags, stats
memex topic resolve <slug>  # Resolve redirect_to chain (exit 0=ok, 1=skip/error)
memex index rebuild         # Rebuild search index (--full for embeddings)
memex index status          # Index health JSON (doc/chunk counts, embedding gaps)
memex index embed-missing   # Embed chunks/obs missing from vec tables (retry after API failures)
memex index migrate-vec     # Truncate vec tables to index_dimensions + add metadata cols (no re-embed)
memex index vacuum          # VACUUM the index to reclaim free pages (e.g. after migrate-vec)
memex session discover      # Find unprocessed sessions
memex session import        # Import discovered sessions (--apply to execute, --exclude ID to skip)
memex session reconcile-orphans  # Clear pending-memo signals whose memo carries their session_id (--apply; --trust-window also clears transcript/date matches)
memex obs topic <slug>      # All observations for a topic (cluster lookup)
memex obs stats             # Observation counts per topic
memex obs retag <old> <new> # Retag observations (for topic merges)
memex obs reassign --from-prefix X --to-prefix Y  # Rewrite obs+chunks doc_path (folder rename SOP)
memex obs untagged          # Observations with no topics (new-topic signals)
memex obs orphans           # Mirror rows whose parent observation is gone (--apply to prune)
memex obs export-sidecars   # One-off migration: write every doc's .obs.jsonl from the DB (--apply)
memex obs ingest-sidecars   # Diff vault sidecars into the index without a full rebuild
memex obs sidecars          # Sidecar health report (missing/orphan/stale/conflicts)
memex backfill obs          # Store supplied observations; requires --replace or --append
memex backfill tokens       # Backfill token counts on transcripts
memex backfill memos        # Backfill has_memo on transcripts
memex backfill topic-tags   # Propagate memo topics to observations
```

## Available Skills

| Skill | Purpose | When to Invoke |
|-------|---------|---------------|
| `recall` | Retrieve session memory — temporal browsing, keyword search, deep cross-session synthesis, or direct file loading | "what did I do yesterday?", "why did we...", "what patterns across...", "load the X topic" |
| `garden-tending` | Full vault lifecycle: diagnose, condense, connect, grow, maintain | "tend the garden", "update project overview", "check vault health", "find broken links" |
| `curator-practice` | Autonomous curator operating philosophy: attention, judgment, initiative | Autonomous tending, "what should I work on next?", scheduled/cron agents |
| `memo-writing` | Format and quality guidelines | `/memex:save`, "remember this", or when [memex] nudge appears |
| `project-consolidation` | Safely merge drifted/duplicate project folders (preserves obs via `obs reassign`) | "consolidate project folders", "merge duplicate projects", "fix detection drift", or after `memex check --folders` reports drift |

Skills are intent-based: Claude decides when to invoke based on user questions. This is more flexible than hooks which run on events.

## Gotchas

Domain-specific gotchas are in `.claude/rules/` and load automatically when working on relevant files. These are universal:

- **Vault selection is explicit or configured** — `memex` and dreamer use the configured vault; dreamer also accepts `--vault <path>`. The source-only `scripts/obsidian_cli.py` takes `--vault <name>`. Run relative script examples from the source checkout, not the vault
- **Observation topic slugs are not validated on insert** — `store_observation_topics` accepts any string. Use only slugs matching `topics/*.md` filenames. Invalid slugs leave tags pointing to nonexistent topics. `memex obs untagged` finds observations with no tags; it does not detect invalid slugs
- **Never delete from `observations` directly — route through `delete_observation_ids`** — three tables mirror it by observation id (`fts_observations`.rowid, `vec_observations`.rowid, `observation_topics`.observation_id). A bare `DELETE FROM observations` leaves mirror rows that every JOIN-ing read path silently discards, so the damage never surfaces in search — it shows up only as counters claiming observations the vault cannot return, and as orphan rows consuming vector-search KNN slots. `_OBS_MIRROR_TABLES` in `observations.py` is the single registry; a new mirror table must be added there or `test_mirror_registry_covers_every_table_referencing_observations` fails. Check any index with `memex obs orphans` (read-only; `--apply` prunes)
- **`memex backfill obs` requires an explicit write mode** — `--replace` deletes the document's existing observations and stores the supplied complete set; `--append` preserves them and adds observations. The Python `store_observations` API also requires `mode`. CLI output reports `mode`, `replaced`, and `skipped_duplicate` alongside stored/embedding counts and the sidecar path; net row loss emits a warning. Never use `--replace` for a partial addition.

- **memex cloud releases are tagged `cloud-v<version>` and are what deployed copies update to** — Deploy-button copies live in users' own repos and pull the newest `cloud-v*` tag through their `update-memex-cloud.yml` workflow (which fetches `cloud/scripts/update-from-upstream.mjs` from upstream on every run, so keep that script backward compatible). To release: bump `cloud/package.json` + `cloud/src/version.ts` together (a test enforces it), set `EXPECTED_SCHEMA` when adding a migration, set `CLOUD_SERVER_VERSION` in `src/memex/cloud.py`, tag `cloud-vX.Y.Z`, and start the release notes with `Server update: required | recommended | none`. Raise `MIN_SERVER_VERSION` only for "required" (push/pull then stop on older servers). Never rename or remove what button copies keep (`wrangler.jsonc` binding names, `package.json` name): the updater leaves those files alone and can only describe the change.

## Where to Go Next

Domain-specific details load automatically via `.claude/rules/` when you work on relevant files:

| Rules File | Covers | Loaded When Editing |
|------------|--------|-------------------|
| `architecture.md` | Memo generation layers, session lifecycle, search pipeline, frontmatter schema | `scripts/`, `src/memex/`, `hooks/`, `commands/`, `skills/` |
| `maintenance.md` | Periodic tasks, dev commands (rebuild, backfill, discover, sync) | `scripts/`, `src/memex/`, `_views/`, `topics/` |
| `configuration.md` | Config paths, path resolution, linking conventions, security | `scripts/`, `src/memex/`, `hooks/`, `.claude-plugin/` |
| `search-and-embeddings.md` | Embedding providers (Gemini primary, LM Studio fallback), chunking, search gotchas | `{scripts,src/memex/scripts}/{search,hybrid_search,embeddings,index_rebuild}.py` |
| `obsidian-cli.md` | Obsidian CLI 1.12.5 commands, SQLite fallback, graph navigation | `scripts/obsidian_cli.py`; `graph_queries.py` and `crystallization_check.py` under `scripts/` or `src/memex/scripts/` |
| `transcripts.md` | Transcript processing, JSONL format, system tag cleaning | transcript-related scripts |
| `hooks.md` | Hook implementation details, timing constraints | `hooks/` |
| `plugin-authoring.md` | Error patterns for commands, skills, hooks, scripts, plugin cache | `commands/`, `skills/`, `hooks/`, `scripts/`, `src/memex/`, `.claude-plugin/` |
| `python-patterns.md` | Python patterns used across the codebase | `scripts/`, `src/memex/`, `hooks/` |