AGENTS.md · git:20261114.a2ec774 · 2026-11-14 · sha256 dc5d4670cd7141b4
AGENTS.md git:20261114.a2ec774A
Immutable. This exact content never changes and is served at /api/v1/blob/dc5d4670cd7141b4.
# Omnesis
Index and search your entire digital life. Fully local, fully private.
`AGENTS.md` is the **agent contract** for working in this repo; `CLAUDE.md` is a symlink to it. It is kept intentionally short. The public docs (under `website/docs/`) are deliberately concise and user-facing; for architectural or behavioral detail, **read the code** — it is the only complete reference.
## Shared agent skills
Repository workflows live in `.agents/skills/<name>/SKILL.md`. Codex discovers
these directly; `.claude/skills/<name>` contains relative directory symlinks to
the same folders for Claude Code. Edit only the canonical files; do not add
copies under `.claude/commands/` or `.claude/skills/`.
Invoke a skill as `$skill-name` in Codex or `/skill-name` in Claude Code. The
shared workflows are `omnesis-review`, `source-review`, `dev-annotations`,
`latency-watch`, and `flaky-watch`. Agent-specific scheduling and delegation behavior is described
inside the relevant skill.
## Where to read what
All paths below are local files. Use `Read` (or `Grep` to find the right page), not URLs.
| Topic | Where |
| ------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------- |
| Engineering conventions (façade splits, defineSource, zod-at-boundary, …) — read first when establishing or imitating a pattern | `docs/conventions.md` |
| iOS-specific agent rules (XcodeGen, device deploy, SwiftUI preview + snapshot self-critique loop, TestFlight) — read before editing anything in `ios/` | `ios/AGENTS.md` (= `ios/CLAUDE.md`) |
| Public docs — what users see; update alongside behavior changes (see § Documentation maintenance) | `website/docs/` |
| Main-thread stalls — the 50 ms rule, the sanctioned fixes, and finding a culprit with the stall log, `/admin/stalls` and flight-recorder profiles | `docs/stall-investigation.md` |
| Architecture, per-source detail, HTTP API, config knobs | the code (repo map below) |
The repo at a glance:
```
packages/core/ shared types, branded IDs, define-source helpers, people-utils, attachments, the shared doctor health check (`@omnesis/core/doctor`)
packages/config/ config schema (omnesisConfigSchema, zod validators, defaults)
packages/gateway-client/ shared HTTP + WS GatewayClient implementation (HttpGatewayClient, GatewayWsClient) — extracted from collector
packages/gateway/ HTTP server, SQLite store, indexer, scheduler, search, analytics-db
packages/collector/ sync engine, source manager, auth subprocess
packages/cli/ unified `omnesis` CLI — read commands hit /search, /documents, /analytics, /status, /whoami; admin commands hit /admin/*; also hosts the daemons (`gateway serve` / `collector run`) and the `service`/`update`/`uninstall` lifecycle commands (`uninstall` discovers an install from disk in `src/uninstall/`)
packages/providers/* one package per provider (google, apple, notion, whatsapp, …)
packages/watch/ the Watch V2 DSL, validator, runtime and compiler — standalone by contract, imports nothing from the gateway; the gateway hosts it through `packages/gateway/src/watch/`
packages/agent-integration/ shared off-host integration runtime for OpenClaw and Hermes — transcript ingestion, subscription delivery, and scoped management tools
extension/ Chrome MV3 browser-capture extension — pairs as a `browser` device with a `write:web` token and pushes visited pages to the gateway-hosted `web` source; experimental Tell Omnesis notes and Find search require gateway `OMNESIS_EXPERIMENTAL=1` and use separately issued create-only and read credentials with independent capability discovery; store listing and privacy declarations under `extension/store/`, packaging in `extension/scripts/` (see docs/releasing.md)
scripts/release/ publish pipeline — stage packages, transform src-pointing manifests to dist at publish time (see docs/releasing.md)
ios/ native iPhone app — pairs with the gateway, hosts Apple Health
website/ static site Cloudflare publishes to omnesis.dev — landing page, public docs (website/docs/), privacy policy, installer mirror
docs/ internal design docs, process docs (e.g. issue-labels.md)
```
## Documentation maintenance — your obligation
The public docs are fourteen hand-written static HTML pages under `website/docs/`, published with the landing site at https://omnesis.dev/docs (the `site` workflow builds the site with `scripts/site/build.mjs` and deploys it to Cloudflare on every push to `main` that touches it and on every release tag: the blog, landing and policy pages come from `main`, while `website/docs/` and the installer files are taken from the newest stable release tag, so the live docs describe what the served installer installs; `main`'s docs are published unlinked at `/docs-edge/`, and each docs tree carries a release stamp its chrome shows). `website/docs/docs.css` + `docs.js` carry the shared chrome (design tokens extracted from `website/index.html`); every page embeds the same nav / sidebar / footer, so structural changes must be applied to all pages.
When you change user-visible behavior — defaults, CLI commands, source semantics, setup flows, new or removed features — check the affected page and update it **in the same commit**:
| Page | Covers |
| ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `website/docs/index.html` | what Omnesis is; core concepts (gateway, collector, devices & tokens and scopes, sources, index, people & links, analytics); data layout; help & support |
| `website/docs/install.html` | installer, Docker, from-source, first run, uninstall |
| `website/docs/setup.html` | portal, certificates & remote access, collector / iOS / Android / browser-extension setup, devices, tokens, repair |
| `website/docs/search.html` | search pipeline and syntax, CLI query commands, delete, SQL analytics |
| `website/docs/agent.html` | built-in agent, memory, conversations, model assignment, standing instructions |
| `website/docs/connect.html` | external agents: access levels, privacy policy, audit, answer API, bring-your-own-agent (MCP), OpenClaw and Hermes |
| `website/docs/apps.html` | getting the apps, mobile-app usage, phone-only sources |
| `website/docs/notifications.html` | what sends a notification, private delivery, direct APNs/FCM setup, push status and repair |
| `website/docs/operating.html` | services, monitoring (status/doctor/logs), config, models, troubleshooting, CLI at a glance |
| `website/docs/security.html` | encryption at rest & keyring, security posture, hardened gateway, TLS lifecycle |
| `website/docs/updating.html` | backup, restore & export, update (single host, fleet, Docker), source-install migration, version compatibility |
| `website/docs/sources.html` | available source list (experimental sources labelled) + the data each contributes; managing sources; sources on several devices |
| `website/docs/building-sources.html` | the source developer contract |
| `website/docs/experimental.html` | experimental-mode gate; Brain and Watch operator guidance |
Authoring rules for these pages:
- **Keep experimental features contained.** Stable pages must not present gated behavior as generally available. Experimental operator guidance belongs only on the dedicated `/docs/experimental` page, must lead with the `OMNESIS_EXPERIMENTAL=1` gate, and must label the behavior unstable. The one exception is experimental sources: they are listed in the `/docs/sources` catalogue, each clearly labelled experimental together with the gate that offers it. When a feature graduates, move its documentation into the relevant stable page as part of graduation.
- **No images.** Terminal interactions use the `.term` component; diagrams are ASCII in a `.term` `<pre>` or plain HTML/CSS.
- Links between docs pages are page-relative and extensionless (`install`, `setup#certificates`, `./` for the overview), so the same pages serve at `/docs/` and `/docs-edge/`; Cloudflare serves `install.html` at `install`. Links to the rest of the site stay root-absolute (`/blog/`, `/privacy`). The site build fails on a link that does not resolve.
- Voice: technical, precise, calm; no marketing adjectives, dated or release-status markers, or issue numbers. The required unstable warning on the Experimental page is the exception. Example data is always fictional (see § Privacy).
- `website/mobile-privacy-policy.html` is the iOS/Android privacy policy. Keep the legacy `/privacy-policy` redirect until every mobile-store listing uses `/mobile-privacy-policy`.
## Source encapsulation
All logic specific to a particular data source lives inside that source's provider package (`packages/providers/<name>/`). No bespoke source-specific code in `core/`, `gateway/`, `collector/`, `cli/`, `portal/`, or `ios/` outside the provider. This includes:
- Branching on a specific source name (e.g. `if (sourceType === "gmail")` in shared code).
- Source-specific unit nouns ("an email" vs. "a file" vs. "a message") hardcoded in shared UI/CLI.
- Source-specific URL handling — deep-link construction, canonicalization, parsing.
- Source-specific display — icons, colors, text strings — in portal or iOS.
- Source-specific schemas, validation, or normalization in shared code.
The `defineSource()` descriptor is the contract. If a consumer needs a piece of source-specific data, the source exposes it via the descriptor (unit noun, icon, color, deep-link builder, …) and consumers read through the registry. If something is missing from the contract, extend the contract — don't hardcode the source name downstream.
Generic abstractions are fine: routing on `:sourceType` as a path parameter, the source registry iterating over all sources, tests that legitimately enumerate sources.
A daily Claude Code routine audits the previous day's commits and opens one GitHub issue per violation, assigned to the user.
## Experimental features
Ship not-yet-battle-tested work in the tree but gate it so it cannot surprise an operator. `OMNESIS_EXPERIMENTAL` reveals **Settings → Omnesis Lab** on the portal and native apps. Brain, Tasks, and Watches require independent persisted Lab opt-ins; the environment flag alone never enables them on a fresh install. Gateway `GET /status` advertises effective flags as `lab.{brain,tasks,watches}`. Lab choices are internal database state, separate from feature settings. Existing task history, existing watches, and an assigned background-agent model preserve the corresponding opt-in during upgrade. Explicit opt-outs survive restart and disabling experimental mode.
The Brain processes data autonomously in the background and uses the internal `background-agent` model role. Chat and the bounded Tell Omnesis note agent use the ordinary `agent` role independently. Tasks can run without the Brain (manual changes and user-requested task tools); background task suggestions and updates require both. When Tasks is disabled, agent prompts, tools, task context, and background task producers omit it. Watches run independently of the Brain. Settings → Sweeps and Debug → Brain require the Brain opt-in; old `/portal/debug/cognition` links remain compatible.
Other experimental surfaces retain their existing environment gate: source descriptors mark `experimental: true`; code reads `experimentalEnabled()` from `@omnesis/core`, or `experimentalVisible()` for synthetic demo discovery. Stable privacy, MCP Answer/Direct, interactive memory, note processing, voice ask, quick capture, and evidence invalidation remain independent of the Brain. Public experimental guidance belongs in `website/docs/experimental.html`; detailed operator references live in `docs/brain.md`, `docs/watch-runtime.md`, and `docs/sweeps.md`.
## Developer annotations — a feedback channel from the operator to you
A second, independent gate — `OMNESIS_DEV_MODE` (parsed like `OMNESIS_EXPERIMENTAL`; `devModeEnabled()` from `@omnesis/core`; advertised as `developer` on `GET /status`) — reveals a hidden **developer-annotations** channel. When it is on, the operator can attach a short free-text data-quality note to whatever entity they're looking at (a document, brief, loop, temporal annotation, agent run, …) straight from the portal (a floating ⚑ button) or the mobile apps (shake-to-annotate). These notes are for **you, the engineer — not the Omnesis agent**: the operator uses them to flag inconsistencies without maintaining an offline list of entity IDs.
**Read the open worklist at the start of a session with `omnesis dev-annotations`** (`--json` for machine parsing; `resolve <id>` / `rm <id>` to close one once you've acted on it). Each note carries the target `{type, id}`, a deep link, and a snapshot of what the operator was looking at. The notes live in the `dev_annotations` table; the `/dev/annotations` routes 404 when dev mode is off. Developer annotations remain absent from the public docs.
## Privacy — never use the user's corpus as inspiration
Omnesis indexes the user's personal data: emails, messages, contacts, health metrics, calendar, files. While working in this repo you may see real names, phone numbers, email addresses, addresses, dates, vendors, and private context belonging to the user, their family, friends, and contacts.
**NEVER use any of that as inspiration for example data, mock fixtures, preview content, test inputs, code comments, commit messages, PR descriptions, or anywhere else in the codebase.** That includes paraphrasing — a "Cadeau Massage Élise" voucher fixture inspired by a real WhatsApp thread with the user's sister is just as bad as copying the thread verbatim. The user's family, friends, vendors, and contacts have appeared in real fixtures; that is the kind of leak this rule exists to prevent.
When you need example data, **invent it from scratch**:
- Names: use clearly fictional names not appearing in the user's corpus (e.g. `Maya Reeves`, `Jamie Lopez`, `David Lin`, `Sarah Mendez`). Don't reuse a name you saw in a search result, email, or person merge.
- Emails: use `example.com` / `example.org` / `example.io` (RFC 2606 reserved domains).
- Phone numbers: use `+1 (555) 010-0xxx` (NANP fictional range) or `+44 7700 900xxx` (Ofcom-reserved UK fiction).
- Addresses: use placeholders like `42 Example Street`.
- Vendors / businesses / venues: invent (e.g. `Stellar Sound`, `Studio Northstar`, `Riverside Estate`). Don't reuse a real vendor name you encountered in the user's invoices, contracts, or threads.
- Stories / scenarios: invent generic ones (heart-rate trend, Q4 budget review, marathon entry form). Don't structure a fixture around a personal incident you observed in the data.
If you're uncertain whether something is from the corpus, assume it is and invent fresh. Extending mock files like `ios/Sources/Omnesis/UI/PreviewMocks.swift` is fine; sourcing the content from the user's actual data is not.
This rule applies to every part of the repo: iOS previews, snapshot tests, gateway tests, provider tests, evals fixtures, docs examples, generated content, comments, anything.
## Agent rules
1. **Plan mode default.** Enter plan mode for any non-trivial task (3+ steps or architectural decisions). If something goes sideways, stop and re-plan rather than push through.
2. **Use subagents liberally.** Offload research, exploration, and parallel analysis to keep the main context window clean. One task per subagent for focused execution.
3. **Verify before done.** Never mark a task complete without proving it works. Run tests, check logs, demonstrate correctness. Ask "would a staff engineer approve this?"
4. **Demand elegance for non-trivial changes.** Pause and ask "is there a more elegant way?" Skip for simple, obvious fixes — don't over-engineer.
5. **Autonomous bug fixing.** Given a bug report, fix it. Point at logs, errors, failing tests, then resolve them.
6. **Simplicity first.** Every change as simple as possible. Impact minimal code.
7. **Root causes.** No temporary fixes. Senior-developer standards.
8. **Learn from corrections.** When the user corrects an approach, save the pattern to your memory system (`~/.claude/projects/.../memory/`) so the next session has it. Iterate until the same mistake stops happening.
## Tests are mandatory
- **CI runs the lanes a pull request affects, all Linux lanes on every push to
`main`, and the full suite twice daily or on manual dispatch**, on GitHub-hosted runners (`.github/workflows/full-validation.yml`);
a newer push to a pull request cancels its older run. A pull request's `scope`
job (`scripts/nx/ci-scope.mjs`) plans its merge commit with the same Nx affected
plan as `checks:plan`: the privacy scan, format and lint of the changed files,
the fast typecheck plus the affected projects' typechecks, and the affected
projects' unit suites always run; the E2E files of the selected bundles, the
Playwright portal lane and the embedder lane run when selected; the Android
lane runs when the plan selects an Android bundle; the native, install,
Docker, harness and security lanes run when the change touches the paths they
cover. The lockfile, a package manifest, a root toolchain config, a workflow,
`packages/core`, the check runners and planner, or an unsupported path run
every Linux lane. The macOS lanes (the macOS Node lane, the Swift linters and
iOS, the Android screenshots, the Mac installer) run on a pull request only
for the paths they cover, never for a shared input: GitHub lends the
repository a few macOS runners at a time. The one exception is iOS, which also
runs when the plan selects an iOS bundle for a gateway wire path the app
decodes. Every push to `main` keeps its own Linux run. Full sweeps at
01:17 and 13:17 UTC include macOS; manual runs default to every lane. A lane a
pull request skipped can redden the next full sweep if the change broke it. Before
handoff, run the affected gate for the actual branch and working tree:
`npm run checks:plan -- --base origin/main`, then
`npm run checks:affected -- --base origin/main`. Nx selects package work
through the dependency graph and adds declared behavioral E2E/native bundles.
**A confirmed unresolved regression on `main` stops unrelated merges.** Fix
it forward or revert the responsible merge; a pull request whose skipped
macOS lane failed on `main` is the first suspect. Inspect the failing job
before treating a red verdict as current: an older commit's failure can be
stale after a relevant fix, and a suspected flake needs supporting evidence.
Record the failed test, the relevant fix or transient cause, and current
validation before allowing unrelated merges. Queued runs are not failures,
but green pull-request checks alone do not dismiss a confirmed regression. Do not
re-push a green pull request only to rebase it: `main` does not require a
rebased branch to merge, and a new push cancels its finished run.
- Establish a focused baseline before editing, then rerun the affected tests while iterating. Re-run `checks:affected` after relevant edits because queued evidence is bound to the planned tree and rejects changed inputs. Use `npm run checks:bundle -- <name>` to add a diagnosed bundle; a narrower manual bundle does not replace the affected gate. `npm run checks:full` remains available for CI, migration validation and manual diagnosis. `npm run test:unit` excludes the spawned-gateway suite; use `npm run test:e2e -- <path/to/file.e2e.test.ts>` for a focused gateway check.
- **No chat or agent-model inference in tests, ever.** Substitute the model at a production seam: replay backends, puppet/cassette models, or scripted verdict servers. Local embeddings are the deliberate exception when a suite is specifically proving semantic retrieval quality; `search-quality.e2e.test.ts` and `embedder-swap.e2e.test.ts` keep their real local embedding dependency, which `scripts/test-embedder.sh` provides.
- Every new feature or bug fix must include tests. Every new source needs sync (bootstrap + incremental), normalization, and error-handling tests — and a `source-review` skill pass (`.agents/skills/source-review/SKILL.md`, the complete source contract as a checklist) before its PR opens.
- Tests use `vitest`. Place test files next to source files as `*.test.ts`. Spawned-gateway E2E tests are named `*.e2e.test.ts` and live under `packages/collector/src/e2e/`. Two shared harnesses: `SyntheticE2EHarness` boots a real gateway against a chosen universe (default: `default`) and drives real sources; every construction must name its `gatewayMode` (`stable`, `experimental`, `synthetic`, or `synthetic-experimental`) so the spawned product surface never inherits the test runner's synthetic flag. `MultiCollectorHarness` (`multi-collector-harness.ts`) boots a gateway plus any number of in-test pseudo-collectors that pair, hold a WS connection and answer source-lifecycle commands — deliberately no `SyncEngine`, so it covers registration and dispatch; its one document seam is `pushDocuments(collector, docs)`. `SyntheticE2EHarness` pairs the universe's declared device roster and syncs every source as the device the manifest attributes it to (see `docs/universes.md`).
- Existing E2E suites worth knowing about before touching their territory: `replay-scenarios.e2e.test.ts` (drives every replay-agent scenario via HTTP/SSE), `golden-corpus.e2e.test.ts` (snapshot-based ranking/people/search regression net over the LIKE-based `/documents/search`), `search-quality.e2e.test.ts` (the SEMANTIC net golden-corpus omits — wires the real local embedder at `:8001` via `extraInference`, asserts recall@k + ordering on the real `POST /search` hybrid pipeline over an invented judged-query set where a lexical baseline loses; required dependency, so an unreachable `:8001` FAILS in CI and only skips on a local dev box; `scripts/test-embedder.sh start` serves the pinned model on the CPU there, as CI does; `OMNESIS_TEST_EMBEDDER_URL`/`OMNESIS_TEST_EMBEDDER_MODEL` override the defaults), `cli.e2e.test.ts` (drives the `cli` binary, including `sources resync` with and without a device), `sources-add-cli.e2e.test.ts` (the `cli` binary with no terminal against a real collector hosting the real Obsidian source on `MultiCollectorHarness`: `sources describe --json` names the keys, a missing setting, an unknown key and a refused folder each exit non-zero naming the fix and register nothing, and `--set` adds the vault unattended), `link-extraction.e2e.test.ts` (asserts linkBackfill + linkStatsRefresh populate `document_links` + `link_stats`), `people-graph.e2e.test.ts` (asserts interactionScoresRefresh fires, mergeRulesEval propagates user-issued merges, autoDetect endpoint stays healthy), `trail-graph.e2e.test.ts` (boots default universe, asserts attachment/dedup/thread links + trail traversal across WhatsApp and Gmail), `multi-collector.e2e.test.ts` + `multi-account.e2e.test.ts` + `source-ownership.e2e.test.ts` (all on `MultiCollectorHarness`: per-device dispatch; one account's removal/pause never disturbs a sibling account of the same source type; cross-collector add refusal + explicit PATCH-deviceId move), `device-roster.e2e.test.ts` (the universe's device roster is what the gateway sees: every seed source is owned by, and syncs as, the device the manifest attributes it to), `source-membership.e2e.test.ts` (two collectors on a `replicated` source: a second add joins instead of 409, member-scoped snapshots and fan-out, Sync now on every member, per-member cursor rows that adopt the shared one, detach and owner hand-over, and a per-device resync that resets one member's cursor only), `needs-auth-reminder.e2e.test.ts` (on `SyntheticE2EHarness` with fake APNs: a collector reporting `needs-auth` earns one reminder per provider connection per device, backed off on the persisted ladder and reset only by that device's own recovery — a sibling member's successful sync leaves a lapsed member's reminder in place), `partitioned-universe.e2e.test.ts` (on `SyntheticE2EHarness` with the `partitioned` universe: two Android phones push identically named day rollups and identically keyed Health Connect rows into partitioned sources; each phone's documents and analytics rows live in its own stream, the union is the corpus, a phone's snapshot, tombstones and exists-checks stay in its stream, a per-device resync wipes one phone's stream and cursor, and a detached phone's stream goes with it), `sync-lease.e2e.test.ts` (on `MultiCollectorHarness`: two collectors race for a handoff source's sync lease — the holder's pages renew it, a non-holder's page is refused, a lapsed lease prefers its online incumbent and passes to a sibling once it is offline, release hands over at once, Sync now reaches the holder; a replicated member without the lease commits its page but defers the snapshot reconcile), `replicated-universe.e2e.test.ts` (on `SyntheticE2EHarness` with the `replicated` universe: two collectors share Things and two phones share Apple Health as members; the corpus equals one host's, snapshot reconcile under two contributors deletes nothing, per-host cursors stay monotone, and a host detaching or being revoked loses no data), `handoff-universe.e2e.test.ts` (on `SyntheticE2EHarness` with the `handoff` universe: two collectors host one Gmail account as a handoff source and every tick runs the collector's own lease branch — the owner's tick takes the lease and leaves one shared `sync_state` row, a member's tick while it is held is skipped with no status event and no write, a lapsed or auth-released lease passes to the sibling's next tick which continues from the shared cursor, and a detached-then-rejoined member is skipped again), `replica-deletion-dispute.e2e.test.ts` (on `MultiCollectorHarness`: two collectors host one replicated source whose replicas disagree about an item — the holder's first tombstone deletes it and resets its sibling once, the sibling's bootstrap makes it disputed, later tombstones are stripped without deleting or resetting, the dispute survives a gateway restart and a per-member resync and is visible on the source and the holding member, the restorer's own tombstone settles it without the lease, and a detached member closes it), `replica-snapshot-dispute.e2e.test.ts` (same harness, no tombstones at all: the holder's corroborated snapshot omissions delete once through the absence sweep and reset the sibling once, the sibling's restore is a dispute the sweep leaves alone, one omission by the restorer never settles it while spaced omissions past the age floor do — naming the item resets the clock — a lease hand-over, a gateway restart, a per-member resync and a detach each leave the ledger consistent, and a third member's verdict cannot settle another member's dispute), `replica-analytics-dispute.e2e.test.ts` (same harness, the analytics plane: every page sent the way a collector tick sends it — rows and tombstones on a fresh epoch, the cursor, then the snapshot; the holder's row tombstone deletes and resets once and its repeats are disputed, a non-holder's fresh tombstone is deferred with its rows kept, the restorer's tombstone settles without a reset, the holder's corroborated omissions delete through the sweep as its verdict and reset every member once while a restorer settles by spaced omissions only, a gateway restart keeps the count, and a detached restorer closes its dispute; after a lease hand-over the new holder is a restorer whose corroborated omissions settle its own dispute; and an absence a detached member's snapshots earned still deletes when swept but carries nobody's verdict), `permission-remediation.e2e.test.ts` (on `MultiCollectorHarness`: a sync failure's structured remedy is served beside its message, a generic failure never gets one, only the member that hit it carries it, it survives a gateway restart, and the member's own recovery clears it), `self-identity-registry.e2e.test.ts` (on `MultiCollectorHarness` with a `config.self`: a collector's self-identity hook push puts the source's LID alias on the self person through the real writer thread, a sibling collector declaring fewer hooks never erases its peer's, and a source added on that sibling still pairs through the merged registry), `pairing-lifecycle.e2e.test.ts` (on `MultiCollectorHarness`: a revoked collector that still hosts sources reads as `needs-pairing` on the gateway and in the CLI while a revoked one with nothing to bring back does not, `omnesis devices repair` mints a code bound to that row, redeeming it restores the same device id with its ownership, membership, cursors and documents intact, and the gateway-host collector's own re-registration reclaims its revoked row; an iPhone code is offered the trusted Tailscale name first and refused a Tailscale IP that an Android code is given, and a code that is not pending gets no addresses), `gateway-lifecycle.e2e.test.ts` (on `MultiCollectorHarness`: a second gateway pointed at a live gateway's config dir is refused by `gateway.lock` before it opens any store and names the owner while the owner keeps ingesting; a replacement started while the owner drains waits for the lock and takes the directory over; a gateway booted after its predecessor was SIGKILLed reclaims the stale lock), `browser-capture.e2e.test.ts` (on `SyntheticE2EHarness`: the extension's push module under Node against a real gateway — a `browser` code minted without naming scopes receives the gateway's default grant, pinned to `defaultScopesForDeviceKind("browser")`; the `write:web` token is refused for any other write; documents and `page_visits` rows land and upsert; the gateway-owned capture policy read with the browser token folds in the collector's owned domains, an exclusion with purge, a pause and their removal are what the gateway serves back, a page deleted for good is named in the policy and refused on re-push as `suppressed` while a copy-only delete lets it return), `browser-extension.e2e.test.ts` (the built extension in a real headless Chromium on `SyntheticE2EHarness`, built with the test manifest — pinned id, host access pre-granted: pairing through the options page stores a `write:web` credential, a routed fixture page dwells into a document and a `page_visits` row, the popup reports it, a CDP-stopped service worker revives and the next page still lands, an exclusion made on the options page lands on the gateway with a purge and a page on the excluded domain queues nothing after a full dwell, excluding the open page from the popup confirms in place, lands on the gateway and stops that page capturing without a reload, the popup's pause is the gateway's pause, a page deleted for good is not captured again, a revoked device makes the popup say re-pair, and unpair leaves no credential, queue item or staged capture; the Playwright Chromium is a required dependency, so the suite fails in CI when it is missing and skips with a notice on a dev box without it), `decision-backends.e2e.test.ts` (a stable gateway and one scripted provider speaking every decision wire format beside an OpenAI-compatible model list: a decision-only backend added and assigned in one config write with no key or consent on this host and refused for every other role, decisions enabled on a backend already serving chat in one write that carries no key, its one key sent to both APIs and a rotated key reaching both, removing its decision API leaving the chat side intact, a model the server lacks refused at assignment, keys kept out of `omnesis.json` and sent as each provider's own header, Workers AI's envelope and OpenAI Decisions' schema, an off-host decision API left unused without consent, the mention worth gate and capture routing served through each, and a moved mention worth threshold re-sorting the judged emails from their stored scores with no new model call), `brain-decision-gate.e2e.test.ts` + `brain-decision-bootstrap.e2e.test.ts` + `brain-decision-replay.e2e.test.ts` (the Brain's worth gate against a scripted TypeSafe System One server or recorded decision cassettes: skipped emails settle with no agent turn, attachments are judged by their parent, an outage fails open, a configured `brain.thresholds.worthGate` gates at its own cut, every judgement lands in the decision ledger and on the runs routes, and gated history shows as its own bootstrap band), `decision-self-hosted.e2e.test.ts` (the mention worth gate with the decision role on self-hosted backends: on an experimental gateway, since chat models serve the role only there, a scripted OpenAI-compatible server is asked for one token with logprobs per question and its label probabilities become the ledger's scores under `http:<backend>/<model>`, a reply without logprobs is recorded `unavailable` and drops nothing, and on a stable gateway a loopback TypeSafe endpoint serves with no key and no remote inference), `brain-record-check.e2e.test.ts` (the record check on the same scripted server: under `enforce` a low-scored new record is not saved and the agent is told not to retry, `shadow` saves it and records the verdict as not enforced, `off` asks nothing, an outage saves the record as `unavailable`, and the tokens are record-check spend that counts no run), `brain-task-checks.e2e.test.ts` (the task checks on the same scripted server: a new title an accepted task would take on its own waits as a proposal when scored as another task and applies when scored as the same one, and a note scored as repeating the task's history is not recorded; each verdict is a `task-check` row on the run), `brain-interest.e2e.test.ts` (the interest engine on the same scripted server: accepting, creating and ignoring tasks, thumbs on brief items and a statement in the Morning brief conversation each land as a signed example and undo takes one back; a new suggestion is judged against them in shadow, withheld or adopted under enforce with an undoable event, an outage fails open and a switched-off ledger asks nothing; a factual-error thumbs becomes a Tell Omnesis note; salience and intake urgency requests carry `owner_interest` only when the ledger holds something alike), `brain-wiki-overlap.e2e.test.ts` (on the Brain Bench with more wiki pages than one library page holds: a new page on a subject an unread current page already covers is refused naming that page, the run enriches the existing page and settles the candidate as merged, and the wiki count is unchanged), `brain-wiki-edit.e2e.test.ts` (on the Brain Bench: a review run enriches an existing page with `knowledge_edit` — one section replaced, one sentence changed — and the stored revision is exactly those edits, the untouched overview byte-identical and every claim kept; an edit without an editing read, to a missing heading, removing a claim without `claimRemovals` or citing a reference at a stale version is a tool error that saves nothing; each write and refusal logs one content-free adoption line), `brain-wiki-not-a-log.e2e.test.ts` (on the Brain Bench with a stream of automated build-status notifications: publishing a page titled with a cited notification's subject line is refused and the page is saved under a name for its subject; when a later batch reaches the page, an edit appending another date-named section to a page whose sections are mostly date-named is refused, the page is queued for whole-page review, and the run updates the current-state passage in place, so no revision grows the log), `brain-status-stream.e2e.test.ts` (on the Brain Bench: a page about a build service that already reads as a log gets exactly one immediate consolidation review the first time a maintenance pass touches it, recorded on the page so no later pass queues another; a stream of routine build-status notifications the discovery worth gate scores low is gated with no agent run, no discovery target and no page work or revision, and each judgement stays in the decision ledger under the current rubric), `brain-organization-worth.e2e.test.ts` (on the Brain Bench with evidence interpreted under the previous discovery rubric: before a chronological cohort starts, members without a current-rubric verdict are judged once through the discovery gate; routine build-status notifications judged low have their organization coverage gated, join no cohort and leave the page about their service without a new revision, members still worth organizing start the cohort, and every judgement is in the decision ledger under the current rubric), `brain-source-deletion.e2e.test.ts` (on the Brain Bench: deleting a source a wiki page cites tombstones the source and invalidates the page instead of deleting it — exactly the claims that relied on the source turn stale in a new revision of the unchanged page — and a maintenance run reviews it and retires it with `knowledge_retire` and a reason, keeping the page and every revision on record), `brain-source-deletion-owners.e2e.test.ts` (on the Brain Bench: a maintenance run turns an email into a loop the owner accepts into Tasks and a document annotation; the collector's sync tombstone then deletes the email, which becomes unreadable, while the task stays in the journal and the annotation in memory — each loses only its link to the email, turns stale with a ledger entry or `unverified`, and its knowledge node is queued for review — and a maintenance run reviews both, keeping the task and retracting the annotation as deliberate decisions), `brain-excluded-claim-repair.e2e.test.ts` (on the Brain Bench: a page with claims citing a source the operator then excludes is repaired after another cited source changes — the excluded-source claims are not offered for review and settle as skipped, a save that removes four of six claims with one reason under a new title is refused and saves nothing, an edit adding a new claim that cites the excluded source is refused as `excluded_source` and no verdict reads that source, the targeted edit leaves the page current and its untouched readable claim pending for review, and every claim survives), `task-conversation.e2e.test.ts` (on `SyntheticE2EHarness` with a scripted `agent` model: "Chat about this" is advertised on `/status` and `/health`, `POST /tasks/:id/conversation` opens an empty conversation carrying the task's card snapshot that resumes like any other, and every turn's system message carries the task as it stands at that turn — after an edit, the new title; after a delete, a notice that it is gone; a plain conversation carries none of it), `maildir-lifecycle.e2e.test.ts` (the real Maildir provider behind a real collector's `SourceManager`, `SyncEngine` and command channel on `MultiCollectorHarness`: added through the portal's resolve-and-add path against a Maildir tree on disk, then delivered to, starred, relabelled, archived, deleted from, made unreadable, restarted and resynced — the pattern for driving a local-file source end to end without the synthetic twin), `account-enrollment-connect.e2e.test.ts` (a real relay with account sign-in, env-configured gateways and a headless Chromium passkey: a gateway with only the flag, a relay URL and a local ACME directory connects from its portal Omnesis account page through sign-in, registration, a DNS-01 certificate over the relay and the account link, explicit account creation with "I already have one" refusing an unknown sign-in, starting over with a different account, and a CLI then joins it automatically; `omnesis cloud connect` links a gateway under the open and allowlist signup policies, pins the one sign-in and records `--ask-first`, and a refused account links nothing), `account-enrollment-devices.e2e.test.ts` (same setup: a freshly linked gateway with no policy chosen joins a trusted sign-in automatically and holds the same sign-in once switched to ask, ask-first holds and trusting a sign-in, a second passkey that joins automatically once the first vouched for it and is held again after its pin is removed, the account page handing its sign-in to the portal, the CLI joining a collector automatically that then syncs over the relay with a device-joined notice, a device that joined automatically administering the gateway at once, which row a sign-in lands on, device-key token rotation, and the portal joining or showing the comparison), `account-enrollment-attacks.e2e.test.ts` (a relay-minted attestation only holds a request, a swapped certificate gets 403 and saves nothing, a captured request cannot join twice, and a leaf without an embedded SCT makes the device require approval), `account-enrollment-sign-in-purpose.e2e.test.ts` (an outsider who gets the owner's passkey or Google sign-in on a gateway sign-up or owner-link page it started with its own device's enrollment nonce is refused with 401 when it offers that proof to the owner's gateway, the link pages wait for "Connect this gateway", and ask-first and a declined sign-in survive disconnecting and relinking while ask-first survives the account's deletion), `account-enrollment-confirmed-signins.e2e.test.ts` (with a local stand-in for Google: an account created with Google on one gateway gets a passkey that countersigns the Google account, a second gateway linked with that passkey joins a Google sign-in with no approval, and a Google account the relay adds to the account, listed alone or written into the signed list, is held), `account-enrollment-flag-off.e2e.test.ts` (without `OMNESIS_CLOUD_RELAY_ENABLED` nothing differs from a gateway without the feature: every enrollment and sign-in route 404s, `/health`, `/status`, the portal session and device lists carry no new fields, the login page is told nothing, strict QR trust modes are refused, `omnesis cloud` is an unknown command and QR pairing scopes are unchanged; extend it with every new surface), `ios/Tests/.../GatewayLiveE2ETests.swift` (Swift transport against a live gateway, invoked via `scripts/run-ios-e2e.sh`).
- **The Brain Bench** (`packages/collector/src/e2e/brain-bench/`, suites `brain-*.e2e.test.ts`) is the end-to-end correctness net for the Brain — read its [`README.md`](packages/collector/src/e2e/brain-bench/README.md) before touching the brain or writing a bench test. A bench boots a real gateway with the whole engine live (waker → queue → rhythm → drainer → run driver → tool layer → gates → cascades) and substitutes only the model, at the production `background-agent` seam. Two lanes: a **puppet** model that decides each turn from a behavior table and emits real tool calls, and **cassettes** — recorded reasonings replayed with live tool calls, so their writes land for real (`scripts/record-brain-scenario.mjs` builds one from a run transcript). A scripted verdict server makes the fail-open `entailment-verifier` gate (annotations, temporal writes, and observations at the morning recheck) reachable in its reject arm, a scripted decision server (or a universe's decision cassettes) stands in for the `decision` model behind the worth gate, and `OMNESIS_BRIEFS_VIRTUAL_CLOCK` drives day boundaries deliberately instead of by sleeping. **The bench asserts correctness, never quality** — that a decision was carried out properly, never that it was a good decision; quality stays in the scorecard/eval lane so improving a prompt can't redden correctness CI. A brain change that touches a tool, gate, cascade, queue mechanic or rhythm task should add or update a bench test in the same PR.
- `npm run test:e2e` uses a memory-aware launcher: at most two E2E files run in parallel, and only on hosts with at least 32 GiB process-visible total / 16 GiB currently available memory; constrained hosts stay sequential. Suites sharing the real embedder, the timing-sensitive load soak, the restart-sensitive encrypted-sidecar suite, and the headless-Chromium extension suite run in a separate serial phase; aggregate machine reporters also use one serial Vitest process so they produce a single valid output file. `OMNESIS_E2E_WORKERS=1|2` is the explicit benchmark/debug override. The launcher also holds a lane lock for the whole run — a ticket directory under the system temp directory, so lanes of users or containers with separate temp directories do not see each other — and a second `test:e2e` on the same machine queues behind the first (naming it) instead of failing both; `OMNESIS_E2E_LOCK_WAIT=0` fails fast instead of waiting, and `OMNESIS_E2E_LOCK_TIMEOUT_MS` caps the wait (two hours by default). Spawned test gateways use small worker pools while preserving every worker path, so bounded file concurrency does not multiply production-sized pools.
- `npm run validate-universes` (= `npx tsx scripts/validate-universes.mjs`) checks every in-tree universe for structural integrity. Also runs in CI before the test suites.
- Gateway DB tests use unique temp paths (`/tmp/omnesis-test-{uuid}.db`).
- Integration tests use ports 17601+ to avoid conflicts with running services.
## Fast inner loops
Pick the **narrowest** signal that covers your change first, then widen to
`npm run checks:affected -- --base origin/main` before handoff. Measured on
the dev box (warm = the project already built/transformed once this session):
- **One test file** — `npm run test:unit:vitest -- <path/to/file.test.ts>` ≈ **0.5s** wall. This is the
tightest loop; use it while iterating on a single unit.
- **One package** — `npm run test:unit:vitest -- packages/<pkg>` ≈ **1.3s** wall for a ~40-file package.
Use it once a file passes, to catch cross-file fallout within the package.
- **A portal-only change** — `npx vitest run packages/gateway/portal` (~2s) is the whole
gate, together with `npm run typecheck`, lint, the PII scan and the mandatory screenshot.
Skip `test:e2e`: `portal.e2e.test.ts` checks that the portal's files are served and that
routes fall back to `index.html`, and no E2E suite renders a portal view, so the lane
costs tens of minutes (plus whatever it queues behind) and covers none of the change.
The portal lane does read the stylesheet — `js/stylesheet.test.ts` parses `css/style.css`
and fails on an unbalanced brace, which otherwise silently drops every rule after it — so
a CSS-only change is not untested, but a screenshot is still the only check on what it
looks like. Shared gateway changes select their declared backend and client
bundles through the affected plan.
- **Developer-script smoke tests** — `npm run test:unit:smoke` runs the gateway-spawning
script checks separately from the bulk unit workers. `npm run test:unit` includes this lane
automatically; the split keeps the isolated gateway boot out of the bulk unit workers.
- **Type errors, one package** — `npx tsc --build packages/<pkg>` builds only that package
plus its dependency chain (warm ≈ 0.2s; cold ≈ 0.8s for a small leaf package, ~3s for
`core` + its deps). The monorepo uses project references, so `tsc --build` is **incremental**
— it consults each package's `.tsbuildinfo` and rebuilds only what changed.
- **`npm run typecheck:fast`** — `tsc --build` over the app packages (`gateway`, `collector`,
`cli`) and their shared-library deps, **skipping the ~30 standalone provider packages**.
Cold ≈ **10s** vs the full `npm run typecheck` (all 45 project references) cold ≈ **16s**;
warm both finish in well under a second. Pass a package path to narrow further:
`npm run typecheck:fast -- packages/<pkg>`.
- **`npm run typecheck:watch`** — `tsc --build --watch`; leave it running in a pane for a
live red/green type signal as you edit, instead of re-invoking the full typecheck each time.
The affected plan is the authoritative branch gate. It includes production and
test typechecks, lint, unit work and declared integration/native bundles for the
current inputs. Full validation remains an explicit diagnostic command and the
uncached main-CI contract.
On a host with managed checks configured, use `npm run checks:status` to inspect
running and queued jobs and `npm run checks:cancel -- <id>` to cancel your job.
Cancel and wait for termination before removing its worktree. See
`docs/agent-gotchas.md` for scoped test filters, logs, lint prerequisites, and
migration conflicts.
The native (iOS/Android) surfaces have the same shift-left split — a **fast pure-logic lane**
(no simulator/emulator build) before the slow visual build. Both bridge to a configured macOS
host over `ssh` (`export OMNESIS_EPIC_MACOS_HOST=<your-macos-ssh-alias>` — unset is a clear
error; never guessed) and keep the macOS scratch checkout **warm** (a persistent dir + reused
build cache) so repeat runs are fast:
- **iOS logic** — `scripts/ios-logic.sh` runs the SwiftPM `swift test` lane (agent
timeline/turn builders, health normalizer/cursor, schemas, offline buffer, markdown,
timing) natively for macOS — **no simulator, no Xcode project, no device build**. Reach for
it before `scripts/ios-snapshot.sh` (the slow simulator-render visual lane). See
`ios/AGENTS.md → Fast inner loops (native lanes)`.
- **Android logic** — `scripts/android-logic.sh` runs `./gradlew <module>:testDebugUnitTest`
for the pure-JVM logic modules (transport decoders, pairing, reducers/models) — **no
emulator, no Roborazzi render**. Reach for it before `scripts/android-render.sh` (the slow
Roborazzi visual lane). See `android/AGENTS.md → Fast inner loops (native lanes)`.
## Code style
- TypeScript strict mode. No `any` except in test mocks.
- Imports use `.js` extension (ESM).
- Use `@omnesis/core` for shared types — never import across packages by relative path.
- Use the structured logger (`createLogger` from `@omnesis/core`). Never `console.log/warn/error` in source code (only the logger itself calls console).
- Logger components follow a hierarchy: `gateway`, `gateway:http`, `collector:sync`, `provider:google`, `source:gmail`, etc. Create child loggers with `log.child("sub")`.
- Logging style: **concise one-liners using template literals**, not structured-data objects. Good: `log.info(\`Sync complete for ${id}: ${count} docs in ${ms}ms\`)`. Bad: `log.info("Sync complete", { id, count, ms })`.
## The writer is one thread
Every write goes through one worker, so anything slow on it stops every other
write on the gateway. Resolve on a reader and hand the writer finished rows;
bound iterating writer ops in sub-batches with a preempt check between them;
and treat a query whose filter no index can serve as work proportional to the
table rather than to the batch. `docs/conventions.md` § "The writer is one
thread" carries the rules and why each exists.
## Database migrations
Any change that modifies the gateway SQLite schema or makes existing data stale needs a migration. Idempotent (safe to run multiple times), runs on startup before the first sync. **Migrations are append-only, contiguous, and kept permanently** — Omnesis now has multiple live users on differing schema versions, so an install several versions behind must upgrade cleanly by replaying the full sequence. Never delete a migration's slot: when a step's transform becomes obsolete, reduce it to a no-op **tombstone** (a no-op `up` with a description saying why) so the `user_version` chain stays gap-free. A contiguity test in `schema.migration.test.ts` reddens the build if the list develops a hole or its head drifts from `LATEST_SCHEMA_VERSION`. A migration normally runs inside the runner's transaction; the `ownTransaction` flag exists only for SQLite's 12-step rebuild of a table that other tables reference (it needs `PRAGMA foreign_keys` OFF and `legacy_alter_table` ON, both inert or harmful inside a transaction) — such a migration owns its BEGIN/COMMIT, runs a table-scoped `PRAGMA foreign_key_check` before committing, and the runner stamps `user_version` afterwards; migration 131 is the reference.
How versioning, migrations, and cross-version compatibility are encoded — the lockstep product version, the machine-readable compatibility manifest (`packages/core/src/compat.ts`, exposed at `GET /admin/compat` and a subset on `GET /health`), the per-store policy classification, the WS/pairing protocol gates, and the `omnesis update` preflight — is encoded in `packages/core/src/compat.ts` (the compatibility manifest and per-store policies, with doc comments) and `packages/cli/src/update/` (the `omnesis update` preflight). Read those when touching versioning; keep `website/docs/updating.html` accurate on the operator-visible behavior (the update's own pre-upgrade backup, forward-only migrations, the restart order and the rollback).
## Releases & packaging
Read `docs/releasing.md` before touching anything publish-related. The essentials:
- `npm run release -- <plan|version|pr|tag|status|github-release>` is the single entry point; it owns everything from the changeset to the annotated tag (including the iOS and Android version fields) and never pushes. Pushing the tag triggers `.github/workflows/release.yml`, which always creates the GitHub Release with the changelog section and publishes the release images unless the repository variable `OMNESIS_IMAGE_PUBLISH` is `0`; its npm publishing step stays a dry run until `OMNESIS_RELEASE_PUBLISH` is `1`.
- All `@omnesis/*` packages version in **lockstep** (one product version, served on `GET /health`, printed by `omnesis --version`, tagged `vX.Y.Z`).
- Repo manifests stay **src-pointing** (`main`/`exports`/`bin` → `src/*.ts`); `scripts/release/` transforms them to `dist/*.js` at publish-stage time, never in the repo.
- Code that spawns a sibling worker thread or subprocess must use `resolveWorkerEntry` / `resolveSubprocessEntry` from `@omnesis/core` so it works both under tsx (dev) and compiled (published).
## Restarting running services
- **Gateway and collector** — remind the user to restart if changes affect runtime behavior.
- **Portal static files** (`packages/gateway/portal/`) — served from disk on every request, no caching. Edits show on the next browser refresh; no restart needed.
## Shell commands
- Use `rm -f` (not bare `rm`). The sandbox runs `rm` interactively, which appears to succeed but silently skips deletion. Verify with `ls`.
- Node 24 is required. Ensure `node` and `npm` resolve to that toolchain before running repository scripts.
- Environment footguns (sandbox-blocked foreground `sleep`, `pkill -f` self-match, zsh `noclobber` `>!`, Playwright `networkidle` on SSE/WS pages, the worktree symlink-farm) are consolidated in [`docs/agent-gotchas.md`](docs/agent-gotchas.md). To kill throwaway processes by pattern without killing yourself or no-op'ing on a wrong pattern, source `scripts/lib/safe_kill` (drops `$$`/`$PPID`, fail-loud on a zero/self-only match) instead of bare `pkill -f`.
## Source roadmap process
A master GitHub issue tracks **which data sources exist, are planned, or are off-limits**. It carries the status table (Implemented / In progress / Planned / Won't implement) and is the chronological ledger via comments.
Process notes:
- **Viability rule.** A source is viable only if Omnesis can authenticate once and then sync from it automatically in the background. Sources whose only ingestion path is a manual user-triggered file export (Google Takeout, Facebook DYI, Instagram personal export, Google Photos Picker) are **Won't implement**.
- **One master, many sub-issues.** New per-source proposals go in their own issue, then become a sub-issue of the master roadmap issue via the GitHub native parent/child relationship. Don't open a new umbrella issue.
- **Won't-implement sources don't get deep-dive issues** — just a row in the roadmap's "Won't implement" table with a one-liner reason.
- **Bug reports & enhancements for shipped sources are NOT sub-issues of the roadmap** — they're standalone issues. The bottom of the roadmap's body lists them by source family for visibility.
- **Update the roadmap when status changes.** Ship a source → move row from Planned to Implemented. Drop a Planned source → move to Won't implement with a reason.
The user-facing explanation of viability and source contributions lives in `website/docs/building-sources.html` and the source list in `website/docs/sources.html`. Update those when the contract changes; the roadmap carries only the status / ledger. The reviewer's checklist for a new source is the `source-review` skill (`.agents/skills/source-review/SKILL.md`) — when the source contract gains or loses an obligation, update that file in the same PR.
## Synthetic-corpus universes
Synthetic-data fixtures (cast, per-source fixtures, replay-agent scenarios, seed source list) live in self-contained universes under `evals/universes/<name>/`. The demo gateway (`scripts/start-demo-gateway.sh --universe <name>`) and the synthetic E2E harness (`new SyntheticE2EHarness({ gatewayMode: "stable", universe })`) both honor this. Default universe is `default`. See `docs/universes.md` for the manifest schema, authoring workflow, gateway-mode contract, and validator.
## iOS development workflow
iOS-specific rules — XcodeGen, device deploy, the mandatory SwiftUI preview + snapshot self-critique loop, TestFlight — live in `ios/CLAUDE.md` (symlinked as `ios/AGENTS.md`). Read it before touching anything under `ios/`.
On a host without Xcode (e.g. the Linux dev box), `scripts/ios-snapshot.sh` bridges the snapshot self-critique loop to a configured macOS build host: set `OMNESIS_EPIC_MACOS_HOST` (unset → a clear error, never a guessed alias), and it rsyncs your **uncommitted** `ios/` tree to a dedicated scratch checkout on the host (hard-fails if that path is the primary checkout or a `_work` runner clone), runs the snapshot suite over `ssh` with code-signing disabled, and rsyncs the PNGs back to `/tmp/omnesis-snapshots/` for you to `Read`. See `ios/AGENTS.md` → "Bridging the snapshot loop from a non-macOS box".
## Portal development workflow
**Mandatory: screenshot + Read + self-critique for any `portal/` change.** The web portal (`packages/gateway/portal/`) gets the same treatment the iOS guide mandates for SwiftUI — handing the human un-self-checked portal UI is a failure mode. After any change that affects what the portal renders, capture the affected route, `Read` the PNG, and critique your own render (did it render the intended content, not a blank/error/empty-state page?) **before** asking the human to look.
Use `scripts/shot-portal.sh <route> [--wait <selector>] [--click <selector>]`: it boots (or reuses) an isolated **synthetic-data** gateway from the current worktree on a high port, navigates the headless portal to `/portal/<route>`, waits on a **selector** (never `networkidle` — the portal holds SSE/WS; see `docs/agent-gotchas.md`), optionally clicks one, and writes a PNG to `/tmp/omnesis-portal-shot-<route>.png` for you to `Read`. **Never shoot the live gateway** — the script only ever drives its own isolated instance (never port 7600 / `~/.config/omnesis`). It is an **on-demand** capability — there is no golden-screenshot corpus to maintain; you capture, look, and critique each time. Routes are the portal's nav routes (`""` and `agent` are both the Ask landing, plus `search`, `people`, `sources`, `capture`, `privacy`, `watches`, `debug`, and `settings` — the last two being tabbed pages, whose tabs are addressed as `debug/data`, `debug/sql`, `debug/graph`, `debug/metrics`, `debug/background-jobs`, `debug/calendar`, `debug/cognition`, `debug/doctor`, and `settings/config`, `settings/omnesis-md`, `settings/models`, `settings/access`, `settings/access/connect` (the connect-an-agent dialog), `settings/policies`, `settings/devices`). Pass `--keep` to leave the gateway up across several shots.
## Known performance work
The gateway's slow-request tail comes from writer-worker queue contention. Before starting any HTTP-latency / writer-worker / backfill-worker performance work, know its levers: the four incremental compute/upsert splits (`refreshPeopleCounts`, `touchTokenUsage` coalescing, `reconcileUnresolvedLinks`, `runMergePass` candidate-selection) and the structural priority-queue alternative. Bench each change before and after.
## Labelling GitHub issues
The repo's issue-label set is defined in [`docs/issue-labels.md`](docs/issue-labels.md) — read it before filing or triaging an issue. It is deliberately minimal: there is **no `type:` / `area:` / `topic:` taxonomy** (those axes were removed) and **no priority axis**. The labels are the GitHub-native states (`good-first-issue`, `help-wanted`, `question`, `duplicate`, `wontfix`, `invalid`) and a few workflow labels. Don't reintroduce the old axes or mint new label families ad hoc; keep `docs/issue-labels.md` and the GitHub label set in sync when the set changes.
## Anchor GitHub issues in the code
When you open a GitHub issue for a new feature, a bug to fix, a planned refactor, or any other tracked work that has an identifiable home in the code, also drop a short comment at the relevant call site(s) referencing the issue (e.g. `// See #312 — planned: …` or `// Known bug: #287`). This is the durable link between the tracker and the code: future agents reading that file will discover the issue without having to search GitHub, and can pull it up for the full context, history, and discussion.
Keep the comment minimal — one line, the issue number, and a few words of "what this issue is about". The issue itself carries the detail; the comment is just a pointer. Remove the comment when the issue is closed and the work is done (no stale `// TODO #123` markers left behind once #123 ships).
## Other instructions
- Always ensure tests pass after implementing a change.
- Always ask whether test coverage is good and whether new tests are needed.
- When fixing a bug, ask whether the fix could introduce staleness or race conditions.
- Always check `README.md` is up to date.
- Always check this file (`CLAUDE.md`) is up to date — it carries agent-specific rules; keep it slim and accurate.
Cloud relay is a separate explicit opt-in: `OMNESIS_CLOUD_RELAY_ENABLED=1`.
Experimental or synthetic mode alone does not enable cloud connectivity, Google
enrollment, or relay settings. Native apps use the same independent opt-in at
build time so Google sign-in can be offered before pairing with a gateway.