CLAUDE.md@packages/events git:20260926.09a45e2A

Immutable. This exact content never changes and is served at /api/v1/blob/ce37ca2491a9b4a7.

# events

## Purpose

The read side of the durability plane: a typed per-session event bus over
OTP `pg`, rebuildable projections folded from a session's committed
stream, and a standalone full-text search service over a repository's
sessions. Nothing here holds authority — every read model is a fold over
the durable store, and every event is a hint that something changed.
WP-K.

## Key Types

- `events/bus.Bus` — an opaque handle to the node-global `pg` scope.
  Groups are keyed `#(session, topic)`, so per-session isolation needs no
  per-session processes and lookups are local-speed ETS reads.
- `events/bus.{SessionKey, key, unidentified_key, key_to_string}` — the
  session half of that group key (`protocol-change/008`). `key(of: id)`
  names a session by its canonical `core/ids.SessionId`;
  `unidentified_key(name:)` is what a publisher with no id yet passes.
  They render to `id:`- and `name:`-prefixed strings, disjoint by
  construction, so a caller-supplied name can never land on — or
  impersonate — an identified session's group.
- `events/bus.{Topic, Event, Published}` — seven topics: six hint topics
  (`Entries`, `Operations`, `Usage`, `Strands`, `Escalations`, `Commits`)
  with deliberately thin events on them, and `Outputs`, whose `ToolOutput(strand,
  op, step, source_index, call_id, stream, tail, total_bytes)` is the bounded rolling window of a
  running tool call's output (`protocol-change/031`) — display state of the
  same standing as `OpTransition`'s phase label, complete in every event
  so a subscriber replaces rather than appends. `OutputStream` (`Stdout` |
  `Stderr`) mirrors the broker's type because this package may not import
  it. `Outputs` also carries `BlockSummary(subject, text)`
  (`protocol-change/050`): a summarizer's label for a long reasoning block
  or delivered advisor message, where `SummarySubject` is
  `SettledBlock(entry, block)` for a committed block (whose label is also
  in the reserved `summary/` cell) or `LiveStream(strand, op, generation)`
  for reasoning still streaming (stored nowhere). `Published(session, event)` is what a subscriber selects, so a
  subscriber joined to several sessions can tell them apart.
- `events/bus.{all_topics, hint_topics, subscribe_all, subscribe_hints}` —
  `subscribe_all` joins all seven; `subscribe_hints` joins the six hint
  topics and is what a pull-driven subscriber (the projection driver)
  takes, so a chatty build does not wake it once per 32 KiB chunk to find
  nothing new in the store.
- `events/bus.bridge` — the adoption seam: turns any subscription-shaped
  event source into bus publishes without this package importing the
  runtime (the composition layer writes the mapping closure, since it is
  the layer that knows both types).
- `events/projection.Projection(state)` — `initial` plus a pure, total
  `apply(state, Change) -> state`. `Change` is `EntryAppended(entry)` or
  `UsageAppended(row)`.
- `events/projection.Checkpoint(state)` — `load`/`save` of a `#(state,
  high_water, generation)` **triple**: state without its high-water is
  meaningless, and either without the rewrite generation they were folded
  under cannot be trusted after a rewrite. `ephemeral()` persists nothing
  and rebuilds every restart.
- `events/projection.{catch_up, rebuild}` — the convergence primitives;
  `stats_projection()` is the shipped example, folding to
  `storage.SessionStats`.
- `events/projection.{Options, Hints, Message(state), start, poke, read,
  sync}` — the driver actor. `Options.logger` is the injected
  `telemetry/log.Logger` a pull fault with no reply channel is reported
  on (§0.2; `log.discard()` for a driver nobody is watching).
  `Options` carries `store` and `generation` as
  *thunks*, called fresh on every pull rather than captured once, because a
  precise rewrite swaps the store handle and bumps its generation
  underneath a long-lived driver. `Hints` is `FromBus(bus, session)` or
  `NoHints`.
- `events/search.{Search, SearchError, Hit, open, sync, query,
  query_in_session, recent_in_session, register_source, source, remove}` — the FTS5 index and its durable per-session
  cursor, in one repository-wide database that is never inside a session
  file. Sessions are named by `SessionId` outright, with no
  caller-supplied-string form: this index spans a repository, which is
  exactly where a file-derived name collides unnoticed.
  `query_in_session` scopes a query in SQL, so `limit` counts hits in that
  session. `recent_in_session` needs no query: it answers one session's
  highest rowids, which FTS5 assigns in insertion order, so they are its
  newest entries, newest first. `search_source` stores host-registered session paths for exact
  reads; consumers validate the source identity before returning content.
  Removing a session drops its text, cursor and source locator together.
- `events/sql` — **generated by parrot, do not edit** (ADR-004 pilot):
  the named static statements compiled from `src/events/sql/search.sql`;
  regenerate with `scripts/gen-sql.sh`.
- `storage/sqlite_policy` supplies the shared database tuning. Search uses its
  defaults and can override typed fields without duplicating PRAGMA strings.
  Open failures close the acquired connection before returning an error.

## Relationships

The `sqlight` module is supplied by the `sqlight_loom` Hex package, whose
`esqlite_loom` dependency carries the native query-retirement repair.
The module API is unchanged; the upstream packages must not coexist with
these forks because they define the same modules.

- **Depends on**: `core` (ids, entries, messages), `storage` (the
  `Storage` record every projection and the search sync scan through),
  `telemetry` (the injected logger a pull fault surfaces on),
  `sqlight` (the FTS5 database), `parrot` (whose `parrot/dev` the
  generated module imports at runtime), `gleam_erlang` + `gleam_otp`.
  `session` is declared in `gleam.toml` — the spec DAG's `K → A,B,C` — but
  nothing in `src` imports it today.
- **Depended on by**: `client` — `client/serve` wires the two production
  publishers (`gateway.tool_output_observer` publishes `ToolOutput` and
  `client/blocksummary` publishes `BlockSummary` on the `Outputs` topic,
  entered through the idempotent `bus.start`), the
  gateway joins `Outputs` under network delivery and every topic as the
  host fixture, and `client/history` is the search service's one consumer:
  a named holder actor owning one `Search`, synced from the runtime
  writer's post-commit publication and queried by the `history_search`
  tool (issue #28, memory stage M1).
- **FFI**: `events/internal/ffi_pg` — the confined binding over OTP's
  `pg` process-group module (Erlang side: `src/events_ffi.erl`).
  `gleam_erlang` ships no `pg` binding, and re-implementing membership
  tracking as a Gleam actor would lose `pg`'s monitor-based crash cleanup
  and local-speed lookups. It is this package's complete inventory of
  impurity.

## Traffic

- **Actor messages**: `projection.Message(state)` — `Hinted` (a cast: an
  event arrived, go pull), `Read(reply)` and `Synchronize(reply)` (calls),
  sent by `poke`, `read`, and `sync`. The bus itself owns no actor; it is
  `pg` membership plus plain sends.
- **Commits**: none. This package never writes to a session store. The
  search index writes only to its own repository-wide database, where the
  index rows and the advanced cursor commit in one transaction.
- **Registers**: none read or written. Operation state, strand results,
  and escalations are named by events only as *hints* — the register
  remains the truth.
- **Wire**: `pg` sends of a `{loom_event, _}` tuple to the local members
  of a group, unwrapped by `ffi_pg.published_payload`. Delivery is
  best-effort plain sends: a subscriber that is down, slow to join, or on
  another node simply misses events, and that is legal by design.

## Invariants

- **The index open refuses a path SQLite would fail on.** `search.acquire`
  calls `storage/sqlite_policy.refusing_unopenable_path` before
  `sqlight.open`, because a failed open frees a block through the binding's
  double close and corrupts whichever connection is handed it next; `packages/storage/CLAUDE.md`
  has the mechanism. The index is the database most exposed to it, since it is
  a deletable projection beside a session file and repairing it means removing
  it.

- **Events are hints; pulls are truth.** An event never carries durable
  content and is never applied as data — it only prompts a catch-up pull
  from the durable store. Drop any subset of events and every read model
  still converges on the next hint, sync, or restart; the lost-event tests
  assert exactly that. `ToolOutput` keeps the rule by carrying display
  state only: the whole bounded window every time, so a lost event is
  restated by the next, and the durable tool result is the truth once the
  call settles. `BlockSummary` is display state of the same kind: a settled
  label's truth is its reserved cell, which a terminal can read back, and a
  live label is replaced by the next one.
- **A projection is rebuildable and carries no authority.** `apply` is
  pure and total, folding unknown shapes as no-ops, so folding the same
  changes in the same order from `initial` always yields the same state —
  which is what makes rebuild and incremental catch-up provably equal.
- **The frontier rule.** A catch-up that issues more than one scan bounds
  *every* scan by a frontier seq read before the first one. Seqs are
  strictly increasing and rows write-once, so `(after, frontier]` is
  immutable by the time either scan runs. Without it, a commit landing
  between the entry and usage scans advances the high-water past entries
  the entry scan never saw, losing them permanently.
- **A checkpoint persists state, high-water, and generation together.**
  State without its high-water is meaningless, and either without the
  generation cannot be told apart from a checkpoint a rewrite has
  invalidated. Persistence is best-effort: a checkpoint that loses writes
  costs a longer catch-up, never correctness.
- **Indexing is pull-based and crash-safe.** Index rows and the advanced
  cursor commit in one transaction, so a crash mid-batch simply re-runs
  the batch into the same state; a lost hint is caught by the next sweep.
- **A rewrite invalidates by generation, on both read models.** A rewrite
  may renumber seqs, so anything folded or indexed under the old numbering
  is void. `events/search` stores the session store's rewrite generation
  (`storage/sqlite.generation`; memory sessions pass `0`) alongside its
  cursor and, on mismatch, drops that session's index rows and re-indexes
  from zero in the same transaction. `events/projection` compares its
  recorded generation against `Options.generation()` on every pull and, on
  mismatch, restarts the fold from `projection.initial` at seq zero against
  the freshly-read store. A cold start has nothing to compare against, so
  it simply records whatever generation the store reports.
- **Search indexes message text and compaction/branch summaries only.**
  Thinking blocks and tool-call arguments are deliberately not indexed,
  and a custom entry indexes to nothing — its seq still advances the
  cursor, so a re-sync finds nothing to redo.
- **A non-positive `limit` is unbounded, and the caller must clamp.**
  The limit reaches SQL `LIMIT ?`, and SQLite reads `LIMIT -1` as *no
  limit* — the opposite of `storage`'s convention. `tools/history`
  clamps to `[1, 50]` before the query is ever built, which is where a
  model-supplied number is stopped.
- **The typed boundary over `pg` is re-established by pairing.** `pg`
  stores plain pids and delivery is an untyped Erlang send; only
  `ffi_pg.publish` ever sends the `{loom_event, _}` tuple and only with
  the caller's typed payload, and `published_payload` is the matching
  unwrap. That pairing is what makes the unchecked `Dynamic -> payload`
  coercion sound.
- **A pull fault with no reply channel is a `warning`, not a silence and
  not an error.** The checkpointed state stands and the next hint
  retries, so the driver is degraded rather than broken — but a driver
  reading a closed or stale handle would otherwise serve its last good
  state forever with nothing surfacing the read failure. An explicit
  `sync` still returns the fault to a caller that is watching.
- **The hand-written schema DDL must stay identical to `sql/schema.sql`**
  — a test pins the two together. Parrot covers named static queries only;
  DDL and pragmas stay out of codegen (ADR-004).
- **A test that opens a real on-disk index file owns a scratch directory
  it deletes and re-asserts absent before use.** `search_test.gleam`'s
  `fresh_index_path` is the shared helper (issue #119): a bare delete of
  the `.db` file with no check, and no cleanup of `-wal`/`-shm`/
  `-journal` siblings, once let a stale file from an interrupted run
  make a fresh open see the database as locked.

## Deep Docs

- [docs/architecture/durability.md](../../docs/architecture/durability.md)
  — the plane these read models sit on: seqs, write-once rows, the store
  as the only authority.
- [docs/adr/004-parrot-sql-codegen.md](../../docs/adr/004-parrot-sql-codegen.md)
  — why `events/sql` is generated and what stays hand-written.
- [docs/spec-gaps.md](../../docs/spec-gaps.md) — "From WP-K (`events`)":
  the frontier rule, checkpoints persisting state with high-water,
  display-string operation events, the `SessionKey` on the bus, and what
  search indexes.
- [Root CLAUDE.md](../../CLAUDE.md) — repo ground rules and the doc graph.