AGENTS.md git:20260830.6ca8d7eA

Immutable. This exact content is served forever at /api/v1/blob/e7e3629b6ab02b20.

# Releases

Changelog indexer and registry for AI agents and developers. The user-facing CLI lives out-of-tree at [buildinternet/releases-cli](https://github.com/buildinternet/releases-cli); this monorepo is the backend (API worker + D1), the remote MCP server, the web frontend, and the managed-agents harness.

## Stack

- **Runtime:** Bun
- **Language:** TypeScript (strict mode)
- **Database:** Cloudflare D1 + Drizzle ORM. No local SQLite path in this repo — `bun:sqlite` is only used for test fixtures (`tests/db-helper.ts`).
- **API:** Cloudflare Worker with Hono (`workers/api/`)
- **MCP:** Remote MCP server (`workers/mcp/`)
- **AI:** Anthropic SDK (`@anthropic-ai/sdk`); managed agents via `@anthropic-ai/claude-agent-sdk`

## Commands

- Lint + format + type-check: `bun run check` (oxlint with `typeCheck` via `oxlint-tsgolint`, then `oxfmt --check`)
- Lint only: `bun run lint` (`bun run typecheck` is an alias)
- Format: `bun run format:check`
- Tests: `bun test` (not part of `check`)
- Targeted `tsc`: `workers/mcp` still uses `npx tsc --noEmit` in CI (carved-out workspace; excluded from root oxlint). `web` and `tests/` have their own tsconfigs for local runs.
- **Evals (`tests/evals/`) are manual and on-demand only.** They call AI APIs, cost money, and take minutes. `bun run eval:evaluation` is the only in-repo suite (URL evaluation, ~30s). Parsing + discovery evals live in the OSS CLI repo.

## Local development

Running the four `dev:*` services behind portless, fresh-worktree bootstrap, worktree teardown, and local D1 schema parity (`db:reset:local` / `db:pull`) and auth rate-limit / signing-secret notes: [local-development.md](docs/architecture/local-development.md).

## Workspaces and carved-out packages

Root `package.json` declares `workers/api`, `web`, and `packages/*` as workspaces. `workers/discovery/`, `workers/mcp/`, and `workers/webhooks/` are intentionally excluded — wrangler manages their dependencies independently.

The root `test` script runs `workers/api` in its **own `bun test` process**, after the rest: `… bun test tests/ web/ workers/discovery workers/mcp workers/webhooks && bun test workers/api`. This is deliberate isolation, not a style choice: bun's `mock.module()` is process-global, keyed by resolved module, and **not restorable per-specifier** (`mock.restore()` doesn't undo it, and re-mocking with the real impl is impossible — every import of the path resolves to the mock). The discovery scrape-fetch tests `mock.module("@releases/adapters/cloudflare", …)` at module scope, and that stub leaks into `workers/api`'s `render-check` tests, which use the real adapter — an ordering-dependent flake. Splitting `workers/api` into a separate process makes the leak structurally impossible. **Keep discovery (and mcp/webhooks) in the root-cwd multi-dir invocation** — that's where their module mocks resolve against the single root-workspace copy of `@releases/adapters` the source uses. Don't move them to `cd workers/discovery && bun test`: from a different cwd the test's `mock.module` and the source resolve `@releases/adapters/*` inconsistently, so the mock misses and discovery's own tests fail in CI. Isolating `workers/api` (rather than discovery) is what keeps both sides resolving correctly.

**Reach for interface injection before process isolation.** Process isolation is the fix of last resort — it only works when the leak crosses a boundary the `test` script can split on. When a mock stubs a _third-party_ module that sibling suites import, prefer passing a fake through the seam the code already has. `packages/ai/src/aisdk-text-model.test.ts` used to `mock.module("ai", () => ({ generateText }))`, which dropped every other export and made `overview-content.ts`'s `import { Output, parsePartialJson } from "ai"` fail with an unhandled `SyntaxError` between tests, ordering-dependent. Spreading the real module back into the factory does **not** rescue it: mocking `ai` at all forces a second evaluation of the module, so `ai/test`'s `MockLanguageModelV3` and the `generateText` it runs through resolve to different instances and the run hangs. The fix was to delete the module mock and drive the real `generateText` with a `MockLanguageModelV3` — a plain argument, scoped to one test, invisible to every other file. Most SDKs ship such a seam (`ai/test` here); use it.

Shared code is split between published npm packages (`@buildinternet/releases-*`) and private in-tree packages (`packages/`):

- `packages/core/` → published as **`@buildinternet/releases-core`** from this monorepo. Pure, runtime-neutral helpers shared with the OSS CLI: DB schema (source of truth), `categories`, `dates`, `changelog-range`, `changelog-slice`, `overview`, `id`, `slug`, `tokens`, `cli-contracts`, `lookup-coordinate`, `fts` (FTS5 query sanitizer — wraps tokens in phrase quotes so `org/repo` and similar punctuation don't error). Consumed here via `workspace:*`; the OSS CLI pulls the published npm version. Schema changes land here first, then get picked up by the CLI on the next version bump.
- `packages/core-internal/` → imported as **`@releases/core-internal`**. Private, workspace-only. DB-coupled / worker-only helpers the thin client doesn't need: `release-upsert` (drizzle upsert config), `hash` (node crypto), webhook surface (`webhook-sign`, `webhook-resilience`, `webhook-url-safety`, `webhook-delivery`, `webhook-alert-format`, plus `release-event` wire types), `schema-coverage` (the `release_coverage` table, part of the drizzle composite schema).
- `packages/api-types/` → published as **`@buildinternet/releases-api-types`** from this monorepo. Wire protocol — request/response shapes served by the API worker, consumed by the MCP worker, web frontend, and the OSS CLI. Consumed here via `workspace:*`; the OSS CLI pulls the published npm version. Wire changes land here first; the CLI bumps its pin when adopting new shapes. Additive by default — renames/removals go through a one-minor-version deprecation alias before removal.
- `packages/adapters/` — adapter primitives (`types`, `source-meta`, `content-hash`), the `github`, `cloudflare`, `crawl`, and `feed` adapters, plus the shared scrape/agent fetch orchestration (`scrape-fetch` / `extract-deps-worker` / `deterministic-update`, persisting via an injected API fetcher — used by the API worker's update workflow and discovery's onboarding tool, #1946). All pure / worker-safe.
- `packages/ai/` → imported as **`@releases/ai-internal`**. `evaluate` (URL recommendation + `buildMetadataFromEvaluation`), `playbook` (deterministic markdown generation), `providers` (provider-detection table), `release-content` (Haiku 4.5 summarization for `title_generated` / `title_short` / `summary` — shared by `scripts/generate-release-content.ts` and the ingest-time hook), `marketing-classifier` (Haiku 4.5 binary verdict on whether a feed item is real product news vs. marketing — used by `fetchOne` when `metadata.marketingFilter` is set). Worker-safe; caller passes the Anthropic client.
- `packages/rendering/` → imported as **`@releases/rendering/*`**. Atom feed helpers, markdown/JSON formatters, and media URL helpers.
- `packages/search/` → imported as **`@releases/search/*`**. Embedding providers/cache, Vectorize hybrid search, and release/entity/changelog embedding pipelines.
- `packages/lib/` — slim private utilities (`config`, `errors`, `source-edit`, Anthropic client/error helpers, managed-agent rate limits, `anthropic-pricing` for list-price cost estimates on managed-agent sessions, `spend-cap` daily-spend KV gate, `session-error-classify`). `logger` is published as `@buildinternet/releases-lib/logger`.

## Managed-agents harness (`managed-agents/`)

The `managed-agents/` directory holds both the deployed agent definitions (`*.agent.yaml`, `*.environment.yaml`) and the harness code that drives them, under `managed-agents/src/`:

- `managed-agents/src/agent/` — the harness itself (`managed-discovery.ts`) plus shared discovery types and the prompt builder in `discovery.ts`. The legacy sandbox-engine `runDiscovery` has been removed; the discovery worker (`workers/discovery/`) is the only production entrypoint.
- `managed-agents/src/shared/` — prompts, typed tools (`agent-tools.ts`), and grader rubrics (`rubrics/*.md`) shared by the harness, the discovery worker, and the eval suite. Imported by workers as `@releases/shared/*` (a bundler/tsconfig alias, not a workspace package).

The `release_coverage` schema lives with the rest of the DB-coupled internals in `packages/core-internal/src/schema-coverage.ts` (imported as `@releases/core-internal/schema-coverage`), not at the repo root.

## Conventions

> Keep entries to **one line: the rule + a pointer to the doc that owns the detail.** When a feature needs a paragraph, that paragraph belongs in `docs/architecture/`, not here. This section has bloated twice from append-on-ship; resist it.

- **This repo is public — keep PII out of committed content.** No absolute home-dir paths (`/Users/<name>/…` — write `~/…` or repo-relative), no personal email addresses (use `@example.com` in fixtures/docs), no customer data or real tokens, in any committed file — including generated plans and specs under `docs/plans/` and `docs/superpowers/`.
- Logging splits by runtime: **worker code** (`workers/*`) MUST log via `logEvent()` from `@releases/lib/log-event` (worker-safe structured JSON); **CLI + runtime-neutral packages** use `@buildinternet/releases-lib/logger` (stderr + `~/.releases/logs/`). Never import the `fs`-backed `@buildinternet/releases-lib/logger` into a worker. Payload conventions, severity, and `Error` unwrapping: [logging.md](docs/architecture/logging.md).
- Source types (fetch adapters): `github`, `scrape`, `feed`, `agent`, `appstore`. Adapter behavior + `appstore` materialization: [ingest.md](docs/architecture/ingest.md).
- **Ingest pipeline** (fetch → parse → insert) — dedup (`UNIQUE(source_id,url)` + `RELEASE_URL_UPSERT`), smart-fetch backoff, URL exclusion (`ignored_urls` org-scoped / `blocked_urls` global, via `isUrlExcluded()`), release suppression (`suppressed=1`), and the ingest-time Haiku 4.5 passes (content summarization, the per-source marketing classifier via `metadata.marketingFilter`, feed-content enrichment via `FEED_ENRICH_ENABLED`): [ingest.md](docs/architecture/ingest.md). Cron/Workflow orchestration: [remote-mode.md](docs/architecture/remote-mode.md).
- **Release importance (1–5, AI-scored, `releases.importance`):** scored inside the same ingest-time `release-content` summarize call, fail-open to `null`; filterable via `?minImportance=` on `GET /v1/releases/latest` (forces a cache bypass); web shows it as a flame glyph at 4-5 only. See [ingest.md → Content summarization](docs/architecture/ingest.md).
- **`source.url` is for humans; fetch routing lives in metadata.** `source.url` is the canonical human-readable URL; machine fetch endpoints (RSS via `metadata.feedUrl`, GitHub-CHANGELOG override via `metadata.githubUrl`, …) live in metadata. Test: would a human ever want to land on this URL? If no, it's metadata. See [remote-mode.md → Display URL vs. fetch routing](docs/architecture/remote-mode.md#display-url-vs-fetch-routing).
- Crawl mode uses Cloudflare's `/crawl` endpoint for multi-page changelogs, stored in `source.metadata.crawlEnabled`. See `packages/adapters/src/crawl.ts`.
- Firecrawl monitoring: external fetch backend for `scrape` sources behind an anti-bot challenge our Browser Rendering can't clear; toggled per source via `source.metadata.firecrawl` (not a new `type`), excluded from the poll-fetch cron, prod-only secrets. **Gotcha: the webhook carries a hunkless whole-document diff (no `@@` headers) — parse only via `addedContentFromDiff`.** See [firecrawl-monitoring.md](docs/architecture/firecrawl-monitoring.md).
- **Full-history backfill** for windowed scrape sources: `POST /v1/workflows/backfill-source { sourceId, markdown?, maxWindows?, dryRun? }` loops extraction over every window and upserts idempotently; deep Firecrawl path routes to durable `BackfillSourceWorkflow` (R2 snapshot + per-window steps, resumable) behind `BACKFILL_WORKFLOW_ENABLED` (default off); returns `202 { instanceId, statusUrl }` async. See [firecrawl-monitoring.md](docs/architecture/firecrawl-monitoring.md).
- **Source staleness signal + render dry-run (#1528):** daily scans on the `0 4 * * *` tick flag overdue first-party (`cron/source-staleness.ts`, `source-staleness`) and Firecrawl (`cron/firecrawl-staleness.ts`, `firecrawl-staleness`) sources; when any are overdue, `sendStalenessDigest` emails operators via `SEND_EMAIL` (logs-only when the rollup is empty). Render dry-run: `POST /v1/sources/:id/fetch?dryRun=true` on a client-rendered scrape source (`crawlEnabled`/`renderRequired`) renders the index once and returns `{ renderCheck, candidateCount, sampleUrls }` with no extraction/MA loop (CLI: `releases source fetch <src> --dry-run`). No feature flag. See [firecrawl-monitoring.md](docs/architecture/firecrawl-monitoring.md).
- **Transactional email** — every message renders through one shell (`renderEmail` from `@releases/rendering/email-shell`): lane + severity rule, inline-styled table HTML plus a matched plain-text part, a copyable URL under every button, a footer that always states why it arrived, markdown-rendered release content, and Gmail Go-To/One-Click annotations (dormant until the domain is registered with Google). See [emails.md](docs/architecture/emails.md).
- **Admin test emails:** `/admin/emails` (admin session) previews every outbound template via fabricated `[test]` sends (`GET/POST /v1/admin/emails/{samples,test}`); live follow digests stay on `POST /v1/admin/digest/test`. See [web.md → Admin hub](docs/architecture/web.md).
- **Raw capture + re-extract:** the steady-state scrape path captures the scraped markdown to `released-raw` behind `raw-snapshot-capture-enabled` (default off; discovery worker POSTs to `POST /v1/orgs/:orgSlug/sources/:sourceSlug/raw-snapshot`, scrape-only, #1283); `POST /v1/workflows/reextract-source { sourceId, snapshotId?, maxWindows?, dryRun? }` re-runs extraction from a stored snapshot with no live scrape, reusing the backfill machinery (#1284). See [firecrawl-monitoring.md](docs/architecture/firecrawl-monitoring.md).
- **In-place release re-fetch:** `POST /v1/workflows/refetch-release { releaseId, url?, dryRun?, force? }` re-fetches ONE release's live page and updates the row in place — same `rel_` id, replaced title/content/publishedAt (media only on extractor hit), AI fields nulled for regen; a stored `#fragment` URL requires an explicit same-host `url`, which rewrites the stored URL to the canonical permalink; placeholder extraction content is rejected and a >50% shrink needs `force: true` (#2077). See [firecrawl-monitoring.md](docs/architecture/firecrawl-monitoring.md).
- **Local ingest** (local Claude Code, no remote dispatch): the `local-ingest` skill (`.claude/skills/local-ingest/`) has the agent fetch + extract releases itself and write via the idempotent `/batch` upsert — skipping the update workflow's server-side extraction and its inference bill. Mandatory `robots.txt`/`Content-Signal` opt-out preflight (`preflight.ts`) refuses `ai-input=no` (`ai-train=no` alone proceeds; `conductor.build` is refused via its `ai-input=no`). See [local-ingest.md](docs/architecture/local-ingest.md).
- **Local backfill workflow** (local Claude Code, no remote dispatch): the `backfill-source` / `backfill-sweep` dynamic Workflows (`.claude/workflows/`) wrap the `local-ingest` primitives in a deterministic harness — fail-closed preflight, explicit window cap, budget-gated extract waves, known-URL dedup, `/batch` upsert by typed `src_` id — to backfill a source's history locally without the remote extraction bill. Front-door: the `backfilling-sources` skill; dry-run is the default. See [local-ingest.md](docs/architecture/local-ingest.md).
- **Opt-out / safety gates fail closed.** On an ambiguous or unparseable response (e.g. an HTML anti-bot challenge), a gate returns the safe verdict (`unknown`/refuse, never `proceed`); when merging multi-value directives the strictest reading wins (a `no` is never overwritten by a later `yes`). Reference gate: `local-ingest/preflight.ts` (`0` proceed / `1` refuse / `2` unknown). See [local-ingest.md → Preflight](docs/architecture/local-ingest.md).
- **Prod D1 local backups:** `bun run db:backup` dumps every real table (auth included, FTS excluded — D1 export can't handle FTS5 virtual tables) to `~/Code/.backups/releases/` with retention rotation; dumps hold real user data — never commit/upload them. See [local-development.md → Prod D1 backups](docs/architecture/local-development.md).
- `daysAgoIso()` from `@buildinternet/releases-core/dates` for date cutoffs.
- **D1's hard limit is 100 bound parameters per prepared statement.** Batch INSERTs chunk at `floor(100 / binds_per_row)` (for `releases`, 13 binds/row → 7 rows/statement); `inArray(...)` lookups chunk at 90 IDs. Raising without re-checking bind count surfaces as a 500 on `/releases/batch`. Capability constants: `@buildinternet/releases-core/d1-limits`.
- **Local wrangler D1 one-shots are wall-clock bounded** via `node scripts/run-timed.mjs <sec> -- …` (kills the whole process group on deadline) so a timed-out agent can't orphan a multi-GB miniflare boot; `db:migrate:local`/`db:reset:local`/`db:query` already route through it, long-lived `wrangler dev`/`preview:*` deliberately do not, and manual local D1 calls should be wrapped the same way. See [local-development.md → Local D1 schema parity](docs/architecture/local-development.md).
- **Entity IDs are single typed nanoids** (`rel_`/`src_`/`org_`/…): the same string is PK and public ID. No dual UUID/`public_id` prep. New SQL goes through `workers/api/src/queries/*`; no new `releases_fts MATCH` sites. Seams + capability map: [storage-portability.md](docs/architecture/storage-portability.md).
- Workflows-based ingest: every per-source fetch runs as a `POLL_AND_FETCH_WORKFLOW` instance, `create()`d from that source's `SourceActor` DO alarm (#1776/#1819 — the actor is the sole fetch driver; per-source tier intervals normal=4h / low=24h + smart backoff pace the alarm). The hourly poll cron is a re-seed heartbeat only — no fan-out, no inline fetch fallback. See [remote-mode.md](docs/architecture/remote-mode.md).
- **OrgActor scrape/agent drain (#1777/#1946):** `SourceActor` self-flags stranded scrape/agent sources and arms a per-org `OrgActor` DO that dispatches one deterministic update run per org via the shared `startDeterministicUpdate` gate (kill switch + spend cap + #1815 scrape lock; no budget/lock logic in the actor). Update runs execute as the API worker's `DeterministicUpdateWorkflow` — `workers/discovery` serves onboarding only (its `/update` route is retired). See [remote-mode.md](docs/architecture/remote-mode.md).
- **Feature flags via Cloudflare Flagship (Tier 1).** Boolean kill switches / rollout gates evaluate at runtime through the `FLAGS` binding; registry is `@releases/lib/flags` (`flag(binding, varValue, def)`); order is Flagship → wrangler var → default, failing open to the var. Adding a flag: add a `FLAGS` entry, convert the read to `await flag(...)`, and create the same kebab-case key in BOTH Flagship apps (`releases-platform{,-staging}`). Numeric tunables and secrets are intentionally NOT in Flagship. See [feature-flags.md](docs/architecture/feature-flags.md).
- **Be judicious with feature flags — do NOT add one by default.** Every flag is permanent maintenance surface: a registry entry, a runtime branch on every read, a key that must be created and kept in sync across BOTH Flagship apps, and a dead code path to eventually retire. A flag per feature is how a flag registry rots into dozens nobody dares delete. **Default to shipping a feature enabled, with no flag.** Reach for one ONLY when there's a concrete reason runtime toggling earns its keep: a genuine kill switch for something risky/expensive/external-facing, a staged rollout you actually intend to ramp, or an operational lever you'd pull without a redeploy. If you can't name which of those applies, you don't need a flag. When unsure, ship it on and add the flag later if a real need appears — adding is cheap, a sprawl of stale flags is not. Prefer one well-scoped kill switch over several fine-grained per-feature toggles.
- Extract tier: `extractFromBody()` branches on body token count — ≤50K one-shot `/v1/messages`, >50K a multi-round tool-use loop via AI SDK (`extract-with-tools-aisdk.ts`; legacy Anthropic SDK loop in `extract-with-tools.ts` only when no `aiSdkModel`), gated behind `EXTRACT_TOOLLOOP_ENABLED` (per-source `metadata.extractStrategy = "toolloop"`); falls back to one-shot on any error. See [extract.md](docs/architecture/extract.md).
- **Classification taxonomy** — source `kind` enum, products, release type (`feature`/`rollup`), tags, categories, collections, and how these axes differ (`kind` vs `type` vs `category`): [taxonomy.md](docs/architecture/taxonomy.md).
- **REST route surface** — route-naming buckets (#494), org-scoped routes + dual-registration + `bare_slug_rejected` (#690/#698), the `/v1/lookups` family (coordinate POST + on-demand materialization, by-domain, slug resolvers), org catalog, entity resolution (IDs over slugs), pagination shape, and the OpenAPI coverage gate (#894): [routing.md](docs/architecture/routing.md).
- **Friendly release URLs** — `/release/rel_<id>-<slug>`: the `rel_` ID (positional, `rel_` + 21 chars — nanoid may contain `-`/`_`) is the only routing key; the slug derives from `title_short` at request time (no stored column, no backfill) and stale/bare forms redirect to canonical with a 308 (`permanentRedirect`). Helpers in `@buildinternet/releases-core/release-slug`. See [routing.md](docs/architecture/routing.md).
- **Error responses** — every non-2xx response is the nested envelope `{ error: { code, type, message, details? } }`; `throw` a `ReleasesError` subclass and let `respondError` serialize it (never hand-roll `c.json({ error })`). Taxonomy source of truth is `packages/core/src/errors.ts`; the three-layer split, producers/consumers, and how to add a code: [errors.md](docs/architecture/errors.md).
- Search-query log: `/v1/search` and the MCP `search` tool write each query (≤200 chars) + mode/counts/duration to `search_queries` (web carries `X-Releases-Surface: web`). Read via `GET /v1/admin/search-queries{,/top}`. Kill switch `SEARCH_QUERY_LOG_DISABLED`. Distinct from `telemetry_events`, which carries only command names and stays PII-clean for the OSS CLI contract.
- Release coverage: multiple releases can cover one launch (marketing post + changelog + app note); canonical + coverage items tracked in `release_coverage`, read paths hide coverage-side rows by default. See [coverage.md](docs/architecture/coverage.md).
- Org overviews: AI-generated `knowledge_pages` (scope `org`) summarize recent activity; display staleness warning `OVERVIEW_STALE_DAYS = 30` from `@buildinternet/releases-core/overview`. Automated regen runs on a daily cron with a per-org cadence (7d default, 2d velocity fast tier, `overview_cadence_days` manual override; #1895). See [web.md → Org overviews](docs/architecture/web.md).
- Collection daily summaries: one `collection_daily_summaries` row per (collection, ET day) — title + one-line summary + bullet takeaways generated nightly over closed ET days via the **shared summarization lane** (reuses `SUMMARIZE_MODEL` + its Haiku fail-open, distinct only by `generationName`; no per-feature model var), gated per-collection by `collections.daily_summary_enabled`; read via `GET /v1/collections/:slug/daily-summaries`, rendered as timeline date headers. See [web.md → Daily summaries](docs/architecture/web.md).
- GitHub CHANGELOG files are fetched alongside tagged releases, stored in `source_changelog_files` (refresh piggybacks on every GitHub fetch); web surfaces them via `GET /v1/sources/:slug/changelog`. See [web.md](docs/architecture/web.md).
- **ISR revalidation is ingest-driven, not clock-driven.** Ingest pings web's `POST /api/revalidate` (`notifyWebRevalidate`); the pages' `revalidate = 86400` is only a backstop. A fetch revalidate anywhere in the ROOT LAYOUT's import graph caps EVERY route in the app — guarded by `web/src/lib/isr.test.ts`. See [web.md → ISR revalidation](docs/architecture/web.md).
- **API-worker → web internal calls share ONE channel credential** (`RELEASES_SERVICE_KEY` / `WEB_SERVICE_KEY`, verified by `verifyServiceKey`), mirroring `RELEASES_PROXY_KEY` inbound. Reuse it for the next internal endpoint — do NOT mint a per-feature secret; it is deliberately not the root API key. See [web.md → ISR revalidation](docs/architecture/web.md).
- Media handling: at ingest, `normalizeMediaUrl()` (`packages/rendering/src/media-url.ts`) strips Next.js/Vercel image-optimizer wrappers so the underlying CDN URL is stored. Ingest-time R2 mirroring runs whenever the `MEDIA` bucket binding is bound (always in prod); an unbound bucket stores third-party URLs verbatim. See [web.md → Media handling](docs/architecture/web.md).
- **Inline hosted-video cards (#1549):** the cron poll-fetch media pre-pass detects Wistia/Loom/Vimeo/YouTube links in a new release's body (`detectInlineVideos`, `packages/rendering/src/video-embed.ts`), resolves a poster via oEmbed, and appends a `{ type:"video", url:<poster>, alt, linkUrl:<watchUrl> }` `media[]` item that rides the existing `processMediaForR2` mirror; web renders a read-only play-thumbnail card (`InlineVideoCard`) linking out. Special-cased to video (first inline-body asset promoted to mirrored media); fail-open, no flag; iframe embed deferred. See [web.md → Inline hosted-video cards](docs/architecture/web.md).
- **Manual release media edit:** `PATCH /v1/releases/:id { media: [...] }` REPLACES a release's stored `media[]` wholesale (curator fix without re-sync); not-yet-mirrored items (no `r2Key`, not on `MEDIA_ORIGIN`) run through the ingest `processMediaForR2` mirror, gated on `env.MEDIA != null`. Cron re-fetch never clobbers this (`onConflictDoNothing` / `RELEASE_URL_UPSERT` only backfills stored-empty media). See [web.md → Media handling].
- **Inline-video retrofit (#1549 backfill):** `POST /v1/workflows/backfill-video { releaseId?|sourceId?|all?, limit?, dryRun? }` (admin, idempotent, no flag) re-runs `detectInlineVideos` over a stored release body, mirrors the poster, and APPENDS the `type:"video"` item to the existing `media[]` (dedup by `linkUrl`) — built on `runVideoBackfill` in `lib/media-backfill.ts`, sibling of `backfill-media`. See [web.md → Inline hosted-video cards](docs/architecture/web.md).
- Org avatars: stored at `orgs/{slug}.{ext}` in `released-media`, served from `https://media.releases.sh/orgs/{slug}.{ext}`; pointer on `organizations.avatar_url`, writable via `PATCH /v1/orgs/:slug { avatarUrl }`. New orgs land `null` with the OG/web fallback to `github.com/{handle}.png` (#982 open). Reuse the `orgs/{slug}.{ext}` key for any new avatar-write path. See [web.md](docs/architecture/web.md).
- Scoped API tokens: opaque `relk_<lookupId>_<secret>` Bearer tokens (`api_tokens` table), scope ladder `read ⊂ write ⊂ admin`; static `RELEASES_API_KEY` is implicit root; kill switch `API_TOKENS_DISABLED`. MCP gates the AI tools + the on-demand `/v1/lookups` fallback on `write` and forwards the caller's own token. See [remote-mode.md → Auth model](docs/architecture/remote-mode.md) and [mcp.md → scope enforcement](docs/architecture/mcp.md). User-owned **API keys** use the Better Auth `@better-auth/api-key` plugin (prefix `relu_`, `apikey` table, per-key rate-limit + metering), **capped read-only** (`USER_API_KEY_MAX_SCOPE = "read"` — write/admin refused at the mint route and clamped at auth time, so a user key can never satisfy `write`), gated by `user-api-keys-enabled`; the `relk_` lane stays for machine principals. Browser `releases login` mints one of these read-only `relu_` keys with no copy-paste via device authorization (RFC 8628; `deviceAuthorization()` + `bearer()`, always registered) — **`verificationUri` must be an absolute web-origin URL, not a relative `/device`** (see remote-mode.md → Auth model).
- **OAuth role provisioning**: a user's OAuth scope ceiling is the Better Auth `user.role` column (`user`→read, `curator`→read+write, `admin`→read+write+admin; NULL → read-only, fail-closed); manage it via the root-key-gated `PATCH /v1/admin/users/role` route / `releases admin user set-role` (audited `role-changed` logEvent), NOT an env var (the old `OAUTH_ADMIN_USER_IDS` bootstrap is gone, #1484). First admin seeded by a one-time D1 write. See [remote-mode.md → Role provisioning](docs/architecture/remote-mode.md).
- **OAuth client provisioning**: register/manage "Sign in with Releases" OAuth clients via the root-key-gated `admin/oauth` route family (`POST/GET/PATCH/DELETE /v1/admin/oauth/clients[...]`, `rotate-secret`); `reloc_` secrets shown once, `trusted` ⇒ `skip_consent`, `tokenEndpointAuthMethod:"none"` ⇒ public/PKCE client. RFC 7591 dynamic client registration is **ON** (`allowDynamicClientRegistration: true`) so agent-run MCP clients self-register at the public `/oauth2/register` endpoint (untrusted ⇒ consent-required, PKCE-required, role-clamped tokens; rate-limited 5/min/IP in prod); the admin route stays for first-party/`trusted` clients, and the plugin's session self-service write endpoints remain admin-only (#1482). See [remote-mode.md → OAuth client provisioning](docs/architecture/remote-mode.md).
- **OAuth resource-server JWT lane**: the REST API + MCP workers verify the AS's JWT access tokens with `jose` + the JWKS endpoint (NEVER by importing better-auth into `workers/mcp` — zod-pin) via the shared `@releases/lib/oauth-jwt` (`isJwtShaped`/`verifyOAuthJwt`); checks sig + `iss` + `aud` + `exp`, maps the `scope` claim (already role-clamped at issuance) onto the `read ⊂ write ⊂ admin` ladder. Additive + fail-consistent: a verify failure is treated like an invalid `relk_` (REST 401 on a write/admin route, public read stays open; MCP → anonymous read, never gates the unauthenticated path). MCP audience via `OAUTH_JWT_{ISSUER,AUDIENCE}` wrangler vars (#1483). Generic MCP clients (MCPJam-class): DCR extra `grant_types` are intersected, DCR `application_type` is defaulted/coerced to `native` for loopback/private-use-scheme redirect clients (opencode/Cursor-class, forward-compat for a future Better Auth 1.7 upgrade), kitchen-sink `scope=` is downscoped, origin and `/mcp` are both valid RFC 8707 resources, and same-environment MCP `aud` is accepted on REST JWT gates so follows tools can forward the Bearer. See [mcp-cimd-interop.md](docs/architecture/mcp-cimd-interop.md) and [remote-mode.md → Auth model](docs/architecture/remote-mode.md).
- **Rate-limit tiers**: three rungs on CF native limiters — anonymous-IP 120 / account-userId 300 / machine-token 600 — selected by the shared `@releases/lib/rate-limit-tiers`; account-tier `relu_` verify is `CREDENTIAL_CACHE`-backed; consumption is the `rate-limit`/`decision` Axiom event. These read-surface tier counters never move to KV/D1 (the separate `/api/auth/*` brute-force limiter is the one limiter that's KV-backed — #1728, see the auth note above). See [remote-mode.md](docs/architecture/remote-mode.md).
- **Entity notices**: a small curator-set note on an org/product/source, stored under `metadata.notice` (`{ message, linkText?, coordinate?, href? }`); set via the entity PATCH route, surfaced as a typed `notice` field + a web banner + an MCP pointer. See [web.md → Entity notices](docs/architecture/web.md#entity-notices).
- **Owner-declared manifest (`releases.json` v2)**: declaration manifest at domain `/.well-known/releases.json` (org + optional `products[]` + top-level `releases[]` locators) and repo-root `releases.json` (product binding + repo `releases[]`); fill-if-empty reconciliation + cost-tiered source materialization; fail-closed, never clobbers curator/editorial fields. See [well-known-config.md](docs/architecture/well-known-config.md).
- **Self-serve listing lane (#1947 phase 2)**: anonymous `POST /v1/listing/{validate,activate}` — live manifest validation + instant-stub activation with a `tracking_requested_at` demand signal; invalid-manifest activation is a 400 `ValidationError` (no 422 in the taxonomy); rate limited per-IP (10/min) and per-domain on activate (3/min), both CF-native; routes live in a new `publicWriteRoutes` namespace bucket (integrity enforced by handler-level guards, not auth middleware); kill switch `listing-self-serve-enabled`. Curator surface is `GET /v1/orgs?trackingRequested=1` (admin-gated inside the handler), not a separate `/admin/*` route. See [well-known-config.md → Self-serve listing](docs/architecture/well-known-config.md).
- **Ownership claims (#1947 phase 3a)**: signed-in-only `POST /v1/listing/claim{,/verify}` + `GET /v1/listing/claims` proves domain control via a `relv_` token, checked either as a `.well-known/releases-verify.txt` file or a `_releases-challenge.<domain>` DNS TXT record (either passes, fail-closed on any ambiguous response); reuses `listing-self-serve-enabled`; unlocks self-serve Tier-1 promotion (follow-up PR), not org edit rights. See [well-known-config.md → Ownership claims](docs/architecture/well-known-config.md).
- **Mobile-app discovery (#1907)**: separate from the manifest — discovers a domain's undeclared native apps from `/.well-known/apple-app-site-association` + `/.well-known/assetlinks.json`. iOS bundle IDs resolve via iTunes `bundleId=` lookup and land as **paused, hidden** `appstore` candidates (curator unpauses); Android package names become an internal `org.metadata.discoveredApps` hint (no `playstore` type). Own 06:30 UTC cron + `POST /v1/orgs/:slug/discover-apps`; gated by `well-known-materialization-enabled`. Requires **hidden-source containment** (public read/search/embed paths filter `is_hidden=0` via `sources_visible`). See [well-known-config.md → Mobile-app discovery](docs/architecture/well-known-config.md).
- **User follows + feed + webhooks**: signed-in users follow orgs/products (`user_follows`); `/v1/me/follows` + `/v1/me/feed` (org follow = its products too). Self-serve **`/v1/me/webhooks`** (org-scoped max 10 + one `scope: "follows"` URL) fans out real-time `release.created` using the follow graph; account UI on `/account/notifications`, CLI `releases webhook …`, API docs at `/docs/api/webhooks`. Gate: Better Auth session OR Bearer **user** principal (`relu_` / OAuth JWT) — same as follows, NOT relk\_/root/anonymous. MCP has follows tools only; webhook MCP deferred ([#1678](https://github.com/buildinternet/releases/issues/1678)). See [routing.md](docs/architecture/routing.md), [web.md](docs/architecture/web.md), [mcp.md](docs/architecture/mcp.md), [docs/webhooks.md](docs/webhooks.md).
- **Stripe customer registration** (`@better-auth/stripe`, billing groundwork — customer management only, no subscriptions yet): `createCustomerOnSignUp` creates a Stripe Customer per user, linked via `user.stripeCustomerId`. GATED on BOTH `STRIPE_SECRET_KEY` + `STRIPE_WEBHOOK_SECRET` resolving (`buildStripePlugin`, same inert-until-provisioned seam as dash/sentinel/social) — secrets bound via Secrets Store (prod bound; staging deliberately unbound until its sandbox Stripe secrets exist — the gate keeps it inert). Worker-compat: Stripe client uses `Stripe.createFetchHttpClient()`. Webhook served at `/api/auth/stripe/webhook` (existing `/api/auth/*` catch-all).
- **Passkeys** (`@better-auth/passkey`, WebAuthn/FIDO2): always-on (no flag, like magic link) — adds the `passkey` table and `signIn.passkey` / `passkey.{add,list,update,delete}` endpoints. **`rpID`/`origin` are pinned to the WEB origin via `derivePasskeyRp(env)` (from `WEB_BASE_URL`), NOT the plugin's `baseURL`-derived default** — the ceremony runs in the browser on `releases.sh`, not the API worker on `api.releases.sh`, so the default would fail every origin/rpID check. The challenge cookie rides the `.releases.sh` cross-subdomain cookie domain. Web: `passkeyClient()` in `auth-client.ts`, a manage panel on `/account` (`passkeys-panel.tsx`), and a passkey button + conditional-UI autofill on the sign-in form. The auth UI itself is **no longer behind `NEXT_PUBLIC_AUTH_UI_ENABLED`** (retired) — it follows `AUTH_CONFIGURED` (= `NEXT_PUBLIC_BETTER_AUTH_URL` set). See [remote-mode.md → Auth model](docs/architecture/remote-mode.md).
- **Account settings — email change + social connections:** `/account` also surfaces an `email-panel.tsx` (Better Auth `changeEmail`, server-enabled via `user.changeEmail` — a confirmation link goes to the user's CURRENT address, `updateEmailWithoutVerification` left off) and a `social-connections-panel.tsx` (`listAccounts` / `linkSocial` / `unlinkAccount`; lists/links/unlinks the providers in `NEXT_PUBLIC_AUTH_SOCIAL_PROVIDERS`, self-renders nothing when none are configured, and relies on Better Auth refusing to unlink the last sign-in method). Provider chrome (`SOCIAL_PROVIDERS`, `PROVIDER_META`) is shared with the sign-in form via `web/src/lib/social-providers.tsx`.
- **Workspaces (Better Auth org plugin)**: user-tenancy "Workspaces" — every user gets a personal one (lazy `session.create.before` provisioning that backfills existing users; deterministic `ws-<userId>` slug for race-dedupe), DISTINCT from the registry `organizations` (indexed vendors). `auth*`-prefixed tables (`organization`/`member`/`invitation`) in `schema-auth.ts`; always-on, no flag; minimal `/account/workspaces` UI. Inert org-keyed Stripe `subscription` seam (`plans: []`, `referenceId` = workspace id, `authorizeReference` owner/admin gate). `member.role` ≠ `user.role`. See [workspaces.md](docs/architecture/workspaces.md).

## Further reading

Deep dives live in `docs/architecture/`. A reader's guide with task-based entry points is at [docs/README.md](docs/README.md).

- [local-development.md](docs/architecture/local-development.md) — portless dev URLs, worktree bootstrap/teardown, local D1 schema parity, auth rate limiting and the local signing secret.
- [staging.md](docs/architecture/staging.md) — the staging read-surface: hosts, per-environment managed agents, disabled crons, access gate, deploy and prod-sync commands.
- [deploy-coupling.md](docs/architecture/deploy-coupling.md) — account-scoped wrangler bindings and open-core boundary for forks/self-hosters.
- [remote-mode.md](docs/architecture/remote-mode.md) — D1, auth model (scoped API tokens), rate limiting, migrations, sessions, cron polling + retier, workflows-based ingest, discovery guardrails.
- [storage-portability.md](docs/architecture/storage-portability.md) — SQLite/D1 seams, `createDb`, entity-ID invariant (single typed nanoid), FTS ownership, backend capability map, and what an optional future Postgres backend would cost (aspirational, not in progress).
- [ingest.md](docs/architecture/ingest.md) — ingest pipeline: source-type adapters, dedup + D1 batching, smart-fetch backoff, URL exclusion / suppression, and the ingest-time AI passes (summarization, marketing classifier, feed enrichment).
- [emails.md](docs/architecture/emails.md) — transactional email: the shared shell, the three lanes, the email-client constraints it encodes, Gmail annotations, and the admin preview catalog.
- [logging.md](docs/architecture/logging.md) — per-runtime logging: `logEvent()` for workers vs. the `fs`-backed logger for CLI/neutral packages, payload conventions, severity, `Error` unwrapping.
- [routing.md](docs/architecture/routing.md) — REST route surface: naming buckets, org-scoped routes + dual-registration, the `/v1/lookups` resolver family + on-demand GitHub materialization, org catalog, entity resolution, pagination shape, OpenAPI coverage gate.
- [errors.md](docs/architecture/errors.md) — standardized error envelope: taxonomy source of truth, the three-layer package split (core/api-types/lib), producers (`respondError`) + consumers, how to add a code.
- [idempotency.md](docs/architecture/idempotency.md) — optional replay-safe POST contract, encryption secret provisioning, supported routes, cleanup, and failure boundary.
- [taxonomy.md](docs/architecture/taxonomy.md) — classification axes: source `kind`, products, release type, tags, categories, collections, and how they differ.
- [semantic-search.md](docs/architecture/semantic-search.md) — Vectorize indexes, hybrid RRF, query cache, related-entity rails.
- [mcp.md](docs/architecture/mcp.md) — remote MCP server, scope enforcement, WebMCP parity, MCP Registry listing.
- [mcp-cimd-interop.md](docs/architecture/mcp-cimd-interop.md) — generic MCP client OAuth (DCR grant_types, kitchen-sink scope, origin vs `/mcp` resource, forwarded Bearer).
- [agents.md](docs/architecture/agents.md) — managed agents (discovery + worker), skills, Claude Code integration.
- [coverage.md](docs/architecture/coverage.md) — release coverage + ingest-time grouping, cron observability.
- [web.md](docs/architecture/web.md) — changelog range/slicing API, GitHub CHANGELOG ingestion, Open Graph images, org overviews, category overlay, collections, media pipeline, org avatars.
- [events.md](docs/architecture/events.md) — release event bus: `ReleaseHub` Durable Object, `GET /v1/releases/stream` WebSocket, fire-and-forget publish from batch + cron ingest.
- [cli-distribution.md](docs/architecture/cli-distribution.md) — OSS repo, npm, Homebrew tap.
- [ai-gateway.md](docs/architecture/ai-gateway.md) — optional Cloudflare AI Gateway passthrough for Anthropic SDK calls; covers direct worker calls, leaves Voyage embeddings + managed-agent internal loops on the direct path.
- [extract.md](docs/architecture/extract.md) — two-tier extraction path: one-shot inline for small bodies, multi-round tool-use loop for large ones, hard fallback to one-shot on any failure. Feature-gated behind `EXTRACT_TOOLLOOP_ENABLED`.
- [local-ingest.md](docs/architecture/local-ingest.md) — local Claude Code onboarding path: the `local-ingest` skill has the agent extract releases itself and write via `/batch` (no remote MA, no extraction billing), gated by a mandatory `robots.txt`/`Content-Signal` opt-out preflight.
- [feature-flags.md](docs/architecture/feature-flags.md) — Cloudflare Flagship Tier-1 boolean flags: registry, evaluation order, per-flag reference, dashboard setup.
- [content-pipelines.md](docs/architecture/content-pipelines.md) — map of every routine/scheduled AI-content pipeline (overview regen, batch summarize, collection summaries, digests, ingest-time passes): schedule, gate, model lane, manual trigger.
- [firecrawl-monitoring.md](docs/architecture/firecrawl-monitoring.md) — external Firecrawl fetch backend for challenge-blocked `scrape` sources: monitor lifecycle, the hunkless `monitor.page` diff wire format, the diff-delta vs. full re-scrape ingest paths, cost gate, poll-fetch exclusion, and staleness resilience.
- [maintenance-workspace.md](docs/architecture/maintenance-workspace.md) — per-user `~/.releases/work/` (tasks / runs / reports) convention for agent-driven admin maintenance; durable, cost-aware trail for the seeding/maintaining/managing/overview skills, reachable across the monorepo and CLI checkouts.
- [well-known-config.md](docs/architecture/well-known-config.md) — owner-declared `releases.json` v2 manifest: host-scoped authority (domain = org + products + release locators, repo = product binding), locator-first matching, cost-tiered source materialization, reconciliation precedence, the sync route + daily sweep.
- [workspaces.md](docs/architecture/workspaces.md) — Better Auth organization module: personal-workspace provisioning (lazy, on session create), the schema/naming split from the registry `organizations`, role separation (`member.role` vs `user.role`), and the inert org-keyed Stripe billing seam.

## Environment

Do not edit `.env` directly. Required vars are documented in `.env.example`. App env vars use the `RELEASES_` prefix (`RELEASES_API_URL`, `RELEASES_API_KEY`, `RELEASES_DATA_DIR`, …).

## Staging

Staging is a read-surface for UI/API iteration plus an agent-iteration sandbox, not a full replica — no crons, no webhooks, no Vectorize, access-key gated. Hosts, deploy commands, and the prod-sync script: [staging.md](docs/architecture/staging.md).

## Legacy naming

The project was originally called "Released"; the rename to "Releases" leaves the Cloudflare resource names deliberately unchanged:

- **Cloudflare resources** keep the `released-` prefix: D1 database `released-db`, and R2 buckets `released-media` (permanent, public media) and `released-raw` (ephemeral raw-page snapshots for backfill — content-hash keyed, 90-day lifecycle, public domain `raw.releases.sh`; see [firecrawl-monitoring.md](docs/architecture/firecrawl-monitoring.md)). The first two predate the rename, so renaming them is a live migration, not a text change; new resources (`released-raw`) keep the prefix for consistency.

Everything else — env vars (`RELEASES_*`), copy, prompts, display names, webhook headers (`X-Releases-*`), package/workspace names, the `~/.releases` data dir, localStorage keys — uses the new name.