git:20260817.34bb80b to git:20260820.8b2295d

# api-routes

## Purpose

Shared Fastify route plugins used by both the local server (`packages/canonry`) and the cloud API (`apps/api`). This is the HTTP surface for the entire platform — ~180 operations across ~27 route files.

## Key Files

| File | Role |

|------|------|

| `src/index.ts` | Plugin entry point, global error handler, `ApiRoutesOptions` interface |

| `src/helpers.ts` | `resolveProject()`, `writeAuditLog()`, `incrementUsage()`, `notProbeRun()` (Drizzle predicate every dashboard/analytics/report/timeline/intelligence read MUST AND-in to exclude probe runs — see root AGENTS.md "Probe runs" section) |

| `src/db-derived-dtos.ts` | `drizzle-zod`-derived row schemas (`projectRowSchema`, `runRowSchema`, `scheduleRowSchema`, `notificationRowSchema`) for the migrated tables. Per-column refinements narrow JSON columns and enum text columns to the typed Zod shapes the DB writes. Use for runtime validation of rows read from these tables; the hand-rolled DTOs in `@ainyc/canonry-contracts` remain the SDK source (see the "Derived row schemas" section below). |

| `src/projects.ts` | Project CRUD routes (largest route file) |

| `src/runs.ts` | Run trigger, status, and list routes |

| `src/results-export.ts` | `GET /projects/:name/results/export` — bulk download of every persisted answer-visibility query × provider observation as a versioned JSON artifact (`canonry.results-export/v1`) or spreadsheet-safe CSV attachment. Probe runs excluded by default (`notProbeRun()`, opt-in `includeProbes`); `kind = answer-visibility` only; inclusive `since`/`until` on run creation. Citation and mention exported independently (tri-state `answerMentioned` preserved, `mentionState` derived); `query` = snapshot-time text with FK fallback so removed queries keep history. Grounding evidence is extracted from the `rawResponse` envelope; the raw provider payload (`apiResponse`) is never exported. CSV cells neutralize leading `=+-@` (spreadsheet injection). Loads the full result set in memory — no pagination (fine single-tenant; revisit past ~100k snapshots). MCP: `excluded-protocol` (bulk attachment). |

| `src/query-replace.ts` | `replaceProjectQueries` — the ONLY way to declaratively replace a project's tracked queries (used by `POST /apply`, `PUT /queries`, `PUT /keywords`). Diffs by `normalizeQueryText`: unchanged texts KEEP their existing rows (ids anchor every historical snapshot's `query_id` FK — delete-all + reinsert orphans the project's whole sweep history), casing-only changes update text in place, incoming duplicates collapse, and only removed rows are deleted (after `preserveSnapshotQueryText`, also exported here, stamps their text onto referencing snapshots). Migration v98 relinks snapshots already orphaned by the pre-fix behavior. |

| `src/auth.ts` | Auth plugin — API key and session validation. Exports `hashApiKey()` (sha256 of a raw `cnry_…` token → `api_keys.key_hash`) and `requireScope()`; both reused by `keys.ts`. The `onRequest` hook also enforces the **global read-only gate**: a read-only key (`isReadOnlyKey(scopes)` from contracts — has `read`, no `*`/`*.write`) is rejected on every mutating HTTP method (POST/PUT/PATCH/DELETE) with `403`; GET/HEAD/OPTIONS pass. Method-based, so a new write route is read-only-protected automatically — see the root AGENTS.md "Deployment Posture" read-only-keys bullet. Also exports `requirePaidReadScope()`, the gate for GETs that SPEND (they call a provider live on the caller's behalf rather than returning stored data): an ALLOW list of `*` / `ads.write` / `ads.approve` / `ads.activate`, so a read-only key, a key scoped to something unrelated, and an empty scope list are all refused. Pair it with `requireAdminSession()` — the scope gate returns early for a signed-in person, so alone it would let a viewer spend. |

| `src/keys.ts` | API key management — `GET /keys` (ungated list, SAFE metadata only: id/name/prefix/scopes/timestamps + `projectId`/`projectName` + derived `readOnly`, never the hash or plaintext), `GET /keys/self` (introspect the CURRENT request's key — ungated read, returns the same SAFE DTO incl. `readOnly`; powers `canonry key whoami` + the MCP read-only auto-detection), `POST /keys` (mint a `cnry_…` token, returns the plaintext ONCE; gated by the `KEYS_WRITE_SCOPE` = `keys.write`), `POST /keys/:id/revoke` (sets `revokedAt`, idempotent, refuses to revoke the currently-authenticating key; gated by `keys.write`). The derived `readOnly` flag comes from `isReadOnlyKey(scopes)` in `toApiKeyDto`. Audit-logs `api-key.created` / `api-key.revoked` (prefix + scopes only, never key material). |

| `src/openapi.ts` | OpenAPI spec generation |

| `src/visibility-attribution.ts` | `buildQueryAttribution` + `resolveCurrentQuery` — historical query attribution by stable `queryId` then snapshot `queryText` fallback |

| `src/report.ts` | Report bundle (uses `visibility-attribution` for historical query resolution) |

| `src/analytics.ts` | Analytics and visibility score endpoints. Model attribution is historical evidence, never project configuration: bucket membership follows `runs.createdAt` (the logical sweep time), not asynchronous snapshot persistence. For each provider and sweep, classify all tracked snapshots as `known`, `unknown`, or `mixed`; `null`/empty model evidence is preserved, and a provider absent from a sweep is not `unknown`. Bounded windows load only the latest prior provider sweep as an unreturned anchor, so the first in-window state can emit a truthful transition. |

| `src/visibility-stats.ts` | `GET /projects/:name/visibility-stats` — aggregated per-query mention/citation counts with a sample size, pooled across many answer-visibility runs (probe-excluded, completed/partial), optional per-provider breakdown. Reads the RAW tri-state `answerMentioned` column (`checked` = non-null) so `null` ("not checked") is never coerced to not-mentioned — `mentionRate = mentioned/checked`, `citedRate = cited/total`. Window via `since`/`until` (ISO), `lastRuns`, or `month=YYYY-MM` (mutually exclusive). `shareOfVoice=1` adds pooled project-vs-tracked-competitor brand-mention share via `buildMentionShare` (opt-in — answer text is loaded only on that path). Pure, testable `computeVisibilityStats` + the route. Also hosts `GET /projects/:name/visibility-compare?from=YYYY-MM&to=YYYY-MM` — statistically honest month-over-month AEO comparison (share-of-voice-led + drift-robust, per-snapshot K-invariant rates, common query+provider basket, Wilson intervals, CI-overlap `verdict`, `modelChanges` diff of the configured model id); the calc is the pure `computeVisibilityCompare` in `src/visibility-compare.ts`. |

| `src/visibility-compare.ts` | Pure `computeVisibilityCompare` (no DB/clock; the `gbp-summary.ts` precedent) — the month-over-month engine behind `GET /visibility-compare`. Restricts to the query+provider basket present in BOTH months, pools per-snapshot rates (K-invariant), computes named + cited share of voice (SoV cancels engine drift so it carries the directional call), attaches Wilson intervals (`wilsonInterval` from contracts) and a CI-overlap `verdict` (`within-noise`/`moved`/`insufficient-data`), and diffs the configured `model` id per provider into `modelChanges` (a config change is visible; a silent upstream version bump is not). Reuses the exported `buildQueryAttribution`/`resolveCurrentQuery` from `visibility-stats.ts`. |

| `src/google.ts` | Google Search Console **and** Google Business Profile (GBP) routes. GSC: OAuth connect/callback, property selection, sync, coverage, plus `GET /gsc/top-pages` (one row per page, `GROUP BY page` + `SUM(clicks)` in SQL so the response is bounded by distinct pages, not by the dimensioned rows behind them). **The dimensioned `gsc_search_data` table is valid for RANKING and invalid for TOTALS**: Google withholds rare/anonymised queries so its sum under-counts clicks, and one impression fans out across every query x page x country x device combination so its sum over-counts impressions (792 vs 1,142 clicks and 45,266 vs 34,916 impressions on one real property-month). `top-pages` therefore sources `totals` from the un-dimensioned `gsc_daily_totals` table, labels it `totalsSource: 'property-daily'`, and returns `null` when no property figure covers the window rather than falling back to the sum; `/gsc/performance/daily` reads the same table through `readGscDailyTotals`. Guarded by `test/gsc-top-pages.test.ts`, whose fixture makes the two sources deliberately disagree. GBP: OAuth connect/callback (shares the Google OAuth client; `gbp` connectionType), `GET /gbp/accounts` (accounts the OAuth user can access — account selection is **per project**), `POST /gbp/locations/discover` (resolves the account: explicit `accountName` > the account the project already tracks > first visible; re-pointing a project at a different account is destructive and requires `switchAccount: true`, which clears the old account's footprint via the shared `clearGbpProjectData` helper) + select/deselect, `POST /gbp/sync` (creates the `gbp-sync` run, fires `onGbpSyncRequested`), and the read endpoints `GET /gbp/locations`, `/gbp/metrics`, `/gbp/keywords`, `/gbp/place-actions`, `/gbp/lodging` (collapses to the latest snapshot per location), and `/gbp/summary` (scopes to the project's SELECTED locations — deselected/stale rows never pollute the aggregate, and `locationCount` matches the data covered — passes the server `asOfDate` to `buildGbpSummary`, which derives the complete-day anchor + freshness + daily timeseries from the data). `DELETE /gbp/connection` clears the project's whole GBP footprint (locations + all synced surfaces), not just the connection. |

| `src/ads-live-delivery.ts` | Pure live-vs-stored comparison engine (no DB/clock/network; the `gbp-summary.ts` precedent) behind `GET /ads/live-delivery`: `buildFieldDeltas` (status / reviewStatus / name; nothing when one side is absent, since `presence` already says it), `buildMetricDeltas` (per-date union of provider rows and in-window stored rollups; the provider's disjoint buckets for one date are summed, because impressions/clicks/spend/conversions are additive while the ratio metrics ctr/cpc/cpm are NOT and survive only verbatim in `liveMetrics`, and spend is converted from the insights API's decimal units to stored micros so both sides are commensurable), `liveComparisonWindow` (the compared date range, dated in the AD ACCOUNT's timezone via `formatIsoDateInTimeZone` because that is the zone the provider buckets by and the zone ads-sync stamps rollup dates with; deriving it in UTC drops the account's current local day for any account east of UTC and admits an unasked-for day at the other end, both of which report as drift that is not real — and the lookback is stepped by `isoDateDaysBeforeInTimeZone`, i.e. in CALENDAR days of that same zone, because a local day is 23 hours when a zone springs forward and 25 when it falls back, so a fixed 24-hour step moves the first day of the window and silently adds or omits a metric date), `buildLiveEntityComparison`, and `summarizeLiveDrift`. The window's `startDate` is ALSO the date the provider range is asked to start at, from the start of that account-local day (`AdsLiveInsightsRequest.startDate`, resolved by the host through `startOfDayHourInTimeZone`, which is hour 00 except on the one day a year a zone that springs forward at midnight has no hour 00), so the first day of the window is a whole day on both sides: when the provider range instead began at the read instant's local hour, that day was a mid-day slice upstream against a full-day rollup and the diff manufactured drift on it on every read. The window's `endDate` is the account's CURRENT local day, and the provider range's UPPER edge is the START of that day (`startOfDayHourInTimeZone`), so the range covers the window's CLOSED days and stops. It cannot reach further: the provider refuses an `until` in the future with `400: time_ranges.end cannot be in the future`, failing the whole read, and it reports a daily bucket only when the range fully covers its boundaries, so no accepted edge returns the open day either. The host therefore reads that day from a SECOND, unranged provider call and merges it in (`readInsightDays` in the host's ads-sync); without it the live side omitted today while the stored side held it, and every read reported a stored-only day. Today is in progress on BOTH sides (live to the read instant, stored to the last ads-sync), so a difference there is snapshot staleness and is reported as the drift it is, while identical partial readings correctly report no drift. |

| `src/gsc-period-comparison.ts` | Pure trailing-period vs prior-equal-period comparison behind the GSC performance tiles (no DB/clock/I/O; the `gbp-summary.ts` precedent). Replaced a percentage taken off the fitted trend line's own START value, which is not a baseline: the fit is unconstrained, so it predicted **-13.98** impressions on day one for a real property and the tile printed nothing on a metric that had grown six-fold, and where it did print it described the LINE rather than the data (average position read as a 45.8% improvement across a window in which the property's real position got WORSE). Splits the authoritative requested window over the CALENDAR, not the sparse row endpoints or row array, because Search Analytics omits zero-data days; a quiet half inside the project's monotonic observed-data frontier is an explicit `empty` period, while a range extending beyond that frontier has no comparison because the absent days are ambiguous, not proven zero. The split is boundary arithmetic rather than a per-day allocation, so an extremely wide custom range stays bounded. Any `dimensioned` evidence makes comparison unavailable because `SUM(gsc_search_data)` is invalid for property totals even on both sides. An odd span drops its OLDEST day so both periods are equal. CTR is each period's own `clicks / impressions` and position is impression-weighted, both matching the window totals: neither is additive, and a mean of daily ratios lets a one-impression day count as much as a thousand-impression one. `change` stays null when the prior period is zero or unmeasured, since growth from nothing has no percentage. Sign is mathematical, so a POSITIVE `position` means a worse rank and the renderer owns the arrow. |

| `src/gbp-summary.ts` | Pure GBP summary calculation module (no DB, no I/O) — `computeMetricTotals`, `computeWindowDelta` (recent-7d vs prior-7d per metric, both windows backfilled with the union of metrics as explicit `0`s; `deltaPct` is `null` when the prior window is `0` to avoid divide-by-zero), `computeFreshness` (reporting-lag detection: `dataThroughDate` = last non-zero day, `latestStoredDate`, `pendingDays` vs the `asOfDate` — only TRAILING zeros count, #658), `buildTimeseries` (per-day pivot over the most recent ~30 days, each day flagged `pending` when it falls in the lag tail), `computeKeywordCoverage` (total + thresholded count/pct), `summarizePlaceActions` (CTA-type presence flags), `summarizeLodging` (lodging / populated / empty counts), and `buildGbpSummary` which composes them — it anchors the recent/prior windows to the **last complete day** (not the lagging tail) so a reporting-lag artifact is never shown as a real decline. Takes an injected `asOfDate` (never reads the clock) so it's deterministic and unit-testable. Tested exhaustively in `test/gbp-summary.test.ts`. |

| `src/bing.ts` | Bing Webmaster Tools routes |

| `src/ga.ts` | Google Analytics 4 routes. **ONE WINDOW PER RESPONSE.** `/ga/traffic` resolves a single `measuredRange` before it builds any query, and the snapshot, AI-referral, and social-referral predicates all derive from it; `windowStart` / `windowEnd` / `windowDays` report it, and `periodStart` / `periodEnd` are retained aliases. Exact 7d/30d/90d reads may use the matching precomputed summary for the denominator and deduplicated users. `all` and an omitted window mean full retained history: totals and sibling history routes stay unbounded. The latest sync summary may supply deduplicated users only when its dates cover every retained detail row; otherwise `totalUsers` is null. Never use that summary to narrow `all`. Any new figure added to `/ga/traffic`, and any new route whose numbers are read beside it, must use the same resolved range. |

| `src/ads.ts` | OpenAI ads (ChatGPT ads) routes: connection/status/sync + nested snapshot/insight/summary reads, and `ads.write`-gated image/campaign/ad-group/ad lifecycle mutations. Creates are forced paused; updates require the entity already be paused plus exact synced `upstreamUpdatedAt`. Activation is exposed only through a separate `ads.activate` scope and a short-lived, human-issued `ads.approve` grant bound to the exact entity tree, advertiser account, and executor key; its bodyless resume route recovers only the existing operation under that exact executor, while generic reconcile rejects activation receipts. Archive remains absent. Every mutation inserts a durable `(project, operationKey)` receipt before I/O, replays identical requests, blocks hash conflicts, records ambiguous outcomes as `unknown`, and verifies/remediates its lifecycle postcondition. Manual generic recovery cannot claim a fresh pending receipt; the shared reconciler uses leases, exponential backoff, a five-attempt quarantine, exact parent/fingerprint checks, and a short-lived credential-fingerprint/account verification cache. Unresolved reads use opaque keyset cursors so permanent rows cannot wedge the first page. Provider credentials stay in config.yaml. The four LIVE planning reads (`/ads/account`, `/ads/geo/search`, `/ads/conversions/pixels`, `/ads/conversions/event-settings`) carry the same `requireAdminSession` + `requirePaidReadScope` pair as `/ads/live-delivery`, applied BEFORE the credential is resolved so a refused caller never reaches the provider: they are provider calls that spend on the ad account (the two conversions lists auto-paginate up to `OPENAI_ADS_MAX_PAGES`), not reads of stored data. The stored-rollup reads (`/ads/insights`, `/ads/summary`, `/ads/delivery-diagnostics`) now disclose the partial day the sync stores: each insight row carries `inProgress`, and each rollup window carries `inProgressDate`, both derived from the ACCOUNT's local date via `adsAccountToday` (a UTC-derived "today" names a different day for an account far enough east or west and would let a running total read as a closed one). Also hosts `GET /projects/:name/ads/live-delivery`, a read-only LIVE passthrough that calls the provider on demand (no sync run, no snapshot tables) through the list/insight-only `AdsLiveDeliveryReader` seam and returns the provider's current status and metrics unaggregated, the matching stored values, and an explicit per-entity delta. It verifies account identity first (the key can have moved), bounds the walk with per-level caps AND a total READER-CALL budget, suppresses stored-only rows whenever the walk truncated or a sub-read failed (unwalked is not absent), and admits reads under TWO separate rules (`429` either way, `error.details.reason` says which). SINGLE-FLIGHT: a per-project in-flight marker, claimed synchronously before any `await` and cleared in a `finally` however the walk ends, refuses a concurrent read for as long as the first walk actually runs — a walk is unbounded in time (about 4040 upstream requests), so expressing exclusion as a duration let the claim expire mid-walk and put two walks on one account. MINIMUM INTERVAL: one read per project per minute between walks, counted from the last attempt that reached provider I/O — a failure after the first provider call holds the interval exactly like a success, because re-arming on failure would turn a provider outage into a retry storm, while a failure BEFORE the provider was touched writes nothing and can be retried at once (no compensating release exists, or is needed: the clock is only ever written where the provider is about to be asked something). `retryAfterMs` is present whenever the interval itself still blocks and absent on a concurrency refusal that has outlived it, since the running walk's remaining time is not knowable. The attempt Map is swept whenever an attempt is recorded so it cannot grow per-project forever; the in-flight Set needs no sweep because the `finally` bounds it to the requests actually running. `bounds` reports two different units on purpose: `maxReaderCalls` (40) counts logical list/insight reads, while each such read auto-paginates up to `maxPagesPerReaderCall` (`OPENAI_ADS_MAX_PAGES`, 100) and an insight read adds one more single-page request for the day in progress, so `maxUpstreamHttpRequests` (about 4040) is the honest upstream ceiling; the HTTP figure is a documented bound, since pagination happens below the reader seam. Every provider failure is reduced to a fixed surface label plus the numeric upstream status, never the upstream message, body, or code, so no credential material can ride an error into a response or a log. |

| `src/intelligence.ts` | Intelligence insights and health snapshot routes |

| `src/report.ts` | `GET /projects/:name/report` (JSON DTO) and `GET /projects/:name/report.html` (standalone downloadable HTML) — aggregated client-facing AEO report bundle (13 sections) |

| `src/report-renderer.ts` | `renderReportHtml(report)` — server-side HTML renderer with inline SVG charts and inline CSS, re-exported from `@ainyc/canonry-api-routes` for the CLI |

| `src/wordpress.ts` | WordPress integration routes |

| `src/traffic.ts` | Server-side traffic ingestion routes — `POST /traffic/connect/cloud-run`, `POST /traffic/connect/wordpress`, `POST /traffic/connect/vercel` (Vercel connect seeds `lastSyncedAt = NOW` so the first scheduled sync uses a tight window — leaving it null would fall back to `DEFAULT_SYNC_WINDOW_MINUTES = 30 days`, which exceeds Vercel `request-logs` retention (~14d) and would make every first sync throw a retention error — and **auto-creates the project's `traffic-sync` schedule** (`*/30 * * * *`, idempotent via the unique `(project, kind)` index, registered with the live scheduler through `onScheduleUpdated`) in the same transaction as the source upsert, so the source actually keeps syncing without a manual `schedule set` step: seeding `lastSyncedAt = NOW` only keeps the FIRST window tight, and the schedule is what stops the watermark drifting into an unbounded — wedging — pull on a later trigger), `POST /traffic/sources/:id/sync`, `POST /traffic/sources/:id/backfill` (async — returns `{ runId, status: "running" }` immediately, background task replaces rollup buckets + sample slice in the window inside one transaction, days clamped to `MAX_BACKFILL_DAYS=30` to match Cloud Logging `_Default` retention, `lastSyncedAt` only advances forward so backfill never undoes incremental progress; supports `cloud-run`, `wordpress`, and `vercel` source types), `POST /traffic/sources/:id/reset` (operator recovery: requires `{ advanceToNow: true }` — advances `lastSyncedAt` to NOW, sets `status` back to `connected`, clears `last_error`; used when an idle source has aged past the upstream retention boundary and every sync now throws), plus reads: `GET /traffic/sources` (list non-archived), `GET /traffic/status` (composite of detail-per-source — single call powering `canonry traffic status`), `GET /traffic/sources/:id` (detail + last-24h totals + latest run, run filtered by `runs.source_id` so multi-source is correct), `GET /traffic/events` (windowed crawler / ai-referral rollups, defaults to last 24h, totals reflect the full window even when `limit` truncates). Credentials resolved through injected stores (`cloudRunCredentialStore`, `wordpressTrafficCredentialStore`, `vercelTrafficCredentialStore`); the per-adapter pull functions and access-token resolver are also injectable for tests. Upstream/auth failures throw `providerError()` (502) so CLI exit codes signal system errors. **Sync dispatcher:** the sync route resolves the source row, sets up the run + shared error path, then branches by `sourceType`. Cloud Run uses a clamped time window (`startTime`/`endTime` + `lastSyncedAt` clamp); WordPress pages through the plugin's opaque `next_cursor` driven by the response's `hasMore` flag, persisting the final cursor to `traffic_sources.last_cursor` inside the same transaction as the rollup writes; Vercel uses a clamped time window like Cloud Run but the `request-logs` endpoint paginates by page number with no resumable cursor, so the window is drained in adaptive time sub-windows (`drainVercelTrafficEvents`, `DEFAULT_VERCEL_MAX_PAGES=50` per sub-window). Two bounds keep a dense or drifted window from wedging the synchronous sync: **(1)** the start is capped to at most `VERCEL_MAX_SYNC_WINDOW_MS=24h` before the sync instant — a watermark that drifted further is clamped forward and the skipped span is surfaced via `warn` (a backfill recovers it); **(2)** the drain runs under a wall-clock budget (`DEFAULT_VERCEL_SYNC_DEADLINE_MS=4m`, override `vercelSyncDeadlineMs`) — on the budget it stops and the route commits the partial window and advances `lastSyncedAt` **only to where it drained** (the additive rollup makes a partial window safe), so the next sync resumes from there instead of one sync grinding for many minutes; if nothing drained before the budget the run **fails (visible)** rather than orphaning a `running` row. Retention is still enforced: if the drain can only serve a clamped tail it fails so `lastSyncedAt` never advances across missing history. Dedupe + rollup + telemetry are shared across all three branches. **Backfill dispatcher:** the backfill route mirrors the same shape — `runBackfillTask` is adapter-agnostic and takes an injected `pullForBackfill: () => Promise<NormalizedTrafficRequest[]>` closure plus a `pullErrorPrefix` string so error attribution stays specific. The route handler validates credentials per `sourceType` up-front, then builds the closure: Cloud Run pulls a single `[startTime, endTime]` window via the Cloud Logging API; WordPress pages through the plugin's `[since, until)` window via opaque cursor; Vercel pulls the `[windowStart, windowEnd]` window with the large `BACKFILL_MAX_PAGES` budget — replace mode, so a budget exhaustion (`hasMore` still true) fails the run loudly rather than wiping the window's rollups and leaving a partial set. All reuse the shared replace-mode rollup transaction and the `lastSyncedAt`-never-rewinds invariant. **Cross-sync dedupe:** for Cloud Run and Vercel, `lastSyncedAt` clamps the fetch window forward to avoid wholesale re-pulls; the boundary second is then deduped via `traffic_sources.last_event_ids` (bounded ring buffer of `MAX_TRACKED_EVENT_IDS=1000` normalized event IDs from prior syncs, persisted inside the same transaction as the rollup writes). New sync IDs are prepended to retained previous IDs so a dup that re-appears across multiple subsequent syncs stays deduped. WordPress reuses the same ring-buffer logic for plugin-side cursor-boundary re-emissions. |

| `src/backlinks.ts` | Backlinks routes — Common Crawl sync + per-project extract. Summary/domains/history default to `source=commoncrawl`; the source discriminator remains readable for inert historical rows from older versions. `GET /projects/:name/backlinks/sources` reports Common Crawl readiness plus retained-source data availability. |

| `src/doctor.ts` | `GET /doctor` and `GET /projects/:name/doctor` — runs check registry, returns `DoctorReport` |

| `src/technical-aeo.ts` | Site Health / Technical AEO routes: legacy score/pages/trend plus bounded crawl summary, persisted graph projection, page inventory, one-level structure, internal-link list/neighbors, semantic subgraph/path/complete-run changes, and opt-in dead-link reads. `POST /technical-aeo/runs` persists normalized request identity before queueing: only identical effective options reuse an active run; a sitemap, budget, depth, or dead-link difference returns `409`. `GET /technical-aeo/runs` is the Site Health scan history: every non-probe site-audit run newest-first, each with `hasCrawlData`. A `runId` naming a real surfaceable run that published no crawl (a legacy score-only scan) gets that route's own no-crawl shape, NOT a 404; only an unknown or foreign `runId` still 404s. Default graph reads select the latest complete non-probe crawl; explicit `runId` may inspect a partial terminal snapshot. The graph read carries `rootNodeKey` so the home page is identified by the server rather than guessed from a path or a depth. The dashboard graph reads only the persisted 20k-node / 50k-edge projection; agent reads traverse canonical page/edge rows without layout coordinates and return bounded/truncated states. Every link-bearing read tags each edge `isTemplate` (nav, header, or footer chrome) plus `templateSource` and `placementOccurrences` (which rule decided it and the DOM evidence behind it), accepts a `linkKind` filter (`all` by default, so an existing caller's counts do not move), and reports `templateDetection`, so an empty content-only list can never be mistaken for a real zero and no count silently mixes the placement and ubiquity rules. `templateSource` is derived at read time from the row plus ITS OWN scan's detection state, which is why `mapCrawlEdge` takes the detection: a scan that never recorded placement cannot report `placement`. It also reads the stored `templateRatio`, so an edge the fallback could not measure (a redirect, a canonical, an unresolved target) reports `unmeasured` instead of being credited to a rule that produced no number. `isTemplate` is a strict boolean on every classified row, so `linkKind=content` returns the same set on the graph read, the link list, and the neighbour read. The graph read adds `totalTemplateEdges` / `totalContentEdges` beside the unchanged `totalEdges`, and its ready layout says whether template links were excluded from the physics. |

| `src/doctor/registry.ts` | `ALL_CHECKS` — single source of truth for the doctor check catalog |

| `src/doctor/runner.ts` | `runChecks()` and `matchesCheckId()` — execute filtered checks, build the report |

| `src/doctor/checks/*.ts` | Individual `CheckDefinition`s (google-auth, gbp-auth, bing-auth, ga-auth, providers, traffic-source, content). `content` covers `content.winnability.coverage`, which measures how many cited-surface domains the shared surface classifier recognizes (own / tracked-competitor / static allow-list / stored discovery `domain_classifications`) so the ownable/ceded gate does not silently fail open, and nudges to set an ICP when the project has none. `gbp-auth` covers `gbp.auth.connection` / `gbp.auth.scopes` / `gbp.account.access` (token refresh + `business.manage` scope + the tracked account is listable via `listAccounts`, mapping `GbpApiError` 0-QPM → a `quota-pending` warn) and `gbp.data.recent-sync` (selected-location sync freshness). The `traffic-source` checks are adapter-agnostic — they query `traffic_sources` directly for connection/recent-data, and dispatch credential / scope validation through `DoctorContext.trafficSourceValidators[<sourceType>]`. v1 registers validators for `cloud-run` (service-account-token resolution), `wordpress` (probe-call against the plugin's REST endpoint), and `vercel` (probe-call against the `request-logs` endpoint — 401/403 maps to `traffic.credentials.unauthorized`), wired from the corresponding credential stores in `index.ts`. Future adapters plug in by adding a key to that map — no doctor-side changes needed. |

| `src/measurement-property-evidence.ts` | `GET /projects/:name/measurement-property-evidence` — cursor-paged evidence for exactly ONE Property out of one revision-pinned run, optionally narrowed by question class, provider, or location. It exists beside `/measurement-report` rather than as a filter on it: the report reconstructs a whole revision (every group, every Target, unpaginated) for a revision the caller names, its `groups`/`targets`/`diagnostics` are run-level and would have to start meaning something narrower whenever a filter was present, and the Branded/Non-brand class lives on a v2 assignment that the v1 revisions `/measurement-report` also serves do not record. Run selection, revision-mismatch refusal, and the plan lookup are the overview's (`activeMeasurementPlan` / `latestMeasurementRun` / `runRevisionMismatch` / `displayedState`), so the two surfaces can never display different runs. Rows come from `buildMeasurementEvidence` — the same `prepareReport` pass the report uses, minus the roll-ups — and are filtered to the usage edges the Property owns, so a sibling's citation is never credited here. `shape` chooses what a ROW is. `sources` (the default, so an existing caller is untouched byte for byte) is one row per cited URL under `evidence`; `answers` is one row per measured answer under `answers`, with the cited URLs nested and both signals on the row. The answer shape is what explains a GAP: an answer that mentioned the Property without linking it, or that named nobody, has no URL to hang a source row on and is absent from the flat shape entirely, so counting source rows understates what was measured. Exactly one page key is returned and the other is ABSENT rather than empty — empty is a statement about the measurement, absent is a statement about which reading was asked for. The two are two readings of one selection, not two computations: `buildMeasurementEvidence` derives the per-URL rows from the answer rows in one pass and the same edge/provider/location predicate narrows both. Paging keys per shape — `[expectedSlotId, usageEdgeId, sourceUrl]` for sources, `[expectedSlotId, usageEdgeId]` for answers, each the exact tuple the kernel sorted by, so an answer's own cited URLs can never straddle a page boundary. The cursor pins the active revision, displayed run, evidence fingerprint, filters, and the SHAPE it was issued for (refused by name on the other); it omits the shape field for the default, so a cursor minted before the parameter existed still walks. A v1 active revision is refused with `validationError` rather than answered with an invented class. An empty page under `measurement.state = not_measured` is the absence of a measurement, NOT a measured zero. |

| `src/discovery/routes.ts` | Tracked-basket discovery routes: `POST /projects/:name/discover/run` (writes `discovery_sessions` + `runs` rows and fires the injected `onDiscoveryRunRequested` callback; concurrent duplicates consolidate onto the in-flight session, whose identity is (project, `icpDescription`, `buyerDescription`, resolved `locations`, canonical `seedProviders` — explicit `['gemini']` normalizes to the omitted default) — buyer changes seed semantics so a different/absent buyer never reuses another buyer's session, while `dedupThreshold` / `maxProbes` / `probeConcurrency` are tuning and are dropped on reuse; any NEW request field must be classified identity-or-tuning the same way (root AGENTS.md "Request Parameters: Identity vs Tuning") — returns `{ runId, sessionId, status: "running" }` immediately; resolves the request's optional `locations` label override against the project's configured locations via `resolveLocations` — unknown label → 400 — and forwards the resolved `LocationContext[]` on the callback so seed generation is geo-constrained), `GET /projects/:name/discover/sessions` (list), `GET /projects/:name/discover/sessions/:id` (detail + per-query probes), `GET /projects/:name/discover/sessions/:id/harvest` (read-only — reads the issued search-query fan-out, e.g. Gemini `groundingMetadata.webSearchQueries`, back out of the session's stored probe `raw_response` via the injected `harvestSearchQueries` seam, then runs the mandatory `gateHarvestedSearchQueries` lexical gate followed by `applyHarvestSemanticNovelty` — an embedding cosine pass over the tracked queries via the injected `embedQueries` seam (the Gemini embedder, same as discovery seeds) that drops paraphrase/synonym duplicates exact-match can't see; degrades to exact-match when embeddings are unavailable, reported as `semanticNoveltyApplied`. Returns candidate seeds ranked by probe recurrence + per-reason rejection stats incl. `semanticDuplicate`; `minProbeHits`/`anchor` query params; issue #713), `GET /projects/:name/discover/sessions/:id/promote` (read-only preview of bucketed queries + recurring suggested competitor domains of **every** classified type so the operator can see what `competitorTypes` would unlock), `POST /projects/:name/discover/sessions/:id/promote` (adopt a completed session's cited + aspirational queries plus recurring competitor domains classified `direct-competitor` into the project by default, tagged `provenance="discovery:<sessionId>"` — add-only, idempotent, single transaction + audit log; `buckets` / `includeCompetitors` / `competitorTypes` request fields scope it). `parseCompetitorMap` normalizes legacy competitor-map JSON (no `competitorType`) to `unknown`; `selectEligibleCompetitors` filters by hit floor + optional type set. |

| `src/discovery/orchestrate.ts` | `executeDiscovery` — pure orchestration with injected `DiscoveryDeps` (seed/embed/probe/classifyDomains). Persists `discovery_sessions` status transitions (`seeding` → `probing` → `completed`/`failed`), embeds + clusters via `clusterByCosine` (default threshold `DISCOVERY_DEFAULT_DEDUP_THRESHOLD`, calibrated so single-link chaining cannot bridge distinct intents), picks shortest-string representatives, records a session `warning` via `seedCollapseWarning` when dedup degenerately collapses the seed set (measured before the probe-budget slice), classifies each probe into cited / aspirational / wasted-surface and persists the probe's `answerMentioned` (the answer-text mention signal the dep computes, independent of citation; nullable for legacy rows), then runs one best-effort `classifyDomains` call to type every recurring cited domain (`direct-competitor` / `ota-aggregator` / `editorial-media` / `other`; failures fall back to `unknown`) and aggregates the typed competitor map. Caps probe budget at 100 default / 500 absolute. When 2+ seed providers ran, dedup uses a MONOTONIC multi-provider merge (`pickCanonicalsWithStats` with a `primaryMask`): it anchors on the primary provider's own clustering (identical to a single-provider run over that subset) and only ADDS novel secondary candidates, so adding a provider can never reduce the canonical count below a single-primary run (guaranteed floor; proven by a property test). Single-provider sessions are byte-identical to the pooled path. Probes run through a bounded worker pool (`probeConcurrency`, default 1 = serial, cap `DISCOVERY_PROBE_CONCURRENCY_CAP` = 8) whose results are collected by canonical index and batch-inserted in canonical order in one transaction — concurrency never changes row order, bucket counts, or failure semantics (the first probe error fails the session). Also persists the seed dep's optional raw-candidate source split onto `discovery_sessions.seed_from_answer_count` / `seed_from_grounding_count` (diagnostics only). Migration 92 widens the diagnostics into full seed provenance: `seed_raw_candidates` (the pre-filter candidate list — every live session becomes a replayable fixture for filter/dedup changes) plus `dedup_cluster_min_sims` / `dedup_band_pair_fraction` / `dedup_pairs_total` (per-cluster cohesion and the ambiguous 0.90-0.97 band mass, the calibration data for any future threshold/linkage decision). Forwards the optional `locations` (resolved `LocationContext[]`) to `deps.seed` so a location-aware seed implementation can geo-constrain its queries, and passes the FIRST resolved location to every `deps.probe` call as the probe geo context (the provider renders it exactly like a sweep location), so probes measure from the buyer's service area instead of nowhere; location-free sessions probe unchanged. Plus pure helpers (`classifyProbeBucket`, `buildCompetitorMap` — accepts an optional classification map, `pickCanonicals`, `markSessionFailed`) for unit testing without spinning up the network. |

### Cloudflare traffic doctor boundary

Doctor behavior must branch on the persisted `configJson.deliveryMode`, not

only `sourceType = cloudflare`. Legacy rows with no mode are direct push.

Only `direct-push` skips pull-watermark lag; `queue-pull` sources use the

ordinary pull checks and a durable source-scoped lease. Queue messages are

acknowledged only after the receipt + rollup transaction commits; a post-commit

ACK failure relies on receipt dedupe for safe redelivery.

`traffic.source.worker-version` applies to both `direct-push` and `queue-pull`.

It compares the last-observed, ingest-recorded `lastWorkerVersion` with the shared

`CURRENT_CLOUDFLARE_WORKER_VERSION`, not the persisted

`configJson.workerVersion`, so a package upgrade detects a stale deployment

before the source is reconnected. This is latest-batch evidence, not proof that

every deployment or queued message runs one version. When `lastWorkerVersion` is null, it warns with

`traffic.worker-version.waiting-for-first-event`. A mismatch warns with

`traffic.worker-version.stale` and requires regeneration and redeployment using

the source's existing delivery mode.

## Discovery replay suite (quality regression)

`test/discovery-replay.test.ts` replays the deterministic seed pipeline (brand filter → exact dedup → cosine clustering → representative pick → collapse warning) against REAL captured sessions in `test/fixtures/discovery-replay/` — five ICP shapes, each with raw candidates + embedding vectors + golden expectations. CI makes zero provider calls. Two assertion tiers: GOLDEN exact equality (a deliberate pipeline change regenerates fixtures via `scripts/capture-discovery-replay-fixtures.ts` in the same PR — never loosen an assertion to pass) and INVARIANTS (canonicals >= the platform gate floor of 8, no collapse warning, branded raw candidates <= 20%). Refresh cost ~$0.50 total; see the capture script header for the procedure.

## Patterns

### Route file structure

Each file exports an async Fastify plugin function:

```typescript

import type { FastifyInstance } from 'fastify'

import type { ApiRoutesOptions } from './index.js'

export async function myRoutes(app: FastifyInstance, opts: ApiRoutesOptions) {

app.get('/my-endpoint', async (request, reply) => {

// handler

})

}

```

### How to add a new route

1. Create a new file in `src/` (or add to an existing domain file).

2. Export an async plugin function following the pattern above.

3. Import and register it in `src/index.ts`.

4. Add the endpoint to the OpenAPI spec in `src/openapi.ts`.

### Error handling

The global error handler in `index.ts` catches `AppError` instances. **Never catch and manually reply.**

```typescript

import { validationError, notFound } from '@ainyc/canonry-contracts'

import { resolveProject } from './helpers.js'

// ✅ Correct — let the global handler serialize

const project = resolveProject(app.db, request.params.name) // throws notFound on miss

if (!body.queries?.length) throw validationError('"queries" must be non-empty')

// ❌ Wrong — duplicates global handler logic

try { resolveProject(app.db, name) } catch (e) { reply.status(e.statusCode).send(e.toJSON()) }

```

### Validation

Use Zod schemas from `@ainyc/canonry-contracts`. Parse with `.safeParse()`, throw `validationError()` on failure.

### Typed responses (Critical)

**Every new route MUST register a Zod schema for its response and reference it via `jsonResponse(...)`.** The SDK (`@ainyc/canonry-api-client`) is regenerated from this spec on every `pnpm gen`; routes that return `rawJsonResponse(..., looseObjectSchema)` give web consumers `Record<string, unknown>` and silently break the end-to-end type pipeline.

The pattern:

1. Define the response shape in `packages/contracts/src/<topic>.ts` as a Zod schema:

```typescript

export const myResponseDtoSchema = z.object({ /* ... */ })

export type MyResponseDto = z.infer<typeof myResponseDtoSchema>

```

2. Register it in `packages/api-routes/src/openapi-schemas.ts` (alphabetized):

```typescript

const SCHEMA_TABLE = {

// ...

MyResponseDto: myResponseDtoSchema,

// ...

}

```

3. Reference it from the route definition in `src/openapi.ts`:

```typescript

responses: {

200: jsonResponse('Successful response.', 'MyResponseDto'),

404: errorResponse('Not found.'),

},

```

4. Run `pnpm --filter @ainyc/canonry-api-client gen` to regenerate the SDK.

`rawJsonResponse(..., looseObjectSchema)` is **capped** by the test

`packages/api-routes/test/no-new-loose-routes.test.ts` — the current count is

the high-water mark, and CI fails if it grows. To add an endpoint without a

schema you'd have to raise the cap, which is reviewable as a deliberate

decision. Don't.

For existing loose endpoints that you're typing for the first time, the same

test's TODO-comment cap will go DOWN by one as you remove the `// TODO: Add

`XxxDto` Zod schema in contracts.` placeholder. Drive both numbers toward 0

incrementally.

### Event callbacks

Routes fire lifecycle hooks via `opts` callbacks — `onRunCreated`, `onProviderUpdate`, `onScheduleUpdated`, `onProjectDeleted`. Fire these **after** the database transaction commits, not inside it.

### Derived row schemas (drizzle-zod)

`src/db-derived-dtos.ts` exports `*RowSchema` Zod validators generated from the Drizzle table definitions via `drizzle-zod`'s `createSelectSchema()`. Per-column refinements narrow:

- JSON columns whose `$type<>` is a TypeScript-only hint (drizzle-zod can't introspect those — it produces a loose `ZodUnion` fallback; the refinement supplies the actual schema)

- text columns whose values are an enum at the API layer (`configSource`, `runKind`, `runStatus`, etc.)

Use them when you want runtime validation of a row at the DB → DTO seam:

```typescript

import { projectRowSchema } from './db-derived-dtos.js'

// .parse(row) verifies the row matches the schema. Throws if the column

// types drifted (e.g. configSource got a value not in the enum).

const validated = projectRowSchema.parse(row)

```

Constraints:

- The hand-rolled DTO schemas in `@ainyc/canonry-contracts` remain the OpenAPI / SDK source. Derived schemas are an internal validator, not a public type — they live in `api-routes` because `contracts` can't import from `db` (db already imports types from contracts for `$type<>`, so the reverse would cycle).

- `db-derived-dtos.test.ts` asserts each derived schema's field set equals the table's column set, plus round-trip parse tests on representative rows. Adding a new column to a covered table fails the field-set test until the refinements + DTO are updated.

- Only the migrated tables have derived schemas today (projects, runs, schedules, notifications). Add more by importing the table, listing refinements for any column whose Zod type the SQL type alone can't express, and extending the test's `ENTRIES` table.

## Common Mistakes

- **Catching `AppError` and manually replying** — duplicates the global handler. Just throw.

- **Importing from `apps/*`** — violates the dependency boundary. This package must be app-agnostic.

- **Hardcoding `/api/v1`** — use the `routePrefix` from plugin registration. Base path support requires this.

- **Forgetting to register new route file in `index.ts`** — the routes won't be mounted.

- **Hand-constructing error JSON** — always use factory functions (`validationError()`, `notFound()`, etc.).

- **Doing async I/O inside transactions** — SQLite transactions must be synchronous.

## See Also

- `docs/architecture.md` — system overview and data flow

- `packages/contracts/` — DTOs, error codes, Zod schemas

- `packages/db/` — database schema and migration patterns