AGENTS.md git:20260928.154bb42A

Immutable. This exact content never changes and is served at /api/v1/blob/7f5462abde986cbe.

# AGENTS.md

Guidance for AI coding agents (Claude Code, Codex, Cursor, Copilot, and others) working in this repository. This is the single source of truth: `CLAUDE.md`, `.github/copilot-instructions.md` and `.cursor/rules/` all point here, so edit this file rather than duplicating its content.

## Build, test, run

**JDK 22 + Maven 3.9+ are required to build the full reactor.** JDK 22 is needed only because `saiku-proptest` (Hegel property tests) uses the Foreign Function & Memory API + a bundled native engine, which is Java-22 bytecode. **Main artifacts still compile at `release 21`, so the shipped runtime (Docker image, launcher fat-JAR) is unchanged and still runs on JRE 21+.** To build without JDK 22, skip the module: `mvn -pl '!saiku-proptest' verify`. The reactor is `saiku-bom → saiku-core → saiku-webapp → saiku-launcher → saiku-proptest`. `saiku-ui` is a separate SvelteKit app and is **not** a reactor module — but `mvn verify` still builds it, via `saiku-webapp`'s frontend plugin (see *Testing baseline*), so a cold build downloads a Node toolchain.

**On Windows, set `git config core.autocrlf false` and re-clone before building.** Git for Windows defaults `core.autocrlf=true`, and the repo has no `.gitattributes` to override it, so checkout rewrites every LF blob to CRLF and `spotless:check` then rejects every Java file — before anything compiles. CI never catches this because CI is Linux-only.

```bash
mvn verify                                    # compile + unit tests + spotless:check (CI's gate)
mvn spotless:apply                            # auto-format Java (Palantir Java Format, bound in root pom)
mvn -pl saiku-core/saiku-service -am test     # one module's tests
mvn -pl saiku-core/saiku-service test -Dtest=RepositoryDatasourceManagerTest  # single test class
mvn -pl saiku-webapp,saiku-launcher clean                     # WIPE stale UI bundles before rebuilding — CRITICAL for UI changes
mvn -pl saiku-launcher -am -Dmaven.test.skip=true package    # build the runnable fat-JAR
mvn -P security verify                        # OWASP dependency-check (opt-in)
./scripts/install-hooks.sh                    # one-time: install pre-commit spotless hook
```

Run the launcher fat-JAR (Picocli + embedded Jetty 12 EE10):

```bash
java -jar saiku-launcher/target/saiku-<version>.jar serve --port 8080 --home ./saiku-home
# <version> is the root pom's <version> — `mvn -q -DforceStdout help:evaluate -Dexpression=project.version`
# UI:    http://localhost:8080/ui/
# REST:  http://localhost:8080/saiku/api/...
# Login: admin / admin
```

For the SvelteKit UI (`saiku-ui/`, version-tracked independently as 3.17.0):

```bash
cd saiku-ui && npm install && npm run dev     # vite dev server
npm run check                                 # svelte-check + tsc
npm test                                      # vitest
npm run lint                                  # ESLint flat config (token-only rule)
npm run storybook                             # Storybook 10.4 design-system catalogue
npm run build                                 # static build → saiku-ui/dist
```

The UI is **Tailwind v4 + design-system primitives + Storybook**. ESLint bans raw tone classes (`bg-emerald-*`, `text-red-*`, `bg-amber-*`, `rose`, `orange`) outside `src/lib/design-system/` — token utilities only.

**The design system is a shared, published package — `saiku-ui/design-system/`, an npm workspace of `saiku-ui`, published as `@concepttocloud/saiku-design-system` (Apache-2.0).** It carries **its own version** and is published by its **own workflow** (`.github/workflows/design-system.yml`, triggered by changes under `saiku-ui/design-system/**`), NOT by `release.yml` — it moves on its own cadence so saiku-cloud isn't waiting on a product release for a token change. Bump `version` in the package's `package.json` and merge to release; the workflow checks the registry first and no-ops if that version is already out. It owns the token layer (`tokens.css`), the Tailwind `@theme inline` bridge (`tailwind.css`), the shadcn-style primitives (Button/Card/Input/Tooltip) and the compounds (PageHeader, FeedbackBanner, Badge, EmptyState, SectionCard, Toast, FormField, SortableColumnHeader). **The Saiku Cloud dashboard consumes the same package from npm**, so a change here ships to both products — that's the point, and it's also the hazard. `src/lib/design-system/` and `src/lib/components/ui/` are now thin barrels re-exporting the package plus the Storybook stories; import from the barrels (`$lib/design-system`, `$lib/components/ui`), never deep paths.

It is a **workspace, not a `file:` dep, and that's load-bearing** — a `file:` dep gives the package its own nested `node_modules/svelte`, and two Svelte installs make `Snippet` from the package structurally incompatible with `Snippet` in the app. The symptom is a wave of "Two different types with this name exist, but they are unrelated" errors in files that have nothing to do with the design system. `npm ci` runs the package's `prepare` (svelte-package) automatically, so `dist/` exists before the app builds; after editing package source run `npm run build --workspace @concepttocloud/saiku-design-system`.

The package's `tailwind.css` deliberately **omits preflight** (saiku-ui's legacy base CSS in `app.css` depends on browser defaults). saiku-cloud, which has always run with preflight, opts back in explicitly. Don't "fix" this by adding preflight to the package.

**Cascade-layer discipline (load-bearing):** every type-selector rule in `app.css` MUST live in `@layer base` so Tailwind utilities (in `@layer utilities`) can override them. Unlayered rules win against ALL layered rules regardless of specificity — that's the CSS cascade-layers spec, and the saiku-ui legacy `a { color: var(--accent) }` ate `text-primary-foreground` on every anchor-as-button until the wrap landed. Only the `*, *::before, *::after { box-sizing }` reset and the `html, body { height/margin }` root zeroing stay unlayered.

**Stale-bundle gotcha with the launcher fat-JAR (bit us twice):** `saiku-webapp`'s maven-war-plugin overlays `saiku-ui/dist/` into `webapp/saiku.war` inside the launcher fat-JAR, but it does NOT purge stale files from `target/`. SvelteKit generates hashed filenames like `_app/immutable/nodes/2.<hash>.js` — every `npm run build` produces new hashes, and every `mvn package` that skips `clean` leaves the old hashes AND the new hashes both in the war. The server's `index.html` points at the new hashes; the old bundle files sit there dead. But if you're debugging via `unzip -l` you can miss which one is live. Fix: `mvn -pl saiku-webapp,saiku-launcher clean` before any UI rebuild — never trust an incremental package on top of an existing `target/` when saiku-ui bundles have changed.

**Never `mvn clean` while the launcher is running:** the launcher's JVM keeps an open file handle to `saiku-launcher/target/saiku-4.6.0.jar` (that's the fat-JAR it was launched from). `mvn clean` wipes the directory, so the file handle stays open (POSIX unlink lets you keep reading) but any path-based operation on that jar starts returning `NoSuchFileException`. Symptom: Mondrian's Calcite backend loads a Rolap schema, tries to JIT-compile a metadata provider via Janino's `ServiceLoader`, the ServiceLoader tries to open `jar:file:.../saiku-4.6.0.jar!/META-INF/services/org.codehaus.commons.compiler.ICompilerFactory` — and hits `NoSuchFileException`. That surfaces as `ServiceConfigurationError: Error accessing configuration file` and every discover/query on an MDX cube returns HTTP 500. The Ossie discover path is unaffected because it never touches Mondrian. Fix: stop the launcher BEFORE `mvn clean`, or accept that after clean you must rebuild AND restart. Don't try to `mvn clean package` mid-session and expect the running JVM to keep working.

Live AI Query API regression suite against a running launcher: `saiku-launcher/test-ai-live.sh` (expects `./run.sh` already started; auth defaults `admin/admin`).

## GitHub Packages auth (non-obvious gotcha)

Saiku depends on Spicule-published artifacts hosted on **GitHub Packages, which requires authentication for downloads even when public**. The root `pom.xml` registers five repos (`github-mondrian-saiku`, `github-olap4j`, `github-olap4j-xmlaserver`, `github-saiku-query`, `github-ossie`); each needs a matching `<server>` entry in `~/.m2/settings.xml` with a GitHub username + PAT. One PAT covers all five — mirror the five `<server>` blocks CI writes in `.github/workflows/ci.yml`. CI uses `secrets.GH_PACKAGES_TOKEN`; the auto-issued `GITHUB_TOKEN` will NOT work cross-org. Local forks of `mondrian-saiku`, `olap4j`, `olap4j-xmlaserver`, `saiku-query` installed in `~/.m2` will be preferred over the remote.

The PAT must be a **classic** token with the `read:packages` scope — [GitHub only supports classic PATs for the Maven registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry), so a fine-grained token 401s. All five source repos are public, so `read:packages` alone suffices; no `repo` scope. **A fine-grained token and a classic token missing `read:packages` produce an identical bare `401 Unauthorized` that never mentions scopes** — so diagnose before re-minting: `curl -sI -H "Authorization: token $PAT" https://api.github.com/user | grep -i x-oauth-scopes`. HTTP 200 with a scope list lacking `read:packages` means the token is valid but under-scoped; classic PAT scopes are editable in place (Settings → Developer settings → Tokens (classic) → tick `read:packages` → Update token) and the value doesn't change, so `settings.xml` needs no edit.

`lib/repo/` is an in-tree file-based Maven repo for vendored deps that aren't reachable on any public remote (currently `miredot-annotations`, Bintray-only). Use `${maven.multiModuleProjectDirectory}` (not `${project.basedir}`) when adding new entries so child modules inherit a URL pointing at the reactor root.

## Architecture

**Multi-module Maven reactor** (all artifacts `org.saikuanalytics:*`, sharing the root pom's `<version>`):

- `saiku-bom` — central dependency catalogue. Versions live here (Spring 6.2.x, Spring Security 6.5.x, slf4j 2.0.16, log4j 2.24.3, Jackson 2.18.6, Jersey 3.1.9, Arrow 18, Caffeine 3.1.8, mondrian fork 4.8.1.x). Child poms reference `${...}` and must `<import>` the BOM via `dependencyManagement`. The root pom shadows the same versions defensively so a child that forgets to import the BOM doesn't silently inherit stale numbers — keep them in sync.
- `saiku-core/saiku-olap-util` — olap4j helpers.
- `saiku-core/saiku-service` — business logic: OLAP queries (`OlapQueryService`, `ThinQueryService`), schema discovery, drill-through, async, totals, AI plan/translate (`olap/ai/`), schema generation (`schema/generate/`), datasource/session/user/cache services.
- `saiku-core/saiku-semantic` — YAML semantic layer (Phase 3 work).
- `saiku-core/saiku-web` — JAX-RS (Jersey 3.1) REST resources under `org.saiku.web.rest.resources.*` (`AiQueryResource`, `Query2Resource`, `OlapDiscoverResource`, `BasicRepositoryResource2`, `DataSourceResource`, `ExporterResource`, `AdminResource`, `InfoResource`, `SessionResource`, `StatisticsResource`, `schemagen/`). REST mount is `/saiku/api/*`. Co-located markdown (`AiQueryPlan.md`) documents agent-facing contracts alongside the code.
- `saiku-webapp` — packages the Servlet web app (`packaging=war`). Spring 6 wiring lives in `src/main/webapp/WEB-INF/applicationContext*.xml` + `spring-rest.xml` + `spring-servlet.xml`. Auth is Spring Security 6.5 (memory-backed for the embedded build via `applicationContext-spring-security-memory.xml`; `users.properties` seeds admin/admin).
- `saiku-launcher` — runnable distribution. Picocli CLI (`SaikuLauncher.ServeCommand`) + embedded **Jetty 12 EE10** WebApp serving the saiku-webapp WAR. Shade-plugin produces `saiku-<version>.jar` containing the WAR under `webapp/saiku.war`, the H2 FoodMart seed SQL, and the Mondrian schema template `seed/foodmart.sds.template`. On first run it materialises `saiku-home/` (data, repository, sessions, logs, plugins) next to the JAR. Sessions persist via Jetty `FileSessionDataStore`.

`saiku-ui` (SvelteKit 2 + Svelte 5 + Vite 6 + monaco + ECharts + apache-arrow) is built independently and served as static assets at `/ui/` — the launcher serves the Maven-built `dist/` from inside the JAR.

**Mondrian backend chain** (see `docs/mondrian-fork.md`): Saiku uses the Spicule fork `pentaho:mondrian:4.8.1.x` which ships a Calcite-based SQL planner alongside the original SqlQuery builder. **Calcite is the default**; force legacy with `-Dmondrian.backend=legacy`. Native Calcite dialect mappings live in `mondrian.calcite.CalciteDialectMap` (H2, HSQLDB, MSSQL, MySQL/MariaDB, Oracle, PostgreSQL); unknown DBs fall through to Calcite's `SqlDialectFactoryImpl` and on failure to legacy with a one-shot WARN. Un-excluding Calcite from `saiku-webapp/pom.xml` pulls in `calcite-core:1.41`, `avatica-core:1.27`, `guava:33.4.8-jre` (pinned — Calcite needs a newer API than Mondrian's historical 18.0), `protobuf-java:3.25.8` (pinned — beats the 2.4.1 transitive from `serenity-bdd → operadriver`).

**AI Query API** (`/saiku/api/ai/*`, see `docs/AI-QUERY-API.md`): typed REST surface so agents query cubes **without ever seeing MDX**. Three core endpoints: `GET /ai/cubes`, `GET /ai/schema/{connection/catalog/schema/cube}` (self-describing — measures, dims, sample members with unique names, JSON Schema, example bodies), `POST /ai/query` (records or matrix output, typed `{value, formatted, unit}` cells). Server validates names against the live cube and returns `VALIDATION_ERROR` 400s with `field` + `available` candidate lists for self-correction. Phase 3 enrichment overlays display-name renames from `<datasource>.generated.json` and accepts either canonical or display names in any field.

**Agent Skills** (`saiku-home/skills/*.md`, see `docs/SKILLS-SPEC.md`, saiku#1426): admin-authored markdown workflows with YAML frontmatter, discoverable from `/ai/ask` and `/ai/ossie/ask`. Scanned lazily on mtime signature; parse errors surface via `GET /ai/skills?errors=true` with stable codes (`INVALID_NAME`, `DUPLICATE_NAME`, etc). Invocation: prefix an ask with `/<skill-name>` (body expanded verbatim) OR ask naturally and let the LLM route via the catalogue in its system prompt. Registry wired in `saiku-beans.xml` (`agentSkillRegistryBean`, root `${saiku.home}/skills`), consumed by `AiAskService` via `setSkills()`. Fresh installs stage `weekly-foodmart-rollup.md` from `saiku-launcher/src/main/resources/seed/skills/` on first boot.

**Agent Spaces** (`saiku-home/agent-spaces/*.json`, see `docs/AGENT-SPACES-SPEC.md`, saiku#1440): named admin-authored personas that scope an ask. Each bundles a `systemPrompt`, `cubeAllowlist`, `skillAllowlist`, and `suggestedPrompts` list. `POST /ai/spaces/{id}/ask` enforces the persona server-side: cube refs outside the allowlist return 403 FORBIDDEN, the system prompt is prepended to the built-in `SYSTEM_PROMPT`, and the skill catalogue seen by the LLM is filtered to the space's allowlist. Users can't override the prompt via `history` or `question`. Registry wired in `saiku-beans.xml` (`agentSpaceRegistryBean`, root `${saiku.home}/agent-spaces`), consumed by `AiAskService` via `setSpaces()`. Fresh installs stage two personas — FoodMart Sales Analyst + FoodMart Finance Ops — from `saiku-launcher/src/main/resources/seed/agent-spaces/`.

**Ossie AI surface** (`/saiku/api/ai/ossie/*`, see `docs/AI-OSSIE-API.md`): SQL-side equivalent of the MDX AI Query API. Same shape (list → schema → query, `VALIDATION_ERROR` self-correction, `POST /ask` for NL), same MCP tools alongside the MDX ones. Uses `bi.saiku.ossie:ossie-core` semantic YAML — datasets/fields/metrics with `ai_context.synonyms` for aliases and `custom_extensions` (`saiku.display`, `saiku.roles`) for enrichment. Ontology block exposed via `GET /ai/ossie/ontology/{c}/{m}` + MCP `describe_ossie_ontology`. Anomaly / forecast endpoints (typed REST at `/ai/anomaly`, `/ai/forecast`) share the AI Query MDX contract — same envelope, same validation.

**Observability** (see `docs/observability.md`): opt-in OpenTelemetry via the OTel Java agent (`io.opentelemetry.javaagent:opentelemetry-javaagent`), vendored into the Docker image at `/opt/saiku/otel/opentelemetry-javaagent.jar`. Activated only when `OTEL_EXPORTER_OTLP_ENDPOINT` is set — the entrypoint (`docker/saiku-entrypoint`) and dist wrappers (`dist/run.sh`, `dist/run.bat`) conditionally append `-javaagent` based on the env var. With the var unset, the agent is never loaded (zero overhead). Auto-instruments Jetty, Jersey, JDBC (every Mondrian SQL emission), `java.net.http.HttpClient`, Log4j 2 (trace_id/span_id in MDC), JVM metrics, DBCP2 pool stats. `log4j2.xml` PatternLayout uses `%notEmpty{[trace_id=%X{trace_id} span_id=%X{span_id}] }` so trace brackets vanish cleanly when the agent isn't attached. Custom spans (Tier 2 — `ThinQueryService.execute`, AI Query metrics, cellset cache) are out of scope until Tier 1 produces data telling us which spans matter.

## Testing baseline

**`TESTING.md` is the source of truth — read it rather than trusting any count restated here.** This section is deliberately thin: it previously carried a duplicated snapshot that rotted badly (it claimed 10 tests total and "zero coverage" for `saiku-core/saiku-web`, a module that by then ran 500 green tests behind a CI floor of 330). A number copied next to its source will drift from it; get current figures by running the suite, not by reading this paragraph. For orientation only: `mvn -B -ntp -DskipITs=false verify` was ~1,480 tests green at 4.6.1, across `saiku-service`, `saiku-web`, `saiku-launcher` (plus failsafe ITs), `saiku-semantic` and `saiku-sql`.

**Test floors are a hard gate.** `.github/test-floors.json` fails CI when a module's surefire total drops below its declared floor (`saiku-core/saiku-service` and `saiku-core/saiku-web` are floored today). Bump a floor explicitly when you add tests; never delete tests to get green.

CI (`.github/workflows/ci.yml`) runs `mvn -B -ntp -DskipITs=false verify` on **ubuntu-latest** against JDK 22 — Linux only, no macOS runner. (All build workflows moved 21→22 for `saiku-proptest`; artifacts still target release 21.) A separate path-filtered `ui` job runs `npm run check` + vitest on Node 20. The `dist` job uploads `saiku-dist-<version>.zip` (launcher fat-JAR + `dist/run.sh` + `dist/run.bat` + `dist/README.md`) as an artifact. Branch protection requires only the aggregate `ci` check, which passes when upstream jobs succeed *or* are legitimately path-skipped.

**`mvn verify` builds `saiku-ui` but does not test it.** `saiku-ui` is not a reactor module, yet `saiku-webapp`'s `frontend-maven-plugin` runs `install-node-and-npm` → `npm ci` → `npm run build` so the war overlay gets a bundle. Expect a Node toolchain download and a long `saiku-webapp` phase on a cold build (~34 min observed). UI tests run only via the `ui` CI job or `npm test` inside `saiku-ui/`.

## Conventions specific to this repo

- **Java format**: Palantir Java Format via Spotless. `mvn spotless:apply` before committing or the pre-commit hook will reject. The check is bound to the `verify` phase — `mvn spotless:check` standalone fails on `saiku-bom` (pom-only, no spotless binding).
- **Spring XML wiring, not JavaConfig**: the webapp wires beans through `applicationContext-*.xml` files. New beans go there, not via `@Configuration`.
- **JAX-RS over Spring MVC**: REST resources are Jersey 3.1 (`jakarta.ws.rs.*`), not Spring `@RestController`. The mount point is configured in `spring-rest.xml`.
- **Reactor compiler pinned to JDK 6 source in root pom** but `saiku-launcher` overrides to `<release>21</release>`. Don't "fix" the root pom — modules that need 21 set it locally. `saiku-proptest` follows the same pattern with `<configuration combine.self="override"><release>22</release>` (Hegel needs 22); the `combine.self="override"` is load-bearing — without it the inherited `source/target 1.6` collides with the bare `<release>` and the compiler plugin errors.
- **`saiku-proptest`** is a test-only module (no main sources, never published) holding Hegel property-based tests against real Saiku code (`JdbcUrlValidator`, `CryptoUtil`, …). It depends on `saiku-web`/`saiku-service` in test scope, needs JDK 22, and its surefire sets `--enable-native-access=ALL-UNNAMED` for Hegel's native engine. See `saiku-proptest/README.md`.
- **Commit message format**: `#<issue> - <description>` per `CONTRIBUTING.md`. Default branch is `development`; PRs go there, not `master`. The repo uses `jgitflow` (`mvn jgitflow:feature-start`, `release-start`, `hotfix-start`).
- **Branching strategy: Gitflow.** All new work goes on a `feature/<name>` branch off `development`. Hotfixes branch off `main` as `hotfix/<name>` and merge back to both `main` and `development`. Release prep happens on `release/<version>` branches off `development`, which then merge to `main` (tagged) and back to `development`. Never push directly to `main` or `development` — always PR. Chores and refactors that don't fit `feature/` use `chore/<name>`. Stick to Gitflow conventions; the `mvn jgitflow:*` commands enforce them automatically when used.
- **Planning docs**: `docs/plans/` holds the Phase 1–7 modernisation plans (build hygiene, FS repo, YAML semantic layer, SvelteKit port, Arrow IPC / async cache, full Svelte rewrite, platform features). The Phase 0 dependency audit is at `docs/phase-0/dependency-audit.md`. Read these before invasive changes.
- **Svelte 5 effect discipline**: never call a state-writing helper synchronously from inside `$effect` if the helper also reads the same `$state`. The effect's dep-tracking captures the read, the write inside the same tick re-queues the effect, and Svelte bails after 128 iterations with `effect_update_depth_exceeded` — visible to the user as a stuck UI state AND every event handler in the component going inert (the whole reactivity graph tears down). Defer with `queueMicrotask(() => fn(...))` or wrap in `untrack(() => ...)` from `svelte`. Diagnostic signal: if multiple buttons in one modal/panel stop working at once, check the browser console for that error code before debugging the buttons individually.
- **Mondrian 4 virtual cubes**: when adding a MeasureGroup to a virtual cube, declare `<NoLink dimension="X"/>` for every dim that doesn't apply — Mondrian doesn't infer it, the cube fails to load with "No link for dimension X in measure group Y". The **canonical, git-tracked FoodMart schema is `saiku-launcher/src/main/resources/seed/FoodMart4.xml`** — edit that. The runtime copy `saiku-home/data/FoodMart4.xml` is under the gitignored `saiku-home/` and is only ever *seeded-if-absent* (`stageSeedAssets` never overwrites an existing home to preserve user edits), so an **established local home never picks up seed changes** — that's how a dev home drifted 2,000+ lines behind (saiku#1662). After editing the seed, re-materialise every local home you test against: delete `saiku-home/data/FoodMart4.xml` (or `cp` the seed over it) and relaunch. The launcher now prints a boot-time WARNING when the two diverge.

## Agent resources

- **Skills** — `.claude/skills/<name>/SKILL.md`: step-by-step workflows for recurring repo tasks (rebuilding the launcher after UI changes, cutting a release). Claude Code loads them automatically; other agents can read them as plain markdown.
- **Prompts** — `.github/prompts/*.prompt.md`: reusable task prompts (fix a bug test-first, add an AI Query API field). Copilot exposes them as `/` commands; for other tools, paste or reference the file.
- **Corrections log** — `.claude/memory/corrections.md`: conventions a human had to correct an agent on. Read it before starting work. When a maintainer corrects you on something this file doesn't already cover, append an entry (format at the top of that file) in the same PR. If a correction is general enough, promote it into this file instead and delete the entry.