arch-scaffold-new ยท diff
git:20260922.28cefc1 to git:20260922.a3c1e91
68 added, 168 removed. Audit B to A.
---
name: arch-scaffold-new
description: "Bootstrap a whole new system in the guideline's shape into an empty folder: the monorepo skeleton, the first API, a worker, a portal, deployment, CI, then the first namespace and entity."
allowed-tools: Read, Grep, Glob, Write, Edit, Agent, Bash(make setup), Bash(make check), Bash(make infra-up), Bash(make migrate), Bash(make migrate-check), Bash(make seed), Bash(make test-integration), Bash(make openapi), Bash(make devx-up), Bash(make test-telemetry), Bash(make traffic PROFILE=light DURATION=30), Bash(uv sync:*), Bash(uv run:*), Bash(pnpm install:*), Bash(pnpm run:*), Bash(pnpm --filter:*), Bash(git init:*), Bash(git status:*), Bash(git rev-parse:*), Bash(python3:*), Bash(git diff:*), Bash(git log:*), Bash(git merge-base:*), Bash(git symbolic-ref:*)
---
# arch-scaffold-new
Conventions: `${CLAUDE_SKILL_DIR}/../_shared/scaffold-conventions.md`.
Sections of `${CLAUDE_SKILL_DIR}/../../architecture.md`: Naming
- Entities, OpContext (Stages, Scopes, The Operator Context), The Storage
+ Entities, Namespaces as Swimlanes, OpContext (Stages, Scopes, The
+ Operator Context), The Business Layer (Operations Without a Principal,
+ Shape of an Operation), The Storage
Layer (Namespace Shape, Storage Root, Defining ORM Classes,
Translation, A Storage Impl, Database Roles, The Second Fence,
- Migrations), Infrastructure (InfraInterface Root), The Network Layer
+ Migrations), Infrastructure (InfraInterface Root, Cache, Buckets,
+ Topics, Queues, Secrets, Idempotency), The Network Layer
(The Gateway; Auth: the Gateway Verifies, the Tenancy Domain Owns;
Realtime at the Edge), Deployment (Cloud: AWS, Infrastructure as Code,
Local: Docker Compose, Twins for External Services, What a Process
- Refuses), Operations, Monorepo Folder Structure (Layout Conventions),
- Documentation as Code, Telemetry, Cross-Cutting Conventions
+ Refuses), Operations (Operator Credentials, Operational Skills,
+ Dashboards and Alarms as Code, Traffic and Stress, The Telemetry Round
+ Trip), Monorepo Folder Structure (Layout Conventions),
+ Documentation as Code (A README at Every Level), Telemetry,
+ Cross-Cutting Conventions
(Exceptions, Configuration, Records of Decisions, Tests), Technology
Choices and How to Override Them (Versions, Overriding a Choice).
## Input
`<target-dir> <root-package> [--first <namespace> <Entity> [field:type ...]] [--codeowners <owner,...>] [--no-portal] [--no-worker]`
Example: `./acme acme --first inventory Warehouse address:str`. Both
positional arguments are required; ask for them when missing.
`<target-dir>` must not exist, or must be empty, or be a fresh
repository holding nothing but `.git`, `README.md`, `LICENSE`, and
`.gitignore` (the shape a hosting service creates); refuse otherwise.
In the fresh-repository case `README.md` and `.gitignore` are replaced,
`LICENSE` is kept, and the `git init` of step 1 is skipped. Those two
replacements are the one exception to the collision rule of the
conventions; any other path that exists is a collision. Refuse when a `.git`
directory exists in a parent of `<target-dir>` (`git rev-parse
--show-toplevel` from it names one), because `git init` never runs
inside an existing repository.
`<root-package>` must not shadow a standard-library module. In this
skill `<root>` is `<root-package>`, and `<root-slug>` its kebab-case
slug, as the conventions' Naming derives it (`acme_corp` gives
`acme-corp`). `--codeowners` names the owners `.github/CODEOWNERS`
lists, the repository's owner by default (the account or organization
in the `origin` URL of `.git/config`; ask when there is none).
## Created
- Everything below is under `<target-dir>/`.
-
- Skeleton:
-
- | File | Holds |
- |---------------------------------------------------|-------------------------------------------------------------------------------------------|
- | `pyproject.toml` | `[tool.uv] package = false`, `[tool.uv.workspace] members` (with a comment citing the root-package ADR by number), `[tool.uv.sources]`, pytest markers `integration`, `e2e`, `slow`, `telemetry` (needs the `devx` profile), and `[tool.arch-check]` with `package = "<root-package>"` and no other key; below it the two option tables the tree's own names need, `[tool.arch-check.options.CTX-26] sites` and `[tool.arch-check.options.CTX-12] tenantless`, which the `tests/unit/` row fills, and no other option; `sites` names each transition by method, `om/src/<root-package>/om/tenancy/impl/manager.py::TenancyManagerImpl.<method>`, one entry per method that constructs a stage, since an entry that constructs nothing is itself a finding |
- | `ruff.toml`, `pyrightconfig.json`, `.python-version` | shared Python lint, format, and type config; `.python-version` and `requires-python` name the latest stable Python release that has a patch release behind it, never one published today, as Technology Choices and How to Override Them (Versions) states |
- | `package.json`, `pnpm-workspace.yaml`, `.nvmrc`, `tsconfig.base.json`, `eslint.config.js` (unless `--no-portal`) | the pnpm workspace over `apps/*` and `clients/*`, and the TypeScript and lint config every member extends; `.nvmrc` names the current active Node LTS release and `packageManager` the latest stable pnpm |
- | `.gitignore`, `.env.example` | ignores; every setting knob documented with its prefix, the `devx` dashboard ports included, the error-tracking DSN pointing at the seeded local GlitchTip key, the three database URLs of the local logins, `<ROOT>_DATABASE_MIGRATION_URL`, `<ROOT>_DATABASE_URL` for the runtime login, and `<ROOT>_DATABASE_SYSTEM_URL`, with the local passwords the compose file's init script creates them with, and every other field of `StorageSettings` (the per-role URLs empty, `<ROOT>_DATABASE_MASTER_URL` empty, since the init script is the local master), and the development seed, each variable under the product prefix (`<ROOT>_SEED_ORG_NAME`, `<ROOT>_SEED_ORG_SLUG`, `<ROOT>_SEED_OWNER_EMAIL=owner@<root-package>.example`, `<ROOT>_SEED_OWNER_PASSWORD=pswd_1234`, and the two local operator identities, `<ROOT>_SEED_OPERATOR_EMAIL=operator@<root-package>.example` with `<ROOT>_SEED_OPERATOR_PASSWORD` on a `read` entry and `<ROOT>_SEED_PROVISIONER_EMAIL=provisioner@<root-package>.example` with `<ROOT>_SEED_PROVISIONER_PASSWORD` on a `write` entry; no TOTP secret and no operator token: the seed enrols no second factor, and `make seed` mints the two local tokens) |
- | `Makefile` | every target the developer and the gate run, self-documented; the full list is below the table |
- | `README.md` | how to set up, run, and check, a quick start (`make up`, the dependencies in containers and the application on the host, with `make down`, `make reset`, and `make urls` beside it, and the steps it wraps for running one at a time: `make setup`, `make infra-up`, `make migrate`, `make seed`, `scripts/dev.sh`) followed by the seeded sign-in (org, owner email, password, all development-only, and local only: a deployed environment is entered through sign-up), and a `Local URLs` table (`http://localhost:<port>`, the port from `.env.example`) listing every `devx` dashboard; the service and app scaffolds of steps 2 and 5 add their rows |
- | `docs/architecture.md` | a one-page "as built" stub linking to the guideline |
- | `specs/architecture.md` | the pointer to the guideline pinned at the release found before writing (the `version` in `plugin.json`), with empty `Substitutions` and `Deviations` tables, as the guideline's adopting guide (`docs/adopting.md` next to it) shows |
- | `docs/adr/0001-root-package.md` | the root package decision |
- | `.github/CODEOWNERS` | the owners `--codeowners` names as the code owners of `deployment/` and `.github/`, whose review the `main` ruleset requires, since a merge to `main` is a staging deploy under a role that writes infrastructure |
- | `docs/adr/0002-technology-choices.md` | the stack as adopted: every technology the guideline names, and per substitution the substitute, the reason, and the rules it must still satisfy |
- | `docs/runbooks/README.md` | where runbooks go |
- | `docs/runbooks/tenant-isolation.md` | the negative control of Cross-Cutting Conventions (Tests): how it is run (the predicate taken out of one query, the suite run with the policy in place and again with it off for that table), and the record of the last run, written by step 8: the query, the table, and what the suite reported each time |
- | `.github/workflows/ci.yml`, `deploy-staging.yml`, `deploy-production.yml`, `release.yml`, `grant-operator.yml`, `state-unlock.yml` | the fast gate and the integration job, the staging and production deploys, the release, the first-operator grant, and the state unlock; the full list is below the table |
- | `deployment/local/docker-compose.yml` | Postgres, with an init script under `deployment/local/postgres/` that creates the three logins of The Storage Layer (The Second Fence) with the local passwords `.env.example` names, under the names every environment uses, `<root>_migration`, `<root>_runtime`, and `<root>_system`, since a policy spells the system login's name: the migration login, granted `CREATE` on the database so it creates and owns the role schemas, the runtime login and the system login, none a superuser and none with `BYPASSRLS`; the grants on the schemas are the migrations' own, below, so the cloud and a role that arrives later get the same; Valkey as the cache, a queue, an object store, each image tagged at its latest stable release; host ports read from `.env` with non-default values, so a second project on the same machine does not collide; a `devx` profile with pgweb, Valkey Admin, the object store's and the queue's consoles where their local images ship one, Jaeger for traces, GlitchTip for errors (seeded with a fixed project key so the local DSN in `.env.example` works without its UI), and the metrics view, on ports from `.env` too |
- | `deployment/local/docker-compose.full.yml` | the same plus the application containers, for the case that asks for them and never the default |
- | `deployment/docker/entrypoint.sh` | the shared image entrypoint |
- | `deployment/terraform/modules/`, `deployment/terraform/bootstrap/{staging,prod}/`, `deployment/terraform/environments/{staging,prod}/`, `deployment/cloud/environments.json` | the modules, the two bootstrap roots, the two environment roots, and the environments' file; the full list is below the table |
- | `scripts/dev.sh` | starts every application process on the host |
- | `scripts/cloud_create.sh`, `scripts/cloud_nuke.sh` | the hands of the create and nuke skills; the full list is below the table |
- | `.claude/skills/<name>/SKILL.md`, one per operational skill: `ops-investigate`, `ops-watch`, `ops-root-cause`, `ops-infra-as-code`, `ops-cloud-deployment-create`, `ops-cloud-deployment-nuke`, `ops-simulate-traffic`, `stress-test-create-or-update`, `stress-test-run` | copied from `${CLAUDE_SKILL_DIR}/../_shared/ops-skills/<name>.md` with the product's name substituted as the conventions state (`acme-ops`, the binary, becomes `<root>-ops`; every other `acme` in a hyphenated name, `acme-<env>-investigate` or `acme-api`, or in a log group, `/acme/<env>/api`, becomes `<root-slug>`; every other `acme` becomes `<root>`, `ACME` its upper case, and `Acme` its CamelCase form); the full list is below the table |
- | `clients/python/` | the Python client, `<root>-client`, in the shape `arch-scaffold-app` defines it, generated from the OpenAPI document step 3's `make openapi` emits, whether or not `--no-portal`, so its operator-plane calls exist before `ops/` is written; the ops package and a remote service impl import it, and the app skill finds it present and skips its row |
- | `ops/` | the workspace member `<root>-ops`; the full list is below the table |
- | `ops/README.md`, `ops/stress/README.md` | how the platform is operated: the roles and profiles, the env file, the nine skills and what each needs, the generator and its profiles; and, under `stress/`, what a scenario holds (profile, duration, ramp, soak, the target p95 and error ratio, the weighted session steps) and that the numbers are the team's |
- | `om/README.md` | the nouns of the object model and how they relate, written for a reader with no code: what a tenant is, who a user and a member are, what the product's entities are and which belong to which, in the product's language, with a link one level down to each namespace's README; no developer instruction and no operator instruction in it |
- | `deployment/README.md` | how it runs: the local stack and its profiles, the two environments and their base domains, the pipeline (a merge deploys staging, a fast-forward to `release` plans production behind an approval), the dashboard and the alarm topic, the two switches, the budget |
- | `llms.txt` | the knowledge map at the root: one section per audience (platform developers, platform operators, tenant users and admins), each listing the documents that audience is served, one link per document; exposure is by listing, never by folder, and the product's language, not the team's |
-
- `Makefile`:
-
- - `setup` (`uv sync`, and `pnpm install` when a `package.json` exists), `infra-up`, `infra-down`, `infra-reset` (the dependencies recreated with their volumes removed and nothing else started, the step a gate and `arch-upgrade-deps` take where a developer takes `reset`), `migrate` (every role, `--all`, as the migration login), `migrate-check` (ORM metadata against the migrated schema, per role), `migrate-roundtrip` (downgrade the latest revision of every role, then upgrade it), `lint` (`ruff check`), `format-check` (`ruff format --check`), `typecheck` (`pyright`), `arch-check` (the guideline's static checker, `ARCH_CHECK ?= uvx --python "$(shell cat .python-version)" --from "git+https://github.com/baristaze/swe_guidelines@v<version>\#subdirectory=checkers" arch-check` at the release found before writing, the same tag `specs/architecture.md` pins; `?=` so a developer offline points it at a local checkout), `check` (`lint`, `format-check`, `typecheck`, `arch-check`, `test-unit`), `test-unit`, `test-integration`, `openapi` (the API's document, then `clients/python/` regenerated from it once it exists), `devx-up` (the stack plus the `devx` profile; `infra-down` stops both), `seed` (the API process's `bootstrap --seed`, the org, its owner, the local read operator, and the local provisioner; then, when absent, the owner-only env file `~/.config/<root>/ops/local.env`, its folder `0700` and the file `0600`, written by the recipe from `.env`: `<ROOT>_API_URL` at the local API, the read operator's token as `<ROOT>_OPERATOR_TOKEN` and the provisioner's as `<ROOT>_PROVISIONER_TOKEN`, each minted by `uv run <root>-api grant-operator --email <address> --mint-token`, which prints a token on its standard output only against a local database, captured by the recipe and never echoed, and no password and no TOTP secret, since an agent never signs in with a password; a token expires within the hour, and `uv run <root>-ops token --env local --identity operator\|provisioner` mints a fresh one into the file the same way, the local GlitchTip as `<ROOT>_ERROR_TRACKER_URL` with the seeded `<ROOT>_ERROR_TRACKER_TOKEN`, and the twins `<ROOT>_PROMETHEUS_URL` and `<ROOT>_JAEGER_URL`, as Operations (Operator Credentials) makes the local stack an environment too), `up` (the dependencies and the `devx` profile in containers, then `migrate`, `seed`, the application on the host through `scripts/dev.sh`, and `urls`; the second compose file is for the case that asks for it and never the default), `down` (stops every container and the host processes, keeps the volumes), `reset` (`down` with the volumes removed, then `up`), `urls` (prints every local URL from `.env`), `test-telemetry` (the round-trip test under the `telemetry` marker, against the `devx` profile), `traffic` (`make traffic PROFILE=light DURATION=30`: the generator through the `<root>-ops` binary, the thirty-second light run being CI's wiring check), self-documented
-
- `.github/workflows/ci.yml`, `deploy-staging.yml`, `deploy-production.yml`, `release.yml`, `grant-operator.yml`, `state-unlock.yml`:
-
- - `ci.yml` sets up uv and pnpm at the releases `.python-version` and `.nvmrc` name and runs `make check`, then `make openapi` and `git diff --exit-code` so a committed OpenAPI document or generated client that drifted fails, then the integration job over the compose stack (`make migrate`, `make migrate-check`, `make migrate-roundtrip`, `make test-integration`, then `make seed`, the API started on the host and waited for on `/healthz`, and `make traffic PROFILE=light DURATION=30`, the gate's thirty-second light run of Operations (Traffic and Stress), the API's log printed on a failure), `terraform fmt -check` and `validate` per root, both bootstrap roots and both environment roots, and an image build
- - `deploy-staging.yml` runs on every push to `main`, and on `workflow_dispatch` for the first deploy `scripts/cloud_create.sh` dispatches, with no approval, in the concurrency group `deploy-staging` that never cancels a run in progress: every job that touches the cloud declares the `staging` environment and reads that environment's variables, so it holds staging's role and nothing else
- - The build jobs hold a push-only role (`<root-slug>-build-staging`, which pushes images and writes the bundle prefix and nothing else) and build the images and the bundles under the commit, push the images and keep the bundle by commit in staging's artifacts bucket, both of which replicate into production's account
- - The apply job alone holds the deploy role and applies `environments/staging/`, whose migration runs as a one-off task on the new image before the rollout, so a merge is the deployment, then reads `/readyz` through the edge and records a deployment on the repository host naming the commit, each image's digest, and the bundle's hash
- - A smoke job follows each apply, staging's and production's: it runs the grant task with `--mint-token --token-secret <root-slug>-<env>-smoke-token` for the smoke identity the environment's `SMOKE_EMAIL` variable names, reads the token from that secret into the job's masked environment, and runs `ops/tests/test_telemetry_roundtrip.py` against the deployed base; while `SMOKE_EMAIL` is empty it is skipped with a notice naming the grant that is due
- - `deploy-production.yml` runs on a push to `release`, and on `workflow_dispatch` on `release` for the first deploy, under the same checks and the same approval, in the concurrency group `deploy-production` that never cancels a run in progress: it checks that `release` is an ancestor of `main` and stops otherwise, declares `production-plan` for the jobs before the approval and `production` for the apply, each reading its own environment's variables, reads the release commit's staging deployment record, looks up the digests and the bundle in production's own registry and artifacts bucket, where replication put them, waiting a bounded time for the copy, and refuses a commit with no successful staging deployment or a copy whose digest or hash differs from the record
- - The fast rollback, a `workflow_dispatch` input, goes to the previous release only, the one production ran before the current, read from its deployment record: it swaps each service's image digest and the portal bundle back to that release's, behind the same approval, runs no migrate and plans no Terraform, and leaves `release` where it is; anything older rolls forward with a fix
- - Saves the plan of `environments/prod/` in production's state bucket under the apply's access (never as a workflow artifact; the masked text rendering is the reviewer's copy), and applies that plan in a job behind the production environment's approval, the migration running as a one-off task on the new image before the rollout, never building; after the apply the job adds the tag `prod-<commit>` to each promoted digest in production's registry, a new tag and never an overwrite, which the registry's immutable tags allow, so the lifecycle that keeps every `prod-` image never expires what production runs or ran
- - `release.yml` runs on dispatch and fast-forwards `release` to the commit of the last successful `deploy-staging` run, read from that run's deployment record, the only push to `release` there is, made with a token of the repository host's app, which the `release` ruleset admits as its one bypass actor
- - The job declares a `release` environment whose deployment-branch policy admits `main` alone and which holds the app's key, and the app's push starts `deploy-production.yml`
- - `grant-operator.yml` is dispatched by a person on an environment's branch with the inputs `email`, `permission` (`read` or `write`), `disable` (a boolean), and `mint_token` (`none`, `provisioner`, or `smoke`), declares that environment, and runs the API image's subcommand as a one-off task under the deploy role, `grant-operator --email <email> --permission <permission>`, or `grant-operator --email <email> --disable` when `disable` is set (production's behind the same approval). The identity signs up first, like any person; the grant puts it on the operator allowlist, and a person then enrols the second factor at the console's first sign-in
- - With `mint_token`, the grant job mints the operator token of a machine identity, the provisioner's or the smoke identity's, and no person's: `grant-operator --email <email> --mint-token --token-secret <root-slug>-<env>-<mint_token>-token` mints a token carrying the entry's one permission and expiring within one hour, stores its digest, and writes the token itself into that secret, never into a log or the job's output. The secret's policy admits the grant task to write it, and the person's identity-center role (whose sign-in carries a second factor) and the deploy workflows' smoke job to read it; the investigate role's fence denies it like every secret value. `<root>-ops token --env <env> --identity provisioner` copies the provisioner's into the env file
- - `state-unlock.yml` is dispatched with a root and a lock id and runs `terraform force-unlock` for that one lock under the environment's deploy role
-
- `deployment/terraform/modules/`, `deployment/terraform/bootstrap/{staging,prod}/`, `deployment/terraform/environments/{staging,prod}/`, `deployment/cloud/environments.json`:
-
- - One module per resource the settings name (database, cache, queue, buckets, secrets, service with rollout limits so a worker never exceeds its desired count, a log group per process with retention, `/<root-slug>/<env>/<process>` (`/<root-slug>/<env>/api`), and a non-essential OpenTelemetry collector beside each task that adds the service and the environment as dimensions and no other beyond a metric's bounded labels and sets the trace's service name to `<root-slug>-<process>` (`<root-slug>-api`), the name X-Ray shows; the load balancer answers `/metrics` with a 404, served at `api.<base_domain>` with its certificate and DNS record), wired in both environments, each passing its `base_domain` (production the product's domain, staging a `staging.` subdomain of it), the API's allowed origins, and every prefix the settings read (buckets, queues, secrets) as variables, the process environment naming each of them, and the database URLs wired as secrets (never a password alone), one per login, plus the master URL `<ROOT>_DATABASE_MASTER_URL`: the migration login's and the master's, which only the one-off migration task holds, and the runtime login's and the system login's, which the services and workers hold, with no secret value in state or a plan: the master password from an ephemeral generator through the database's write-only password attribute, or managed by the database service, each login's password from an ephemeral generator, and each URL's secret version written through its write-only attribute, never computed as an output; the API's `totp_encryption_key` is a secret the same way, from an ephemeral generator, wired into the API's task alone
- - The target group's health check and each task definition's own health check on `/healthz`, never `/readyz`
- - The ECS deployment circuit breaker with rollback on every service
- - The API's migration as a one-off task on the new image before its rollout, which the rollout depends on: it first runs `migrate ensure-logins`, which connects under the master URL and nothing else does, creates each of the three logins by its fixed name when absent, sets its password from its URL's secret, and grants the migration login `CREATE` on the database; then `migrate upgrade --all` runs the migrations as the migration login
- - The API's `grant-operator` subcommand as a second one-off task definition on the same image, `<root-slug>-<env>-grant-operator`, holding the runtime login's and the system login's URLs as secrets and never the migration login's or the master's; `grant-operator.yml` runs it with the email and the permission as the command's arguments, and the environment root outputs its name; the environment root creates the two token secrets, `<root-slug>-<env>-provisioner-token` and `<root-slug>-<env>-smoke-token`, empty, with no version, so no token is in state; this task's role may write both and no other secret value, and the deploy role may read the smoke token, which its smoke job presents
- - Every ECS service waits for its steady state (`wait_for_steady_state`), so a rollout the circuit breaker rolled back fails the apply
- - Environment names match the settings' cloud-environment set
- - `deployment/cloud/environments.json` naming the region and, per environment, its `account_id`, its `admin_profile` (`<root-slug>-<env>-admin`), its `sso_profile` (`<root-slug>-<env>`), and its public names, read by every root and both scripts, with every provider's `allowed_account_ids` pinned to the environment's account
- - One bootstrap root per account, `bootstrap/<env>/`, over a shared `account` module: the state bucket `<root-slug>-state-<account id>`, versioned, holding state and nothing else, the artifacts bucket `<root-slug>-artifacts-<account id>`, versioned, holding the bundles kept by commit and nothing else, with a lifecycle that outlasts the last few releases and, in production, a refusal of a second write to a key under the bundle prefix, the image registry with immutable tags, scan on push, and a lifecycle that keeps a bounded number of images and never one tagged `prod-`, the repository host's identity federation and the roles it trusts (staging's push-only build role and its deploy role, production's two, one that plans and one that applies, so the approval gates the one that writes, each trusting its repository-host environment, the ref of its branch, and the repository by `repository_id` and `repository_owner_id`), the permissions boundary every role a deploy role creates must carry, each deploy role denied a role created without it and every change to the boundary, to the deploy roles, to the bootstrap's roles and trust, and to the bootstrap's state key, a trail of the account's API calls in a bucket of its own, the investigate role `<root-slug>-investigate-<env>` (read-only over every signal and every resource description, the state prefix readable so `terraform plan -lock=false -refresh=false` runs, and fences denying every secret value, every data bucket's objects, the database connect, and the other environment's tags) trusting the identity center's everyday role of its own account by pattern, with no cloud user and no access key anywhere, the budget (`monthly_budget_usd`, alerts at 50, 90, and 100 percent actual and 100 forecast to `owner_email`) and the anomaly monitor, and one hosted zone per public name
- - Staging's bootstrap root replicates its registry and its artifacts bucket into production's account under `replicate_to_production`, and production's grants those two writes, scoped to its repositories and to the bundles' prefix, and nothing else
- - Per environment root, the alarm topic `<root-slug>-<env>-alarms` with its email subscription and the default alarm set (load balancer 5xx ratio, unhealthy targets, database CPU, database free storage, running tasks below desired for the API and the worker, load balancer p95, the outbox's lag, with or without a worker, from the gauge the sweep sets, the worker's or, under `--no-worker`, the API's, and, with a worker, the queue's: the oldest waiting item's age, failed work, and parked records, from the gauges the worker's sweep sets), the dashboard `<root-slug>-<env>` with the same panels as the local metrics view, target-tracking autoscaling on every service behind the root switch `autoscaling_enabled` (off by default, every lever below it on, so one flip scales the environment), the root switch `destroyable` (off by default: `force_destroy` on the buckets, and outside production `skip_final_snapshot` on the database; production's database always leaves its final snapshot and keeps its automated backups, `delete_automated_backups = false`), the database's deletion protection under a variable of its own, `database_deletion_protection` (`true` in production, set in `environments/prod/terraform.tfvars`, so a merged change and never a script turns it off), outside production no recovery window on the secrets, the services' `desired_count` ignored once autoscaling owns it, private subnets with a named egress (a NAT gateway, or the private endpoints of the registry, the secret store, the logs, the object store, and the queue), encryption at rest on every store, TLS required by the database, and transit encryption on the cache, whose client connects over TLS (`cache_url` is a `rediss://` URL in every deployed environment, and the infra settings refuse `redis://` unless the environment is `local`), so every connection to either store is encrypted, point-in-time recovery with a declared retention and two zones in production once customers depend on it, production's load balancer and distribution behind the managed web firewall or an ADR saying why not, retention on every log group, and the environment tag through the provider's `default_tags`
-
- `scripts/cloud_create.sh`, `scripts/cloud_nuke.sh`:
-
- - The hands of the create and nuke skills, both with `--dry-run` printing every command they would run: both clear the keys exported in the shell, take `<env>` under the `admin_profile` the environments' file names for it, and refuse unless `aws sts get-caller-identity` answers that environment's `account_id`, asked again before every apply
- - Create applies `bootstrap/<env>/` with local state and moves it into the bucket it made (staging's root turning `replicate_to_production` on once production's bucket exists), writes each public name's NS delegation at the domain's DNS host through its API with a token read from the environment, the run's one credential outside the account, scoped to the domain's zone and held only for the run, writes the profile `<root-slug>-<env>-investigate` into `~/.aws/config` chained by `source_profile` from the `sso_profile`, writes the owner-only env file `~/.config/<root>/ops/<env>.env` (`<ROOT>_API_URL`, and the lines `<ROOT>_OPERATOR_TOKEN`, `<ROOT>_PROVISIONER_TOKEN`, `<ROOT>_ERROR_TRACKER_URL`, and `<ROOT>_ERROR_TRACKER_TOKEN` left empty, since no operator exists until the pipeline's first-operator job has run and the operator has enrolled; the file never holds a password or a TOTP secret), prints the two error-tracker lines as the one part of the env file a person fills by hand, once they have made the environment's project in the error tracker, creates the repository's two branch rulesets when absent (`main`: a pull request with one approving review and a code owner's review, every required check, no force push, no deletion; `release`: no push, no force push, no deletion, and one bypass actor, the repository host's app that `release.yml` pushes with, named by the app id the script takes as an input, `--app-id`; a person makes the app by hand before the first run and stores its private key in the `release` environment, and the script lists both among its manual steps), creates the GitHub environments (`staging`; `production-plan` with no reviewer and `production` with the required reviewer; and `release`, whose deployment-branch policy admits `main` alone, for `release.yml`), each with its deployment-branch policy (`staging` from `main`, both production environments from `release`), and sets each one's variables under the same names (`AWS_ROLE_ARN`, `TF_STATE_BUCKET`, `ARTIFACTS_BUCKET`, the public names, the alarm address), dispatches staging's first deploy through the pipeline and, for production, names the order that comes first (staging again, a merge to `main`, then the release), and prints the smoke test as the step that follows the first-operator grant: the smoke test reaches the operator plane with the smoke identity's token, so it passes once `grant-operator.yml` has granted the smoke identity a `read` entry and minted its token, and never before
- - Nuke takes `<env>`, refuses `production` unless `--confirm production` is typed, the root's `database_deletion_protection` reads `false` on `release`, and the applied state no longer protects the database (the released change is what lifted it; the script never sets that variable), checks out the environment's exact deployed commit (`release` for production; for staging, the commit of staging's last successful deploy, read from its deployment record, never the tip of `main`) into a clean worktree of its own and refuses a dirty one, applies the `destroyable` switch from there, empties the data buckets, destroys the environment root, removes the environment's GitHub environments and their variables, the investigate profile from `~/.aws/config`, and the env file, and prints what remains (production's final snapshot, and the bootstrap root: the zones, the state prefix, the images, the roles)
-
- `.claude/skills/<name>/SKILL.md`, one per operational skill: `ops-investigate`, `ops-watch`, `ops-root-cause`, `ops-infra-as-code`, `ops-cloud-deployment-create`, `ops-cloud-deployment-nuke`, `ops-simulate-traffic`, `stress-test-create-or-update`, `stress-test-run`:
-
- - Copied from `${CLAUDE_SKILL_DIR}/../_shared/ops-skills/<name>.md` with the product's name substituted the same way, nothing else edited: each states its role as Operations (Operational Skills) names it, the credential it holds, what it reads, what it never does, and its report. The investigator and supporter skills hold `<root-slug>-<env>-investigate` and read the owner-only env file `~/.config/<root>/ops/<env>.env`, except `ops-infra-as-code`, which plans against the cloud and reads no env file
- - Create and nuke hold the environment's `admin_profile` alone, compared with its `account_id` in `deployment/cloud/environments.json`
- - The traffic and stress skills have no role of their own, and `ops-simulate-traffic` and `stress-test-run` read the env file for the provisioner identity and hold the investigate profile only to read the signals back from a cloud environment
- - `stress-test-create-or-update` writes a file and holds no profile and no env file. Every one takes `--env staging\|production`, and every one but create and nuke also takes `local`, reading the `devx` stand-ins
-
- `ops/`:
-
- - The workspace member `<root>-ops`, package `<root>.ops`, binary `<root>-ops`: `traffic --env <env> --profile light\|regular\|heavy\|stress [--duration S] [--orgs N] [--report path]`, the one generator (its tenants and their members created through `POST /v1/admin/orgs` and `POST /v1/admin/orgs/{org_id}/members` by the provisioner identity of the env file and named with the run id, and removed through `DELETE /v1/admin/orgs/{org_id}` when the run ends, a failure included, the tenants it could not remove named in the report; `--orgs 0` drives the seeded people and needs no provisioner, so it is local only and refused against a cloud environment, which has no seeded people), riding `clients/python/` and the operator plane, driving the edge and never a manager, with realistic sessions (sign in, list, add a handful, edit, complete, reopen, move, list, delete one, read events, one socket that sees its own change, sign out; the socket is the generator's own, opened with `websockets` on `/v1/realtime/socket?ticket=` after `POST /v1/realtime/tickets` through the client, since `clients/python/` is REST only) and four profiles differing by tenants, members, concurrency, and think time, reporting requests by route and status, p50, p95, p99, and the error ratio
- - `stress --scenario ops/stress/<name>.yaml --env <env> [--report path]`, the same generator at the scenario's profile with its ramp, soak, and target
- - `signals check --env <env> --request-id <id> [--since-minutes N] [--log-file PATH]` over `SignalsInterface` (`log_lines`, `metric_delta`, `trace`, `error_event`) with `SignalsLocalImpl` (the Prometheus HTTP API, the Jaeger HTTP API, GlitchTip's REST API with the token the seed creates, the captured stdout) and `SignalsCloudImpl` (CloudWatch Logs Insights, `GetMetricData` on the product's namespace, X-Ray, the error tracker's REST API)
- - `size --env <env>` (tenants, users, the main entity written in the last day, through `GET /v1/admin/size`)
- - `token --env <env> --identity operator\|provisioner`, which writes an operator token into the env file as `<ROOT>_OPERATOR_TOKEN` or `<ROOT>_PROVISIONER_TOKEN` without printing it. For the operator, the person runs it in their own terminal: it asks there for the email, the password, and the TOTP code, signs in, and calls the mint route `POST /v1/admin/me/tokens` with `permission: read` and an `Idempotency-Key` of its own, so no agent ever holds a password or a code. For the provisioner it reads the token the grant job wrote into `<root-slug>-<env>-provisioner-token`, under the person's `sso_profile`. Against `local` it runs `uv run <root>-api grant-operator --mint-token` for the seeded identity, as `make seed` does. Every other command reaches the operator plane with the token alone, and answers an expired one by naming this command; every command that reads the env file refuses one that is not owner-only (`0600`), and none prints a value from it
- - `ops/tests/test_telemetry_roundtrip.py` under the `telemetry` marker, which starts the API as a real process with the OTLP endpoint and the DSN set, drives one session, and reads every signal back by request id, so the same test runs as the deployed smoke test with a different base; it reaches the operator plane with an operator token, `<ROOT>_OPERATOR_TOKEN` locally and the smoke identity's token when deployed
-
- OM distribution, under `om/`:
-
- | File | Holds |
- |----------------------------------------|-----------------------------------------------------------------------------------------------|
- | `pyproject.toml` | `<root>-om`; `pydantic`, `pydantic-settings`, `sqlalchemy[asyncio]`, `asyncpg`, `alembic`, and `<root>-infra` as a workspace source (`build_managers` takes `InfraInterface`; the OM depends on infra and never the reverse) |
- | `src/<root>/om/base.py` | `Platform`, the five mixins with `PROVENANCE_FIELDS` beside them (the constant naming `created_at`, `created_by`, `deleted_at`, and `deleted_by`, which every copy on update leaves as stored), `FrozenMapping`, `new_id`, `utcnow`, `EMPTY_UUID` |
- | `src/<root>/om/opcontext.py` | `SecurityContext` with `user_id`, `org_id`, `role`, `permissions`, `teams`, `credential_kind`, and `credential_id` (ids and facts, never a `User` or `Org` entity; a socket ticket re-checks the credential by its id), `AppContext`, the stages `RequestContext` (request id, app, trace id, the `traceparent` a handoff carries on, the causing request a handoff names), `IdentityContext(RequestContext)` (identity id, email, credential), `OpContext(RequestContext)` (with the `org_id`, `user_id`, `credential_kind`, and `credential_id` properties), and `OperatorContext(IdentityContext)` (with `permissions: tuple[OperatorPermission, ...]`, what its allowlist entry grants, read or read and write, or `ENROL` alone while the identity has no confirmed TOTP secret), each produced by one transition on the tenancy manager; the scopes `RequestScope`, `TenantScope`, `ActorScope(TenantScope)`, `CredentialScope`, and `ProvenanceScope(ActorScope, RequestScope)` as `Protocol`s of read-only properties; `Role` (`owner`, `admin`, `member`, `viewer`, ordered `owner > admin > member > viewer`, each role's permissions a superset of the next one's, which a unit test holds to the permission table, so "the lower role" means the later one in that order; and the role reserved for services, outside the order), `Permission`, `OperatorPermission`, `CredentialKind` (`password`, the sign-in credential; `api_key`; `tenant_session`; `operator_token`; and `internal`, the per-call credential a peer service mints, which the gateway parses), and `AppType` are declared here, and the role-to-permission table in `tenancy/types/` reads them, so this module imports nothing above `base.py` and no module needs `TYPE_CHECKING` to stay acyclic |
- | `src/<root>/om/exceptions.py` | `PlatformException` with `http_status` and `code`; `NotFound`, `Conflict`, `PreconditionFailed` (412, code `precondition_failed`, the compare-and-set that found another version than the caller expected), `ValidationFailed` (422, code `validation_failed`), `NotAuthorized`, `NotAuthenticated`, `Unavailable` (the shape of a dependency that cannot be reached: an open breaker, a refused admission, a backend that is down) |
- | `src/<root>/om/root.py` | `build_managers(storage, infra, options) -> Managers`, `options` a frozen `ManagerOptions` that each root, the API's container and a worker's, builds from its own settings and passes whole: the tenancy manager's bounds and, once the work namespace exists, `item_retention`, which `build_managers` passes to the work manager |
- | `src/<root>/om/storage/root.py` | `StorageInterface` with `healthcheck` and `close`, and `InsertOutcome` (`INSERTED`, `ID_EXISTS`, `KEY_EXISTS`), the answer of a create that can collide on more than one key, both re-exported from `storage/__init__.py`, since the guideline puts the root at `<root>.om.storage` |
- | `src/<root>/om/storage/roles.py` | `DatabaseRole`, the table-to-role map `TABLE_ROLES`, `TenancyScope` (`system`, `org`, `identity`, `both`), and the table-to-scope map `TABLE_SCOPES` beside it (the names `arch-check` reads by default), naming the column the policy rests on for an `identity` or `both` table, as The Storage Layer (The Second Fence) states; `orgs` is `system`, the platform's global list of tenants |
- | `src/<root>/om/events/` | the `events` namespace of the guideline's Realtime at the Edge: `Event(Identifiable)` with `org_id`, `seq`, `kind`, `target_id`, `actor_id` (the principal of the write, `EMPTY_UUID` for the platform), and a typed payload of ids only, never a personal field's value, its `activity`-role table, storage with the named atomic `append_event` that takes the event whose id is the outbox row's id, does nothing and returns the stored event when that id is already written, the cursor's increment rolled back with it, so the relay at once and the sweep relaying one row leave one event and one `seq`, and otherwise assigns `seq` (per tenant, gapless, from a `cursors` row per tenant in the same role, `UPDATE ... SET head = head + 1 ... RETURNING head` inside the append's transaction, the row inserted on the tenant's first event; never `MAX(seq) + 1` with a retry), `read_head(org_id)` from the same row, which the events manager's `get_head(ctx)` reads, `read_after(org_id, after_seq, limit)`, and `count_since(since)`, the operator plane's count across tenants, a system-scope read, and a manager the outbox relay calls to record one event per entity write, because every push is also a record; an event carries ids only, never a field's value |
- | `src/<root>/om/audit/` | the `audit` namespace, the cross-cutting swimlane of Namespaces as Swimlanes: `AuditEntry(Identifiable)` with the same shape as an `Event` plus the request id and the app (`seq`, `kind`, `target_id`, `actor_id`, a typed payload, which may carry values, `request_id`, `app`), as Realtime at the Edge states, and no `org_id`: it is a tenant entity, read under a context, and `org_id` is its table's storage column; its `activity`-role table, storage with the named atomic `append_audit_entry` that assigns `seq` and `read_after(org_id, after_seq, limit)`, and `AuditManagerInterface`, which the dead-letter path of a worker and the operator plane write through |
- | `src/<root>/om/outbox/` | the transactional outbox of Database Roles; the full list is below the table |
- | `src/<root>/om/idempotency/` | the edge idempotency marker: `IdempotencyMarker(Identifiable, Created)`, as The Network Layer (The Gateway) declares it, its `core`-role table unique on `(org_id, user_id, key)`, storage, and a manager with `begin`, `finish`, the release, and the take-over, each one conditional write with its guard in the statement, and `purge_markers(rctx)`, the sweep's purge of markers past their retention over the storage's `purge_markers(before, limit)`, as the marker table of The Network Layer (The Gateway) shows; an operator route's marker is keyed under `EMPTY_UUID` as the `org_id` with the operator's identity id as the `user_id`, so its calls run on the system login, among the enumerated system-scope methods; so a replayed creating request dedupes on a durable unique index like every queue handler, and the cache is only a read-through |
- | `src/<root>/om/{tenancy,events,audit,outbox,idempotency}/README.md` | one per namespace, one level below `om/README.md` and linked from it, in the product's language: what its nouns are, what can happen to them, and which rules hold (for `tenancy`, the org, the member, the role, the session, the key, the operator allowlist, the operator token), and no developer or operator instruction, as Documentation as Code (A README at Every Level) states |
- | `src/<root>/om/storage/settings.py` | `StorageSettings`, one `BaseSettings` under the product prefix holding every database setting: the three shared URLs, the per-role URLs `<ROOT>_DATABASE_URL_<ROLE>` and `<ROOT>_DATABASE_SYSTEM_URL_<ROLE>` (`<ROLE>` the role in upper case, `CORE`, `ACTIVITY`, `QUEUE`; empty means the shared URL), the master URL, and the pool size, checkout timeout, and statement timeout, shared and per role the same way; every container and the migration CLI read it, and a unit test holds each of its fields in `.env.example` |
- | `src/<root>/om/storage/migrate.py` | `run_sql(role, file)`, which runs the file as one transaction and never splits it on `;`, since a `DO` block holds semicolons of its own, the check that a SQL file names only tables of its role, the ORM-versus-schema comparison, and the migration CLI (`upgrade --role <role>` or `--all`, `check`, both as the migration login, and `ensure-logins`, the cloud migration task's first step, the one command that reads `<ROOT>_DATABASE_MASTER_URL`) |
- | `src/<root>/om/storage/tables/base.py` | `Base` deriving the schema from the role map, the mixins with sort-order bands, `GlobalIdentifiableMixin`, `FeedIdentifiableMixin`, whose `org_id` carries no single-column index (a feed table composes it instead of redeclaring `org_id`), and `IdentityScopedMixin`, which carries `id` and `identity_id` and no `org_id`, and which every `identity`-scoped table composes (the tenancy tables of a person before any tenant, `sessions` among them) |
- | `src/<root>/om/storage/utils/translation.py` | `to_row`, `to_model`, `apply_row` |
- | `src/<root>/om/storage/impl/pg_base.py`, `postgres.py`, `memory_base.py`, `memory.py` | the Postgres base with `_upsert(table, entity, outbox_rows=(), *, org_id=None, user_id=None, identity_id=None)` (the outbox rows inserted in the same commit), `_insert` with the same arguments (doing nothing on an existing id or unique key and reporting which, as an `InsertOutcome`, the outbox rows landing only when the insert won, so a create returns the row as stored on a retry), and per-statement role routing, and one session funnel, `_session_for(stmt, *, org_id=None, user_id=None, identity_id=None)`, that routes the statement to its role, selects the system login's engine when `org_id` is `EMPTY_UUID` and the runtime login's otherwise, and sets `app.org_id`, `app.user_id`, and `app.identity_id` with `set_config(..., true)` so they die with the transaction; the memory base with the same two, landing the outbox rows in the outbox memory storage the root hands it; both roots, `StoragePostgresImpl` and `StorageMemoryImpl`, named like every other impl; `StoragePostgresImpl` reads the runtime login's URL and the system login's URL per role from `StorageSettings`, each defaulting to its shared value, opens one engine per distinct URL, and every pool declares `pool_size` and `pool_checkout_timeout` from settings, per role and defaulting to the shared value the way the role URLs do, so a checkout that waits past the bound fails instead of queueing without end, and every session applies `statement_timeout` from the same settings, so a statement past its deadline is cancelled and surfaces as a failure rather than holding its connection, as The Storage Layer (Database Roles, A Storage Impl) states |
- | `src/<root>/om/tenancy/` | the tenancy namespace: its entities, its manager's operations, storage impls, and tables; the full list is below the table |
- | `migrations/alembic.ini`, `migrations/env.py`, `migrations/sql/{core,activity}/`, `migrations/versions/{core,activity}/` | one chain per role, run as the migration login from its own URL: the initial `core` migration (tenancy, the outbox row, the idempotency marker) and the initial `activity` migration (events with its per-tenant cursor row, audit entries), since `make migrate` (every role) and `make migrate-check` of step 7 run every role a table was declared in; each chain's first migration creates its role's schema and grants the runtime and system logins `USAGE` on it and, through default privileges of the migration login, `SELECT`, `INSERT`, `UPDATE`, and `DELETE` on its tables; every table's policy is created in the same migration as the table, with `ENABLE ROW LEVEL SECURITY` and `FORCE ROW LEVEL SECURITY`, in the shape its tenancy scope implies, the `org` and `identity` expressions each ending in the one system-login clause, `OR (current_setting('app.org_id', true) = '<EMPTY_UUID>' AND current_user = '<system_login>')`, with the system login's name, `<root>_system` in every environment, in place of `<system_login>`, and dropped in the down file; a later migration that changes data takes the data-migration shape of the conventions, `NO FORCE ROW LEVEL SECURITY` around its statements and `FORCE` again in the same transaction, failing when the rows it touched differ from the rows it meant to touch; the `queue` chain arrives with `arch-scaffold-worker` |
- | `tests/contracts/` | the storage contract cases as plain modules, parameterised by a storage fixture, so the unit suite runs them over memory and the integration suite over Postgres (pytest `--import-mode=importlib`, since two distributions each have a `tests/`); every named atomic method is raced, not only called: two callers at once against the ticket redemption, the idempotency take-over, `remove_membership` against the same member issuing an API key (no key of the removed member stays live), and later the claim, asserting that exactly one wins, over memory and over the engine |
- | `tests/unit/` | tenancy on write, both roots built whole (every getter of `StoragePostgresImpl` and `StorageMemoryImpl`, every capability of the infra root, and every field of `Managers` exists after construction; none is built on first use), the request-stage list (every manager method that takes the request stage instead of a later one, enumerated, `sign_up` and the sign-in among them, so a new one fails the test), the contract cases over memory. The role map, the construction sites of the stages, the tenant-less storage methods, the import direction, the interface check, and the migration heads are not tests: `arch-check` decides them in `make check`, and the root `pyproject.toml` lists the sites under `[tool.arch-check.options.CTX-26] sites` and the tenant-less methods under `[tool.arch-check.options.CTX-12] tenantless`, each list held both ways; `tenantless` names the five lookups before an identity is known as `TenancyStorageInterface.read_identity_by_email_digest`, `TenancyStorageInterface.read_identity_by_issuer_subject`, `TenancyStorageInterface.read_api_key_by_digest`, `TenancyStorageInterface.read_session_by_digest`, and `TenancyStorageInterface.redeem_socket_ticket`, the sign-in failure counter over the `system` table `sign_in_failures`, `TenancyStorageInterface.read_sign_in_failures`, `TenancyStorageInterface.record_sign_in_failure`, and `TenancyStorageInterface.clear_sign_in_failures`, the global reads of the `orgs` table, `TenancyStorageInterface.read_orgs` and `TenancyStorageInterface.read_live_org_ids`, the operator plane's counts, `TenancyStorageInterface.read_size` and `EventsStorageInterface.count_since`, the outbox's sweeps, `OutboxStorageInterface.read_pending`, `OutboxStorageInterface.read_oldest_pending`, and `OutboxStorageInterface.purge_done`, and the sweep's other purges, `IdempotencyStorageInterface.purge_markers`, `TenancyStorageInterface.purge_socket_tickets`, and `TenancyStorageInterface.purge_sessions`; `sites` names `TenancyManagerImpl.sign_up`, `sign_in`, `authenticate`, `authenticate_login`, `admit_operator`, `redeem_socket_ticket`, `bootstrap`, and `service_contexts`, each by its full path, and the worker scaffold adds `claim_context` |
- | `tests/integration/` | the migration check (ORM metadata against the migrated schema, per role), the tenancy policy check (`pg_class` and `pg_policies` read for every table in the scope map and asserted against the scope it declares, since the schema diff does not see policies), the login check (on each of the three logins' live connections, `current_user` is neither superuser nor `BYPASSRLS`; the runtime login owns no table; the runtime login naming the system scope reads nothing), the backfill check, over a test-only fixture the check creates as the migration login and drops after: the `org`-scoped table `core.backfill_fixture` (`id`, `org_id`, and the column `flag boolean NOT NULL DEFAULT false`) with the policy every `org` table carries, `ENABLE` and `FORCE ROW LEVEL SECURITY` included, and a test-only data migration in the conventions' shape that sets `flag = true WHERE NOT flag`, run through `run_sql` over seeded rows of two tenants: it touches every row of both, and the check reads each back with `flag` set; the same migration without the `NO FORCE` does not fail on its own, since under `FORCE` the count and the update both see zero rows and the count check passes, so the check's own read of both tenants' rows is what catches it, and the contract cases over Postgres |
-
- `src/<root>/om/outbox/`:
-
- - The transactional outbox of Database Roles: `OutboxRow(Identifiable, Created)`, as The Storage Layer (Namespace Shape) declares it, with `org_id`, `kind`, `target_id`, `payload` (a `FrozenMapping` of ids only, never a personal field's value; `{}` for an entity change, since the row already names its target), `actor_id` (the principal of the write, `EMPTY_UUID` for the platform), `request_id`, `traceparent` (the causing request's trace context, the header and not the id, empty when no tracer was configured), `app` (an `AppContext`), and `done_at`, and no `created_by`, since the row names the actor of the write it announces
- - The helper `outbox_row(ctx, kind, target_id, payload)` that builds it from a `ProvenanceScope`, which every manager write uses
- - Its `core`-role table, the table `org`-scoped in `TABLE_SCOPES`, storage with `mark_done(org_id, row_id)` under the row's tenant and three system-scope methods that open their session with `org_id=EMPTY_UUID`, `read_pending(limit)`, `read_oldest_pending()` (the created time of the oldest pending row, for the lag gauge), and `purge_done(before)`, and `OutboxRelayInterface.relay(org_id, row)` with its impl, which dispatches on the row's `kind` and marks the row done, idempotent on the row's id, the id every destination is written under: a row announcing an entity change appends the `Event` with the row's actor through the events manager and publishes `ENTITY_CHANGED` with the `seq` the append assigned, since only the relay knows it, and a row whose kind is `work.<kind>`, `<kind>` the `WorkKind` member's name in lower case (`work.notify_shipment`), enqueues the item through the work manager's `enqueue_relayed(org_id, row)`, which publishes `WORK_AVAILABLE` as every enqueue does (`arch-scaffold-worker` creates that namespace; until it does, the relay has the one branch)
- - A durable effect never rides a topic alone. What must happen after a write rides an outbox row or a work item, and a topic, at most once, carries hints only (`ENTITY_CHANGED`, `WORK_AVAILABLE`), so a lost message delays an effect and never drops it
- - A manager takes the relay by interface and calls it after every write; a relay in the request path never raises, since the write has committed: a failure is logged and left to the sweep, and the request answers as the success it was. The worker sweep calls the relay's three sweep operations, each on the sweep pass's `RequestContext` and listed in the request-stage test: `relay_pending(rctx, limit)`, which relays each row `read_pending` returns through `relay(org_id, row)`; `purge_done(rctx)`, which purges done rows past retention; and `read_lag(rctx)`, the age of `read_oldest_pending()`, for the lag gauge
- - The relay's tests: a relay whose event append fails leaves the write answered and the row pending, and the sweep's relay of a row the request path already relayed leaves one event
-
- `src/<root>/om/tenancy/`:
+ Everything the references below list is under `<target-dir>/`. Those
+ lists are long, so each one lives beside this file and is read by the
+ step that names it, when that step runs and not before.
- - The namespace shape with `Org`, `Identity`, `User`, `Membership`, `Session`, `ApiKey`, its manager (sign up: `sign_up(rctx, email, password, display_name, org_name, org_slug)` creates the identity, its first org, and the owner membership in one transaction and answers as the sign-in does, refusing an email or a slug already held with `Conflict` (the slug as the org insert's `KEY_EXISTS` on the slug's unique index, never a read by slug) and raising `NotFound` while its options' `signup_enabled` is false; list an identity's memberships, `get_memberships(ictx, limit)`, bounded, the same `MembershipChoiceView`s the sign-in answers with; sign in, `sign_in(rctx, email, password, totp_code)`, which returns the identity stage and the sign-in credential, as The Business Layer (Operations Without a Principal) states, and delays each email after a failed attempt: it counts failures in the `system`-scope `core` table `sign_in_failures`, keyed by the email's digest, through `read_sign_in_failures`, `record_sign_in_failure`, and `clear_sign_in_failures` on `TenancyStorageInterface`; every failure doubles the delay before the next attempt for that email is checked, from a base to a cap its options carry, a success resets it, and an attempt inside the delay is refused `429` before the password is checked, so an unknown email is delayed the same as a known one; authenticate a credential, `authenticate(rctx, bearer)` into `OpContext` and `authenticate_login(rctx, bearer)` into the identity stage from the login credential or a live session, their lookups before an identity is known, `read_identity_by_email_digest`, `read_identity_by_issuer_subject` (the external provider's find-or-create, over the identity's `issuer` and `subject`, unique together), `read_api_key_by_digest`, `read_session_by_digest`, and `redeem_socket_ticket` on `TenancyStorageInterface`, each a system-scope storage method that takes no tenant, opens its session with `org_id=EMPTY_UUID` passed explicitly, runs on the system login, and is listed as tenant-less, everything after the lookup under the scope it found; a session carries two lifetimes from the options, an idle one measured from its `last_seen_at`, which `authenticate` and `authenticate_login` move forward at most once a minute, and an absolute one measured from its creation, and a session past either is refused `401`; an operator token follows the idle lifetime too, inside its one-hour `expires_at`; an API key's effective role is the lower of the role it was issued with and its issuer's current role, read at every use, so a demoted issuer's keys are demoted with it; verify the TOTP code a sign-in presents against the identity's enrolled secret, refusing a code already used in its time step, and record the verified factor on the sign-in credential; sign out, `sign_out(ictx)`, ending the credential the identity stage came from and, when that credential is a session, publishing `SESSION_REVOKED` for it, as revoking a session does and as removing a member does for each of the member's sessions in that tenant; issue and exchange tokens, an exchange presented with a session ending that session in the same write (an API key is issued with a required `expires_in`, capped by a settings option and never `None`; the wire request defaults inside the cap; its role is at most the issuer's own, and a request for a higher one is refused `403`), bootstrap the first org and operator and return the owner's context, `bootstrap(rctx, ...)`, grant and disable an operator's allowlist entry for the `grant-operator` subcommand, `grant_operator(rctx, email, permission)` and `disable_operator(rctx, email)`, admit an operator, `admit_operator(ictx)`, which admits an allowlisted identity only when its sign-in credential carries a verified second factor and admits one with no confirmed secret with `OperatorPermission.ENROL` alone (a secret minted and not yet confirmed is not enrolled), refuses an identity stage whose `credential_kind` is `tenant_session` or `api_key`, and admits one whose `credential_kind` is `operator_token` with that token's one permission (a `write` token's context holding `READ` and `WRITE`, since `write` implies `read`), the one named exception to "a password alone never admits", enrol and confirm an operator's TOTP secret, `enrol_totp(octx)` and `confirm_totp(octx, totp_code)`, the secret stored encrypted under the `totp_encryption_key` its options carry, mint an operator token, `issue_operator_token(octx, permission, expires_in)` for the mint route, refused unless the operator stage came from a sign-in with a verified second factor, so a token never mints a token, and `grant_operator_token(rctx, email, expires_in)` for the grant job's `--mint-token`, an operation on the request stage; either mints a `Session` of kind `operator` for the identity, found through `read_session_by_digest` like every session, carrying one `OperatorPermission` no wider than the identity's entry, expiring within one hour (`expires_in` 3600 seconds when absent, and never more), stored as its digest, and returned once as an `Issued...` shape, one service context per live tenant for sweeps, `service_contexts(rctx)`, paging the live tenants through `read_live_org_ids(after, limit)`, the sweep's two purges, `purge_socket_tickets(rctx)` over the storage's `purge_socket_tickets(now, limit)` (tickets redeemed or past their expiry) and `purge_sessions(rctx)` over `purge_sessions(now, idle_before, limit)` (sessions ended, past `expires_at`, or idle since before `idle_before`), each one cross-tenant statement bounded by a batch size under the system scope, issue and atomically redeem the socket ticket, `issue_socket_ticket(ctx)` and `redeem_socket_ticket(rctx, ticket)` into `OpContext`, and for every trait an entity composes the operation that exercises it: update a member's role, remove a member (one named atomic storage method, `remove_membership(org_id, user_id)`, which deletes the membership, revokes every API key the member issued in that tenant, and ends the member's sessions there in one transaction), update a user, revoke a session, delete an org on the operator plane; the operator plane's own operations, each taking `OperatorContext` first and the tenant as a parameter and never a context, which `routers/admin.py` of the service calls: `get_operator(octx)`, `get_size(octx)` (tenants and users through `TenancyStorageInterface.read_size()`, and the rows written in the last day, one event per write, through the events manager's `count_events_for_operator(octx, since)` over `EventsStorageInterface.count_since(since)`, both system-scope reads across tenants), `get_orgs(octx, limit)` over `read_orgs(limit)`, `get_org(octx, org_id)`, `get_members(octx, org_id, limit)`, and the tenant's events feed through the events manager's `get_events_for_operator(octx, org_id, after_seq, limit)` with `request_id` and `app` on each entry, each read of a tenant's rows logged with the tenant and the operator; `create_org(octx, ...)` and `add_member(octx, org_id, ...)`, the provisioning, each opening with `octx.require(OperatorPermission.WRITE)`, so a `read` entry is refused; every copy on update starts from the stored row, the caller's fields with `set(PROVENANCE_FIELDS) | set(<Entity>.MANAGER_OWNED_FIELDS)` excluded, and the create that issues an API key re-mints the secret on a rerun that finds the row, in the same named atomic write, guarded by the attempt the idempotency marker holds, returning a fresh `Issued...View` with the same id), storage impls, tables. An entity composes only the mixins a manager operation exercises
- - `sessions`, the one table of every session, `identity`-scoped on `identity_id`: `id`, `identity_id`, `kind` (`tenant` or `operator`), `org_id` (a tenant session's org, `EMPTY_UUID` for an operator token), `permission` (an operator token's one `OperatorPermission`, empty for a tenant session), `last_seen_at`, `expires_at`, and `ended_at`, beside the token's `digest`, unique, which `read_session_by_digest` reads. An operator token is a row of it, with no table of its own. A tenant session authenticates as `CredentialKind.TENANT_SESSION`, an operator token as `CredentialKind.OPERATOR_TOKEN`
- - `MANAGER_OWNED_FIELDS` on each built-in entity, the fields its manager sets and a caller never writes: `Org` `("slug",)`; `Identity` `("email", "password_hash", "totp_secret", "totp_confirmed_at", "issuer", "subject")`; `User` `("identity_id",)`; `Membership` `("user_id", "role")`, since a role changes only through the update of a member's role; `Session` every field but `id`; `ApiKey` `("digest", "role", "issued_by", "expires_at", "revoked_at")`; and `()` on the records no caller updates (`Event`, `AuditEntry`, `OutboxRow`, `IdempotencyMarker`). A versioned entity adds `version` to its tuple
+ | Reference | Holds | Read by |
+ |----------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|
+ | `${CLAUDE_SKILL_DIR}/references/skeleton.md` | the workspace and tool config, the `Makefile`, the README, the docs, the two ADRs and the runbooks, the local compose stack, the Terraform modules and roots with the environments' file, the two cloud scripts, the six workflows, and the nine operational skills | step 1 |
+ | `${CLAUDE_SKILL_DIR}/references/object-model.md` | the OM distribution under `om/`: the base module, the context stages and scopes, the exceptions, the storage layer with its roles, tables, translation, impls, and migrations, and the `tenancy`, `events`, `audit`, `outbox`, and `idempotency` namespaces with their tests | step 1 |
+ | `${CLAUDE_SKILL_DIR}/references/infrastructure.md` | the infra distribution under `infra/`: the cache, buckets, topics, queues, and secrets capabilities, observability and the trust store, and the configured and local roots | step 1 |
+ | `${CLAUDE_SKILL_DIR}/references/ops-package.md` | `clients/python/`, the Python client generated from the API's document, and `ops/`, the `<root>-ops` member with the traffic generator, the stress runner, the signals interface, and the telemetry round trip | step 3 |
+ | `${CLAUDE_SKILL_DIR}/references/api-sweep.md` | the API process's own sweep, `services/api/.../sweep.py`, the loop of `arch-scaffold-worker` without the queue, written only when the tree has no worker | step 4, with `--no-worker` |
- Infra distribution, under `infra/`:
+ A reference file is detail. These are the lines a run must never miss,
+ so they stay here:
- | File | Holds |
- |---------------------------------------------|-----------------------------------------------------------------------------------------|
- | `pyproject.toml` | `<root>-infra`; `valkey-glide`, `aioboto3`, `opentelemetry-sdk`, `prometheus-client`, `sentry-sdk`, `truststore`; no dependency on `<root>-om`: `TopicPayload` is a frozen base declared here with `extra="ignore"`, and the system scope is the zero UUID checked by value |
- | `src/<root>/infra/root.py` | `InfraInterface` with `start` and `close` |
- | `src/<root>/infra/exceptions.py` | `InfraException` with `http_status` and `code`, the root of every exception infra raises, since infra imports nothing from the OM, as Cross-Cutting Conventions (Exceptions) states |
- | `src/<root>/infra/{cache,buckets,topics,queues,secrets}/` | each: the interface (with `start()` and `close()`, returning `None` where an impl holds nothing), a memory or local impl that behaves like the hosted one (a queue twin does not deduplicate), the cloud impl translating driver errors into an `InfraException` and counting outcomes; `topics/` defines `Topics.WORK_AVAILABLE` and `Topics.ENTITY_CHANGED` with their payloads, the latter's `EntityChangedPayload` carrying `org_id` (from the base), `kind`, `target_id`, `seq`, and `actor_id`, and its one control kind, `SESSION_REVOKED`, carrying the session id as `target_id`, the identity id as `actor_id`, and `seq` as `None`, as the guideline's Topics names them, the two every later step produces or routes |
- | `src/<root>/infra/observability.py`, `trust.py` | logging with the filter that puts the service, the environment, the request id, and the causing request on every record, error reporting (on only when the DSN is set and not `off`; events tagged with service, release, request id), tracing, the OS trust store |
- | `src/<root>/infra/impl/settings.py`, `impl/configured.py`, `impl/local.py` | settings, the configured root that picks impls and refuses unsafe combinations (a `redis://` `cache_url` outside `local` among them), the all-local root |
- | `tests/` | every capability over the local impls, one test per operation of its interface; and the settings check: every field of `InfraSettings` appears in `.env.example` under its prefix (the test reads the file, so a knob added to settings and not documented fails the fast gate) |
+ - The guideline release is pinned once. `specs/architecture.md` names
+ the tag the conventions found before writing, and the `Makefile`'s
+ `arch-check` runs that same tag.
+ - Three logins reach the database, as the conventions state. The local
+ compose file's init script creates them under the names every
+ environment uses, `<root>_migration`, `<root>_runtime`, and
+ `<root>_system`, since a policy spells the system login's name.
+ - Every table declares its role and its tenancy scope, and the
+ migration that creates the table creates its policy, as the
+ conventions state. A chain's first migration also creates its role's
+ schema and grants the runtime and system logins on it.
+ - No secret value in Terraform state or a plan. Every password comes
+ from an ephemeral generator through a write-only attribute, and no
+ URL secret is computed as an output.
+ - Every health check, the target group's and each task definition's
+ own, is on `/healthz`, never `/readyz`.
+ - A durable effect never rides a topic alone. What must happen after a
+ write rides an outbox row or a work item, and a topic, at most once,
+ carries hints only (`ENTITY_CHANGED`, `WORK_AVAILABLE`), so a lost
+ message delays an effect and never drops it.
+ - An event, an audit entry's payload of ids, an outbox row, and a
+ socket frame carry ids only, never a personal field's value.
+ - Every settings knob is documented. A unit test holds every field of
+ `StorageSettings`, of `InfraSettings`, and of each process's own
+ settings to `.env.example` under its prefix.
## Changed
| File | Change |
|------|--------|
- | (none) | The tree is new; every later step appends to the files above. |
+ | (none) | The tree is new; every later step appends to the files the references name. |
## Procedure
1. `git init` in `<target-dir>`, nothing staged (skipped when the
target was a fresh repository). It comes first so that every step
after it, and every skill this one follows, lists its files from
- `git status`. Then write the skeleton (leaving `clients/python/` and
- `ops/` with its `README.md` to step 3), then the OM distribution
- and the infra distribution, and run `make setup`. The fast gate runs from
+ `git status`. Then write the skeleton, reading
+ `${CLAUDE_SKILL_DIR}/references/skeleton.md` before it (leaving
+ `clients/python/` and `ops/` with its `README.md` to step 3); then
+ the OM distribution, reading
+ `${CLAUDE_SKILL_DIR}/references/object-model.md` before it; then the
+ infra distribution, reading
+ `${CLAUDE_SKILL_DIR}/references/infrastructure.md` before it; then
+ run `make setup`. The fast gate runs from
step 2 on. The nine operational skills are part of the skeleton:
copy each template under `${CLAUDE_SKILL_DIR}/../_shared/ops-skills/` to
`.claude/skills/<name>/SKILL.md` with `acme` substituted, as the
- Created table states, and change nothing else in them.
+ skeleton reference states, and change nothing else in them.
2. Read `${CLAUDE_SKILL_DIR}/../arch-scaffold-service/SKILL.md` and
follow its Created, Changed, and Procedure with these arguments:
`api --realtime --container` (omit `--realtime` with `--no-portal`).
The operator plane's routes arrive with it.
- 3. `make openapi`, then write `clients/python/` generated from the
+ 3. Read `${CLAUDE_SKILL_DIR}/references/ops-package.md`, then
+ `make openapi` and write `clients/python/` generated from the
document it emitted, whether or not `--no-portal`; then write
`ops/` and `ops/README.md` over that client, add both members to
the workspace, and run `uv sync`. The ops package rides the client
and the operator plane, so it is written after both exist.
4. Unless `--no-worker`, read
`${CLAUDE_SKILL_DIR}/../arch-scaffold-worker/SKILL.md` and follow
it with `maintenance NOOP --container`: a worker whose only work is
the maintenance sweep, ready for real kinds. With `--no-worker`,
the sweep moves into the API process's lifespan, so outbox rows a
- crash left behind are still relayed: write
- `services/api/src/<root>/services/api/sweep.py`, the sweep of
- `arch-scaffold-worker`'s loop without the queue (the relay's
- `relay_pending(rctx, limit)` and `purge_done(rctx)`; the purges of
- idempotency markers, socket tickets, and sessions, through the
- idempotency manager's `purge_markers(rctx)` and the tenancy
- manager's `purge_socket_tickets(rctx)` and `purge_sessions(rctx)`,
- which step 1 wrote; the purge of soft-deleted rows, once an entity
- composes the mixin, per service context, each context built by
- the tenancy manager's `service_contexts(rctx)`; and the outbox lag
- gauge, which the outbox-lag alarm reads here as it reads the
- worker's), on a timer at
- `sweep_interval`, which this step adds to the API's settings and
- to `.env.example`, started in the lifespan
- after `start()` and cancelled before `close()`, every step
- idempotent and wrapped. Every API replica runs it, which is safe
- because every step is idempotent and each purge is bounded by a
- batch size.
+ crash left behind are still relayed: read
+ `${CLAUDE_SKILL_DIR}/references/api-sweep.md` and write
+ `services/api/src/<root>/services/api/sweep.py` from it. Whichever
+ path ran, the sweep sets the gauge under the one name the
+ outbox-lag alarm and the copied `ops-investigate` skill read,
+ `<root>_outbox_lag_seconds`, so a tree with no worker is watched
+ like a tree with one. Every API replica runs the sweep, which is
+ safe because every step is idempotent and each purge is bounded by
+ a batch size.
5. Unless `--no-portal`, read
`${CLAUDE_SKILL_DIR}/../arch-scaffold-app/SKILL.md` and follow it
with `portal --kind portal`, including its Terraform and deploy
rows: the portal's bucket and distribution exist in every
environment before this step is done. Its Python client row is
skipped, since step 3 wrote the client.
6. With `--first`, read
`${CLAUDE_SKILL_DIR}/../arch-scaffold-namespace/SKILL.md` and follow
it with `<namespace> <Entity> <field:type ...>`.
7. `make check` and `make openapi`, so the portal's generated types
and the Python client carry the routes of step 6; then, when Docker
is available, `make infra-up`, `make migrate`, `make migrate-check`,
`make seed` twice (the second run changes nothing, and leaves
`local.env` as the first wrote it), and `make test-integration`,
only against the
compose stack of step 1: refuse when any database URL `StorageSettings`
resolves (the three shared URLs and every per-role URL, from the
environment, `.env`, or the settings default) is not a local
address.
8. When Docker is available, run the negative control of
Cross-Cutting Conventions (Tests) once. Take the tenant predicate
out of one query of a storage impl over Postgres (the first
entity's list with `--first`, else a tenancy list). Run
`make test-integration` with the table's policy in place: it stays
green, the second fence holding. Turn the policy off for that
table (`ALTER TABLE ... NO FORCE ROW LEVEL SECURITY` and `DISABLE
ROW LEVEL SECURITY`, through `uv run` over the local migration
login's URL, since only the owner alters a table), and run `make test-integration` again: it fails, and the
failures name the cross-tenant case of that method beside the
policy check. Put the predicate back, turn the policy on again the
same way (`ENABLE` and `FORCE ROW LEVEL SECURITY`), and run
`make test-integration` green. Record both runs in
`docs/runbooks/tenant-isolation.md`: the query, the table, and
what the suite reported each time. A run two that stays green is a
defect of the suite: name it in the output and stop.
9. When Docker is available, `make devx-up`, then
`make test-telemetry`: the round trip starts the API as a real
process, drives one session, and reads the counter, the trace, the
error event, and the log line back by request id through the
`devx` twins. Then `make traffic PROFILE=light DURATION=30`, the
thirty-second light run, the same one CI's integration job runs.
Both are a wiring check of the edge, the
client, the generator, and the signals, and never a stress test;
a stress test has a scenario and a target, and is the platform
developer's to run.
10. Before the review, sweep the tree for the four misses a fresh
scaffold makes most, and fix each: a setting the Terraform root
does not pass to the service, a mutating manager operation whose
first line is not `ctx.require(...)` or `octx.require(...)`,
leaving out the operations the conventions exempt (the request
stage, the identity stage, and the outbox handoff), a socket route mounted
outside the gateway, a route that writes a durable row (201 or 202)
without the `Idempotency-Key` dependency. Then read
`${CLAUDE_SKILL_DIR}/../arch-review-full/SKILL.md`
and run it over the whole tree; the checker run and the git reads
it takes are in this skill's tools for that step. Close every high
finding and rerun `make check`; list the rest in the output for the
person. A high
finding on a fresh tree is a defect of this skill: name it in the
output so it can be closed at the source.
Stop at the first step whose gate fails and report where it stopped.
## Output
As `${CLAUDE_SKILL_DIR}/../_shared/scaffold-conventions.md` states,
plus one line: the tree is uncommitted, and the first commit is the
user's.