malloy-compliance-review · git:20261106.58c1127 · 2026-11-06 · sha256 951ee73becc0042f
malloy-compliance-review git:20261106.58c1127B
Immutable. This exact content never changes and is served at /api/v1/blob/951ee73becc0042f.
No tool to install first
claude plugin marketplace add https://markdownregistry.com/mp/a/art_enpij6mkpryn4vzo/951ee73becc0042f99279ca3854840fe43970d63877904269c4e6add1793ad79.json
Then /plugin install malloy-compliance-review@markdownregistry-enpij6mkpryn4vzo in Claude Code. It installs this exact version, the one graded B here, pinned to the commit it was graded at, and leaves it there.
---
name: malloy-compliance-review
description: 'Check a Malloy package and its controls against a written access policy: turn the policy''s prose into a control register, run compliance.py check against a served package (structure, access as each role, and the values themselves), judge what needs a person, and produce a report and a signable attestation. Use when the user asks whether a model "complies with", "satisfies" or "meets" a policy, HIPAA, SOC 2, PCI or the like, asks for a compliance review or attestation, or wants the review in CI.'
---
<!--
Copyright (c) Credible Data Inc.
SPDX-License-Identifier: MIT
-->
# Reviewing a Malloy Package Against an Access Policy
> A policy is prose: who may read which data, and under what conditions. A compliant package is
> controls a machine checks: gates on the analytical domains, classifications on the fields, a
> ceiling on what each domain carries. This skill connects the two. It writes the policy down as a
> register of checks, runs them against the served package, and reports each control as holding,
> failing, needing a person, or not provable from the model at all.
The review has two halves, and they are kept apart on purpose:
- **The register** (`policies/controls.json`) is the policy as checks. You draft it from the prose;
the policy owner approves it. It changes when the policy changes.
- **The check** (`scripts/compliance.py check`) runs the register against the package as it is
served. It changes nothing, reads only aggregates, and exits 1 when a control fails (3 when nothing failed but something could not be checked), so the same
command gates every change in CI.
What Publisher refuses at load (a domain exposing a class its `#(carries)` omits) the check reports
as the control it breaks. A join that reaches a gated source around its gate is only a `join-bypass`
warning on the server, in every package, so the check fails it: this review is the hard stop. It
does not re-derive either.
## Before you start
- A Publisher serving the package, reachable over REST (default `http://localhost:4000`). Run the
review against a server with **no gateway in front**: each persona's givens are sent with the
request and honored, which is how the check asks "what would this role see". Never point it at a
production endpoint, where a trusted tier sets those givens and the probe would be meaningless.
A pass certifies the package's controls. In production they hold only behind the gateway that
authenticates users, sets the givens and blocks raw SQL, so a report says that alongside the verdict.
- The policy document.
- A classified model. If fields are unclassified, `skill:malloy-classify` first.
## Workflow
### 1. Draft the register from the policy
Read the whole policy. For each numbered requirement, write one control with its id and text as
the policy states them, and the checks that prove it. Then the context the checks need: the class
scheme, the personas the policy names, what each persona may read on each domain, and the rules on
particular domains (de-identified, never carries a class, a cap). `reference/register.md` is the
format and `reference/checks.md` says what each check proves.
Rules for drafting:
- **One control per requirement, in the policy's words.** The attestation quotes them back.
- **A control with no check is `not_provable_here`**, with a `why` naming who can prove it (a log
store's retention, a training record). Saying so is a result, not a gap to paper over.
- **Name a persona for every workforce class the policy names**, and one that should be refused
(a partner without an agreement). The refusals are half of what the access check proves.
- **Givens come from the model.** A persona's givens are the model's `#(secure)` givens, named
exactly; `get_context` lists them on each domain, and `GET …/packages/{pkg}/entities` marks each with `secure: true` in its model's `givens`.
Show the register to the user and stop for approval. It is the policy owner's statement of what
the policy means; you drafted it.
### 2. Run the check
From the package directory:
```bash
python3 <skills>/malloy-compliance-review/scripts/compliance.py check \
--environment <env> --package <pkg> --register policies/controls.json --out-dir review
```
It reads the package's classification summary (`GET …/packages/{pkg}/classification`), the model read's gates, `get_context` over
the MCP endpoint (`--mcp`, default `http://127.0.0.1:4040/mcp`) and `GET /api/v0/status`. It probes
every published domain as every persona with a count, queries every source the package does not
publish (read from the `.malloy` files in the current directory, or `--package-dir`) to confirm it
refuses, and profiles every published dimension with aggregates (seconds to a minute for 140
fields). It writes `review/report.json` and `review/report.md`. Against a server that is not on
localhost, `--mcp` is required (a usage error otherwise).
Exit codes: **0** every control holds, **1** a control fails, **2** usage (a bad flag, or a register that is missing, not JSON or has no `controls`),
**3** nothing failed but something could not be checked, **4** an internal error (traceback on
stderr). A probe the server neither answered nor refused (a 400, a 401, a 404 that is not the query
boundary's refusal, or that names the `--model` itself as missing) is never a pass: it needs review, so a typo cannot make a control hold.
`--allow-review` turns 3 into 0 for a pipeline that triages review items by hand.
### 3. Judge what needs a person
Read `review/report.md`. `reference/report.md` explains each section. For each control:
- **Holds**: report it with its evidence. Do not restate it as stronger than the evidence.
- **FAILS**: name the domain and field from the finding, the requirement it breaks, and the
smallest change to the model that would fix it. A failure the server refused at load (the report
says the package is serving a stale model) is already contained: the change was not deployed.
Say both. A `join-bypass` failure is not contained: the server serves that package with a warning.
- **Needs review**: a check that could not decide (free text on a de-identified domain, a probe
the server neither answered nor refused, a cap on a field that was not profiled, a field whose
declared class is lower than what its expression reads). Look at the evidence and say what you concluded and why; if it needs the
policy owner, say so.
- **Not provable here**: say who can prove it.
Do not fix the model in the same step. Report first; a fix is a separate change that goes through
the same review.
### 4. Attest
```bash
python3 <skills>/malloy-compliance-review/scripts/compliance.py attest \
--report review/report.json --out review/attestation.md
```
The attestation lists every control with its status and signature lines for the policy owner.
It is generated from the report, not edited: if a status is wrong, fix the register or the model
and rerun.
### 5. In CI
Run the same `check` on every change to a `.malloy` file, against a Publisher started on the
change's checkout, and fail the build on any non-zero exit: 1 and 4 are never acceptable, and 3
means a person has to look. `docs/compliance-review.md` has a GitHub Actions example. Upload
`review/` as the build's artifact so the evidence travels with the change.
```yaml
- name: Review
run: |
python3 <skills>/malloy-compliance-review/scripts/compliance.py check \
--environment ci --package <pkg> --register policies/controls.json --out-dir review
# add --allow-review to pass on exit 3 (needs review, none failing)
- if: always()
uses: actions/upload-artifact@v4
with: { name: compliance-review, path: review/ }
```
## Rules
- **Aggregates only.** The check never selects a row. Do not add a query that does, and do not
read rows to investigate a finding: the finding's evidence is a count for that reason.
- **The register is approved, not inferred.** Never change an expected outcome to make a check
pass. A check that fails because the register is wrong is fixed by the policy owner.
- **Report, then fix.** A review that edits what it reviews proves nothing.
- **Exact names.** Domain, persona and given names in the register must match the served model;
the check refuses an unknown check name and reports an unknown domain as a failure.