git:20261006.ff75382 to git:20261012.55216ae

10 added, 322 removed. Audit A to A.

What this update changes in the checks

No check changed: every fixed check gives the same answer before and after this update.

A tripped check is a label, not a verdict: read the matched text and the lines below. This week's updates that newly trip a check.

---
name: malloy-notebooks
- description: Create Malloy notebooks (a .malloy file under notebooks/, written as a one-column layout of tiles) for interactive data stories and reports. Use when user asks to "create a notebook", "make a notebook", "data story", or needs to build reports/visualizations that read top to bottom. Legacy .malloynb files are read, not newly authored.
+ description: "Renamed to malloy-reports. A notebook is now a report with layout=linear: a .malloy file with an ## artifact tag, written at the package root. Load malloy-reports instead when the user asks for a notebook, a data story, or a report that reads top to bottom."
---
+
<!--
Copyright (c) Credible Data Inc.
SPDX-License-Identifier: MIT
-->
- # Malloy Notebooks
-
- Files: `notebooks/<slug>.malloy`. Author a notebook as a one-column layout of tiles (next section). Notebooks written as `run:` cells, and existing `.malloynb` files, are still read and served, but a new notebook is never a `.malloynb`.
-
- ## Scope: when to use this skill vs `skill:malloy-analysis-report`
-
- This skill is for **curated notebooks** that live in a package. Interactive parameter widgets render automatically from the `given:` declarations the notebook file declares or imports.
-
- For **ad-hoc reports** generated by an analysis agent, combining queries into a quick narrative artifact, use `skill:malloy-analysis-report` instead. Ad-hoc reports also inherit whatever the source declares.
-
- ## What makes a file a notebook
-
- A **notebook** is a `.malloy` file directly under the package's top-level `notebooks/` directory (not a subdirectory) whose model-level notes include `## artifact { kind=notebook … }`. The artifact tag turns the file into a notebook. A `.malloy` file under `notebooks/` with no artifact tag is a shared include, not a notebook, and is left out of the notebook list. Publisher serves the notebook at `/<env>/<pkg>/notebooks/<slug>` and lists it beside any `.malloynb` files. The tag's `kind=` decides the document's kind, not the folder.
-
- The file is ordinary Malloy: it compiles as one model, it is validated by the same `/compile` call as any model, and its cells run against the whole file (every `import` and `given:` in scope).
-
- ## Author a notebook as a layout of tiles
-
- A notebook is a dashboard with one column. Its artifact tag lists the tiles in reading order, each either a query (`"source -> view"`, always a quoted string) or a prose block (`name { kind=text }`, a bare name), and the file defines the views and the prose:
-
- ```malloy
- ##! experimental.givens
- ##" One category at a time: how it sells across the year and which brands carry it.
- ##| artifact { kind=notebook title="Category review"
- tiles=[
- intro { kind=text },
- "category_tiles -> revenue_trend",
- "category_tiles -> brand_ranking"
- ]
- }
- |##
- import { order_items, products } from "../storefront.malloy"
-
- #(description="Narrow to one product category")
- # label="Category" control=select suggest { source=products dimension=category }
- given: CATEGORY :: filter<string> is f''
-
- ##|(markdown) intro
- ## Category review
-
- Pick a **Category** in the controls above and every chart below re-runs for it.
- |##
-
- source: category_tiles is order_items extend {
- # line_chart
- # label="Revenue by month"
- view: revenue_trend is sales_by_month + { where: category ~ $CATEGORY }
-
- # bar_chart
- # label="Top brands"
- view: brand_ranking is top_brands + { where: category ~ $CATEGORY }
- }
- ```
-
- - **The tag is one line (`## artifact { … }`) or a `##|` … `|##` block** with the same text inside, the form to use once the list is long: one tile per line, `}` on its own line, `|##` closing at the opener's column. Both read, lint and edit the same, and the builder keeps whichever form the file has. `tiles=[…]` is the notebook: tiles render top to bottom in list order, one column wide. `dashboard { columns }` other than 1 has no effect on a notebook, and a `colspan` or `break` on a tile entry is ignored with a warning (`notebook-tile-layout-ignored`).
- - **Quote every query entry.** `"orders_tiles -> headline"` is a string; `orders_tiles -> headline` unquoted does not parse, and the lint says so (`notebook-artifact-unparsed`) rather than reading the file as a cell notebook.
- - **A query tile is a `view:` on a source**, named as `"source -> view"`. Put the views on a `<source>_tiles` extension as thin wrappers (`view: revenue_trend is sales_by_month + { where: … }`): the modelled view keeps its chart tag, the wrapper says which controls the tile answers to and carries the tile's `# label`. A `run:` in a layout notebook is never shown, and the lint reports it (`notebook-layout-run`): define a view, list it in `tiles`, and delete the `run:`.
- - **A prose tile is a `##|(markdown) name` block** listed as `name { kind=text }`. The name is one bare word on the opener line, the body starts on the next line (a heading goes inside the block), and `|##` closes it at the opener's column. Every text entry needs its block, and every named block needs an entry written with `{ kind=text }`: a bare `intro` without it is the query-tile form (it names a `query:` in the file), so a block of that name is not shown.
- - **Place the file in one order: header, imports and givens, then the prose blocks in tile order, then the `<source>_tiles` extension.** Tiles read in `tiles=[…]` order wherever their blocks sit; grouping the blocks before the extension is the convention the builder writes, and it inserts a new block after the last one.
- - **Chart tags go on the view** (`# line_chart`, `# label="…"`), as on a dashboard; the `malloy-dashboards` skill has the tag set and the lint.
- - **Givens work as in any notebook**: declare `given:` above the view that reads it as `$NAME`, with the controls tags shown.
- - **Tiles run through the model query endpoint**, and the notebook also carries cells made from its tiles, so cell runs, `get_context` and notebook chat work on it as on any notebook.
- - **The tag's `kind=` decides the document's kind, not the folder.** Keep notebooks in `notebooks/` and dashboards in `dashboards/`; a file in the other folder works, and the lint notes it (`notebook-other-folder`). An untagged `.malloy` file in `notebooks/` is a shared include.
-
- The rest of this skill (the header rules, `given:`, charts, style) applies to both forms. The sections on cells describe the older form, where the file is a sequence of `run:` statements and `(markdown)` notes: Publisher still reads it, and a person saving it in the Console converts it to a layout.
-
- ## The five authoring rules (cell notebooks)
-
- Rules 1 and 4 hold for both forms; rules 2, 3 and 5 are about `run:` cells and prose notes.
-
- 1. **The header is `##!` flags, `//` comments and unnamed `"` notes (the description), then `## artifact { kind=notebook … }`.** Nothing else goes above the tag. A statement or a tag above `## artifact` is an error, and the notebook is served with that error instead of opening.
- 2. **Prose is a `(markdown)` annotation, and the number of `#` says what it belongs to.** Floating markdown is a cell of its own: `##(markdown) text` (contiguous lines with no blank line between them merge into one cell) or a `##|(markdown)` block, with the body starting on the next line and the `|##` closer at the opener's column. Attached markdown, `#(markdown) text` or a `#|(markdown)` … `|#` block, belongs to the statement below it and renders with that cell. Keep the parentheses: `##| markdown` reads its prose as ordinary tags, and `##|markdown` draws a malformed-route warning. No body line starts with `|##`, and a blank line follows the closer. A `## Heading` line is model tags, not prose: write a heading inside a `##|(markdown)` block.
- 3. **Render tags sit directly above `run:`, with nothing between.** A `#"` directly above the `run:` is its caption; `# bar_chart`, `# label="..."` and the other render tags go in the same block. Attached `#(markdown)` goes above the `#"` caption when both are present, and renders as a header above the result.
- 4. **`given:` uses `NAME :: filter<T> is f''`, bound with `~`, declared before first use.** Put `##! experimental.givens` at the top of the file.
- 5. **Trailing prose is `##(markdown)`, never `#(markdown)` or `#"`.** Both belong to the statement below them, and at the end of a file there is none, so Malloy refuses a `#"` and the lint reports a `#(markdown)` as an error. An `import` or an `export` takes no annotation either, so prose above one is `##|(markdown)`.
-
- ## How the file becomes cells
-
- Cells come from the file's own notes and statements, in file order (imported files add none):
-
- | In the file | Cell |
- |---|---|
- | `##!` lines, `## artifact`, and every `"` note above the tag | the header: no cell |
- | `##(markdown)` lines (contiguous) or a `##|(markdown)` block, after the tag | one markdown cell |
- | `#(markdown)` or a `#|(markdown)` block above a `run:`, `source:`, `query:`, `given:` or `type:` | no cell of its own: it renders with that statement's cell (on a `run:`, a header above the result and above the `#"` caption) |
- | `run:` with its contiguous `#` tag block, including `run: <named query>` | one query cell |
- | `import`, `source:`, `query:`, `given:`, `export { … }`, `type:` | one definition cell **per statement** |
- | a non-`"` `##` note after the tag (`## title=…`) | not a cell |
-
- The unnamed `"` notes above `## artifact` are the notebook's description. Definition cells render folded; only query cells run. Earlier spellings are still read: `##"`, `##|"`, `##(text)` and `##|(text)` notes below the tag are markdown cells, like `(markdown)`. Write `(markdown)`.
-
- ## Compile errors and checking a notebook
-
- Compile the file with `/compile`, at the path it will have and with `"scope": "file"`. A file under `notebooks/` also gets the notebook lint: a file that would not open (a statement above the tag, an `## artifact` tag that does not parse, a prose line the reader cannot place) comes back as an `error` problem naming the line and the fix, and the package warnings carry the same finding after a reload. Read those before you call the notebook done, then open it and look.
-
- ## Multiple Models & Cross-Model Joins
-
- A notebook file compiles with **all** its `import`s in scope, so it can import several `.malloy` models, and a single cell can reference, and **join**, sources from different imported files. Import each model you need at the top; no "re-export" wrapper model is required.
-
- **When the package root holds an `index.malloy`, cells read only what it exports.** A cell over a source `index.malloy` does not export answers 404, even when the notebook imports that source's file. A source the notebook derives from an exported one (`source: mine is customers extend { ... }`) still works. A cell's own source over a raw table (`duckdb.table(...)`) is refused. To use a source in a notebook, add it to the `export { ... }` in `index.malloy`. A package with no `index.malloy` and no legacy `explores` has no such limit.
-
- ```malloy
- import "flights.malloy"
- import "carriers.malloy"
-
- ##(markdown) Flights joined to carriers, across two imported files.
-
- run: flights extend {
- join_one: carriers on carrier = carriers.code
- } -> {
- group_by: carriers.nickname
- aggregate: flight_count
- }
- ```
-
- Imports are file-wide: declare them at the top, before the cells that use them. A query cell runs as a *restricted query* against the already-compiled file, so an `import` inside a `run:` is rejected (`file imports are not permitted in a restricted query`).
-
- ## No Data-Specific Insights in Markdown
-
- **Notebooks must NOT contain findings about current data values.** Data refreshes will make these stale. Use markdown for framing questions and structural narrative, not for stating results.
-
- ```malloy
- ##(markdown) WRONG: will become stale when data refreshes. Revenue spiked 23% in March.
-
- ##(markdown) RIGHT: frames the question and lets the query answer it. How is revenue trending?
- ```
-
- ## A complete cell notebook
-
- One notebook with all three prose forms: floating markdown cells, attached markdown headers, and a `#"` caption.
-
- ```malloy
- ##! experimental.givens
- ##" One category at a time: how it sells across the year and which brands carry it.
- ## artifact { kind=notebook title="Category review" }
- import { order_items, products } from "../storefront.malloy"
-
- ##|(markdown)
- ## Category review
-
- Pick a **Category** in the controls above and every chart below re-runs for it.
- Leave the control empty to read the whole catalog.
- |##
-
- #(description="Narrow to one product category")
- # label="Category" control=select suggest { source=products dimension=category }
- given: CATEGORY :: filter<string> is f''
-
- ##(markdown) How is revenue trending for the selected category?
-
- #(markdown) ### Revenue by month
- #" Revenue by month for the selected category
- # line_chart
- # label="Revenue by month"
- run: order_items -> sales_by_month + { where: category ~ $CATEGORY }
-
- ##(markdown) Which brands are behind those numbers?
-
- #" The eight brands with the most revenue in the selected category
- # bar_chart
- # label="Top brands"
- run: order_items -> top_brands + { where: category ~ $CATEGORY }
-
- ##(markdown) Last, the best-selling products, defined once as a named query and then run.
-
- query: top_products_in_category is order_items -> top_products + { where: category ~ $CATEGORY }
-
- #(markdown) ### Best sellers
- run: top_products_in_category
- ```
-
- The `##|(markdown)` block and each `##(markdown)` line are floating markdown cells. The two `#(markdown)` lines are attached: each renders as a header with the `run:` below it, and on the first the header sits above the `#"` caption. The second `run:` has only a caption. The `given:` and the `query:` are definition cells, and each `run:` is a query cell. The header ends at `## artifact`.
-
- ## Interactive Parameters (`given:`)
-
- Interactive parameters are common and important. Add them to most notebooks. **Use `given:`.** Never add a `#(filter)` annotation, even if the model you are extending already has some: it is deprecated and the notebook does not render it (see below).
-
- A notebook's parameter surface comes from the `given:` declarations the file declares or imports. Publisher's notebook UI renders a Filters panel above the notebook: declared givens become parameter inputs, the values a user sets are forwarded to Malloy's runtime, and every query cell re-executes with those values applied. A `given:` is read by the cells below it, so declare it before the first cell that uses it.
-
- The widget for each parameter follows the given's declared Malloy type (`string`, `string[]`, `number`, `boolean`, `date`/`timestamp`, `filter<T>`); see `docs/givens.md` for the full type table. A `#(description="...")` annotation on the given renders as helper text under its input. It draws a cosmetic `malformed-route` warning that you can ignore; see `docs/givens.md` § Annotations. A control tag (`control=select suggest { source=… dimension=… }`) goes on the `given:` as shown in the complete notebook above.
-
- `malloy-model` § Access Control covers the syntax and the `#(authorize)`/`#(access_filter)` gating story built on top of givens.
-
- ## Legacy: `#(filter)` annotations
-
- `#(filter)` is deprecated, and the notebook no longer renders controls for it. A model that uses `#(filter)` is not filterable from a notebook, and a cell whose source has a `required` `#(filter)` fails, because nothing in the notebook can supply the value. Never add a `#(filter)` annotation. To make such a model filterable, migrate it to givens: `malloy-model` § Legacy: reading an existing `#(filter)` model has the mapping, including `required`.
-
- ## Notebook Style: Iterative Analysis
-
- **Build a story.** Each query should reveal something that motivates the next query. Start broad, then drill into what's interesting. Frame questions in markdown, let queries answer them.
-
- **Pattern:** Question, Query, Next Question, Drill
-
- ```malloy
- ##(markdown) How is revenue trending?
-
- # line_chart
- run: orders -> { group_by: order_month; aggregate: revenue }
-
- ##(markdown) What's driving the biggest changes?
-
- # bar_chart
- run: orders -> { group_by: category; aggregate: revenue; order_by: revenue desc; limit: 10 }
-
- ##(markdown) How does the top category break down?
-
- run: orders -> { aggregate: order_count, avg_order_value; where: category = 'Electronics' }
- ```
-
- **When to use other styles:**
- - **Dashboard with filters:** User asks for "filterable dashboard". Build a dashboard (the `malloy-dashboards` skill) rather than a notebook.
- - **EDA/summary:** User asks for "overview". Run views like `summary`, `by_month`, `by_category` in sequence.
-
- ## View Refinement
-
- To add `limit`, `where`, or other options to an existing view, use `+`:
-
- ```malloy
- run: source -> my_view + { limit: 15 }
- run: source -> my_view + { where: status = 'active', limit: 10 }
- ```
-
- ## Template
-
- ```malloy
- ##! experimental.givens
- ##" [One-line description of the notebook.]
- ##| artifact { kind=notebook title="[Title]"
- tiles=[
- intro { kind=text },
- "main_source_tiles -> broad_tile",
- "main_source_tiles -> drill_tile"
- ]
- }
- |##
- import "model.malloy"
-
- ##|(markdown) intro
- [Framing question: what are we trying to understand?]
- |##
-
- source: main_source_tiles is main_source extend {
- view: broad_tile is [broad view]
- view: drill_tile is [drill view]
- }
- ```
-
- Define each listed view on the `_tiles` extension (thin wrappers over the modelled views), and compile with `"scope": "file"`.
-
- ## Editing in the Console
-
- A person can edit a `notebooks/*.malloy` notebook in the Console (an **Edit** button; **New** on the
- package page starts one). A layout notebook is edited in place: tiles are reordered, added and
- removed, text blocks and the tag are rewritten, and everything else in the file survives byte for
- byte. A cell notebook opens converted and unsaved, and Save writes the conversion: each `run:`
- becomes a `view:` on a `<source>_tiles` extension appended to the file (a `query:` used by exactly
- one run is folded into its view), each prose note becomes a `##|(markdown) text_N` block, and
- definitions, imports, givens and comments stay where they were. The first Save asks before it
- rewrites the file ("Convert this notebook?"); Cancel writes nothing, and once saved the builder
- cannot undo the conversion, so the file's history in its repository is the way back. Write a cell
- notebook so it converts:
-
- - Every `run:` is `<source> -> <view or query body>` with a named source. An inline `extend` before
- the arrow, a source that is not a name, or a refinement (`q + { … }`) of a multi-stage query is
- refused, and so is a run that resolves through more than ten named queries.
- - A chart line the editor does not model (for example `# bar_chart { size=spark }`) is kept, and the
- Viz type picker is disabled for that tile with the reason shown.
- - The editor opens a notebook **read-only**, saying why, when it cannot place cells one by one:
- two statements or notes on one line, text after a block closer, a comment straddling two cells, a
- lone carriage return (use LF or CRLF), a statement above the `## artifact` tag, a statement no
- cell can hold, or a tag value Malloy cannot read, such as a malformed date literal
- (`@2024-13-01`).
- - A save whose text declares a real `#(authorize)` or `#(access_filter)` gate outside prose is
- refused with a 400. Put gates in a model file the notebook imports.
-
- Viz type choices are From the view, Table, Line, Bar, Big value, Scatter, Shape map and Segment map.
- Every choice is listed; one the view cannot render is disabled with its reason beside it.
-
- ## Existing `.malloynb` files
-
- A `.malloynb` notebook (cells delimited by `>>>markdown` and `>>>malloy`) is a deprecated format. Publisher keeps read-only support so existing notebooks still open and run, and the bundled examples no longer ship one. **Do not create a new `.malloynb`.** To give a story a new home, write a `.malloy` notebook as above. Two things to know when maintaining an existing one:
-
- - **Compile errors in `.malloynb` files are NOT shown in the IDE linter**, and the notebook lint above does not cover them. You only see errors when cells are executed, so test queries in the model first.
- - Imports are notebook-wide and belong in a setup cell at the top, never inside a `run:` cell.
-
- ## Common Mistakes
-
- | Mistake | Fix |
- |---------|-----|
- | `view_name { limit: 10 }` | Use `+`: `view_name + { limit: 10 }` |
- | `# currency` on non-money | Only use `# currency` for monetary values |
- | A statement or a `#` tag above `## artifact` | Only `##!` flags, `//` comments and unnamed `"` notes go in the header. Move the statement below the tag. |
- | `## Heading` for a section title | That line is model tags, not prose. Write the title inside a `##|(markdown)` block. |
- | `##| markdown` or `##|markdown` (no parentheses) | Write `##|(markdown)`. |
- | `##(markdown)text` or `##|(markdown)text` (no space after the route) | Malloy drops the note. Write `##(markdown) text`. |
- | `##"`, `##|"`, `##(text)` or `##|(text)` prose below the tag | Still read as markdown cells; prefer `##(markdown)` or `##|(markdown)`. |
- | Text on the opener line after `##|(markdown)` | Only a name may follow it, and a notebook has no use for one. Put the text on the next line. |
- | `#(markdown)` above an `import` or `export` | Those take no annotation. Write `##|(markdown)`. |
- | A `#` tag separated from its `run:` by another statement | Render tags sit directly above the `run:`. Move the tag. |
- | `#(markdown)` or `#"` as the last thing in the file | Trailing prose is `##(markdown)`; both need a statement below them. |
- | `given:` used before it is declared | A cell reads only the givens declared above it. Move the `given:` up. |
- | `##|(markdown) name` block in a cell notebook | A name means something only where a `tiles=[name { kind=text }]` entry lists the block (a dashboard or a layout notebook). In a cell notebook, drop it: `##|(markdown)`. |
- | `run:` in a notebook that lists `tiles=[…]` | Never shown. Define a `view:`, list `source -> view` in `tiles`, delete the `run:`. |
- | `tiles=[intro { kind=text }, orders_tiles -> headline]` (entry not quoted) | Every `source -> view` entry is a quoted string: `"orders_tiles -> headline"`. A text entry is a bare name followed by `{ kind=text }`. Unquoted, the tag does not parse and no tile is read. |
- | A bare `intro` in `tiles` for a `##|(markdown) intro` block | A bare name is the query-tile form for a `query:`. Write `intro { kind=text }`. |
- | `colspan` or `break` on a tile entry of a notebook | Ignored (a notebook is one column) and reported as `notebook-tile-layout-ignored`. Remove them. |
- | `#(filter) {"type": "Star"}` (JSON-blob form, on a dimension) | Unsupported legacy syntax, and `#(filter)` in any form is deprecated. Declare a `given:` on the model's source instead. |
- | Data-specific insights in markdown | Don't write "Revenue grew 23%." Frame questions instead. Data refreshes will make findings stale. |
- | `import` inside a `run:` cell | Imports are file-wide, put them at the top. An in-query import is rejected: `file imports are not permitted in a restricted query`. |
- | Cross-model join inside a single ad-hoc report cell | A `skill:malloy-analysis-report` cell renders against a single model, so a cross-model join there won't render. Build a published notebook (which runs against the whole file) for a cross-model join. |
+ # Malloy notebooks: renamed
- ## Best Practices
+ This skill was merged into `skill:malloy-reports`. Read that skill instead; it holds everything this one did.
- 1. Start with markdown framing the question (`##|(markdown)` for the title block, `##(markdown)` for each question)
- 2. Import each model the notebook needs at the top; cells can reference and join sources across all imports (see Multiple Models & Cross-Model Joins)
- 3. Each query should follow from what the previous one could reveal
- 4. One query per cell
- 5. Charts render only the FIRST aggregate
- 6. Add interactive filters to most notebooks with `given:` declarations (see Interactive Parameters)
- 7. Never include data-specific findings in markdown. Frame questions, let queries answer
- 8. `/compile` the file before saving, then reload the package and read its warnings
+ What changed, in one place:
- ## Done
+ - A notebook is a **report** with `layout=linear`. A dashboard is the same report with `layout=grid`.
+ - A report can live anywhere in the package. Write a new one at the package root, not in `notebooks/`.
+ - Write `layout=linear` in the `## artifact` tag. `kind=notebook` is the deprecated spelling.
+ - A `# drill { to=… }` target is the report's package path without `.malloy`.
- Step complete. Output: a `notebooks/<slug>.malloy` file with an `## artifact { kind=notebook tiles=[…] }` tag.
+ This name stays only so that hosts and installs that still ask for `malloy-notebooks` find the new skill.