git:20260929.c5cf71d to git:20261008.60767d1

143 added, 150 removed. Audit A to A.

What this update changes in the checks

No check changed: every fixed check gives the same answer before and after this update.

A tripped check is a label, not a verdict: read the matched text and the lines below. This week's updates that newly trip a check.

---
name: publisher-release
description: Cut a Malloy Publisher release, and write the RELEASE_NOTES.md entries a release ships. Use when asked to release, cut a release, ship a version, or publish Publisher to npm/Docker — and when a change needs a release note, or you are deciding whether it does and what version to stamp on it.
---
<!--
Copyright (c) Credible Data Inc.
SPDX-License-Identifier: MIT
-->
# Releasing Publisher
A release is one `workflow_dispatch` of `Release (NPM + Docker)`
- (`.github/workflows/release.yml`). Everything else in this skill exists because
- work has to land on `main` **before** that dispatch, or be written **after** it
- completes — and the step that is always skipped is the last one.
+ (`.github/workflows/release.yml`), followed by one pull request: the
+ `release/v<version>` branch the release published from, merged back into
+ `main`. That branch already carries the version bump and the stamped release
+ notes, so **you never create a branch or stamp anything by hand**. You open the
+ PR from the branch the workflow pushed.
Read `.github/workflows/CONTEXT.md` (path from the repo root) before acting. It carries the publishing rules that are not guessable from the
YAML, and it is the authority when this file and it disagree.
## Release notes: when to write one, and what number goes on it
`RELEASE_NOTES.md` is not written at release time. **The PR that changes the
behaviour writes the note**, in the same PR, as a `## [Unreleased]` section —
which is the only way it gets written by someone who knows what changed. The
- release then carries it to users on its own: `gh-release` appends every
- `[Unreleased]` section to the release page, then pushes a branch stamping those
- headings with the version that shipped them and prints a link to open it as a PR.
- Writing the section is the whole job; opening and merging that PR is the one
- thing the release cannot do for itself.
+ release then carries it to users on its own: `prepare` stamps every
+ `[Unreleased]` heading with the release's version on the release branch, and
+ `gh-release` puts those sections on the release page. Writing the section is
+ the whole job; merging the release branch back is the one thing the release
+ cannot do for itself.
### Does this change need one?
`gh-release` already attaches an auto-generated "What's Changed" list of merged
PRs. **That list is sufficient for a routine patch**, so a note is not a per-PR
chore. Write one when the PR list would leave a reader unable to act — when a
user or operator has to *do* something, or would otherwise draw a wrong
conclusion from an unchanged-looking system:
- A **breaking change**, or anything needing a migration step. Mark the heading
`(BREAKING)`.
- A **new or removed API field, endpoint, or response shape** — anything that
breaks a strict generated client, or that consumers should move onto.
- A **deprecation**: what still works, for how long, and what to move to.
- **Changed meaning of an existing signal** — a metric label, a counter, an
error code. A value that keeps its name and changes its basis is the case
most worth writing, because nothing else will surface it.
- **A silent failure that is now visible**, or a bug whose symptom users have
been living with. Say who was affected and how to tell.
- A **new capability with a cost or a limit** worth knowing before adopting it.
Skip it for refactors, tests, docs, CI, and internal changes with no observable
effect. When unsure, ask whether a reader upgrading blind could be surprised. If
not, the PR list covers it.
Write it for the person upgrading: what changed, what breaks, what to do. The
existing sections are the house style — prose, not bullets-only, and specific
about the failure mode.
### What number goes on it
Three states, in order:
1. **`## [Unreleased] — <what changed>`** while the change sits on `main`
unreleased. This is what an authoring PR writes. Always.
2. **Amend that same section in place** if a follow-up PR lands before the
release and changes the same behaviour. Do not add a second section — the two
ship together as one story, and the reader has never seen the first version.
#1024 (pre-aggregation off by default) and #1030 (on by default) are one
section for exactly this reason.
3. **`## [<version>] — <what changed>`** once a release has shipped it. **CI
- writes this** — `gh-release` pushes a `release-notes-stamp-<version>` branch
- after the release is cut, and a human opens and merges it. You do not stamp by
+ writes this** — `prepare` stamps it on `release/v<version>`, and it
+ reaches `main` when that branch's PR merges (step 6). You do not stamp by
hand, and you do not guess the number in advance.
Once a section is stamped, it is history and does not get rewritten. A follow-up
that changes that behaviour opens a **new** `[Unreleased]` section referencing
the shipped version by number, the way the build-failures section names 0.0.245
and 0.0.246 when describing what 0.0.247 changed.
- Never write a version number into a note yourself *in advance*. Release numbers
- are assigned at dispatch time and a release can fail, so a section stamped early
- can name a version that does not exist — which is exactly why the stamp runs
- after `gh release create` succeeds and not before. Stamping a version that has
- **already** shipped is a different act and sometimes necessary, because the
- stamp branch needs a human to open and merge, and can be missed; step 3 covers
- it, including why the obvious command for it is destructive.
+ Never write a version number into a note yourself. Release numbers are assigned
+ at dispatch time and a release can fail. A failed release's stamp is harmless
+ only because its branch is never merged; the next dispatch cuts a new branch
+ from `main`, which still reads `[Unreleased]`.
## The versioning policy
**`0.MINOR.PATCH` for every published package, while pre-1.0.** It is the policy,
not yet the state: `skills` follows it, `sdk`/`app`/`server` are still on `0.0.x`
until `0.2.0` is cut, `create-malloy-package` versions on its own `0.0.x` line, computed at release time, and the Python client
declares `0.1.0` — sharing `0.1.x` with `skills`. Nothing enforces the *shape* of
the number for any of them — skills and create-malloy-package resolve their next
version from npm `latest` at release time, and moving either onto a new minor
line is a judgement call made with an explicit `-f version=`, not something CI
demands.
- **MINOR = a breaking change. PATCH = everything else.** A breaking change is
one that turns a package which loaded or built yesterday into one that does
not: a refused annotation, a refused `#@ persist` combination, a load-time
failure on syntax that previously passed. It also covers a removed endpoint or
response field, like 0.0.242's `/pages` → `/data-apps` rename.
- **This is what makes a range usable.** `^0.0.250` resolves to
`>=0.0.250 <0.0.251` — exactly one version — and `~0.0.250` to
`>=0.0.250 <0.1.0`, a range whose upper half does not exist. So on `0.0.x` no
consumer can say "patches yes, breaking no". `^0.2.0` resolves to
`>=0.2.0 <0.3.0`, which they can. (On any `0.x` line `^` and `~` are
identical — both stop at the next minor.)
- **No `1.x`.** Publisher is preview, so a major is a deliberate decision made
with an explicit `-f version=`; there is no `major` option on the dispatch. If
it ever happens, note `@malloy-publisher/sdk@1.0.1` **is** published (an old
off-line publish `latest` moved off long ago) while `1.0.0` is free, and `app`
and `server` have no `1.x` at all — so a 1.x line has to start at `1.0.0` and
skip `1.0.1`, or start at `1.0.2`.
- **The `sdk`/`app`/`server` train has NOT moved yet.** Its `latest` is still
`0.0.250` and the policy takes effect at its first minor release, intended to
be **`0.2.0`** — not `0.1.0`, because `skills` already occupies `0.1.x` and two
trains sharing a minor invites reading one for the other. Until that release is
cut, a default dispatch derives `0.0.251`, `0.0.252`, … and that is correct.
**Do not "correct" it with an explicit version.** `skills` is already on
`0.1.x` and already follows the policy.
Read a release note's own claims rather than its heading when deciding. A section
can be titled `(BREAKING)` while its bullets say *"this cannot affect an existing
package"*, and the reverse happens too. Propose the level and get the
maintainer's call rather than inferring it from the diff alone — a narrow break
may still be judged patch-worthy, and that judgement is theirs.
### How to bump
`prepare` derives the version from **npm's `latest`**, so a routine release needs
no version input at all:
```bash
# a patch — the default
gh workflow run release.yml --repo malloydata/publisher --ref main
# a breaking release
gh workflow run release.yml --repo malloydata/publisher --ref main -f bump=minor
```
The floor is `max(npm latest, main declared)` by version sort, and it **fails
closed**: if the registry will not answer, `prepare` refuses rather than falling
back to the file. An explicit `-f version=` still wins outright over `bump`, and
is what you want for a major, for a prerelease, or to name the number exactly.
Two things worth knowing about the derived path:
- The chosen level is applied **once**. If that number's release branch or tag
already exists, the walk past it is by *patch* — so `bump=minor` colliding with
an existing `0.3.0` gives `0.3.1`, not `0.4.0`.
- `bump=minor` is computed off npm's current `latest`, which is not always the
number the policy wants. Off a `0.0.x` floor a minor bump lands on `0.1.0` —
the line `skills` occupies. That is why the pending move to `0.2.0` needs an
explicit version rather than `bump=minor`.
An earlier version of this section said *"`prepare`'s default version is not
trustworthy — always pass `-f version=`"*. That was true and is no longer:
#1037 changed the floor to `max(npm latest, main declared)` with fail-closed
behaviour, so the hole it warned about — a default run walking up from `main`'s
stale number, finding a gap below `latest`, and moving the dist-tag **backwards**
with nothing failing — is closed. Passing an explicit version is now a choice,
not a precaution.
## The version trains
| Packages | Version | Decided by | Missing bump caught by |
| --- | --- | --- | --- |
- | `sdk`, `app`, `server` | lockstep | `release.yml` itself, on a `release/sdk-<v>` branch | n/a — the release sets it |
+ | `sdk`, `app`, `server` | lockstep | `release.yml` itself, on a `release/v<v>` branch | n/a — the release sets it |
| `skills` | its own line, `package.json` carries a fixed `0.0.0-dev` placeholder | release time, from npm's own state (`scripts/independent-version.mjs`); you, for a minor | n/a — nothing is committed ahead of time to forget |
| `create-malloy-package` | its own line, same placeholder | release time, the same way | n/a, same reason |
| `malloy-publisher-sdk` (Python) | its own line | you, by hand, on `main` | `python-sdk.yml` PR check — but see below |
**Skills and create-malloy-package no longer have a version to forget.** Nothing
is bumped ahead of a release: both packages' `package.json` declare `0.0.0-dev`
in the repo, and what actually publishes is decided at release time by
`scripts/independent-version.mjs`, called from `scripts/publish-packages.sh`, and
written into the manifest at publish time. So the pre-release hand-audit that
used to live in step 2 below (comparing `main`'s declared version against npm)
no longer applies to either package — there is nothing on `main` to compare.
The two are decided differently:
- **skills** publishes when its published content changed since npm `latest`'s
`gitHead` — a diff of `skills/`, `packages/skills/`, `bun.lock` and the root
`package.json`, excluding `skills/README.md` and
`packages/skills/src/*.spec.ts`. Changed publishes one patch above the
highest published plain version; unchanged skips.
- **create-malloy-package** publishes on every non-prerelease release,
unconditionally, because it bakes the server's npm `latest` into every
workspace it scaffolds — a release changes what it ships even when its own
directory did not. The only skip is a re-run of the same release, detected by
its published `publisherServer` field already matching.
The publish version is one patch above the HIGHEST published plain version
(`npm view <pkg> versions --json`), not `latest` plus one — a `latest` dist-tag
rolled back by hand after a bad release must not make this recompute a version
that's already published. `latest`'s `gitHead` stays the content-diff baseline
above; only the version arithmetic reads the full versions list.
A hand dispatch of either child workflow with no `version` input publishes one
patch above the highest published version, the same guard the old PR check
used to enforce, now run once at dispatch time. **A minor or major bump is a
hand dispatch with `-f version=`.** After a hand-dispatched skills minor,
hand-dispatch the scaffolder too — it depends on skills' version — or wait for
the next release, which republishes the scaffolder anyway.
One caveat: the **Python** check has nothing to *catch* until the first publish
lands, because `malloy-publisher-sdk` is not on PyPI at all and a project-level
404 is its pass. (It can still go red on an unreadable `pyproject.toml`, a
version that is not `major.minor.patch`, or a registry that answers neither 200
nor 404.) It starts enforcing for real the moment a version is up there, so treat
the Python version as unenforced only until then.
**Its PyPI publish is paused, though.** The `publish_pkg python-client` line at
the end of `scripts/publish-packages.sh` is commented out, so no release
dispatches `python-sdk.yml` today. Before restoring that line, read *The first
PyPI publish* in `.github/workflows/CONTEXT.md`: `PYPI_TOKEN` has to be
account-scoped for a first upload (a project-scoped token cannot exist for a
project that does not), the name has to still be free, and `0.1.0` is what
ships — PyPI filenames can never be reused, so move the version before that
run if it is going to move at all. A failure there is cheap: it is dispatched
last, nothing depends on it, and re-running the job skips whatever already
published.
`main`'s `packages/sdk/package.json` used to lag npm permanently. It no longer
- should: the post-release stamp PR resets those three files to the version that
- shipped. But that PR needs a human to merge it, so **`main` is truthful only if
- the last one landed** — which is why `prepare` still takes the max of npm and the
- file rather than trusting either. Ask npm when you want to know what is
+ does: merging the release PR (step 6) brings those three files to the version
+ that shipped, and CI's `Release sync` check fails while a shipped
+ release's PR is unmerged. `prepare` still takes the max of npm and the file for
+ its floor rather than trusting either. Ask npm when you want to know what is
published.
## Order, and why each step is where it is
### 1. Establish where things actually stand
```bash
git fetch --tags origin
npm view @malloy-publisher/server version # the real current version
git log --oneline "v$(npm view @malloy-publisher/server version)..origin/main"
```
That commit list is what this release ships. If it is empty, there is nothing to
release.
### 2. Skills and create-malloy-package need no pre-flight bump check
Nothing is committed ahead of time for either package, so there is nothing to
confirm here: `scripts/independent-version.mjs` reads npm's own state at release
time and decides publish/skip/version from it, not from what `main` declares.
Skip straight to *Sanity-check the notes* below.
If you want to see what the release will decide before dispatching, ask npm the
same questions the release does:
```bash
npm view @malloy-publisher/skills dist-tags.latest
npm view "@malloy-publisher/skills@$(npm view @malloy-publisher/skills dist-tags.latest)" gitHead
git diff --quiet "<that gitHead>" origin/main -- skills/ packages/skills/ bun.lock package.json \
':!skills/README.md' ':!packages/skills/src/*.spec.ts' && echo unchanged || echo changed
npm view @malloy-publisher/create-malloy-package dist-tags.latest
npm view "@malloy-publisher/create-malloy-package@$(npm view @malloy-publisher/create-malloy-package dist-tags.latest)" publisherServer
```
`changed` (or any diff not `unchanged`) means skills will publish this release;
`unchanged` means it skips. create-malloy-package publishes regardless, unless
its `publisherServer` already equals the version this release is about to ship —
that only happens on a re-run.
Also check the Python client. Its PyPI publish is paused (work in progress): the
release does not dispatch python-sdk.yml until the `publish_pkg python-client`
line at the end of scripts/publish-packages.sh is restored, so this reading does
not change between releases for now. Its version is in `pyproject.toml`, not a
`package.json`, and PyPI answers 404 for the whole project until the first upload
lands — so "PyPI: 404" here is the expected reading today and NOT a reason to
skip the comparison next time.
```bash
printf 'python-client: PyPI %s, main %s\n' \
"$(curl -sS --max-time 20 https://pypi.org/pypi/malloy-publisher-sdk/json \
| python3 -c 'import json,sys; print(json.load(sys.stdin)["info"]["version"])' \
2>/dev/null || echo 404)" \
"$(python3 -c 'import tomllib; print(tomllib.load(open("packages/python-client/pyproject.toml","rb"))["project"]["version"])')"
```
#### The scaffolder's server pin is derived now — do not set it by hand
`create-malloy-package` pins the server its generated workspaces run
(`SERVER_VERSION` in `packages/create-malloy-package/src/scaffold.ts`). **That pin
is substituted at publish time** from npm's current `@malloy-publisher/server`
`latest`, in the runner's working tree only, so the value committed to the repo is
a dev default and there is nothing to bump before a release. Its publish job then
reads the line back, confirms it matches the registry, and confirms that server
still documents `--host`.
This is what 0.0.250 was about, and it is worth knowing why the old instruction
existed: the pin was hand-maintained and the publish job refused to ship unless it
equalled npm's `latest`, so a release that forgot it went red 25 minutes in, after
`skills` had already published. Setting it in the pre-release PR is now wrong
rather than merely unnecessary — the substitution overwrites it, and a committed
value that happens to differ is not a problem to fix.
The derivation is only correct because `publish-packages` waits for `publish-npm`.
It did not until 0.0.250, and on `needs: prepare` alone this job races the server
publish, so the substituted value would be a release behind and every generated
workspace would pin the previous server. **If that `needs:` is ever narrowed back
to `prepare` alone, this breaks silently rather than loudly.**
The `needs:` is not enough on its own, because npm can take minutes to show a
version after it publishes. In 0.7.0 `latest` read 0.7.0 seven minutes after
`publish-npm` finished. So `publish-packages.sh` also waits, up to 15 minutes, for
the server's `latest` to read the new version before dispatching the scaffolder.
It then passes that version to the scaffolder as its `server_version` input,
because another runner can still read the old `latest` for a few minutes; the
scaffolder waits until its own `latest` matches and refuses to pin anything else.
If the wait gives up, neither the scaffolder nor python-client was dispatched.
Once `latest` reads the new version, re-run the `publish-packages` job (Re-run
failed jobs), not the whole release.
- ### 3. Sanity-check the notes
+ ### 3. Check the last release merged back, and read what ships
- `gh-release` reads `RELEASE_NOTES.md` itself: it appends every `## [Unreleased]`
- section to the release page, then pushes a branch stamping those headings. There
- is nothing to paste, so this step is a read and one check.
+ `prepare` stamps `RELEASE_NOTES.md` itself and `gh-release` reads it back, so
+ there is nothing to paste. Whether the last release merged back is CI's job:
+ `release-sync.yml` compares npm's `latest` with `packages/sdk/package.json`
+ on every PR, every push to `main`, and after every release run. Read its
+ result rather than repeating the comparison:
```bash
- grep -n '^## \[Unreleased\]' RELEASE_NOTES.md
- node scripts/release-notes.mjs extract | head -40
- # a previous release's stamp, still waiting on a human
- git ls-remote --heads origin 'refs/heads/release-notes-stamp-*'
+ gh api repos/malloydata/publisher/commits/main/check-runs \
+ --jq '.check_runs[] | select(.name == "Release sync") | "\(.conclusion) \(.html_url)"'
+ # the narrative this release will stamp and ship
+ git fetch origin
+ NOTES="$(mktemp)" && git show origin/main:RELEASE_NOTES.md > "$NOTES"
+ RELEASE_NOTES_FILE="$NOTES" node scripts/release-notes.mjs extract | grep '^## '
```
- What you are checking is that the sections listed are the ones this release
+ **If the check failed, do not dispatch.** The previous release's PR has not
+ merged, so `main` still reads `[Unreleased]` for sections that release already
+ shipped, and this release would stamp and publish them again. The check's error
+ names the version; finish step 6 for it, which is also the only thing that turns
+ the check green. `release.yml` does not run this check itself, so nothing stops
+ a dispatch but you.
+
+ What you are checking in the extract is that the sections listed are the ones this release
actually ships. A section describes work merged to `main`, so anything sitting
there goes out with this release whether or not it was written for it. Nothing
listed is fine and common — the generated PR list carries a routine patch.
If a section is present that should **not** ship yet, the work behind it is
already on `main` and the note is telling the truth; the fix is a release, not
an edit.
- **A previous release's stamp left unmerged is the one that will bite you**, which
- is what the third command is for. The stamp cannot land itself: `main` requires a
- pull request, so the release pushes the branch and stops. Until someone opens and
- merges it, its sections still read `[Unreleased]` and *this* release re-appends
- the previous release's narrative to its own page — and `main` still declares the
- version before last. The second is harmless (`prepare` takes the max of npm and
- the file), the first is not.
-
- `git ls-remote` rather than `gh pr list --search`, deliberately. That search is
- full text, not title-scoped, so it matches any PR whose body merely discusses the
- stamp — including release-prep PRs, which all explain the mechanism. The branch
- name is exact, and it also catches a branch that was pushed but never opened,
- which the PR search cannot see at all. If you want the PR too:
- `gh pr list --repo malloydata/publisher --search '"stamp" in:title' --state open`.
- The commit title is `chore(release): stamp <version> on main`; 0.0.250 used
- `docs: stamp the release notes shipped in <version>`, so search on `stamp` rather
- than either full phrase.
-
- So: if the branch is there, open and merge it before dispatching.
-
- #### If the branch is gone too
-
- Then stamp by hand — the one case where you write a version number into a note
- yourself, because the release that shipped it is already public and its number is
- no longer a guess. **Scope it first.** The bare command is destructive:
-
- ```bash
- # DESTRUCTIVE — rewrites EVERY [Unreleased] section in the file
- node scripts/release-notes.mjs stamp <version>
- ```
-
- Unscoped it stamps every `[Unreleased]` heading, including the ones *this*
- release is about to ship. Those get labelled with a version that never contained
- them and vanish from the next release's page — the exact failure `--titles` was
- added to prevent. And you cannot just pass `--titles`: that file lives in
- `$RUNNER_TEMP` and died with the run.
-
- Re-establish the scope by comparing the shipped release page against the file.
- The page carries two `## ` headings of its own — `## Release v<version>` from the
- job's header and `## What's Changed` from `--generate-notes` — so filter those or
- you are comparing 8 lines against 6 and never get a match:
-
- ```bash
- gh release view "v<version>" --repo malloydata/publisher --json body -q .body | grep '^## ' | grep -vE '^## (Release v|What'"'"'s Changed)' | sort
- node scripts/release-notes.mjs extract | grep '^## ' | sort
- ```
-
- Both then print stripped titles, so the sets are directly comparable. If they are
- **identical**, every remaining section belongs to that release and the unscoped
- stamp is safe. If the file has extras, write just the shipped headings — verbatim
- from `RELEASE_NOTES.md`, `[Unreleased]` marker included — into a file and pass
- `--titles <that file>`. Either way, commit on a branch and open a PR.
-
- The lost branch carried the version reset too, so add it to the same commit —
- this one has no destructive edge, and it is a no-op if `main` already declares
- the version:
-
- ```bash
- node scripts/set-version.mjs <version> \
- packages/sdk/package.json packages/app/package.json packages/server/package.json
- ```
-
- 0.0.249 is the worked example, and it is why this section exists: it put all six
- of its sections on its release page and stamped none of them, because the step
- still pushed straight to a protected `main`. The unscoped stamp was safe there
- only because the two sets matched once filtered — six sections on the page, the
- same six in the file.
-
### 4. Dispatch
`prepare` derives the version from npm's `latest` and fails closed if the registry
will not answer, so a routine patch needs no input. See *How to bump* above for
the policy behind the choice.
```bash
# a patch
gh workflow run release.yml --repo malloydata/publisher --ref main
# a breaking release
gh workflow run release.yml --repo malloydata/publisher --ref main -f bump=minor
# an exact number: a major, a prerelease, or a line change like 0.0.x -> 0.2.0
gh workflow run release.yml --repo malloydata/publisher --ref main -f version=<next>
```
**Do not merge to `main` while it runs.** `publish-packages` aborts if `main`
moves under a watched path mid-release. A `RELEASE_NOTES.md`-only merge is not
- watched, but the window is short — just wait.
+ watched, but the window is short — just wait. **Do not push to
+ `release/v<version>` while it runs either**: `npm-sdk.yml` and
+ `docker-image.yml` check it out by name, so a push mid-run can publish two
+ different commits under one version.
### 5. Verify what actually shipped
A green tick is not evidence. Read the run's **job summary**, which names each
independently-versioned package as published or skipped and why.
```bash
npm view @malloy-publisher/server version
npm view @malloy-publisher/skills version
gh release view "v<version>" --repo malloydata/publisher
```
- ### 6. Open and merge the stamp
+ ### 6. Open the release PR, and get it merged
- Half verification, half the one task the release genuinely cannot finish itself.
- Two things `gh-release` did, both visible without leaving the run:
+ This is the one step the release cannot finish itself, and as the agent running
+ this skill **you open the PR**. The branch already exists and already holds
+ everything: `prepare` committed the three `packages/{sdk,app,server}/package.json`
+ versions and the stamped `RELEASE_NOTES.md` headings to `release/v<version>`
+ as one commit. Do not create another branch and do not stamp anything by hand.
- - The release page carries the narrative under the generated header. The job
- logs `attached N narrative section(s)`, or `no [Unreleased] sections` when
- there were none.
- - A `release-notes-stamp-<version>` branch is pushed, and the job summary's
- *Release notes and version* section carries a compare link to open it as a PR.
+ Open it once the run has finished, for any release whose sdk reached npm
+ (`npm view @malloy-publisher/sdk version` reads it). That includes one whose
+ `gh-release` then failed: it shipped, and `release-sync.yml` stays red until
+ this PR merges. A release that never reached npm has nothing to merge back:
+ leave its branch alone (the next dispatch skips past it) and dispatch again.
+ A prerelease or a `+build` version never gets a PR.
- **That branch carries two things**, despite its name: the stamped
- `RELEASE_NOTES.md` headings, and `main`'s three `packages/{sdk,app,server}/
- package.json` files reset to the version that shipped. It no longer touches
- `skills` or `create-malloy-package`: both carry a fixed `0.0.0-dev` placeholder,
- decided at release time from npm's own state, so there is nothing left for this
- step to move ahead of. The summary line names both. Either can legitimately be
- zero — nothing to stamp, or `main` already declaring that version — so when both
- are zero no branch is pushed, and the summary says so.
+ **First confirm the branch is there and holds only the release commit**, against
+ `origin/main`:
- The name is unchanged on purpose: it is the identifier this skill and
- `CONTEXT.md` both tell you to look for, and renaming it would break the recovery
- below to buy nothing.
+ ```bash
+ V=<version>
+ git fetch origin
+ # 1. prepare pushed it
+ git ls-remote --exit-code --heads origin "release/v$V"
+ # 2. one commit ahead of main, titled chore(release): $V
+ git log --oneline "origin/main..origin/release/v$V"
+ # 3. only the three manifests and RELEASE_NOTES.md
+ git diff --stat "origin/main...origin/release/v$V"
+ # 4. the [$V] sections it stamped, which should match the release page
+ NOTES="$(mktemp)" && git show "origin/release/v$V:RELEASE_NOTES.md" > "$NOTES"
+ RELEASE_NOTES_FILE="$NOTES" node scripts/release-notes.mjs extract "$V" | grep '^## '
+ # 5. no PR for it yet
+ gh pr list --repo malloydata/publisher --head "release/v$V" --state all
+ ```
- **Follow that link, open the PR, merge it.** The run stops at a branch on
- purpose: a PR opened by a person triggers `pull_request`, so its checks run and
- any maintainer can merge it, where one opened by the workflow would trigger no
- workflows at all and only an admin could ever merge it. One click buys back the
- check suite. Left unopened it costs exactly what a failed stamp used to — the
- next release re-appends this release's narrative to its own page, and so does the
- one after — and `main` keeps declaring the pre-release version.
+ Read them in order, and stop at the first surprise:
- Of those two, **the notes half is the one that compounds**; the version half is
- self-correcting, because `prepare` takes `max(npm latest, main declared)` and so
- derives the right floor whether or not this PR landed. That is exactly why that
- floor is not "simplified" to reading the file.
+ 1. **No branch:** `prepare` failed before pushing, so nothing published. Read
+ the run; there is nothing to merge back.
+ 2. **More than one commit:** someone pushed to the release branch. Read those
+ commits before opening anything.
+ 3. **Any other file:** stop and show the user. The release branch should carry
+ nothing else.
+ 4. **No sections:** normal for a routine patch. Otherwise compare with
+ `gh release view "v$V" --repo malloydata/publisher`.
+ 5. **A PR exists:** use it rather than opening a second. If it was closed
+ unmerged, reopen it.
+ Then open it against `main`:
+
```bash
- gh release view "v<version>" --repo malloydata/publisher --json body -q .body | head -40
- git ls-remote --heads origin 'refs/heads/release-notes-stamp-*'
- # after merging: main should now declare what shipped
- git fetch origin main && git show origin/main:packages/sdk/package.json | grep '"version"'
+ gh pr create --repo malloydata/publisher --base main --head "release/v$V" \
+ --title "chore(release): $V" \
+ --body "Merges the v$V release branch back into main: sets sdk, app and server to $V and stamps the RELEASE_NOTES.md sections v$V shipped.
+
+ Release: https://github.com/malloydata/publisher/releases/tag/v$V"
```
- **Merging this PR is no longer time-sensitive against `publish-packages`.** The
- branch only touches `packages/{sdk,app,server}/package.json` and
- `RELEASE_NOTES.md`, neither of which is a path `publish_indep_pkg`'s "main moved"
- guard watches for skills or create-malloy-package, so merging it while
- `publish-packages` is still polling npm cannot abort either dispatch. Merge it
- whenever is convenient.
+ The title matters: the repo squash-merges, so it is what `main`'s history keeps.
+ The workflow does not open this PR itself because a PR opened with
+ `GITHUB_TOKEN` triggers no workflows: `main`'s required checks would never run
+ and only an admin could merge it. Opened on the user's credentials, the checks
+ run and any maintainer can merge.
- The stamp step is `continue-on-error`, deliberately: the release is already
- public and correct by then, and reddening a finished release over a docs commit
- would send someone hunting a publishing problem that does not exist. So a missing
- branch is a real possibility — **read the job summary rather than assuming**. It
- always writes a line, including `No [Unreleased] sections to stamp` when there
- was nothing to do, so silence there means the step died and not that the release
- had no narrative. Re-running the job does not help: `gh release create` fails on
- the existing tag, and the stamp step is skipped behind it. Recover by hand as in
- step 3.
+ **`main` requires the branch to be up to date, and `RELEASE_NOTES.md` usually
+ conflicts.** Any PR merged since the release was cut makes the branch stale. A new
+ `[Unreleased]` section lands directly above the first stamped heading, which git
+ treats as one conflicting hunk. Merge `main` in locally, with a sign-off because
+ the DCO check is required:
- A **prerelease** skips the stamp, matching the rest of the job.
+ ```bash
+ git fetch origin
+ git switch -c "release/v$V" "origin/release/v$V"
+ git merge --signoff origin/main
+ # Only if the merge stopped on a RELEASE_NOTES.md conflict: keep BOTH sides,
+ # main's new [Unreleased] sections exactly as they are and this branch's [$V]
+ # headings, then conclude it. The package.json version lines never conflict;
+ # only release PRs change them.
+ # git add RELEASE_NOTES.md && git commit -s --no-edit
+ node scripts/release-notes.mjs extract "$V" | grep '^## ' # exactly v$V's sections
+ node scripts/release-notes.mjs extract | grep '^## ' # only what merged since
+ git push origin "release/v$V"
+ ```
+ The two `extract` lines are the check that the resolution is right: the first
+ must list the same sections as the release page, the second only sections
+ merged after the release was cut. Pushing to the release branch is safe once the
+ run has finished, since the tag pins the published commit.
+
+ Ask the user to merge the PR, or merge it if they asked you to. Merging deletes
+ `release/v$V` (the repo deletes head branches on merge); the tag keeps the
+ commit. Then confirm:
+
+ ```bash
+ git fetch origin main
+ git show origin/main:packages/sdk/package.json | grep '"version"' # $V
+ gh release view "v$V" --repo malloydata/publisher --json body -q .body | head -40
+ ```
+
+ If the release page is missing narrative that `[$V]` sections on the branch
+ carry, `gh-release` logged a `could not read RELEASE_NOTES.md` warning. Add the
+ output of `extract "$V"` to the page by hand with `gh release edit`.
+
+ Left unmerged, `release-sync.yml` fails on `main` and on every PR, naming
+ this version; merging this PR is the only fix. Confirm it went green on `main`
+ after the merge (the first command in step 3).
+
## If it fails
- **Only `publish-packages` is red** → the sdk/app/server release completed and
the tag exists; that job sits outside `gh-release`'s `needs`. Use *Re-run
failed jobs*, which re-enters that job alone and skips whatever already landed.
**Do not re-run the release** — it walks the version forward and burns three
npm versions.
Dispatching the children by hand also works, but it is *not* the same thing and
the skip does not come with it. Only `publish-packages` asks the registry and
skips; each child carries a **Verify this version is not already published**
step that `exit 1`s on a version npm already holds. So dispatching a child that
already published fails the run rather than no-opping. Ask npm first and
dispatch only the package still missing, keeping skills before the scaffolder:
```bash
npm view @malloy-publisher/skills version
npm view @malloy-publisher/create-malloy-package version
```
- **"main moved during this release"** → expected and retryable, same recovery.
It can fire on the second package after the first already published, so read
the job summary rather than assuming nothing shipped.
- **Anything after `npm-sdk.yml` published** → not resumable at the same version.
npm versions are immutable; move forward.
## Prereleases
Any hyphen in the version skips `gh-release` *and* both independently-versioned
packages, because their own versions carry no hyphen and would take over the
`latest` tag. Ship those from an ordinary release.
+
+ A prerelease is also never stamped and never merged back. `main`'s
+ `[Unreleased]` sections stay as they are and ship with the next ordinary
+ release, and `extract <prerelease>` finds nothing, because no heading carries a
+ prerelease number.