push · diff

git:20260812.a9f50e1 to git:20260817.072dbb0

25 added, 9 removed. Audit A to A.

---
name: push
description: The authoritative pre-push procedure for the cgg repo. Run before every `git push` (and before opening a release). It defines how to refresh every autogenerated README component (mermaid graphs, self-stats, benchmark table, language count/table), how to keep the Limitations and Potential-future-improvements sections honest against the code, the rule that EVERY factual claim in README.md must be neurosymbolically validated — proven by a deterministic symbolic check (cgg itself, rg/grep, cargo test, docs-check.py, `cgg --help`), not asserted from memory — that every CHANGELOG entry must carry a measured `### Performance` block (scripts/perf-compare.sh), that silent/transient functionality bugs get bold prominent disclosure, and how version numbers are incremented (Cargo.toml workspace + internal pins, Cargo.lock, CHANGELOG.md) and published (scripts/release.sh, publish-crates.sh, publish-python.sh). Trigger when about to push, cut a release, or bump a version in the cgg repo.
---
# push — the cgg pre-push contract
A push is a publish. The README is cgg's public contract: agents read
it, users trust its numbers, and the autogenerated blocks are assumed
to match the binary that produced them. This skill is the gate that runs
**before `git push`** so nothing stale, unproven, or mis-versioned
leaves the repo.
It composes with two existing pieces and adds the parts they don't cover:
- `.githooks/pre-commit` regenerates the three embedded mermaid blocks
and the self-stats line, runs the test suite, installs the fresh
binary to `$CGG_INSTALL_DIR`, and runs `docs-check.py`.
- The `docs-sync` skill is the per-edit drift checklist for prose/tables.
This skill is the **whole-README, whole-repo final pass** plus the
**versioning and publishing procedure**. Do all of it before pushing.
For a release specifically, `scripts/release.sh` already automates most
of §1–§4a — see §4b.
## 0. Preconditions
```bash
cargo build --release -p cgg # the binary every regen below uses
```
If the build fails, stop — every downstream number would be wrong.
## 1. Refresh every autogenerated README component
The README contains five machine-generated regions. Never hand-edit
them; regenerate from the freshly built binary so they describe the code
you are about to push.
| Region | Marker / location | Regenerate with |
| --- | --- | --- |
| cgg-walk graph | `<!-- cgg:begin:walk -->` … `:end:walk` | `cgg ./crates/cgg-walk -t mermaid -o walk.mmd` then `scripts/update-readme-graphs.py walk walk.mmd …` |
| cgg-lang graph | `<!-- cgg:begin:lang -->` … `:end:lang` | `cgg ./crates/cgg-lang/src/{detect,parser,lib}.rs -t mermaid -o lang.mmd` then `scripts/update-readme-graphs.py … lang lang.mmd` |
| self-analysis graph | `<!-- cgg:begin:self -->` … `:end:self` | `cgg ./crates --filter 'cgg::analyze_in_pool$' -n 1 -o self.mmd` then `scripts/update-readme-graphs.py … raw:self self.mmd` (verbatim — the README presents it as the literal output of the command printed above it) |
| self-analysis stats | `<!-- cgg:begin:self-stats -->` … `:end:self-stats` | that same run's **stderr**, piped to `scripts/update-readme-stats.py` |
| benchmark table | the per-language table under `## Benchmark` | `scripts/benchmark.sh` (clones repos; multi-minute) then `scripts/update-readme-stats.sh` / `scripts/patch-readme-stats.py` |
The showcase filter is `cgg::analyze_in_pool$` — **not** `cgg::run$`,
which since the 0.6.0 library split is a shim in `main.rs` whose name
several test helpers also share. `docs-check.py` check 7 asserts the
same filter appears in `.githooks/pre-commit`,
`scripts/update-readme-stats.sh`, `scripts/patch-readme-stats.py`,
`README.md` and `CLAUDE.md`, and that the generated block still spans at
least three crates.
The first four are exactly what `.githooks/pre-commit` runs — so a
normal `git commit` already refreshes them. **Verify, don't assume:** the
graphs and self-stats can be stale on disk if a prior commit used
`--no-verify` or `CGG_SKIP_PRECOMMIT=1`. Run the regen and confirm
`git diff README.md` shows either nothing or only the expected
regenerated lines.
The **benchmark table is NOT touched by the hook.** If your change could
move per-language numbers (new language, resolver change, stdlib tuning,
perf work), re-run `scripts/benchmark.sh` and patch the table. If it
can't, say so explicitly in the push notes rather than shipping numbers
you didn't re-measure — silent staleness reads as "freshly measured."
## 2. Keep Limitations and Potential-future-improvements honest
`## Limitations` and `## Potential future improvements` rot silently
because no generator owns them. Before every push, re-derive each bullet
from the code:
- **Every "future improvement" must still be unimplemented.** For each
bullet, find the symbolic proof that it has NOT happened (or has only
partially happened). If the code now does it, move it out of the
list — to the feature table, the CHANGELOG, or delete it. Partial
progress gets rewritten to name exactly what remains (e.g. "8 stdlib
lists remain: csharp, fortran, java, …"), never left as a blanket
claim.
- **Every "limitation" must still be true.** A limitation that the code
has since fixed is a lie in the contract. Prove each one still holds
(no watch mode → no `watch`/`notify` in `cli.rs`; Tcl/Hack absent →
not in `plugins.rs`; macros not expanded → no preprocessor sim; no
cache → nothing writes one).
Validation patterns (these are the symbolic half of §3, applied to
prose). **Every one of these has been run and produces the output it
claims** — a validation command that silently errors or greps the wrong
file "proves" the claim by accident:
```bash
# Dynamic dispatch lives in the library, NOT main.rs. Grepping main.rs
# returns nothing and would wrongly "prove" the feature is gone.
rg -n 'trait_impl_target' crates/cgg/src/lib.rs
git log --oneline -- crates/cgg-core/src/stdlib/ # which stdlib lists got cleaned?
# ripgrep has no -E for "extended regex" — -E is --encoding, and
# `rg -niE 'watch|daemon'` exits 2 with "unknown encoding".
rg -ni 'watch|daemon|notify' crates/cgg/src/cli.rs # watch mode landed yet?
# The calls read `reg.register(...)`; a `plugin.register` grep returns 0.
grep -c 'register(' crates/cgg-lang/src/plugins.rs # 44 languages
```
## 3. Neurosymbolic validation of ALL README claims (mandatory gate)
**Rule: no claim ships unproven.** Before push, every factual assertion
in `README.md` must be validated *neurosymbolically* — the neural half
(reading the prose, deciding what it asserts) is paired with a
**symbolic** half (a deterministic check that proves it against the
actual code or tooling). An assertion you can only support "from
memory" or "it was true last release" is not validated; soften it,
remove it, or go prove it.
Symbolic oracles available in this repo — prefer them in this order:
1. **cgg itself** — the strongest oracle. Counts, edges, cross-file
numbers, "resolves method dispatch", "finds cross-file edges":
run cgg on a fixture/the repo and read the real numbers.
2. **`cgg --help`** — the source of truth for both the CLI flag table
and the usage synopsis. `docs-check.py` gates each separately and
must pass.
3. **`cargo test --workspace`** — behavioral claims backed by tests
(detection rules, resolver behavior, format output).
4. **`rg`/`grep` over `crates/`** — existence claims: a language is
supported, a flag exists, an FFI form is detected, a resolver phase
runs in a given order. Confirm the command actually matched
something; an empty result is not a proof of absence unless you
verified the pattern works elsewhere.
- 5. **`scripts/docs-check.py`** — eleven consistency invariants
- (numbered 0–9 in its docstring, plus the framework-rule/`APPS`
- check). **Must exit 0.**
+ 5. **`scripts/docs-check.py`** — fourteen consistency invariants
+ (numbered 0–12 in the body, plus the unnumbered
+ framework-rule/`APPS` check). Checks 11 and 12 gate the *content*
+ of the skill files themselves, including this one. **Must exit 0.**
6. **`scripts/benchmark.sh`** — the only oracle for the benchmark
numbers. If you didn't run it, you didn't validate them.
7. **`scripts/determinism-sweep.py`** — for any "identical at any
`--jobs`" / reproducibility claim. It strips the embedded timing
fields, which a naive byte-diff does not; that false positive has
been hit three times.
8. **`scripts/framework-coverage.py`** — for framework claims: which
rules actually fire on a real application, as opposed to on a
fixture written to pass.
Claim classes and their required proof:
| Claim in README | Symbolic proof required |
| --- | --- |
| "Supports N languages" / language table | `grep -c 'register(' plugins.rs`, registry test, `docs-check.py` |
| Any CLI flag / default | appears in `cgg --help` (table **and** synopsis) |
| Counts in self-stats / benchmark | regenerated by the binary / `benchmark.sh` |
| "resolves X" / "links cross-file" / FFI | a cgg run that produces the edge, or a passing test |
| Framework rule counts / coverage | `rules.rs` parsed, plus `framework-coverage.py` |
| Pipeline phase order | matches `cgg::analyze_in_pool` in `crates/cgg/src/lib.rs` (NOT `main.rs` — that is a thin CLI shim whose `run` just calls `cgg::analyze` and emits) |
- | Install commands | actually run them: `cargo install cgg`, `pip install cgg-callgraphgenerator`. The npm package `cgg-callgraphgenerator` is **not published**; do not document it as installable. |
+ | Install commands | actually run them: `cargo install cgg`, `pip install cgg-callgraphgenerator`, `npm install cgg-callgraphgenerator`, and the GitHub binary tarball. All four channels are published; verify the version each registry actually serves rather than assuming the tag published it. |
| A limitation / future gap | the absence proof from §2 |
If a claim has no available oracle, it does not belong in the README as
a fact — rephrase it as clearly-marked intent or cut it.
## 4. Version increment policy
cgg is pre-1.0 and follows a loose SemVer (see `CHANGELOG.md` preamble:
the default graph only ever *grows*). Increment by the largest category
the change touches:
- **PATCH** (`0.6.2 → 0.6.3`) — bug fixes, doc-only changes, stdlib list
tuning, internal refactors. No new user-visible surface.
- **MINOR** (`0.6.3 → 0.7.0`) — additive contract surface: a new
language, a new flag, a new output format, a new resolver phase, new
audit fields. This is the common case for feature work.
- **MAJOR** (`→ 1.0.0`) — reserved for the 1.0 stability commitment.
Pre-1.0, behavior changes that would otherwise be "breaking" ship as
MINOR **with a `### Compatibility / migration` note in the CHANGELOG**;
the default graph must remain a superset of the prior release.
When you bump, change **all** of these in one commit, then validate:
1. `Cargo.toml` → `[workspace.package] version` (one line).
2. `Cargo.toml` → the **six** internal pins under
`[workspace.dependencies]`: `cgg`, `cgg-core`, `cgg-lang`,
`cgg-resolve`, `cgg-format`, `cgg-walk`. All must match the
workspace version. (`cgg` itself is pinned there too, because
`cgg-py`, `cgg-ffi` and `cgg-node` depend on the library.)
3. `Cargo.lock` → refresh by running `cargo build` (rewrites the **nine**
`cgg*` entries — the six above plus `cgg-ffi`, `cgg-node`, `cgg-py`).
Stage the result; the pre-commit hook does not.
4. `CHANGELOG.md` → add `## [x.y.z] - YYYY-MM-DD` (today's date) at the
top, with `### Added` / `### Changed` / `### Fixed` / `### Compatibility`
subsections as applicable. Match the existing entry's voice.
5. `CHANGELOG.md` → **a `### Performance` block is mandatory in every
entry.** See §4a.
Symbolic check after bumping (substitute the version you are leaving):
```bash
rg -n '0\.6\.2' Cargo.toml # must return nothing for the old version
cargo build --release -p cgg # Cargo.lock now consistent
./target/release/cgg --version # prints the new version
```
## 4a. Performance is a release-note requirement
**Every CHANGELOG entry carries a `### Performance` block. No
exceptions, including for releases that "shouldn't" affect latency —
that claim is exactly the one worth checking.**
cgg's pitch is that it is fast enough to run before every edit. A
release that quietly regresses latency breaks that pitch silently, and
four releases shipped before this rule with no numbers at all.
Generate the numbers, never estimate them:
```bash
scripts/perf-compare.sh # vs the last `chore: release` commit
scripts/perf-compare.sh <ref> <runs> # explicit baseline / sample count (default 7)
```
It builds the baseline in a **separate git worktree**, so it works with
a dirty tree and never touches your working copy. Output is a markdown
table ready to paste.
- **`perf-compare.sh` is a 9-repo smoke test, not a corpus.** Its fixed
- set is `rust-ripgrep python-flask js-express go-fzf c-jq cpp-spdlog
- csharp-serilog swift-alamofire cpp-nlohmann-json`. One of those, `c-jq`,
+ **`perf-compare.sh` measures the whole corpus** as of 0.7.0. It used to
+ be nine hand-picked repos — `rust-ripgrep python-flask js-express go-fzf
+ c-jq cpp-spdlog csharp-serilog swift-alamofire cpp-nlohmann-json` —
+ totalling 1.8 seconds, or 0.7% of the corpus's 258s of work, and
+ containing none of the repositories where 0.7.0's superlinear resolver
+ paths lived; it would have reported *flat* for a fix that took
+ `erlang-otp` from 3h40m to 24.5s. Set `REPOS=(...)` to override for a
+ deliberate one-off. The cautionary history below is why. One of those
+ nine, `c-jq`,
is pathologically sensitive: 69 analyzed files with the largest being
22% of the bytes, so wall time is the critical path of a single file and
any scheduling change amplifies. It regresses in every paired trial at
an unstable +23% to +45%.
That mattered. A "+4–6.8% corpus-wide regression" was reported in two
CHANGELOG entries on the strength of this 9-repo set — and it was wrong.
The paired A/B over 29 repositories puts the **median per-repo delta at
+0.0%**, total wall at +2.7%, with **10 of 29 faster**. `c-jq` carried a
ninth of the weight of a headline number that did not generalise, and
`cpp-nlohmann-json` looked like a second case at +9.5% in one run and
came back +0.0% in another — noise.
So: use `perf-compare.sh` to *detect* a possible move, and widen the
sample before *describing* one. Never promote a 9-repo delta to a
corpus-wide claim.
The block must state, alongside the table:
- **Machine load at measurement time.** The script prints it
(`load : …` from `uptime`). A loaded box changes the answer
materially.
- **Whether the comparison is like-for-like.** If the new version
enables work by default that the baseline lacked, the delta is the
cost of new default behaviour, not overhead. Say which.
- **An explanation for every regression over ~5%** on a repo whose
baseline exceeds 150 ms. Below that, jitter reads as several percent
and the script says so.
- **Whether the move is corpus-wide or one repository**, with the
repo-shape evidence (file count, biggest file as a share of bytes) if
it is the latter.
- **Test-suite and pre-commit-hook timings**, since those are the
latency contributors developers feel most often.
A regression you can explain is fine to ship. A regression you did not
measure is not. A regression measured on a sample too small to
generalise is worse than either, because it sends someone optimising
the wrong thing.
## 4b. Cutting an actual release
`scripts/release.sh` runs the gates, the measurements and the drafting
in the order that stops prose being written before numbers exist. **It
never commits, tags or pushes** — it prints the commands and stops.
```bash
scripts/release.sh --purpose "what this release is for"
scripts/release.sh --quick # gates only; no perf, no prose
scripts/release.sh --skip-ai # gates + measurements, no prose
scripts/release.sh --version X.Y.Z --baseline <ref>
```
Its phases: preflight → gates (build, test, clippy, fmt, docs-check,
determinism) → measure (perf vs previous release, corpus stats,
coverage) → document → verify → report. Artifacts land in
`target/release-prep` by default.
Publishing, after the release commit and tag:
```bash
scripts/publish-crates.sh --dry-run # package + verify, upload nothing
scripts/publish-crates.sh # crates.io, in dependency order
scripts/publish-python.sh --check # build wheel + twine check, no upload
scripts/publish-python.sh # manylinux wheel -> PyPI
```
`publish-crates.sh` exists because a workspace cannot be published in
one command: each crate's dependencies must already be live on crates.io
before the next can even be packaged, and a version once published can
never be reused — so getting the order wrong burns a version number.
`publish-python.sh` builds inside maturin's manylinux container because
PyPI rejects plain `linux_x86_64` wheels; the distribution is
`cgg-callgraphgenerator` and the import is `cgg`.
- **There is no npm publish script and `cgg-node` is not published.** Do
- not add an `npm install` line to any doc until it is.
+ **npm publishes from CI, not from a script.** `release.yml` runs `napi
+ prepublish` for the five platform packages and then `npm publish
+ --ignore-scripts` for the root, and polls `npm view` for up to five
+ minutes afterwards because `dist-tags` propagation lags a successful
+ publish — on v0.6.7 a single read one second later said the old version
+ and failed the job after everything had shipped.
+
+ The root package needs `--ignore-scripts`: its `prepublishOnly` runs
+ `napi prepublish` a second time, and `napi` is not on PATH in that step.
+ Without it the five platform packages publish and the root does not,
+ leaving `npm install` on the previous release.
## 4c. Functionality bugs get bold disclosure
**A bug that changed results silently is disclosed prominently, in full,
and near the top of its CHANGELOG entry — not buried in a list.**
The class that matters most is a bug producing a *plausible wrong
answer*. A crash announces itself. A graph that is quietly missing edges,
or that differs run to run, does not: every number published while it was
live was wrong without saying so, and nobody had cause to re-check.
Rules:
1. **Transient / nondeterministic bugs lead the `### Fixed` section**,
with a blockquote callout above it so a scanner cannot miss it. Same
input, different answer is the most serious class cgg has.
2. **Quantify the blast radius.** Not "fixed a resolver bug" — state
what it did and how much: *"1,404 of 2,858 edges on phoenix"*,
*"1460/1463/1466/1469 across ten runs"*. A reader needs to know
whether their own results were affected.
3. **Say which releases were affected**, and say so explicitly if
previously published numbers (README tables, benchmark rows) were
wrong as a result. This applies to *wrong analysis in the CHANGELOG
itself*: the "+4–6.8% corpus-wide regression" of 0.6.0/0.6.1 was
retracted in 0.6.2 under its own heading, not quietly dropped.
4. **Never report a delta against a nondeterministic or unrepresentative
baseline as a finding.** Check the baseline is stable first; if it is
not, report the range, not the difference. A single draw compared
against a fixed value is not a measurement — it produced a spurious
"−5 edges" in this project's own notes before being caught.
5. **State the mechanism**, so a reader can judge whether their codebase
hits it — e.g. *"routine in C/C++, where many directories carry their
own `common.h`"*.
6. **Add the regression test in the same commit**, and name it in the
entry. A silent bug that had no test is two defects.
A leak is in this class too, not merely a performance note: the
`type_hints.rs` `Box::leak` was harmless while the pipeline was a
short-lived binary and became ~161 bytes per call once 0.6.0 made it a
library, a Python module and a C ABI. `docs-check.py` check 9 now
requires every deliberate leak in the pipeline crates to be listed in
`ALLOWED_LEAKS` with a reason that does not depend on the process
exiting.
Being unflattering in a release note is cheap. A user discovering the
bug themselves, after trusting a number cgg gave them, is not.
## 5. Final push sequence
```bash
# 1. build + regen autogen (§1) + validate claims (§2,§3)
# 2. bump version if warranted (§4)
scripts/perf-compare.sh # §4a — paste into the CHANGELOG
cargo test --workspace # green (also gated by the hook)
python3 scripts/docs-check.py # exit 0 (also gated by the hook)
git add -A
git commit -m "…" # pre-commit hook re-regens graphs/stats, re-runs tests + docs-check
git push origin main
```
Let the pre-commit hook run — do not `--no-verify` a push. If the hook
restages README.md, that is the autogen blocks correcting themselves;
inspect the diff, it should be only generated regions. Note the hook
also installs the freshly built binary to `$CGG_INSTALL_DIR` (default
`~/.local/bin/cgg`), so anything else on the machine picks up the code
you are about to push.
## What this skill does NOT do
- It does not replace `docs-sync` (the per-edit drift checklist) — it
assumes that ran and does the final whole-repo sweep.
- It does not auto-bump the version — versioning is a judgment call
(§4); decide the category, then apply it everywhere at once.
- It does not invent benchmark numbers — unmeasured numbers are not
shipped; either re-run `benchmark.sh` or state that they were not
re-measured.
- It does not commit, tag or push for you, and neither does
`scripts/release.sh`.