v0.3 to v0.3

---

name: pollard-weights

description: >-

Quantize any LLM to a memory-fit GGUF (and vLLM/SGLang GPTQ) the RIGHT way with

Pollard Weights. Use this whenever asked to quantize / shrink / "make a Pollard

build" of a model, or to plan its fit on a machine or wafer. It routes dense vs

MoE models down different paths — using the wrong one wastes hours.

version: "0.3"

---

# Pollard Weights — agent skill

Pollard makes a frontier model fit a given machine's memory by **deciding where the

bits go** instead of painting every weight the same. This skill tells an agent the

correct path for *this* model so results are good and no time is wasted.

## ⚠️ STEP 0 — ASK THE USER (do this before building; don't assume)

The machine you BUILD on is usually **not** the machine the model will RUN on (users build on a big-disk

box for a phone, a Pi, another server, a 5090, a wafer…). `--ram`/`--vram` is the **TARGET device's**

budget, NOT this machine's — so you cannot infer it. **Before building, ask the user two things** (unless

they already gave them in the prompt):

1. **Target device & budget** — where will the model RUN, and how much RAM/VRAM does it have? (e.g. "iPhone

~6GB", "Jetson 8GB", "a box with 24GB", "a 256GB server", "512GB wafer node"). Pollard targets **ANY**

size — MB to TB — there is NO fixed 16–24GB range; that's just a common demo number.

2. **Preference** — *fit-a-budget* (best quality that fits that device, default) or **max-shrink** (smallest

possible, quality be damned — `--ram <small>` / `--allow-1bit`)?

Then build for the TARGET: `pollard-fit --ram <TARGET_GB>` (or `pollard-calc --ram <TARGET_GB>` first to

show what fits). Use `--ram auto` ONLY when the build machine IS the target. If you truly can't ask (no

interactive user), state your assumption explicitly ("assuming a 16GB target — tell me the real target to

re-fit") rather than silently picking one.

## Don't know the model type? Use the autoaware entry point

```bash

pollard --gguf model-f16.gguf --run # ONE-SHOT GGUF: auto-calib -> auto-imatrix -> flagship mix

pollard --hf Qwen/Qwen3-8B --run # ONE-SHOT from a HF repo: download -> convert -> build

pollard --hf ./my-local-model --run # ...or a model already on disk (any arch)

pollard --hf Qwen/Qwen3-8B --format gptq --run # GPTQ for vLLM/SGLang (from HF weights)

pollard --hf Qwen/Qwen3-8B --format mlx --run # MLX for Apple Silicon

pollard --hf Qwen/Qwen3-8B --format exl3 --run # EXL3 (exllamav3) — smoothing + Calib 3.0 gold, one-shot

pollard --hf Qwen/Qwen3-8B --format mx --run # MX: Blackwell/Rubin NVFP4 / any-GPU W4A16 (compressed-tensors)

pollard --gguf model-f16.gguf --imatrix model.imatrix --run # bring your own imatrix (skips auto-calib)

```

Point it at **any input** (HF repo id, local HF dir, or an f16 GGUF) and pick **any output** (`--format

gguf` default · `gptq` vLLM/SGLang · `mlx` Apple · `exl3` exllamav3 · `mx` Blackwell/Rubin/any-GPU compressed-tensors).

GPTQ/MLX/EXL3/MX emit straight from HF weights (no GGUF). **All five lanes run the GOLD method one-shot:**

smoothing is default-ON for the low-bit lanes (GPTQ/EXL3/MX), allocation is auto-measured (`pollard-probe`,

`--no-measure` to skip) for GPTQ/MLX/MX, and EXL3 packs Calib 3.0 to `-cd` + keeps its native allocator (the

8.670 win). EXL3 is the heavy trellis lane (hours by format) — use it only for the exllama runtime.

**True one-shot:** with no `--imatrix`, `pollard` auto-builds one (a Calib 3.0 multi-domain corpus via

`pollard-calib` → `llama-imatrix`), so the DEFAULT output is the flagship mix with **zero manual

steps** — the user supplies only the f16 GGUF. (`--no-auto-imatrix` = stock K-quant ladder only; big

MoE: pass a lower `--ngl` / compute the imatrix on a Q6_K host — see the coverage note below.)

### Scope & modes (accurate)

- **Supported arch classes** (auto path, regression-tested): **dense** (Qwen/Llama/Gemma/Mistral),

**MoE** (Qwen3-A3B/Mixtral/DeepSeek/Ling), **MLA-MoE** (DeepSeek, Tencent HY4), **GLM4-MoE**

(GLM-4.5/4.6/5.x — routes MoE, NEXTN/MTP tail edge-protected; `test_glm_moe_routing`). A genuinely

**new family** one-shots *if* its tensors match the feature rules — a 2-min `automap --dry-run`

confirms (auto-pin covers any unknown tail); if a new tensor family appears, add a **detection rule**,

never a per-model recipe (Ref-pipeline).

- **Fast one-shot vs measured (both real modes, not a caveat):** the default = locked recipe priors +

Calib 3.0 imatrix (minutes). The **measured-allocation** max quality is the opt-in `--benchmark` /

`pollard-sensitivity` path (hours). The 10-minute path is excellent; the beats-uniform gold-card

numbers come from the measured path.

- **Five output lanes** (same Pollard allocation, one command each): **GGUF** (llama.cpp) · **GPTQ**

(vLLM/SGLang) · **MLX** (Apple Silicon, mixed 4/8; real emit needs `mlx_lm`) · **EXL3** (exllamav3,

Blackwell) · **MX** (NVFP4/MXFP4 on Blackwell/Vera Rubin FP4 cores, via llm-compressor; Rubin serves from vLLM's cu134-nightly image). GGUF is the flagship;

the rest emit straight from HF weights. For a *specific* model, a one-line load in the target runtime

is still the sensible final ship check.

- ✅ **EXL3 low-bit WORKS — with preconditioning.** At low bit, run `pollard-hf-smooth` on the fp16 model

first (or `pollard-doctor --repair`): a massive-activation input channel otherwise collapses the trellis

global scale and silently wrecks a layer. MEASURED (Qwen2.5-3B, wikitext): smoothed 4bpw → **PPL 8.70**,

≈ the 8bpw build's 8.28; *without* smoothing 4bpw was PPL 3090 (garbage). EXL3 needed on Blackwell:

manual MSVC env, CUDA 12.8 to match torch cu128, Calib-3.0 `standard_cal_data` (wheel omits it). See notes.

- ✅ **THE WIN — the Pollard method beats EXL3 on EXL3's own allocator, trellis atoms, and custom kernel.**

**MEASURED (Qwen2.5-3B, wikitext, equal 4.00 bpw):** broken 3090 → smoothed **8.699** → smoothed + our

Calib 3.0 **8.670**, vs EXL3 out-of-box **8.699** — a win on their own turf, untuned and with no extra

calibration. Smoothing alone takes unusable low-bit (PPL 3090) to 4bpw ≈ 8bpw quality at HALF the size.

The gold path: **smoothing + Calib 3.0 + EXL3's native allocator** — `pollard --format exl3` runs it by

default. Keep the native allocator (it's strong on its own atoms and the win doesn't need reallocation);

DON'T port the GGUF K-quant role-map — it's a dead lever on trellis atoms (measured worse, ~50 min

wasted). A finer per-tensor-KL recipe in EXL3's atom space is open R&D; until one beats the gold recipe,

use the gold path. Judge everything by real reconstruction (`pollard-verify`), never proxy_err.

`pollard` reads the arch, decides **dense vs MoE**, and dispatches to the correct path

(dense → the imatrix K-quant ladder via `pollard-fit` **plus the IQ1_KT mixed-precision

flagship** — the hand-coded mix, the dense repos' headline build; MoE → `pollard-automap`

expert-allocation) —

the user never picks a tool. The manual steps below are exactly what it runs; read on to

drive a path yourself or understand what `pollard` chose.

> ### Build ≠ benchmark — the default is FAST

> Making a Pollard model is just the **build** (quantize): **one model, minutes, no eval.**

> The PPL / KLD / top-1 / 3-bar comparison / sensitivity sweep are the **benchmark** — they

> only *measure* the model and take **hours**. They are **opt-in**, not part of a normal build.

> - **Just make the model:** `pollard --gguf model.gguf --run` (one model, no eval).

> - **Reproduce our gold-card numbers:** add `--benchmark`, or score an existing model with the

> drop-in **`pollard-bench --gguf model.gguf --ref f16.gguf`** (PPL + KLD + top-1; `--vs` for head-to-head).

> - **If a build LOOPS on free-gen:** `pollard-bench --gguf model.gguf --coherence` runs the gate —

> generates, detects loops, sweeps sampling, and returns **PASS** (+the sampling to ship on the card)

> or **BELOW-FLOOR** (loops under every sampling → NOT a sampling problem; bump `--body` one tier and

> rebuild). `--quick` is a fast one-prompt post-build sanity. Sampling-first ALWAYS before a tier change.

>

> Never run the benchmark or the sensitivity sweep as part of a plain build — that's the

> mistake that turned a minutes-long shrink into a 3-hour run. See `benchmarks/README.md`.

## Winning default paths vs losing/fallback (so nobody re-hits the 3-hour trap)

| model | ✅ WINNING default (proven, minutes) | ⚠️ fallback / ⛔ losing (never the default) |

|---|---|---|

| **DENSE** | imatrix K-quant ladder (`pollard-fit`) **+ the IQ1_KT mixed-precision flagship** (the hand-coded mix — won 7B/14B) | ⛔ sensitivity SWEEP on dense = loses (no expert redundancy) → `pollard-sensitivity` refuses it |

| **MoE** | the **trellis mixed-precision mix** (`automap` WITH an imatrix) | ⛔ `automap --no-imatrix` K-quant mix = **DEPRECATED**, does NOT beat stock `Q2_K` (imatrix-free build → just use a stock K-quant ladder) · ⛔ sensitivity SWEEP on a big MoE = ~2·layers full-model quantizes = HOURS → refused unless `--allow-slow` |

| **MoE variants** (MLA / hyper-connection / DSA-indexer — Deepseek, Tencent Hy4/hy_v4, etc.) | **still just MoE → THE MoE recipe.** `automap` detects the features generically (labels e.g. `MoE +MLA+hyper-conn+DSA-indexer`) and routes by `is_moe`, NOT by model name. The MoE `attn_q/k/v` rules substring-match MLA (`k_b`/`kv_a_mqa`/`v_b`/`q_a`/`q_b`); the recipe also covers `attn_gate`/`hc_*_fn`/`indexer.proj`/`output_hc_fn`/dense-blk0 (no-op on Qwen3). Same policy, same tiers | ⛔ do NOT build a per-model recipe (esp. from a buggy run) · these need a COVERED imatrix (many experts — the coverage lesson at scale) |

The **mixed-precision hand-coded mix** (crush body → protect attn/down/first-last) is the BUILD,

proven by palette/lowbit and shipped across GGUF (`automap`), torch/GPU (`gptq --recipe`), and

vLLM (`export`). The **KL/PPL/scorecard/3-bar board** is the BENCHMARK — opt-in, `benchmarks/`.

## STEP 0 — detect the model type FIRST (this decides everything)

```bash

pollard-calc --model <hf-id-or-gguf> # prints arch: "dense" or "moe", params, layers, KV

```

- **dense** (Llama, Qwen2.5, Gemma, Mistral, …) → the **imatrix** path.

- **MoE** (Qwen3-A3B, Mixtral, DeepSeek, Ling, gpt-oss, …) → the **automap** path.

> **Rule: `imatrix = dense`, `automap = MoE`.** The tools ENFORCE this — `pollard-automap`

> and `pollard-sensitivity` refuse a dense model (pass `--allow-dense` only for research).

> Running the measured-KL sweep on a dense model is a multi-hour no-op; don't.

---

## DENSE path — imatrix-guided K-quants

The win on a dense model is the **importance matrix**, not per-layer reallocation

(measured: the KL knapsack does NOT beat uniform on dense — no expert redundancy).

```bash

# 1. f16 GGUF (if starting from HF)

python convert_hf_to_gguf.py <hf-dir> --outtype f16 --outfile model-f16.gguf

# 2. imatrix from a MIXED-domain calib (prose + code; not one register)

llama-imatrix -m model-f16.gguf -f calib.txt -o model.imatrix -ngl 99 --chunks 100

# 3. build the ladder you want to ship (imatrix-guided). Pick sizes for the audience:

llama-quantize --imatrix model.imatrix model-f16.gguf model-IQ4_XS.gguf IQ4_XS

llama-quantize --imatrix model.imatrix model-f16.gguf model-Q6_K.gguf Q6_K

# ...or let pollard-fit pick the best quant for a RAM budget:

pollard-fit --gguf model-f16.gguf --ram 16 --imatrix model.imatrix --out model-pollard.gguf

```

Ship a **ladder** (≥3 files, e.g. IQ3_S / IQ4_XS / Q6_K) so the HF repo's quant +

hardware-compatibility sidebar populates (it is one row PER GGUF FILE, not from the card).

Do **not** run `pollard-sensitivity` on a dense model.

Optional flagship extreme rung: an **automap 1-bit trellis mix** (the "gold-card")

*is* automap-on-dense — allowed only with `--allow-dense` (see below).

---

## MoE path

### MoE QUALITY win — the MEASURED allocator (this is the proven one; do NOT reinvent it)

The quality win on a MoE (and on dense) is **`pollard-sensitivity` → `pollard-fit --sensitivity`**:

it CRUSHES one tensor group at a time and measures the *true KL cost*, then a knapsack spends

bits where the measurement — not a guess — says they matter. **Measured, shipped, and proven

to beat uniform imatrix-IQ at matched size on both a MoE (granite-3B-a800m, 40 experts) AND a

dense (Qwen2.5-1.5B)** — see `assets/benchmarks/kl_win.png`. Do not build a second allocator; use this one.

```bash

pollard-sensitivity --gguf moe-f16.gguf --imatrix moe.imatrix --eval held.txt --out moe.sens.json

pollard-fit --gguf moe-f16.gguf --ram 16 --imatrix moe.imatrix --sensitivity moe.sens.json --out moe-pollard.gguf

```

Cost: the sweep is ~2·num_layers passes (one-time per model). Note: `imatrix magnitude LIES`

(big activations ≠ high KL) — that's exactly why the measured sweep exists; don't allocate on

raw imatrix importance.

### MoE trellis (imatrix) — the extreme-low 1-bit-class variant (only with a covered imatrix)

`pollard-automap` emits the 3-bar 1-bit-class build (uniform-IQ1 / PollardMix / uniform-IQ2).

This is the extreme-low showcase (the 7B/14B gold-cards), NOT the general allocator above.

### MoE trellis (imatrix) — the extreme-low variant (only with a covered imatrix)

```bash

# 1. f16 GGUF + imatrix — MoE NEEDS a DIVERSE calib (prose+code+varied) or rare experts

# never route and get NO importance data. Use --chunks 200+.

llama-imatrix -m moe-f16.gguf -f calib_diverse.txt -o moe.imatrix -ngl 99 --chunks 200

# 2. FULL tensor list (dry-run at a non-low type so it can't abort mid-list):

llama-quantize --dry-run moe-f16.gguf x.gguf Q6_K > tensors.txt

# 3. automap: emits the MoE recipe (crush ffn_*_exps, protect router/ffn_down_exps/shared/attn),

# AUTO-PINS any imatrix-uncovered expert to q6_K so the build can't hard-fail.

pollard-automap --tensors tensors.txt --model moe-f16.gguf --imatrix moe.imatrix --out build.bat

# 4. run build.bat -> uniform-IQ1 / PollardMix / uniform-IQ2 + PPL each

```

**Watch automap's output:** if it reports "PINNING <many> uncovered tensor(s)", the

imatrix under-covered experts — the mix will BLOAT. Re-run the imatrix with a larger,

more diverse calib (more experts routed). (Build it on a Q6_K host + copy `up_exps`→`gate_exps`.)

**Accept-the-Mix gate:** keep it only if PollardMix ≤ uniform-low size + ~8–10% AND it

beats uniform-low on PPL/KL/top-1. If it drifts toward uniform-high size, reject and

tighten the crush.

### MoE with NO imatrix — just use a stock K-quant ladder (`--no-imatrix` is DEPRECATED)

`pollard-automap --no-imatrix` (a K-quant "mix") is **DEPRECATED** — measured on Qwen3-30B-A3B

it did **NOT** beat the stock `Q2_K` preset, so it's a losing path with no reason to exist.

The imatrix is no longer a swamp (build it on a **Q6_K** host + copy `up_exps`→`gate_exps` for

SwiGLU coverage → minutes). So: **make an imatrix and use the trellis mix (the winner); if you

truly won't, just build a stock K-quant ladder (`pollard-fit`)** — that's the honest imatrix-free

build, not a fake mix that loses to it.

**MoE measured K-quant ladder (alternative to the trellis mix)** — the KL knapsack that

beats uniform, for shipping IQ3/IQ4/Q6 sizes:

```bash

pollard-sensitivity --gguf moe-f16.gguf --imatrix moe.imatrix --eval held.txt --out moe.sens.json

pollard-fit --gguf moe-f16.gguf --ram 16 --imatrix moe.imatrix --sensitivity moe.sens.json --out moe-pollard.gguf

pollard-experts --gguf moe-f16.gguf --imatrix moe.imatrix # measured hot-expert report

```

---

### Giant MoE that won't fit even quantized (e.g. Kimi K2, ~1T params) — PRUNE first

Quant has a floor: params x 1 bit. A 1T MoE is ~125 GB at 1-bit and unusable that low — you

**cannot quantize a trillion-param MoE onto one small box.** The lever is **expert pruning**:

a big MoE has many cold/redundant experts (Kimi K2: 384 experts, 8 active). Drop the cold

third-to-half FIRST (structurally smaller model), THEN quantize:

```bash

pollard-pack --gguf kimi-f16.gguf --emit-plan plan.json # rank experts (forecast + which to drop)

pollard-prune --gguf kimi-f16.gguf --keep 0.5 --out kimi-pruned.gguf # execute: rewrite a smaller GGUF

pollard-automap --tensors <dry-run kimi-pruned> --model kimi-pruned.gguf --imatrix kimi.imatrix --out build.bat

```

`--score imatrix` (with a diverse-calib imatrix) keeps the experts the calib actually routed

to (REAP-correct); `magnitude` is the zero-calib default. Keep >= the model's active-expert

count. **Producing the artifact needs the source weights + storage + a big box** — the tool

runs on the box that holds the model; a user without the disk downloads the finished small

build instead of making it.

### Models with a lookup table (n-gram / per-layer embeddings) — keep the table on the SSD

Qwen3.8-Flash-Next (a 51B n-gram PLE table), Gemma 4 E-series (per-layer embeddings), Engram / LongCat n-gram

embeddings: a big table that is only ever READ, a few rows per token. llama.cpp serves those rows straight from

the file (`--load-mode mmap --lazy-mode on`), so the table needs neither RAM nor VRAM. **Do not let it eat the

RAM budget:** `pollard-fit` does this by default (`--disk auto`) — the table is pinned at `--disk-type` (q8_0,

near-lossless; disk is cheap) and every GB of `--ram` goes to the transformer. On a 24 GB box with a 20B table

that is the difference between the layers at Q2_K and at Q6_K.

```bash

pollard-ngram inspect model-f16.gguf # tables, their size, does this runtime lazy-read the arch

pollard-fit --gguf model-f16.gguf --ram 24 --disk auto # table on the SSD at q8_0, transformer gets the budget

pollard-ngram run build.gguf --ram 24 --vram 12 # the launch line (mmap + lazy + table on CPU)

pollard-ngram watch --pid <llama-server pid> # disk writes vs swap while it runs (10+ min)

```

Reading the table never writes to the SSD. **Writes during a run are swap**: the model was loaded with

`--no-mmap` / `-lm none|mlock` (copies the table into RAM), or the rest of the model plus KV doesn't fit.

`watch` says which. Reads far under the drive's rated speed are normal (small random reads: latency-bound).

`--disk auto` only applies to architectures whose llama.cpp loader marks the table lazy (read from

`runtime/llama.cpp/src/models/*.cpp`); otherwise it says so and keeps the table in the budget.

## Make your own foundation model (`pollard-forge`)

Pollard builds models, not only quants of other people's. `pollard-forge` is the small-open-model recipe the labs

use (NVIDIA Minitron/Nemotron-style): **new** (a fresh qwen3 / llama / qwen2 architecture at a size preset or exact

dims; tokenizer copied or trained), **prune** (carve a smaller model out of a teacher: drop the layers with the

lowest block influence and the FFN channels that fire least, measured on `--calib`), **train** (pretrain,

continue, or distill from the teacher: CE + a distillation loss -- `--loss kl` (temperature-scaled, default) /

`torchrun`), **card** (lineage from `forge.json`, license required).

```bash

pollard-forge prune --teacher Qwen/Qwen3-4B --calib calib.txt --keep-layers 0.75 --ffn 0.6 --out my-2b

pollard-forge train --model my-2b --teacher Qwen/Qwen3-4B --kd 0.7 \

--data hf:HuggingFaceFW/fineweb-edu@0.6 --data chat.jsonl@0.4 --tokens 2e9 --save-every 500

pollard-forge card --model my-2b --name "My-2B" --license apache-2.0

pollard --hf my-2b --run # GGUF ladder + card like any model

```

Keep chat data in the mix for an instruct model (raw web text alone erases chat — measured on STQ1_0). A pruned or

distilled model inherits its teacher's license terms; the card refuses to guess one. Checkpoints save in bf16

(`--save-dtype`); mid-run checkpoints stay fp32 so `--resume` is exact.

## Release a model on EVERY lane (`pollard release`)

One plan, every lane, each lane a full LADDER of rungs: GGUF (the publish ladder via `pollard --hf --ladder`, the

way we build), MLX 2/3/4/5/6/8-bit, GPTQ int2/3/4/8, MX NVFP4/MXFP4/pure FP8 W8A8, EXL3 several bpw. Smoke

preflight first, serial under the machine lock, resumable (state file + MANIFEST: built+verified rungs are never

rebuilt), every rung through `pollard-verify --gate`, one card per lane (`pollard-card --verified-only`), upload

only with `upload: true` in the plan AND `--upload`.

```bash

pollard release --init release.yaml --preset moe # text | vlm | decision | moe (recipes/release*.yaml)

pollard release --hf Qwen/Qwen3-8B --plan release.yaml # print the full plan (default)

pollard release --hf Qwen/Qwen3-8B --plan release.yaml --lanes gguf,mlx --run # the Mac

pollard release --hf Qwen/Qwen3-8B --plan release.yaml --lanes gptq,mx,exl3 --run # the CUDA box

```

Repos: `<Org>/<M>-Pollard` (GGUF, files `<M>-Pollard-<TAG>.gguf`) and `<Org>/<M>-Pollard-MLX|GPTQ|MX|EXL3` (one

subfolder per rung, default rung at the root). Lanes this machine cannot build are skipped with the command to run

them where they can. The EAGLE3 `draft` lane is declared but not wired (no pollard-draft yet).

### The publish gate (`pollard-verify --gate gates.yaml`)

Per-rung thresholds (default <- `lanes.<lane>` <- matching `rungs` patterns): KLD max, top-1 min, task retention

min (pollard-taskeval), translation retention, tool-call pass (pollard-toolcall), decision agreement, GGUF runtime

(stock vs ik_llama from the tensor types -- an `IQ4_XS` whose attn_v got IQ4_K is ik-only and FAILS a `runtime:

stock` gate) and coherence. An unmeasured threshold FAILS. Every lane is measured on the same scale (mean KLD + top-1

vs the source): gguf via llama-perplexity, mlx via mlx_lm, gptq/mx via transformers; exl3 is gated on its assembled

forward. `pollard-verify --print-gates` = the defaults (`recipes/gates.yaml`). Non-zero exit = do not publish.

### MLX range search (`pollard-mlx --range search|minmax`)

Default `search`: each group-of-64's scale/bias is chosen by GPQ's shrink-grid search + weighted LS refit,

scored on error weighted by the input channel's activation energy (`--imatrix`: llama.cpp .dat/.gguf with

llama-family names mapped, or .npz/.safetensors keyed by module path; without it, plain weight error). MLX's own

min/max is always a candidate, so no group gets worse. Output = standard MLX affine (same packing, same

scale/bias tensors) -- mlx-lm loads it unchanged. Synthetic: weighted error ~0.35x of `mx.quantize` at 3/4-bit;

GPQ measured ~2x lower KL at 4-bit. Real KL A/B still pending (`notes/mlx-range-search.md`). `pollard --format

mlx` passes `--imatrix` through when given.

### EXL3 recipe bridge (`pollard-exl3 --recipe none|pollard|exl3|FILE`)

ExLlamaV3 >= 1.6 (MIT) converts from a per-tensor `-rcp recipe.yaml`. `--recipe pollard` emits one from Pollard's

measured sensitivity (`--sensitivity`) at `--bpw`: EXL3's own tensor list (exllamav3 graph on the box, safetensors

headers elsewhere), sc_optimize's KL law `S_t * 4^(p-K)`, greedy marginal-KL-per-bit at the budget, fused

q/k/v + gate/up tied; checked against EXL3's parser (v1.6.2 vendored in tests). `exl3` = EXL3's own sc_optimize

recipe (`--exl3-recipe`, or prints the sc_* pipeline). DEFAULT stays `none` (the gold path) until the head-to-head

(`notes/exl3-recipe-bridge.md`: budgeted vs Pollard recipe vs sc_optimize, same bpw, KL on held-out) says otherwise.

### NVFP4 Four Over Six (`pollard-mx --fp4-scale 6|4over6`)

Four Over Six (MIT HAN Lab, arXiv 2512.02010, MIT licence): each NVFP4 block's E4M3 scale maps the block max to

4 OR 6 (tensor scale against 256, not 448), keeping whichever block reconstructs better -- Pollard scores it by

imatrix-weighted error (`--imatrix`), MSE without one. Post-pass over llm-compressor's checkpoint; output stays

standard NVFP4 (compressed-tensors layout, fused q/k/v + gate/up keep one shared global scale). Synthetic:

~0.75x weighted error vs plain. Default stays `6` until the box A/B (`notes/nvfp4-four-over-six.md`).

### Calibration profiles (`pollard-calib --profile`)

`general` (Calib 3.0), `translation` (parallel pairs through the model's own template), `agent` (tool-call traces +

schemas through the template), `code`, `auto` (from pollard-modelkind; `pollard` passes it by default,

`--calib-profile` to override). `--model` renders chat samples with the model's own template; `--self-generate`

(`--server URL` or `--self-generate-gguf f16.gguf`) swaps in the f16 model's own answers. Sources are open-licensed

and pinned; ids/revisions/licences go to `<out>.sources.json` and the MANIFEST.

**Reasoning (thinking) models -- automatic.** `pollard-modelkind` renders the chat template (jinja2, no tokenizer)

to read the model's OWN thinking delimiters (`<think>`, `<seed:think>`, `[THINK]`, harmony

`<|channel|>analysis`, `<|START_THINKING|>`, `<mm:think>`, Gemma `<|channel>thought` ...), whether a variable

switches it (`enable_thinking` / `thinking` / `thinking_mode` / `reasoning_effort` no_think) and whether the

generation prompt already opens it (always-on); names/tags/config are the fallback (R1-distill templates strip the

block; `*-Instruct-2507` / `non-thinking` releases are excluded). `pollard-calib` then adds a **reasoning share

(`--reasoning-share`, default 0.35)** of DeepSeek-R1 traces (OpenThoughts-114k metadata, Apache-2.0, pinned;

OpenR1-Math fallback; bundled seeds offline) rendered EXACTLY as the model emits them -- the template's own

rendering when it renders the final turn's thinking, else its generation prompt + delimiters + end-of-turn. A

toggleable model gets thinking-ON for the reasoning share and thinking-OFF for the rest (template kwargs).

`--self-generate` keeps the model's own thinking (reasoning_content, or split off inline) and rejects blocks the

budget cut off. **Sanity gate:** the corpus is NOT written (non-zero exit) if a reasoning model's calibration has

zero thinking blocks in its delimiters; the counts land in `<out>.sources.json` (`reasoning.report`).

`pollard-calib --check old.txt --model <m>` audits an existing corpus (the LFM2.5-24B corpus had zero `<think>`).

`--reasoning on|off|toggle` overrides; `--allow-no-think` writes anyway. Eval side: taskeval / toolcall / the

coherence gate strip thinking for ANY delimiter style (`pollard_modelkind.split_reasoning`); an unclosed block =

no answer (`unclosed-think` in pollard-toolcall, which gives reasoning models a 1024-token budget). KL/verify

keep the thinking: the held-out split carries the same reasoning share, so KL is measured on what the model emits.

### Retention vs f16 on the card

`pollard-taskeval --results results.json --rung <tag>` (any suite, plus `--suite mt` = WMT24++ / FLORES+ chrF++ per

language, optional COMET-22), `pollard-toolcall --results`, `pollard-decision --results` all merge into one

results.json; `pollard-card --results` renders the "Retention vs f16" table and the per-language translation table.

## Fine-tune on your own data, then ship it (`pollard tune`)

"Fine-tune Qwen for payments risk review" → follow **`skills/pollard/fine-tune-and-serve/SKILL.md`** (data

checklist, privacy, reading the eval). Short form:

```bash

pollard tune --base Qwen/Qwen3.5-9B --data cases.csv --task classify --label-field outcome \

--weight-field amount_usd --time-field created_at --out runs/risk # prep -> LoRA -> merge -> eval

pollard-tune ship --out runs/risk --harness claude-code # ladder + card w/ eval + serve line

```

LoRA/QLoRA on mlx (Apple Silicon), TRL+PEFT or Unsloth (CUDA); the eval compares base vs tuned (vs

`--compare-url`) on a leak-checked held-out split: per-class P/R/F1, dollar-weighted recall, latency P50/P99.

`--method grpo` / `sft+grpo` trains with RL on a class-balanced label reward instead of (or after) SFT

(mlx-lm-lora on a Mac, TRL GRPOTrainer elsewhere); chat data stays SFT-only.

## Runtime targets (where the build will actually run)

- **llama.cpp / ik_llama.cpp / Ollama / LM Studio** — the GGUF above runs as-is. The

trellis `*_KT` quants need `ik_llama.cpp`; K-quants run in any recent llama.cpp.

- **vLLM / SGLang** (GPU servers, DGX-Spark crowd) — GGUF/trellis does NOT run there.

Export a **GPTQ** checkpoint instead:

```bash

pollard-export --model <hf> --sensitivity model.sensitivity.json --calib calib.txt --out ./out

vllm serve ./out --quantization gptq # 4/8-bit dynamic mix, Marlin-accelerated

```

Constraints (enforced): bits ∈ {4,8} only (Marlin), never split bits inside a fused

group (q/k/v share; gate/up share), `group_size 128`, `desc_act False`. SGLang mixed-bit

is fragile → `--uniform` for SGLang. **vLLM reserves a large KV pool up front** — a model

that fits in GGUF can OOM in vLLM; budget weights + KV + activations, run one load test.

- **Cerebras wafer** — `pollard-pack --gguf model.gguf --target wse3t` forecasts

wafers-per-model + a sensitivity-ranked expert-prune plan. Capacity PLANNER only

(16-bit resident, forecast not benchmark); MoE-only lever.

## Optional — abliteration (opt-in, off by default)

`pollard-abliterate --model <hf> --harmful A.txt --harmless B.txt --out ./abl` removes the

refusal direction on the FP16 weights BEFORE quantizing (composes with any build). It costs

some coherence — measure the PPL/KL delta vs the un-ablated build before shipping.

## Preconditioning levers (optional — apply to the f16 BEFORE the imatrix/quantize)

These rotate/smooth the weights so low-bit quant lands better. Bit-width-dependent (they

help at some tiers, hurt at others) — measure with `pollard-eval`/`pollard-kl` before shipping.

```bash

pollard-smooth --gguf model-f16.gguf --out model-smooth.gguf --alpha 0.5 # AWQ-style channel smoothing

pollard-rotate --gguf model-f16.gguf --out model-rot.gguf --kind orthogonal # QuIP#/QuaRot rotation (also --kind block --block 32)

pollard-precondition --gguf model-f16.gguf --out model-pre.gguf # dynamic selector: pick the best lever per bit-width

pollard-gptq --model <hf-dir> --bits 4 --groupsize 128 --out ./gptq # full-Hessian error-feedback GPTQ (beats RTN)

```

## Utilities & metrics

```bash

pollard-calc --model <hf-or-gguf> --ram 16 --ctx 8192 # will it fit? KV-cache + total RAM pre-flight

pollard-run --gguf model-pollard.gguf --vram auto --launch # launch a build (llama-server), VRAM-aware

pollard-eval --gguf model-pollard.gguf --ref model-f16.gguf --eval held.txt # trajectory-divergence quality

pollard-kl --model <hf> --eval-file held.txt --calib-file calib.txt # KL-to-f16 + top-1 (small models, torch)

pollard-probe --model <hf> --eval held.txt --out model.sens.json # = pollard-sensitivity, CHEAP mode (same profile output, no GGUF/imatrix sweep — any box)

pollard-scorecard --results results.json --out scorecard.md # the standardized publish scorecard

pollard-health # cross-vendor accelerator degradation check

pollard-fit-dit --gguf dit-model-f16.gguf --ram 16 --out dit-pollard.gguf # memory-fit builds for DiT (image/video) archs

pollard-lowbit --model <hf> --bits 1 --keep 0.01 --levels 3 --eval-file held.txt # PROVEN low-bit levers: outlier-catch + residual carousel

pollard-palette --model <hf> --calib-file calib.txt --eval-file held.txt --target-bpw 1.58 1.3 # PROVEN measured mixed-alphabet allocation (beat uniform ternary -22%/-35%)

```

## FlyBrain — give a Pollard build memory with no context window

A quantized model still has the context problem: its KV cache grows with every token, and past the

window the beginning is gone. `pollard-flybrain` attaches a measured fruit-fly connectome (MaleCNS

v1.0) to a **frozen** model as an associative memory that never grows — so a Pollard build can be

small AND answer questions about text it can no longer see.

Measured on Qwen2.5-0.5B-Instruct. A six-letter string is stated once, buried under filler, and asked

about far beyond the window; the model never sees more than 128 tokens at a time, so it can never see

the fact and the question together. Training words are drawn fresh for every example and never reused

— there is nothing to memorise. 48 samples per row, through the shipped code path:

| | accuracy |

|---|---:|

| no brain (the fact is outside the window) | **0.0%** |

| **+ fly brain** | **93.8%** |

| control — a word the document never contained | 0.0% |

| control — brain read a *different* document | 0.0% |

Holds as documents grow far past what it trained on, with the state size unchanged:

| document | no brain | + fly brain | state |

|---|---:|---:|---|

| 768 tokens (6 windows) | 0.0% | 100.0% | 8.8 MB |

| 1,536 tokens (12 windows) | 0.0% | 100.0% | 8.8 MB |

| 3,072 tokens (24 windows) | 0.0% | 96.9% | 8.8 MB |

**State is 8,552 slots × 328 = 11.2 MB, constant at any length.** The state update costs ~3% of one

model forward (2.1 ms vs 73.4 ms) and fits on CPU at 3.6 ms, so it can run beside the model instead

of competing with it.

### The full package for a user

1. Quantize as normal (`pollard --hf <model> --run`, or the dense/MoE path above).

2. Train a brain **for that backbone** — they do not transfer between families:

```bash

pollard-flybrain --train 1400 --model <hf-id-or-path> \

--connectome graph.feather --signs signs.npy \

--probes corpus.txt --brain MyModel-FlyBrain.pt

```

3. Bind it at run time, read a document of any length, and persist the session:

```python

from pollard_flybrain import FlyBrain

brain = FlyBrain.load("MyModel-FlyBrain.pt", device="cuda")

brain.bind(model, tok) # backbone is never modified

brain.feed(open("long_document.txt").read()) # any length, 128 tokens at a time

print(brain.answer("Question: what is the secret word? Answer:"))

brain.save_state("session.flystate") # 8.8 MB — and it does not grow

brain.load_state("session.flystate") # resume mid-thought

```

### What to tell the user honestly

- **It retrieves the identifying token, not yet whole strings.** Exact first token ~94%; spelling the

full six-letter word back is ~2.5%, because training supervised only the first token. Say

"finds the fact" and not "reproduces the text" until that number moves.

- **It is not faster than attention at short lengths** — about 10–28% slower at ≤4K tokens with

128-token windows, because the brain still runs the model over every window. At 1,024-token windows

it reads a 4K document in 0.90× attention's time while holding a quarter of the KV. The durable win

is **flat memory**, not throughput.

- **One brain per backbone.** The address matrix has that model's hidden size and the token codes come

from its output embedding; `bind()` refuses a mismatch rather than producing confident nonsense.

- **The brain remembers; it does not reason.** Language, code and vision come from the backbone.

8,552 slots are a memory, not a mind.

- **It needs the extra:** `pip install 'pollard-weights[flybrain]'` (torch, transformers, pandas,

scipy). Core Pollard stays light.

- The wiring is doing the work, not recurrence alone: a degree-preserving shuffle — same neurons,

synapses, degrees and weights, only the connectivity randomised — carries 2.2–2.6× less traffic.

### Gotchas that waste a run

Every one of these was measured the hard way, and each alone pins accuracy at exactly 0%.

- **Never truncate the prompt to a token budget** — cut the *filler* instead. Building

`fact + filler + question` and slicing to N tokens removes the question from the end of every

example, and the task then has no answer at all. This sat under an entire night of work and produced

eight different architectures that all scored zero.

- **Key on the context BEFORE a token, value on the token.** Keyed on the fact itself, a random string

is unfindable — the question shares nothing with it (measured 0.00×–0.007× against a filler token).

Keyed one token earlier, on "…the secret word is", the question finds it at 1.6×–4.3×.

- **Addresses must be sharp.** `elu(x)+1`-style dense positive features make every query match

everything about equally; top-5 then stalls around 8% no matter how wide the state gets. Use a

softmax over slots — but standardise or norm-pin the logits, or they grow until the softmax is

one-hot and two addresses overlap by exactly zero.

- **Generate training words on the fly.** On a fixed 300-word list the brain hits ~12% and then 0% as

soon as words are drawn fresh: it learned 300 codes, not a mechanism. Store the *host's* token code

(`P @ out_embedding[token]`, read back with `P.t()`) so unseen words work by construction.

- **The brain must be audible.** Its vote needs standardising to the backbone's logit scale and a

mixing weight that does not start at zero — measured 58.3% for the brain alone against 8.3% for the

mix when it was too quiet. Give that scalar its own (larger) learning rate.

- **Hidden-state similarity will not give you addressing for free.** Across all 25 layers the question

looks *more* like random filler than like the fact's context. The address function has to be

supervised explicitly; nothing in the answer loss teaches it.

- **Controls must exclude first-token collisions.** 200 random six-letter words share only 26 distinct

first tokens, so a "wrong" word often shares the right one's first token and the control reports

leakage that is not there (6.2% → 0.0% once excluded).

- Train against the backbone the user will actually run. `bind()` refuses a hidden-size mismatch.

## Guards & gotchas (why runs fail or waste time)

- **Dense + sensitivity/automap = REFUSED** (multi-hour no-op / wrong tool). Use imatrix.

- **MoE + short/prose-only imatrix** = uncovered experts → build hard-fails or bloats.

Diverse calib, `--chunks 200+`. automap auto-pins the uncovered ones as a safety net.

- **HF sidebar empty** = the repo has too few GGUF files. Ship the ladder (1 row/file).

- **Start from f16/bf16**, never a re-quantized file (compounds loss).

- **New/newer arch fails to quantize** = rebuild the runtime llama.cpp (`git pull && cmake --build`).

- **Bench alone** — a parallel eval/build contaminates tok/s.

- **FlyBrain does not transfer between backbones** — train one per model family (~20 min), or it contributes nothing.

## Full feature index (keep this list in sync with the tools)

| tool | use it for | model type |

|---|---|---|

| `pollard` | **autoaware entry** — detect dense/MoE + run the right BUILD | any |

| `pollard-bench` | **drop-in benchmark** — PPL + Mean/Median KLD + top-1 for a model, `--vs` for head-to-head | any |

| `pollard-calc` | pre-flight: arch, params, "will it fit?" (KV-aware) | any |

| `pollard-fit` | memory-fit build for a RAM budget (K-quant ladder / MoE knapsack) | any |

| `pollard-automap` | MoE expert-allocation mix + auto-pin uncovered experts | **MoE only** |

| `pollard-sensitivity` | measured per-group KL profile (the knapsack input) | **MoE only** |

| `pollard-probe` | `pollard-sensitivity` **cheap mode** — same profile, in-process, no GGUF/imatrix sweep | any |

| `pollard-pack` | Cerebras wafer capacity + expert-prune plan (forecast) | MoE (lever) |

| `pollard-prune` | REAP-style expert pruning — drop cold experts, rewrite a SMALLER GGUF | **MoE only** |

| `pollard-ngram` | n-gram / per-layer embedding tables on the SSD: inspect, launch line, swap watch (`pollard-fit --disk`) | models with a lookup table |

| `pollard-forge` | make your own model: new architecture, Minitron-style prune, pretrain / distill, lineage card | any decoder-only |

| `pollard-release` | **every lane as a ladder** (gguf/mlx/gptq/mx/exl3), gated, carded, resumable, upload on double opt-in (also `pollard release`) | any |

| `pollard-tune` | fine-tune on YOUR data (LoRA/QLoRA SFT and/or GRPO: mlx / TRL / Unsloth), eval base vs tuned vs frontier, ship + serve | any HF causal LM |

| `pollard-export` | vLLM/SGLang GPTQ checkpoint (4/8 dynamic; `--bits 2/3/4/8` ladder rungs) | any |

| `pollard-abliterate` | optional refusal-direction ablation (pre-quant) | any |

| `pollard-experts` | measured hot-expert report | **MoE only** |

| `pollard-run` | launch a build (llama-server), VRAM-aware | any |

| `pollard-fit-dit` | memory-fit builds for DiT (image/video) archs | any (DiT) |

| `pollard-gptq` | full-Hessian error-feedback GPTQ (beats RTN) | any |

| `pollard-smooth` | AWQ-style channel smoothing (preconditioning) | any |

| `pollard-rotate` | QuIP#/QuaRot rotation (preconditioning) | any |

| `pollard-precondition` | dynamic selector for the best lever per bit-width | any |

| `pollard-eval` | trajectory-divergence quality metric | any |

| `pollard-kl` | KL-to-f16 + top-1 metric (small models, torch) | any |

| `pollard-scorecard` | the standardized publish scorecard | any |

| `pollard-lowbit` | PROVEN low-bit levers: outlier-catch (SpQR) + residual carousel (AQLM) | any (torch, small model) |

| `pollard-palette` | PROVEN measured mixed-alphabet allocator (prune/binary/ternary/2b) — beat uniform ternary −22%/−35% | any (torch, small model) |

| `pollard-flybrain` | **connectome memory** — attach/train a fly brain so a build has no context window | any (torch) |

| `pollard-ggufcheck` | which runtime can actually load a GGUF — reads ggml types + arch, not filenames | any |

| `pollard-card` | the master HF model-card template (measured numbers, retention vs f16, verified column, correct runtime column) | any |

| `pollard-recard` | bring already-published repos onto that template without losing measured numbers | any |

| `pollard-ls` | list the Pollard workspace so you never hunt for a build | any |

| `pollard-reclaim` | free disk a finished model still holds, once it is verifiably published | any |

| `pollard-runtime` | are the llama.cpp builds current, and do they carry our patches? | — |

| `pollard-verify` | the publish GATE: KLD/top-1 vs source on every lane (gguf/mlx/gptq/mx/exl3) + `--gate gates.yaml` per-rung thresholds (retention, tool calls, runtime, coherence) | any |

| `pollard-refcheck` | prove the reference forward is sane BEFORE measuring anything against it | any |

| `pollard-doctor` | diagnose/repair a model for low-bit: is this convert healthy, and why not? | any |

| `pollard-errsrc` | attribute measured quant error to a tensor AND its trigger | any |

| `pollard-errtype` | classify WHY a tensor quantizes badly, so the right lever is chosen | any |

| `pollard-archfp` | what IS this architecture structurally — and which known one is it a twin of? | any |

| `pollard-onboard` | audit a NEW architecture and emit a ready-to-submit contribution | any |

| `pollard-envmatch` | build a custom-arch model in a Python env matched to its converter | any |

| `pollard-calib` | calibration corpus: Calib 3.0 + `--profile general/translation/agent/code/auto`, model-template rendering, `--self-generate`, automatic reasoning share in the model's own think delimiters + sanity gate (`--check`) | any |

| `pollard-probes` | task-accuracy probes on a Pollard'd GGUF via llama.cpp's own runner | any |

| `pollard-taskeval` | task scores + RETENTION vs f16 (lm-eval suites, `--suite mt` chrF++/COMET per language), `--results` for the card | any |

| `pollard-toolcall` | does the rung still emit valid tool calls (vs the f16), `--results` for the card + gate | any (agent models) |

| `pollard-decision` | decision-model board vs f16 (agreement, option KL, calibration), `--results` for the card + gate | decision models |

| `pollard-serve-eval` | A/B a served quantized model against its baseline, on the same traffic | any |

| `pollard-hf-smooth` | activation-aware SmoothQuant preconditioning for an HF model | any |

| `pollard-mlx` | emit an MLX (Apple Silicon) mixed-bit model with Pollard's allocation (`--bits 2/3/4/5/6/8` rungs); `--range search` (default) = imatrix-weighted per-group scale/bias search (`--imatrix`), still standard MLX affine | any |

| `pollard-mx` | emit FP4/FP8 for Blackwell/Rubin (NVFP4 default, MXFP4, pure FP8 W8A8 `--scheme FP8`; `--kv-fp8`, `--moe-protect-attn-only`, `--fp4-scale 4over6` Four Over Six block scales) | any |

| `pollard-exl3` | emit an EXL3 model carrying Pollard's allocation; `--recipe pollard` writes an EXL3 `-rcp` recipe from the measured sensitivity, `--recipe exl3` uses sc_optimize's | any |

| `pollard-exl3-band` | band-parallel EXL3 conversion for models that do not fit one GPU | any |

| `pollard-health` | cross-vendor accelerator degradation check | — |

> **MAINTENANCE:** this skill must track the tools. When a feature is added/changed,

> update the decision tree AND this index in the same PR, or agents will run Pollard

> wrong. Verify the tool list against `pyproject.toml [project.scripts]`.