kb · git:20261117.82b2487 · 2026-11-17 · sha256 99abc626d2ab700c

kb git:20261117.82b2487A

Immutable. This exact content never changes and is served at /api/v1/blob/99abc626d2ab700c.

No tool to install first
claude plugin marketplace add https://markdownregistry.com/mp/a/art_d2gr6jpulh2q5vu2/99abc626d2ab700c0369f4c159cd91f081fc158d137260d2b6f34cebe02bcac6.json

Then /plugin install kb@markdownregistry-d2gr6jpulh2q5vu2 in Claude Code. It installs this exact version, the one graded A here, pinned to the commit it was graded at, and leaves it there.

---
name: kb
description: >
  What makes a kernel fast on gfx950 (MI350/MI355X) inside AKLR, and how to write it in FlyDSL, Triton/Gluon or
  HIP (the flydsl/, triton/ and hip/ folders: limits, errors, debugging, each language's own guides). Includes
  a dense-GEMM package (fp8, MXFP4, MXFP8, bf16/fp16, int8, small-M shapes included): dispatch rounds
  and split-K, main-loop pipelines, LDS/swizzle/DMA data paths, XCD tile order and scheduling hints,
  epilogues, autotuning, attribution, correctness gates and A/B measurement. Also a MoE package (fused MoE at
  any token count, and the expert grouped GEMM: tile-to-group mapping, order under token skew, sorted-token
  rows, transposed operands and K varying per group, in bf16, fp8, MXFP8 and MXFP4), an attention package,
  symptom pages, pitfalls and delivered kernels, trace capture and analysis,
  perf-regression bisection, the CDNA4 (gfx950) ISA reference guide with guides to looking things up in it,
  and gfx942 (MI300X) GEMM notes. Use when optimizing a kernel in any of these languages, or finding out why one
  is slow or wrong.
allowed-tools: Read Edit Bash Grep Glob Agent
gens: [gfx950, gfx942]
---

# AKLR kernel knowledge base

What makes a kernel fast on AMD GPUs inside AKLR, by op and by symptom. The op cards (`gemm/`, `moe/`,
`attention/`) and the cases come from FlyDSL kernels: their mechanisms carry to other languages, their numbers were
measured on FlyDSL code unless a sentence says otherwise (re-derive them: § Rules that hold everywhere). How to write
it in a language is in that language's folder: `flydsl/SKILL.md` (with the FlyDSL side of each op card in
`flydsl/<op>.md`), `triton/SKILL.md`, `hip/SKILL.md`.

Paths below are relative to this folder, except those starting with `aklr/`, relative to the repository root, and
`aiter/` or `csrc/`, relative to aiter's. File and line citations are to ROCm/FlyDSL `v0.3.2` (`dd8373349`),
AMD-AGI/Primus-Turbo main `7615a5a2` (`ca1fdd67` for the expert GEMM in `moe/`) and ROCm/aiter `dbd8bf5`
(`aiter` in `stack.json`); check out those revisions to follow them. FlyDSL's own skills are linked at `v0.3.2`, not
copied: `flydsl/guides.md` and `profile.md` give each link with what this repository corrects in it; a
`§` marked as FlyDSL's is that skill's own numbering.

## Stop: what a AKLR kernel cannot do

`aklr/ops/README.md` § The kernel section and ABI is the authority; where this skill disagrees, the README
wins. Check a plan against this table before writing code.

| Rule | What it rules out here |
|---|---|
| Outputs pass the definition's `compare` block; under `determinism: bitwise`, the default, two runs are byte-identical (`runs_agree`, `aklr/core/contract/semantics/_compare.py`) | Reducing in arrival order (atomic split-K, stream-K fix-ups): reduce in a fixed order (`gemm/correctness.md` §8, `gemm/grid_pricing.md` §2) |
| A Point is served by the file with the best measured `speedup` over the baseline, and only from `AKLR_MIN_SPEEDUP` (1.05) | Judging on a mean: compare each Point with the faster of the kernel AKLR serves there and the baseline |
| Launch code (`run` and what it calls) does not synchronize, call `.item()` / `.cpu()`, print, read a clock or the environment, or change torch state (SA3) | Timing or racing configurations in `run` |
| Compilation and the choice of configuration happen in a module-level `@functools.cache` function on a Point's first call; later calls may be inside a CUDA graph capture | The first-call race of `gemm/autotune.md` §1 as written: choose from the shape, with a table raced offline |
| Inputs other than the mutable ones are read-only (SA2); nothing derived from an argument is kept across calls, and there is no `global` or `nonlocal` (SA1) | Caching a repacked weight: repack on every call, or serve the contract's shuffled layout variant where it has one |
| Scratch is allocated in the call (`torch.empty`: contents undefined) or once per shape in the compile function and rewritten before it is read; no call depends on what an earlier call left | Counters that live across calls: zero workspaces in each call before any read (`gemm/small_m.md` §4). A slot stamped with this launch's 64-bit tag, queue bits included, needs no zeroing as long as no leftover word equals the tag: such a split-K fold is one launch on a `torch.empty` workspace (`gemm/wait_free_fold.md` § Folding without a counter) |
| Not in the README or its audit, but keep to it: no workgroup waits for another without a way out: a launch's workgroups need not all be resident (a stream limited to part of the CUs, another kernel holding CUs), so a wait on a flag another workgroup writes is bounded, and on timeout hands its share to the last to arrive (`attention/split_kv.md` §2 Bounded wait); the driver's `--stress` passes at every Point the file lists | A split-K fold or split-KV merge whose owner polls its partners with no bound, or gives up and writes: merge in a separate launch, have each publish and count in on an atomic counter and the last to arrive merge, or wait bounded with hand-off, in a fixed order |
| `run(*, <kernel_call_keys>)` writes the preallocated outputs and returns nothing; the entry file's docstring describes the kernel, with no history or speedup (SA4) | Allocating outputs; importing sibling files other than relatively (the package name is a hash) |

## How to look things up

- Before opening a page, write down the diagnosis so far (the regime and the trace or counter that shows it),
  the plan, and the one question whose answer would change the plan.
- Pick one page and one section with the tables below, and read that, not the folder.
- Then note what changed in the plan, and leave one line in the round's notes: `KB: matched <page> §<n>` or
  `KB: no-match <question>`. The no-match lines are what this skill is missing.
- Confirm a change by what its page says to check (final ISA, kernel descriptor, a counter), not by the total
  time alone.
- Before starting on an op, look in [cases/INDEX.md](cases/INDEX.md) for a delivered kernel of the same op and
  regime, and in [pitfalls/INDEX.md](pitfalls/INDEX.md) for its traps.

## By task

| Task | Read in this order |
|---|---|
| Write or optimize a dense GEMM (fp8 / MXFP4 / MXFP8 / bf16) | `gemm/INDEX.md` → the card it routes to |
| A GEMM with M ≤ 256 (as in decode or speculative verify) | `gemm/small_m.md` |
| A GEMM on gfx942 (MI300X) | `gemm/gfx942.md` |
| A MoE layer at any token count (MXFP4 experts, routing sort, stage 1 / stage 2), or its expert GEMMs as a grouped GEMM: per-size configuration, tile and occupancy, XCD placement, low precision and scale preshuffle, sorted-token rows, K varying per group, correctness, measurement | `moe/INDEX.md` → the card § it routes the problem to; main loop and data path in `gemm/` |
| Write or optimize an attention kernel: many query rows per head, the dQ dK dV gradients, or few query rows per head with split KV | `attention/INDEX.md` → the card it routes to |
| Dump a kernel's IR and final ISA; read registers, LDS, spills, waits and hot-loop counts | `gemm/mainloop.md` §10 (FlyDSL's `FLYDSL_DUMP_IR=1`: `flydsl/gemm.md` § mainloop §10); `analysis/isa_diff.py` compares two dumps |
| Capture a kernel trace and find the hotspot, or find the commit behind a performance regression | `profile.md` |
| EP dispatch/combine against TP for a MoE layer at a few tokens per GPU, or a collective's time in a multi-GPU trace | `communication/INDEX.md` |
| What an instruction, encoding, counter, wait state or hardware limit really is | `hardware/gfx950/INDEX.md` → `hardware/gfx950/lookup.md`, `hardware/gfx950/matrix_core.md`, `hardware/gfx950/memory_and_sync.md`; gfx942 (MI300X): `hardware/gfx942/INDEX.md` |
| The language: its API, compile errors, debugging, its own guides | the language's folder: `flydsl/SKILL.md`, `triton/SKILL.md`, `hip/SKILL.md` |
| A Triton or Gluon kernel: authoring, debugging, tuning, numerics, Gluon layouts, memory, MFMA and inline asm, AKLR's Triton kernels by family, its GEMM, MoE, attention and GatedDeltaNet kernels | `triton/SKILL.md` → the page its task table names |

## By symptom

| What the timings, trace or counters show | Read (then the symptom page, for what to rule out first) |
|---|---|
| Not yet known whether it is compute-, memory- or launch-bound | `gemm/attribution.md` §1–3 |
| Fewer workgroups than CUs, or a last dispatch round mostly empty | `gemm/grid_pricing.md` §1–2, §10; `patterns/low_cu_utilization.md` |
| M ≤ 256 on a row bound by launch, reduction and latency, not by reading B | `gemm/small_m.md` §2–3; `patterns/launch_floor_small_m.md` |
| LDS bank conflicts | `gemm/datapath.md` §3; `patterns/lds_bank_conflicts.md` |
| MFMAs wait on global loads; bytes / time near the measured stream bandwidth | `gemm/datapath.md` §5, §7; `patterns/memory_bound.md` |
| Spills, VGPR / AGPR pressure, one wave per SIMD | `gemm/mainloop.md` §4, §7, `gemm/grid_pricing.md` §4; `patterns/register_pressure.md` |
| The store costs more than expected | `gemm/epilogue.md` §2–3 |
| Memory-bound with a linear tile order: tiles that share a panel land on different XCDs | `gemm/schedule_and_locality.md` §2 |
| Stalls on `s_waitcnt` | `gemm/schedule_and_locality.md` §4, `hardware/gfx950/memory_and_sync.md` §1 |
| A fixed cost per tile: time not proportional to K | `gemm/attribution.md` §7, `gemm/grid_pricing.md` §6 |
| Clocks drop under load; slower than the instruction count says | `gemm/attribution.md` §4 |
| A gain that does not reproduce, or an A/B inside the noise | `gemm/measurement.md` §1, §4 |
| Outputs that differ between runs | `gemm/correctness.md` §3, §8 |
| A change wins on some shapes and loses on others | `gemm/autotune.md` §1–2; `patterns/shape_divergence.md` |

## Rules that hold everywhere

- **Structure before parameters.** Try fusion, pipelining and layout changes before knob tuning; tune last.
- **Reuse the mechanism, re-derive the number.** Constants in the cited kernels (tile and preload tables,
  wait counts, adoption margins, cache thresholds) were tuned on specific shapes.
- Numbers marked *(field)* are measured values: use them for direction and order of magnitude, not as
  targets. *(field, E1)* points to the line under the page's `## Evidence` that gives the conditions.

## Also in AKLR

- **The aiter these cards describe is the baseline**, ROCm/aiter at the commit `aiter`
  in `stack.json`: AKLR does not ship, patch or fall back to it, so its `aiter/` paths are that
  revision's, read as reference code. Price aiter through the op's `<contract>_baseline`
  (`aklr/ops/<op>/baseline.py`), the call bench times. At a Point no kernel serves, the api returns a
  `FallbackReason` and its caller runs its own path. AKLR's plugins route a model to plugin ops and to some
  `aklr/ops` ops (vLLM's GEMM and MoE layers, sglang's DSA paged MQA logits; `aklr/plugins/README.md`), so an
  `aklr/ops` kernel runs in a model only where such a route or the engine calls its api.
- **Workflow** (`aklr/ops/README.md` § Workflow): export the Points, fill `kernel.sources`, then
  `python -m aklr export check` and `python -m aklr validate --base origin/main`. The local result does not
  count; CI measures. A `platform_failure` whose message names rocprofiler-sdk or the graph replays, or says "the
  kernel tracer failed" or "calibration could not restore", is a tracer fault, already measured again in fresh
  processes: rerun (`aklr/core/validation/stages/_measure.py`, `pitfalls/profiler_misses_kernels.md`).
- **A short kernel sits on AKLR's timing floor.** Up to measurement profile 18 (Kineto, with
  `GPU_FORCE_QUEUE_PROFILING` forced) AKLR read about 4 µs per kernel however little it did, so a short kernel and
  its baseline read about the same. Main's profile 19 (its own rocprofiler-sdk tracer, queue profiling not forced)
  still has that floor, about 4.4 µs: a bmm candidate 1.50× faster back to back read 1.13× on it. Time an empty
  kernel, the baseline and your candidate on current main before deciding a Point has no headroom
  (`pitfalls/aklr_dispatch_floor.md` § Root cause).

## Folders and adding a page

`CONTRIBUTING.md`: what each folder holds, the page kinds, and how a page gets in.