git:20261025.e6d217a to git:20261027.bf64f38

- # CLAUDE.md

-

- Guidance for Claude Code when working in this repository.

-

- ## Benchmarks

-

- - During development, run benchmarks in quick mode: `QUICK=1 benchmarks/run.sh`, with `FILTER='*SimpleTest*'`

- (any BenchmarkDotNet filter) to run only the cases you are working on. Quick mode measures each case for 6 to 10

- iterations, stopping once its error is under 2%, so it takes about 30–45 minutes for everything (Nino, MemoryPack,

- MessagePack and protobuf-net).

- - Both modes run v3 3.3.13 (from NuGet: `benchmarks/Nino.Benchmark` built with `NINO_BENCH_V3=true`) once, directly

- before v4, on the same machine; MemoryPack runs in both, so its two results show how much the machine moved.

- - Quick numbers are for comparing your own changes back to back on one machine. Never publish them: the numbers

- in README.md, docs/v4-benchmarks.md, the chart and the release notes come from a full `benchmarks/run.sh` run

- (or the release workflow), whose job id stays `ShortRun-.NET 10.0`.

- - See benchmarks/README.md for the options.

-

- ## Machine code

-

- A change to a read or write path (Nino.Core, Nino.Immutable, the code the generator emits) can slow cases it does not

- touch: everything a top-level call reaches is inlined into one method, under one inlining budget. Before committing

- one, run `python3 benchmarks/asm.py compare origin/main` (about two minutes, see benchmarks/README.md) and explain or fix

- every method it lists. What has gone wrong before:

- - A branch whose condition is a method call (`RawFormatter.Is`, `NinoBlock.Size`, ...) is compiled and inlined even

- for the types where it is dead, and uses up the budget there. Keep rare or large paths (vector loops, element by

- element loops, string allocation, recycling) out of line (`NoInlining`), with only the common path inline.

- - Never pass `ref reader` or `ref writer` to a method that is not inlined, even in a dead branch: the address keeps

- the struct in memory for the whole method. Pass it by value and return it (`reader = ReadEach(values, reader)`).

- - Mark every small helper a read or write path calls `AggressiveInlining`, even one whose arguments are constants

- and that looks certain to fold: in a large method the JIT's inliner gives up on it. `NinoLanes.Paired`, called

- with constants in the AVX2 group loop, stayed a call per load and spilled the vectors around it: AVX2 writes took

- twice as long as SSSE3's. Look for such calls in the Tier1 listing of every ISA a path takes (`--asm`, with

- `DOTNET_EnableAVX512=0` and `DOTNET_EnableAVX2=0` too).

- - A JIT-time check made of several helper calls is one `static readonly` field instead (`NinoElements.Traits`,

- `NinoBlock<T, TFormatter>.Size`): one load, nothing to inline.

- - Vector loops: bump `ref` cursors and count down (`at = ref Unsafe.Add(ref at, step)`, `while (--n != 0)`), never

- `(nuint)i * size` per iteration; take everything that does not change per iteration (masks, indices) out of the

- loop into locals.

- - On Arm64 (and anywhere without AVX-512) `Vector512<T>` and `Vector256<T>` are structs of 16-byte vectors: passed

- into an inlined loop body they are copied through the stack every iteration. Take their `Vector128` parts out once,

- before the loop, and pass those.

- - Arm64 `tbl` with two to four table registers needs consecutive registers: build the tuples from loads made once

- per iteration, before any lookup (`(w0, w1)`, `(w1, w2)`), never from per-lookup loads, or the JIT adds `mov`s.

- Each table register is a micro-op on Neoverse cores: give each lookup only the registers its bytes come from, and

- store only the bytes the values fill (`NinoBlock.WireTables64`); check each shape, as types that compile to the

- same code can differ by the data (`(byte, int)` took twice as long as `(short, int)`).

- - Look at Arm64 code too, not only x64: build for `linux-arm64` and run under `qemu-aarch64-static` with

- `DOTNET_TieredCompilation=0 DOTNET_JitDisasm='<method>' DOTNET_JitStdOutFile=<file>` (the code paths only a type

- outside the benchmark suite takes, such as `(byte, int)` for the 64-byte permutes, need a small program of their

- own). A clean loop is loads (`ldp`), lookups, stores (`stp`) and the cursor adds; `stp`/`ldp` to `[fp, ...]`, `mul`

- or `mov v..` in it are the signs above.

- - Every optimization has to hold on each platform Nino runs on: x64 with AVX-512, AVX2 or SSSE3 only, Arm64, Arm32

- and other 32-bit targets, WebAssembly and anything without SIMD (`DOTNET_EnableHWIntrinsic=0`), Mono and Unity

- (IL2CPP included), .NET Framework. Fix a problem everywhere it occurs, not only in the case that showed it, and say

- in the commit body which platforms a change speeds up and which keep the old path. A path through a runtime's

- internals (`CollectionViews`: the fields of `List<T>`, `HashSet<T>`, `Dictionary<TKey, TValue>`, ...) declares a

- layout for each implementation (.NET 64-bit and 32-bit, .NET Framework, Mono), checks it against an instance with

- known contents at startup and uses the public API when none matches; run the tests on .NET, the `net472` build under

- Mono and the `linux-arm` and `linux-arm64` builds under qemu, and keep the fallback (the enumerator) out of line.

- - Building a runtime's own objects without its public API (`Nino.Immutable/HashTrees.cs`: an `ImmutableDictionary`

- or `ImmutableHashSet` built as its tree, instead of one builder `Add` at a time; `SortedSetNodes` in Nino.Core: a

- `SortedSet<T>`'s red-black tree; `ReadLinked`: a `LinkedList<T>`'s ring of nodes; `ConcurrentTables.Builder`: a

- `ConcurrentDictionary`'s bucket chains): one way for every runtime where

- one works, and a separate way per runtime (behind `#if` or a startup check) only where none does. What works

- everywhere so far:

- - Mirror classes that declare the fields of the real class in its order, read and written through `Unsafe.As`:

- every runtime lays out two classes with the same field list the same way. Check the field lists of every version

- of the library users may have (System.Collections.Immutable 1.5 to 10.0 have the same fields), and check the

- layout at startup against an instance with known contents, through the public API; use the public API when it

- fails.

- - Never call an instance method on a mirrored object (a `MemberwiseClone` helper in the mirror class): Mono and

- IL2CPP take a shared generic method's context from `this`'s class, and Mono aborts (an assertion, not an

- exception, so no `catch` helps). Static code only.

- - Allocate a mirror instance with `new` and set its type word to the real type's (`ObjectTypes.Retype`): the same

- fields mean the same size and the same references for the GC. Faster than `RuntimeHelpers.GetUninitializedObject`

- (13 ns on .NET 10 but 77 on .NET 8, FormatterServices 59 on Mono) and needs nothing IL2CPP may lack. The type word

- sits where the reference points (CoreCLR: method table, one header word; Mono and IL2CPP: vtable or class, then

- the monitor, two words), found at startup; call `GetType()` on a retyped object out of line (`NoInlining`), or

- the JIT folds it to the type it allocated (Arm32 did).

- - Run it on .NET 8 and 10, the `net472` build under Mono (also `MONO_GC_DEBUG=verify-before-collections`), Arm32

- under qemu, with a GC stress (forced compacting collections between builds, then every object checked).

- Under qemu-arm (Arm32), set `DOTNET_EnableWriteXorExecute=0`: with W^X the runtime writes dynamic methods

- (`Expression.Compile`, reflection invoke stubs) through a second mapping qemu-user does not see, and they crash.

- Set `DOTNET_INTERNAL_ThreadSuspendInjection=0` too: qemu (8.2 and 10.2) aborts in about one run in four

- (`thumb_tr_translate_insn: Assertion (dc->base.pc_next & 1) == 0`) when the GC's suspension signal lands.

- To run tests under qemu-aarch64, pass `-cpu neoverse-n1` (listings can keep the default): qemu 8.2's default CPU

- (`max`) has LRCPC2, whose `ldapur`/`stlur` (what the optimizing JIT emits for `Volatile.Read`/`Write` of a field)

- it emulates wrong. With or without Nino's code, runs crashed (`ToImmutableDictionary`), or a static field written

- by `Volatile.Write` read as never written elsewhere; `DOTNET_TieredCompilation=0` moves the failures, it does

- not avoid them. CI's "Runtime internals" job runs `tests/Nino.RuntimeTests` on .NET (x64, Arm64, Arm32 under

- qemu, Windows x86), Mono (x64, Arm64) and .NET Framework 4.8 (64-bit and 32-bit): give a new path a case there

- with known contents, and its startup flag a row in its `FastPaths`, so a runtime that falls back to the public

- API fails.

- - Prefer one path for every platform. A platform-specific fast path (an intrinsic, `UnsafeAccessor` on .NET 8+,

- `allows ref struct` on .NET 9+, a runtime's internal layout) is allowed only behind `#if` or a startup check, with

- the shared path kept as the fallback; say in a comment which platforms take it, and test that both paths write and

- read the same bytes (run the suite once with the fast path off: `DOTNET_EnableHWIntrinsic=0`, the `netstandard`

- builds of Nino.Core, Mono).

- - The signs in the listing: `Reader:Skip`, `Reader:CopyTo` or `NinoUnsafe:Read` as calls (the budget ran out);

- `NinoReadContext:Return` or `NinoWriteContext:Return` calls and `[rbp-0x..]` loads and stores of the cursor (the

- reader or writer was not kept in registers).

-

- ## Security

-

- Nino reads untrusted data (network messages, files, request bodies): crafted bytes must never crash the process, read

- outside the input, allocate out of proportion to it, recurse without a bound or take time out of proportion to it.

- - Every check of the data throws `NinoDataException`, through a `NinoThrow` helper (out of line). The caller's

- mistakes and the write side (a graph nested too deep or holding itself, a collection changed while written) keep

- `InvalidOperationException`. Tests assert the exact type (`Assert.ThrowsExactly`).

- - Check a length against the bytes left (`TryReadLength`, `NinoLimits.TooLong`) before allocating; grow, never

- presize, what an element larger in memory than on the wire claims. Every recursive path counts depth: class bodies

- through `Depth`, structs and collections that can hold themselves through `EnterStruct`/`ExitStruct` on both the

- reader and the writer, against `NinoSettings.MaxDepth`.

- - Reject bytes no writer writes: a `bool` or presence byte other than 0 or 1 (`ReadBool`, `ReadPresent`,

- `NinoChecks.Bools`), keyed members that do not fill their entry (`EndMember`), sections of deferred bodies that do

- not match their region. A raw block copy of a type with a `bool` in it goes through `NinoLayout<T>.CopiesAsIs`,

- never `IsCanonical` alone.

- - A hash table Nino fills counts the keys in each bucket against `NinoSettings.MaxHashCollisions` (`HashTables`:

- `TryFill`, `Insert`, or `Added`/`Walk` after the table's own add, as `HashTables.Plan` says; `ConcurrentTables` for

- `ConcurrentDictionary`). A new path that adds keys from the data to a hash table does one of these.

- - `NinoSettings.Untrusted` holds the checks that cost time on every read (seeded comparers, value checks of raw

- structs, trailing bytes). It is a constant once frozen (`NinoLimits.Untrusted`): keep each test of it a single

- `if` on the field, with the work out of line, so the default path compiles to what it was; prove it with

- `benchmarks/asm.py compare`.

- - A change here runs the fuzzer (`tests/Nino.Fuzz`: `selftest`, then a timed run with and without

- `DOTNET_EnableHWIntrinsic=0`), the hostile-input unit tests (`StrictReadTests`, `HashFloodingTests`,

- `UntrustedModeTests`, `SettingsTests`, the nesting tests) and the conformance corpus in .NET, Go, Rust and TypeScript.

- A rule the other runtimes must follow goes into `docs/spec/wire-format.md` §9.

-

- ## Source generator

-

- - A generator bug shows up as code that does not compile for one shape of type nobody tested (#203: every generic

- abstract class or interface). Take a name the model may leave empty (`BaseFormatter`, `DispatchRootFormatter`,

- `Construction.Creator`, ...) through `TypeModel.Need`, never by interpolating it directly. Decide what a type gets

- from the model's rules (`DispatchesSubtypes`, `IsDispatchRoot`, ...), not from a new check next to them.

- - A feature that adds a way to declare a type (a kind, an attribute option, a place) adds a dimension to

- `tests/Nino.UnitTests/Corner/ShapeMatrixTests.cs`, so it is compiled and round-tripped with every other one.

- - Generated registration code may call back into Nino.Core (`NinoPolymorphic.Register(Type)`), which asks every

- registry again: anything a registry calls must be re-entrant (`NinoRegistry.RegisterType` guards it per thread).

-

- ## TypeScript runtime (runtimes/typescript)

-

- `src/nino.ts` is the runtime Nino.Codegen embeds and nino-gen writes next to the generated TypeScript. The

- reader takes hostile input as the Go and Rust ones do: check bounds and lengths before allocating, count depth as .NET

- does and defer past it, never build objects whose keys come from the data on a plain object (`__proto__`): use `Map`.

- Before committing: `npm ci && npm test` (type check and the `node:test` suite) in `runtimes/typescript`, and the

- `Nino.Codegen.Integration.Tests` and `Nino.Codegen.Tests` .NET tests.

-

- ## Go runtime

-

- `runtimes/go` (package `nino`, see its README.md) reads and writes the wire format for Go; `internal/sample` holds the

- corpus's types with codecs in the style a generator would write. Both performance and safety on untrusted input are

- requirements:

- - Never panic on any input, and never rely on `recover`: check every bound, and keep errors sticky (after a failure

- the reader returns zeros, empty slices and null, so every loop ends). Check a length against the bytes left

- before allocating, with .NET's caps (`SliceCap`, `capacity`); grow, never presize, what the data only claims.

- - Count depth exactly as .NET does (`ReadClass`, `WriteClass`, regions and sections): the bytes must match, and

- the stack must stay bounded however deep the data. A polymorphic slot accepts only the ids of the types it allows.

- - Hot paths: no reflection, no boxing into `any` outside polymorphic slots; a run of fixed-size members is one

- `Take`/`Extend` and `binary.LittleEndian` at constant offsets; blocks of primitives are one copy (guarded for

- big-endian hosts). Keep rare paths (`Fail`, growth, deferral) in `//go:noinline` functions. Check

- `go build -gcflags=-d=ssa/check_bce/debug=1` and `-gcflags=-m` on what you change, and keep writing a value free

- of allocations (`bench`, `-benchmem`).

- - Struct tags (`tagged.go`, `layout.go`, `marshal.go`) are the one place reflection is allowed: a plan built once

- per type, then walked. Anything the wire format cannot express is a `*TagError` naming the field when the plan

- is built, never a silent default; reads go through the same `Reader` calls as generated code, so the guarantees

- above hold there too. The sample types carry the tags and `NinoLayout` a generator emits (README, "Tags on

- generated structs"); keep them and `internal/sample/tagged_test.go` in step with the hand-written codecs.

- - Before committing: `go vet ./...`, `go test -race ./...` with the conformance corpus present, a short run of

- `FuzzReader`, `FuzzDecode`, `FuzzTagged` and `FuzzTaggedDecode`, and the benchmarks back to back against the

- previous commit for a change to a read or write path. The runtime's module has no dependencies; competitors live in the `bench` module.

-

- ## Rust runtime (runtimes/rust)

-

- - Security: the reader takes hostile input. No input may panic it, read out of bounds, recurse without a bound or

- allocate out of proportion to the data. The crate is `#![deny(unsafe_code)]` and denies `indexing_slicing`,

- `unwrap_used`, `expect_used`, `panic` and `arithmetic_side_effects` (lib.rs): use `get`, `checked_*`/`wrapping_*` with

- a comment saying why it cannot wrap, and errors. The one module allowed `unsafe` is `src/simd.rs` (with its submodules

- `simd/x86.rs` and `simd/neon.rs`, a kernel per instruction set): the string writer's transcoding, and the reader's

- decoding of Latin-1 and UTF-16 into a `String`'s spare capacity (`from_utf8_unchecked` only on what the decoder itself

- writes from code units, so valid UTF-8 by construction; unpaired surrogates become U+FFFD). The reader's bounds checks

- stay safe code: an unsafe decoder takes a `&[u8]` already taken from the input. Every block has a `// SAFETY:`

- comment, its tests compare every block kind and decoder with a plain one for every length up to 256 and random input,

- and the `rust-miri` job in `.github/workflows/runtimes.yml` runs them under Miri (x86_64 with SSE2 only, SSSE3,

- SSE4.1, AVX2 and AVX-512, Arm64, s390x). A kernel for an instruction set stays only where it measured faster on the

- CPUs it targets (`cargo bench -p nino-bench --bench text`, by class and length, and `--bench compare`), with the

- others kept as they were. Add `unsafe` only for a measured win, and only there.

- - Allocation: a collection reserves `r.capacity::<T>(count)`, never `count` itself: the per-read budget (16 bytes per

- input byte plus 4 KiB) stops nested collections each pre-allocating against the same bytes (the fuzz found that).

- Check a count against the bytes left (`length(min_size)`) before reading elements.

- - Depth: class bodies recurse up to `DEPTH_INLINE` and are deferred past it, as .NET does; structs count toward the

- limit through `enter_struct`/`exit_struct`. Never add a recursive path that skips them.

- - Stricter than .NET, never looser: reject what .NET never writes (bools other than 0/1, out-of-range dates, decimal

- flags, non-minimal BigInteger, raw padding, duplicate keys, malformed sections), and read everything it writes.

- A new check goes in README.md's "Stricter than .NET" list.

- - Performance: the generated (and derived) code is what the templates write: a run of fixed-size members is one

- bounds check (`r.array::<N>()`, `w.reserve(n)`), `#[inline]` on every small codec, and rare or large paths

- (`latin1_wide`, UTF-16 decoding, building an `Error`) `#[inline(never)]`. Compare with `cargo bench -p nino-bench`

- back to back on one machine; never publish the numbers.

- - The derive (`nino-serialization-derive`, in `nino-derive/`) generates the same code as the templates; anything the

- wire format cannot express is a compile error naming the field, with a case in `tests/compile_fail/`. Give a new

- attribute a corpus or derive test.

- - Before committing: `cargo fmt --all --check`, `cargo clippy --workspace --all-features --all-targets -- -D warnings`

- and `cargo test --workspace --all-features` with `NINO_CORPUS` set (the conformance, derive and fuzz tests skip

- without a corpus). `tests/conformance/types.rs` is generated by `tests/conformance/gen.py`; edit the generator.

- Builds share one `CARGO_TARGET_DIR`: Rust targets are large.

-

- ## Code generation for other languages

-

- `src/Nino.Codegen` (nino-gen: packed as the Nino.Codegen build package and, through `src/Nino.Codegen.Tool`, as the

- Nino.Codegen.Tool .NET tool) writes TypeScript, Go and Rust from the published layouts (docs/codegen.md):

- - Every option is defined once, in `Options.cs` (`CodegenOptions.All`: its command-line name, the `<NinoCodegen>`

- metadata that sets it, its language and values); `build/Nino.Codegen.targets` passes each metadata as that option,

- and `Nino.Codegen.Integration.Tests` checks the two agree and builds projects with the targets.

- - Logic lives in the IR (`Model/`) and each target's C# (`Targets/`), never in the templates (`Templates/*/*.sbn`),

- which only assemble text from what the target hands them. A decision about the bytes goes into the IR once, for

- every language.

- - Every template or target change is tested end to end against the corpus: regenerate the committed files

- (`NINO_UPDATE_GENERATED=1 dotnet test tests/Nino.Codegen.Tests`), then run the TypeScript (`npm test`), Go

- (`go test -race ./...`) and Rust (`cargo test --workspace --all-features` with `NINO_CORPUS` set) checks, which read,

- write back and attack the generated code with the corpus's valid, malicious and truncated cases.

- - Generated code follows each runtime's performance and safety rules above (bounds checked before allocating, depth

- counted as .NET does, one bounds check per run of fixed-size members, rare paths out of line): what the

- hand-written samples do, the templates write. Anything a language cannot express is a NINOGEN002 warning naming

- the type, never code that reads differently.

-

- ## Commit messages and release notes

-

- - Conventional Commits: `type(scope): summary`, plus a body that says why. Types: `feat`, `fix`, `perf`,

- `refactor`, `docs`, `test`, `ci`, `build`, `chore`; the scope is the area (`core`, `generator`, `collections`,

- `benchmarks`, ...). The release notes group commits by type (`.github/scripts/release_notes.py`, `classify`).

- - Release notes are generated (`.github/scripts/release_notes.py`): the commit list, and highlights from trailers in

- the message's last paragraph, next to `Co-Authored-By`. Give a user-facing change one:

- - `Wire-Format:` anything that changes the bytes Nino writes, and for which types;

- - `Added:` a new API, attribute or behavior;

- - `Performance:` what got faster, for which types, and by how much (measured);

- - `Fixed:` a user-visible bug.

- Write each as one sentence or two for users, in Markdown, continuing over indented lines if needed. Put

- `Closes #N` in the body above the trailers; the issue is appended to the entry. Leave tests, CI, docs and

- refactors without trailers. The docs site's changelog (nino.docs) is written from the same trailers.