AGENTS.md@benchmarks ยท diff
git:20261029.0fb7586 to git:20261030.e7612a4
1 added, 1 removed. Audit A to A.
# Benchmarks: file map
> Read when: adding, changing or running a .NET benchmark, or a script that turns benchmark results into the README, docs or a release body.
BenchmarkDotNet projects and the Python and shell scripts around them. benchmarks/README.md is the reference for
every option, class and environment variable: read it instead of guessing. The runtimes outside .NET keep their
benchmarks in `runtimes/*/bench`. `audit-code` audits the scripts, `benchmark` runs them, `audit-perf` and `optimize` use them.
```
README.md options of run.sh, the classes, MessagePack / protobuf-net, member by member, machine code
(asm.py, cross JIT), the runtimes' benchmarks
run.sh the published runtime benchmark: v3 3.3.13 then the full v4 suite on one machine (QUICK=1,
FILTER=...); writes benchmarks/results/ (ignored)
Nino.Benchmark/ the BenchmarkDotNet suite: Nino against MemoryPack, MessagePack and protobuf-net per class
(SimpleTest, PolymorphicTest, GenericPolymorphicTest, PooledTest, PooledGraphTest,
ReferenceTest, VersionTolerantTest, FixedSizeTest, CommonTypesTest, CollectionsTest,
BclTypesTest, ScatteredTest, UntrustedTest), CpuProbe (block copies against Memmove),
Verify.cs (--verify: every case reads back right), NinoJobAttribute (the job, quick mode), PayloadColumn; V3/ +
NINO_BENCH_V3=true: the v3 3.3.13 build of SimpleTest; -p:NinoBenchLegacy for older refs
Nino.GeneratorBenchmark/ the source generator on a synthetic workload: generator time, generated size, emit,
analyzer, re-generation after edits
shards.json the classes and the filters of each CI shard: "report" (report.yml, x64 and Arm64) and
"ab" (ab-bench.yml)
ab.py two github reports of the same benchmarks (base, head): head against base and MemoryPack
asm.py the Tier1 machine code of the cases: dump / diff / compare REF (`compare origin/main`
before committing a read or write path change)
jitwarm/ the startup hook asm.py runs Nino.Benchmark with: calls each case until Tier1, then exits
cross-jit.sh installs the JIT for the other 64-bit architecture, so DOTNET_JitDisasm prints x64 and
Arm64 code on one machine
members/ member-by-member v3 against v4 (run.sh: v3, v4, v3 in process; DISASM=1 for listings):
shared/Parts.cs built as v3/P3.csproj (NuGet 3.3.13) and v4/P4.csproj; table.py,
split-asm.py; results/ ignored
bdn.py the one parser of BenchmarkDotNet github reports, units and FLOOR_NS (1 ns), for the
- scripts below
+ scripts below; show: the time format of every release-body table
merge_reports.py merges the reports of parallel jobs into one report (report.yml)
compact_report.py shortens a merged report for a release body (125,000-character limit)
compare_v3.py v3 3.3.13 against v4 from two SimpleTest reports, with both MemoryPack runs
readme.py writes the numbers of README.md / README.zh.md, the chart (docs/assets) and the runtime
tables of docs/v4-benchmarks.md from a joined report
release_body.py the release body's benchmark sections (Performance Report, v3 comparison, Arm64) for
report.yml; the docs site finds them by their headings
runtimes_report.py the release body's Go, Rust and TypeScript sections from each benchmark's output
unity_report.py the release body's Unity section from the IL2CPP player's results (unity.yml artifacts)
test_unity_report.py tests of unity_report.py on fixtures/
fixtures/ unity-bench-<version>/ results.xml + build.txt, and unity-bench-empty/, for those tests
.gitignore results/
```
## Rules for these files
- Headings the release body uses (release_body.py, runtimes_report.py, unity_report.py) are read by the docs site
(Nino.Docs `scripts/sync-benchmarks.ts`): change one only together with its parser there.
- A script that reads BenchmarkDotNet reports does it through `bdn.py` (one parser, one unit table, one floor);
the 1 ns floor is the docs site's (`src/lib/benchmarks.ts`) too.
## Running them
How to run, A/B on runners and what may be published: the `benchmark` skill.