# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What this is

**DwarfStar** (`ds4`) is a self-contained native inference engine **purpose-built for DeepSeek V4 Flash** (and, on very high-memory machines, DeepSeek V4 PRO). It is *not* a generic GGUF runner and does **not** link against GGML — it reimplements the loading, tokenizer, prompt/DSML rendering, KV cache, graph scheduling, server API, and a native coding agent for this one model family. It only runs the DeepSeek V4 GGUFs published for this project (asymmetric quant: routed MoE experts at `IQ2_XXS`/`Q2_K`, everything else left high-precision). `README.md`(英文主版) / `README.zh-CN.md` 是对外门面(卖点+指标+用法+工作台截图 docs/img/), 算法/完整结果表/没走通的路/手动安装在 `docs/TECHNICAL.md` / `docs/TECHNICAL.zh-CN.md`; V4 时代的完整功能说明(服务端 API、磁盘 KV、运行时参数、Capability Evaluation)归档在 `docs/archive/README_v4_upstream.md`; 模型见 `docs/MODEL_CARD.md`。

Two production backends, both run the full V4.1 1017 base (113.6 GB) on one 128 GB machine:
- **CUDA on Linux (DGX Spark / GB10)**: the deployment target; every end-to-end gate and every number we publish is measured here.
- **Metal on Apple Silicon (128 GB Mac)**: full model runs since PR #3 (2026-10-19, contributor's M5 Max: decode 35–42 t/s, prefill 627 t/s on 25.5k).
  Our own Mac has **16 GB** ⇒ on it only offline/kernel suites (`--metal-v41`, `--metal-kernels`, `--server`); Mac full-model numbers in the docs are the contributor's, say so.
- The **CPU path is reference/debug only**.

## Layout (2026-08-25 重构后)

单文件 ≤500 行是硬规矩（`make linecount` 守卫，豁免走 `.linecount-exempt`，只收
vendored/单函数 EXCEPTION/冻结转录三类）。源码按模块住 `src/`：

```
src/common/   全仓唯一的格式基元: GGUF 读取(ds4_gguf)、量化块 dequant(ds4_quantfmt,
              含 iq2_xxs 编码/解码双表正名)、E4M3/E2M1(ds4_fp8.h)、f16(ds4_float.h)、
              safetensors(ds4_st.c)。引擎与 gguf-tools 共用这一份——改格式只改这里。
src/core/     引擎主体(原 ds4.c 24.6k 行拆 60 文件): gguf 加载/权重绑定/CPU 前向/
              GPU graph 编排/tokenizer/采样/session 与 payload。内部头 core_internal.h。
src/metal/    Metal 后端(原 ds4_metal.m 拆 62 文件); shader 在顶层 metal/(运行时按
              相对路径拼接加载, 移动该目录=运行时炸)。V4.1 原语(前向/反传/草稿蒸馏)在
              metal_v41_*.m / metal_bwd_*.m / metal_draft_kd.m + metal/v41_*.metal(2026-10-08);
              只缺解码整步 graph(Metal 无流捕获, core 走直发)。单测 `ds4_test --metal-v41`。
              整模型在 Mac 跑通靠 PR #3(2026-10-19): ①VQ blob 不在 span 里, metal_model_map.m 首用时懒映射
              并入常驻集(不入集驱动卡 30~1000 ms; 这些视图不进 --mem-budget-mb 加载闸) ②推理路预填 half 激活
              快核(metal/v41_prefill_mm.metal、v41_vq_mm.metal, TensorOps/simdgroup)只在契约
              ds4_gpu_v41_set_half_prefill(1) 期间走, 由 ds4_v41_state.half_prefill 驱动(generate/req/score 置 1);
              训练原语与 --metal-v41 留 f32 原核(反传重算要逐位一致), CUDA 侧是空实现。
src/cuda/     CUDA 后端: ds4_cuda.cu 是聚合根, 分片在 src/cuda/*.inc.cu(单 TU 语义)。
              ds4_gpu.h(伞头)+六个子头是 Metal/CUDA 共同契约, 已带 extern "C" 守卫,
              CUDA 直接 include——契约漂移=编译错误。
src/dist/     分布式运行时(层切片/TCP 协议/coordinator/worker/TP)。
src/server/   HTTP 服务(OpenAI/Anthropic/Responses); 内嵌测试已解耦到 tests/。
src/assistant/ 工作台个人助理(住 ds4-train 主进程; 每次工具调用 fork 子进程先上内核沙盒: Linux Landlock+seccomp / macOS Seatbelt; 火狐 BiDi 上网; 方案 docs/assistant.md; 技能 skills/<名>/SKILL.md 见 docs/assistant_skills.md; 任务 tasks/<id>.json + 专属会话 + 调度线程, 周期由模型用 task_check_in 定, 见 docs/assistant.md §6; 通知 = Web Push(密码学 src/push/ 纯 C: P-256/AES-GCM/HKDF/VAPID, 发送走 curl), notify 工具 + 任务做完/长运行两条机械规则, 页面开着在看时不推改页内弹, §6.1; 单测 ds4_asst_test + ds4_push_test)。
src/db/       连接器的数据库协议(10-19, 方案 docs/assistant_connectors.md): 纯 C 线协议, 零依赖 —— BSON + MongoDB OP_MSG + SCRAM-SHA-256(PostgreSQL 共用) /
              PostgreSQL v3 / MySQL 握手+两种口令插件(SHA-1、RSA-OAEP 大数在 mysql_auth.c) / Redis RESP2; SQLite 走沙盒里的 sqlite3。只读是代码保证(只有读命令、
              只读事务、命令白名单)。种类表 db.c DB_KINDS 驱动页面下拉与 query_db 的 schema; 连接器本体 src/assistant/asst_conn.c, 工具 tools/tool_db.c; 单测 ds4_asst_test 的 asst db。
src/agent/    终端编码 agent。src/cli/ src/eval/ src/bench/ 各叶子程序。
src/kv/       磁盘 KV checkpoint。src/web/ agent 网页抓取。
gguf-tools/   离线量化与反修工具链(独立 Makefile): quantize/(HF→GGUF 量化器与编码格式)
              amp/(反修: 放大器解算/逐层调优/侧车链) calib/(标定数据面) bench/(判决尺:
              anchor_metrics/pubbench/quality-testing) scripts/(活脚本) data/(冻结数据)
              legacy/(冻结旧代) docs/ migrate/(迁移金标账本 golden.txt)。
tests/        ds4_test 套件(tests/t_*.c)+server 测试(server_tests_*.c)+单元测试
              (unit/)+金标夹具(fixtures/quantfmt: dequant 七类型逐字节金标)。
docs/         根目录以外的全部 md(MODEL_CARD/CONTRIBUTING/AGENT 与各战役方案稿 speed/decode/back 等); 根目录只留 README×2、CLAUDE.md、fable5.md。
docs/archive/ 已收官战役的设计稿。根目录还剩: 公共头(ds4.h/ds4_gpu*.h 等)、
              vendored(rax/linenoise)、小模块(ds4_z/ds4_loss/ds4_corr/ds4_zchain/
              ds4_multimodal/ds4_spatial/ds4_css/ds4_posttrain)、Makefile、文档。
```

## Build

The build is a single hand-written `Makefile` (no CMake/configure). Output binaries land in the repo root.

```sh
make              # macOS: Metal 版 ds4, ds4-server, ds4-bench, ds4-eval, ds4-agent, ds4-train, ds4-seg (default target)
make cuda-spark   # Linux: = cuda-generic 的别名(2026-10-07 起无 Spark 专属旗; 统一内存/独显由引擎运行时按设备属性判)
make cuda-generic # Linux: CUDA for a generic local GPU (CUDA_ARCH=native)
make cuda CUDA_ARCH=sm_120   # Linux: CUDA with an explicit nvcc arch
make cpu          # CPU-only reference/debug build (adds -DDS4_NO_GPU)
make clean
make -C gguf-tools all amp legacy calib bench tools-test   # 工具链全目标(Mac/Linux 自适应)
```

- On **Linux**, plain `make` only prints the target list — you must pick a `cuda-*`/`cpu` target explicitly.
- C is **C99**, `-O3 -ffast-math`, native CPU arch. GPU kernels: `src/metal/*.m` (Objective-C, `-fobjc-arc -fno-common`) 运行时拼接 `metal/*.metal`；`ds4_cuda.cu`(聚合根) 由 `nvcc` 编译。
- 对象文件放源文件旁(`src/<mod>/*.o`)。每个程序一个 `<MOD>_OBJS` 变量；`CORE_OBJS` = core+common+dist+后端。CPU 变体同源加 `-DDS4_NO_GPU`(`*_cpu.o`)。

## Run

```sh
./download_model.sh q2-imatrix   # fetch a model into ./gguf/ and point ./ds4flash.gguf at it
./ds4 -p "Explain Redis streams" # one-shot;  no -p => interactive REPL
./ds4-server --cuda -m gguf/v41/<model>.gguf --zchain <反修目录>   # V4.1: 上下文只有 1M 一个取值, 没有 --ctx(spark 上用 gguf-tools/scripts/serve_1m_spark.sh)
./ds4-server --metal -m <model>.gguf --zchain <反修目录> --engram-dir <分片目录> --mem-budget-mb 110000   # 128 GB Mac; 下载走 hf_install.sh download(无 Linux 预检), 用法见 README "Get started"
#   Mac 上工作台(ds4-train)的下载/加载按钮没人跑过; 整模型只有贡献者的 M5 Max 跑过 ds4-server 这条
#   起服后 GET /monitor = 监控页(web/monitor.html, 指标照 Strata Monitor), GET /metrics = 数据(JSON; Accept: text/plain 给 Prometheus 文本)
./ds4-agent                      # in-process native coding agent (sessions in ~/.ds4/kvcache)
./ds4-train --port 8000          # ★工作台主进程, 永远不退★(10-10 用户定): GET / = web/studio.html(聊天/模型/训练/语料/记录), 全部 /api/*; spark 上 train_ui_spark.sh start
#   模型 = 它直接 fork 的子进程 ds4-server(src/train/train_model.c, 绑 127.0.0.1:8001, 起跑清单/冒烟/看门狗都在 C 里), /v1/… /monitor /metrics 与热切(/api/models/plugins)、清单(/api/models/list)由主进程转发(src/train/train_proxy.c)
#   训练 = 它自己的作业线程(src/train/train_job.c: 停模型子进程 → fork ./ds4 --ptrain → 选轮(train_gate.c: 留出损失比第 0 步低且收口不少于部署态的轮里最低者; wt2 两臂配对 ± SE 只报数不否决, 10-11 用户定) → 子进程按 serve_pick.txt 装回来)
#   ★页面不调任何 shell 脚本★(10-10 用户令, 只剩下载 hf_install.sh); serve_1m_spark.sh / z_nightly_spark.sh / v41_judge.sh 只是命令行工具, 产物目录与格式和 C 一致
#   模型页"加载"= 写 gguf/serve_pick.txt(C 直接写, 格式与 serve_1m_spark.sh pick 同)+ 重起子进程; 聊天页下拉 = 热切侧车, "后训练"开关 = 挂/卸当前侧车关联的最新 ③(server_plugins.c, 不重装底座)
#   ★训完不自动挂★(10-10 用户: "训练完就是训练完不要直接挂"): ③ 只落盘并关联训练时装着的侧车(base.fnv), 开关打开才挂, 切侧车跟着换
```

`./ds4flash.gguf` (a symlink) is the default model for every binary; pass `-m gguf/<file>` to override. Use `--metal`/`--cuda`/`--cpu` to force a backend, `--chdir /path/to/ds4` when launching from elsewhere so `metal/*.metal` resolves.

## Test

Read `docs/CONTRIBUTING.md` before changing inference code — it defines the correctness and speed regression tracks.

```sh
make test                      # ds4_unit(离线金标) + linecount 守卫 + 抽取器自测 + ds4_test 全套
./ds4_test --list              # 11 个 suite; 模型缺失的套件打 SKIP 不 FAIL
./ds4_test --server            # 104 项服务端单测(离线): API 解析/DSML/流式/KV-disk
./ds4_test --engine-units      # 采样与 per-request 惩罚单测(离线)
./ds4_test --rax               # 基数树单测(离线)
./ds4_test --metal-kernels     # 孤立 Metal kernel 数值(要 Metal 设备, 不要模型)
./ds4_test --tp-allreduce      # TP 传输回环(离线)
./ds4_test --logprob-vectors   # 对官方 API 逐位对拍(要真模型)
./ds4_test --long-context / --tool-call-quality / --local-golden-vectors / --metal-short-prefill / --metal-tensor-equivalence   # 其余真模型套件
./ds4_unit                     # src/common 金标: dequant 七类型逐字节 + fp8 + GGUF 往返
make cuda-regression           # Linux/CUDA only(首跑含 PTX JIT 可能超时, 复跑为准)
make -C gguf-tools tools-test  # 工具链 9 项 -D*_TEST 自测
```

Override test inputs with `ds4_test` flags: `--model FILE`, `--vector-file FILE`, `--long-prompt FILE`, `--local-golden FILE`, `--mpp-case NAME`, `--keep-metal4`.

For generation-drift-sensitive changes, also run the deterministic q1..q4 eval gate (expected token counts are in `docs/archive/README_v4_upstream.md` → Capability Evaluation):

```sh
./ds4-eval -m ds4flash.gguf --plain --questions 4 --tokens 2048 --temp 0 --seed 1
```

Speed regressions use `ds4-bench` (instantaneous prefill/gen t/s at context frontiers, not a whole-run average):

```sh
./ds4-bench -m ds4flash.gguf --prompt-file speed-bench/promessi_sposi.txt \
  --ctx-start 2048 --ctx-max 65536 --step-incr 2048 --gen-tokens 128 --csv /tmp/ds4-speed.csv
```

Quantization/GGUF changes are scored with `gguf-tools/bench/quality-testing` (`make -C gguf-tools quality-score`, then `score_official` + `compare_scores`; lower `avg_nll` is better).

## Architecture (the cross-file picture)

- **Backend abstraction.** `ds4_gpu.h`(伞头, 六子头带 extern "C") is the GPU tensor/graph
  interface, implemented by `src/metal/` and `src/cuda/`; `-DDS4_NO_GPU` selects the CPU
  reference in `src/core/`. `src/core/` owns everything host-side. Model weights are
  **mmap-backed and never eagerly copied** — Metal wraps mmap regions as no-copy
  `MTLBuffer` views and uses an `MTLResidencySet` to budget GPU VM.

- **Compressed KV cache is a first-class *disk* citizen.** DeepSeek V4's KV is heavily compressed (raw sliding window + ratio-4 indexer-selected + ratio-128 compressed layers). `src/core/core_payload*.c`/`core_snapshot_*.c` serialize the `DSV4` session payload; `src/kv/` manages the on-disk KV cache (`<sha1>.kv`, plain read/write — *not* mmap). The server keeps exactly **one live in-memory checkpoint**; the disk cache is the resume mechanism. File format (48-byte `KVC` header, rendered text, `DSV4` payload, optional `KTM` tool-id map) documented in `docs/archive/README_v4_upstream.md` → Disk KV Cache.

- **Server (`src/server/`).** OpenAI/Anthropic/Responses-compatible HTTP. Inference serialized through one graph worker. Key subtlety: stateless clients resend JSON tool calls, so the server keeps an **exact-DSML replay map** (tool id -> exact sampled DSML bytes, radix-tree via `rax.c`) so re-rendered prompts byte-match the live KV checkpoint. During tool-call *syntax* the server forces `temperature=0`.

- **Distributed inference (`src/dist/`).** Layer-sliced across machines (`--role coordinator|worker`, `--layers 0:19` / `20:output`). Custom binary TCP protocol; activations flow worker→worker, coordinator owns tokenization/sampling. A rolling 64-bit token-prefix hash guards stale worker KV. Saved sessions use the same single-file `DSV4` payload.

- **Native agent (`src/agent/`).** Drives inference in-process; the session *is* the on-disk KV cache. `/list`, `/switch <sha>`, `/save`, `/del`, `/strip`.

- **反修/量化工具链 (`gguf-tools/`).** 独立 Makefile。判决尺=参考前向(`caliper_ref.sh`,
  五指标, 见 memory 铁律); 迁移金标账本在 `gguf-tools/migrate/golden.txt`。格式基元
  一律走 `src/common`——工具侧禁止再抄第二份 dequant/GGUF/safetensors 实现。

**全仓零 env 配置**(2026-08-31 大扫除收口): 引擎与工具链不读任何 `DS4_*` 环境变量——行为写死进代码(命名常量+why 注释), 取料/运维入口全走 CLI 参数(`--cap-dir`/`--eval-ids`/`--mem-budget-mb`/`--spec` 等, 见 `docs/archive/README_v4_upstream.md` → Runtime Configuration)。**禁止新增 env 配置**(铁律 2026-08-22), 脚本 `${VAR:-默认}` 直喂二进制同样算。

## 当前主记录

**`fable5.md` 是项目主记录**(用户 2026-07-02 指定): 结论、判决、战役日志全在那里,
重构(2026-08-25, restructure 分支)的全部记录也在。`docs/archive/` 存已收官战役的
设计稿(双机 q2 提速的 project.md 等 6 月文稿在归档前的根目录历史里)。北极星=
合适体积+模型全能力还原(非编程域专项), "go"命名的战役设施已随 2026-08-25 重构清退。

## Guardrails (hard)

- Watchdog red line **12/12 GiB** (both hosts on dual-Mac runs); `--mem-budget-mb` + L1 resident gate must refuse startup over budget (flag 不传=护栏不武装, 加载大模型的脚本必须传). **Any model-loading script must prove memory safety (RSS budget + watchdog) before it runs.** Never double-load an 80+ GiB base on one host.
- After a local rebuild, consider whether the other machine needs the synced binary too (shared engine objects). Never delete files on the other host without explicit confirmation — "cleanup" means killing the process, not removing files.
- **Per-merge correctness gates**: `make test` 全绿(含 linecount); 推理路改动加
  `--dump-logprobs` parity 与 `ds4_test --logprob-vectors`(有模型的机器);
  routing/quant/repack changes additionally run `ds4-eval q1..q4 --temp 0 --seed 1`。
  量化质量判决只认参考前向尺(caliper), 真代码基准必须 completions 口径(memory 铁律)。

## Project rules (from `docs/AGENT.md` — follow these)

- **No C++.** Pure C99, with Objective-C only where Metal requires it. Python 禁止进入
  数值/算法链(全仓零 Python 裁决; 仅存的 .py 是外部 API 采集/绘图/金标夹具生成器)。
- **Keep model loading mmap-backed**; do not eagerly copy the full GGUF.
- **Correctness before speed.** Don't keep a faster path with unexplained attention/KV/logit drift.
- **单文件 ≤500 行**(make linecount 强制); 注释写"为什么"不复述代码。
- **★只写 GPU 版本, 禁写 CPU 路径★**(铁律 2026-08-28): 新代码一律只走 GPU; 缺原语就接
  生产的 GPU 实现, 不许为了"能编过"补标量参考版。单测/探针同样必须走生产 GPU 路
  (`-DDS4QUANT_CUDA` + 链 `vq_gpu.o` + `-lcudart -lcublas -lstdc++`; `vq_gpu.o` 是 C++
  目标, 缺 `-lstdc++` 会报 `__cxa_guard_acquire` 未定义)。实撞代价: 单测走了
  CPU 参考路, 量化报 4.33s/矩阵(推一层 55 分钟), 而生产 GPU 一层只要 60-80s —— **失真 40 倍
  且自洽**, 拿它对比目标全是废话。已有的 CPU fallback 是历史遗留, 不扩散不新增, 不拿它出
  任何速度/质量读数。
- **macOS CPU danger:** running the CPU inference path on macOS can crash the kernel — avoid large CPU runs there.
- **Instance lock is intentional:** do not run multiple huge-model processes concurrently.

## Debugging

```sh
./ds4 --dump-tokens -p "..."                              # tokenize (incl. DS4 specials) and exit
./ds4 --dump-logprobs /tmp/out.json --temp 0 -p "..."     # greedy continuation + top-k alternatives
./ds4 --score-ids ids.txt --score-out /tmp/score.bin      # teacher-forced 逐位 logits(解码路, 100% 决定论)
./ds4-server --trace /tmp/ds4-trace.txt ...               # log rendered prompts, cache decisions, tool events
```