DESIGN.md@docs Β· git:20260731.4bb913c Β· 2026-07-31 Β· sha256 1a08e6d40a1622e6

DESIGN.md@docs git:20260731.4bb913cA

Immutable. This exact content never changes and is served at /api/v1/blob/1a08e6d40a1622e6.

> ## πŸ›οΈ Historical design record β€” frozen 2026-07-15
>
> **This is the original pre-build design doc, kept for history. It is NOT the current architecture.**
> Read **[ARCHITECTURE.md](ARCHITECTURE.md)** for what the code actually is today. The build diverged
> from this plan in several deliberate ways:
>
> - **The design is built, not "pending sign-off."** The status line below is frozen at the design pass.
> - **The llama.cpp engine shipped.** It's described here as *planned* (Β§1, Β§6, Β§2.5); it is now a
>   second, user-selectable engine behind the same `LLMEngine` protocol, with a `RoutingEngine` that keeps
>   one resident. See [WIRING.md](WIRING.md) and ARCHITECTURE.md.
> - **The catalog grew far past Bonsai.** This doc seeds only Bonsai (Β§1.1, Β§3). The shipped Featured
>   catalog is **12 models across 5 families** (Bonsai, Qwen, Hunyuan, DeepSeek, Gemma), plus an **Explore**
>   tier that browses Hugging Face live (MLX + GGUF) and flags community models *Unverified*.
> - **The accent is no longer teal.** The teal "on-device intelligence" hue (Β§1.2, Β§8) was replaced by an
>   **ink-wash (ζ°΄ε’¨)** palette β€” warm rice-paper surfaces and a cinnabar-seal accent, sampled from the app icon.
> - **Features this doc doesn't cover, now shipped:** tool calling + an **MCP** client (remote Streamable-HTTP
>   servers), a **model-aware context ladder** (rungs capped by the model, re-scored per rung), the
>   **thinking-disclosure** interaction, and a stock, editable **system prompt**.
> - **β€œNothing leaves the device” below describes the inference path, not every optional feature.** Model
>   discovery/downloads contact Hugging Face, and web/Wikipedia/MCP tools contact their documented remote
>   endpoints only when enabled or invoked. The app has no account, analytics, or telemetry.
>
> Everything below is preserved verbatim as the record of the initial design intent.

---

# mobileLLM β€” Design

*An open-source, private, on-device runner for open-weight LLMs on macOS + iOS (Swift + MLX). Everything
runs on the user's own silicon β€” no account, no cloud, nothing leaves the device. It is built to host
many open-weight model families over time: Prism ML's Bonsai (1-bit) is the first included, and the goal
is **β‰₯7 families**. Two inference engines sit behind one protocol β€” an **MLX** engine today, and a
**llama.cpp** engine planned for large models on memory-tight phones (mmap'd weights). The 1-bit weight
format (`Q1_0`) is already merged into mainline llama.cpp.*

Status: **design approved-for-build pending sign-off.** Produced from a multi-agent design pass
(foundation audit Β· engine wiring Β· model catalog Β· architecture Β· product/UX Β· adversarial critique),
2026-07-15. Every "verified" number below comes from a Hugging Face primary source (config.json /
safetensors index), not a model card.

---

## 1. What we're building & the honest constraints

A commercial-grade, private chat app (ChatGPT-app calm Γ— Raycast keyboard-speed Γ— Linear structure)
where the model runs on the user's own silicon. Two facts shape **every** decision:

1. **Weights must be RESIDENT (MLX engine).** LLM decode is bandwidth-bound at batch-1: streaming
   weights from flash = seconds/token (unusable). On the MLX engine there is **no** two-phase /
   weight-streaming rung β€” a model either fits in RAM or it cannot run, and the only memory lever is
   the KV cache. (Fitting a large model on a memory-tight phone is exactly what the planned llama.cpp
   engine's mmap'd weights are for, trading tok/s for footprint.)
2. **1-bit needs the fork.** `bits=1` is not in upstream MLX (only {2,3,4,5,6,8}); the 1-bit Metal
   kernel lives in `ml-explore/mlx` PR **#3161** (open/unmerged) and ships in the PrismML forks
   (`PrismML-Eng/mlx`, `PrismML-Eng/mlx-swift@prism`). **mobileLLM uses the fork**, quarantined behind
   the `LLMCore` package so it is the only place in the app that ever sees a non-upstream MLX.

### 1.1 Verified model facts (HF, live)

| Model | Repo | model_type | Layers / hidden / vocab | Native ctx | On-disk |
|---|---|---|---|---|---|
| **Bonsai-27B** | `prism-ml/Bonsai-27B-mlx-1bit` | `qwen3_5_text` (hybrid GDN) | 64 (48 linear + **16** full) / 5120 / 248 320 | 262 144 | **5.13 GB** |
| Ternary-27B | `prism-ml/Ternary-Bonsai-27B-mlx-2bit` | `qwen3_5_text` | same | 262 144 | **8.49 GB** |
| **Bonsai-8B** | `prism-ml/Bonsai-8B-mlx-1bit` | **`qwen3`** (dense) | 36 / 4096 / 151 669 | 65 536 | **1.28 GB** |
| Ternary-8B | `prism-ml/Ternary-Bonsai-8B-mlx-2bit` | `qwen3` | 36 / 4096 / 151 669 | 65 536 | **2.30 GB** |
| Bonsai-4B | `prism-ml/Bonsai-4B-mlx-1bit` | `qwen3` | 36 / 2560 / 151 669 | 32 768 | **0.63 GB** |
| Bonsai-1.7B | `prism-ml/Bonsai-1.7B-mlx-1bit` | `qwen3` | 28 / 2048 / 151 669 | 32 768 | **0.27 GB** |

Quant: `group_size 128`; 1-bit `{bits:1}` (Binary), ternary `{bits:2}` (Ternary). EOS `<|im_end|>`.
Every repo ships `chat_template.jinja` (ChatML/Qwen) and is **thinking-mode capable** (`<think>`).
License **Apache-2.0**.

**Two corrections the research surfaced (important):**
- **Only the 27B is `qwen3_5`** (Gated-DeltaNet hybrid β†’ needs `Qwen35.swift` + `MambaCache`, already
  in mlx-swift-lm). **8B / 4B / 1.7B are plain `qwen3` dense** β†’ the standard `Qwen3.swift` (upstream).
  The small models are architecturally simpler.
- **AWQ-4bit and gguf variants are excluded.** AWQ is AWQ-gemm (non-MLX-native) and bundles an fp16
  vision tower (6–19 GB); gguf is a llama.cpp format (out of scope for the MLX engine β€” a target for
  the planned second engine). The real stock-MLX path is the **ternary 2-bit** variants.

### 1.2 The feasibility matrix (peak β‰ˆ weights + ~0.5 GB runtime + KV@ctx)

Usable app ceilings: **8 GB iPhone 16 Pro β‰ˆ 5.3 GB** (jetsam ~5.5 GB) Β· 12 GB iPhone β‰ˆ 8.5 GB Β·
Mac β‰ˆ RAM βˆ’ 4 GB.

| Variant (peak@4K) | 8 GB iPhone 16 Pro | 12 GB iPhone | Mac |
|---|---|---|---|
| **27B 1-bit** β€” ~5.9 GB | πŸ”΄ **not supported** (weights 5.13 alone β‰ˆ ceiling; short-context can't help β€” it's the *weights*, not the KV) | 🟒 green | 🟒 green |
| 27B ternary-2bit β€” ~9.3 GB | πŸ”΄ | πŸ”΄ (weights 8.49 > 8.5) | 🟒 green |
| **8B 1-bit** β€” ~2.4 GB | 🟒 **green (the iPhone hero)** | 🟒 | 🟒 |
| 8B ternary-2bit β€” ~3.4 GB | 🟒 | 🟒 | 🟒 |
| 4B / 1.7B (either quant) | 🟒 | 🟒 | 🟒 |

> **Honesty note (critique A1) + user decision:** on paper the 27B weights (5.13 GB) sit right at the
> 8 GB 16 Pro's ceiling before a single KV byte, so it is *expected not to fit* and short-context can't
> help (it's the weights, not the KV). **The user wants the device, not an estimate, to decide.** Fit
> badges therefore remain honest, advisory labels, but every installed variant has the same **Use**
> action and always attempts the real load (with the `increased-memory-limit` entitlement + KV-4bit +
> a low context cap for the best shot). iOS may terminate a process that exceeds its limit; the next
> launch remains safe because launch never loads weights automatically. On Mac + 12 GB iPhone it is a
> normal green flagship. **8B-1bit stays the recommended iPhone default, not an enforced fallback.**

**Defaults:** Mac β†’ 27B-1bit Β· 12 GB iPhone β†’ 27B-1bit Β· **8 GB iPhone β†’ 8B-1bit** (27B-1bit selectable
as Experimental).

**Accent:** mobileLLM's identity is a **teal "on-device intelligence" hue**. Light `#0D9488` / dark
`#2DD4BF`, with a low-opacity `accentSoft`; the `Theme` token indirection makes this a self-contained ramp.

The 27B's hybrid attention is a genuine selling point: only its **16 full-attn layers grow a KV cache**
(the 48 Gated-DeltaNet layers hold fixed `MambaCache` state) β†’ **near-constant memory as context grows**
(~64 KB/token vs 8B's ~144 KB/token).

---

## 2. Architecture

Three tiers: an **MLX-free foundation layer** (kept inline for v1), a **fork-quarantined engine**, and the app.

```
mobileLLM.app  (Xcode target β€” build via xcodebuild only; SwiftPM can't compile the fork's Metal)
β”œβ”€β”€ imports  Shared* (copied), LLMCore
β”œβ”€β”€ @MainActor  ChatStore Β· ConversationVM Β· views (Chat / History / Models / Settings)
└── ModelManager (actor)  β€” download + lifecycle orchestration

LLMCore  (Swift package β€” THE ONLY place the fork appears)
β”œβ”€β”€ LLMEngine (actor) Β· ThinkSplitter Β· Sampling Β· LLMMemoryGovernor Β· LLMCatalog Β· schema
└── deps β†’ nanguoyu/mlx-swift-lm (fork; one repointed dep line β†’ PrismML-Eng/mlx-swift@prism)

Shared* β€” MLX-FREE foundation code (v1: kept inline in the app; see Β§5)
β”œβ”€β”€ UI: Theme, Motion, Chip, Segmented, studioCard, toastBanner
└── Runtime: ModelDownloader (resumable .part streaming), DownloadMeter, MemoryProbe,
             DeviceTier, ThermalGovernor(clearCache injected), GenerationControl, DurableStore
```

### 2.1 The fork quarantine (SPM identity trick)

`ml-explore/mlx-swift` and `PrismML-Eng/mlx-swift` share SPM identity `mlx-swift` β†’ an app can hold
only one. mobileLLM resolves the **fork** everywhere. The mechanism: fork `mlx-swift-lm` and edit its
**single** dependency line to point at `PrismML-Eng/mlx-swift@prism`; `LLMCore` depends on that fork.
Vendor both fork repos as git submodules under `Packages/` with `.package(path:)` (lets the exact
Metal tree be reviewed before it ships).

> **Critique B1 / B4 (do this FIRST, before any UI):** the real risk is not URL identity β€” it's whether
> the pinned `mlx-swift-lm` compiles against the fork's `mlx-swift 0.31.1` base (version constraint +
> API drift: `KVCache` shape, `generate` signature). This can require editing the version requirement
> and rebasing the prism kernels onto a newer base β€” budget multi-day. **Gate the pin on: clean
> `xcodebuild` + a `bits=1` smoke-decode on Bonsai-8B returning non-garbage.** Vendor with
> `--recurse-submodules`; CI-assert the generated `quantized.h` is present. Pin the branch HEAD
> (full mobile-decode kernel stack), not the old `v0.0.1-prism` tag.

### 2.2 `LLMEngine` (actor, in LLMCore)

Sits one level below `ChatSession` so the product can regenerate / edit / branch, trim context, and
instrument peak memory.

```swift
public actor LLMEngine {
    func load(_ id, variant, weightsDir, progress) async throws   // LLMModelFactory.loadContainer
    func unload() async                                            // container=nil; MLX.Memory.clearCache()
    func generate(messages:[ChatTurn], params:Sampling) -> AsyncThrowingStream<Delta, Error>
    enum Delta { case reasoning(String); case answer(String); case done(Stats) }
    struct Stats { promptTokens, genTokens; promptTPS, tokensPerSecond; peakMemoryBytes; stopReason }
    struct Sampling { maxTokens; temperature=0.7; topP=0.95; topK=20; repetitionPenalty=1.05;
                      seed?; thinking=true; contextTokenCap=8192; kvBits=4; quantizedKVStart=256 }
}
```

- **load** β†’ `LLMModelFactory.loadContainer(directory:)` (offline; weights arrive via Β§5). The factory
  builds `Qwen35Model` (27B) or `Qwen3Model` (8B/4B/1.7B) and rebuilds `Linear→QuantizedLinear` from
  `{bits:1, group_size:128}` β€” **this is where the fork kernel is mandatory** (upstream asserts
  `bits∈{2,…}` β†’ a first-decode assert is the canary that the wrong mlx-swift resolved). iOS:
  `MLX.Memory.cacheLimit = 32<<20` (weights are resident; give the reuse pool little).
- **generate** runs in `container.perform { … }`: build `UserInput(chat:)`, set
  `additionalContext=["enable_thinking": params.thinking]`, apply the repo `chat_template.jinja` via
  swift-jinja, `resetPeakMemory()`, iterate `MLXLMCommon.generate`. Each chunk β†’ `ThinkSplitter.feed`
  β†’ `.reasoning`/`.answer`; `.info` β†’ `.done(Stats)` reading `MLX.Memory.peakMemory`. Per-token:
  `Task.checkCancellation()` + `thermal.throttleIfNeeded()`.
- **KV 4-bit**: `kvBits=4` quantizes each cache to `QuantizedKVCache` after `quantizedKVStart` tokens;
  it correctly **skips `MambaCache`** (never quantize recurrent state). Enforce `contextTokenCap` by
  history-trim before prefill (always keep the system turn).

### 2.3 State & data flow (tap β†’ streamed token)

`@MainActor @Observable ChatStore` owns all UI state; the engine is an actor; they talk over the
`AsyncThrowingStream`. On send: append user + empty-assistant messages, build `[ChatTurn]` from
history trimmed to `contextTokenCap` + system prompt, launch `genTask`; each delta hops to `@MainActor`
to grow a small `StreamingState { reasoning, answer, phase, stats }`; `.done` commits into the message
and `ConversationStore.save`. Only the streaming strings mutate per token (no array churn β†’ smooth).

> **Critique D1 / D3 / D4 (production correctness):**
> - **Stop is a cooperative boundary-stop**, not instant. A long **prefill** on 27B is one
>   uninterruptible MLX op β€” Stop lands at the next token boundary. UX says "Stopping…", never "instant".
> - **Backgrounding with 5 GB resident = jetsam bait.** On `scenePhase β†’ .background`: cancel + commit
>   any partial answer, lower `cacheLimit` + `clearCache()`, and for the largest models on 8 GB
>   proactively `unload()` (warm-reload on foreground with a toast). Define this β€” it's the line between
>   "resumes where I left off" and "app was killed".
> - **Model swap must serialize** cancel β†’ *await GPU drain* β†’ `clearCache()` β†’ load, behind a
>   `switching` state, or two weight sets briefly co-reside (10 GB) or buffers free under a live kernel.

### 2.4 Persistence β€” files, not SwiftData

Both stores build on `DurableStore` β€” a recovery skeleton with a versioned Codable manifest, atomic
writes, **corrupt-manifest β†’ backup-not-wipe recovery**, and bad-record skipping. Losing a long chat
is a serious failure β€” keep that defensive posture.

- **ConversationStore** (actor): one `conversation-<uuid>.json` per thread + a light `index.json`
  (id, title, updatedAt, model, count) for fast list rendering. `Message { role, answer, reasoning?,
  stats?, parentID? (branching) }`. Atomic per-file writes. Single writer (critique G1).
- **ModelRegistryStore** (actor): `registry.json` β€” per variant install state; source of truth for the
  model manager's tri-state + offline load; rebuildable by re-scanning the weights dir.
- Weights live under a **no-backup-flagged** dir (multi-GB shards must not hit iCloud backup).

### 2.5 Memory / thermal governance & errors

`LLMMemoryGovernor.plan(model, variant, device, context) -> LLMFit` β€” resident-only, no streaming rung.
`peak = onDiskBytes + ~0.5 GB + KV(ctx)`; `peak ≀ 0.70Β·ceiling` β†’ green; `≀ ceiling` β†’ amber+maxContext;
`weights alone > ceiling` β†’ gray/unsupported. These are advisory planning labels only: neither an
`unsupported` estimate nor live free-memory pressure blocks activation. Every installed model is attempted
and the engine/device supplies the real result. Ship `increased-memory-limit` (helps 12 GB, never claimed
for 8 GB). **Thermal**: the `ThermalGovernor`
calls `throttleIfNeeded()` on a wall-clock boundary (~every 250 ms of decode, not only per-N-tokens);
`.critical` β†’ bounded pause then recoverable `pausedForHeat` (the one hard anti-shutdown guarantee).

One `LLMError` enum surfaced via `.toastBanner` + inline, always with a forward action:
`pausedForHeat` (auto-resumes) Β· `weightsCorrupt` (re-download, resumes) Β· `forkKernelMissing` (CI canary)
Β· `cancelled` (commit partial, not an error).

---

## 3. Model catalog (extensible)

```swift
struct LLMModel { id; displayName; family; publisher; summary; license; architecture; variants; default }
struct LLMArchitecture { modelType; swiftModelClass; hidden; layers; vocab; tieWordEmbeddings;
                         attention: AttentionShape; nativeContext; thinkingCapable; eos; chatTemplate }
enum AttentionShape { case fullAttention(kvHeads,headDim,layers)               // dense qwen3
                      case hybridLinear(fullLayers,kvHeads,headDim,recurrent) } // qwen3_5 GDN
struct LLMVariant { quant: QuantSpec; backend: Backend; onDiskBytes; source }
enum Backend { case mlxFork      // 1-bit, PrismML fork
               case mlxStock     // ternary-2bit, upstream bits∈{2,…}
               case awqUnsupported }  // excluded
```

Seed: `bonsai-27b` (qwen3_5, 1-bit + ternary), `bonsai-8b` / `bonsai-4b` / `bonsai-1.7b` (qwen3,
1-bit + ternary). Adding Qwen3/Llama/Mistral later = append an `LLMModel` with the right
`modelType`/`swiftModelClass` (LLMModelFactory keys) β€” no schema change.

> **Critique E1 (scope honestly):** the **download + UI layer is family-agnostic**, but the **governor's
> memory model is qwen-shaped**. It can't yet express sliding-window attention (Mistral/Gemma β€” bounded
> KV) or MoE (all experts resident dominates memory). Ship the catalog as **"qwen-family v1"**; add
> `slidingWindow(...)` / `moe(residentExpertBytes:...)` cases when a non-qwen model is actually added,
> so fit-estimates never silently lie.

---

## 4. Product & interaction design (commercial-grade)

North star: **on-device is the hero** β€” every apparent limitation (offline, thermal, OOM, download)
is truthfully reframed as privacy / control / protection. **Never dead-end** β€” every error carries a
forward action; irreversible actions get **undo**, not just confirmation.

**Information architecture.** iPhone: 3 tabs β€” **Chat** (NavigationStack: list β†’ thread), **Models**,
**Settings**; active model is a header subtitle tap-target β†’ quick switcher sheet. Mac:
`NavigationSplitView` (sidebar · conversation list · thread), full menu-bar map (⌘N new, ⌘. stop,
⌘R regenerate, ⌘L switch model, βŒ˜β‡§T thinking, ⌘F find), and a v1.1 `MenuBarExtra` βŒ₯Space quick-ask.

**The chat surface.** User = right-aligned accent bubble; assistant = full-width document text (no
bubble) so markdown/code read cleanly. Composer: multiline auto-grow; **one morphing Send↔Stop button**
(no reflow); inline **🧠 Thinking** toggle; **live context meter** `1,240 / 8K` (amber @85%, red @98%).
Send β†’ user bubble springs up β†’ **warming shimmer** (owns prefill latency honestly) β†’ tokens stream
with a blinking caret, **incremental markdown**, sticky-bottom autoscroll + a "βŒ„ N new" pill when
scrolled away.

**The signature interaction β€” thinking disclosure.** While `<think>` streams, a disclosure auto-expands
("Thinking…", dimmed backstage text); when the first answer token arrives it **auto-collapses** with a
spring, its label morphing to "Thought for 4.2s", spotlighting the answer. Tap to re-read. Setting:
auto-collapse (default) / always-expand / hidden.

> **Critique F1 (correctness):** `ThinkSplitter` must have a **`finish()`** that flushes the residual
> buffer at stream end (it withholds up to `tag.countβˆ’1` trailing chars as a possible partial tag β€”
> without a flush the final chars are lost). Validate `<think>` boundary detection against real Bonsai
> output (special-token vs literal string) on-device before trusting substring matching.

**Rich content.** GFM markdown (scrollable tables), fenced code cards (language label + Copy, syntax
highlight, horizontal scroll β€” never wrap code), inline-code chips, LaTeX (`$…$`/`$$…$$` with raw-TeX
fallback). Message actions (hover Mac / long-press iPhone): Copy Β· Regenerate Β· Read-aloud Β· **Fork
from here** Β· rate (local only). **Edit-and-resend** truncates + re-generates with a `β€Ή1/2β€Ί` branch
pager (v1.0). **Quiet stats:** per-message footer `Bonsai 8B Β· 41 tok Β· 23 tok/s Β· stop: eos`; opt-in
`βŒ˜β‡§I` HUD (tok/s Β· peak vs ceiling Β· thermal dot).

**Conversation management.** Recency-grouped list (Pinned/Today/…); auto-title (first user line now,
model-summarized later as a *preemptible idle* job β€” critique F4); full-text search; new/rename/pin;
**soft-delete + Undo toast** (safety rule against irreversible loss); export MD/JSON; fork.

**Model management.** Catalog cards with the **device-recommended default pinned first**, a
**Segmented** 1-bit ↔ ternary selector that live-updates the fit badge (`LLMFitBadge` bound to `LLMFit`)
+ size, and a **bytes/speed/ETA download meter** with **pause/resume**. One active model
(bandwidth-bound = one resident); switching is first-class (unload→load warm toast).

> **Critique D2 (honest download copy):** the downloader is a `URLSessionDataTask` streaming into
> `.part` β€” iOS **background** URLSession supports only download/upload tasks, **not** data tasks. So
> **downloading continues only in the foreground** (resume-on-relaunch works). Copy: *"Keep mobileLLM
> open while downloading β€” it resumes automatically if interrupted."* A true background
> `URLSessionDownloadTask` path (loses mid-file resume) is a later option, not a v1 claim.

**Onboarding (3 calm screens):** promise (private, on-device, no account) β†’ pick first model
(device-recommended pre-selected, honest size, resumable download) β†’ "You're set" with example prompts.
First-run also carries a one-line content note (critique H1: on-device β‰  no liability).

**States, a11y, i18n.** Every state has warm, action-oriented copy (empty / no-model / warming /
streaming / stopped / load-error / thermal-pause / context-full / offline / model-deleted).
VoiceOver labels on all controls (fit-badge text *is* its label); Dynamic Type to XXXL (code scrolls,
never clips); all copy in a `.xcstrings` catalog from day one; motion via `Motion` (Reduce-Motion β†’ 
crossfades, steady caret).

---

## 5. Foundation layer (MLX-free)

**v1 = keep the ~6 MLX-free foundation files inline in the app. Do NOT extract a standalone shared
package yet.**

> **Critique B3:** extracting an `AppFoundation` package now would force designing a stable public API
> before it has more than one real consumer β€” premature abstraction. The files are small,
> self-contained, and already MLX-free β€” keep them inline, ship, and defer the shared-package extraction
> to a deliberate later pass, once the API has settled.

| Component | MLX-coupled | v1 action |
|---|---|---|
| Design system: Theme / Motion / Chip / Segmented / studioCard / toastBanner | No | **inline** the tokens + shared controls; add the app's own controls per surface |
| `DownloadMeter` + `formatETA` | No | **inline** verbatim |
| Resumable streaming downloader | No (Foundation + CryptoKit + Hub) | **inline**; parameterize the component-set check (LLM repos ship a flat file list); give it an LLM-specific on-disk manifest name |
| `MemoryProbe` (`phys_footprint`) | No | **inline** verbatim (jetsam-accurate) |
| `DeviceTier` | No | **inline** (LLM-relevant fields only) |
| `ThermalGovernor` | Barely (`MLX.Memory.clearCache()`) | **inline with `clearCache` injected** as `@Sendable ()->Void` β†’ MLX-free |
| `DurableStore` recovery skeleton | No | **adapt** β†’ `ConversationStore` / `ModelRegistryStore` |
| `ModelsView` / `SettingsView` view builders | No | **adapt** (retarget to `LLMModel`, `FitBadge` β†’ `LLMFitBadge`) |

The two hardest subsystems β€” the resumable multi-GB downloader and the thermal governor β€” are
effectively MLX-free already: that is the big head start.

---

## 6. Roadmap

**MVP β€” prove the hard tech + one clean loop**
1. **Fork pin + `bits=1` smoke-decode gate FIRST** (Bonsai-8B β†’ non-garbage). This is the schedule risk;
   settle it before any UI.
2. Stand up the MLX-free foundation files inline; bring up `LLMCore` + `LLMEngine`.
3. **8B-1bit + 27B-1bit, iPhone + Mac.** 8B-1bit is the recommended green path everywhere;
   **27B-1bit ships in the MVP too** (user decision β€” they want to personally test whether it runs on the
   8 GB 16 Pro). 27B is green on Mac/12 GB and carries an honest advisory warning on the 8 GB phone, but
   the same **Use** action still attempts it.
4. One conversation surface: send β†’ stream β†’ thinking disclosure (with `finish()`) β†’ honest
   boundary-Stop β†’ persist (`DurableStore` + corrupt-recovery).
5. Model manager: foreground download (honest copy), fit badge (green/gray), delete.
6. Advisory fit planning + `.critical` thermal pause + the backgrounding policy (Β§2.3).

**v1.0**
- **27B-1bit on Mac + 12 GB iPhone** (after 8B validates); ternary-2bit stock variants.
- Edit-and-resend **branch pager**, fork-from-here, regenerate-with-different-model.
- Full stats HUD, LaTeX, syntax highlighting, full-text search, `.xcstrings`.

**v1.1+**
- **Second inference engine (llama.cpp)** behind the same protocol β€” mmap'd weights to fit large models
  on memory-tight phones (the 1-bit `Q1_0` format is already upstream in llama.cpp).
- `AppFoundation` shared-package extraction, once the foundation API has settled (a separately-tested refactor).
- `MenuBarExtra` quick-ask; background `URLSessionDownloadTask` path; vision (mmproj) when MLX supports it.

**Device testing is LAST** (per the user's instruction; the phone isn't connected to the Mac yet).
Simulator has no Metal path for the 1-bit kernels β†’ inference is validated on real devices only.

---

## 7. Risks

| Risk | Impact | De-risk |
|---|---|---|
| **Fork ↔ mlx-swift-lm version/API rebase** | High (schedule) | Compile-gate + `bits=1` smoke-decode **before UI**; vendor submodules with recurse; may need to edit version constraint / rebase kernels β€” budgeted. |
| **8 GB iPhone Γ— 27B** | Crash | 8B-1bit is recommended; 27B carries an advisory warning but remains attemptable, and launch never reloads it automatically. |
| **Backgrounding a 5 GB resident model** | Jetsam kill | Explicit background policy: commit partial, lower cache, unload the largest models. |
| **Metal builds via Xcode only** | CI/dev friction | `xcodebuild` everywhere; copied MLX-free code keeps a fast `swift test` loop. |
| **Supply chain (unmerged #3161)** | Maintenance | Pin exact SHAs; **exit path:** when #3161 merges + ships in a tagged mlx-swift, repoint the fork back to upstream and delete it β€” the on-disk affine-quant format survives the merge (gate on a byte-identical logits check first), no re-download. |

---

## 8. Decisions (locked 2026-07-15)
1. **Accent** β€” mobileLLM's own **teal** intelligence hue (light `#0D9488` / dark `#2DD4BF`).
2. **MVP models** β€” **8B-1bit + 27B-1bit**, iPhone + Mac. 27B ships so the user can personally test the
   8 GB 16 Pro (Experimental Β· Try-anyway); 8B-1bit is the safe default.
3. **Name** β€” `mobileLLM` (bundle id TBD at Xcode-project creation, e.g. `wang.mobilellm`).
4. **Repo** β€” **private** GitHub repo under the user's account.

Device testing is LAST (phone not yet connected to the Mac).