---
name: spark-model-scheduler
version: "0.3.0"
description: Plan and check unified-memory use on a single DGX Spark (GB10, one memory pool shared by CPU and GPU) before starting or switching inference services - LLM, vision model, ComfyUI image or video generation. Use before bringing the stack up, before a video generation stage, when a model load fails or the machine starts swapping, or when the user asks whether a model "fits" next to the others.
license: BUSL-1.1
metadata:
  author: 贺家豪 (Jiahao He)
  category: production
  tags:
    - dgx-spark
    - unified-memory
    - vllm
    - scheduling
---

# Spark model scheduler

DGX Spark has one ~121 GB pool for the OS, every LLM's weights and KV cache, and every
diffusion model. A service that loads fine alone can kill another one later. Decide the
budget first, then start things.

## Workflow

1. Measure before acting:

   ```bash
   cinefoundry mem status
   ```

   It reports total and available unified memory, per-container memory, and ComfyUI's
   loaded-model state. `nvidia-smi` shows no total on GB10; trust `/proc/meminfo`.
   Per-container numbers cover host allocations only: CUDA memory is not charged to the
   container's cgroup, so a vLLM or SGLang container holding 100 GB can show under 5 GB.
   Use the drop in available memory, not `docker stats`, to size a GPU service.

   **Available is not free.** `mem status` reports `MemAvailable`, which counts the page cache as reclaimable. On DGX Spark CUDA can only allocate memory that is truly free (`MemFree`): it does not make the kernel drop cached pages for it. Every model file read from NVMe - a 22B video transformer, its text encoder, the image models, a vLLM reload after sleep - stays in the page cache, so a machine that shows 60 GB available can have far less free, and a GPU load fails or a sampler runs short. Before every GPU phase, and always before the video phase, compare and evict:

   ```bash
   grep -E '^(MemFree|MemAvailable|Cached):' /proc/meminfo
   sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'
   ```

   Plan against `MemFree` after the eviction, not against `MemAvailable` before it.
2. Check the plan against the budget:

   ```bash
   cinefoundry mem plan --need nemotron=30 --need stepvl=20 --need comfyui=45
   ```

   Exit code 1 means the plan does not fit with the reserve (default 10 GB). Reduce
   `--gpu-memory-utilization` / `--max-model-len` on the vLLM services, or stage the work.
3. Keep one diffusion family resident at a time. With the image, edit and video models all left in ComfyUI next to the two LLMs, the GPU driver started failing allocations (`NV_ERR_NO_MEMORY` in the kernel log) and a 25-minute job died on a request timeout. Release at every phase boundary: hero still -> frames -> clips -> interpolation -> voice -> music. The harness frees ComfyUI and records available memory at each boundary (`enterPhase`); the page-cache eviction above is the operator's step.
4. Stage work by phase when it does not fit at once. The Cinefoundry pipeline is naturally
   phased: text planning -> images -> video -> composition. Diffusion memory is only
   needed in the middle two phases, and the vision model only for breakdown and QA.
   The video phase is the longest: the default 30-second master is six 5-second clips, each
   generated by LTX-2.5 Fast NVFP4 at native 1080p and then interpolated by RIFE, one at a time
   with the model kept loaded between clips and released after the last.
5. Release diffusion memory at each boundary:

   ```bash
   cinefoundry mem free
   ```
6. Do not run a second GPU workload (an eval, another film) next to a film job; they share one pool and one GPU.
7. Record `cinefoundry mem status` before and after each phase into `eval/` - these numbers are
   the platform evidence for the submission. Report measured values only.

## Reference budget (measure and replace)

| Service | Planned | Notes |
|---|---:|---|
| Nemotron 3.5 Lightning 30B-A3B NVFP4 | 35 GB | measured on GB10 at 0.25 of the pool, 64K context |
| Step3-VL-10B FP8 | 24 GB | measured: 14.25 GiB weights; 0.20 of the pool gives a 47,616-token KV cache |
| ComfyUI, Qwen-Image fp8 | 32 GB | 20.4 GB model + 9.4 GB text encoder |
| ComfyUI, Qwen-Image-Edit-2509 | 32 GB | same size as Qwen-Image; a different family, so release one before loading the other |
| ComfyUI, Wan2.2 I2V A14B fp8 | 30 GB | two 14.3 GB experts loaded in turn + 6.7 GB text encoder + activations |
| ComfyUI, Wan2.2 TI2V-5B | 20 GB | 10 GB model + 6.7 GB text encoder + activations |
| ComfyUI, LTX-2.5 Fast 22B NVFP4, native 1080p | about 45 GB | measured 2026-10-17: lowest free memory 8.4 GB with both LLMs resident; 1088x1920, 121 frames, about 4 min per 5 s clip |
| ComfyUI, RIFE 4.26 interpolation (x5 to 120 fps) | about 20 GB | 601 float frames at 1088x1920 (about 15 GB) plus the encoder; run after the video model is freed |
| Qwen3-TTS voice container | about 8 GB | runs in its own container; free ComfyUI first (a film died here with CUDA out of memory before this was enforced) |
| ComfyUI, ACE-Step music | 9 GB | 7.7 GB checkpoint |
| Nemotron 3.5 Super VL 120B-A12B NVFP4 (planned) | 75 GB weights, about 91 GB with KV cache | vLLM recipe minimum for one Spark; cannot stay resident next to Step3-VL and diffusion - must sleep (level 2) outside the text stages. Measure before relying on it |
| OS, Cinefoundry, reserve | 15 GB | |

vLLM's `--gpu-memory-utilization` is a fraction of the whole pool and is claimed at
startup, so start LLM services first and let ComfyUI allocate on demand.

Phase plan the harness follows (sleep level 2 discards a service's weights and reloads them from NVMe on wake):

| Phase | Awake | Asleep |
|---|---|---|
| text (brief, script, compliance, storyboard) | planner, reviewer | - |
| hero still and frames | vision gate | planner |
| clips and interpolation | - | planner, vision gate |
| cut (voice, music, assembly) | woken in the background | - |

Step-3.7-Flash (198B) needs ~116 GB even at IQ4_XS: it cannot share the machine with
diffusion. Use it through the StepFun API, or run it alone in a dedicated planning phase.