unsloth · git:20261012.dcfba03 · 2026-10-12 · sha256 a533ae02f7fa5015
unsloth git:20261012.dcfba03A
Immutable. This exact content never changes and is served at /api/v1/blob/a533ae02f7fa5015.
--- name: unsloth description: >- Install, operate, and troubleshoot Unsloth Core, Studio, and Desktop for local inference and model training. Use for Unsloth environment setup, LoRA/QLoRA supervised fine-tuning, chat templates, GRPO and preference training, vision and audio workflows, checkpoint recovery, multi-GPU training, and model export to GGUF or serving engines. Do not use for framework-neutral ML strategy or operating a standalone inference server unrelated to Unsloth. license: MIT metadata: research-date: "2026-10-12" source: "https://github.com/unslothai/unsloth" --- # Unsloth Turn an Unsloth request into a reproducible installation, bounded experiment, or verified model artifact. Unsloth provides an optimized Python training stack (Core), a browser interface (Studio), and a native application (Desktop). Capabilities differ by product, backend, model, and release. Read the relevant official page before selecting a new hardware or model path; never promote a vendor speed or minimum-memory claim into a promise about the user's workload. ## When not to use - For deciding whether ML, retrieval, or fine-tuning is appropriate, use `ml-engineering`; return here for Unsloth execution. - For operating exported models in a standalone engine, use `llama-cpp` or `vllm`. This skill owns the export and parity checks at that boundary. - For container platform design use `platform-engineering`; for Docker Compose mechanics use `docker-compose`. Do not turn this skill into a platform manual. ## Start here 1. Identify the requested surface: Core Python/notebook, Studio web UI, or Desktop. Determine inference, SFT, preference optimization, online RL, or another model-specific training task. Default to Core for reproducible coded training, and Desktop/Studio when a GUI is requested. 2. Discover OS, Python, accelerator/backend, free VRAM/RAM/disk, installed versions, model ID and revision, input format, target context, and output destination. Use existing task context; ask only for unresolved choices that affect feasibility. Run the read-only helper before prescribing packages. 3. Establish a measurable baseline, held-out dataset, budget, and stop condition before training. Capture these in [the run record](templates/run-record.md). 4. This skill can install software, download weights, consume GPU resources, write checkpoints, expose services, and publish models. Before first mutation: > Confirm the target, scope, and rollback path before acting. Read-only discovery may proceed without confirmation. Existing explicit authorization can establish these facts; do not ask again when the session already supplies them. Keep a previous environment/artifact available; use a new run directory. Publishing and opening remote access need their own authorized destination/scope. Deletion needs an explicit directive. ## Load only the relevant guide | Situation | Read | |---|---| | Explain Unsloth, select a surface, assess fit | [Capabilities and decisions](references/capabilities.md) | | Install, upgrade, select hardware/backend, use containers | [Installation](references/installation.md) | | Run the observed RTX 5070 Ti text-SFT smoke recipe | [Validated executable SFT](references/validated-sft.md) | | Prepare data, run text SFT, choose LoRA settings | [Data and supervised training](references/sft.md) | | GRPO, DPO, vision/audio, continued pretraining, multi-GPU | [Advanced training](references/advanced-training.md) | | Save/resume, export, run inference, use Studio API | [Artifacts and serving](references/artifacts-serving.md) | | Import errors, OOM, zero loss, bad exports, hangs | [Troubleshooting](references/troubleshooting.md) | | Check currency, resolve conflicting docs, find a model notebook | [Source ledger](references/sources.md) | ## Bounded workflow 1. **Prove the environment.** Inventory first, then select the official backend recipe. Use an isolated environment. Restart notebook kernels after package changes. Import Unsloth before Transformers/PEFT/TRL in training code. Check the accelerator in the same Python process/environment that will train. 2. **Prove the data.** Audit structure and exact duplicates with the helper; separately check source/group leakage, permission, quality, and tokenizer lengths. Render real examples using the selected tokenizer's chat template. Inspect a collated batch, including decoded non-ignored labels. A syntactically valid conversation is not proof of correct loss masking. 3. **Prove a small run.** Start with a supported model-specific notebook/script, pin its revision and dependencies, and adapt one dimension at a time. Execute a short run with finite loss and a saved/reloaded artifact before committing the full budget. Record peak memory and tokens/sec after warmup. 4. **Train with checkpoints.** Track training and validation signals separately. End at the budget, nonfinite loss, repeated infrastructure failure, or the agreed regression threshold. Do not tune on the final test set. 5. **Compare behavior.** Run the base model and candidate on identical held-out prompts with equivalent decoding, task scoring, and critical failure checks. Training loss alone does not demonstrate improvement. Keep negative results. 6. **Verify delivery.** Save tokenizer/processor and adapter or merged weights; test reload in a fresh process. Compare target-runtime behavior after export. Report exact artifacts, versions, checks actually run, missing evidence, and how to resume or roll back. ## Validated GPU workflow For an executable text-SFT baseline, read [the observed recipe](references/validated-sft.md). It pins the public Qwen3-0.6B revision and resolved dependencies and uses `scripts/sft_smoke.py`, `scripts/sft_audit.py`, `scripts/verify_export.py`, and `scripts/gpu_completion.cpp` with `assets/smoke-config.json`, `assets/train.jsonl`, `assets/heldout.jsonl`, and `assets/requirements-gpuslut01.lock`. The scripts implement target-retention and actual collated-label audits, fixed-prompt base/adapter scoring, full checkpoints, fresh-process reload, stateful resume, merge, and GPU F16 GGUF parity. Read the recipe before execution; its package installation and GPU work mutate only the authorized isolated directory. No automatic CPU fallback, upload or service start. This observed path is BF16 LoRA on one RTX 5070 Ti. Other installation/backend, QLoRA, UI, advanced-training and quantized-export guidance remains unverified by this run. Output-quality eval manifests are review assets, not executed semantic passes. Do not generalize the toy formatting-task score to real workloads. ## Local helpers Run these commands from this skill's root with Python 3.10+. They require no Unsloth installation, GPU, network, or credentials and do not modify inputs. ```bash python3 scripts/unsloth_preflight.py doctor --json python3 scripts/unsloth_preflight.py dataset train.jsonl --eval-file valid.jsonl --json python3 -m unittest discover -s scripts -p 'test_*.py' ``` `doctor` inventories installed package metadata and available executables; it does **not** certify compatibility or execute package imports. Dataset auditing supports only text SFT JSONL (`text` or `messages`, one consistent format per file pair). It rejects unsupported multimodal/tool/preference shapes explicitly. It checks exact content duplicates and cross-split overlap, ignores row metadata when comparing content, and emits counts/line numbers without training text. Exit 0 means the selected checks completed without blocking findings, 1 means findings or incomplete scanning, 2 means invalid CLI arguments. No tokenization, semantic deduplication, quality assessment, or GPU execution is implied. ## Non-obvious failure boundaries - The driver's advertised CUDA capability, Torch's compiled runtime, and an installed CUDA toolkit are different facts. Do not select a wheel by copying the `nvidia-smi` version string blindly. - QLoRA trains adapters on a quantized base; it does not make all training state four-bit. GGUF inference quantization is a separate artifact/workflow. - Never use a Llama response delimiter for another model without inspecting rendered tokens. Zero supervised tokens can masquerade as a fast, low-loss run. - A LoRA export is not a standalone model or a full optimizer checkpoint. - Multiple GPUs do not automatically pool memory under DDP; model replication and sharding solve different problems. - A working NVIDIA Core recipe is not a Mac/AMD/Intel recipe. Current Mac UI/MLX support does not prove the same Core CUDA script runs on Apple Silicon. - Documentation and installed TRL can differ on `processing_class`/`tokenizer` and `max_length`/`max_seq_length`; inspect the installed signature and matched notebook instead of combining snippets from different versions. ## Completion and escalation For research: deliver sourced decisions and explicit unknowns. For installation: prove the requested surface starts and the selected backend runs a tiny operation. For training: deliver the run record, held-out comparison, reloadable artifact, and recovery instructions. For export: additionally prove the intended runtime loads it and passes the agreed parity checks. If hardware or access is absent, deliver the prepared commands and mark execution unverified. After three diagnostic passes without new evidence, stop and provide the minimal reproducer, version inventory, observed failing layer, attempted hypotheses, and the specific missing prerequisite. Do not loop through global reinstalls.