git:20260702.df864d8 to git:20260728.dcb3761

---

name: cosmos3-codebase-nav

description: >

Navigate the Cosmos3 package codebase to find where parameters, configs, defaults,

scripts, and documentation live. Use when the user asks "where is X in cosmos3",

"how do I find the config for Y", "where are the defaults", "where do I change a

parameter", or any question about locating files, modules, or settings. Also use

when the user opens or edits files and needs orientation.

---

# Cosmos3 Codebase Navigation

## When to use this skill

- Use this skill when an agent is navigating the Cosmos3 package

- Use this skill to answer "where is X", "how do I find the config for Y", or any file-location question

- Use this skill when the user opens or edits cosmos3 files and needs orientation

## Path convention

All paths below are relative to this file's location (`.agents/skills/cosmos3-codebase-nav/`). The repo is laid out as:

- `cosmos_framework/` — main training package (data, model, trainer, callbacks, checkpoint, utils, …).

- `cosmos_framework/configs/base/experiment/` — vfm (generator) experiment SKUs referenced by `[train.train_policy].experiment` in the recipe TOMLs.

- `cosmos_framework/configs/base/reasoner/experiment/` — vlm (reasoner) experiment SKUs.

- `cosmos_framework/inference/` — inference subpackage (args, model, inference engine, defaults, Ray serving, common helpers).

- `cosmos_framework/scripts/` — top-level entry-point scripts (train, inference, eval, export_model, convert_model_to_dcp, upsample_prompts, caption_from_video, captions_to_sft_jsonl, action_policy_server, …). Invoked as `python -m cosmos_framework.scripts.<name>`.

- `examples/toml/sft_config/<recipe>.toml` + `examples/launch_sft_<recipe>.sh` — paired SFT recipes (training entry-point input). The shell sources `examples/_sft_launcher_common.sh`, which forwards into `cosmos_framework.scripts.train --sft-toml=...`.

- `cosmos_framework/configs/toml_config/` — pydantic schemas (`sft_config.py`) and helpers that validate the recipe TOML at load time.

## Quick Reference

### Where parameters and defaults live

| What you're looking for | File |

| --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |

| Sampling params (num_steps, guidance, shift, fps, etc.) | `../../../cosmos_framework/inference/args.py` → `SamplingArgs`, `SamplingOverrides` |

| Per-modality default values | `../../../cosmos_framework/inference/defaults/<mode>/sample_args.json` |

| Setup params (parallelism, checkpoints, model path) | `../../../cosmos_framework/inference/args.py` → `OmniSetupArgs`, `OmniSetupOverrides` |

| Common args base classes | `../../../cosmos_framework/inference/common/args.py` → `ArgsBase`, `OverridesBase` |

| Ray serving parallelism presets | `../../../cosmos_framework/inference/ray/configs/latency.yaml`, `../../../cosmos_framework/inference/ray/configs/throughput.yaml` |

| Feature flags | `../../../cosmos_framework/utils/flags.py` |

| Prompt upsampler system prompt | `../../../cosmos_framework/inference/defaults/prompt_upsampler.txt` |

| Video captioner system prompt | `../../../cosmos_framework/inference/defaults/video_captioner.txt` |

| SFT recipe TOMLs (paired with `examples/launch_sft_*.sh`) | `../../../examples/toml/sft_config/<recipe>.toml` |

| SFT pydantic schema (validates the recipe TOML) | `../../../cosmos_framework/configs/toml_config/sft_config.py` |

| Training experiment SKUs (vfm) | `../../../cosmos_framework/configs/base/experiment/` |

| Training experiment SKUs (vlm / reasoner) | `../../../cosmos_framework/configs/base/reasoner/experiment/` |

| Example inputs | `../../../inputs/omni/t2i.json`, `../../../inputs/omni/t2v.json`, `../../../inputs/omni/i2v.json`, … |

### Config defaults resolution chain

When a user runs inference, default parameter values are resolved in this order:

```

cosmos_framework/inference/defaults/<mode>/sample_args.json # 1. Per-modality JSON defaults (num_steps, guidance, shift, fps, etc.)

↓

_load_modality_defaults() in cosmos_framework/inference/args.py # 2. Loaded and cached at import time

↓

SamplingArgs / SamplingOverrides # 3. Pydantic models with field-level validation

↓

OmniSampleOverrides.build_sample() # 4. Merges user overrides → final resolved args

↓

_RESOLUTION_SHIFT_DEFAULTS[model_size, resolution] # 5. Model+resolution shift override (if user didn't set shift)

↓

CLI flags (--guidance, --shift, etc.) # 6. User overrides from command line

```

The `_RESOLUTION_SHIFT_DEFAULTS` table in `../../../cosmos_framework/inference/args.py` (on `OmniSampleOverrides`) overrides the default `shift` based on model size and resolution, unless the user explicitly specified `--shift`.

| Mode | Default file | Key defaults |

| ------------- | --------------------------------------------------------------------------- | ---------------------------------------------- |

| `text2image` | `../../../cosmos_framework/inference/defaults/text2image/sample_args.json` | `num_frames=1`, `guidance=6.0`, `shift=10.0` |

| `text2video` | `../../../cosmos_framework/inference/defaults/text2video/sample_args.json` | `num_frames=189`, `guidance=6.0`, `shift=10.0` |

| `image2video` | `../../../cosmos_framework/inference/defaults/image2video/sample_args.json` | `num_frames=189`, `guidance=6.0`, `shift=10.0` |

Action and video2video modes also have defaults under `cosmos_framework/inference/defaults/{image2image,video2video,forward_dynamics,inverse_dynamics,policy}/sample_args.json`.

Users can also supply a custom defaults file per-request via the `defaults_file` field in sample arguments (see `../../../docs/inference.md`).

### Where to make changes

| Task | Edit |

| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |

| Change a built-in default value | `../../../cosmos_framework/inference/defaults/<mode>/sample_args.json` |

| Add a new CLI parameter | `SamplingArgs` + `SamplingOverrides` in `../../../cosmos_framework/inference/args.py`, then add to each `sample_args.json` |

| Change parallelism presets | `../../../cosmos_framework/inference/ray/configs/latency.yaml` or `throughput.yaml` |

| Add a new script | `../../../cosmos_framework/scripts/` — follow `inference.py` as the pattern |

### Key entry points

| Entry point | How to run |

| ---------------------------------- | -------------------------------------------------------------------------------------------- |

| Batch inference | `python -m cosmos_framework.scripts.inference` |

| Training | `python -m cosmos_framework.scripts.train --sft-toml=examples/toml/sft_config/<recipe>.toml` |

| Online serving (Ray) | `python -m cosmos_framework.inference.ray.serve` |

| Submit to Ray server | `python -m cosmos_framework.inference.ray.submit` |

| Gradio UI | `python -m cosmos_framework.inference.ray.gradio` |

| Prompt upsampling | `python -m cosmos_framework.scripts.upsample_prompts` |

| Model export (HF) | `python -m cosmos_framework.scripts.export_model` |

| DCP conversion | `python -m cosmos_framework.scripts.convert_model_to_dcp` |

| Diffusers conversion | `python -m cosmos_framework.scripts.convert_model_to_diffusers` |

| Video captioning | `python -m cosmos_framework.scripts.caption_from_video` |

| Captions → SFT JSONL | `python -m cosmos_framework.scripts.captions_to_sft_jsonl` |

| Action policy server (LIBERO HTTP) | `python -m cosmos_framework.scripts.action_policy_server_libero` |

| Action policy server (RoboLab WS) | `python -m cosmos_framework.scripts.action_policy_server_robolab` |

### Documentation

| Doc | Covers |

| ----------------------------------- | ---------------------------------------------------------- |

| `../../../AGENTS.md` | Commands, rules, key file locations (read this first) |

| `../../../README.md` | Overview, quickstart, examples |

| `../../../docs/setup.md` | Installation, environment, checkpoints |

| `../../../docs/code_structure.md` | Repo layout and per-subpackage tour of `cosmos_framework/` |

| `../../../docs/inference.md` | Sample args, default values, custom defaults |

| `../../../docs/training.md` | SFT / post-training workflow |

| `../../../docs/faq.md` | FAQ, tips, and troubleshooting |