mujoco v1.0.2A

Immutable. This exact content is served forever at /api/v1/blob/f9b1fca694d158f9.

---
name: mujoco
version: 1.0.2
description: >
  MuJoCo for lightweight, contact-rich robot manipulation simulation on
  macOS/Linux, especially single-arm grasping without ROS: headless offscreen
  rendering, MJCF models, mujoco_menagerie assets, damped-least-squares inverse
  kinematics, and empirical grasp calibration. Use when: 'MuJoCo', 'MJCF',
  'mjpython', 'MUJOCO_GL', 'menagerie', 'SO-101' / SO-ARM100 arm, 'offscreen
  render', 'inverse kinematics' / 'IK', 'grasp' / 'pick and place' in sim,
  hand-building a manipulation env, or headless mujoco.Renderer determinism.
  Load after architect or simulation route a non-ROS single-arm manipulation
  task here (Gazebo and Isaac Sim are the ROS / GPU-photoreal alternatives).
  Defers to MuJoCo's own docs and mujoco_menagerie for the physics/model API;
  embeds the macOS-arm64 platform and grasp/IK gotchas hard-won from a real
  SO-101 build. Not for: ROS-integrated sim (gazebo), GPU-parallel RL or
  photoreal synthetic data (isaac-sim / isaac-lab), or LeRobot policy
  training/eval mechanics (lerobot).
---

# mujoco

MuJoCo 3.x for robium's lightweight, non-ROS manipulation side: a fast
contact-rich physics engine with an offscreen renderer, well suited to
hand-building a single-arm grasping/pick-and-place environment when the full
ROS + Gazebo stack (or a GPU-gated Isaac stack) is more than the task needs.
This skill stays deliberately thin on the MuJoCo API itself: the model format
(MJCF), the Python bindings, and the robot assets in mujoco_menagerie are all
documented upstream and move release to release. What it embeds instead is the
set of hard-won gotchas from the vla-trial SO-101 build (macOS arm64, MuJoCo
3.x, observed 2026-07-13/14): the macOS headless-render backend, version-pin
traps, offscreen-render determinism, and the IK / grasp-calibration failure
modes that cost the most time and give no error message when they bite.

## When to use this skill

- Simulating a low-DOF arm (grasping, pick-and-place, contact-rich
  manipulation) where ROS is not required and a GPU-photoreal simulator is
  overkill; MuJoCo runs natively and fast on Apple Silicon.
- Rendering camera observations headless for a policy/dataset pipeline, and
  needing that rendering to be deterministic/reproducible.
- Writing an inverse-kinematics reach + grasp routine by hand (damped
  least-squares), and debugging why the arm won't reach or knocks the object
  over.
- Loading a robot from mujoco_menagerie (e.g. the SO-101 / SO-ARM100 arm) and
  adapting its scene into a manipulation task.
- The trigger phrases in the description: 'MuJoCo', 'MJCF', 'mjpython',
  'MUJOCO_GL', 'menagerie', 'SO-101', 'offscreen render', 'IK', 'grasp'.
- Cross-references: go to the sibling skill instead when the question is:
  - A ROS 2-integrated simulation (ros_gz bridge, SDF worlds) → the `gazebo`
    skill.
  - Photoreal rendering, synthetic data at scale, or GPU-parallel RL → the
    `isaac-sim` / `isaac-lab` skills (GPU-gated).
  - Which simulator to pick at all → the `simulation` umbrella (routes here for
    lightweight non-ROS manipulation).
  - LeRobot dataset/policy training/eval mechanics → the `lerobot` skill;
    MuJoCo here is only the sim that produces observations/actions.
  - The whole-stack decision this feeds into → the `architect` skill.

## Key directives

- **Delegation posture: embed + links.** The robium-specific,
  hard-won gotchas (macOS render backend, version pins, offscreen-render
  determinism, IK/grasp failure modes) live here because they are not written
  down anywhere upstream. For the physics/model API itself (MJCF syntax, the
  `mujoco` Python bindings, `mj_step`/`mj_forward`, the `Renderer` class, and
  robot assets) point upstream to MuJoCo's docs and the mujoco_menagerie repo
  (see References); do not restate the API from memory, it changes across 3.x
  releases.
- **On macOS, `MUJOCO_GL=cgl` is the only headless render backend.** <!-- id: macos-gl-cgl-only --> osmesa and
  egl are Linux-only (mujoco#2164). Set it before importing `mujoco`.
- **Watch the version-pin trap.** <!-- id: mujoco-version-pin-trap --> Menagerie's SO-101 needs MuJoCo ≥3.1.3, but
  common gym wrappers pin it *down* (gym-hil pins `mujoco<3.9`, gym-xarm pins
  `<3.0`); depending on one of those can silently downgrade MuJoCo below the
  model's requirement. Pin MuJoCo explicitly and verify the installed version.
- **Hand-written IK fails silently.** <!-- id: ik-fails-silently --> A damped-least-squares solver does not
  raise on an unreachable target or on a bad grasp orientation; it returns a
  locally-optimal pose short of the goal. Probe reachability and calibrate
  grasps empirically (see Usage patterns); never assume a returned qpos means
  success.
- **Verify the asset matches the robot's intended work envelope** <!-- id: verify-asset-work-envelope --> before
  blaming IK. Menagerie scenes are generic and can put the floor, object spawn,
  or camera in the wrong place for the arm (see Platform gotchas).

## Quick start

**1. Install (pin MuJoCo explicitly, above the menagerie floor):** <!-- id: pin-mujoco-install -->

```bash
uv add "mujoco>=3.1.3"   # current 3.10 as of 2026-07; SO-101 needs >=3.1.3
```

Do not let a gym wrapper (gym-hil, gym-xarm) choose the version transitively;
it can pin MuJoCo below what the model needs.

**2. Get the robot asset from mujoco_menagerie, not upstream MJCF.** <!-- id: use-menagerie-not-upstream-mjcf --> Clone
[mujoco_menagerie](https://github.com/google-deepmind/mujoco_menagerie) and use
its SO-101 model (the robotstudio_so101 directory, which ships a pick-and-place
scene, manipulation-tuned collision geometry, and a camera mount). Do NOT use
the upstream TheRobotStudio/SO-ARM100 MJCF: its own README admits the gripper
linear-joint mapping is not reflected in the model.

**3. Headless offscreen render (macOS):**

```python
import os
os.environ["MUJOCO_GL"] = "cgl"   # macOS headless backend; must precede import
import mujoco

model = mujoco.MjModel.from_xml_path("scene.xml")
data = mujoco.MjData(model)
renderer = mujoco.Renderer(model, height=256, width=256)
# warm-up: one throwaway full reset+render (see determinism gotcha below)
mujoco.mj_resetData(model, data)
mujoco.mj_forward(model, data)
renderer.update_scene(data, camera="fixed_cam")
_ = renderer.render()
```

**4. Interactive viewer needs `mjpython`.** `mujoco.viewer.launch_passive`
requires launching with `mjpython`, not `python` (and is buggy under `uv`,
mujoco#1923). Offscreen rendering (step 3) has no such requirement.

For MJCF syntax, the bindings, and API signatures, go to MuJoCo's own docs
(References) before hand-writing model or physics code.

## Usage patterns

**Headless render is not the bottleneck on Apple Silicon.** <!-- id: render-not-bottleneck --> Measured on the
vla-trial build: ~84 fps at 256x256 via `MUJOCO_GL=cgl`, against a 60 fps
floor; `mj_step` ~0.012 ms vs ~11.8 ms per render. This kills the common
upstream worry that "offscreen render is extremely slow": for a low-DOF arm
scene it is comfortably real-time. Render, don't skip frames to save time.

**Deterministic offscreen rendering** (needed for reproducible datasets/evals):

- **Use a static xyaxes look-at camera, not `mode="targetbody"`.** <!-- id: static-camera-not-targetbody --> A
  target-tracking camera re-runs its smoothing filter on every
  `update_scene()`, making the rendered pose call-history-dependent (observed
  up to 85 LSB drift) even when the physics is bit-identical.
- **Warm up the renderer with one throwaway full reset + render in `__init__`.** <!-- id: renderer-warmup-reset -->
  A fresh `mujoco.Renderer`'s first reset→`mj_forward`→render cycle is not
  repeatable against later renders (~6 LSB). A raw pre-`mj_forward` warm-up does
  NOT consume the cold cycle; it must be a full reset+render.
- Ruled out as a cause: MSAA offsamples (a red herring here).

**Reachability probing (IK).** <!-- id: reachability-probing-ik --> Because the DLS solver returns silently on an
unreachable target, probe the full task footprint × every lift height
*independently* before trusting the env. Menagerie's default box spawn
(0.5, 0, 0.03) is unreachable by the SO-101 mount, for example.

**Empirical grasp calibration.** <!-- id: empirical-grasp-calibration --> The end-effector site is not the grasp point.
Calibrate the gripper-local grasp offset by experiment: teleport the object to
a grid, close, lift, and keep the offsets that actually lift. Key facts:

- Calibration is a function of **pose**, not a gripper property; recalibrate <!-- id: grasp-calibration-is-pose-function -->
  after any wrist-orientation change.
- It is millimeter-sensitive and non-monotonic: a 3 mm offset change cost 6/10 <!-- id: grasp-offset-millimeter-sensitive -->
  success in one sweep. Sweep the grasp offset *and* the pedestal height
  end-to-end rather than tuning one in isolation.

**Wrist-roll orientation is the silent half of position-only IK.** <!-- id: wrist-roll-position-only-ik --> On a 5-DOF
arm, position-only IK leaves wrist roll free. At roll≈0 the pinch axis was 91%
vertical, so a 4.2 cm aperture could not span a 6 cm cube (closed on air,
0/10). Diagnostic: print the world-frame pinch axis and check it is
perpendicular to the grasped dimension. Fix: solve wrist roll as a 1-D root
find (`pinch_z(roll)` is a smooth sinusoid), then re-solve position with roll
pinned via jnt_range; adding roll to the DLS objective *diverges*. This took
the oracle grasp from 0/10 to 8/10. (Position-only IK also gives no signal
about the orientation it chose; a level-finger constraint may be genuinely
infeasible on a 5-DOF arm.)

**Debugging "arm won't reach / knocked the object over":**

- **Bisect IK vs obstruction** <!-- id: bisect-ik-vs-obstruction --> by deleting/banishing the object and re-running
  the same pose. Cube present: arm_err=0.18, saturated_joints=[1 2]; cube
  banished: arm_err=0.0006, saturated=[]. Instantly tells you whether IK or a
  collision is the problem.
- **Check `qfrc_actuator` vs `forcerange`** <!-- id: check-qfrc-actuator-forcerange -->: saturated joints mean BLOCKED,
  not slow. Raising settle steps (e.g. 12→60) is a dead end when joints are
  saturated.
- **The first waypoint can sweep the arm through the object** <!-- id: first-waypoint-sweep-collision -->: a position
  servo takes an arbitrary joint path to the target. Raising the initial
  approach waypoint (0.16→0.20 m) fixed a 2/10 → 10/10 grasp.

**Resolve gripper geoms by body, not name.** <!-- id: gripper-geoms-by-body --> Menagerie leaves mesh geoms
UNNAMED, so a name-based gripper-geom list silently misses the fixed-jaw
collision mesh, a false-success risk in contact checks. Collect gripper geoms
by `model.geom_bodyid` membership instead.

**Determine gripper polarity empirically.** <!-- id: gripper-polarity-empirical --> Drive the gripper joint to each end
of its `actuator_ctrlrange` and measure the fingertip gap. SO-101 is
LOW=closed / HIGH=open; the upstream SO-ARM100 docs claim the opposite and even
admit their own MJCF disagrees, so measure, don't trust the docs.

## Platform gotchas

- **macOS headless render backend is `MUJOCO_GL=cgl` only** (osmesa/egl are
  Linux-only, mujoco#2164). Set it before `import mujoco`.
- **The macOS CGL renderer is thread-affine: a cross-thread first render is a
  silent deadlock.** <!-- id: cgl-thread-affine-deadlock --> A `mujoco.Renderer` created on one thread hangs forever
  (no error, no timeout, no exception) the first time a *different* thread
  renders on it; the stack sits in cgl `make_current`. This bites frameworks
  that run handlers on worker threads (e.g. Gradio): a first `env.reset()`
  called from a worker freezes the stream. Fix: construct the env/renderer in
  the same thread that runs the episode (a per-run env in the handler), not once
  at import on the main thread. (observed 2026-07-15, vla-trial)
- **Interactive viewer needs `mjpython`**, <!-- id: mjpython-viewer-required --> not `python`, and is buggy under
  `uv` (mujoco#1923). Offscreen rendering does not need `mjpython`.
- **No off-the-shelf SO-101 Gymnasium env exists.** <!-- id: no-so101-gym-env --> gym-hil is Franka-only;
  LIBERO is Linux-only. A language-conditioned SO-101 sim env must be
  hand-built; budget for it rather than expecting to import one.
- **Menagerie assets can be geometrically wrong for the robot.** <!-- id: menagerie-assets-geometrically-wrong --> The SO-101
  scene put the floor at the arm's base level; a real rig has base + objects on
  a raised surface. A 6 cm pedestal restored the intended work envelope (fixing
  a forced ~32° down-pitch). Check the asset world matches the robot's intended
  envelope before blaming the IK solver.

## Customization

- **Different arm from menagerie:** the failure modes above (silent IK,
  unnamed geoms, empirical gripper polarity, static-camera determinism) are
  general MuJoCo/menagerie traps, not SO-101-specific. Re-run the empirical
  calibration (grasp offset grid, gripper-polarity sweep, reachability probe)
  for the new model rather than porting the SO-101 numbers.
- **Different task envelope:** re-check the scene geometry (floor height,
  object spawn, camera pose) against the robot's reach before writing task
  logic; a pedestal or a re-placed spawn is often the fix, not the IK.
- **Feeding a LeRobot pipeline:** MuJoCo here produces the observations and
  executes actions; the dataset format, recording, and policy training/eval
  belong to the `lerobot` skill; keep the sim env and the dataset layer
  separate.

## References

- Upstream (primary sources; check before writing model or physics code):
  [MuJoCo documentation](https://mujoco.readthedocs.io/) (MJCF, Python
  bindings, the `Renderer` API), [google-deepmind/mujoco GitHub
  repo](https://github.com/google-deepmind/mujoco) (releases and the issues
  cited above: #2164 render backends, #1923 viewer-under-uv),
  [mujoco_menagerie](https://github.com/google-deepmind/mujoco_menagerie)
  (robot assets, incl. the SO-101 model; prefer this over upstream
  TheRobotStudio/SO-ARM100 MJCF). All facts above observed on MuJoCo 3.x,
  macOS arm64, 2026-07-13/14 (vla-trial SO-101 build); re-verify version-tied
  claims against current releases.
- Sibling skills: the `simulation` umbrella (routes here for lightweight
  non-ROS manipulation; owns the Gazebo-vs-Isaac selection), `gazebo` (ROS 2
  simulation), `isaac-sim` / `isaac-lab` (GPU-gated photoreal / RL sim),
  `lerobot` (policy training/eval that consumes this sim's observations),
  `architect` (routes the whole stack decision here).

## Changelog

<!-- One dated line per battle-tested change, added by skill-author hardening sessions. -->

- 1.0.2 (2026-08-03): style pass; removed em dashes throughout (no content changes).

- 1.0.1 (2026-08-01): anchor IDs added to claim-bearing items (learning-engine Phase 1); no content changes.

- 1.0.0 (2026-07-31): created: captured from the vla-trial SO-101 manipulation build (2026-07-13/14).