llms-full.txt · git:20261106.a7c2b12 · 2026-11-06 · sha256 131b1b9591c93dff
llms-full.txt git:20261106.a7c2b12B
Immutable. This exact content never changes and is served at /api/v1/blob/131b1b9591c93dff.
# Three-Live & XR-Live Complete LLM Reference (`llms-full.txt`)
> Auto-generated from `docs/` and `skills/`. Do not edit by hand; run `node scripts/build_llms_txt.js`.
================================================================================
SOURCE: docs/llm-guide.md
================================================================================
# LLM & Agent Authoring Guide (`three-live` & `xr-live`)
This guide is designed for **LLMs, coding assistants, and autonomous agents**
generating 3D and Spatial Computing (AR/VR/XR) code with `<three-live>` and
`<xr-live>`.
- **Repository Agent Instructions:** [`AGENTS.md`](../AGENTS.md)
- **Installable Agent Skills:**
- [`skills/three-live-authoring/SKILL.md`](../skills/three-live-authoring/SKILL.md) (building 3D/XR scenes and apps)
- [`skills/xr-live-plugin-dev/SKILL.md`](../skills/xr-live-plugin-dev/SKILL.md) (contributing cross-platform traits and plugins)
- **Single-File Context Feeds:** [`/llms.txt`](../llms.txt) (concise index) and [`/llms-full.txt`](../llms-full.txt) (complete reference)
You can copy the **System Prompt Snippet** below directly into any LLM prompt or
agent skill to guarantee valid, zero-hallucination output.
--------------------------------------------------------------------------------
## Copy-Paste System Prompt Snippet for LLMs
```markdown
You are authoring 3D and Spatial Computing scenes using Google's `three-live` and `xr-live` web components.
### 1. Choose the Right Component & Bundle
- Use `<three-live>` (`three-live.min.js` or `three-live-full.min.js`) for 3D product viewers, procedural Three.js scenes, `KHR_interactivity` behavior graphs, PBR Next materials, and 1-tap mobile AR handoff (Quick Look / Scene Viewer / WebXR).
- Use `<xr-live>` (`xr-live.min.js` or `xr-live-full.min.js`) when the scene requires interactive 6DoF WebXR, declarative spatial traits (`userData.xr`), spatial HTML cards (`<div slot="spatial">`), depth occlusion & physics (`xr-depth`), 25-joint hand tracking (`xr-hands`), any of the 14 spatial plugins (`head-follow`, `manipulation-handles`, `world-understanding`, `gemini-live`, `spatial-paint`, `depth-particles`, `splat-loader`, `spatial-measure`, `magnetic-snap`, `gaze-dwell`, `exploded-view`, `persistent-anchors`, `spatial-locomotion`, `shared-room`), or any of the 6 `xr-live-sim` simulation plugins (`character-controller`, `steering-agents`, `rigid-body`, `hit-zones`, `objectives`, `synth-audio`; available in `xr-live-sim.min.js` and `xr-live-full.min.js`).
### 2. Procedural `<script type="text/three">` Rules
- Inside `<script type="text/three">`, the globals `THREE` and `scene` (a `THREE.Scene` instance) are pre-defined.
- ALWAYS add the `allow-client-scripts` attribute to any `<three-live>` or `<xr-live>` element that contains an inline `<script type="text/three">`; without it the script is refused and nothing renders.
- Add all meshes, groups, and lights directly to `scene.add(...)`, and give interactive meshes a unique `mesh.name = '...'`.
- Attach canonical spatial traits via `mesh.userData.xr = { ... }`:
- `grab`: `'1-hand' | '2-hand' | true`
- `scale`: `{ min: 0.1, max: 5.0 } | true`
- `billboard`: `'capsule' | 'cylindrical' | 'spherical' | true`
- `follow`: `'head' | 'hand-left' | 'hand-right'` or `{ mode: 'head', lazy: true, distance: 1.2, deadzoneDeg: 15, stiffness: 6, height: -0.1, faceUser: true }`
- `anchor`: `'floor' | 'wall' | 'ceiling' | 'table' | 'horizontal' | 'vertical'` or `{ target: 'table', id: 'part-1', persist: true }`
- `physics` / `rigidBody`: `'dynamic' | 'static' | 'kinematic' | true` or `{ body: 'dynamic' | 'static' | 'kinematic', shape: 'auto' | 'box' | 'sphere' | 'cylinder', mass: 1, restitution: 0.35, friction: 0.5, linearDamping: 0.05, angularDamping: 0.1, gravityScale: 1, velocity: [0, 0, 0], angularVelocity: [0, 0, 0], lockRotation: false }`
- `orbit`: `{ target: [x, y, z] | 'nodeName', radius: 0.5, speed: 0.8, height: 0, faceCenter: true }`
- `handles`: `{ rotateAxis: 'y' | 'none', minScale: 0.2, maxScale: 3.0 } | true`
- `dimensions`: `{ unit: 'auto' | 'm' | 'cm' | 'mm' | 'in' | 'ft-in', axes: ['x', 'y', 'z'], color: '#38bdf8', showOnSelect: false } | true`
- `snapTo`: `{ grid: 0.05, angleDeg: 15, surface: 'floor' | 'wall' | 'ceiling' | 'any' | 'none', socketTag: 'dock', radius: 0.15 } | true`
- `snapSocket`: `'dock'` or `{ tag: 'dock', radius: 0.15, alignRotation: true, highlightColor: '#22c55e' }`
- `gaze`: `{ dwellMs: 1200, action: 'select' | 'ask', repeat: false } | true`
- `explode`: `{ direction: [0, 1, 0], distance: 0.35, layer: 1, leaderLine: true } | boolean`
- `navigable`: `true | 'walkable' | 'blocked'`
- `sync`: `boolean` (multi-user `shared-room` transform sync; defaults to `true` on grabbable nodes)
- `particles`: `{ preset: 'rain' | 'snow' | 'sparks' | 'confetti', count: 400, area: [2, 2, 2], speed: 1.0 }`
- `holdToAsk`: `{ prompt: 'What is this?', holdMs: 600 } | true`
- `worldUnderstanding`: `{ planes: true, meshes: false, reticle: true }`
- `splatUrl`: `'https://.../capture.spz'` (or `.splat`)
- `splat`: `{ url: 'https://.../capture.spz', shDegree: 3, maxSplats: 500000 }`
- `bloom`: `{ strength: 1.2, threshold: 0.8, radius: 0.5 } | boolean | number`
- `outline`: `{ color: '#38bdf8', thickness: 0.03 } | boolean | string`
- `ik`: `{ root: 'RootNode', mid: 'MidNode', effector: 'EffectorNode', target: 'TargetNodeName' | [x, y, z], pole: 'PoleNodeName' | [x, y, z], solver: 'two-bone' | 'fabrik', iterations: 10, tolerance: 0.001, weight: 1 }`
- `text`: `{ text: '3D Label', size: 0.2, depth: 0.04, bevelEnabled: true }`
- `audio`: `{ src: 'https://.../sound.mp3', refDistance: 1.0, loop: false }`
- `character`: `true` or `{ mode: 'third-person' | 'first-person' | 'top-down' | 'fixed-camera', speed: 4, jumpVelocity: 5.5, gravity: 15, radius: 0.35, height: 1.7, cameraDistance: 3.5, cameraHeight: 1.6 }`
- `agent`: `'patrol'` or `{ behavior: 'patrol' | 'chase' | 'flee' | 'follow' | 'wander', speed: 2.5, waypoints: [[x, y, z], ...], loop: true, sightDistance: 8, sightAngleDeg: 110, reachDistance: 0.6, target: 'character' | 'nodeName', alertBehavior: 'chase' }`
- `integrity`: `100` or `{ max: 100, current: 100, group: 'hazard' | 'player' | 'prop', scoreValue: 25, removeOnDeplete: true, hitCooldownMs: 120, contactAmount: 25 }`
- `zone`: `{ shape: 'box' | 'sphere' | 'cylinder', size: [2, 2, 2], radius: 1.5, height: 2, triggerGroup: 'player', action: 'checkpoint' | 'hazard' | 'complete' | 'fail' | 'event', ratePerSec: 20, once: false, color: '#22c55e' }`
- `spawner`: `{ template: 'BotNode', intervalSec: 3, maxActive: 5, maxTotal: 15, radius: 1.5, autoStart: true }`
- `objective`: `'Inspect valve'` or `{ id: 'valve-1', label: 'Inspect valve', type: 'reach' | 'clear' | 'collect' | 'activate' | 'hold', points: 50, order: 1, required: true, holdSec: 1.5, reachRadius: 1.2 }`
- `audioCues`: `{ onHit: 'thud', onDeplete: 'buzz', onCollect: 'chime', onObjective: 'beep', onJump: 'whoosh', onScenarioPass: 'sweep', onScenarioFail: 'alarm', pitchHz: 440, gain: 0.7 }`
- Attach declarative `KHR_interactivity` behaviors via `mesh.userData.interactivity = { onSelect: [...], onHoverIn: [...], onStart: [...] }` using actions `toggleTransform`, `interpolate`, `setProperty`, `toggleVisibility`, `cycleMaterialColor`, `playAnimation`, `stopAnimation`, or `sendEvent`.
- NEVER use `GLTFLoader`, `OBJLoader`, `TextureLoader`, `EffectComposer`, `ShaderMaterial`, `RawShaderMaterial`, or `requestAnimationFrame` inside `<script type="text/three">`.
- NEVER create a `THREE.WebGLRenderer` or attach DOM canvas elements inside `<script type="text/three">`.
- Respect strict scene quotas: max 1,000 meshes, 100,000 vertices, 300,000 indices, 16 textures, 3-second execution timeout.
### 3. Allowed `THREE.*` Constructors inside `<script type="text/three">`
- Math & IK: `Vector2`, `Vector3`, `Vector4`, `Euler`, `Quaternion`, `Matrix3`, `Matrix4`, `Color`, `MathUtils`, `Clock`, `IKSolver` (`solveTwoBone`, `solveFABRIK`)
- Geometries & CAD: `BoxGeometry`, `PlaneGeometry`, `SphereGeometry`, `CylinderGeometry`, `ConeGeometry`, `TorusGeometry`, `TorusKnotGeometry`, `LatheGeometry`, `TubeGeometry`, `Shape`, `Path`, `ShapeGeometry`, `ExtrudeGeometry`, `TextGeometry`, `CSG` (`union`, `subtract`, `intersect`), `BufferGeometry`, `BufferAttribute`
- Materials: `MeshStandardMaterial`, `MeshBasicMaterial`, `MeshLambertMaterial`, `MeshPhongMaterial`, `MeshPhysicalMaterial` (plus `mesh.userData.variants` for `KHR_materials_variants`)
- Scene Graph & Instancing: `Scene`, `Group`, `Mesh`, `InstancedMesh` (`setMatrixAt`, `getMatrixAt`, `setColorAt`, `getColorAt`, `count`), `Object3D` (`position`, `rotation`, `scale`, `matrix`, `add`, `remove`, `traverse`)
- Lights & Cameras: `AmbientLight`, `DirectionalLight`, `PointLight`, `Camera`, `PerspectiveCamera`
```
--------------------------------------------------------------------------------
## DO vs. DO NOT Cheat Sheet
| DO (Supported & Fast) | DO NOT (Unsupported / Anti-Pattern) |
| :------------------------------------------------------------------------------ | :------------------------------------------------------------------ |
| Use `new THREE.MeshStandardMaterial({ color, metalness, roughness })` | Do NOT use `THREE.ShaderMaterial` or custom GLSL strings in scripts |
| Add objects to the global `scene` (`scene.add(mesh)`) | Do NOT instantiate `new THREE.WebGLRenderer()` or call `renderer.render()` |
| Set `mesh.userData.xr = { grab: '2-hand', handles: { rotateAxis: 'y' } }` | Do NOT write manual WebXR session setup (`navigator.xr.requestSession`) |
| Set `mesh.userData.interactivity = { onSelect: [{ action: 'toggleTransform', ... }] }` | Do NOT try to access `window` or `document` inside sandboxed `<script type="text/three">` |
| Use `<div slot="spatial" data-position="x y z" data-xr-traits="grab billboard">` for spatial HTML UI | Do NOT use `new THREE.GLTFLoader().load(...)` inside procedural scripts (use `src="..."` or `<xr-anchor src="...">`) |
### Permissive Fallback Shims & Diagnostic Warnings (JS & Rust Compiler)
While LLMs should prefer the canonical APIs above for highest fidelity, both the
JavaScript authoring runtime (`three-author`) and the native Rust compiler
(`compiler/`) include **permissive fallback shims** so common LLM-generated
Three.js assumptions never crash or abort a 3D/XR scene:
- **Shaders (`ShaderMaterial`, `RawShaderMaterial`):** Gracefully downgraded
to `MeshStandardMaterial`, preserving `uniforms.color.value` /
`params.color` and opacity so the mesh renders in 3D and XR.
- **Polyhedra & Extended Geometries (`IcosahedronGeometry`,
`OctahedronGeometry`, `DodecahedronGeometry`, `TetrahedronGeometry`,
`CapsuleGeometry`, `RingGeometry`, `CircleGeometry`):** Mapped to
parameter-compatible `SphereGeometry` or `CylinderGeometry` primitives.
- **Lights (`SpotLight`, `HemisphereLight`, `RectAreaLight`):** Mapped to
`PointLight` or `AmbientLight`.
- **Unknown / Hallucinated Classes:** Any unknown `THREE.*` class resolves to
a safe recursive no-op proxy, and `new THREE.Mesh(geo, mat)` automatically
substitutes a fallback `BoxGeometry` / `MeshStandardMaterial` if passed a
non-geometry or non-material proxy.
- **Browser Boilerplate (`requestAnimationFrame`, `window`, `document`):**
Safely shimmed in headless/Worker/Rust sandboxes so trailing render loops
(`requestAnimationFrame(animate)`) or `window.innerWidth /
window.innerHeight` checks do not throw `ReferenceError`.
- **Diagnostic Warnings:** Every degraded feature logs a deduplicated
`console.warn` and is recorded in `THREE.getAuthorWarnings()` (reset via
`THREE.clearAuthorWarnings()`).
--------------------------------------------------------------------------------
## Canonical Few-Shot Templates
### Template 1: Interactive 3D Scene with `KHR_interactivity` & AR Handoff (`<three-live>`)
```html
<script type="module" src="/dist/three-live-full.min.js"></script>
<three-live allow-client-scripts camera-controls ar auto-rotate shadow-intensity="0.7" style="width: 100%; height: 500px; display: block;">
<script type="text/three">
const base = new THREE.Mesh(
new THREE.CylinderGeometry(0.6, 0.7, 0.2, 32),
new THREE.MeshStandardMaterial({ color: 0x1e293b, metalness: 0.8, roughness: 0.2 })
);
base.name = 'pedestal';
scene.add(base);
const crystal = new THREE.Mesh(
new THREE.SphereGeometry(0.35, 32, 32),
new THREE.MeshStandardMaterial({ color: 0x38bdf8, metalness: 0.1, roughness: 0.1 })
);
crystal.name = 'crystal';
crystal.position.set(0, 0.55, 0);
crystal.userData.interactivity = {
onSelect: [
{ action: 'toggleTransform', property: 'translation', from: [0, 0.55, 0], to: [0, 0.85, 0], duration: 0.35 },
{ action: 'cycleMaterialColor', colors: [[0.22, 0.74, 0.97, 1], [0.66, 0.33, 0.97, 1]], duration: 0.25 }
]
};
scene.add(crystal);
const light = new THREE.DirectionalLight(0xffffff, 2.0);
light.position.set(2, 4, 3);
scene.add(light);
</script>
</three-live>
```
### Template 2: Full Spatial Computing Scene with Extended XR Traits (`<xr-live>`)
```html
<script type="module" src="/dist/xr-live-full.min.js"></script>
<xr-live
allow-client-scripts
xr-mode="ar"
xr-depth="occlusion-physics"
xr-hands="gestures"
xr-simulator="living-room"
world-understanding="planes reticle"
spatial-paint="#38bdf8"
camera-controls
style="width: 100%; height: 600px; display: block;">
<!-- Interactive procedural 3D objects with canonical userData.xr traits -->
<script type="text/three">
const sculpture = new THREE.Mesh(
new THREE.TorusKnotGeometry(0.18, 0.06, 64, 16),
new THREE.MeshStandardMaterial({ color: 0xa855f7, metalness: 0.6, roughness: 0.25 })
);
sculpture.name = 'sculpture';
sculpture.position.set(0, 1.1, -1.0);
sculpture.userData.xr = {
grab: '2-hand',
handles: { rotateAxis: 'y', minScale: 0.3, maxScale: 3.0 },
holdToAsk: { prompt: 'What is this sculpture?', holdMs: 600 }
};
scene.add(sculpture);
const satellite = new THREE.Mesh(
new THREE.SphereGeometry(0.05, 24, 24),
new THREE.MeshStandardMaterial({ color: 0x38bdf8 })
);
satellite.name = 'satellite';
satellite.userData.xr = {
orbit: { target: 'sculpture', radius: 0.38, speed: 1.0, faceCenter: true }
};
scene.add(satellite);
</script>
<!-- Live HTML spatial card floating next to the 3D object -->
<div slot="spatial" data-position="0.45 1.1 -1.0" data-xr-traits="grab billboard">
<div style="padding: 14px; background: #111827; color: #f9fafb; border-radius: 12px; width: 220px;">
<h4>Spatial Control Panel</h4>
<p>Drag corner handles to scale, or pinch in empty space to paint in 3D.</p>
<button onclick="console.log('Clicked in 3D space!')">Inspect Details</button>
</div>
</div>
</xr-live>
```
### Template 3: Collaborative CAD Studio (`exploded-view` + `spatial-measure` + `magnetic-snap` + `shared-room`)
```html
<script type="module" src="/dist/xr-live-full.min.js"></script>
<xr-live
allow-client-scripts
spatial-measure="all"
measure-unit="cm"
magnetic-snap="all"
gaze-dwell="auto"
explode="0.4"
explode-mode="axial-y"
section-plane="x:0"
persistent-anchors="cad-review"
persist-url-hash
locomotion="teleport snap-turn"
shared-room="cad-review-room"
peer-name="Reviewer"
peer-color="#38bdf8"
camera-controls
style="width: 100%; height: 600px; display: block;">
<script type="text/three">
const deck = new THREE.Mesh(
new THREE.CylinderGeometry(1.4, 1.4, 0.04, 48),
new THREE.MeshStandardMaterial({ color: 0x0f172a, roughness: 0.85 })
);
deck.name = 'walkable-deck';
deck.position.set(0, -0.45, -1.2);
deck.userData.xr = { navigable: 'walkable', explode: false, sync: false };
scene.add(deck);
const chassis = new THREE.Mesh(
new THREE.CylinderGeometry(0.28, 0.32, 0.16, 36),
new THREE.MeshStandardMaterial({ color: 0x334155, metalness: 0.75, roughness: 0.25 })
);
chassis.name = 'turbine-chassis';
chassis.position.set(0, 0.75, -1.2);
chassis.userData.xr = {
dimensions: { unit: 'cm', axes: ['x', 'y', 'z'] },
snapSocket: { tag: 'rotor-bay', radius: 0.18, highlightColor: '#22c55e' },
explode: false
};
scene.add(chassis);
const shroud = new THREE.Mesh(
new THREE.CylinderGeometry(0.24, 0.26, 0.22, 36),
new THREE.MeshStandardMaterial({ color: 0x38bdf8, metalness: 0.65, roughness: 0.2 })
);
shroud.name = 'upper-shroud';
shroud.position.set(0, 0.96, -1.2);
shroud.userData.xr = {
grab: '1-hand',
snapTo: { socketTag: 'rotor-bay', radius: 0.18, angleDeg: 15 },
explode: { direction: [0, 1, 0], distance: 0.32, layer: 2, leaderLine: true },
gaze: { dwellMs: 1000, action: 'select' },
anchor: { id: 'upper-shroud', persist: true },
sync: true
};
scene.add(shroud);
const key = new THREE.DirectionalLight(0xffffff, 2.2);
key.position.set(2, 4, 3);
scene.add(key);
scene.add(new THREE.AmbientLight(0xffffff, 0.6));
</script>
</xr-live>
```
--------------------------------------------------------------------------------
## 5. Closed-Loop AI Agent Verification (`three-live-cli`)
AI coding agents can validate, visually inspect, and simulate
`<script type="text/three">` scenes, `.html` host documents, and `.glb` files in
under `50 ms` without launching a headless browser:
```bash
# 1. Structured JSON Diagnostics (TL100-TL402) + Spatial Frustum/Grounding/Lighting Linter
three-live-cli validate scene.js --json
# 2. 4-Up Quad Contact Sheet (Perspective + Front + Right + Top with 1m Grid & AABB Labels)
three-live-cli render scene.js -o preview.png --view=quad --grid --annotations
# 3. Multi-Frame Temporal Filmstrip or Animated GIF89a (`animate(t)` + KHR_interactivity Tweens)
three-live-cli render scene.js -o anim.png --frames=4 --duration=2.0 --trigger-select=crystal
# 4. Deterministic 60Hz Scenario & KHR_interactivity Simulation
three-live-cli sim scene.js --duration=5.0 \
--actions='[{"at":0.2,"type":"select","target":"crystal"},{"at":0.5,"type":"moveCharacter","direction":[0,0,-1],"duration":1.5,"jump":true}]' \
--json
```
### Diagnostic Code Summary (`TL100–TL402`)
| Code | Category | Meaning & Fix |
| :------ | :------- | :----------------------------------- |
| `TL100` | Compiler | JavaScript syntax error, uncaught |
: : : runtime exception, or sandbox quota :
: : : violation (includes 1-based :
: : : `line`/`column`). :
| `TL101` | Compiler | Unsupported material |
: : : (`ShaderMaterial`, :
: : : `RawShaderMaterial`) downgraded to :
: : : `MeshStandardMaterial`. :
| `TL102` | Compiler | Unsupported geometry |
: : : (`IcosahedronGeometry`, :
: : : `OctahedronGeometry`, :
: : : `DodecahedronGeometry`, :
: : : `TetrahedronGeometry`, :
: : : `TubeGeometry`, :
: : : `PolyhedronGeometry`) approximated :
: : : as sphere/cylinder. :
| `TL103` | Compiler | Blocked network/loader API |
: : : (`THREE.GLTFLoader`, :
: : : `TextureLoader`, `fetch`) inside :
: : : sandbox; use `<three-live :
: : : src="...">`. :
| `TL104` | Compiler | `requestAnimationFrame` used in |
: : : static `.glb`/`.usdz` export; prefer :
: : : `userData.xr.orbit` or :
: : : `userData.interactivity`. :
| `TL105` | Compiler | Unknown `THREE.*` property |
: : : intercepted by chaining proxy. :
| `TL201` | Schema | Unrecognized `userData.xr` or |
: : : `userData.interactivity` key with :
: : : Levenshtein typo suggestion (e.g., :
: : : `grabbable` -> `grab`, `onClick` -> :
: : : `onSelect`). :
| `TL202` | Schema | Out-of-range numeric trait clamped |
: : : to valid domain (`dwellMs`, :
: : : `bloom.strength`, :
: : : `outline.thickness`, :
: : : `splat.shDegree`). :
| `TL203` | Schema | Unsafe URL or anchor ID rejected |
: : : (`safeMediaUrl`, `isAllowedLiveUrl`, :
: : : `safeAnchorId`). :
| `TL204` | Schema | Broken node reference |
: : : (`orbit.target`, :
: : : `ik.root`/`mid`/`effector`/`target`, :
: : : `spawner.template`, :
: : : `interactivity.*.target`). :
| `TL205` | Schema | Unnamed interactive or spatial mesh |
: : : (`mesh.name === ""`); assign a :
: : : unique `mesh.name`. :
| `TL301` | Spatial | Empty scene (`meshCount == 0` or |
: : : `vertexCount == 0`). :
| `TL302` | Spatial | Camera position is enclosed inside a |
: : : mesh's world AABB. :
| `TL303` | Spatial | Mesh is outside the camera frustum |
: : : (or scenes fill `< 1%` / `> 95%` of :
: : : the viewport). :
| `TL304` | Spatial | Scene diagonal is microscopic (`< |
: : : 0.01 m`) or gigantic (`> 200 m`, :
: : : e.g. millimeter units). :
| `TL305` | Spatial | Metallic PBR materials (`metalness > |
: : : 0.5`) with no scene lights and no :
: : : `environment`. :
| `TL306` | Spatial | Two non-floor sibling meshes share |
: : : `> 80%` AABB intersection volume :
: : : (accidental duplicate placement). :
| `TL307` | Spatial | Scene or `character`/`physics: |
: : : "dynamic"` mesh is floating :
: : : (`min.y > 0.25 m`) or buried (`min.y :
: : : < -0.5 m`). :
| `TL401` | HTML | `<three-live>` / `<xr-live>` |
: : : contains `<script :
: : : type="text/three">` without :
: : : `allow-client-scripts`. :
| `TL402` | HTML | `<xr-live>` spatial attributes/slots |
: : : (`xr-hands`, `gemini-live`, :
: : : `plugins`, `slot="spatial"`) placed :
: : : on `<three-live>`. :
================================================================================
SOURCE: skills/three-live-authoring/references/xr-traits-and-plugins.md
================================================================================
# `<three-live>` & `<xr-live>` Complete Traits, Plugins & Events Reference
## 1. `<xr-live>` Host Attributes
`<xr-live>` inherits every `<three-live>` attribute (`src`, `ios-src`, `poster`, `scale`, `camera-orbit`, `camera-target`, `min-camera-orbit`, `max-camera-orbit`, `field-of-view`, `selected-mesh`, `camera-controls`, `ar`, `ar-modes`, `ar-scale`, `ar-placement`, `script-src`, `sandbox`, `allow-client-scripts`, `exposure`, `environment-image`, `skybox-image`, `shadow-intensity`, `shadow-softness`, `tone-mapping`, `autoplay`, `animation-name`, `auto-rotate`) and adds:
Attribute | Values | Default | Description
:-------------------- | :--------------------------------------------------------------------------------------------- | :---------- | :----------
`xr-mode` | `'ar' \| 'vr' \| 'inline' \| 'auto'` | `'auto'` | Target WebXR session mode.
`xr-depth` | `'occlusion' \| 'occlusion-physics' \| 'mesh' \| 'none'` | `'none'` | Hardware depth occlusion and analytical heightfield physics collision.
`xr-hands` | `'gestures' \| 'joints' \| 'pinch' \| 'none'` | `'none'` | 25-joint `XRHand` tracking and gesture classification (`pinch`, `grab`, `point`, `palm-up`).
`xr-simulator` | `'living-room' \| 'office' \| 'tabletop' \| 'none'` | `'none'` | Enables the desktop 6DoF keyboard/mouse simulator with room floor/table planes.
`xr-profile` | `'quest3' \| 'android-xr' \| 'visionos' \| 'auto'` | `'auto'` | Target hardware profile hint.
`sound-feedback` | boolean attribute | `false` | Plays procedural Web Audio earcons (`grab`, `release`, `bounce`, `hover`) on interaction.
`world-understanding` | `'' \| 'planes' \| 'planes reticle' \| 'false'` | — | Activates `xr-live-world-understanding` plane/mesh tracking and optional gaze reticle.
`spatial-paint` | `'' \| '#rrggbb' \| 'false'` | — | Activates `xr-live-spatial-paint`; pinching in empty space extrudes Catmull-Rom 3D tube strokes.
`spatial-measure` | `'' \| 'bounds' \| 'tape' \| 'all' \| 'none'` | — | Activates `xr-live-spatial-measure` 3D dimension callouts and/or interactive AR tape measure.
`measure-unit` | `'auto' \| 'm' \| 'cm' \| 'mm' \| 'in' \| 'ft-in'` | `'auto'` | Default display unit for `spatial-measure`.
`magnetic-snap` | `'' \| 'grid' \| 'surface' \| 'sockets' \| 'all' \| 'none'` | — | Activates `xr-live-magnetic-snap` socket, grid, surface, and yaw snapping.
`snap-grid` | `number` or `'gx gy gz'` (meters) | — | Default Cartesian snap grid spacing in meters.
`snap-angle` | `number` (`0..180` degrees) | `0` | Default rotational yaw snap increment in degrees.
`gaze-dwell` | `'' \| 'auto' \| 'false' \| 'none'` | — | Activates `xr-live-gaze-dwell` hands-free eye/head gaze reticle and radial dwell selection.
`dwell-time` | `number` (`250..10000` ms) | `1200` | Default dwell duration in milliseconds for `gaze-dwell`.
`explode` | `number` (`0..1`) | `0` | Normalized multi-part assembly explosion factor (`xr-live-exploded-view`).
`explode-mode` | `'radial' \| 'axial-x' \| 'axial-y' \| 'axial-z' \| 'hierarchical'` | `'radial'` | Automatic explosion vector strategy when parts omit `userData.xr.explode.direction`.
`explode-scale` | `number` | `1.0` | Multiplier applied to per-part explosion distances.
`section-plane` | `'none' \| 'x:<d>' \| 'y:<d>' \| 'z:<d>' \| '-x:<d>' \| '-y:<d>' \| '-z:<d>' \| 'interactive'` | `'none'` | Real-time GPU cross-section clipping plane in scene (model) space (`d` in meters; the cut follows the model through AR placement and turntable rotation) with optional draggable gizmo frame.
`persistent-anchors` | `''` or `string` (namespace) | — | Activates `xr-live-persistent-anchors` cross-session transform persistence (`xr-live:anchors:<ns>`).
`persist-url-hash` | boolean attribute | `false` | Encodes persisted node poses into `#xr-state=<base64url>` (`<= 8192` bytes) for link sharing.
`locomotion` | `'' \| 'teleport' \| 'snap-turn' \| 'smooth' \| 'teleport snap-turn' \| 'none'` | — | Activates `xr-live-spatial-locomotion` parabolic teleportation, snap-turns, or smooth glide.
`snap-turn-deg` | `number` (degrees) | `30` | Horizontal thumbstick snap-turn increment in degrees.
`comfort-vignette` | boolean attribute | `true` | Shows a `120ms` peripheral FOV comfort vignette during teleports and snap-turns.
`shared-room` | `'<room-id>'` or `'wss://...'` | — | Joins a multi-user room via `BroadcastChannel("xr-live:room:<id>")` or a `wss://` relay (`xr-live-shared-room`).
`shared-room-voice` | boolean attribute | `false` | Opts in to voice **signaling** in `shared-room`: relays WebRTC offer/answer/ICE messages (`peerrtcsignal` event, `broadcastRtcSignal()`) and computes per-peer `voiceGain` (`computeSpatialVoiceGain`). Your app owns `getUserMedia`, `RTCPeerConnection`, and audio playback.
`peer-name` | `string` (max `32` chars) | — | Local peer display name broadcast in `shared-room`.
`peer-color` | `'#rrggbb'` | `'#38bdf8'` | Local peer 3D visor & laser ray tint color in `shared-room`.
`bloom` | `boolean \| string` | — | Enables HDR emissive bloom (`bloom`, `bloom="1.4"`, or `bloom="strength:1.2 threshold:0.75 radius:0.5"`).
`outline` | `boolean \| string` | — | Enables silhouette outlines on `selected-mesh` or nodes with `userData.xr.outline` (`outline="#38bdf8 0.03"`).
`variant-name` | `string` | — | Selects active `KHR_materials_variants` colorway name (`viewer.variantName` / `viewer.availableVariants`).
`webmcp` | boolean attribute | `false` | Registers `inspect_xr_scene`, `place_xr_object_at_gaze`, and `capture_xr_view` on `navigator.modelContext`.
`webmcp-allow-depth` | boolean attribute | `false` | Opt-in gate allowing `capture_xr_view` to include the 8×8 metric `depthGridMeters` array.
`webmcp-omit-pose` | boolean attribute | `false` | When present, `inspect_xr_scene` omits `headPosition` entirely.
## 3. Declarative `KHR_interactivity` Authoring (`userData.interactivity`)
Any mesh or group inside `<script type="text/three">` can declare interactive state machines compiled into standard Khronos `KHR_interactivity` graphs (`three-live/src/interactivity-compiler.ts`):
- **Triggers:**
- `onSelect`: Fired when clicked/tapped or pinched in XR (`pointer/onSelect`).
- `onHoverIn` / `onHoverOut`: Fired on pointer/ray hover enter and exit.
- `onStart`: Fired automatically when the scene loads (`event/onStart`).
- `onReceive`: `{ [eventName: string]: Action | Action[] }` — reacts to custom events sent by `sendEvent`.
- **Actions (single object or array executed in sequence):**
- `{ action: 'toggleTransform', property: 'translation' | 'rotation' | 'scale', to: number[], from?: number[], duration?: number }`
- `{ action: 'interpolate', target?: string, property: 'translation' | 'rotation' | 'scale' | 'baseColorFactor' | 'metallicFactor' | 'roughnessFactor' | 'emissiveFactor', to: number | number[], duration?: number }`
- `{ action: 'setProperty', target?: string, property: string, value: any }`
- `{ action: 'toggleVisibility', target?: string }`
- `{ action: 'cycleMaterialColor', colors: Array<[number, number, number, number?]>, duration?: number }`
- `{ action: 'playAnimation', animation?: number }` / `{ action: 'stopAnimation' }`
- `{ action: 'sendEvent', event: string, values?: Record<string, any> }`
---
## 4. Programmatic Plugin Controllers (`xr-live/spatial`)
When importing from `xr-live/spatial` (or using `xr-live-spatial.min.js` /
`xr-live-full.min.js`):
- **Gemini Live (`gemini-live.ts`):**
```ts
import { connectGeminiLive } from 'xr-live/spatial';
const live = connectGeminiLive(xrElement, {
url: 'wss://your-backend.example.com/gemini-live', // wss:// required (or ws://localhost)
model: 'models/gemini-2.0-flash-exp',
systemInstruction: 'You are a spatial assistant.',
frameIntervalMs: 1000, // clamped to [250, 60000] ms
streamAudio: true, // streams 16 kHz PCM mic chunks
playAudio: true, // plays streamed 24 kHz PCM model audio
speakReplies: false, // SpeechSynthesis fallback
detectObjects: true, // auto-registers report_objects tool -> xr-object-detected events
});
```
- **Spatial Paint (`spatial-paint.ts`):**
```ts
import { getSpatialPaintController } from 'xr-live/spatial';
const paint = getSpatialPaintController(xrElement);
paint.color = [0.22, 0.74, 0.97, 1.0];
paint.radius = 0.008;
paint.undo(xrElement.viewer);
paint.clear(xrElement.viewer);
```
- **World Understanding (`world-understanding.ts`):**
```ts
import { detectObjects, parseDetections, XRTapeMeasure } from 'xr-live/spatial';
const detections = parseDetections('[{"label":"chair","box":[200,300,800,700]}]');
const located3D = detectObjects(xrElement, detections);
```
- **Spatial Measure, Exploded View, Persistent Anchors, Locomotion & Shared
Room:**
```ts
import {
getSpatialMeasureController,
getMagneticSnapController,
getGazeDwellController,
getExplodedViewController,
getPersistentAnchorsController,
getSpatialLocomotionController,
getSharedRoomController,
} from 'xr-live/spatial';
xrElement.setExplode(0.65); // Smoothly animate exploded-view [0..1]
xrElement.clearMeasurements(); // Clear locked 3D tape-measure segments
const token = xrElement.exportAnchorState(); // Base64url layout snapshot (<= 8 KB)
xrElement.importAnchorState(token); // Restore layout snapshot programmatically
xrElement.teleportTo([0, 0, -1.5]); // Teleport viewer rig + emit xrteleport
const peers = xrElement.getSharedPeers(); // Active RemotePeerState[] in shared-room
```
---
## 5. DOM Events Summary
Event Name | Dispatched By | `event.detail` Payload
:--------------------------------- | :--------------------------- | :---------------------
`load` | `<three-live>` / `<xr-live>` | `{ model: GLBData }`
`error` | `<three-live>` / `<xr-live>` | `{ error: Error }`
`meshclick` | `<three-live>` / `<xr-live>` | `{ meshName, hitPoint, normal, distance }`
`camera-change` | `<three-live>` / `<xr-live>` | `{ theta, phi, distance }`
`ar-status` | `<three-live>` / `<xr-live>` | `{ status: 'session-started' \| 'not-presenting' \| 'failed', error? }`
`interactivityEvent` | `KHR_interactivity` VM | `{ id: string, values: Record<string, any> }`
`xr-select-start` / `move` / `end` | `XRInputManager` | `{ hand: 'left' \| 'right' \| 'screen', pivotPosition, quaternion, rayOrigin, rayDirection, hit }`
`xr-handles-change` | `manipulation-handles` | `{ mesh, scale: number, yawDeg: number }`
`xr-object-detected` | `world-understanding` | `{ label: string, position: [number, number, number], distanceMeters: number, source: 'plane' \| 'depth' \| 'fallback' }`
`xr-transcription` | `gemini-live` | `{ speaker: 'user' \| 'model', text: string, isFinal: boolean }`
`xr-hold-to-ask-listening` | `gemini-live` | `{ mesh: string }`
`xr-hold-to-ask` | `gemini-live` | `{ mesh: string, transcript: string, prompt: string }`
`xr-gemini-interrupted` | `gemini-live` | `{}`
`xr-paint-stroke` | `spatial-paint` | `{ index: number, points: number, lengthMeters: number }`
`measurecomplete` | `spatial-measure` | `{ start: [x,y,z], end: [x,y,z], distanceMeters: number, formatted: string }`
`xrsnap` | `magnetic-snap` | `{ node, socket, position: [x,y,z], quaternion: [x,y,z,w], mode: 'socket' \| 'grid' \| 'surface' \| 'angle' }`
`xrunsnap` | `magnetic-snap` | `{ node, socket }`
`gazeenter` / `gazeleave` | `gaze-dwell` | `{ node, point: [x,y,z] }` / `{ node }`
`gazedwell` | `gaze-dwell` | `{ node, point: [x,y,z], action: 'select' \| 'ask' }`
`explodechange` | `exploded-view` | `{ factor: number, source: 'attribute' \| 'gesture' \| 'api' \| 'peer' }`
`anchorsrestored` | `persistent-anchors` | `{ count: number, source: 'xr-anchor' \| 'storage' \| 'url-hash' }`
`xrteleport` | `spatial-locomotion` | `{ from: [x,y,z], to: [x,y,z], yawDeg: number }`
`peerjoin` / `peerleave` | `shared-room` | `{ peerId: string, name: string, color: string }`
`peergrabconflict` | `shared-room` | `{ peerId: string, nodeId: string }`
`peersync` | `shared-room` | `{ peerId: string, nodeId: string }`
`characterjump` / `characterland` | `xr-live-sim` | `{ nodeId, position }` / `{ nodeId, fallSpeed }`
`agentstatechange` / `agentreach` | `xr-live-sim` | `{ nodeId, state, targetId }` / `{ agentId, targetId, position }`
`xrhit` / `xrdepleted` | `xr-live-sim` | `{ targetId, sourceId, amount, remaining, point }` / `{ targetId, group, scoreValue }`
`xrzoneenter` / `xrzoneexit` | `xr-live-sim` | `{ zoneId, nodeId, group }`
`xrspawn` | `xr-live-sim` | `{ spawnerId, instanceId, index }`
`xrobjective` | `xr-live-sim` | `{ nodeId, tag, type, amount, totalScore, remainingRequired, outOfOrder }`
`xrscorechange` | `xr-live-sim` | `{ score, delta, attemptsLeft, timeRemaining }`
`xrscenarioend` | `xr-live-sim` | `{ outcome: 'pass' \| 'fail' \| 'timeout', finalScore, elapsedSec }`
`xr-webmcp-capture` | `xr-extras` (WebMCP) | Fired when `capture_xr_view` captures a frame (`data-xr-webmcp-indicator="active"`).
--------------------------------------------------------------------------------
## 6. Simulation Plugins (`xr-live-sim`)
Load `xr-live-sim.min.js` (global `XRSim`, included in `xr-live-full.min.js`) or
import `xr-live/sim`. Systems step at a fixed 60 Hz (at most 5 steps per frame);
at most 128 spawned instances exist per scene.
Trait clamping (defaults in parentheses):
- **`character`:** `speed` 0.1–20 (4), `runMultiplier` 1–4 (1.6),
`jumpVelocity` 0–30 (5.5), `gravity` 0–50 (15), `radius` 0.05–5 (0.35),
`height` ≥ 2 × radius (1.4), `cameraMode` (`third-person`), `cameraDistance`
0.5–50 (3.5), `cameraHeight` −10–50 (1.8).
- **`agent`:** `mode` (`wander`), `speed` 0.1–20 (2), `turnSpeed` 0.1–50 (6),
`sightRadius` 0–100 (6), `stopDistance` 0.01–10 (0.6), `separationRadius`
0–10 (0.8), up to 32 `waypoints`. `chase`, `flee`, and `follow` default
`target` to `'user'`.
- **`integrity`:** `max` 1–1e6 (100), `group` (`neutral`), `cooldownMs`
0–10000 (250), `hideOnDepleted` (true), `scoreValue`, `contactAmount` (0).
- **`zone`:** `shape` (`box`), `size` 0.01–1000 (1), `radius` (0.5), up to 16
`watchGroups` (empty watches every tracked node).
- **`spawner`:** `intervalSec` 0.05–3600 (3), `maxActive` 1–32 (8),
`totalCount` (0 = unlimited), `spawnRadius` 0–100 (0).
- **`objective`:** `type` (`pickup`), `amount` (10), `radius` 0.05–50 (0.65),
`order` 0–1000 (0), `respawnSec` 0–3600 (0). Only pickups spin and bob by
default.
- **`audioCues`:** event-to-cue map, `pitchJitter` 0–0.5 (0.08), `volume` 0–1
(1), `spatial` (true).
Host attributes: `character-controller`, `character-speed`, `character-jump`,
`touch-joystick`, `steering-agents`, `hit-zones`, `fire-rate`,
`projectile-speed`, `projectile-gravity`, `hit-amount`, `objectives`,
`scenario-timer`, `target-score`, `attempts`, `sim-hud`, `synth-audio`, and
`synth-volume`. `XRSim.resetSimScenario(element)` restarts the scenario.
================================================================================
SOURCE: skills/three-live-authoring/references/recipes.md
================================================================================
# Copy-Paste Recipes for `<three-live>` and `<xr-live>`
All 9 recipes below use features available in the repository today and work
without external build steps.
## Recipe 2: Spatial UI Card & Depth-Physics Ballpit (`<xr-live>`)
```html
<script type="module" src="/dist/xr-live-full.min.js"></script>
<xr-live
allow-client-scripts
xr-mode="ar"
xr-depth="occlusion-physics"
xr-hands="gestures"
xr-simulator="living-room"
sound-feedback
camera-controls
style="width: 100%; height: 600px; display: block;">
<script type="text/three">
const palette = [0x38bdf8, 0xf43f5e, 0x22c55e, 0xeab308, 0xa855f7];
for (let i = 0; i < 12; i++) {
const ball = new THREE.Mesh(
new THREE.SphereGeometry(0.08, 24, 24),
new THREE.MeshStandardMaterial({
color: palette[i % palette.length],
metalness: 0.2,
roughness: 0.25
})
);
ball.name = `ball-${i}`;
ball.position.set((i % 4 - 1.5) * 0.22, 1.3 + Math.floor(i / 4) * 0.22, -1.1);
ball.userData.xr = {
grab: '1-hand',
physics: 'dynamic',
restitution: 0.78
};
scene.add(ball);
}
const light = new THREE.DirectionalLight(0xffffff, 2.0);
light.position.set(1, 3, 2);
scene.add(light);
</script>
<div slot="spatial" data-position="0.55 1.25 -1.1" data-xr-traits="grab billboard">
<div style="padding: 16px; background: #0f172a; color: #f8fafc; border-radius: 14px; width: 220px;">
<h4 style="margin: 0 0 8px;">Physics Controls</h4>
<p style="margin: 0 0 12px; font-size: 13px;">Grab spheres or bounce them off real-world surfaces.</p>
<button onclick="document.querySelector('xr-live').transitionToVR({ durationMs: 400 })">
Toggle VR Environment
</button>
</div>
</div>
</xr-live>
```
---
## Recipe 3: Precision Manipulation Handles, Lazy Head-Follow HUD & Orbiting Satellites
```html
<script type="module" src="/dist/xr-live-full.min.js"></script>
<xr-live
allow-client-scripts
xr-simulator="office"
camera-controls
style="width: 100%; height: 560px; display: block;">
<script type="text/three">
// Centerpiece with 4 corner scale handles + pedestal yaw rotation bar
const sculpture = new THREE.Mesh(
new THREE.TorusKnotGeometry(0.22, 0.07, 64, 16),
new THREE.MeshStandardMaterial({ color: 0x6366f1, metalness: 0.7, roughness: 0.2 })
);
sculpture.name = 'sculpture';
sculpture.position.set(0, 0.9, -1.2);
sculpture.userData.xr = {
grab: '2-hand',
handles: { rotateAxis: 'y', minScale: 0.4, maxScale: 2.5 }
};
scene.add(sculpture);
// Satellite orbiting the named 'sculpture' node
const orbiter = new THREE.Mesh(
new THREE.SphereGeometry(0.05, 24, 24),
new THREE.MeshStandardMaterial({ color: 0x38bdf8, emissive: 0x0284c7 })
);
orbiter.name = 'orbiter';
orbiter.userData.xr = {
orbit: { target: 'sculpture', radius: 0.45, speed: 1.2, height: 0.08, faceCenter: true }
};
scene.add(orbiter);
// Lazy head-follow status pill with 18-degree angular deadzone
const hudPill = new THREE.Mesh(
new THREE.BoxGeometry(0.28, 0.08, 0.02),
new THREE.MeshStandardMaterial({ color: 0x0f172a, roughness: 0.4 })
);
hudPill.name = 'hud-pill';
hudPill.userData.xr = {
follow: {
mode: 'head',
lazy: true,
distance: 1.1,
deadzoneDeg: 18,
stiffness: 6,
height: -0.22,
faceUser: true
}
};
scene.add(hudPill);
</script>
</xr-live>
```
---
## Recipe 4: Multimodal Gemini Live Assistant + Hold-to-Ask + World Understanding
```html
<script type="module">
import '/dist/xr-live-full.min.js';
import { connectGeminiLive } from '/dist/xr-live-spatial.min.js';
const xr = document.getElementById('gemini-xr');
xr.addEventListener('xr-transcription', (e) => {
console.log(`[${e.detail.speaker}] ${e.detail.text}`);
});
xr.addEventListener('xr-object-detected', (e) => {
console.log(`Located ${e.detail.label} at`, e.detail.position);
});
// Connect only after user gesture (never on page load):
document.getElementById('connect-btn').addEventListener('click', () => {
connectGeminiLive(xr, {
url: 'wss://your-proxy.example.com/gemini-live',
frameIntervalMs: 1000,
streamAudio: true,
playAudio: true,
detectObjects: true
});
});
</script>
<button id="connect-btn">Start Gemini Live Session</button>
<xr-live
id="gemini-xr"
allow-client-scripts
world-understanding="planes reticle"
xr-depth="occlusion"
camera-controls
style="width: 100%; height: 560px; display: block;">
<script type="text/three">
const artifact = new THREE.Mesh(
new THREE.CylinderGeometry(0.18, 0.22, 0.4, 32),
new THREE.MeshStandardMaterial({ color: 0xf59e0b, metalness: 0.6, roughness: 0.3 })
);
artifact.name = 'turbine-core';
artifact.position.set(0, 1.0, -1.0);
artifact.userData.xr = {
grab: '1-hand',
holdToAsk: {
prompt: 'Explain the role of this turbine core and inspect its orientation.',
holdMs: 600
}
};
scene.add(artifact);
</script>
</xr-live>
```
---
## Recipe 5: 3D Spatial Paint, Depth Particles & Gaussian Splats
```html
<script type="module" src="/dist/xr-live-full.min.js"></script>
<xr-live
allow-client-scripts
spatial-paint="#38bdf8"
xr-depth="occlusion"
xr-simulator="tabletop"
camera-controls
style="width: 100%; height: 560px; display: block;">
<script type="text/three">
// Depth-colliding snow emitter volume
const snowVolume = new THREE.Mesh(
new THREE.BoxGeometry(0.01, 0.01, 0.01),
new THREE.MeshBasicMaterial({ color: 0xffffff })
);
snowVolume.name = 'snow-emitter';
snowVolume.position.set(0, 1.4, -1.2);
snowVolume.userData.xr = {
particles: { preset: 'snow', count: 500, area: [2.5, 2.0, 2.5], speed: 0.9 }
};
scene.add(snowVolume);
// 3D Gaussian Splat node (.splat or .spz)
const splatAnchor = new THREE.Group();
splatAnchor.name = 'garden-splat';
splatAnchor.position.set(0, 0.8, -1.2);
splatAnchor.userData.xr = {
grab: '2-hand',
splatUrl: 'https://example.com/assets/bonsai.spz'
};
scene.add(splatAnchor);
</script>
</xr-live>
```
---
## Recipe 6: Native Android (Kotlin) & iOS/visionOS (Swift) Integration
### Android (Google Filament + Jetpack XR SceneCore)
```kotlin
import com.google.threelive.ThreeLiveRuntimeSession
import com.google.threelive.XRLiveSceneCoreHelper
import com.google.threelive.filament.ThreeLiveFilament
// 1. Install the Filament renderer once at startup
ThreeLiveFilament.install()
// 2. Compile procedural Three.js and step XR traits / handles / particles natively
val session = ThreeLiveRuntimeSession.fromScript(scriptSource)
session.stepFrame(dtSeconds = 1f / 60f)
// 3. Or attach traits directly to an Android XR SceneCore entity from GLB bytes
XRLiveSceneCoreHelper.attachTraitsFromGlb(xrSession, rootEntity, glbByteBuffer)
```
### iOS & visionOS (Apple RealityKit)
```swift
import ThreeLiveCompiler
import RealityKit
@MainActor
func mountSpatialScene(script: String, in arView: ARView) async throws {
let entity = try await RealityKitHelper.loadEntity(from: script)
XRLiveRealityKit.installTraits(on: entity, in: arView)
}
```
--------------------------------------------------------------------------------
## Recipe 7: CAD Exploded View, Cross-Section Slicer, 3D Dimensions & Gaze Dwell
```html
<script type="module" src="/dist/xr-live-full.min.js"></script>
<xr-live
allow-client-scripts
explode="0.55"
explode-mode="axial-y"
section-plane="interactive"
spatial-measure="all"
measure-unit="mm"
gaze-dwell="auto"
dwell-time="950"
camera-controls
style="width: 100%; height: 560px; display: block;">
<script type="text/three">
const base = new THREE.Mesh(
new THREE.CylinderGeometry(0.26, 0.3, 0.14, 36),
new THREE.MeshStandardMaterial({ color: 0x334155, metalness: 0.8, roughness: 0.25 })
);
base.name = 'housing-base';
base.position.set(0, 0.75, -1.1);
base.userData.xr = {
explode: false,
dimensions: { unit: 'mm', axes: ['x', 'y', 'z'], color: '#38bdf8' }
};
scene.add(base);
const shroud = new THREE.Mesh(
new THREE.CylinderGeometry(0.24, 0.26, 0.14, 36),
new THREE.MeshStandardMaterial({ color: 0x38bdf8, metalness: 0.65, roughness: 0.2 })
);
shroud.name = 'upper-shroud';
shroud.position.set(0, 0.92, -1.1);
shroud.userData.xr = {
explode: { direction: [0, 1, 0], distance: 0.38, layer: 2, leaderLine: true },
gaze: { dwellMs: 900, action: 'select' }
};
scene.add(shroud);
scene.add(new THREE.DirectionalLight(0xffffff, 2.2));
</script>
</xr-live>
```
--------------------------------------------------------------------------------
## Recipe 8: Magnetic Socket Snapping, `#xr-state` URL Persistence & XR Locomotion
```html
<script type="module" src="/dist/xr-live-full.min.js"></script>
<xr-live
allow-client-scripts
magnetic-snap="all"
snap-grid="0.05"
snap-angle="15"
persistent-anchors="workbench"
persist-url-hash
locomotion="teleport snap-turn"
snap-turn-deg="30"
xr-simulator="office"
camera-controls
style="width: 100%; height: 560px; display: block;">
<script type="text/three">
const floor = new THREE.Mesh(
new THREE.PlaneGeometry(4, 4),
new THREE.MeshStandardMaterial({ color: 0x0f172a, roughness: 0.9 })
);
floor.name = 'walkable-floor';
floor.rotation.x = -Math.PI / 2;
floor.userData.xr = { navigable: 'walkable' };
scene.add(floor);
const socketPedestal = new THREE.Mesh(
new THREE.CylinderGeometry(0.16, 0.18, 0.04, 32),
new THREE.MeshStandardMaterial({ color: 0x1e293b })
);
socketPedestal.name = 'dock-socket';
socketPedestal.position.set(0, 0.8, -1.1);
socketPedestal.userData.xr = {
snapSocket: { tag: 'bay-a', radius: 0.18, alignRotation: true, highlightColor: '#22c55e' }
};
scene.add(socketPedestal);
const moduleCube = new THREE.Mesh(
new THREE.BoxGeometry(0.18, 0.18, 0.18),
new THREE.MeshStandardMaterial({ color: 0x6366f1, metalness: 0.5, roughness: 0.25 })
);
moduleCube.name = 'snap-module';
moduleCube.position.set(0.35, 0.9, -1.1);
moduleCube.userData.xr = {
grab: '1-hand',
snapTo: { socketTag: 'bay-a', radius: 0.18, grid: 0.05, angleDeg: 15 },
anchor: { id: 'snap-module', persist: true }
};
scene.add(moduleCube);
scene.add(new THREE.DirectionalLight(0xffffff, 2.0));
</script>
</xr-live>
```
--------------------------------------------------------------------------------
## Recipe 9: Zero-Backend Multi-User Co-Presence & Shared Spatial Paint (`shared-room`)
```html
<script type="module" src="/dist/xr-live-full.min.js"></script>
<xr-live
allow-client-scripts
shared-room="studio-room-1"
peer-name="Designer"
peer-color="#38bdf8"
spatial-paint="#38bdf8"
camera-controls
style="width: 100%; height: 560px; display: block;">
<script type="text/three">
const artifact = new THREE.Mesh(
new THREE.TorusKnotGeometry(0.18, 0.055, 64, 16),
new THREE.MeshStandardMaterial({ color: 0xa855f7, metalness: 0.65, roughness: 0.2 })
);
artifact.name = 'shared-artifact';
artifact.position.set(0, 1.0, -1.1);
artifact.userData.xr = {
grab: '2-hand',
handles: { rotateAxis: 'y', minScale: 0.4, maxScale: 2.5 },
sync: true
};
scene.add(artifact);
scene.add(new THREE.DirectionalLight(0xffffff, 2.2));
</script>
</xr-live>
```
--------------------------------------------------------------------------------
## Recipe 10: Timed Training Drill with a Safety Zone (`xr-live-sim`)
Ordered task steps, a countdown, and an alarm zone around a robot cell. Load
`xr-live-sim.min.js` (or use `xr-live-full.min.js`). Taps complete steps in
order; out-of-order taps play an `error` cue.
```html
<script src="/dist/xr-live-sim.min.js"></script>
<xr-live
id="drill"
allow-client-scripts
objectives
scenario-timer="60"
synth-audio
style="width: 100%; height: 520px; display: block;">
<script type="text/three">
const mat = (color) => new THREE.MeshStandardMaterial({ color, roughness: 0.4 });
// Ordered, required task steps (tap them in sequence)
['Step_1_Valve', 'Step_2_Filter', 'Step_3_Cap'].forEach((name, i) => {
const part = new THREE.Mesh(new THREE.BoxGeometry(0.2, 0.2, 0.2), mat(0x38bdf8));
part.name = name;
part.position.set(i * 0.4 - 0.4, 1.0, -1.0);
part.userData.xr = {
objective: { type: 'checkpoint', order: i + 1, required: true, amount: 10 },
audioCues: { onObjective: 'confirm' }
};
scene.add(part);
});
// Robot cell: alarm while the AGV is inside
const cell = new THREE.Mesh(
new THREE.BoxGeometry(1.2, 1, 1.2),
new THREE.MeshStandardMaterial({ color: 0xef4444, transparent: true, opacity: 0.15 })
);
cell.name = 'Robot_Cell';
cell.position.set(1.4, 0.5, -2);
cell.userData.xr = {
zone: { shape: 'box', size: [1.2, 1, 1.2], color: '#ef4444', watchGroups: ['agent'] },
audioCues: { onZoneEnter: 'alarm' }
};
scene.add(cell);
// AGV patrolling through the cell
const agv = new THREE.Mesh(new THREE.BoxGeometry(0.4, 0.2, 0.6), mat(0xf59e0b));
agv.name = 'AGV';
agv.position.set(-1.5, 0.1, -2);
agv.userData.xr = {
agent: { mode: 'patrol', speed: 0.8, waypoints: [[-1.5, 0.1, -2], [2.5, 0.1, -2]] }
};
scene.add(agv);
scene.add(new THREE.DirectionalLight(0xffffff, 2.2));
scene.add(new THREE.AmbientLight(0xffffff, 0.6));
</script>
</xr-live>
<script>
const drill = document.getElementById('drill');
drill.addEventListener('xrobjective', (e) => {
if (e.detail.outOfOrder) console.warn('Out of order:', e.detail.nodeId);
});
drill.addEventListener('xrzoneenter', (e) => {
console.log(e.detail.nodeId, 'entered', e.detail.zoneId);
});
drill.addEventListener('xrscenarioend', (e) => {
console.log('Outcome:', e.detail.outcome, 'in', e.detail.elapsedSec, 's');
});
</script>
```
--------------------------------------------------------------------------------
## Recipe 11: Procedural CAD Extrusion, CSG Booleans, 3D Text & GPU Instancing (`<three-live>`)
```html
<script type="module" src="/dist/three-live.min.js"></script>
<three-live
allow-client-scripts
camera-controls
auto-rotate
ar
tone-mapping="aces"
bloom="1.2"
outline="#38bdf8 0.025"
style="width: 100%; height: 520px; display: block;">
<script type="text/three">
const profile = new THREE.Shape();
profile.absarc(0, 0, 0.55, 0, Math.PI * 2, false);
const hole = new THREE.Path();
hole.absarc(0, 0, 0.22, 0, Math.PI * 2, true);
profile.holes.push(hole);
const steel = new THREE.MeshStandardMaterial({ color: 0x94a3b8, metalness: 0.78, roughness: 0.24 });
const rawFlange = new THREE.Mesh(
new THREE.ExtrudeGeometry(profile, { depth: 0.16, bevelEnabled: true, bevelThickness: 0.02, bevelSize: 0.015 }),
steel
);
rawFlange.rotation.x = -Math.PI / 2;
const slotCutter = new THREE.Mesh(new THREE.BoxGeometry(1.3, 0.14, 0.16), steel);
slotCutter.position.set(0, 0.12, 0);
const machined = THREE.CSG.subtract(rawFlange, slotCutter);
machined.name = 'machined-flange';
machined.userData.xr = { outline: { color: '#38bdf8', thickness: 0.02 } };
scene.add(machined);
const label = new THREE.Mesh(
new THREE.TextGeometry('CAD R1', { size: 0.14, depth: 0.035, bevelEnabled: true }),
new THREE.MeshStandardMaterial({ color: 0xf8fafc, emissive: 0x0ea5e9, emissiveIntensity: 0.45 })
);
label.position.set(-0.32, 0.28, 0);
scene.add(label);
const bolts = new THREE.InstancedMesh(
new THREE.CylinderGeometry(0.035, 0.035, 0.22, 16),
new THREE.MeshStandardMaterial({ color: 0xf59e0b, metalness: 0.9, roughness: 0.15 }),
12
);
const m = new THREE.Matrix4();
const p = new THREE.Vector3();
const q = new THREE.Quaternion();
const s = new THREE.Vector3(1, 1, 1);
for (let i = 0; i < 12; i++) {
const a = (i / 12) * Math.PI * 2;
p.set(Math.cos(a) * 0.39, 0.09, Math.sin(a) * 0.39);
m.compose(p, q, s);
bolts.setMatrixAt(i, m);
}
scene.add(bolts);
scene.add(new THREE.DirectionalLight(0xffffff, 2.4));
scene.add(new THREE.AmbientLight(0xffffff, 0.6));
</script>
</three-live>
```
--------------------------------------------------------------------------------
## Recipe 12: Articulated Robotic Arm with 2-Bone / FABRIK IK & Voice Signaling (`<xr-live>`)
```html
<script type="module" src="/dist/xr-live-full.min.js"></script>
<xr-live
allow-client-scripts
bloom="1.3"
outline="#38bdf8 0.025"
tone-mapping="agx"
shared-room="robotics-cell"
shared-room-voice
camera-controls
style="width: 100%; height: 540px; display: block;">
<script type="text/three">
const armMat = new THREE.MeshStandardMaterial({ color: 0x64748b, metalness: 0.8, roughness: 0.25 });
const shoulder = new THREE.Group();
shoulder.name = 'Shoulder';
shoulder.position.set(0, 0.75, -1.1);
const upper = new THREE.Mesh(new THREE.BoxGeometry(0.08, 0.08, 0.32), armMat);
upper.position.set(0, 0, 0.16);
shoulder.add(upper);
const forearm = new THREE.Group();
forearm.name = 'Forearm';
forearm.position.set(0, 0, 0.32);
const lower = new THREE.Mesh(new THREE.BoxGeometry(0.065, 0.065, 0.28), armMat);
lower.position.set(0, 0, 0.14);
forearm.add(lower);
shoulder.add(forearm);
const wrist = new THREE.Mesh(
new THREE.SphereGeometry(0.055, 20, 20),
new THREE.MeshStandardMaterial({ color: 0x38bdf8, emissive: 0x0284c7, emissiveIntensity: 0.6 })
);
wrist.name = 'WristEffector';
wrist.position.set(0, 0, 0.28);
wrist.userData.xr = {
ik: { root: 'Shoulder', mid: 'Forearm', effector: 'WristEffector', target: 'GripperTarget', pole: 'ElbowPole' }
};
forearm.add(wrist);
scene.add(shoulder);
const pole = new THREE.Group();
pole.name = 'ElbowPole';
pole.position.set(0, 1.35, -0.95);
scene.add(pole);
const target = new THREE.Mesh(
new THREE.SphereGeometry(0.065, 24, 24),
new THREE.MeshStandardMaterial({ color: 0x22c55e, emissive: 0x16a34a, emissiveIntensity: 0.8 })
);
target.name = 'GripperTarget';
target.position.set(0.22, 1.02, -0.78);
target.userData.xr = { grab: '1-hand', outline: { color: '#22c55e', thickness: 0.025 } };
scene.add(target);
scene.add(new THREE.DirectionalLight(0xffffff, 2.2));
</script>
</xr-live>
```
================================================================================
SOURCE: skills/three-live-authoring/references/debugging-and-csp.md
================================================================================
# Debugging, Sandboxing & CSP Guide (`<three-live>` & `<xr-live>`)
## 1. Why Nothing Renders (Top 4 Causes)
1. **Missing `allow-client-scripts` attribute:**
- **Symptom:** `<three-live>` or `<xr-live>` with an inline `<script type="text/three">` stays blank and dispatches an `error` event with `CLIENT_SCRIPTS_DISABLED_ERROR`.
- **Fix:** Add `allow-client-scripts` to the element (`<three-live allow-client-scripts ...>`) or set `ThreeLiveElement.allowClientScripts = true` before connecting.
2. **Bundler moved the Worker engine asset (`sandboxEngineSrc`):**
- **Symptom:** When bundling with Vite, Webpack, or Rollup instead of loading `/dist/three-live.min.js` via `<script>`, the sandbox Worker cannot auto-detect the IIFE bundle URL and fails closed.
- **Fix:** Explicitly point `ThreeLiveElement.sandboxEngineSrc` at the IIFE `.min.js` bundle:
```js
import { ThreeLiveElement } from 'three-live';
ThreeLiveElement.sandboxEngineSrc =
new URL('three-live/dist/three-live.min.js', import.meta.url).href;
```
3. **Strict Host CSP (`require-trusted-types-for 'script'` or blocked `blob:` Workers):**
- **Symptom:** The host page's Content Security Policy blocks Worker creation or script evaluation.
- **Fix:** Set `sandbox="iframe"` on the element and configure `ThreeLiveElement.sandboxIframeSrc` to point at `three-live-sandbox.html`, which runs inside an opaque-origin `<iframe sandbox="allow-scripts">`.
4. **Calling `new THREE.GLTFLoader()` or `fetch()` inside `<script type="text/three">`:**
- **Symptom:** External model does not appear because `fetch` and `XMLHttpRequest` are permanently revoked inside the sandbox Worker.
- **Fix:** Use `<three-live src="/path/to/model.glb">` or `<xr-anchor src="/path/to/model.glb">`.
## 3. Enforced Sandbox Quotas
Every procedural compilation (`toCanvas3D()`, `toGLB()`, `toUSDZ()`) runs `SceneQuotaValidator` with these defaults:
- **Max Script Size:** `1,048,576` bytes (`1 MiB`)
- **Max Meshes:** `1,000`
- **Max Vertices:** `100,000`
- **Max Indices:** `300,000`
- **Max Textures:** `16` (max dimension `2048×2048`)
- **Execution Watchdog:** `3,000 ms` wall-clock timeout (`250 ms` CPU ceiling inside Rust Boa compiler)
- **Max Concurrent Sandbox Workers:** `32`
--------------------------------------------------------------------------------
## 4. Network CSP & Plugin Security Quotas (`gemini-live`, `shared-room`, `persistent-anchors`)
- **Host `connect-src` for Live Relays:** The procedural
`<script type="text/three">` Worker always runs with network access revoked
(`connect-src 'none'`). Host-level plugins (`connectGeminiLive()` and
`shared-room="wss://..."`) connect from the main thread and enforce `wss://`
(or loopback `ws://localhost` / `ws://127.0.0.1` / `ws://[::1]`). If your
page serves a strict Content Security Policy and uses remote WebSocket
relays, allow your relay origin in `connect-src` (e.g.
`connect-src 'self' wss://your-relay.example.com`). Same-origin
`shared-room="room-id"` uses `BroadcastChannel` and needs no `connect-src`
allowance.
- **Persistence & Co-Presence Caps:**
- `persistent-anchors`: `MAX_PERSISTED_NODES = 128`,
`MAX_HASH_BYTES = 8192` (`#xr-state` tokens larger than `8 KB` or
containing prototype-pollution keys are rejected).
- `shared-room`: `MAX_PEERS = 16`, `MAX_PAYLOAD_BYTES = 16384` (`16 KB`),
`MAX_REMOTE_STROKES = 64`, `POSE_TICK_INTERVAL_MS = 50` (`20 Hz`),
`SOFT_LOCK_TTL_MS = 2000`.
--------------------------------------------------------------------------------
## 5. Closed-Loop CLI Diagnostics (`TL100–TL402`), Headless Contact Sheets & Scenario Simulation
Use `three-live-cli` to verify `.js`, `.html`, and `.glb` scenes without a
browser:
```bash
# 1. Validate syntax, schema, HTML attributes, and 3D spatial layout
three-live-cli validate scene.js --json
# 2. Render a 2x2 quad contact sheet (Camera, Front, Right, Top) with 1m grid & AABB labels
three-live-cli render scene.js -o preview.png --view=quad --grid --annotations
# 3. Render a 4-frame temporal filmstrip or animated GIF with KHR_interactivity onSelect trigger
three-live-cli render scene.js -o anim.png --frames=4 --duration=2.0 --trigger-select=crystal
# 4. Run a 60Hz deterministic scenario simulation
three-live-cli sim scene.js --duration=5.0 \
--actions='[{"at":0.2,"type":"select","target":"crystal"},{"at":0.5,"type":"moveCharacter","direction":[0,0,-1],"duration":1.5}]' \
--json
```
### Diagnostic Code Table (`TL100–TL402`)
Code | Severity | Condition | Actionable Fix
:------ | :-------- | :----------------------------------------------------------------- | :-------------
`TL100` | `error` | Syntax error, runtime exception, or quota exceeded | Check `line`/`column` in JSON output and fix JS syntax or reduce vertex/mesh count.
`TL101` | `warning` | `ShaderMaterial` / `RawShaderMaterial` fallback | Replace with `THREE.MeshStandardMaterial` or `THREE.MeshPhysicalMaterial`.
`TL102` | `warning` | Polyhedron/Tube geometry fallback | Use `SphereGeometry`, `CylinderGeometry`, `ExtrudeGeometry`, `LatheGeometry`, or `THREE.CSG`.
`TL103` | `warning` | `THREE.GLTFLoader` / `TextureLoader` / `fetch` | Load external `.glb` via `<three-live src="model.glb">` or `<xr-anchor src="model.glb">`.
`TL104` | `info` | `requestAnimationFrame` in static export | Use `userData.xr.orbit` or `userData.interactivity` so motion survives `.glb`/`.usdz` export.
`TL105` | `warning` | Unknown `THREE.*` constructor/method | Remove unsupported Three.js helper call.
`TL201` | `warning` | Unknown `userData.xr` or `userData.interactivity` key | Apply the Levenshtein suggestion in `suggestion` (e.g. `grabbable` -> `grab`).
`TL202` | `warning` | Out-of-range numeric trait clamped | Keep `gaze.dwellMs` in `[200, 10000]`, `bloom.strength` in `[0, 5]`, `outline.thickness` in `[0.002, 0.25]`.
`TL203` | `error` | Unsafe media/live/room URL or anchor ID | Use `https://` (or clean relative paths) for media, `wss://` for relays, and alphanumeric anchor IDs.
`TL204` | `error` | Broken target node reference | Ensure `orbit.target`, `ik.*`, `spawner.template`, and `interactivity.*.target` match an existing `mesh.name`.
`TL205` | `warning` | Unnamed interactive/spatial mesh | Set a unique `mesh.name = "..."` before attaching `userData.xr` or `userData.interactivity`.
`TL301` | `error` | Empty scene (`meshCount == 0`) | Call `scene.add(mesh)` with at least one non-empty geometry.
`TL302` | `warning` | Camera inside mesh AABB | Move camera outside `summary.worldBounds` or shrink the enclosing mesh.
`TL303` | `warning` | Mesh outside camera frustum or extreme framing | Reposition the mesh or adjust camera `position` / `lookAt` so `screenCoveragePct` is between `5%` and `85%`.
`TL304` | `warning` | Extreme scene scale (`< 0.01 m` or `> 200 m`) | Author dimensions in meters (`1 unit = 1 meter`); scale millimeter meshes by `0.001`.
`TL305` | `warning` | Unlit metallic PBR material | Add a `DirectionalLight` / `AmbientLight` or set `environment` on `<three-live>`.
`TL306` | `warning` | Co-located sibling meshes (`> 80%` overlap) | Offset sibling positions or use `THREE.CSG.subtract(a, b)` for boolean cutouts.
`TL307` | `info` | Floating or buried scene / character | Align `min.y` to `0.0` (`mesh.position.y = height / 2`) so AR/VR placement sits on the floor.
`TL401` | `error` | Inline `<script type="text/three">` without `allow-client-scripts` | Add `allow-client-scripts` to `<three-live>` or `<xr-live>`.
`TL402` | `warning` | `<xr-live>` feature used on `<three-live>` | Upgrade the tag from `<three-live>` to `<xr-live>` and load `xr-live.min.js`.
================================================================================
SOURCE: docs/three-live.md
================================================================================
# `<three-live>` Reference Guide
`<three-live>` is a lightweight WebGL2 3D viewer, procedural Three.js JIT
compiler, and `<model-viewer>` drop-in replacement with universal Augmented
Reality handoff (iOS Quick Look, Android Scene Viewer, and WebXR).
For interactive 6DoF WebXR spatial computing, declarative spatial traits
(`grab`, `scale`, `billboard`, `anchor`), spatial HTML cards (`<div
slot="spatial">`), depth/physics, and 25-joint hand tracking, see
**[`docs/xr-live.md`](xr-live.md)**. See **[`README.md`](../README.md)** for JS
bundles and entry points.
--------------------------------------------------------------------------------
## Quickstart
```html
<script type="module" src="/dist/three-live.min.js"></script>
<three-live
src="/models/chair.glb"
poster="/images/chair_poster.webp"
camera-controls
camera-orbit="30deg 20deg 3m"
selected-mesh="cushion"
ar
ar-modes="quick-look scene-viewer webxr"
auto-rotate
style="width: 100%; height: 500px; display: block;">
</three-live>
<script>
const viewer = document.querySelector('three-live');
// Listen for mesh part selection
viewer.addEventListener('meshclick', (e) => {
console.log('Selected part:', e.detail.meshName);
});
// Reset camera view
viewer.resetCamera();
</script>
```
--------------------------------------------------------------------------------
## Observed Attributes
| Attribute | Type | Bundle | Description |
| :--- | :--- | :--- | :--- |
| `src` | `string` | `three-live-viewer`+ | URL or path to binary `.glb` model. |
| `ios-src` | `string` | `three-live-viewer`+ | Optional URL or path to Apple `.usdz` model for iOS Quick Look (auto-transcoded from GLB if omitted). |
| `poster` | `string` | `three-live-viewer`+ | URL to a placeholder image shown before 3D rendering initializes. |
| `scale` | `string` | `three-live-viewer`+ | Uniform or per-axis scale multiplier `"x y z"` (e.g. `"2 2 2"`). |
| `camera-orbit` | `string` (str/JSON) | `three-live-viewer`+ | Initial camera orbit angles and distance (`"45deg 20deg 3m"` or `{"theta": 0.8, "phi": 0.35, "zoom": 3}`). |
| `camera-target` | `string` | `three-live-viewer`+ | Orbit center coordinate `"x y z"` (e.g. `"0m 1.5m 0m"`, defaults to model bounding center). |
| `min-camera-orbit` | `string` | `three-live-viewer`+ | Lower bounds for camera orbit `"theta phi radius"` (e.g. `"-45deg 0deg 1m"` or `"auto 0deg auto"`). |
| `max-camera-orbit` | `string` | `three-live-viewer`+ | Upper bounds for camera orbit (e.g. `"auto 90deg auto"` prevents orbiting beneath the ground plane). |
| `field-of-view` | `string` | `three-live-viewer`+ | Vertical field of view angle (e.g. `"45deg"`, clamps between `10deg` and `90deg`). |
| `selected-mesh` | `string` | `three-live-viewer`+ | Name of the sub-mesh to highlight. |
| `camera-controls` | `boolean` | `three-live-viewer`+ | Enables interactive orbit/pan/zoom camera controls (disabled by default unless `camera-controls` is present, matching `<model-viewer>`). |
| `ar` | `boolean` | `three-live-viewer`+ | Enables the augmented reality launcher button. |
| `ar-modes` | `string` | `three-live-viewer`+ | Space-delimited AR modes: `quick-look`, `scene-viewer`, `webxr`. |
| `ar-scale` | `string` | `three-live-viewer`+ | Controls whether the model can be scaled in AR: `"auto"` (default) or `"fixed"`. |
| `ar-placement` | `string` | `three-live-viewer`+ | Controls AR surface placement mode: `"floor"` (default) or `"wall"`. |
| `script-src` | `string` | `three-live`+ | URL or path to a server-generated procedural Three.js scene script. |
| `sandbox` | `string` | `three-live`+ | Mandatory sandbox mode (`"worker"` or `"iframe"`). Default: `"worker"`. |
| `allow-client-scripts` | `boolean` | `three-live`+ | Explicit opt-in allowing client-modified script strings via `executeScript()` or inline `<script>`. |
| `exposure` | `number` | `three-live-viewer`+ | Linear exposure applied to all lighting before tone mapping, like model-viewer (default: `1.0`). |
| `environment-image` | `string` | `three-live-studio` | Equirectangular `.hdr`/`.pic` radiance map or LDR environment image for ambient reflections and SH irradiance. |
| `skybox-image` | `string` | `three-live-studio` | Visible background equirectangular skybox image URL rendered behind the 3D scene. |
| `shadow-intensity` | `number` | `three-live`+ | Ground contact shadow opacity from `0.0` (off) to `1.0` (default: `0.7`). |
| `shadow-softness` | `number` | `three-live`+ | Radial Gaussian blur falloff of contact shadow from `0.0` (hard) to `1.0` (soft). |
| `tone-mapping` | `string` | `three-live-viewer`+ | Tone mapping mode: `"neutral"` (default, Khronos PBR Neutral), `"aces"` (ACES Filmic), `"agx"` (AgX Filmic), `"reinhard"`, or `"none"`. |
| `variant-name` | `string` | `three-live-studio` | Active `KHR_materials_variants` product material variant name (e.g. `"Midnight Black"`). Also readable/writable via `viewer.variantName` and `viewer.availableVariants`. |
| `bloom` | `boolean \| string` | `three-live-studio` | Enables HDR emissive bloom (`bloom`, `bloom="1.4"`, or `bloom="strength:1.2 threshold:0.75 radius:0.5"`). |
| `outline` | `boolean \| string` | `three-live-studio` | Enables silhouette outlines on `selected-mesh`, nodes with `userData.xr.outline`, or all foreground meshes (`outline`, `outline="#38bdf8"`, or `outline="#38bdf8 0.025"`). |
| `autoplay` | `boolean` | `three-live-viewer`+ | Automatically starts keyframe animation playback on model load. |
| `animation-name` | `string` | `three-live-viewer`+ | Name of specific glTF animation clip to play (defaults to first clip). |
| `auto-rotate` | `boolean` | `three-live-viewer`+ | Automatically rotates the camera around the model. |
--------------------------------------------------------------------------------
## `<model-viewer>` Drop-in Imperative API
`<three-live>` provides full drop-in compatibility for `<model-viewer>`
JavaScript configurators, hotspot managers, and AR launchers:
- **Camera & Screenshot:** `viewer.getCameraOrbit()`,
`viewer.getCameraTarget()`, `viewer.getFieldOfView()`,
`viewer.jumpCameraToGoal()`, `viewer.toDataURL()`, `await viewer.toBlob()`
- **AR Status & Activation:** `viewer.canActivateAR` (boolean getter), `await
viewer.activateAR()`
- **Surface Raycasting:** `viewer.positionAndNormalFromPoint(pixelX, pixelY)`
returns `{ position: { x, y, z, toString() }, normal: { x, y, z, toString()
} } | null`
- **Live Material Editing & `KHR_materials_variants`
(`viewer.model.materials`, `viewer.availableVariants`,
`viewer.variantName`):**
```js
// Switch a glTF KHR_materials_variants colorway (or USDZ VariantSet)
console.log(viewer.availableVariants); // ['Midnight Black', 'Titanium Silver']
viewer.variantName = 'Titanium Silver';
// Or edit PBR material factors directly
const mat = viewer.model.getMaterialByName('Leather') || viewer.model.materials[0];
mat.pbrMetallicRoughness.setBaseColorFactor([0.8, 0.2, 0.1, 1.0]);
mat.pbrMetallicRoughness.setRoughnessFactor(0.35);
```
--------------------------------------------------------------------------------
## DOM Events
| Event | Detail Payload | Description |
| :--- | :--- | :--- |
| `load` | `{ model: GLBData }` | Fired when the 3D model or procedural scene finishes loading. |
| `error` | `{ error: Error }` | Fired if model loading or script execution fails. |
| `camera-change` | `{ theta, phi, distance }` | Fired whenever the camera orbit changes. |
| `ar-status` | `{ status: 'session-started' \| 'not-presenting' \| 'failed', error? }` | Fired when AR session status transitions (`<model-viewer>` standard). |
| `meshclick` | `{ meshName, hitPoint, normal, distance }` | Fired when a user clicks or taps a sub-mesh. |
--------------------------------------------------------------------------------
## Custom AR Button Overlay (`slot="ar-button"` / `activateAR()`)
Provide a child element with `slot="ar-button"` to replace the default AR button
overlay with your own custom styled button. `<three-live>` automatically hides
its built-in button, manages visibility when `ar` is enabled, and triggers
`activateAR()` on click:
```html
<three-live src="/models/chair.glb" camera-controls ar style="width: 100%; height: 500px;">
<button slot="ar-button" class="custom-ar-btn">
✨ View in Your Space
</button>
</three-live>
```
--------------------------------------------------------------------------------
## Visual Extensions & Plugins
### 3D Hotspots & Annotations (`three-live/studio`)
Import `three-live/studio` (or use `three-live-full.min.js`) to attach
interactive HTML buttons or callouts pinned to 3D surface coordinates. Hotspots
automatically orbit with the camera and support surface normal occlusion
culling:
```html
<script type="module" src="/dist/three-live-full.min.js"></script>
<three-live src="/models/astronaut.glb" camera-controls camera-target="0 1.2 0" style="width: 100%; height: 500px; display: block;">
<button slot="hotspot-visor" data-position="0 1.5 0.3" data-normal="0 0 1" class="hotspot">
Gold-coated Visor
</button>
<div slot="hotspot-pack" data-position="0 1.3 -0.4" data-normal="0 0 -1" class="hotspot">
Life Support Unit
</div>
</three-live>
```
Update a hotspot dynamically at runtime using `viewer.updateHotspot({ name,
position, normal })`.
### PBR Rendering, Tone Mapping, Post-Processing & Lighting Extensions
- **Core WebGL2 PBR (`three-live-viewer`+):**
- **Screen-Space Derivative Normal Maps:** Perturbs normals via GLSL
`dFdx`/`dFdy` without requiring glTF vertex tangents.
- **5 Filmic & Commerce Tone Mapping Modes (`tone-mapping`):** Supports
`"neutral"` (default Khronos PBR Neutral for accurate e-commerce
colors), `"aces"` (Narkowicz ACES Filmic), `"agx"` (Sobotka AgX Filmic
with smooth highlight chroma desaturation), `"reinhard"`, and `"none"`.
- **GPU Instancing (`EXT_mesh_gpu_instancing` / `THREE.InstancedMesh`):**
Renders up to `MAX_INSTANCE_COUNT = 10,000` instances per mesh with
per-instance `MAT4` transforms and optional per-instance `COLOR_0` tints
(one `drawElementsInstanced` call per primitive, with per-instance
vertex attributes), exporting to glTF `EXT_mesh_gpu_instancing` and USDZ
`PointInstancer`.
- **HDR Emissive Bloom & Silhouette Outlines (`three-live/studio` /
`postprocessingPlugin`):**
- **Bloom (`bloom` attribute or `mesh.userData.xr.bloom`):** Dual-Kawase
MIP-chain HDR bloom and soft emissive halo generation
(`{ strength: 1.2, threshold: 0.8, radius: 0.5 }`).
- **Selection & Highlight Outlines (`outline` attribute or
`mesh.userData.xr.outline`):** Inverted-hull and screen-space Sobel
silhouette outlines (`{ color: '#38bdf8', thickness: 0.03 }`),
automatically highlighting `selected-mesh` or outlined nodes.
- **PBR Next Materials, HDRI & Contact Shadows (`three-live/studio`):**
- **glTF PBR Next Extensions:** Shader support for Clearcoat
(`KHR_materials_clearcoat`), Transmission
(`KHR_materials_transmission`), Dispersion (`KHR_materials_dispersion`),
Volume Attenuation (`KHR_materials_volume`), Iridescence
(`KHR_materials_iridescence`), and Sheen (`KHR_materials_sheen`).
- **HDRI Radiance Maps & Spherical Harmonics:** Set
`environment-image="/env/studio.hdr"` to parse Radiance `.hdr` RGBE
textures, prefilter GGX specular mipmaps, and project 9-coefficient
Spherical Harmonics diffuse irradiance.
- **Analytical Ground Contact Shadows:** Set `shadow-intensity="0.7"` and
`shadow-softness="0.5"` for soft ground ambient occlusion without shadow
map pass overhead.
### Animations: Rigid-Body TRS, Skeletal/Morph & Inverse Kinematics (IK)
- **Rigid-Body Keyframe Animations (`three-live-viewer`+):** Node translation,
rotation, and scale (`TRS`) keyframe tracks are built into all bundles
(`el.availableAnimations`, `el.play()`, `el.pause()`).
- **GPU Skeletal Skinning, Morph Targets & Cross-Fading
(`three-live/animation`):** Import `three-live/animation` (or use
`three-live-full.min.js`) to enable 64-joint GPU vertex skinning
(`JOINTS_0`/`WEIGHTS_0`), morph target blending (`weights`), and smooth clip
cross-fading (`el.crossFadeTo('Walk', 0.3)`).
- **Analytical 2-Bone & FABRIK Inverse Kinematics (`ikPlugin` /
`userData.xr.ik` / `THREE.IKSolver`):** Attach `mesh.userData.xr.ik = {
root: 'Shoulder', mid: 'Forearm', effector: 'Wrist', target: 'TargetNode',
pole: 'ElbowPole', solver: 'two-bone' | 'fabrik', iterations: 10 }` to solve
robotic arms, mechanical linkages, and character limbs at runtime across
Web, Android, and iOS, or call `THREE.IKSolver.solveTwoBone(...)` /
`THREE.IKSolver.solveFABRIK(...)` directly inside
`<script type="text/three">`.
### Declarative `KHR_interactivity` Behavior Graphs (`userData.interactivity`)
Inside `<script type="text/three">`, any `THREE.Mesh` or `THREE.Group` can
attach declarative `userData.interactivity` triggers (`onSelect`, `onHoverIn`,
`onHoverOut`, `onStart`, `onReceive`) that compile into standard Khronos
`KHR_interactivity` bytecode embedded inside exported `.glb` files and executed
by the `three-live/studio` VM on Web, Android, and iOS:
```html
<three-live allow-client-scripts camera-controls style="width: 100%; height: 480px; display: block;">
<script type="text/three">
const button = new THREE.Mesh(
new THREE.BoxGeometry(0.8, 0.3, 0.8),
new THREE.MeshStandardMaterial({ color: 0x38bdf8, metalness: 0.4, roughness: 0.3 })
);
button.name = ' interactive-box';
button.userData.interactivity = {
onSelect: [
{
action: 'toggleTransform',
property: 'translation',
from: [0, 0, 0],
to: [0, 0.4, 0],
duration: 0.35
},
{
action: 'cycleMaterialColor',
colors: [
[0.22, 0.74, 0.97, 1.0],
[0.13, 0.77, 0.37, 1.0],
[0.96, 0.25, 0.37, 1.0]
],
duration: 0.25
}
]
};
scene.add(button);
</script>
</three-live>
```
Supported action objects in `onSelect` / `onHoverIn` / `onHoverOut` / `onStart` / `onReceive[eventName]`:
- `toggleTransform`: `{ action: 'toggleTransform', property: 'translation' | 'rotation' | 'scale', from?: number[], to: number[], duration?: number }`
- `interpolate`: `{ action: 'interpolate', target?: string, property: 'translation' | 'rotation' | 'scale' | 'baseColorFactor' | 'metallicFactor' | 'roughnessFactor' | 'emissiveFactor', to: number | number[], duration?: number }`
- `setProperty`: `{ action: 'setProperty', target?: string, property: string, value: unknown }`
- `toggleVisibility`: `{ action: 'toggleVisibility', target?: string }`
- `cycleMaterialColor`: `{ action: 'cycleMaterialColor', colors: Array<[number, number, number, number?]>, duration?: number }`
- `playAnimation` / `stopAnimation`: `{ action: 'playAnimation', animation?: number }` / `{ action: 'stopAnimation' }`
- `sendEvent`: `{ action: 'sendEvent', event: string, values?: Record<string, unknown> }` (dispatches `interactivityEvent` on the host element).
### Built-in `EXT_meshopt_compression` & Pluggable Decoders
`EXT_meshopt_compression` and `KHR_meshopt_compression` (`ATTRIBUTES` vertex
codec v0/v1, `TRIANGLES` index codec, and `INDICES` index-sequence codec, plus
the `OCTAHEDRAL`, `QUATERNION`, `EXPONENTIAL`, and `COLOR` filters) are decoded
**out of the box** in pure TypeScript with zero WASM dependencies
(`decodeMeshoptBuffer` / `decodeMeshoptGLB` in
`three-live/src/plugins/meshopt.ts`). The decoders are byte-exact with the
reference meshoptimizer library, so `gltfpack` output loads directly, including
its `KHR_mesh_quantization` attributes (normalized / integer positions, normals,
UVs, skin weights, morph deltas and animation outputs) and the
`KHR_texture_transform` it uses to dequantize UVs. For Draco or KTX2 Basis
Universal textures, register external decoders on demand:
```javascript
import { ThreeLiveElement } from 'three-live';
ThreeLiveElement.registerDecoder('draco', myDracoDecoder);
```
--------------------------------------------------------------------------------
## Procedural Scene Scripting (`three-author`)
`<three-live>` supports two procedural scene compilation workflows:
1. **Server-Generated Scene Scripts (Default — `script-src` /
`loadScriptFromSrc(url)`):** The client passes a script URL directly to the
sandbox (`<three-live script-src="https://.../scene.js">`). The sandbox
Worker loads and compiles the script directly via `importScripts` without
requiring `'unsafe-eval'` or `new Function()`.
2. **Client-Modified Scene Scripts (Opt-In — `allow-client-scripts` /
`ThreeLiveElement.allowClientScripts = true`):** For interactive code
editors or client-modified scripts (`el.executeScript(code)` or inline
`<script type="text/three">`), explicitly opt in via the
`allow-client-scripts` attribute or `ThreeLiveElement.allowClientScripts =
true`:
```html
<three-live
allow-client-scripts
camera-controls
style="width: 100%; height: 500px; display: block;">
<script type="text/three">
const geo = new THREE.BoxGeometry(1, 1, 1);
const mat = new THREE.MeshStandardMaterial({ color: 0x4285f4 });
const mesh = new THREE.Mesh(geo, mat);
scene.add(mesh);
</script>
</three-live>
```
### Security & Sandboxing Architecture
Three-Live operates under a **fail-closed** security model for untrusted or
AI-generated script execution:
- **Isolated Web Worker (default, `sandbox="worker"`):** Scripts run in a
dedicated Worker with no DOM access. Before any author code runs, the worker
revokes its own ambient authority — `fetch`, `XMLHttpRequest`, `WebSocket`,
`EventSource`, `importScripts`, `navigator` (and therefore `sendBeacon`),
`indexedDB`, `caches`, `Worker`, `SharedWorker`, `BroadcastChannel`,
`SharedArrayBuffer` and `Atomics` are all made permanently `undefined`. The
channel back to the host is captured in a closure and `self.postMessage` is
replaced with a throwing stub, so a script cannot forge a result message.
- **Execution watchdog:** A script that exceeds
`SECURITY_3D_SANDBOX_TIMEOUT_MS` (3 s) has its worker terminated and raises
an `error` event. At most `SECURITY_3D_MAX_CONCURRENT_TASKS` (32) sandbox
workers may be alive at once.
- **Quota enforcement:** `Scene.toCanvas3D()`, `toGLB()` and `toUSDZ()` all
run `SceneQuotaValidator`, so no serialization path can skip the vertex,
index, mesh and texture ceilings. Scripts larger than
`SECURITY_3D_MAX_SCRIPT_BYTES` (1 MiB) are rejected before execution.
- **Fail-Closed Guarantee & Mandatory Sandboxing:** Sandboxing is strictly
required for all live procedural compilation and cannot be bypassed.
`sandbox="none"` and `sandbox="unsafe-host-realm"` are prohibited and
rejected with a `SecurityError`. If Web Workers or the sandbox iframe are
unavailable or blocked by Content Security Policy (CSP), Three-Live **fails
closed**: it halts execution, emits an `error` event, and throws rather than
evaluating code in the host realm.
- **Opaque-Origin Iframe + Worker Isolation (`sandbox="iframe"`):** For hosts
enforcing strict CSP (`require-trusted-types-for 'script'`, no
`'unsafe-eval'` on the host document), set `sandbox="iframe"` or configure
`ThreeLiveElement.sandboxIframeSrc` to point at `three-live-sandbox.html`.
Three-Live mounts a hidden `<iframe sandbox="allow-scripts">` (omitting
`allow-same-origin` so the iframe and its Worker run in a unique `null`
opaque origin) with its own `connect-src 'none'` CSP, completely isolating
script evaluation from the host page's origin, cookies, storage, and CSP.
#### Pointing the sandbox at the engine
The worker loads the Three-Live runtime by URL. When the library is included via
a classic `<script src="three-live.min.js">` tag this is auto-detected. If you
bundle Three-Live with webpack, Vite or Rollup, set it explicitly:
```js
import {ThreeLiveElement} from 'three-live';
ThreeLiveElement.sandboxEngineSrc =
new URL('three-live/dist/three-live.min.js', import.meta.url).href;
// Optional: enable opaque-origin <iframe sandbox="allow-scripts"> isolation
// across all <three-live> elements on strict-CSP pages:
ThreeLiveElement.sandboxIframeSrc =
new URL('three-live/dist/three-live-sandbox.html', import.meta.url).href;
```
The URL must point at an **IIFE** build (`*.min.js`), be same-origin or
CORS-enabled, and be permitted by your CSP `script-src`. If it cannot be
resolved, sandboxed execution fails closed rather than falling back to the host
realm. This mirrors how Monaco, PDF.js and Tesseract.js require their worker
assets to be located.
### Supported Procedural Scripting Subset
- **Math & Core:** `Vector2`, `Vector3`, `Vector4`, `Euler`, `Quaternion`,
`Matrix3`, `Matrix4`, `Color`, `MathUtils`, `Clock`, `IKSolver`
(`solveTwoBone`, `solveFABRIK`).
- **Primitives, CAD & Text Geometries:** `BoxGeometry`, `PlaneGeometry`,
`SphereGeometry`, `CylinderGeometry`, `ConeGeometry`, `TorusGeometry`,
`TorusKnotGeometry`, `LatheGeometry`, `TubeGeometry`, `Shape`, `Path`,
`ShapeGeometry`, `ExtrudeGeometry` (with holes and multi-segment bevels),
`TextGeometry` (built-in 3D extruded vector font), `CSG` (`THREE.CSG.union`,
`THREE.CSG.subtract`, `THREE.CSG.intersect`), `BufferGeometry`,
`BufferAttribute`.
- **Materials & Variants:** `MeshStandardMaterial`, `MeshBasicMaterial`,
`MeshLambertMaterial`, `MeshPhongMaterial`, `MeshPhysicalMaterial` (mapped
to glTF PBR metallic-roughness + PBR Next extensions, plus
`mesh.userData.variants` for `KHR_materials_variants`).
- **Scene Graph & GPU Instancing:** `Scene`, `Group`, `Mesh`, `InstancedMesh`
(`setMatrixAt`, `getMatrixAt`, `setColorAt`, `getColorAt`, `count`),
`Object3D` (`position`, `rotation`, `scale`, `matrix`, `add`, `remove`,
`traverse`).
- **Lights & Cameras:** `AmbientLight`, `DirectionalLight`, `PointLight`,
`Camera`, `PerspectiveCamera`.
- **Permissive LLM Fallbacks & Warnings:** Unsupported Three.js shaders
(`ShaderMaterial`, `RawShaderMaterial`), polyhedra/extended geometries
(`IcosahedronGeometry`, `OctahedronGeometry`, `CapsuleGeometry`,
`RingGeometry`, `CircleGeometry`), lights (`SpotLight`, `HemisphereLight`,
`RectAreaLight`), and headless browser globals (`requestAnimationFrame`,
`window`, `document`) gracefully degrade to standard PBR/primitive
equivalents without crashing the 3D/XR scene. All fallback events emit a
deduplicated warning and can be inspected via `THREE.getAuthorWarnings()`.
- **Scene Quotas:** Max **1,000 meshes**, **100,000 vertices**, **300,000
indices**, **16 textures** (max 2048×2048), and a **5-second** wall-clock
execution watchdog.
================================================================================
SOURCE: docs/xr-live.md
================================================================================
# `<xr-live>` spatial computing and plugins guide
`<xr-live>` extends `<three-live>` into a full spatial computing runtime for
**WebXR (Android XR, Meta Quest, and Apple Vision Pro)**, **mobile web AR**, and
**native Android (`Google Filament` and Jetpack XR `SceneCore`) and iOS or
visionOS (`Apple ARKit` and `RealityKit`)**.
It brings declarative spatial traits (`userData.xr`), 14 optional **spatial
plugins** for advanced features (`head-follow`, `manipulation-handles`,
`world-understanding`, `gemini-live`, `spatial-paint`, `depth-particles`,
`splat-loader`, `spatial-measure`, `magnetic-snap`, `gaze-dwell`,
`exploded-view`, `persistent-anchors`, `spatial-locomotion`, and `shared-room`),
interactive HTML and CSS spatial cards (`<div slot="spatial">`), real-world
depth occlusion and heightfield physics, 25-joint hand gesture recognition, a
desktop 6DoF simulator, WebMCP browser AI tools, and zero-webview native mobile
AR and XR runtimes.
For standard 3D viewing, `<model-viewer>` drop-in compatibility,
`KHR_interactivity`, and PBR materials, see
[`docs/three-live.md`](three-live.md). For end-to-end recipes, see
[`docs/cookbook.md`](cookbook.md).
--------------------------------------------------------------------------------
## Why `<xr-live>` pairs WebXR with native Filament and RealityKit runtimes
When building spatial features for phones, tablets, and headsets, relying on
browser WebXR (`navigator.xr`) alone has three practical limitations:
* **iOS Safari, `WKWebView`, and Android System `WebView` don't support
WebXR:** You can't start a WebXR session in Safari on iPhone or iPad, or
inside an embedded `WKWebView` (iOS) or `android.webkit.WebView` (Android)
in a native mobile app. Handing a static `.usdz` or `.glb` file to
`AR Quick Look` or `Scene Viewer` displays the geometry, but loses
JavaScript `animate()` loops, `KHR_interactivity` behavior graphs, and
interactive spatial plugins.
* **Browser WebXR doesn't use native spatial compositors:** In Chrome on
Android, WebXR renders WebGL2 in the browser's GPU process rather than using
Google Filament or Jetpack XR SceneCore's `HomeSpace` shared space. On
visionOS, browser WebXR is restricted to a fullscreen VR session without
camera passthrough or SwiftUI Shared Space integration.
* **Bundle size on the web:** Shipping every XR subsystem in a single bundle
adds unnecessary weight when a page only needs basic model viewing or one or
two spatial tools.
`<xr-live>` addresses both web and native apps from a single scene definition:
* **On the web (`<xr-live>`):** Starts with small core bundles (`< 51 KB gz`
for `xr-live-viewer.min.js`, or `< 72 KB gz` for `xr-live.min.js` with live
Three.js authoring) and optional plugins for advanced features (`~2–5 KB gz`
each, or `< 57 KB gz` for all 14 combined).
* **In native Android, iOS, and visionOS apps (`XRLiveRuntimeSession` and
`XRLiveRealityKitSession`):** The in-process Rust compiler
(`ThreeLiveCompiler`) compiles `<script type="text/three">` on-device,
serializes normalized `userData.xr` traits into `.glb` (`extras.xr` and
`ANDROID_xr_scenecore`) and `.usdz` (`customData.xr`), and drives **Google
Filament and Jetpack XR SceneCore** on Android and **Apple ARKit and
RealityKit** on iOS and visionOS at 60 to 120 Hz—so interactive AR,
`animate()` loops, `KHR_interactivity`, and all 14 spatial plugins work
natively on iOS and Android without a `WebView`.
--------------------------------------------------------------------------------
## Quickstart
### Load the all-in-one bundle in HTML (`xr-live-full.min.js`)
```html
<script type="module" src="/dist/xr-live-full.min.js"></script>
<xr-live
allow-client-scripts
xr-mode="ar"
xr-depth="occlusion-physics"
xr-hands="gestures"
xr-simulator="living-room"
world-understanding="planes reticle"
spatial-paint="#38bdf8"
sound-feedback
camera-controls
style="width: 100%; height: 600px; display: block;">
<!-- Procedural 3D nodes with canonical userData.xr traits -->
<script type="text/three">
const sculpture = new THREE.Mesh(
new THREE.TorusKnotGeometry(0.18, 0.06, 64, 16),
new THREE.MeshStandardMaterial({ color: 0x6366f1, metalness: 0.7, roughness: 0.2 })
);
sculpture.name = 'sculpture';
sculpture.position.set(0, 1.0, -1.1);
sculpture.userData.xr = {
grab: '2-hand',
handles: { rotateAxis: 'y', minScale: 0.4, maxScale: 2.5 },
holdToAsk: { prompt: 'Describe this sculpture.', holdMs: 600 }
};
scene.add(sculpture);
</script>
<!-- Live HTML and CSS spatial UI card rendered inside the 3D and XR scene -->
<div slot="spatial" data-position="0.45 1.1 -1.1" data-xr-traits="grab billboard">
<div style="padding: 16px; background: #0f172a; color: #f8fafc; border-radius: 16px; width: 240px;">
<h3>Spatial control card</h3>
<p>Pinch to grab, drag corner handles to scale, or pinch in empty space to paint.</p>
<button id="buy-btn">Inspect in AR</button>
</div>
</div>
</xr-live>
```
### Register optional plugins for advanced features
When bundling with Vite, Rollup, esbuild, or Webpack, you can pair the
`<xr-live>` core (`three-live/xr-live` at `< 72 KB gz`, or
`three-live/xr-live-viewer` at `< 51 KB gz`) with only the specific plugins your
application uses:
```ts
import { XRLiveElement } from 'three-live/xr-live';
import { registerXRTraitSerializer } from 'three-live/src/xr-trait-schema';
import { spatialMeasurePlugin } from 'xr-live/src/plugins/spatial-measure';
import { magneticSnapPlugin } from 'xr-live/src/plugins/magnetic-snap';
import { explodedViewPlugin } from 'xr-live/src/plugins/exploded-view';
// Register only the plugins your scene uses (~2-5 KB gz each)
registerXRTraitSerializer();
XRLiveElement.registerPlugin(spatialMeasurePlugin);
XRLiveElement.registerPlugin(magneticSnapPlugin);
XRLiveElement.registerPlugin(explodedViewPlugin);
```
To load all 14 spatial plugins without a bundler, import
`/dist/xr-live-spatial.min.js` (`< 57 KB gz`) or `/dist/xr-live-full.min.js`
(`< 188 KB gz`, which also includes `xr-live-sensors` and the 6 `xr-live-sim`
simulation plugins).
--------------------------------------------------------------------------------
## Observed attributes (`<xr-live>` and `<xr-anchor>`)
`<xr-live>` inherits all
[`<three-live>` attributes](three-live.md#observed-attributes) and adds the
spatial attributes listed in the following table:
| Element | Attribute | Type and values | Default | Description |
| :------------ | :-------------------- | :------------------ | :---------- | :------------------------------- |
| `<xr-live>` | `xr-mode` | `'ar' \| 'vr' \| 'inline' \| 'auto'` | `'auto'` | Target WebXR session mode (`ar` requests `immersive-ar`; `vr` requests `immersive-vr`). |
| `<xr-live>` | `xr-depth` | `'occlusion' \| 'occlusion-physics' \| 'mesh' \| 'none'` | `'none'` | Enables WebXR `depth-sensing`, PBR depth occlusion, and analytical heightfield physics. |
| `<xr-live>` | `xr-hands` | `'gestures' \| 'joints' \| 'pinch' \| 'none'` | `'none'` | Enables 25-joint `XRHand` tracking (`hand-tracking`) and gesture classification. |
| `<xr-live>` | `xr-simulator` | `'living-room' \| 'office' \| 'tabletop' \| 'none'` | `'none'` | Enables the desktop 6DoF keyboard and mouse simulator (`WASD`, `Q` and `E`, mouse look, and virtual hands). |
| `<xr-live>` | `xr-profile` | `'quest3' \| 'android-xr' \| 'visionos' \| 'auto'` | `'auto'` | Target spatial hardware profile hint. |
| `<xr-live>` | `sound-feedback` | `boolean` | `false` | Plays procedural Web Audio earcons (`grab`, `release`, `bounce`, `hover`) and mobile haptics. |
| `<xr-live>` | `world-understanding` | `'' \| 'planes' \| 'planes reticle' \| 'false'` | — | Enables `xr-live-world-understanding` plane and mesh detection and the gaze surface reticle. |
| `<xr-live>` | `spatial-paint` | `'' \| '#rrggbb' \| 'false'` | — | Enables `xr-live-spatial-paint`; pinching in empty space extrudes 3D Catmull-Rom tube strokes. |
| `<xr-live>` | `spatial-measure` | `'' \| 'bounds' \| 'tape' \| 'all' \| 'none'` | — | Enables `xr-live-spatial-measure` 3D bounding-box dimension callouts and AR point-to-point tape measure. |
| `<xr-live>` | `measure-unit` | `'auto' \| 'm' \| 'cm' \| 'mm' \| 'in' \| 'ft-in'` | `'auto'` | Default measurement display unit for `spatial-measure`. |
| `<xr-live>` | `magnetic-snap` | `'' \| 'grid' \| 'surface' \| 'sockets' \| 'all' \| 'none'` | — | Enables `xr-live-magnetic-snap` socket, Cartesian grid, surface, and yaw-angle snapping. |
| `<xr-live>` | `snap-grid` | `number` or `'gx gy gz'` (meters) | — | Default Cartesian snap grid spacing in meters (for example, `'0.05'` or `'0.05 0.05 0.05'`). |
| `<xr-live>` | `snap-angle` | `number` (`0..180` degrees) | `0` | Default yaw rotation snap increment in degrees. |
| `<xr-live>` | `gaze-dwell` | `'' \| 'auto' \| 'false' \| 'none'` | — | Enables `xr-live-gaze-dwell` hands-free eye and head gaze reticle and radial dwell selection. |
| `<xr-live>` | `dwell-time` | `number` (`250..10000` ms) | `1200` | Default gaze dwell duration in milliseconds before triggering selection. |
| `<xr-live>` | `explode` | `number` (`0..1`) | `0` | Normalized multi-part assembly explosion factor (`xr-live-exploded-view`). |
| `<xr-live>` | `explode-mode` | `'radial' \| 'axial-x' \| 'axial-y' \| 'axial-z' \| 'hierarchical'` | `'radial'` | Automatic explosion vector strategy when a part omits `userData.xr.explode.direction`. |
| `<xr-live>` | `explode-scale` | `number` | `1.0` | Global distance multiplier applied to per-part explosion offsets. |
| `<xr-live>` | `section-plane` | `'none' \| 'x:<d>' \| 'y:<d>' \| 'z:<d>' \| '-x:<d>' \| '-y:<d>' \| '-z:<d>' \| 'interactive'` | `'none'` | Real-time GPU cross-section clipping plane in scene (model) space (`d` in meters; the cut follows the model through AR placement and turntable rotation) with optional draggable gizmo frame. |
| `<xr-live>` | `persistent-anchors` | `''` or `string` (namespace) | — | Enables `xr-live-persistent-anchors` cross-session transform persistence (`xr-live:anchors:<ns>`). |
| `<xr-live>` | `persist-url-hash` | `boolean` | `false` | Syncs persisted node poses into `#xr-state=<base64url>` (`<= 8 KB`) for zero-backend link sharing. |
| `<xr-live>` | `locomotion` | `'' \| 'teleport' \| 'snap-turn' \| 'smooth' \| 'teleport snap-turn' \| 'none'` | — | Enables `xr-live-spatial-locomotion` parabolic teleportation, thumbstick snap-turn, or smooth glide. |
| `<xr-live>` | `snap-turn-deg` | `number` (degrees) | `30` | Horizontal thumbstick snap-turn angle in degrees. |
| `<xr-live>` | `comfort-vignette` | `boolean` | `true` | Renders a `120ms` peripheral FOV comfort vignette during teleports and snap-turns. |
| `<xr-live>` | `shared-room` | `'<room-id>'` or `'wss://...'` | — | Enables `xr-live-shared-room` multi-user co-presence via `BroadcastChannel` or WebSocket relay. |
| `<xr-live>` | `shared-room-voice` | `boolean` | `false` | Opts in to voice **signaling** in `shared-room`: relays WebRTC offer/answer/ICE messages (`peerrtcsignal` event, `broadcastRtcSignal()`) and computes per-peer `voiceGain` (`computeSpatialVoiceGain`). Your app owns `getUserMedia`, `RTCPeerConnection`, and audio playback. |
| `<xr-live>` | `peer-name` | `string` (max `32` chars) | — | Local peer display name broadcast to room participants. |
| `<xr-live>` | `peer-color` | `'#rrggbb'` | `'#38bdf8'` | Local peer 3D visor and laser ray tint color in `shared-room`. |
| `<xr-live>` | `bloom` | `boolean \| string` | — | Enables HDR emissive bloom (`bloom`, `bloom="1.4"`, or `bloom="strength:1.2 threshold:0.75 radius:0.5"`). |
| `<xr-live>` | `outline` | `boolean \| string` | — | Enables silhouette outlines on `selected-mesh` or nodes with `userData.xr.outline` (`outline="#38bdf8 0.03"`). |
| `<xr-live>` | `variant-name` | `string` | — | Active `KHR_materials_variants` product material variant name. |
| `<xr-live>` | `webmcp` | `boolean` | `false` | Registers spatial AI tools (`inspect_xr_scene`, `place_xr_object_at_gaze`, `capture_xr_view`) on `navigator.modelContext`. |
| `<xr-live>` | `webmcp-allow-depth` | `boolean` | `false` | Opt-in gate required before `capture_xr_view` exposes the 8×8 metric `depthGridMeters` array. |
| `<xr-live>` | `webmcp-omit-pose` | `boolean` | `false` | Omits `headPosition` from `inspect_xr_scene` responses. |
| `<xr-anchor>` | `surface` | `'floor' \| 'wall' \| 'table' \| 'ceiling'` | `'floor'` | Target semantic surface for declarative child placement. |
| `<xr-anchor>` | `magnetic` | `boolean` | `false` | Continuously snaps the child anchor to the nearest matching surface plane. |
--------------------------------------------------------------------------------
## Canonical `userData.xr` spatial traits
Defined in
[`three-live/src/xr-trait-schema.ts`](../three-live/src/xr-trait-schema.ts),
`mesh.userData.xr` traits are normalized, clamped to identical ranges, and
exported into `.glb` (`nodes[i].extras.xr` and
`nodes[i].extensions.ANDROID_xr_scenecore`) and `.usdz`
(`customData = {dictionary xr = {...}}`) so scenes behave identically on Web,
Android, and iOS or visionOS. The following table describes each trait key:
| Trait key | Type and options | Description |
| :----------------------- | :----------------- | :--------------------------- |
| **`grab`** | `'1-hand' \| '2-hand' \| true` | 6DoF single-hand grab and rotate (`'1-hand'`) or two-hand grab, pinch-scale, and twist (`'2-hand'`). Legacy `grabbable: true` is also accepted. |
| **`scale`** | `{ min?: number, max?: number } \| true` | Two-handed pinch scaling clamped to `[min, max]` (defaults to `{ min: 0.05, max: 5 }`). |
| **`billboard`** | `'capsule' \| 'cylindrical' \| 'spherical' \| true` | Orients the node toward the viewer (`'capsule'` and `'cylindrical'` rotate around the Y axis; `'spherical'` faces the camera in 3D). |
| **`follow`** | `'head' \| 'hand-left' \| 'hand-right'` **or** `{ mode, lazy, distance, deadzoneDeg, stiffness, height, faceUser, snapOnStart }` | String values apply an eager spring follow; object values activate lazy head or hand follow with an angular deadzone (`deadzoneDeg`, default `15°`) and a critically damped spring (`stiffness`, default `6`). |
| **`anchor`** | `'floor' \| 'wall' \| 'ceiling' \| 'table' \| 'horizontal' \| 'vertical'` **or** `{ target?, id?, persist? }` | Pins the node to a detected real-world plane or simulator surface; setting `persist: true` (or `id`) persists user-moved transforms across reloads and `#xr-state` links. |
| **`physics`** / **`rigidBody`** | `'dynamic' \| 'static' \| 'kinematic'` **or** `{ body?, shape?: 'auto' \| 'box' \| 'sphere' \| 'cylinder', mass?, restitution?, friction?, linearDamping?, angularDamping?, gravityScale?, velocity?, angularVelocity?, lockRotation? }` | Steps 60 Hz deterministic Sequential Impulse rigid-body physics (gravity, restitution bounce, friction, stacking, sleeping, character push, projectile knockback, and depth heightfield collisions) and exports USDZ `PhysicsRigidBodyAPI` / `PhysicsMassAPI`. |
| **`orbit`** | `{ target: [x,y,z] \| string, radius?: number, speed?: number, height?: number, phase?: number, faceCenter?: boolean }` | Orbits around a 3D world point or another mesh's `name` at `speed` rad/s. |
| **`handles`** | `{ rotateAxis?: 'y' \| 'none', minScale?: number, maxScale?: number } \| true` | Adds four top-corner uniform scale handles and a bottom pedestal yaw-rotation bar; emits `xr-handles-change`. |
| **`dimensions`** | `{ enabled?: boolean, unit?: 'auto' \| 'm' \| 'cm' \| 'mm' \| 'in' \| 'ft-in', axes?: Array<'x' \| 'y' \| 'z'>, color?: string, showOnSelect?: boolean } \| true` | Renders live `W × H × D` bounding-box dimension guide bars and camera-facing vector-glyph metric or imperial labels (`spatial-measure`). |
| **`snapTo`** | `{ grid?: number \| [x,y,z], angleDeg?: number, surface?: 'none' \| 'floor' \| 'wall' \| 'ceiling' \| 'any', socketTag?: string, radius?: number } \| true` | Snaps grabbed meshes to matching `snapSocket` anchors (with `1.25×` breakaway hysteresis), Cartesian grids, surfaces, and yaw increments (`magnetic-snap`). |
| **`snapSocket`** | `string` **or** `{ tag: string, radius?: number, alignRotation?: boolean, highlightColor?: string }` | Marks a node as a magnetic snap target socket with a pulsing proximity highlight ring (`magnetic-snap`). |
| **`gaze`** | `{ enabled?: boolean, dwellMs?: number, repeat?: boolean, action?: 'select' \| 'ask' } \| true` | Opts a mesh into hands-free eye and head gaze dwell (`dwellMs` in `[250, 10000]`, default `1200`), firing `meshclick` and `KHR_interactivity` (`'select'`) or Gemini Live (`'ask'`). |
| **`explode`** | `{ enabled?: boolean, direction?: [x,y,z], distance?: number, layer?: number, leaderLine?: boolean } \| boolean` | Configures per-part explosion vector, distance (`0..10m`), staggered `layer` (`1..10`), and dashed leader lines (`exploded-view`); `false` pins a base part in place. |
| **`navigable`** | `boolean \| 'walkable' \| 'blocked'` | Marks floor and platform meshes as valid (`'walkable'`) or forbidden (`'blocked'`) landing targets for parabolic XR teleportation (`spatial-locomotion`). |
| **`sync`** | `boolean` | Controls whether user manipulations on a node synchronize across peers in `shared-room` (defaults to `true` on grabbable nodes; set `false` to keep local-only). |
| **`particles`** | `{ preset: 'rain' \| 'snow' \| 'sparks' \| 'confetti', count?: number, area?: number \| [x,y,z], speed?: number }` | Turns the node into a particle emitter volume (`count` up to `2,000`) that collides with the floor and WebXR depth surfaces. |
| **`holdToAsk`** | `{ prompt?: string, holdMs?: number } \| true` | Press-and-hold on the mesh starts speech recognition and sends a contextual prompt to Gemini Live; emits `xr-hold-to-ask`. |
| **`worldUnderstanding`** | `{ planes?: boolean, meshes?: boolean, reticle?: boolean, objects?: boolean }` | Opts the scene into WebXR plane and mesh detection and the gaze surface reticle (`__xr_reticle__`). |
| **`splatUrl`** | `string` (`https://` or relative `.splat` or `.spz` URL) | Loads and back-to-front depth-sorts 3D Gaussian Splats (`.splat` or Niantic `.spz` v2 and v3 with anisotropic 3D covariance and SH degree 1–3, up to `1,000,000` splats). |
| **`splat`** | `string` **or** `{ url: string, shDegree?: 0 \| 1 \| 2 \| 3, maxSplats?: number }` | Structured 3D Gaussian Splat trait controlling Spherical Harmonics evaluation degree (`0..3`) and splat cap. |
| **`bloom`** | `{ strength?: number, threshold?: number, radius?: number } \| boolean \| number` | Enables HDR emissive bloom on the scene/node (`strength` in `[0, 5]`, `threshold` in `[0, 2]`, `radius` in `[0, 1]`). |
| **`outline`** | `{ color?: string, thickness?: number } \| boolean \| string` | Renders an inverted-hull / screen-space silhouette outline (`thickness` in `[0.002, 0.25]`) around the node. |
| **`ik`** | `{ root: string, mid?: string, effector: string, target: string \| [x, y, z], pole?: string \| [x, y, z], solver?: 'two-bone' \| 'fabrik', iterations?: number, tolerance?: number, weight?: number }` | Solves analytical 2-bone (`root` → `mid` → `effector`) or FABRIK inverse kinematics toward `target` (and optional `pole`, alias `poleTarget`) every frame. |
| **`text`** | `{ text: string, size?: number, depth?: number, bevelEnabled?: boolean, bevelThickness?: number, bevelSize?: number }` | Extruded 3D vector text metadata exported to glTF `extras.xr.text` and USDZ `customData.xr`. |
| **`audio`** | `string` **or** `{ src: string, refDistance?: number, loop?: boolean }` | Attaches a 3D `PannerNode` HRTF audio source to the mesh. |
| **`pointerEvents`** | `'none'` | Excludes the mesh from raycast picking so rays pass through it. |
--------------------------------------------------------------------------------
## Spatial plugin suite
All 14 spatial plugins are included in `xr-live-full.min.js` and
`xr-live-spatial.min.js`, or can be imported individually from
`xr-live/src/plugins/<name>` and registered via
`XRLiveElement.registerPlugin(plugin)`. Every plugin has a corresponding native
implementation in Android (`mobile/android/src/main/java/com/google/threelive/`)
and iOS or visionOS (`mobile/ios/Sources/ThreeLiveCompiler/`):
* **`head-follow` (`xr-live-head-follow`):** Implements `FollowHead` lazy HUD
behavior (angular deadzone plus critically damped spring
`stepCriticallyDampedSpring`) and `Orbit` satellite motion
(`userData.xr.orbit`).
* **`manipulation-handles` (`xr-live-manipulation-handles`):** Attaches four
top-corner uniform-scale handles and a bottom pedestal yaw-rotation bar to
any mesh with `userData.xr.handles`. Dispatches `xr-handles-change`
(`{ mesh, scale, yawDeg }`).
* **`world-understanding` (`xr-live-world-understanding`):** Tracks WebXR
`detectedPlanes` (`MAX_TRACKED_PLANES = 64`) and `detectedMeshes`, updates a
surface-normal gaze reticle (`__xr_reticle__`), provides `XRTapeMeasure`,
and unprojects 2D bounding boxes (`[ymin, xmin, ymax, xmax]`) into 3D world
space via `detectObjects(element, detections)`, dispatching
`xr-object-detected`.
* **`gemini-live` (`xr-live-gemini-live`):** Streams viewport frames and 16
kHz PCM microphone audio to a Gemini Live `wss://` endpoint
(`connectGeminiLive(element, options)`), plays 24 kHz PCM model audio with
barge-in cancellation (`xr-gemini-interrupted`), emits live
`xr-transcription` (`{ speaker, text, isFinal }`), answers `report_objects`
tool calls with 3D unprojection, and wires `userData.xr.holdToAsk`
press-and-hold queries (`xr-hold-to-ask-listening`, `xr-hold-to-ask`).
* **`spatial-paint` (`xr-live-spatial-paint`):** Enabled via the
`spatial-paint` attribute (for example, `spatial-paint="#ff3366"`). Pinching
in empty space smooths input points with a uniform Catmull-Rom spline and
extrudes twist-free parallel-transport tubes (`TubeStrokeBuilder`), emitting
`xr-paint-stroke` (`{ index, points, lengthMeters }`). Supports programmatic
`undo()` and `clear()` via `getSpatialPaintController(element)`.
* **`depth-particles` (`xr-live-depth-particles`):** Simulates `rain`, `snow`,
`sparks`, and `confetti` emitters (`userData.xr.particles`, up to
`MAX_EMITTERS = 8` and `2,000` particles per emitter) with a deterministic
PRNG and real-world WebXR depth collision (`isOccludedByDepth`).
Browser depth particles use shared static cross-quads when the viewer supports
instance offsets. Position uploads are 12 bytes per particle update instead of
96. Older viewers retain the expanded geometry fallback. Existing mesh-instance
matrices and colors keep their current rendering path.
* **`splat-loader` (`xr-live-splat-loader`):** Streams `.splat` (32-byte
records) and Niantic `.spz` (v2 and v3, raw or gzipped) 3D Gaussian Splat
files for any node with `userData.xr.splatUrl` or `userData.xr.splat`,
reconstructing full 3D covariance matrices (`computeSplatCovariance3D`),
projecting EWA 2D elliptical splat axes (`projectSplatCovariance2D`),
evaluating view-dependent Spherical Harmonics degrees 1–3
(`evaluateSplatSH`), and sorting splats back-to-front with a 16-bit counting
sort with cached unchanged depth orders. Moving views refresh every fourth
draw cycle; parallel stereo eyes with identical depth rows share one GPU
order, while divergent eyes retain independent orders (at most two).
Spherical-harmonic color refresh remains independent of index uploads.
* **`spatial-measure` (`xr-live-spatial-measure`):** Renders live `W × H × D`
bounding-box dimension callout bars and procedural 3D vector-glyph labels
(`userData.xr.dimensions` or `spatial-measure="bounds"`), plus an
interactive point-to-point AR and 3D tape measure (`spatial-measure="tape"`
or `"all"`, up to `MAX_TAPE_SEGMENTS = 16`) that emits `measurecomplete` and
clears via double-pinch or `element.clearMeasurements()`.
* **`magnetic-snap` (`xr-live-magnetic-snap`):** Provides declarative socket
snapping (`userData.xr.snapSocket` and `userData.xr.snapTo.socketTag`) with
`1.25×` breakaway hysteresis and a pulsing preview ring (`1.5×` radius),
plus Cartesian grid (`snap-grid`), rotational yaw (`snap-angle`), and
real-world or simulator surface (`floor`, `wall`, `ceiling`, or `any`)
quantization. Dispatches `xrsnap` and `xrunsnap`.
* **`gaze-dwell` (`xr-live-gaze-dwell`):** Hands-free eye and head gaze
raycasting for Apple Vision Pro, Android XR, and accessibility (`gaze-dwell`
or `userData.xr.gaze`). Applies EMA ray smoothing (`alpha = 0.25`), a `1.5°`
micro-saccade grace cone, a `150ms` grace delay, and a constant-angular-size
surface-normal progress ring, dispatching `gazeenter`, `gazeleave`, and
`gazedwell` (routing `'select'` to `meshclick` and `KHR_interactivity`, or
`'ask'` to `gemini-live`).
* **`exploded-view` (`xr-live-exploded-view`):** Explodes multi-part CAD and
engineering assemblies (`explode="0..1"`, `explode-mode`,
`userData.xr.explode`) with staggered per-layer timing (`layer: 1..10`),
spring-smoothed interpolation, dashed leader lines, two-hand XR spread
gestures, and a real-time GPU fragment cross-section clipping slicer
(`section-plane="x:0"` or `"interactive"`). Dispatches `explodechange`.
* **`persistent-anchors` (`xr-live-persistent-anchors`):** Restores and
auto-saves user-manipulated transforms (`persistent-anchors` or
`userData.xr.anchor.persist`) across a three-tier pipeline:
`#xr-state=<base64url>` shareable URL hashes (`persist-url-hash`, capped at
`MAX_HASH_BYTES = 8192`), namespaced `localStorage`
(`MAX_PERSISTED_NODES = 128`), and WebXR `XRAnchor` persistent handles.
Dispatches `anchorsrestored`.
* **`spatial-locomotion` (`xr-live-spatial-locomotion`):** Comfortable
room-scale and architectural XR navigation
(`locomotion="teleport snap-turn"` or `userData.xr.navigable`) with a
ballistic parabolic teleport arc and cyan or red landing ring,
hysteresis-guarded thumbstick snap-turns (`snap-turn-deg`, default `30°`),
optional smooth glide, and a `120ms` peripheral FOV comfort vignette
(`comfort-vignette`). Dispatches `xrteleport`.
* **`shared-room` (`xr-live-shared-room`):** Zero-dependency multi-user
co-presence (`shared-room="<room-id>"` via same-origin
`BroadcastChannel("xr-live:room:<id>")` or cross-device `wss://` relay,
`MAX_PEERS = 16`, `MAX_PAYLOAD_BYTES = 16384`). Broadcasts `20 Hz` quantized
head poses and tinted 3D peer visor pills and laser rays, enforces `2000ms`
optimistic soft-lock ownership leases on grabbed nodes (local grabs on a
node another peer holds are refused; near-simultaneous grabs resolve to the
lowest `peerId` on every peer), synchronizes `exploded-view` factors and
`spatial-paint` 3D tube strokes (`MAX_REMOTE_STROKES = 64`), and supports
opt-in WebRTC voice **signaling** (`shared-room-voice`: relays
offer/answer/ICE via `broadcastRtcSignal()` / `peerrtcsignal` and computes
per-peer `voiceGain` + stereo pan with `computeSpatialVoiceGain`; your app
creates the `RTCPeerConnection`s and plays the audio, and signaling is
visible to every room member). Dispatches `peerjoin`, `peerleave`,
`peersync`, and `peergrabconflict`.
--------------------------------------------------------------------------------
## Simulation plugins (`xr-live-sim`)
The optional `xr-live-sim` bundle adds six plugins for interactive simulations:
games, training drills, safety walkthroughs, and robot-cell or factory-floor
rehearsals. Each plugin reads `userData.xr` traits from your scene and
`<xr-live>` attributes from the host element, so a scene needs no per-frame
code.
Load the bundle after the core, or use `xr-live-full.min.js`, which includes it:
```html
<script src="/dist/xr-live.min.js"></script>
<script src="/dist/xr-live-sim.min.js"></script> <!-- < 26 KB gz, global XRSim -->
```
With a bundler, import `xr-live/sim`. Importing the module registers all six
plugins.
| Plugin | What it does |
| :--------------------- | :-------------------------------------------------- |
| `character-controller` | Moves the first `character` node with WASD or arrow keys, a gamepad, or a touch joystick. Jump with Space, sprint with Shift. Slides along `collidable` nodes, follows with the camera, and switches `idle`, `walk`, `run`, and `jump` clips. |
| `steering-agents` | Moves `agent` nodes with `patrol`, `chase`, `flee`, `follow`, or `wander` steering, with separation and wall push-out. |
| `rigid-body` | Steps `rigidBody` / `physics` nodes (`dynamic`, `static`, `kinematic`) at 60 Hz with a deterministic Sequential Impulse solver (box, sphere, and cylinder colliders, restitution, friction, stacking, sleeping, character push, and projectile knockback). |
| `hit-zones` | Tracks `integrity`, fires projectiles or hitscan rays, applies contact amounts, emits zone enter and exit events, and runs `spawner` nodes. |
| `objectives` | Tracks pickups, checkpoints, keys, time bonuses, and restores, plus score, attempts, a countdown, and the pass, fail, or timeout outcome. |
| `synth-audio` | Plays procedural cues (no audio files) mapped per node by the `audioCues` trait. |
### Simulation traits
Add these keys to `userData.xr` on any mesh or group:
| Trait | Shape |
| :---------- | :------------------------------------------------- |
| `character` | `true`, a camera mode, or `{ speed, runMultiplier, jumpVelocity, gravity, radius, height, cameraMode, cameraDistance, cameraHeight, clips: { idle, walk, run, jump } }`. Camera modes: `third-person`, `first-person`, `top-down`, `fixed`. |
| `agent` | A mode string or `{ mode, target, speed, turnSpeed, sightRadius, stopDistance, separationRadius, waypoints, loopWaypoints }`. `target` is a node name or `'user'` (the character, else the viewer). |
| `rigidBody` / `physics` | `'dynamic' \| 'static' \| 'kinematic' \| true` or `{ body, shape: 'auto' \| 'box' \| 'sphere' \| 'cylinder', mass, restitution, friction, linearDamping, angularDamping, gravityScale, velocity, angularVelocity, lockRotation }`. |
| `integrity` | A max number or `{ max, current, group, showBar, cooldownMs, hideOnDepleted, scoreValue, contactAmount }`. Hits skip nodes in the shooter's own `group`. |
| `zone` | `'box'`, `'sphere'`, or `{ shape, size, radius, watchGroups, color }`. |
| `spawner` | A template node name or `{ template, intervalSec, maxActive, totalCount, spawnRadius }`. The template node is hidden. |
| `objective` | A type or `{ type, amount, tag, radius, required, order, spinSpeed, bobAmplitude, respawnSec }`. Types: `pickup`, `checkpoint`, `key`, `time-bonus`, `restore`. `order > 0` enforces ascending order. |
| `audioCues` | `{ onTap, onObjective, onHit, onDepleted, onZoneEnter, onZoneExit, onSpawn, onJump, onLand, pitchJitter, volume, spatial }`. Cue names: `confirm`, `error`, `alarm`, `warning`, `pickup`, `jump`, `beam`, `impact`, `burst`, `powerup`, `step`, `pass`, `fail`. |
| `collidable` | `true` makes the node a wall or floor for characters and agents. |
`audioCues` is separate from the file-based `audio` trait. Characters and agents
face local +Z.
### Simulation attributes
| Attribute | Values | Default |
| :--------------------- | :----------------------- | :----------------------- |
| `character-controller` | `third-person`, `first-person`, `top-down`, `fixed-camera`, `none` | trait value |
| `character-speed` | meters per second | trait value |
| `character-jump` | jump velocity in meters per second | trait value |
| `touch-joystick` | `auto` (touch screens), `always`, `none` | `auto` |
| `steering-agents` | `false` or `none` pauses agents | on |
| `hit-zones` | `projectile` (or empty), `hitscan`, `contact` | off |
| `fire-rate` | shots per second | `4` |
| `projectile-speed` | meters per second | `14` |
| `projectile-gravity` | meters per second squared | `0` |
| `hit-amount` | integrity removed per hit | `25` |
| `objectives` | present enables; `none` disables | on with objective traits |
| `scenario-timer` | countdown in seconds | none |
| `target-score` | score that passes the scenario | none |
| `attempts` | lives before `fail` | unlimited |
| `sim-hud` | `auto`, `minimal`, `none` | `auto` with `objectives` |
| `synth-audio` | present enables; `muted` disables | off |
| `synth-volume` | `0` to `1` | `0.7` |
To restart a scenario, call `XRSim.resetSimScenario(element)` (or import
`resetSimScenario` from `xr-live/sim`). The Aether view restarts when its
`scenarioResetKey` prop changes.
### Simulation events
All events bubble from the host element:
| Event | `detail` |
| :----------------- | :----------------------------------------------------- |
| `xrcollision` | `{ nodeId, otherId, normal, impactSpeed }` |
| `xrsleep` | `{ nodeId }` |
| `characterjump` | `{ nodeId, position }` |
| `characterland` | `{ nodeId, fallSpeed }` |
| `agentstatechange` | `{ nodeId, state, targetId }` |
| `agentreach` | `{ agentId, targetId, position }` |
| `xrhit` | `{ targetId, sourceId, amount, remaining, point }` |
| `xrdepleted` | `{ targetId, group, scoreValue }` |
| `xrzoneenter` | `{ zoneId, nodeId, group }` |
| `xrzoneexit` | `{ zoneId, nodeId, group }` |
| `xrspawn` | `{ spawnerId, instanceId, index }` |
| `xrobjective` | `{ nodeId, tag, type, amount, totalScore, remainingRequired, outOfOrder }` |
| `xrscorechange` | `{ score, delta, attemptsLeft, timeRemaining }` |
| `xrscenarioend` | `{ outcome: 'pass' \| 'fail' \| 'timeout', finalScore, elapsedSec }` |
### Example: a timed training drill
```js
const scene = new THREE.Scene();
['Valve', 'Filter', 'Cap'].forEach((name, i) => {
const part = new THREE.Mesh(
new THREE.BoxGeometry(0.2, 0.2, 0.2),
new THREE.MeshStandardMaterial({ color: 0x38bdf8 })
);
part.name = name;
part.position.set(i * 0.4 - 0.4, 1, 0);
part.userData.xr = {
objective: { type: 'checkpoint', order: i + 1, required: true },
audioCues: { onObjective: 'confirm' }
};
scene.add(part);
});
return scene;
```
```html
<xr-live objectives scenario-timer="45" synth-audio>
<script type="text/three">/* the scene above */</script>
</xr-live>
```
Taps complete steps in order; an out-of-order tap plays `error`. Completing all
required steps ends the scenario with `pass`, and the timer ends it with
`timeout`.
### Native runtimes
The Android (`XRSimWorld.kt`) and iOS or visionOS (`XRSimWorld.swift`) runtimes
step the same systems at a fixed 60 Hz and match the web runtime on two shared
golden scenes in `testdata/golden/`: `sim_scene.json` (movement, agents,
contacts, zones, and scoring) and `sim_visuals_scene.json` (projectiles, bursts,
spawns, integrity bars, hit flashes, zone outlines, and the follow camera).
The shared core also decides what to draw, so every platform shows the same
feedback:
* `CharacterController.cameraRig` gives the follow-camera target, polar angle,
and distance (`null` for `fixed-camera` and during XR sessions).
* `HitZones.visuals` lists hit flashes, integrity bars, projectiles, burst
fragments, and colored zone outlines.
On Android, `XRSimRenderBridge` draws both through the renderer; the Filament
renderer shows spawned instances as gltfio instances that share the template's
GPU buffers and textures. On iOS and visionOS, `XRSimRealityKitBridge` draws
them with RealityKit and clones the template entity for spawned instances. The
Aether `ThreeLive` view wires these up, together with an on-screen joystick,
jump and fire buttons, and the score HUD.
--------------------------------------------------------------------------------
## Spatial DOM cards (`<div slot="spatial">`)
Any child element with `slot="spatial"` is projected into 3D space as an
interactive textured quad inside WebXR and inline 3D views:
* **Sanitized SVG `<foreignObject>` rasterization:** Renders live HTML and CSS
into a crisp GPU texture with automatic XSS hardening (strips dangerous tags
and handlers, blocks `javascript:` and `data:text/html` URIs, and escapes
XML entities).
* **3D raycast hit-testing:** Controller rays, hand pinch rays, and pointer
clicks compute UV intersection coordinates and dispatch synthetic `click`,
`pointerdown`, `pointermove`, and `pointerup` events to the underlying DOM
elements (for example, `<button>`).
* **Trait support:** Add `data-xr-traits="grab billboard"` and
`data-position="x y z"` directly on `<div slot="spatial">`.
--------------------------------------------------------------------------------
## Imperative methods and DOM events
### Imperative methods
* `await element.enterSpatialAR(sourceOverride?, forceFallback?)`: Launches
WebXR `immersive-ar`, iOS `AR Quick Look` (`.usdz`), or Android
`Scene Viewer` (`intent://`).
* `element.transitionToVR({ color?: [r, g, b], durationMs?: number })`:
Smoothly fades the viewport background from transparent AR passthrough to an
opaque VR environment.
* `element.transitionToAR({ durationMs?: number })`: Smoothly fades the
viewport background back to transparent AR passthrough.
* `await element.captureViewFrame({ format?, quality?, includeDepth? })`:
Captures the rendered viewport (`dataUrl`, `base64`), camera `viewMatrix`
and `projectionMatrix`, `headPosition`, and optional 8×8 `depthGridMeters`.
* `element.stepSpatialFrame(dtSeconds)`: Steps all spatial subsystems
(automatically called during WebXR and simulator frames).
* `element.clearMeasurements()`: Clears all active and locked 3D tape-measure
segments (`spatial-measure`).
* `element.setExplode(factor)`: Smoothly animates the assembly explosion
factor (`0..1`) and emits `explodechange` (`exploded-view`).
* `element.savePersistedAnchors()` and `element.clearPersistedAnchors()`:
Saves or clears persisted anchor poses across `localStorage`, `#xr-state`,
and WebXR handles (`persistent-anchors`).
* `element.exportAnchorState()` and `element.importAnchorState(token)`:
Exports or imports a compact base64url layout state token
(`persistent-anchors`).
* `element.teleportTo([x, y, z])`: Teleports the viewer rig to `[x, y, z]` and
emits `xrteleport` (`spatial-locomotion`).
* `element.getSharedPeers()`: Returns the list of active remote peers
(`RemotePeerState[]`) in `shared-room`.
### Spatial DOM events
The `<xr-live>` element dispatches the custom events listed in the following
table:
| Event | `event.detail` payload | Description |
| :------------------------- | :--------------- | :------------------------------------ |
| `xr-select-start`, `xr-select-move`, `xr-select-end` | `{ hand: 'left' \| 'right' \| 'screen', pivotPosition, quaternion, rayOrigin, rayDirection, hit }` | Fired on controller trigger, hand pinch, or screen touch ray. |
| `xr-handles-change` | `{ mesh, scale: number, yawDeg: number }` | Fired when a `userData.xr.handles` corner scale handle or yaw bar is dragged. |
| `xr-object-detected` | `{ label: string, position: [number, number, number], distanceMeters: number, source: 'plane' \| 'depth' \| 'fallback' }` | Fired for each 2D detection unprojected into 3D by `world-understanding`. |
| `xr-transcription` | `{ speaker: 'user' \| 'model', text: string, isFinal: boolean }` | Fired as Gemini Live user and model speech transcripts arrive. |
| `xr-hold-to-ask-listening` | `{ mesh: string }` | Fired when a press-and-hold on a `holdToAsk` mesh begins listening. |
| `xr-hold-to-ask` | `{ mesh: string, transcript: string, prompt: string }` | Fired when a `holdToAsk` voice or text query is dispatched to Gemini Live. |
| `xr-gemini-interrupted` | `{}` | Fired when user speech barges in and cancels Gemini Live audio playback. |
| `xr-paint-stroke` | `{ index: number, points: number, lengthMeters: number }` | Fired when a 3D `spatial-paint` tube stroke completes. |
| `measurecomplete` | `{ start: [x,y,z], end: [x,y,z], distanceMeters: number, formatted: string }` | Fired when a `spatial-measure` point-to-point tape segment is locked. |
| `xrsnap` | `{ node, socket, position: [x,y,z], quaternion: [x,y,z,w], mode: 'socket' \| 'grid' \| 'surface' \| 'angle' }` | Fired when `magnetic-snap` snaps a grabbed or released mesh. |
| `xrunsnap` | `{ node, socket }` | Fired when a mesh is pulled past the `1.25×` breakaway radius of a `snapSocket`. |
| `gazeenter`, `gazeleave` | `{ node, point }` or `{ node }` | Fired when the smoothed `gaze-dwell` ray enters or leaves a gazeable mesh. |
| `gazedwell` | `{ node, point: [x,y,z], action: 'select' \| 'ask' }` | Fired when radial gaze dwell reaches `100%`. |
| `explodechange` | `{ factor: number, source: 'attribute' \| 'gesture' \| 'api' \| 'peer' }` | Fired when `exploded-view` updates the assembly explosion factor. |
| `anchorsrestored` | `{ count: number, source: 'xr-anchor' \| 'storage' \| 'url-hash' }` | Fired when `persistent-anchors` restores saved node poses. |
| `xrteleport` | `{ from: [x,y,z], to: [x,y,z], yawDeg: number }` | Fired when `spatial-locomotion` teleports the viewer rig. |
| `peerjoin`, `peerleave` | `{ peerId: string, name: string, color: string }` | Fired when a remote participant joins, disconnects, or times out of `shared-room`. |
| `peergrabconflict` | `{ peerId: string, nodeId: string }` | Fired when a remote participant wins a near-simultaneous grab of the same node and the local hold is released. |
| `peersync` | `{ peerId: string, nodeId: string }` | Fired when `shared-room` applies a remote node transform, explode factor (`__explode__`), or paint stroke (`paint:<id>`). |
| `xr-webmcp-capture` | `{}` | Fired when a WebMCP agent invokes `capture_xr_view` (sets `data-xr-webmcp-indicator="active"`). |
--------------------------------------------------------------------------------
## Native mobile spatial bridges (no WebViews required)
`xr-live` compiles scenes into self-contained `.glb` and `.usdz` binaries with
embedded spatial trait metadata (`extras.xr`, `ANDROID_xr_scenecore`, and USDZ
`customData.xr`), allowing native Android and iOS or visionOS apps to run full
6DoF spatial experiences natively:
### Android (`mobile/android/src/main/java/com/google/threelive/`)
* **[`XRLiveRuntimeSession.kt`](../mobile/android/src/main/java/com/google/threelive/XRLiveRuntimeSession.kt),
[`XRSpatialFeatures.kt`](../mobile/android/src/main/java/com/google/threelive/XRSpatialFeatures.kt),
and modular feature controllers (`SpatialMeasure.kt`, `MagneticSnap.kt`,
`GazeDwell.kt`, `ExplodedView.kt`, `PersistentAnchors.kt`,
`SpatialLocomotion.kt`, `SharedRoom.kt`):** Executes lazy head-follow
springs, orbits, manipulation handles, Catmull-Rom spatial paint tubes,
depth-colliding particles, Gaussian splat sorting, 3D dimension callouts and
tape measure, magnetic socket and grid snapping, gaze dwell, exploded-view
assemblies and section clipping, persistent anchors, parabolic locomotion,
multi-user room sync, and `KHR_interactivity` behavior graphs natively on
Google Filament. In a shared room, start host grabs with
`XRSpatialFeatures.requestGrab(node)` (or check the return value of
`XRLiveRuntimeSession.setNodeGrabbed(node, true)`) and cancel the gesture
when it returns `false`: another peer holds the node. Near-simultaneous
grabs resolve like the web (lowest peer ID wins) and report
`Listener.onPeerGrabConflict`.
* **[`XRLiveSceneCoreHelper.kt`](../mobile/android/src/main/java/com/google/threelive/XRLiveSceneCoreHelper.kt):**
Jetpack XR SceneCore integration for Android XR headsets and phones,
attaching `MovableComponent`, `ResizableComponent`, `Anchor`, and billboard
updates directly from compiled `.glb` bytes.
* **[`XRGeminiLiveController.kt`](../mobile/android/src/main/java/com/google/threelive/gemini/XRGeminiLiveController.kt):**
Native Kotlin WebSocket and PCM audio streaming client for Gemini Live.
### iOS and visionOS (`mobile/ios/Sources/ThreeLiveCompiler/`)
* **[`XRLiveRealityKitSession.swift`](../mobile/ios/Sources/ThreeLiveCompiler/XRLiveRealityKitSession.swift),
[`XRLiveRealityKit.swift`](../mobile/ios/Sources/ThreeLiveCompiler/XRLiveRealityKit.swift),
[`XRSpatialRealityKitFeatures.swift`](../mobile/ios/Sources/ThreeLiveCompiler/XRSpatialRealityKitFeatures.swift),
and modular feature controllers (`SpatialMeasure.swift`,
`MagneticSnap.swift`, `GazeDwell.swift`, `ExplodedView.swift`,
`PersistentAnchors.swift`, `SpatialLocomotion.swift`, `SharedRoom.swift`):**
Native Swift and RealityKit runtime supporting all 14 spatial plugin
behaviors, `userData.xr` traits, manipulation handles, spatial paint tubes,
particle emitters, Gaussian splats, `KHR_interactivity`, and
`XRGeminiLiveController.swift`. In a shared room, start host grabs with
`XRSpatialRealityKitFeatures.requestGrab(_:)` and cancel the gesture when it
returns `false` (another peer holds the node); near-simultaneous grabs
resolve like the web and call `onPeerGrabConflict`.
### Gemini Live session ownership
Disconnect, remote closure and replacement revoke old capture, timer, transport,
transcription, audio and recognition callbacks. Teardown detaches old state before
external callbacks run. Reentrant replacement keeps the newer session and closes
superseded transports. Synchronous setup failures clean up the failed session.
Ordinary text turns retain the existing text-only wire format.