llms.txt · diff
git:20261005.c2bfddb to git:20261009.c6b0c13
27 added, 9 removed. Audit A to A.
# Helios Engine
> Framework-agnostic programmatic video creation for the web
Helios is a modern engine for creating videos programmatically using standard web technologies—HTML, CSS, and JavaScript. Unlike Remotion (React-only), Helios works with any framework or vanilla JS.
## Quick Facts
- **License**: Elastic License 2.0 (free commercial use, no SaaS resale)
- **Status**: Beta / Self-Driving (Core: Stable v5.13.0+, Studio: Alpha v0.107.1+, Player: v0.78.1+, Renderer: v1.78.3+, CLI: Beta v0.45.2+, Infrastructure: v0.25.0+)
- **Frameworks**: React, Vue, Svelte, Solid, vanilla JS
- **Animation**: Standard CSS, Web Animations API (WAAPI), any animation library
## Core API
```typescript
import { Helios } from '@helios-project/core';
// Define strict types for input props (Generic TInputProps) and structure
const helios = new Helios<{ title: string }>({
duration: 10, // seconds
fps: 30,
width: 1920,
height: 1080,
loop: false,
playbackRate: 1,
volume: 1,
muted: false,
autoSyncAnimations: true, // Auto-sync CSS/WAAPI
inputProps: { title: "Hello World" }, // Dynamic data via constructor
captions: [{ id: "1", start: 0, end: 1000, text: "Welcome" }], // SRT string or objects (milliseconds)
timeline: {
tracks: [
{
id: "main",
clips: [{ id: "clip1", source: "video.mp4", start: 0, duration: 300 }]
}
]
}
});
// Reactively subscribe to state changes
helios.subscribe(({ currentFrame, inputProps, availableAudioTracks, activeClips }) => {
// Drive animations with progress or props
// availableAudioTracks is AudioTrackMetadata[] (id, src, startTime, duration, fadeInDuration, fadeOutDuration, fadeEasing)
});
// CRITICAL: Ensure fonts, images, and media are ready before rendering
await helios.waitUntilStable();
helios.play();
```
## Packages
- `@helios-project/core` - Headless video engine (state, timeline, subscriptions)
- `@helios-project/player` - Web Component player (`<helios-player>`) and `ClientSideExporter`
- `@helios-project/renderer` - Server-side rendering (Playwright, FFmpeg)
- `@helios-project/studio` - Browser-based development environment (Alpha)
- - `@helios-project/cli` - Command-line interface (init, add, components, list, remove, update, build, preview, job, skills, merge, diff, deploy, studio, render)
+ - `@helios-project/cli` - Command-line interface (init, add, components, list, remove, update, build, preview, job, skills, merge, diff, deploy, studio, render, still, sheet, verify, check, analyze, mcp)
- `@helios-project/infrastructure` - Distributed rendering orchestration and cloud execution adapters
## Animation Approach
Helios uses native browser APIs, not proprietary hooks:
```css
/* Standard CSS animations work */
@keyframes fadeIn {
from { opacity: 0; }
to { opacity: 1; }
}
.element { animation: fadeIn 1s ease-out; }
```
The engine controls `document.timeline.currentTime` to seek animations frame-accurately.
## Architecture
### Rendering Paths
1. **Canvas-to-Video** (MVP): WebCodecs API for Three.js, Pixi.js, WebGL
2. **DOM-to-Video**: Playwright screenshots for HTML/CSS/SVG
### Components
- **Helios class**: Headless state machine (currentFrame, isPlaying, timeline, etc.)
- **Framework adapters**: React hook, Vue composable, Svelte store, Solid signal
- **Web Component**: `<helios-player>` for browser preview and client-side export
- **Renderer**: FFmpeg with piped frames (no temp files) and pluggable RenderExecutor
## vs Remotion
| | Helios | Remotion |
|---|--------|----------|
| Framework | Any | React only |
| Animation | CSS, WAAPI | `interpolate()`, `spring()` |
| Pricing | Free | $100+/mo for 4+ devs |
| Maturity | Beta | Production-ready |
| Distributed rendering | Local (Beta) / Cloud (GCP/AWS/Azure/Cloudflare/Fly.io/Docker/K8s/Modal/Deno Deploy/Vercel/Hetzner/Cloudflare Sandbox) | Available |
+ ## Making and Checking a Video (CLI)
+ A video page defines `window.renderAt(t)`; every frame is a pure function of `t` (seconds).
+ ```bash
+ helios render page.html -o out.mp4 --duration 12 [--audio song.mp3] [--preset medium]
+ helios still page.html --at 0.5,3 # PNGs, no encode
+ helios sheet page.html --every 1 --duration 12 # labelled contact sheet (also --at, --strip a:b)
+ helios verify page.html --duration 12 # frames depend only on t (in order vs reverse)
+ helios verify page.html --duration 12 --cues lyrics.srt [--json] # every timed cue (.srt/.vtt/.json) on screen at its time
+ helios check out.mp4 [--json] # WCAG 2.3.1 flashing, colour tags, frame count
+ helios analyze song.mp3 # song.beats.json: beats, downbeats (drifting tempo), kick/snare/hat, hits, risers, sections, envelopes
+ helios mcp # plugin MCP server over stdio
+ ```
+ - Canvas pages declare drawn text for the cue check: `window.heliosDrawnText?.add(text)` (undefined during normal renders).
+ - Renders are BT.709: converted with the BT.709 matrix and tagged (primaries, transfer, matrix, tv range).
+ - Registry components (`helios add <name>`): `beat-clock` (reads the analyze JSON: beat/bar/phase, pulse, envelope level, hits, sections) and `cursor` (scripted pointer; actions keyed to arrival/press times, minimum-jerk reaches, Fitts' law timing).
+
## Agent Skills
- Skills for AI agents are in `.agents/skills/helios-skills/skills/`:
- - `.agents/skills/helios-skills/skills/core/SKILL.md` - Core API patterns, state management, subscriptions
- - `.agents/skills/helios-skills/skills/renderer/SKILL.md` - Rendering pipeline usage
- - `.agents/skills/helios-skills/skills/player/SKILL.md` - Player component usage
- - `.agents/skills/helios-skills/skills/studio/SKILL.md` - Studio usage and extensions
- - `.agents/skills/helios-skills/skills/getting-started/SKILL.md` - Initial scaffolding and basic tasks
- - `.agents/skills/helios-skills/skills/workflows/*.md`, `.agents/skills/helios-skills/skills/examples/*.md` & `.agents/skills/helios-skills/skills/guided/*.md` - Task-specific workflows, examples, and guided skills
- - `.agents/skills/skill-creator/SKILL.md` - Meta-skill for generating new agent skills
+ The entry skill ships in the Helios agent plugin (`plugins/helios/`); the rest of the catalog is in `skills/` (install with `npx skills add BintzGavin/helios`):
+ - `plugins/helios/skills/make-video/SKILL.md` - Entry skill: one HTML page with `window.renderAt(t)`, rendered frame-exact to MP4, checked with stills and contact sheets
+ - `skills/core/SKILL.md` - Core API patterns, state management, subscriptions
+ - `skills/renderer/SKILL.md` - Rendering pipeline usage
+ - `skills/player/SKILL.md` - Player component usage
+ - `skills/studio/SKILL.md` - Studio usage and extensions
+ - `skills/getting-started/SKILL.md` - Initial scaffolding and basic tasks
+ - `skills/workflows/*/SKILL.md`, `skills/examples/*/SKILL.md`, `skills/guided/*/SKILL.md` & `skills/design/*/SKILL.md` - Task-specific workflows, examples, guided and design skills
+ - `.agents/skills/skill-creator/SKILL.md` - Meta-skill for generating new agent skills (for this repo's agents)
## Key Files for Agents
- `README.md` - Vision document
- `docs/BACKLOG.md` - Feature backlog
- `docs/PROGRESS-*.md` - Completed work (per domain)
- `docs/status/[ROLE].md` - Domain-specific status
- `.sys/llmdocs/context-*.md` - Architecture summaries
- `packages/core/src/index.ts` - Core public API
- `packages/player/src/index.ts` - Player public API
- `packages/renderer/src/index.ts` - Renderer public API
- `packages/cli/src/index.ts` - CLI public API
- `packages/infrastructure/src/index.ts` - Infrastructure public API
- `packages/studio/src/server/mcp.ts` - Studio MCP Server
- `packages/studio/src/server/plugin.ts` - Studio Vite Plugin
- `packages/studio/src/server/render-manager.ts` - Studio Render Manager
## Diagnostics
```typescript
const report = await Helios.diagnose();
// {
// waapi: boolean,
// webCodecs: boolean,
// offscreenCanvas: boolean,
// webgl: boolean,
// webgl2: boolean,
// webAudio: boolean,
// colorGamut: 'srgb' | 'p3' | 'rec2020' | null,
// videoCodecs: { h264: boolean, vp8: boolean, vp9: boolean, av1: boolean },
// audioCodecs: { aac: boolean, opus: boolean },
// videoDecoders: { h264: boolean, vp8: boolean, vp9: boolean, av1: boolean },
// audioDecoders: { aac: boolean, opus: boolean },
// userAgent: string
// }
```
## Roadmap
### V1.x (Active)
- **Studio**: v0.107.1+ (Completed: Improve Omnibar Test Coverage, Asset Drag and Drop, Export Job Spec)
- **CLI**: v0.45.2+ (Completed: CLI Command Coverage Tests V9, CLI Docker Adapter Regression Tests, CLI Registry Types Regression Tests)
- **Core**: v5.13.0+ (Completed: Generic Input Props, Active Clips, Composition Schema)
- **Player**: v0.78.1+ (Completed: Document getSchema API Parity, Bridge Coverage Completeness, Bridge Coverage Expansion)
- **Renderer**: v1.78.3+ (Completed: Validate HW Accel, Orchestrator Plan, Refactor Media Sync Logic)
- **Infrastructure**: v0.25.0+ (Completed: Optimize GCS Uploads, Orchestrator Test Coverage Expansion, InMemoryJobRepository Coverage)
### AI Integration
- **Studio MCP Server**: Inspect, create, and render compositions via LLM tools (Done)
- **System Prompt**: Programmatic context generation for agents (Done)
- **Helios Assistant**: Context-aware documentation search (Done)
+ - **Plugin MCP Server** (`helios mcp`): tools `preview_video`, `get_frames`, `verify_video` (purity + timed-text cues), `render_video` (checked with `helios check` when done), `get_render_status`, `cancel_render`, `analyze_audio`, `helios_library`; prompts `make_video`, `music_video` (Done)
### V2 (Next)
- Advanced audio engine (Tone.js integration)
### V3 (Future)
- Native Node.js encoding (beamcoder)
## Links
- Repository: https://github.com/gavinbintz/helios
- Documentation: ./docs/
- Examples: ./examples/