Immutable. This exact content is served forever at /api/v1/blob/1e0d9ebe8ce3ec9a.
---
name: remote-gpu
category: compute
description: >
Orchestrate a headless GPU server (NUC, cloud VM, or any SSH-accessible machine) from a local
Mac or workstation. Use when: (1) Running GPU workloads remotely (training, inference, video
generation), (2) Triggering Claude Code sessions on a remote machine, (3) Launching autoany
EGRI loops, symphony orchestrations, or training scripts on remote hardware, (4) Managing jobs
on a headless GPU server (submit, monitor, cancel, download results), (5) Setting up SSH
tunnels or API bridges to a GPU machine, (6) Orchestrating multi-machine agent workflows.
Triggers on: remote gpu, headless server, gpu server, remote training, remote inference,
ssh gpu, nuc server, remote claude code, remote agent, gpu orchestration.
---
# Remote GPU Orchestrator
Operate a headless GPU server from your Mac. Submit jobs (training, inference, agents),
monitor progress, and retrieve results — all over SSH or HTTP API.
## Architecture
```
┌─────────────────┐ SSH / HTTP API ┌──────────────────────┐
│ Mac (Control) │ ──────────────────────────▶ │ NUC / GPU Server │
│ │ │ │
│ - Claude Code │ Commands: │ - RTX 4090 (12GB) │
│ - This skill │ submit_job │ - gpu-server.py │
│ - gpu-remote.sh │ check_status │ - Job queue │
│ │ stream_logs │ - Claude Code │
│ │ download_results │ - autoany / symphony│
│ │ run_claude_session │ - LTX-2 / training │
└─────────────────┘ └──────────────────────┘
```
## Quick Setup
### 1. Configure SSH Access
```bash
# On Mac — set up passwordless SSH to NUC
ssh-keygen -t ed25519 -f ~/.ssh/nuc_gpu
ssh-copy-id -i ~/.ssh/nuc_gpu.pub user@NUC_IP
# Add to ~/.ssh/config
cat >> ~/.ssh/config << 'EOF'
Host nuc-gpu
HostName NUC_IP_ADDRESS
User YOUR_USER
IdentityFile ~/.ssh/nuc_gpu
Port 22
ServerAliveInterval 60
EOF
# Test
ssh nuc-gpu "nvidia-smi"
```
### 2. Install Server on NUC
```bash
# SSH into NUC
ssh nuc-gpu
# Copy and start the server
pip install fastapi uvicorn psutil
python gpu-server.py --port 8420 --workdir ~/gpu-jobs
```
Or run `scripts/setup-nuc.sh nuc-gpu` from Mac to automate.
### 3. Use from Mac
```bash
# Via SSH (simplest)
source scripts/gpu-remote.sh
gpu-submit "python train.py --epochs 10" --workdir ~/project
gpu-status
gpu-logs job-abc123
gpu-download job-abc123
# Via HTTP API (if gpu-server.py running)
curl http://nuc-gpu:8420/submit -d '{"command":"python train.py"}'
curl http://nuc-gpu:8420/jobs
```
## Job Types
### Training Runs
```bash
# Submit a training job
gpu-submit "cd ~/project && python train.py --config config.yaml" \
--name "lora-training-v2" \
--workdir ~/project
# Monitor GPU usage during training
gpu-watch # streams nvidia-smi every 5s
```
### Video Generation (LTX-2)
```bash
gpu-submit "cd ~/LTX-2 && source .venv/bin/activate && \
python -m ltx_pipelines.run \
--config configs/ltx-2.3-22b-distilled-2stage.yaml \
--quantization fp8-cast \
--prompt 'A drone shot over mountains at dawn' \
--height 704 --width 1216 --num_frames 97 \
--output /tmp/output.mp4" \
--name "ltx-mountains" \
--download /tmp/output.mp4
```
### Claude Code Sessions
```bash
# Start a Claude Code session on the NUC
gpu-claude "Fix the failing tests in ~/project" --workdir ~/project
# Start with a specific branch
gpu-claude "Implement the feature described in PLAN.md" \
--workdir ~/project --branch feature/new-api
```
### Autoany EGRI Loops
```bash
# Run an EGRI optimization loop on GPU
gpu-submit "cd ~/autoany && cargo run -- \
--config egri.toml \
--max-iterations 50 \
--target-metric accuracy" \
--name "egri-optimization"
```
### Symphony Orchestrations
```bash
# Launch a symphony workflow on GPU
gpu-submit "cd ~/symphony && cargo run -- \
orchestrate workflow.toml" \
--name "symphony-pipeline"
```
## Commands Reference
All commands work via the `gpu-remote.sh` shell functions:
| Command | Description |
|---------|-------------|
| `gpu-submit CMD` | Submit a job, returns job ID |
| `gpu-status` | Show all jobs and GPU state |
| `gpu-logs JOB_ID` | Stream logs from a job |
| `gpu-cancel JOB_ID` | Cancel a running job |
| `gpu-download JOB_ID [FILE]` | Download job output files |
| `gpu-watch` | Live GPU monitoring (nvidia-smi) |
| `gpu-claude PROMPT` | Start Claude Code session on NUC |
| `gpu-ssh` | Interactive SSH to NUC |
| `gpu-sync DIR` | rsync a directory to/from NUC |
| `gpu-tunnel PORT` | SSH tunnel a port from NUC to localhost |
### Options
```
--name NAME Human-readable job name
--workdir DIR Working directory on NUC
--branch BRANCH Git branch to checkout before running
--download FILE Auto-download this file when job completes
--gpu GPU_ID Target GPU index (default: 0)
--timeout SECS Job timeout (default: 3600)
```
## HTTP API (gpu-server.py)
If running the Python API server on the NUC:
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/submit` | POST | Submit job `{command, name, workdir, timeout}` |
| `/jobs` | GET | List all jobs with status |
| `/jobs/{id}` | GET | Job detail (status, logs, files) |
| `/jobs/{id}/logs` | GET | Stream job logs (SSE) |
| `/jobs/{id}/cancel` | POST | Cancel running job |
| `/jobs/{id}/files` | GET | List output files |
| `/jobs/{id}/files/{name}` | GET | Download a file |
| `/status` | GET | GPU info, disk, memory |
See `references/api-reference.md` for full API documentation.
## Configuration
Create `~/.config/gpu-remote/config.toml` on Mac:
```toml
[server]
host = "nuc-gpu" # SSH host alias or IP
port = 8420 # API server port (if using HTTP)
user = "your-user" # SSH user
mode = "ssh" # "ssh" or "api"
[defaults]
workdir = "~/gpu-jobs"
timeout = 3600
gpu_id = 0
[sync]
exclude = [".git", "node_modules", "__pycache__", ".venv"]
```
## Troubleshooting
- **SSH timeout:** Add `ServerAliveInterval 60` to SSH config
- **CUDA OOM:** Check `gpu-status` for other jobs using VRAM, cancel or wait
- **Job stuck:** Use `gpu-logs JOB_ID` to check output, `gpu-cancel JOB_ID` to kill
- **Server down:** SSH in and restart: `ssh nuc-gpu "python gpu-server.py &"`
- **File transfer slow:** Use `gpu-sync` (rsync) instead of individual downloads