seed-test-data · git:20261118.4546234 · 2026-11-18 · sha256 b1506a5bfbfc6009
seed-test-data git:20261118.4546234A
Immutable. This exact content never changes and is served at /api/v1/blob/b1506a5bfbfc6009.
No tool to install first
claude plugin marketplace add https://markdownregistry.com/mp/a/art_snq5ivmsbxks7yui/b1506a5bfbfc6009b4c47f012abf826eeb5952148380652e8c1ccfa1348b1dbc.json
Then /plugin install seed-test-data@markdownregistry-snq5ivmsbxks7yui in Claude Code. It installs this exact version, the one graded A here, pinned to the commit it was graded at, and leaves it there.
--- name: seed-test-data description: | Seed reproducible local Langfuse data in ClickHouse and Postgres. Use for complex traces, long sessions, v3/v4 events, bulk list data, or frontend rendering and performance tests; never use ad hoc scripts or raw inserts. --- # Seed Test Data One-shot deterministic test data for local Langfuse. The CLI handles env loading, preflight checks, ClickHouse/Postgres writes, readback verification, and prints UI deep links plus a machine-readable JSON summary (last stdout line). ## If anything fails, run doctor first ```bash pnpm run seed -- doctor ``` Prints PASS/WARN/FAIL per dependency (Postgres, migrations, project, ClickHouse, v4 dev tables, Redis, MinIO, web app) with the exact fix command for every failure. Do not debug Docker/ClickHouse manually before running this. ## Need → command | I need... | Command | | ------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Managed in-app assistant skills | `pnpm run seed -- assistant-skills` (imports bundled SKILL.md files through the synthetic seed key; production label, in-app-agent tag; reuses unchanged versions) | | A very complex observation tree (v3) | `pnpm run seed -- trace-tree --observations 5000 --depth 12 --breadth 500` | | The same tree readable in the v4 events UI | add `--v4` (writes `events_full`; `events_core` fills via MV) | | Async parents whose subtree outlives their own span (subtree wall-clock duration badge) | add `--async-parents` to `trace-tree` (root + hub end immediately while children keep running) | | A realistic agent flow over a timeline (graph view + scrubbable timeline) | `pnpm run seed -- agent-timeline --turns 6 --v4` (LangGraph refine loop planner→retriever→generator→critic→loop, staggered in time; add `--timing-only` for the pure timing fallback) | | A trace that is large as a GRAPH (many distinct node names + connections, the trace-graph layout stress) | `pnpm run seed -- agent-graph --v4` (~1,350 distinct connections from 350 observations; `--nodes 120 --steps 100 --parallel 8` crosses the layout ceiling, `--nodes 80 --steps 30 --parallel 4` is small-but-dense) | | A dozen SMALL traces, each a different timeline shape (the everyday case, not a stress test) | `pnpm run seed -- timeline-shapes --v4` (12 hand-timed traces of 4-25 observations: rag answer, streamed chat with a TTFT split, 8-way fan-out, retry backoff with widening gaps, a 13-minute wait on a human, one slow tool dwarfing everything, an error cascade with failover, in-flight spans with no end time, zero-duration checkpoints, a ten-level ladder, 24 flat siblings, a three-turn agent loop with think time; `--shape <slug>` for one) | | A row carrying every annotation at once, to judge whether the timeline is overloaded | `pnpm run seed -- timeline-annotated --v4` (ONE trace, 12 observations: one/two/four scores — the last collapsing into `+1` — one and twelve comments, a streaming first-token mark, costs and durations spread so the heat map paints some rows, plain rows beside them for contrast, and all three label placements on screen together) | | A multi-HOUR agent run (wall clock dominated by waiting, as production ones are) | `pnpm run seed -- agent-timeline --turns 120 --turn-gap-ms 60000 --v4` (~2.5h; the gap is jittered up to 2x, so the work lands at a few percent of the trace instead of packing every span into a few seconds) | | A demo-grade, real-looking agent trace (videos, screenshots, docs) | `pnpm run seed -- support-agent --v4 --id-prefix <hex>` (one fixed, fully handcrafted support-copilot refund run: guardrails, parallel context fan-out, 3-turn ReAct loop with real payloads/costs; deterministic — reseed with a FRESH prefix for a clean take; the prefix is the trace id, so a hex prefix reads like production) | | A demo-grade, real-looking multi-user SESSION (session timeline videos, screenshots, docs) | `pnpm run seed -- incident-session --id-prefix inc-4471-checkout-latency` (one fixed, handcrafted incident channel: 7 turns, 4 named users, accumulating chat history with reasoning and markdown, tool calls paired to TOOL rows plus two untraced ones, a WARNING and an ERROR tool, a nested sub-agent, a refusal, typed session scores and comments; v4 on by default; the prefix is the session id) | | A plain trace with no agentic types (collapsed-by-default graph panel) | add `--plain` to `trace-tree` (SPAN/GENERATION/EVENT only) | | A trace with MORE observations than the detail view loads (v4 caps the tree at 10k, `startTime ASC` — the chronological tail is missing) | `pnpm run seed -- trace-tree --observations 12000 --stride-ms 10 --v4` (`--stride-ms` starts each observation index × N ms in, so start times are unique and strictly increasing: observation index < 10000 loads, `-obs-10000` and up fall past the cap. Without it thousands of rows share one millisecond and the boundary is arbitrary. Re-runs that CHANGE timing flags need a fresh `--id-prefix`: `start_time` is an events ORDER BY key, so both versions persist in the same trace) | | An extremely DEEP single-chain trace (tree depth = observation count; layout stress) | `pnpm run seed -- deep-chain --v4` (1401 sequential generations, each the sole child of the previous — the mis-parented-instrumentation shape from LFE-10959 that collapses tree/timeline layouts; `--observations N` to change depth) | | A super tough session (v3 legacy session view) | `pnpm run seed -- long-session --traces 300 --observations-per-trace 8` | | Diverse v4 session shapes (chat / coding-agent / mixed / media) for the session-detail view | `pnpm run seed -- session-shapes --shape all` (the `agent` shape has I/O on AGENT/TOOL with no GENERATION — pre-LFE-10520 the "first generation" default rendered empty cards for it; the current "All observations with I/O" default renders it correctly; v4 on by default) | | Many sessions for the sessions TABLE and its filters / search bar (not the detail view) | `pnpm run seed -- session-variety --sessions 120 --days 14` (searchable topic ids, 1-3 userIds and 1-3 tags per session, four environments, session metadata `tier`/`region`/`channel`/`topic`, numeric + categorical + boolean session scores, comments on a quarter; v4 on by default. Widen the time-range picker past the default 1 day.) | | Session messages carrying Langfuse media references (inline image, several refs in one message, bare refs in a content array, link-only payload) | `pnpm run seed -- session-shapes --shape media` — uploads the image/audio/pdf fixtures to MinIO and links them to the observation, so the inline chip and the "Media" strip both resolve (LFE-14815, LFE-9577) | | Many traces for list/filter performance | `pnpm run seed -- many-traces --count 100000 --days 14` | | Many readable names and varied facets in v4 Traces and Observations | `pnpm run seed -- filter-variety --traces 1200 --names 240 --v4` (240 distinct root names with common/rare counts; types, environments, tags, levels, models, users, sessions, and metadata; links generations to existing project prompts) | | Long-window v4 traffic with cost/latency/token OUTLIERS (outlier chart strip, LFE-14451) | `pnpm run seed -- outlier-traffic --days 90` (diurnal base load + deterministic spikes + hour-long ×8-latency incidents; root AGENT + GENERATION carrying cost + TOOL per trace; v4 on by default) | | AI Gateway traffic for the org settings Analytics tab (cost per user / API key / model) | `pnpm run seed -- gateway-traffic --days 30` (writes into the org's gateway ingestion project; creates gateway keys bound to org members up to `--keys` plus one service key bound to no user, one GENERATION per request attributed like gateway ingestion does (`service_name = langfuse-ai-gateway`, gateway key id in `ingestion_api_key`), plus a few SDK events the tab must exclude; v4 on by default) | | AI Gateway Budgets page (spend over time per key / user, key and user budgets, users table pagination and search) | `pnpm run seed -- gateway-budgets` (also runs in `pnpm run ch:seed`; adds 36 gateway-only members `<first>.<last>.<n>@example.com`, one gateway key each plus a service key, ~17k requests over 90 days with a growth trend and quiet weekends, five key budgets sized to 8%–112% usage and three user budgets sized to 20%–125%; v4 on by default) | | Scores with spaces in the name (filter/grammar testing) | `pnpm run seed -- scored-traces --traces 24 --v4` | | Numeric score distributions (wide, narrow, negative, near zero, span across zero) | `pnpm run seed -- score-distributions` | | Custom model definitions reachable from a trace (price editor entry points) | `pnpm run seed -- custom-models --v4` (a tiered model with a condition-gated second tier and one usage type priced at 0, a single-tier model, and a generation whose model matches no definition so its badge opens the create dialog) | | Many project-owned evaluators for gallery infinite-scroll testing | `NEXTAUTH_URL=https://pr-<N>.preview.langfuse.com pnpm run seed -- evaluator-gallery --count 200` (uses the default synthetic seed API key and reconciles deterministic evaluator names; also works against local seeded environments when `NEXTAUTH_URL` is omitted) | | Lots of scores on every node (dense score badges, tree-row overflow testing) | add `--scores-per-node 12` to `trace-tree` (N distinct scores per observation; try `--depth 2 --breadth 44` for many tall sibling rows) | | Extra trace tags, incl. mixed case/accents (tag filter ordering) | add `--tags "Zebra,apple,Ärger"` to `trace-tree` (comma-separated, appended to the scenario's own tags) | | Varied human-annotation queues (annotate UI / keyboard testing) | `pnpm run seed -- annotation-queue --core-items 12` (creates a "core types" queue covering every score-field render path + an "edge cases" queue with archived/stale/partial scores and observation/session/deleted/completed items) | | Huge/malformed/unicode payloads | `pnpm run seed -- trace-tree --payload-bytes 1000000 --payload-style malformed` (styles: json, text, malformed, unicode, bignum, base64) | | Big integers beyond 2^53-1 (number-precision testing) | `pnpm run seed -- trace-tree --observations 1 --payload-style bignum` | | Huge base64 data-URI in ChatML IO (multimodal crash shape, LFE-10152) | `pnpm run seed -- trace-tree --observations 30 --payload-bytes 20000000 --payload-style base64 --v4` (one unbroken multi-MB base64 token in trace + root-observation IO; max 50 MB) | | See all scenarios and flags | `pnpm run seed -- list --json` | | Predict without writing | add `--dry-run` | For comparing filters across experiments, run `pnpm run seed -- experiment-filters`: three runs over 60 shared items with shifted numeric/categorical/boolean scores, status, item/event metadata, missing scores and missing results. Per-experiment expected matches are documented in `packages/shared/scripts/seeder/README.md`. For a v4 experiment with chat messages and nested JSON input/output, run `pnpm run seed -- experiment-io`. It creates one dataset, one experiment, and three items, then prints the experiment results link. Add `--compare` for a second run on that dataset whose chat output is long enough to need a taller comparison row. Set `NEXTAUTH_URL` to your local app URL when using a port other than 3000. For Topics discovery, held-out assignment, or evaluation fixtures, follow the [Topics fixture walkthrough](../../../packages/shared/scripts/seeder/README.md#topics). For Topics results UI checks without model keys, `pnpm run seed -- topics-map` publishes ready-made maps. ## Contract For managed in-app assistant skills, run `pnpm run seed -- assistant-skills`. It imports the bundled catalog through the synthetic seed API key, reuses unchanged versions, labels them `production`, and adds the `in-app-agent` tag (`--tag` overrides it). See the [local assistant setup](../../../packages/shared/scripts/seeder/README.md#managed-assistant-skills) for the worker configuration. Add `--demo` for an instructions-only `local-assistant-check` skill; ask the assistant "Run the local assistant check skill." and expect "Hosted skills are connected." - Last stdout line is a JSON summary: `traceIds`, `sessionIds`, `counts`, `verified` (ClickHouse readback), `links` (UI deep links). Use `--json` to suppress progress logs. Non-zero exit = data did not land; the error includes a `fix:` line. - Deterministic: same `--seed` (default 42) and flags → same ids (ids never contain dates), with timestamps anchored to the current UTC day. Re-running within the same day overwrites in place; a later-day re-run updates the same ids with re-anchored timestamps (the previous day's rows persist under their old dates until then). Independent copies come only from `--id-prefix`. - Default project is the seeded `7a88fb47-b4e2-43b8-a06c-a5ce950dc53a` (login `demo@langfuse.com` / `password`); override with `--project`. - Open the printed `links` in the browser to verify visually. The v4 events-backed UI is the per-user "Fast (Preview)" sidebar toggle, or `LANGFUSE_MIGRATION_V4_WRITE_MODE=events_only` server-side. ## Extending Add a scenario in `packages/shared/scripts/seeder/scenarios/`: a plain function using the deterministic `Rng` (never `Math.random`), register it in `scenarios/index.ts`, and update the table in `packages/shared/scripts/seeder/AGENTS.md` and this skill. Scenario names, flags, and JSON keys are additive-only contracts. Design rationale: `packages/shared/scripts/seeder/README.md`.