llms.txt@site/public · git:20261007.323e65a · 2026-10-07 · sha256 d93d9ffbd4273c2e

llms.txt@site/public git:20261007.323e65aB

Immutable. This exact content never changes and is served at /api/v1/blob/d93d9ffbd4273c2e.

# humanish

> You can't run a user study on an app that has no users yet. humanish runs one anyway. One command
> puts a synthetic participant with a persona and a task in front of your app in a real browser; what
> comes back is what they did, where they got stuck, and what it cost.

## When to use humanish

Reach for it when the question is "what happens when someone who is not us tries this?" and
the answer needs evidence rather than a prediction:

- a first-time visitor, a keyboard-only user, or a phone user has to get through a real flow
  (signup, first task, the thing the README promises) on a web app you maintain;
- a maintainer wants a defect report they can act on: screenshots, the participant's own words,
  a per-task funnel, a fail-closed verify grade, and a public-safe issue draft;
- a change under the UI (a probe, a prompt, a model) has to be checked against an instrument with
  known answers: `bench/` carries the planted-defect arms (58 of 60 defects reported over four
  runs, nothing invented in 15 clean runs, receipts with run ids).

Not the tool for load testing, pixel diffing, or a decision about people that needs people; the
limits page below says where simulated users diverge from real ones.

How an agent calls it: `npx humanish init --yes`, then `npx humanish run try-live` (a real drawDB study, with a $2 estimated model-spend
cap and separately billed desktop time). `npx humanish run first-run` is a synthetic evidence
preview with no keys or spend; it does not study an app. Read results with
`npx humanish runs --json` and `npx humanish review --run latest --json`; draft feedback with
`npx humanish feedback issue --run latest --repo owner/repo`. `--json` is available where supported; `tui` is a human surface and refuses detected agent sessions.

## Install

```
npm i -D humanish @e2b/desktop
npx humanish init --yes
npx humanish keys set e2b
npx humanish keys set openai
npx humanish doctor
npx humanish run try-live
```

Node 22.19 or newer. Install `humanish` and `@e2b/desktop` in the same project for live runs;
a one-shot `npx humanish@latest` can miss the optional desktop peer. The keyless preview
only needs `humanish`.

- Quickstart: https://humanish.dev/docs
- How a study works (study, run, participant, persona, actor, subject, route): https://humanish.dev/docs/concepts
- Your own app: https://humanish.dev/docs/your-app
- Results and feedback: https://humanish.dev/docs/read-results
- Paired Save-button study: https://humanish.dev/docs/save-button-study
- TodoMVC keyboard repair: https://humanish.dev/docs/todomvc-edit-study
- What a study costs, and budgets: https://humanish.dev/docs/what-a-study-costs
- Study file reference (every field, persona traits): https://humanish.dev/docs/study-files
- Trust boundaries and credentials: https://humanish.dev/docs/trust-boundaries
- Computer-use controls: https://humanish.dev/docs/computer-use
- Local coding agents: https://humanish.dev/docs/local-agents
- Observer, TUI, and JSON alternatives: https://humanish.dev/docs/review-surfaces
- CLI reference: https://humanish.dev/docs/cli

For coding agents: `npx skills add danielgwilson/humanish --skill humanish`

## Commands

<!-- humanish-cli:start -->

This command index is generated from the shipped CLI. Full arguments and options: https://humanish.dev/docs/cli

- `humanish init`: Set up committed humanish/ source files and ignored .humanish/ runtime state.
- `humanish doctor`: Explain project readiness and missing humanish setup.
- `humanish run`: Run a study, as a dry run or with live participants. This is the everyday command.
- `humanish watch`: Run a study, open its Observer and keep the shell attached.
- `humanish observe`: Open a saved run in the Observer, served on http://127.0.0.1; without --run, the latest. With --all, serve the run library, opening on --run when given.
- `humanish verify`: Check a run's evidence and share safety.
- `humanish review`: Show a run's review: verdict, summary, gaps and the findings of its analysis, with the evidence each one cites.
- `humanish analyze`: Analyze retained participant evidence into versioned findings. Selected text and captures go to the chosen remote analyst. Opening Observer never starts analysis.
- `humanish analyze list`: List immutable analysis versions, including failed attempts.
- `humanish analyze show`: Print a run's analysis findings with the evidence each one cites. --json prints the validated analysis record and its correction history. Defaults to the latest usable version.
- `humanish analyze correct`: Append a human review note bound to one exact finding version; original claims remain intact.
- `humanish feedback`: Draft public-safe feedback issues from a run. humanish never calls the GitHub API.
- `humanish feedback list`: List recorded feedback candidates and any saved draft.
- `humanish feedback draft`: Generate a public-safe feedback draft from verified evidence.
- `humanish feedback verify`: Verify the feedback draft for public issue eligibility.
- `humanish feedback issue`: Print Markdown for a public GitHub issue. Does not mutate GitHub.
- `humanish feedback issue-url`: Print a prefilled public issue URL. Does not mutate GitHub.
- `humanish export`: Export a run as self-contained Observer HTML, or a separately verified redacted bundle workspace. HTML requires share_ready unless --local-only; bundle format requires --redact-screenshots and preserves the original.
- `humanish runs`: List this project's runs and which one is latest.
- `humanish stats`: Cost, outcome, and duration roll-ups across run history. Estimates stay labelled; unknown costs count as unknown.
- `humanish study`: List, show and check studies in humanish/studies/.
- `humanish study list`: List the studies in this project, committed and local.
- `humanish study show`: Show a study's parsed file and its warnings without running it.
- `humanish study check`: Check the study file and, with --reachability, its named endpoints. humanish doctor --study <study> checks this machine.
- `humanish migrate`: Convert humanish.lab.v2 study files to humanish.study.v3. A file under a labs/ directory moves to the matching studies/ directory; any other file is rewritten in place. An original that a failure leaves out of place is kept as <name>.v2.bak. Pass the files to convert; without any, the six study directories are scanned. Every destination, moved key and dropped key is printed before anything is written. A file that changes during the run is kept and listed.
- `humanish comms`: Local email capture, real receiving connections, checks and cleanup recovery.
- `humanish comms providers`: List installed communication provider capabilities. No network requests.
- `humanish comms connections`: Manage project-local non-secret connection profiles. A study explicitly selects its receiving connection.
- `humanish comms connections list`: Show saved connections and local credential status; does not authenticate with a provider.
- `humanish comms connections add`: Save an AgentMail connection profile. Does not write a key, alter a study or contact the provider.
- `humanish comms check`: Check connection and credential presence; --online authenticates without creating inboxes.
- `humanish comms configure`: Preview or save a local receiving-enabled copy of a supported study. No provider requests.
- `humanish comms recover`: Inspect interrupted email leases; --apply deletes only privately recorded resources owned by this project and account.
- `humanish comms catch`: Run the email catch on this host so humanish can study an app it does not provision. Your app posts its email sends here; the persona opens /inbox; humanish drains GET /deliveries and writes digest-only evidence. Point your study's comms.email.external.catchBaseUrl at this server.
- `humanish runtime`: Prepare or inspect the local browser runtime.
- `humanish runtime status`: Check local Docker, virtualization and the cached browser image without downloads, and how many participant desktops fit.
- `humanish runtime setup`: Download and install the local browser image. Does not start a study or use model quota.
- `humanish reclaim`: Kill an interrupted run's sandboxes: the exact ids journaled in its sandbox-receipts.ndjson, and any sandbox E2B lists with this run's owner tags (a sandbox whose id never reached a receipt). It lists only sandboxes that match every tag of this run. --check asks E2B whether each still exists and kills nothing. Needs E2B_API_KEY in the environment or in --dotenv.
- `humanish keys`: Show which provider keys are set and where each comes from, and manage the humanish user key store that provider-key discovery reads.
- `humanish keys status`: Show each provider key humanish uses: where it comes from, or the command that adds it. Values are never printed.
- `humanish keys set`: Store one provider key in the user store (0600), prompted with hidden input. With no key named, ask for each missing key in turn.
- `humanish keys unset`: Remove one key from the user store.
- `humanish keys list`: List the key names in the user store. Values are never printed.
- `humanish telemetry`: Show or change anonymous usage collection.
- `humanish telemetry status`: What is collected, and whether it is on.
- `humanish telemetry enable`: Turn anonymous usage collection on.
- `humanish telemetry disable`: Turn anonymous usage collection off.
- `humanish tui`: Browse studies and runs in an interactive terminal UI. It refuses detected agent sessions and non-TTY input or output; agents use humanish study list --json, humanish study show <study> --json and humanish runs --json.

<!-- humanish-cli:end -->

`humanish tui` is for a human: interactive stdin and stdout. It refuses
detected coding-agent sessions even with a TTY (`HUMANISH_TUI_AGENT_SESSION`), or
non-TTY input/output (`HUMANISH_TUI_REQUIRES_TTY`); both return exit code 2.
`--json` reports that refusal; it does not make the TUI usable by an agent.
Use `humanish study list --json`, `humanish study show <study> --json`, and
`humanish runs --json` for read-only browsing. `humanish review --run latest --json`
reads an existing result. `humanish run <study> --json` starts a run and may
incur provider spend; it is not a browsing command. `--force` is only for a person
actually at the keyboard; it does not bypass TTY or Node requirements.
Details: https://humanish.dev/docs/review-surfaces#for-coding-agents-and-scripts

## Credentials

Choose where the participant runs before requesting credentials. Local browser studies
use a supported Codex CLI with file-backed ChatGPT login on Linux x64 with
Docker/KVM/TUN or M3+ Mac with Lima. They need neither E2B nor an OpenAI API key;
participants and the separate default analyst use account quota and remote
inference. Setup: https://humanish.dev/docs/your-app#study-your-running-local-app

A hosted desktop with an OpenAI participant needs:

- `E2B_API_KEY`: the hosted sandbox the persona works in. Set it with
  `npx humanish keys set e2b`, or `e2b auth login`.
- `OPENAI_API_KEY`: the model driving the persona. Set it with `npx humanish keys set openai`.
  Alternatively a signed-in local agent (Codex, Claude Code) can supply the participant, but
  `E2B_API_KEY` is still required for that hosted desktop. Analysis defaults to
  the OpenAI API separately; an account participant does not change that default.

`humanish doctor --study <study> --json` checks the study's setup and names
the command to fix missing prerequisites without printing key values. Persisted text is scrubbed for known
secret values and patterns; this is not a guarantee that all sensitive content is detected.

Node 22.19 or newer.

## Worked example: the Excalidraw study

Run `cua-2026-08-07T17-44-48-760Z-87389419` (2026-08-07): four computer-use participants on
hosted 1920×1080 desktops drove a commit-pinned clone of Excalidraw
(excalidraw/excalidraw@4872083c), an open-source virtual whiteboard. 3/4 participants passed;
one gave up; `humanish verify` ran 16/16 checks. Wall-clock 6m 06s; estimated cost
~$1.54 (rates as of 2026-08-05).

Participants and verbatim outcomes:

- diagram-login-flow — persona final report: "Done"
- sticky-notes — persona final report: "Done."
- sketch-shapes — recorded reason (harness; the persona emitted no final message): "gave up: 8 consecutive turns with no change to the UI state"
- export-drawing — persona final report: "Done"

Excalidraw is the application studied; it is not a humanish adopter or endorser.

## Credentials and sharing

- Computer-use model keys stay on the host. Terminal actors receive a command-scoped runtime key by default; opt-in `execution.runtimeAuth: openai-egress` keeps the raw key outside but still permits sandbox processes to spend through the proxy. See https://humanish.dev/docs/trust-boundaries#store-credentials.
- Evidence lands in gitignored `.humanish/` by default. Exposing Observer or sharing an export makes it accessible to others.
- `humanish feedback issue` renders a draft; no GitHub API call exists on that path.
- Verify fails closed: a bundle that cannot pass every gate grades `local_only` or `blocked`, never `share_ready`.

## Known failure modes

The limits, cited: what the computer-use substrate cannot do yet, where simulated users
diverge from real ones, what a passing `humanish verify` grade does not prove, and when to
recruit real people instead. Includes first-party run ids for our own failed participants, and
the published numbers against simulated users.

- https://humanish.dev/failure-modes

## Component registry

humanish.dev serves a shadcn-compatible component registry at `https://humanish.dev/r/<name>.json`.
Items: `humanish-tokens` (the site's token system), `terminal-cast` (static terminal
transcript), `persona-lane` (one participant with screenshot, verbatim report, honest status
chip), `pinned-replay` (the scroll-pinned run walkthrough, with the Excalidraw study's
metadata as demo data — screenshots not included).

Install directly:

```
npx shadcn@latest add https://humanish.dev/r/pinned-replay.json
```

Or configure the namespace in `components.json` and add by name:

```json
{ "registries": { "@humanish": "https://humanish.dev/r/{name}.json" } }
```

```
npx shadcn@latest add @humanish/terminal-cast
```

## Links

- GitHub: https://github.com/danielgwilson/humanish
- npm: https://www.npmjs.com/package/humanish