llms-full.txt · git:20261101.a295790 · 2026-11-01 · sha256 acfc753da7e43d04
llms-full.txt git:20261101.a295790C
Immutable. This exact content never changes and is served at /api/v1/blob/acfc753da7e43d04.
# skein-js — full documentation
> The open-source alternative to LangGraph Platform for TypeScript: a self-hosted Agent Protocol
> runtime for LangGraph.js servers or local SQLite applications, and a drop-in replacement for the LangGraph CLI.
> This file concatenates the user-facing docs for wholesale ingestion. It is GENERATED by
> scripts/generate-llms-full.mjs — run `pnpm docs:llms` to regenerate. The curated index is llms.txt.
<!-- source: README.md -->
# skein-js
[](https://www.npmjs.com/package/skein-js) [](https://www.npmjs.com/package/skein-js) [](./LICENSE) [](https://github.com/skein-js/skein-js/actions/workflows/ci.yml) [](https://skein-js.github.io/skein-js/)
**Run LangGraph.js agents on your own infrastructure.**
skein-js is an open-source TypeScript runtime for hosting
[LangGraph.js](https://docs.langchain.com/oss/javascript/langgraph/overview) agents. LangGraph defines
and executes your graphs; Skein provides the API, persistence, streaming, background work, and console
around them.
## Why skein-js?
A graph that works in a script still needs a backend before people can use it. Conversations need
state, long-running work needs a queue, and people need a way to see what happened or approve the
next step. Building and maintaining that infrastructure takes time away from your application.
Skein supplies that backend. Keep your graph code, models, and tools, then run it as a service on
your infrastructure or inside an application you already ship. It is an Apache-2.0 alternative to
LangGraph Platform (now LangSmith Deployment), with no commercial license key required.
Build a customer-facing assistant, a research tool, or a workflow that waits for human approval.
Your existing LangGraph SDK clients and Agent Chat UI connect through the same Agent Protocol:
the HTTP and streaming API around your graphs.
## What you get
- **Conversation state and long-term memory.** Keep conversation history and recall information
across threads. Use Postgres for shared servers or SQLite for local persistence.
[State and persistence](./docs/state-and-context.md) · [Memory](./docs/memory.md)
- **Streaming with standard LangGraph clients.** Deliver tokens, tool activity, and graph updates as
they happen. Connect with the LangGraph SDK and its frontend bindings, including React's
`useStream`. [Streaming](./docs/streaming.md) · [Client integration](./docs/react-sdk.md)
- **Human approvals and checkpoint history.** Expose LangGraph interrupts to your users, resume
after a decision, and explore a different outcome from a saved checkpoint.
[Human-in-the-loop](./docs/human-in-the-loop.md) · [Time travel](./docs/threads.md#time-travel-re-run-a-turn-a-different-way)
- **Background jobs and schedules.** Let work continue after a request ends, run agents on a
schedule, and receive a callback when a run finishes.
[Background jobs](./docs/background-jobs.md) · [Schedules](./docs/crons.md)
- **Channels and durable callbacks.** Connect authenticated events from your systems to a graph and
deliver its outcome to an allowed destination. Inspect and replay webhook deliveries.
[Channels](./docs/channels.md) · [Webhooks](./docs/webhooks.md) · [Native access and BullMQ](./docs/native-access.md)
- **Console and observability.** Test graphs, inspect conversations, and manage approvals in your own
console. Connect LangSmith, PostHog, or OpenTelemetry for tracing and analytics.
[Console](./docs/console.md) · [Observability](./docs/observability.md)
## Quick start
> 🚧 **Status: pre-alpha, but end-to-end.** Development and self-hosted production work today, with
> Fetch, Express, Fastify, NestJS, and Next.js adapters. See the [roadmap](./docs/roadmap.md).
Use **Node 24** (recommended), then create a project:
```bash
npm create skein-js@latest my-agent
cd my-agent
npm run dev
```
Accept the scaffolder's defaults to install dependencies and start with an echo graph that needs no
API key, database service, or Docker. Open <http://localhost:2024/console>, create a thread, and send a
message. The server runs at <http://localhost:2024>.
[Your first agent](./docs/your-first-agent.md) walks through adding a model and tools, then deploying.
For workspace setup and generator options, see [scaffolding](./docs/scaffolding.md).
### Already using LangGraph?
Install the CLI in your existing project:
```bash
npm install -D skein-js
```
Then change your development script:
```diff
- "dev": "langgraph dev"
+ "dev": "skein dev"
```
Keep your `langgraph.json`, graph exports, and clients. See
[LangGraph CLI compatibility](./docs/langgraph-cli-compat.md) for supported commands, fields, and
limitations.
## Run it your way
Each path uses the same Agent Protocol engine. Choose where the runtime lives:
| Path | Use it for | Guide |
| --------------------- | ------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| **Standalone server** | A shared agent service with Postgres and Redis, deployed as Node or a container | [Server deployment](./docs/deploy.md) |
| **Existing backend** | Graphs mounted alongside your Express, Fastify, NestJS, or Next.js routes | [Embedding](./docs/embedding.md) · [Adapters](./docs/adapters.md) |
| **Local application** | A desktop app, CLI, or daemon with one Node process owning a SQLite file | [Local deployment](./docs/local-deployment.md) |
The default development server uses memory drivers. Select durable storage when you need state to
survive beyond development. Your application chooses its models, credentials, and tools in every
path.
## The console
The console is served by your own process at `/console`. Send a message in the playground, follow
live runs, review pending approvals, and browse saved memory and schedules.
It is enabled during development and **off by default in production**. The
[console guide](./docs/console.md) covers access, configuration, and the full tour.
## Documentation
[Read the docs](https://skein-js.github.io/skein-js/) for the complete guides and reference.
- [Getting started](./docs/getting-started.md) — adopt Skein with a new project or an existing graph.
- [Features](./docs/features.md) — find a capability and its guide.
- [Agent Protocol](./docs/agent-protocol.md) — API resources and authentication.
- [Recipes](./docs/recipes/index.md) — practical patterns for serving and running agents.
- [Agent skills](./docs/agent-skills.md) — setup guidance for AI coding assistants.
LangChain Deep Agents support is **experimental**: see the
[Deep Agents recipe](./docs/recipes/deep-agents.md).
For AI tools, [llms.txt](./llms.txt) indexes the docs and [llms-full.txt](./llms-full.txt) bundles them.
## Examples
Start with [triage-agent](./examples/triage-agent) for background work, schedules, and approvals
without an API key, or [chat-app](./examples/chat-app) for a research assistant with a Next.js UI.
The [examples directory](./examples) also includes framework integrations, migration, and local apps.
For durable embedded calls and external command queues, use the
[native/BullMQ approval example](./examples/native-transports) and
[security guide](./docs/native-access-security.md).
## Contributing & feedback
[Open an issue](https://github.com/skein-js/skein-js/issues) for bugs and compatibility reports, or
see [CONTRIBUTING.md](./CONTRIBUTING.md) to contribute. Changes and releases are in the
[changelog](./CHANGELOG.md).
## Sponsorship
Sponsored by [Garatropic Studios](https://garatropic.com), supporting engineering and ongoing
maintenance. To support development, [get in touch](https://github.com/skein-js/skein-js/issues).
## License
[Apache-2.0](./LICENSE)
---
<!-- source: docs/index.md -->
## Why skein-js? {#why-skein-js}
skein-js is an open-source TypeScript runtime for hosting LangGraph.js agents. LangGraph defines and
executes your graphs; Skein provides the API, persistence, streaming, background work, and console
around them.
A working graph still needs a backend before people can use it. Skein supplies that infrastructure
so you can ship an assistant, research tool, or approval workflow while keeping your graph code,
models, and tools. It is an Apache-2.0 alternative to LangGraph Platform (now LangSmith Deployment)
that you run on your own infrastructure.
## Quick start {#start}
Use **Node 24** (recommended):
```bash
npm create skein-js@latest my-agent
cd my-agent
npm run dev
```
Accept the scaffolder's defaults to install dependencies and try an echo graph that needs no API key,
database service, or Docker. Open `http://localhost:2024/console` and send a message.
[Your first agent](./your-first-agent.md) takes you through adding a model and tools, then deploying.
Skein is **pre-alpha**; see the [roadmap](./roadmap.md) for support and known gaps.
### Already using LangGraph?
Install the CLI in your project with `npm install -D skein-js`, then replace `langgraph dev` with
`skein dev`. Keep your `langgraph.json`, graph exports, and LangGraph SDK clients.
[CLI compatibility](./langgraph-cli-compat.md) documents supported commands, fields, and limitations.
## Run it your way
| Path | Use it for | Guide |
| --------------------- | ------------------------------------------------------------------------ | ----------------------------------------- |
| **Standalone server** | A shared service with Postgres and Redis | [Server deployment](./deploy.md) |
| **Existing backend** | Graphs alongside your Express, Fastify, NestJS, or Next.js routes | [Embedding](./embedding.md) |
| **Local application** | A desktop app, CLI, or daemon with one Node process owning a SQLite file | [Local deployment](./local-deployment.md) |
Each path uses the same Agent Protocol engine. The default development server uses memory drivers;
choose durable storage when you need persistence beyond development.
## See it running {#see-it}
The console is served by your own process at `/console`. Test a graph, follow live runs, review
approvals, and browse memory and schedules. It is enabled in development and off by default in
production. [Take the tour](./console.md).
<div class="light-only">
</div>
<div class="dark-only">
</div>
## Where to go next
Browse [features](./features.md) for capabilities, [recipes](./recipes/index.md) for practical
patterns, or [examples](https://github.com/skein-js/skein-js/tree/main/examples) for runnable
projects. Connect traces and analytics with [observability](./observability.md).
LangChain Deep Agents support is **experimental**: see the
[Deep Agents recipe](./recipes/deep-agents.md).
---
<!-- source: docs/why.md -->
# Why skein-js
You can build an agent in an afternoon. Putting it in front of real users is the part nobody warns
you about. This page is the honest version of how that goes, what your options are, and where
skein-js fits — including where it doesn't.
The short version: **LangGraph.js defines and executes the agent workflows. skein-js is the
self-hosted server runtime around them.** You can host several task-specific graphs behind one
server, with conversations, streaming, background work, and a console already wired up.
That runtime can also be embedded in a local application. Choose SQLite for a desktop app, CLI,
daemon, or one Node owner; choose Postgres/Redis for a shared server. Both reuse the existing
Agent Protocol and LangGraph execution. [Local deployment and examples →](./local-deployment.md)
## How everyone gets here
<ol class="skein-steps">
<li>
**You write an agent, and it works.** Twenty lines of LangGraph. It answers questions, calls a tool,
returns a result. You demo it and people are impressed.
</li>
<li>
**Someone asks to use it.** So you wrap it in an HTTP handler. Fine — an hour's work.
</li>
<li>
**"Can it remember what I said?"** Now you need conversations that persist, keyed to a user, restored
on the next request. That's a schema, a migration, and a serialization format for graph state.
</li>
<li>
**"Why does it take nine seconds to say anything?"** It doesn't — it's just not streaming. So you add
server-sent events. Then a client drops mid-answer and you add reconnect-and-replay.
</li>
<li>
**"Legal needs to approve before it sends."** Your agent has to _stop_, wait for a human who might
answer tomorrow, and resume from exactly where it paused — across a deploy. That's checkpointing.
</li>
<li>
**"Run it every morning."** A scheduler. Which must not fire twice when you scale to two servers.
</li>
<li>
**"Tell our CRM when it's done."** A webhook. Which must not be lost when the receiver is redeploying,
so now you own a retry queue and an outbox.
</li>
<li>
**Something breaks at 2am** and you have no idea which conversation, which step, or what the agent
actually saw.
</li>
</ol>
None of that was your product. All of it is load-bearing. It's a few thousand lines you'd own
forever, and every item has a failure mode you find in production rather than in review.
**skein-js is that entire list, already built, on your own infrastructure.**
## Your four options
<div class="skein-cards">
<div class="skein-card">
<span class="ico">🔨</span>
### Build it yourself
Total control, no dependency. Also the few thousand lines above, plus the ongoing cost of keeping up
with a protocol other people's clients expect.
</div>
<div class="skein-card">
<span class="ico">💳</span>
### LangGraph Platform
The managed option, and genuinely good. Paid per seat plus metered compute; production self-hosting
is an Enterprise add-on requiring a commercial licence key.
</div>
<div class="skein-card">
<span class="ico">🐍</span>
### aegra
The leading _open_ self-hosted alternative, and the project that inspired this one. Excellent — and
Python only, so a TypeScript team runs a second language in production.
</div>
<div class="skein-card">
<span class="ico">🧵</span>
### skein-js
Open source, self-hosted, TypeScript. Your database, your servers, no licence key, no per-run bill —
and a one-word path back out if you change your mind.
</div>
</div>
## How they compare
| | LangGraph Platform | aegra | **skein-js** |
| ----------------------------- | -------------------------- | ----------- | ------------------------------------------------ |
| Language | Python + JS | Python only | **TypeScript** |
| Licence | Elastic License 2.0 | Apache-2.0 | **Apache-2.0** |
| Self-host in production | Enterprise add-on + key | ✅ | ✅ **no key** |
| Cost | Per-seat + metered compute | Free | **Free** |
| Your own database | Managed, or Enterprise | ✅ | ✅ |
| Studio-style UI | Hosted, needs an account | — | ✅ **yours, at `/console`** |
| Drop-in for the LangGraph CLI | — | Partial | ✅ **one word** |
| HTTP framework | Theirs | FastAPI | **Express · Fastify · NestJS · Next.js · Fetch** |
## Your stack is already TypeScript
Look at what you've already got. The frontend is TypeScript. The API is TypeScript. The types are
shared across them, the CI is one pipeline, the team reviews each other's code. Then the agent shows
up, and it's the one component anyone suggests you write in a different language.
That's a real cost, and it's usually undercounted. The expensive part isn't the extra runtime — it's
that **nothing crosses the boundary**:
- **Types stop dead.** The agent's input and output are the shapes your app cares most about, and
they become a Zod schema on one side and a Pydantic model on the other, kept in step by hand. You
find out they've drifted in production, not at compile time.
- **You can't reuse what you've already written.** The validation, the domain rules, the permission
checks, the money and date formatting — your API has all of it, and the agent can reach none of it.
So you duplicate it, and the two copies drift; or you put it behind HTTP, and the agent makes a
network call to your own API to ask what a customer's plan is.
- **The agent's output is a UI contract, and it's the one that hurts most.** A node returns a
structured result — a booking, a diff, a set of options, a chart spec — and a component in your app
renders it. In one language that's a single exported type shared by the node that produces it and
the component that draws it, with the compiler catching a mismatch before you ship. Across two it's
a shape declared in Pydantic, re-declared in TypeScript, and rendered on trust. Same story for a
tool's schema: one Zod object can validate the model's arguments _and_ generate the form your UI
shows.
- **Two of everything around the code.** Two dependency trees to patch, two test runners, two CI
pipelines, two Dockerfiles, two streams of security advisories, two ways to configure a logger.
- **Review narrows.** A team fluent in TypeScript reviewing Python is working outside its
day-to-day, so in practice one or two people end up owning the agent. That shows up as slow reviews
long before it shows up as an incident.
- **A smaller pool of people who can fix it.** At 2am the question isn't which language is nicer,
it's who on the team can read the stack trace.
**The honest counterpoint:** Python's ML ecosystem is deeper, and that's not close. If your agent
does real numerical work, or reaches a model or library that only exists in Python, pay the cost —
it's worth it, and [aegra](https://github.com/aegra/aegra) is very good.
But that isn't the common case. The common case is an LLM behind an HTTP API, calling tools that are
your own services. Every major model provider ships a first-class TypeScript SDK, LangGraph has a
JavaScript implementation, and the whole thing can be one language, one deploy, one set of types.
Agent tooling in JavaScript has lagged the Python side — not because the ecosystem is smaller, but
because the infrastructure kept getting built there first. skein-js is one piece of closing that gap:
the production server was the missing part, so we built it, in the open, for the language most
product teams already ship.
## The licensing part, plainly
You _can_ self-host LangGraph Platform (now LangSmith Deployment). But production self-hosting is an
**Enterprise add-on that requires a commercial licence key** — the platform's server runtime is
source-available under the
[Elastic License 2.0](https://www.elastic.co/licensing/elastic-license), which is not an open-source
licence. The managed **Plus** plan is **$39 / seat / month**, includes one small serverless
deployment, and meters additional compute and storage on top. Fully self-hosted and hybrid deployment
are Enterprise-only, with custom pricing.
_As of August 2026 — see [langchain.com/pricing](https://www.langchain.com/pricing) and the
[self-hosting docs](https://docs.langchain.com/langsmith/self-hosted). Always verify current terms._
If you're a solo developer, a small team, or just trying an idea, that model is a poor fit. You
shouldn't need a commercial licence or a per-run bill to ship a side project.
## The same stack in dev and production
With a managed platform, local development is an _approximation_ of production. You develop against a
dev server, ship to someone else's service, and the gap between the two is where the surprises live —
because the thing you deploy to is a thing you cannot run yourself.
Self-hosted and open source removes that gap entirely:
```bash
skein dev --store postgres --queue redis # the real drivers, on your machine
skein up # app + Postgres + Redis, via Compose
skein build # the image you deploy — `skein start` runs inside it
```
The same checkpointer, the same migrations, the same BullMQ queue semantics, the same code path. When
something misbehaves in production you can reproduce it locally, because there is no privileged
environment you're locked out of.
And when reproducing isn't enough, you can read the server — all of it, Apache-2.0 — set a breakpoint
in it, and patch it. That's not a small thing at 2am.
## We want leaving to be easy
This is the part that's easy to leave unsaid.
skein-js is a **drop-in for the LangGraph CLI on an unchanged `langgraph.json`**. That's usually
pitched as "migration is one word" — and it is. But the same fact runs in both directions: if your
team grows, you want SLAs and managed operations, and LangGraph Platform starts making sense, then
**switching back is also one word**.
That's deliberate. Low lock-in in both directions means you can start free, ship something real, and
adopt a managed platform if and when it's actually worth paying for — instead of choosing your
production architecture on day one, under uncertainty, and living with it.
It's also the standard we hold ourselves to: we'd rather you stay because skein-js keeps being the
right fit than because leaving got expensive.
## Won't it drift out of compatibility?
The reasonable worry about any community project tracking a commercial one: it falls behind, breaks
subtly, and strands you.
That isn't the shape of this, because **compatibility here isn't maintained by hand**. The wire
format is `@langchain/langgraph-sdk`'s own types — the SDK's types _are_ the contract, so `useStream`
works by construction rather than because someone keeps a copy in sync. Your agent runs on
LangGraph's runtime, saves through its checkpointers, and your `langgraph.json` is read by its
parser.
That reuse is possible because on JavaScript the pieces are **MIT-licensed**, including the Agent
Protocol server internals
([`@langchain/langgraph-api`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/langgraph-api)).
skein-js wraps them; it doesn't clone them, so there's no second implementation to fall behind.
skein-js builds only what open source genuinely lacks: durable production storage and queueing, the
framework adapters, the console, and the drop-in CLI. The package-by-package ledger of what's reused
versus rebuilt is in
[reuse.md](https://github.com/skein-js/skein-js/blob/main/docs/reuse.md).
## Why we bet on LangGraph
skein-js is built for LangGraph.js, and that isn't incidental — it's the choice the whole project
rests on. Here's the reasoning, because if you're adopting skein you're adopting that bet too.
The short version: it isn't a new or niche framework. It's LangChain's, it's MIT, and it pulls
[millions of npm downloads a week](https://www.npmjs.com/package/@langchain/langgraph) — so the part
of your stack doing the actual agent work is the well-trodden part.
**Control flow is explicit.** You break the agent into discrete steps — nodes — connected by a state
they each read and write, and routing is a decision you can point at. As LangChain's own
[Thinking in LangGraph](https://docs.langchain.com/oss/javascript/langgraph/thinking-in-langgraph)
puts it, "you can always understand what your agent will do next by looking at the current node."
That sounds academic until you need to pause halfway through, resume after a deploy, or ask what the
agent saw three steps ago — at which point an explicit graph is the difference between a feature and
a rewrite.
**Its persistence primitives are exactly what a server needs.** Checkpointers aren't a nice-to-have
we work around; they're the substrate. LangGraph writes a checkpoint at node boundaries, and
human-in-the-loop, time travel, resumable runs and crash recovery all fall out of that one fact — a
run that stops resumes from the node it stopped in, days later, on a different machine. A framework
without it would make most of this page impossible to build.
**The client ecosystem already exists.** `useStream`, the SDK, Agent Chat UI, Studio — all of it
speaks one protocol, and the wire types are the SDK's own. Betting on LangGraph means your users'
frontends work by construction rather than through a compatibility layer someone maintains.
**And the graph is portable.** Nothing in your graph code is skein-specific — no imports, no
decorators, no lifecycle hooks. It's the same file that runs under `langgraph dev` or on LangGraph
Platform. That's what makes [leaving easy](#we-want-leaving-to-be-easy) in both directions.
### So, am I locked in?
To LangGraph, **today, mostly yes** — and it's worth saying plainly rather than overselling a seam.
Your agent runs on LangGraph's runtime, state persists through its checkpointers, `langgraph.json`
uses its parser. It isn't merely the supported path; it's the complete one.
What _is_ decoupled is narrower:
- **The protocol engine carries no graph runtime.** `@skein-js/agent-protocol` installs without
LangGraph or LangChain, and a test pins that. The LangGraph binding is a separate package using
only the engine's public entry point.
- **The runner seam is real, but partial.** The engine drives an `AgentGraph` — `stream` and
`getState` required, the rest optional — so another runtime _can_ serve the protocol. It would
bring its own persistence, and lose what's LangChain-specific by construction: `events` stream mode
is a LangChain demux, and time travel needs `updateState` + `getStateHistory` you'd write yourself.
The limits are listed honestly in [building a runner](./building-a-runner.md#known-limits).
- **Your clients were never tied to skein.** They speak the Agent Protocol, so on that side the thing
you'd migrate is a URL.
Practically: if you're not writing LangGraph.js graphs, skein-js probably isn't for you yet.
What we deliberately **don't** do is argue LangGraph beats every alternative — that's LangChain's
case to make, and their [docs](https://docs.langchain.com/oss/javascript/langgraph/overview) make it.
The above is why _we_ built on it.
## Who this is for
<div class="skein-cards">
<div class="skein-card">
### A good fit
TypeScript teams who want their agents and their data on their own infrastructure · anyone whose
compliance story rules out a managed control plane · developers who want to ship a side project
without a per-run bill · teams already on the LangGraph CLI who need a production story.
</div>
<div class="skein-card">
### Probably not for you
Python-first teams — use [aegra](https://github.com/aegra/aegra), it's good · teams who want managed
operations, SLAs and a vendor to call — that's what LangGraph Platform is for · anyone who wants
`skein deploy` to a hosted platform, which is a deliberate non-goal.
</div>
</div>
## What we deliberately don't do
Being honest about the edges is part of the pitch:
- **No hosted platform.** Self-hosted by design; there's no managed target to deploy to.
- **No WebSocket transport.** Server-sent events cover the client experience and the React SDK
doesn't care.
- **No sub-minute schedules**, and no backfilling schedules missed during an outage.
- **No exactly-once webhook delivery.** Delivery is durable — the callback commits in the same
transaction as the run's terminal status, and is retried until it lands. What it is not is
exactly-once, which is why every attempt carries a stable dedup key.
- **We don't restate LangGraph's docs.** We document our _conformance_ and the delta.
The full list of what's shipped, in preview, and planned is on the [features page](./features.md) and
the [roadmap](./roadmap.md).
## Convinced, or curious?
```bash
npm create skein-js@latest my-agent
cd my-agent && npm run dev
```
No API key, no database, no Docker. [Your first agent](./your-first-agent.md) takes it from there —
or if you already have a LangGraph project, [change one word](./langgraph-cli-compat.md).
---
<!-- source: docs/your-first-agent.md -->
# Your first agent
From an empty directory to a deployed agent server. No LangGraph experience assumed — the parts of
it you need are explained here, where you need them.
If you already have a LangGraph.js graph or a `langgraph.json`, you want
[getting-started.md](./getting-started.md) instead: there is nothing to scaffold, and adopting skein
is a one-line change.
**Prerequisites:** Node 24 is recommended, with Node 22.12+ within 22.x also supported, and a package
manager. Nothing in this guide needs an API key until you choose a model, and nothing needs Docker
until deployment. For a workspace, start with the [monorepo guidance](./scaffolding.md#nx-and-other-monorepos).
## 1. Create the project
```bash
npm create skein-js@latest my-agent
```
`pnpm create skein-js@latest my-agent` and `yarn create skein-js@latest my-agent` do the same thing.
Keep `@latest` to request the latest published scaffolder.
Interactive runs ask about the model provider, ESLint and Prettier, local storage, dependency
installation, and git initialization. **Pick "None" for the provider**, keep memory storage and
ESLint/Prettier enabled, and accept dependency installation. The directory is already supplied above;
it is prompted when omitted. `--yes` accepts defaults without prompting.
You can add a model in a minute. The generated echo graph runs without credentials.
Then:
```bash
cd my-agent
npm run dev
```
You now have an agent server on `http://localhost:2024`.
## 2. Look at it before you read any code
Open `http://localhost:2024/console`.
That's the **skein console**, served by your own process — no account, no hosted service, no tunnel.
Create a thread, send "hello", and watch the run execute. You will get `echo: hello` back, because
the graph you just created echoes its input.
This is worth doing before anything else: everything below is about changing what happens between
your message and that reply, and it helps to have seen the loop close.
## 3. What you actually got
The project includes tests, default ESLint/Prettier tooling, and a `.nvmrc` selecting a supported
Node version. The three files to understand first:
**`src/echo-graph.ts`** — your agent. We'll come back to it.
**`langgraph.json`** — how skein finds your graphs:
```json
{
"node_version": "24",
"graphs": { "echo": "./src/echo-graph.ts:graph" },
"env": ".env"
}
```
`"./src/echo-graph.ts:graph"` means _the export named `graph` in that file_. The key — `echo` — is
what clients ask for by name. This is the same file format the LangGraph CLI reads, which is why
skein is a drop-in for it.
**`package.json`** — the commands that are the whole lifecycle:
| | |
| ---------------------- | ------------------------------------------------------------- |
| `npm run dev` | what you're running: hot reload, in-memory state, zero setup |
| `npm run dev:services` | Postgres + Redis in Docker, for the durable `dev` and `start` |
| `npm run dev:postgres` | the same hot reload, against those services instead |
| `npm run build` | compile your graphs to plain JavaScript in `.skein/` |
| `npm start` | serve that build — this is what production runs |
(If you picked Postgres at the storage prompt, `dev` is already the durable one and the in-memory
spelling is `dev:memory` instead — both are always there.)
The rest: `.env` (ready to use — the model key is commented out, the service URIs are not) and
`.env.example` (a committed reference copy of it), `compose.dev.yaml` (the services for `start`),
`tsconfig.json`, a test, and a README.
## 4. Understanding the graph
Here is the whole agent:
```ts
import { AIMessage, type BaseMessage } from "@langchain/core/messages";
import { MessagesAnnotation, StateGraph } from "@langchain/langgraph";
function echo(state: typeof MessagesAnnotation.State): { messages: BaseMessage[] } {
const last = state.messages.at(-1);
const text = typeof last?.content === "string" ? last.content : "";
return { messages: [new AIMessage(`echo: ${text}`)] };
}
export const graph = new StateGraph(MessagesAnnotation)
.addNode("echo", echo)
.addEdge("__start__", "echo")
.addEdge("echo", "__end__")
.compile();
```
Four ideas, and they are the only LangGraph concepts you need to get started:
**State** is the data flowing through your agent. `MessagesAnnotation` is a ready-made state that
holds a list of chat messages and knows to _append_ new ones rather than replace the list. Most
conversational agents want exactly this.
**A node** is a function. It receives the current state and returns only the part that changed — here,
one new message. It is a plain function: you can call it, test it, and step through it in a debugger.
**Edges** say what runs next. `__start__` and `__end__` are the built-in entry and exit points, so
this graph is "start → echo → done." Real agents branch here — that is where the "graph" part earns
its name.
**`.compile()`** turns the definition into something runnable. What it returns is what skein serves.
Try it: change `echo:` to `you said:` in `src/echo-graph.ts` and save. The server hot-reloads and
**keeps your existing threads** — send another message in the console and you will see the new reply
in the same conversation.
Those four ideas are enough to finish this guide. When you want the rest — custom state, branching,
subgraphs, `interrupt()` — [LangGraph essentials](./langgraph-essentials.md) is the short version,
with a link out to the LangChain docs on each one.
## 5. Give it a real model
Now swap the echo for an LLM. Install a provider:
```bash
npm install @langchain/anthropic
```
Create `src/agent-graph.ts`:
```ts
import { ChatAnthropic } from "@langchain/anthropic";
import { createAgent } from "langchain";
const model = new ChatAnthropic({ model: "claude-sonnet-5", temperature: 0 });
export const graph = createAgent({ model, tools: [] });
```
`createAgent` is LangChain's prebuilt agent loop: call the model, and if it asks for a tool, run the
tool and call the model again. You did not have to build that loop. It lives in the `langchain`
package — LangGraph v1 deprecated its own `createReactAgent` and moved it there.
Register it in `langgraph.json`:
```json
{
"graphs": {
"echo": "./src/echo-graph.ts:graph",
"agent": "./src/agent-graph.ts:graph"
}
}
```
Then give it a key — `.env` is already there, with every line commented out:
```bash
# in .env, set ANTHROPIC_API_KEY= — from https://console.anthropic.com/settings/keys
```
Restart, pick `agent` in the console, and you are talking to a real model. Tokens stream as they are
generated.
> Prefer to skip this assembly? `npm create skein-js@latest my-agent --provider anthropic` scaffolds
> all of it, plus a working tool, in one step.
## 6. Give it a tool
Talking is half of it. Tools are what let an agent actually do things:
```ts
import { tool } from "@langchain/core/tools";
import { z } from "zod";
const getWeather = tool(
async ({ city }: { city: string }) => {
const geo = await fetch(
`https://geocoding-api.open-meteo.com/v1/search?name=${encodeURIComponent(city)}&count=1`,
).then((response) => response.json());
const place = geo.results?.[0];
if (!place) return `I couldn't find ${city}.`;
const forecast = await fetch(
`https://api.open-meteo.com/v1/forecast?latitude=${place.latitude}` +
`&longitude=${place.longitude}¤t=temperature_2m`,
).then((response) => response.json());
return `It's ${forecast.current.temperature_2m}°C in ${place.name}.`;
},
{
name: "get_weather",
description: "Get the current weather for a city.",
schema: z.object({ city: z.string().describe("City name, e.g. 'Nairobi'") }),
},
);
```
Pass it in — `createAgent({ model, tools: [getWeather] })` — and ask "what's the weather in
Nairobi?". The console shows the model deciding to call the tool, the result coming back, and the
final answer. (Open-Meteo needs no API key of its own.)
The `description` and `schema` are how the model knows when to use it. Write them for a reader who
has no other context, because that is exactly the model's situation.
## 7. Talk to it from your own code
The server speaks the standard Agent Protocol, so the official SDK works — there is no skein client:
```ts
import { Client } from "@langchain/langgraph-sdk";
const client = new Client({ apiUrl: "http://localhost:2024" });
const thread = await client.threads.create();
for await (const event of client.runs.stream(thread.thread_id, "agent", {
input: { messages: [{ role: "user", content: "what's the weather in Nairobi?" }] },
})) {
console.log(event.event, event.data);
}
```
For a UI, the `useStream` React hook talks to the same server with only a URL change — see
[react-sdk.md](./react-sdk.md). Vue, Svelte and Angular work too; so do
[Agent Chat UI](https://github.com/langchain-ai/agent-chat-ui) and LangGraph Studio.
## 8. Remember things between conversations
Threads persist a conversation. A **store** persists across conversations — what a user told you last
week. Any node can reach it:
```ts
import { getStore } from "@langchain/langgraph";
async function remember(state: typeof MessagesAnnotation.State) {
const store = getStore();
await store.put(["users", "alice"], "prefers", { units: "celsius" });
const saved = await store.get(["users", "alice"], "prefers");
// …use saved.value in your prompt
}
```
The default `dev` uses memory. A server can use Postgres with vector search for semantic recall;
a local application can use SQLite with text/filter search. Your node code does not change.
See [storage.md](./storage.md) and [memory.md](./memory.md).
## 9. Ship it
`dev` is not the production path. Production is `build` + `start`:
```bash
npm run dev:services # Postgres + Redis via Docker
npm run build # graphs → plain JavaScript in .skein/build
npm start
```
Nothing to edit first: the `POSTGRES_URI` and `REDIS_URI` in `.env` already match those services, and
`npm start` loads them itself.
`build` compiles your TypeScript graphs ahead of time; `start` serves that output with no TypeScript
toolchain in the loop. It is exactly what the production container runs.
The default `start` selects Postgres and Redis for durable state, queued work, and multiple
instances. It is one deployment option. For a desktop app, CLI, daemon, or single Node owner,
choose [SQLite](./local-deployment.md): scaffold with `--storage sqlite` to generate SQLite
dev/start scripts without Compose. The same graphs, checkpoints, and protocol are used; the
application decides whether to retry an abandoned attempt.
For a container, `npx skein up` brings up the whole stack, and `npx skein build` produces an image.
Then pick a host: [Cloud Run](./deploy-cloud-run.md), [Fly](./deploy-fly.md),
[Railway](./deploy-railway.md), [Render](./deploy-render.md), [AWS](./deploy-aws.md),
[Kubernetes](./deploy-kubernetes.md), or [a plain VPS](./deploy-vps.md). Full guide:
[deploy.md](./deploy.md).
## Where to next
**Read [building blocks](./building-blocks.md).** It is the map of everything this guide
skipped — checkpoints, threads, multitask, interrupts, memory, what production actually needs — a few
lines each, so you can tell which page you want before you need it.
If you would rather understand the graph model first,
[LangGraph essentials](./langgraph-essentials.md) covers state, reducers, branching, subgraphs and
`interrupt()`, and links out to the LangChain docs on each.
Then, as you need them:
- [Recipes](./recipes/) — auth, human-in-the-loop, background runs, CORS
- [The console](./console.md) — what else that UI does: time travel, interrupt approvals, crons
- [Agent Protocol](./agent-protocol.md) — every endpoint your server exposes
- [Scaffolding reference](./scaffolding.md) — every flag, and the Nx generators
---
<!-- source: docs/getting-started.md -->
# Getting started
A guided, end-to-end path to a running Agent Protocol server, then to production. Pick the path that
matches what you have right now:
- **Nothing yet** → [Path A](#path-a--starting-from-scratch): one command scaffolds a working project.
- **A `langgraph.json`** → [Path B](#path-b--i-have-a-langgraphjson-drop-in): a one-line drop-in.
- **A compiled graph in code** → [Path C](#path-c--i-have-a-graph-in-code-embed): wrap it in `deps`.
All three produce the **identical** server. For a terse reference instead of a walkthrough, see
[using-skein.md](./using-skein.md).
Deployment is a separate choice: keep the default memory drivers while developing, use
[Postgres/Redis](./deploy.md) for a shared server, or choose
[SQLite for a local app or one Node process](./local-deployment.md). The graphs and Agent Protocol
remain the same.
## Prerequisites
- **New projects (Path A):** Node 24 is recommended. Generated engines support `^22.12.0 || ^24.0.0 || ^26.0.0`.
- **Existing graphs (Paths B and C):** Node ≥ 20 and a package manager (`pnpm`/`npm`).
- **Node 22, 24 or 26** when selecting SQLite; desktop examples build with Node 24.
- For Paths B and C, a
**[LangGraph.js graph](https://docs.langchain.com/oss/javascript/langgraph/graph-api)** — a
`CompiledStateGraph` from `@langchain/langgraph`. Path A writes one for you.
## Path A — Starting from scratch
No graph, no config, no LangGraph experience. One command:
```bash
npm create skein-js@latest my-agent
cd my-agent
npm run dev # → http://localhost:2024, console at /console
```
You get a `langgraph.json`, a keyless echo graph you can edit, a test, and the whole
`dev` → `build` → `start` lifecycle wired up. ESLint and Prettier are enabled by default;
`--no-linting` opts out. The default memory development server needs no API key or database.
[**Your first agent**](./your-first-agent.md) walks through it end to end — what each generated file
does, the four LangGraph concepts you need, adding a model and a tool, then deploying.
[scaffolding.md](./scaffolding.md) is the flag-by-flag reference, and covers doing it by hand if you
would rather not run a scaffolder.
## Path B — I have a `langgraph.json` (drop-in)
If you already run `langgraph dev`, this is a one-line change. Keep your `langgraph.json` exactly as
it is:
```jsonc
// langgraph.json
{ "graphs": { "agent": "./src/agent.ts:graph" } }
```
Swap the CLI:
```diff
- "dev": "langgraph dev",
+ "dev": "skein dev",
```
```bash
pnpm add -D skein-js
pnpm dev # skein dev — in-process, hot-reload, on http://localhost:2024
```
`skein dev` loads your TypeScript graphs through vite (no separate loader), hot-reloads on save, and
persists dev state across restarts. `.env` counts as a save: filling in a model key there reloads the
graphs, so a graph that could not load for want of a credential starts working without a restart. This is the full LangGraph CLI surface —
[langgraph-cli-compat.md](./langgraph-cli-compat.md) documents every field and command.
Prefer to mount it inside your own Express/Fastify/Nest/Next app instead of the CLI? Point an adapter
at the same config — `{ config: "./langgraph.json" }` — using the snippets in
[Mount it on your framework](./using-skein.md#mount-it-on-your-framework).
## Path C — I have a graph in code (embed)
No `langgraph.json`, no CLI — bring the compiled graph you already hold and wrap it into a
`ProtocolDeps` with `embedInMemoryGraphs`, then hand `{ deps }` to any adapter:
```ts
// server.ts
import { createExpressServer } from "@skein-js/express";
import { embedInMemoryGraphs } from "@skein-js/server-kit";
import { graph } from "./agent.js"; // your CompiledStateGraph
const server = await createExpressServer({ deps: embedInMemoryGraphs({ agent: graph }) });
await server.listen(2024);
console.log("Agent Protocol on http://localhost:2024");
```
```bash
pnpm add @skein-js/express @skein-js/server-kit @langchain/langgraph
npx tsx server.ts # or your usual TS runner
```
All three paths produce the **identical** Agent Protocol server. See [embedding.md](./embedding.md) for the
graph-map/factory semantics and how `overrides` swaps in production drivers.
## Talk to your server
With the server running on `:2024`, drive it with the standard SDK — no skein-specific client:
```ts
import { Client } from "@langchain/langgraph-sdk";
const client = new Client({ apiUrl: "http://localhost:2024" });
const thread = await client.threads.create();
const reply = await client.runs.wait(thread.thread_id, "agent", {
input: { messages: [{ role: "user", content: "hello" }] },
});
console.log(reply);
```
Stream tokens as they arrive instead of waiting:
```ts
for await (const event of client.runs.stream(thread.thread_id, "agent", {
input: { messages: [{ role: "user", content: "hello" }] },
})) {
console.log(event.event, event.data);
}
```
`"agent"` is the `assistant_id`, which defaults to the `graph_id`. The full endpoint surface is in
[agent-protocol.md](./agent-protocol.md); the stream wire format is in [streaming.md](./streaming.md).
## Add a web UI
The `useStream` React hook streams over the same server — point it at your URL:
```tsx
import { useStream } from "@langchain/react";
function Chat() {
const thread = useStream({ apiUrl: "http://localhost:2024", assistantId: "agent" });
return (
<button onClick={() => thread.submit({ messages: [{ role: "user", content: "hi" }] })}>
Send
</button>
);
}
```
A browser on a different origin needs CORS enabled — see the
[CORS recipe](./recipes/serving.md#cors-for-a-browser-client). For a same-origin full-stack app (no CORS), the
[`nextjs-app`](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app) example serves the protocol and the UI from one Next.js app.
See [react-sdk.md](./react-sdk.md).
## Go to production
The default dev server uses in-memory drivers. For durability and horizontal scale, swap in Postgres (state +
checkpoints) and Redis (run queue + cross-instance streaming). Nothing else about your server
changes — only how `deps` is built.
```ts
// embed path → durable, reading POSTGRES_URI / REDIS_URI from the environment
import { embedPostgresGraphs } from "@skein-js/runtime";
import { createExpressServer } from "@skein-js/express";
import { graph } from "./agent.js";
const { deps, dispose } = await embedPostgresGraphs({ agent: graph });
const server = await createExpressServer({ deps });
await server.listen(2024);
process.on("SIGTERM", async () => {
await server.close();
await dispose(); // release the pools it opened
process.exit(0);
});
```
From a `langgraph.json`, either call `buildRuntime({ store: "postgres", queue: "redis" })` and pass its
`deps` **and** `channels` to the adapter, or skip code entirely: `skein dev --store postgres --queue
redis`, and `skein build` / `skein up` to containerize. Redis is optional for a single instance but
required to run more than one. Details:
[embedding.md](./embedding.md#going-to-production), [storage.md](./storage.md),
[runs-and-redis.md](./runs-and-redis.md), [deploy.md](./deploy.md).
### Or choose SQLite for a local application
Scaffold with `npm create skein-js@latest my-app -- --storage sqlite` for SQLite dev/start scripts,
or install `@skein-js/local` and the upstream SQLite saver in an existing project:
```bash
pnpm add @skein-js/local @langchain/langgraph-checkpoint-sqlite@~1.0.4
pnpm skein dev --store sqlite --queue sqlite
```
The CLI defaults to `.skein/state.sqlite`. For an Electron utility process, a Tauri Node backend,
or a CLI using graphs in code, use `embedSqliteGraphs` and let the host own startup and shutdown.
See [local deployment](./local-deployment.md) for embedding, examples, and failure accounting.
## Where to next
- [Your first agent](./your-first-agent.md) — the from-zero walkthrough, if you took Path A.
- [Recipes](./recipes/) — auth, human-in-the-loop, long-term memory, CORS, background runs, deploy.
- [Using skein-js](./using-skein.md) — the terse consumer/agent cheat-sheet.
- [Examples](https://github.com/skein-js/skein-js/tree/main/examples) — a runnable project per framework and pattern.
- [Overview & architecture](./index.md) · [Agent Protocol surface](./agent-protocol.md)
---
<!-- source: docs/scaffolding.md -->
# Scaffolding a project
`create-skein-js` generates a working skein-js project from an empty directory. It is the fastest
path from nothing to a running agent server, and it is optional — everything it emits is a file you
could write yourself, and the last section shows you exactly that.
For a guided walkthrough rather than a reference, start with
[your first agent](./your-first-agent.md).
## Quick start
```bash
npm create skein-js@latest my-agent
```
```bash
pnpm create skein-js@latest my-agent
yarn create skein-js@latest my-agent
npx create-skein-js@latest my-agent
```
**Keep the `@latest`** to request the latest published scaffolder instead of relying on a cached
version. Node 24 is recommended; generated projects support `^22.12.0 || ^24.0.0 || ^26.0.0`.
For an existing workspace, choose a new child directory such as `apps/my-agent`. Use the
[Nx generator](#nx-and-other-monorepos) in Nx workspaces.
## What it generates
The default storage choice is `memory`. It keeps development simple and supplies Postgres/Redis
services for the default production `start`. Choose `--storage sqlite` for a local application;
its generated scripts and files differ as described below.
```text
my-agent/
├── langgraph.json Points skein at your graphs; the LangGraph CLI's format
├── package.json The skein CLI lifecycle as scripts
├── tsconfig.json Strict, ESM, bundler resolution
├── .nvmrc Recommended Node version, or a compatible workspace pin
├── eslint.config.mjs Recommended rules, unless inherited or tooling is disabled
├── .prettierrc.json printWidth: 100, unless inherited or tooling is disabled
├── .prettierignore Generated output, runtime state, dependencies and lockfiles
├── vitest.config.ts
├── compose.dev.yaml Postgres + Redis — what `start` needs
├── .env Ready to use, gitignored — never overwritten if one already exists
├── .env.example A committed reference copy of it
├── .gitignore
├── README.md Explains each of these files
└── src/
├── echo-graph.ts Runs with no API key, no network
├── echo-graph.test.ts So `npm test` is green on the first commit
└── agent-graph.ts Only with --provider: a ReAct agent with a working tool
```
The scripts are the whole skein CLI lifecycle:
| Script | Command | What it does |
| -------------- | ------------------------------------------------- | ----------------------------------------------------------- |
| `dev` | `skein dev --port 2024` | In-memory drivers, hot reload, state persisted to `.skein/` |
| `dev:services` | `docker compose -f compose.dev.yaml up -d --wait` | Postgres + Redis, for the durable `dev` and for `start` |
| `dev:postgres` | `skein dev … --store postgres --queue redis` | The same hot reload, against the drivers production uses |
| `build` | `skein build --artifact-only` | Graphs → plain JavaScript in `.skein/build` |
| `start` | `skein start -c .skein/build/langgraph.json` | Serve that build — the production entrypoint |
| `typecheck` | `tsc --noEmit` | |
| `test` | `vitest run` | Run the generated echo graph test |
| `lint` | `eslint src --max-warnings=0` | ESLint, enabled by default |
| `format` | `prettier --write .` | Apply formatting, enabled by default |
| `format:check` | `prettier --check .` | Check formatting, enabled by default |
Those are the default memory scripts. Pick Postgres and `dev` becomes the Postgres/Redis command,
with the in-memory alternative emitted as `dev:memory` instead of `dev:postgres`.
`dev:services` uses `--wait` so it returns once both healthchecks pass, not merely once the containers
exist — otherwise the very next command races first-boot `initdb`.
`start` names the artifact's own `langgraph.json` because `skein start` serves a _build_, not a source
project: it wants `schemas.json` beside the config it loads, and that file only exists in
`.skein/build`. Run from the project root like this, it also picks up the `POSTGRES_URI` and
`REDIS_URI` from the project's `.env` — `skein start` reads a conventional `.env` from its working
directory as well as from the config's, because an artifact deliberately carries none of its own (it
is the Docker build context). So `dev:services && build && start` works with nothing to edit first.
Two deliberate choices worth knowing:
- **The default `dev` needs no credential or service.** The `echo` graph is always present and
first in `graphs`, so the first request works with an empty `.env` and memory drivers.
- **Memory and Postgres projects include `compose.dev.yaml`**, and the `POSTGRES_URI` /
`REDIS_URI` in `.env` are live rather than commented out. The default `skein start` selects
`--store postgres --queue redis` and requires those services, so a project shipping that
`start` script has to ship both the services and the URIs that reach them, or the script is a trap.
The values are the ones the generated compose file serves, so there was never anything to decide.
### SQLite projects
```bash
npm create skein-js@latest my-app -- --storage sqlite
cd my-app
npm run dev
npm run build
npm start
```
SQLite projects install `@skein-js/local`, LangGraph, and its SQLite saver. Both `dev` and `start`
select `--store sqlite --queue sqlite`, with the database at `.skein/state.sqlite`; `dev:memory`
keeps the memory alternative. There is no Compose file, service-start script, or Postgres/Redis
environment configuration. Graphs still build to `.skein/build` and the database stays outside it.
Use Node 24 (recommended), Node 22.12+ within 22.x, or Node 26. See [local deployment](./local-deployment.md) for embedding and desktop examples.
## Options
```text
create-skein-js [directory]
-m, --provider <name> none | google | anthropic | openai (default: prompted, else none)
--storage <name> memory | postgres | sqlite (default: prompted, else memory)
--pm <name> npm | pnpm | yarn | bun (default: detected)
--linting / --no-linting ESLint and Prettier (default: yes)
--no-workspace-update Leave workspace registration manual
--no-install Skip installing dependencies (else: prompted, default yes)
--no-git Skip initializing a git repository (else: prompted)
-y, --yes Accept every default; never prompt
-f, --force Scaffold into a directory that is not empty
-v, --version
-h, --help
```
**Passing flags through npm** needs a `--` separator. `pnpm create` and `npx` do not:
```bash
npm create skein-js@latest my-agent -- --provider anthropic
pnpm create skein-js@latest my-agent --provider anthropic
```
### `--provider`
| | Package added | `.env.example` gains | Emits `agent-graph.ts`? |
| ------------------ | ------------------------- | -------------------- | ----------------------- |
| `none` _(default)_ | — | — | no |
| `google` | `@langchain/google-genai` | `GOOGLE_API_KEY` | yes |
| `anthropic` | `@langchain/anthropic` | `ANTHROPIC_API_KEY` | yes |
| `openai` | `@langchain/openai` | `OPENAI_API_KEY` | yes |
With a provider you also get a ReAct agent wired to a live weather tool that needs no key of its own,
so the agent is genuinely runnable the moment you add your model key.
Until the key is set, `agent` fails to load naming the variable it wants — a
[load-failure block](./errors-and-logging.md#the-load-failure-block), while `echo` keeps serving.
`skein dev` watches `.env`, so filling the key in takes effect on save.
### Behaviour you can rely on
- **It never hangs unattended.** Prompts appear only when both streams are a TTY, `--yes` was not
passed, and `CI` is unset. Otherwise it takes the flag, then the default — safe inside a Dockerfile
or a CI job.
- **A failed install is not fatal.** Your files are already written; the closing output just adds
`install` back to the steps.
- **Scaffolding into a fresh clone works.** A directory holding only `.git`, `LICENSE`, editor
folders or `.DS_Store` counts as empty, so "create an empty repo, clone it, scaffold into it" needs
no `--force`.
- **git is skipped inside an existing work tree**, so it never nests a repository in yours — and the
closing output says so, rather than leaving you to infer it from a missing `.git`. A failure (git
absent, or no `user.email` configured) is reported too, and distinctly: the two used to be the same
silent non-event.
- **The version is pinned to a matching runtime.** Because every `packages/*` shares one version,
`create-skein-js@x.y.z` pins `skein-js@^x.y.z` — the scaffolder and the runtime it scaffolds are
always the same release.
## Tooling and Node versions
ESLint and Prettier default to enabled in both entrypoints. The CLI asks **Y/n** interactively;
`--linting` and `--no-linting` bypass that prompt. `--yes`, CI and non-TTY runs enable tooling.
The Nx option `linting` defaults to `true`; use `--linting=false` to opt out.
Projects provide `lint`, `format`, and `format:check`. Missing tooling uses recommended JavaScript
and TypeScript ESLint rules and Prettier with `printWidth: 100`. Existing root ESLint and Prettier
configurations are inherited independently. Inherited tooling uses a generated `scripts/skein-tools.mjs`
helper; a standalone project uses the commands shown above. Generated output, dependencies, runtime
state and package-manager lockfiles are ignored. Missing tool dependencies are added independently
of inherited configuration, and new files follow workspace formatting; see the
[workspace tooling and caching](#workspace-tooling-and-caching).
The shared policy recommends **Node 24** with engines `^22.12.0 || ^24.0.0 || ^26.0.0`. It derives `.nvmrc`
and `langgraph.json.node_version` from the same selection. Compatible numeric workspace pins in
`.nvmrc` or `.node-version` are honored. Unsupported numeric pins fail before writing; unresolved
aliases such as `lts/*` produce an explanation and fall back to Node 24. Node 26 is an additional
supported runtime; production defaults and desktop packaging remain on Node 24. Yarn users on Node 26
must install Corepack separately because it is no longer bundled with Node.
## Nx and other monorepos
For Nx **23.2.1 through 23.x**, install the generator at the root:
```bash
pnpm add -Dw @skein-js/nx
pnpm exec nx g @skein-js/nx:application --directory=apps/my-agent --storage=sqlite
pnpm install
pnpm exec nx dev apps-my-agent
```
Options are `directory` (required), `provider`, `storage`, and `linting`.
The generator supports integrated workspaces (root dependencies) and package workspaces (application
dependencies). It extends the TypeScript base, preserves root lifecycle scripts and leaves installation
manual. Use `--dry-run` to preview without writing. This replaces issue #19's earlier decision to
document manual wiring only; existing projects can retain their targets.
Targets use existing `nx:run-commands`. `start` depends on `build`, live targets are uncached,
and only `.skein/build` is a build output. SQLite state and WAL/SHM files stay outside cached
inputs and outputs. Retain explicit artifact selection and storage flags, for example
`skein start -c .skein/build/langgraph.json --store sqlite --queue sqlite`.
When a diagnostic suggests a config, keep the original command's other options, including custom
`--sqlite-path` settings.
For npm, pnpm, Yarn, Bun, Turborepo or Lerna workspaces:
```bash
npm create skein-js@latest apps/my-agent
```
The CLI retains that directory, detects the workspace and skips installation, including with `--yes`.
Install dependencies from the workspace root after generation. Missing registration
is prompted with a yes default; `--yes` accepts and `--no-workspace-update` leaves manual
instructions. Existing globs, exclusions, YAML comments, key order, versions and unrelated settings are
preserved. A pnpm root without `pnpm-workspace.yaml` gets one containing its existing `package.json`
workspace patterns and exclusions, since pnpm ignores `package.json` `workspaces`. Invalid declared
workspaces are diagnosed even when another manifest field is invalid. Excluded paths, invalid root
metadata, duplicate names and package-manager conflicts fail before writing.
Workspace `packageManager` metadata takes precedence over lockfile detection. Compatible root or
pnpm catalog ranges are reused, then unanimous compatible sibling ranges. Remaining dependencies use
tested defaults and report differences. Integrated Nx workspaces reject root dependency conflicts
without changing existing versions. For new child directories in Nx workspaces the CLI directs you
to the generator before writing.
### Workspace tooling and caching
Scaffold into a new child directory. The CLI rejects an existing workspace root, including with
`create-skein-js . --force`, before writing files. Nx workspaces use the application generator.
Configuration inheritance and tool dependencies are independent. Existing root ESLint and Prettier
configurations stay untouched. A tool declared at the root is reused; otherwise the generated package
gets the missing tool dependency (at the root for integrated Nx layouts). Plugins imported by a root
configuration remain dependencies of that configuration. Install manually from the workspace root.
Both entrypoints format new files with inherited Prettier settings, including `singleQuote` and `semi`.
Cached Nx targets include `^default` dependency sources. Build, typecheck and test depend on `^build`,
so imported workspace libraries build first and source edits invalidate the application's cache.
Keep the workspace's dependency graph accurate and retain the generated SQLite/WAL/SHM exclusions.
`start` depends on the application's build; live hosts remain uncached.
## Scaffold validation
The internal `scripts/scaffold-cases.json` catalog drives 96 generation combinations and 33 installed
consumers: every package-manager/storage and provider/storage pair, opt-outs, independent tooling
inheritance, both Nx layouts on Node 22.12 and 24, and three additional Node 26 consumers.
CI builds and packs the commit once and distributes tarballs. Consumers install outside the checkout,
verify transitive Skein artifact hashes, run typechecking, tests, applicable lint/format checks and
build, then send echo requests to development and artifact servers. Dummy keys allow provider graph
building without model calls. SQLite always executes on Node. Required durable cases fail without Docker.
CI pins npm 11.12.1, pnpm 9.15.0 and Yarn 4.9.2; Bun is provisioned by `setup-bun`. Yarn consumers
use its node-modules linker. These checks also gate npm publication. Registry smoke tests and
SQLite/desktop platform gates remain. The curated cases cover the presence of inherited tooling;
they do not yet cover every limitation listed above. See the contributor
[scaffold validation guide](./testing.md#scaffold-consumers) for local commands and coverage details.
The new `@skein-js/nx` package follows the fixed release group. Its tarball includes generator
metadata, schema and declarations without a private runtime dependency.
### First publication
Publishing runs through `.github/workflows/release.yml` after the reusable CI gate. Before the first
release containing `@skein-js/nx`, maintainers must verify that the workflow's optional `NPM_TOKEN`
bootstrap credential is configured and authorized to create that package. Passing consumer tests
does not verify registry permissions.
After the package exists, configure its npm trusted publisher for repository `skein-js/skein-js`
and workflow `release.yml`, allowing direct `npm publish` as used by the workflow. Remove the
bootstrap token once every published package has a working trusted publisher. See npm's
[trusted publishing documentation](https://docs.npmjs.com/trusted-publishers/).
## If you don't want the scaffolder
Nothing here is load-bearing. Two alternatives:
**Copy a runnable example.** The [`examples/`](https://github.com/skein-js/skein-js/tree/main/examples)
directory has one project per framework and pattern:
```bash
npx degit skein-js/skein-js/examples/express-basic my-agent
```
Note that this copies from `main`, which tracks unreleased work: the examples depend on
`workspace:*` versions that only resolve inside the monorepo, so you will need to replace those with
real version ranges. The scaffolder exists partly to avoid exactly that.
**Write the three files yourself.** A skein project is a graph, a `langgraph.json`, and the CLI:
```bash
npm install -D skein-js
npm install @langchain/core @langchain/langgraph
```
```ts
// src/graph.ts
import { AIMessage } from "@langchain/core/messages";
import { MessagesAnnotation, StateGraph } from "@langchain/langgraph";
export const graph = new StateGraph(MessagesAnnotation)
.addNode("echo", (state) => ({
messages: [new AIMessage(`echo: ${state.messages.at(-1)?.content}`)],
}))
.addEdge("__start__", "echo")
.addEdge("echo", "__end__")
.compile();
```
```json
// langgraph.json
{ "node_version": "24", "graphs": { "agent": "./src/graph.ts:graph" }, "env": ".env" }
```
Add `"type": "module"` to your `package.json`, then `npx skein dev`. That is the entire contract —
see [langgraph-cli-compat.md](./langgraph-cli-compat.md) for every field it accepts.
## See also
- [Your first agent](./your-first-agent.md) — the guided version of all of this
- [Getting started](./getting-started.md) — the paths for when you already have a graph
- [LangGraph CLI compatibility](./langgraph-cli-compat.md) — every `langgraph.json` field
The generated JavaScript client dependency follows the runnable examples: currently
`@langchain/langgraph-sdk` `^1.12.7`. React frontends use the dedicated `@langchain/react`
package; see [the frontend guide](./react-sdk.md) for thread subscriptions and interrupt responses.
---
<!-- source: docs/agent-skills.md -->
# Agent skills for building with skein-js
The [Agent Skills](https://agentskills.io/specification) in this repository guide coding agents through building a LangGraph.js project with skein-js. They are optional: skein-js works without an AI coding agent or an installed skill.
| Skill | Use it for |
| ------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------- |
| [`skein-js`](https://github.com/skein-js/skein-js/tree/main/skills/skein-js) | Scaffolding or embedding graphs in a server, desktop app, CLI, or daemon |
| [`skein-js-nx`](https://github.com/skein-js/skein-js/tree/main/skills/skein-js-nx) | Nx application generator, workspace tooling, dependency inputs, and runtime-state cache boundaries |
| [`skein-js-turborepo`](https://github.com/skein-js/skein-js/tree/main/skills/skein-js-turborepo) | Turbo tasks, storage selection, build outputs, and runtime-state cache boundaries |
## Server or local application
The skills cover the same LangGraph execution and Agent Protocol in either deployment. Choose
SQLite explicitly for a desktop app, CLI, daemon, or one Node owner; choose Postgres/Redis for a
shared server. New projects recommend Node 24 and enable ESLint/Prettier by default; compatible
workspace pins and explicit tooling opt-outs are preserved.
The local guidance covers `--storage sqlite`, `embedSqliteGraphs`, host-owned startup/shutdown,
and persistent state after reopening. Workspace skills keep live SQLite databases and their WAL/SHM
files out of build caches while preserving the application's build artifacts. Models, tools,
credentials, and decisions to retry or resume remain application choices. Start with the
[local deployment guide](./local-deployment.md) and its Electron, Tauri, and CLI examples.
Workspace guidance covers independent configuration and tool dependency inheritance, generated
formatting, and safe child-directory generation. Nx guidance preserves dependency cache inputs and
build ordering. See [workspace tooling and caching](./scaffolding.md#workspace-tooling-and-caching).
## Find and install a skill
The skills live in this repository's `skills/` directory. With a recent [GitHub CLI that supports `gh skill`](https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-cloud-agent/add-skills), inspect and install one into your project:
```bash
gh skill preview skein-js/skein-js skein-js@main
gh skill install skein-js/skein-js skein-js@main --agent codex
```
Replace `codex` with `cursor`, `github-copilot`, or `claude-code` to install for that agent. Add `--scope user` to make it available across your projects; the default scope is the current project. Install a workspace skill the same way, substituting its name. `@main` selects the current skill even before the next skein-js release; use `@<tag>` to select a release instead. Run `gh skill update` later to get changes from this repository.
You can also copy a skill directory into your agent's project skill location. Codex, Cursor, and GitHub Copilot read `.agents/skills/<skill-name>/SKILL.md`; Claude Code reads `.claude/skills/<skill-name>/SKILL.md`. Copy the whole directory, including any supporting files. See each agent's documentation for user-wide installation paths.
Once installed, the agent sees the skill's name and description and may select it for a matching task. To select it explicitly, ask it to use the `skein-js` skill while describing the project you want. A skill checked into this repository is available to agents working **on skein-js**; installing it in your own project makes it available to agents building **with skein-js**.
Upgrading an npm package does not refresh separately installed or copied skills. Update those from
this repository too; `@main` follows current guidance, while `@<tag>` selects that release's snapshot.
---
<!-- source: docs/using-skein.md -->
# Using skein-js in your app
A dense, task-oriented cheat-sheet for **consuming** skein-js — pick a framework, stand up an Agent
Protocol server around your LangGraph.js graph, and call it. Written to be skim-friendly for humans
_and_ for AI coding agents building on skein. (Working _on_ skein itself? See
[AGENTS.md](https://github.com/skein-js/skein-js/blob/main/AGENTS.md).)
## The one thing to know: the `{ config } | { deps }` seam
Every framework adapter takes the **same options bag** (`SkeinRuntimeOptions`). You choose one of two
inputs, plus optional common fields:
```ts
// EITHER: let skein build an in-memory runtime from a langgraph.json (dev / zero-setup)
{ config: "./langgraph.json", importModule? }
// OR: bring your own assembled ProtocolDeps (production drivers, custom auth, in-code graphs)
{ deps }
// plus common: { logger?, cors?, warm? }
```
`config` → in-memory drivers, hot-reload, great for dev. `deps` → whatever you assembled (Postgres +
Redis for a server, SQLite for a local app, or an in-code graph map). **Same server either way** — only the wiring differs.
## Install
Starting from nothing? `npm create skein-js@latest my-agent` writes the project for you and you can
skip this section.
Otherwise pick your framework adapter; `@langchain/langgraph` is always a peer dependency (bring your
graph).
```bash
pnpm add @skein-js/express @langchain/langgraph # Express
pnpm add @skein-js/fastify @langchain/langgraph # Fastify
pnpm add @skein-js/nestjs @langchain/langgraph # NestJS
pnpm add @skein-js/nextjs @langchain/langgraph # Next.js
```
For server drivers add `@skein-js/runtime` (assembles Postgres/Redis). For local SQLite graphs,
use `@skein-js/local` with the upstream SQLite saver. Prefer the CLI on-ramp?
`pnpm add -D skein-js` and run `skein dev` — a drop-in for `langgraph dev`.
## Three on-ramps
**A — You have nothing yet.** Scaffold a working project — a `langgraph.json`, a keyless graph, a
test, and the `dev`/`build`/`start` lifecycle:
```bash
npm create skein-js@latest my-agent
```
See [scaffolding.md](./scaffolding.md), or [your-first-agent.md](./your-first-agent.md) for the
walkthrough.
**B — You have a `langgraph.json`** (or use the LangGraph CLI today). Change one script and keep the
config unchanged:
```diff
- "dev": "langgraph dev",
+ "dev": "skein dev",
```
Or point an adapter at the config: `{ config: "./langgraph.json" }`. See
[langgraph-cli-compat.md](./langgraph-cli-compat.md).
**C — You have a compiled graph in code** (no config, no CLI). Wrap it into `deps` and pass `{ deps }`:
```ts
import { createExpressServer } from "@skein-js/express";
import { embedInMemoryGraphs } from "@skein-js/server-kit";
import { graph } from "./my-graph.js";
const server = await createExpressServer({ deps: embedInMemoryGraphs({ agent: graph }) });
await server.listen(2024);
```
`embedInMemoryGraphs(graphs, { overrides? })` builds a `ProtocolDeps` (store, queue, bus,
checkpointer). See [embedding.md](./embedding.md).
## Mount it on your framework
Each adapter ships a **standalone** server (`create*Server`) and an **embed-alongside-your-app** path.
All accept the `{ config } | { deps }` seam above.
```ts
// Express — standalone, or skeinRouter({...}) to mount on an existing app
import { createExpressServer } from "@skein-js/express";
const server = await createExpressServer({ config: "./langgraph.json" });
await server.listen(2024);
// Fastify — standalone, or app.register(skeinPlugin, { prefix: "/agent", config })
import { createFastifyServer } from "@skein-js/fastify";
await (await createFastifyServer({ config: "./langgraph.json" })).listen(2024);
// NestJS — imports: [SkeinModule.forRoot({ config: "./langgraph.json" })]
import { createNestServer } from "@skein-js/nestjs";
await (await createNestServer({ config: "./langgraph.json" })).listen(2024);
// Next.js — App Router catch-all: app/api/[...path]/route.ts
import { createSkeinRouteHandlers } from "@skein-js/nextjs";
export const runtime = "nodejs";
export const { GET, POST, PUT, PATCH, DELETE, OPTIONS } = createSkeinRouteHandlers({ deps });
```
Each adapter has a **standalone** entry (a dedicated graph server) and an **embed-alongside-your-app**
entry, each with a runnable example:
| Framework | Package | Standalone (dedicated server) | Embed in an existing app | Examples |
| --------- | ------------------- | --------------------------------------- | --------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Express | `@skein-js/express` | `createExpressServer` | `skeinRouter` (mount the `Router`) | [express-basic](https://github.com/skein-js/skein-js/tree/main/examples/express-basic), [embed-graph](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) |
| Fastify | `@skein-js/fastify` | `createFastifyServer` | `skeinPlugin` (`register` under a `prefix`) | [fastify-basic](https://github.com/skein-js/skein-js/tree/main/examples/fastify-basic), [fastify-app](https://github.com/skein-js/skein-js/tree/main/examples/fastify-app) |
| NestJS | `@skein-js/nestjs` | `createNestServer` | `SkeinModule.forRoot` (import it) | [nestjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-basic), [nestjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-app) |
| Next.js | `@skein-js/nextjs` | — (the route handlers _are_ the server) | `createSkeinRouteHandlers` (App Router) · `createSkeinPagesHandler` (Pages) | [nextjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app), [nextjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-basic) |
[`embed-graph`](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) is the framework-agnostic **in-code** pattern
(`embedInMemoryGraphs` + `{ deps }`, no `langgraph.json`) shown on the Express adapter; the same `deps`
works with any adapter above. [`react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream) is a browser frontend
for any of them. See [Expand your setup](#expand-your-setup) to grow from here.
### Where to point your client when embedding
The protocol is served wherever you mount it, so set the client's `apiUrl` to the **mount root**, not
the server root. How you set the mount differs per adapter:
```ts
// Express — mount the router under a path
const { router } = await skeinRouter({ deps });
app.use("/agent", router); // → apiUrl: http://localhost:2024/agent
// Fastify — the plugin is encapsulated, so `prefix` isolates skein's routes + CORS
await app.register(skeinPlugin, { prefix: "/agent", deps }); // → .../agent
// NestJS — no skein-side option: it follows your app's global prefix
app.setGlobalPrefix("api"); // → apiUrl: http://localhost:2024/api
// Next.js — the catch-all's location, via `basePath` (defaults to "/api")
// app/api/[...path]/route.ts → apiUrl: http://localhost:3000/api
```
Mount at the root (no prefix) and `apiUrl` is just the server root. NestJS is the odd one out: it
reads the mount from the framework rather than from an argument you pass, so there is no skein-side
option to keep in sync with `setGlobalPrefix`.
## Local applications with SQLite
Use `embedSqliteGraphs` from `@skein-js/local` for an Electron utility process, a packaged Tauri
Node backend, a CLI, or a separate daemon. Supply an absolute application-data path, then use the
protocol service directly or pass `deps` to an existing adapter. The host owns process lifetime.
On the config path, install the local package and saver, then select both SQLite drivers:
```bash
skein dev --store sqlite --queue sqlite
skein start -c .skein/build/langgraph.json --store sqlite --queue sqlite
```
Both default to `.skein/state.sqlite`; `--sqlite-path` selects a different file. SQLite has one owner
per database. Unclaimed jobs persist, but abandoned attempts become failures rather than replaying
input. See [local deployment](./local-deployment.md) for the full setup and examples.
## Shared server deployment (Postgres + Redis) {#go-to-production-postgres--redis}
Swap the in-memory `deps` for durable ones — everything else stays the same. Two ways:
```ts
// In code: durable deps around graphs you hold (reads POSTGRES_URI / REDIS_URI)
import { embedPostgresGraphs } from "@skein-js/runtime";
import { createExpressServer } from "@skein-js/express";
const { deps, dispose } = await embedPostgresGraphs({ agent: graph });
const server = await createExpressServer({ deps });
await server.listen(2024);
process.on("SIGTERM", () => dispose().then(() => process.exit(0)));
```
```ts
// From a langgraph.json: pick drivers explicitly
import { buildRuntime } from "@skein-js/runtime";
const rt = await buildRuntime({
configPath: "./langgraph.json",
store: "postgres",
queue: "redis",
});
const server = await createExpressServer({
deps: rt.deps,
channels: rt.channels,
cors: rt.cors,
});
```
Redis is optional but **required to run more than one instance** (the in-memory queue is
process-local). Or skip the code entirely: `skein dev --store postgres --queue redis`, and
`skein build` / `skein up` for a container. See [embedding.md](./embedding.md#going-to-production),
[storage.md](./storage.md), [runs-and-redis.md](./runs-and-redis.md).
## Call the server
Any Agent Protocol client works — no custom SDK. The two you'll reach for:
```ts
// Node / server-to-server — @langchain/langgraph-sdk
import { Client } from "@langchain/langgraph-sdk";
const client = new Client({ apiUrl: "http://localhost:2024" });
const thread = await client.threads.create();
const input = { messages: [{ role: "user", content: "hello" }] };
const reply = await client.runs.wait(thread.thread_id, "agent", { input });
for await (const ev of client.runs.stream(thread.thread_id, "agent", { input })) console.log(ev);
```
```tsx
// Browser — @langchain/react useStream (SSE)
import { useStream } from "@langchain/react";
const thread = useStream({ apiUrl: "http://localhost:2024", assistantId: "agent" });
thread.submit({ messages: [{ role: "user", content: "hello" }] });
```
`assistantId` defaults to the `graph_id`. See [react-sdk.md](./react-sdk.md),
[streaming.md](./streaming.md).
## Endpoint surface
skein implements the standard Agent Protocol REST + SSE contract, so the SDK maps onto it directly.
The resources: **assistants** (a served graph + its schemas), **threads** (persistent conversations),
**runs** (`/runs/wait`, `/runs/stream`, background `/threads/{id}/runs` with join + cancel), and a
long-term **store** (`/store/items`, semantic `/store/items/search`). The full endpoint inventory and
the auth route→permission map live in [agent-protocol.md](./agent-protocol.md).
## Package → import map
| You want to… | Import | From |
| ----------------------------------------------- | ------------------------------------------ | -------------------------- |
| Serve on Express / Fastify / Nest / Next | `create*Server` / `skein*` / `SkeinModule` | `@skein-js/<framework>` |
| Embed a graph in code (in-memory) | `embedInMemoryGraphs` | `@skein-js/server-kit` |
| Embed a graph in code (durable Postgres) | `embedPostgresGraphs` | `@skein-js/runtime` |
| Assemble prod deps from a `langgraph.json` | `buildRuntime` | `@skein-js/runtime` |
| Implement a storage driver / handle edge errors | `SkeinStore`, `SkeinHttpError` | `@skein-js/core` |
| Put skein on a framework we don't ship | `skeinRoutes`, `createProtocolRuntime` | `@skein-js/agent-protocol` |
## Expand your setup
Grow from the minimal server without rewrites — each step changes one thing:
- **Add another graph.** Add an entry to the graph map (`embedInMemoryGraphs({ echo, agent })`) or to
`langgraph.json`'s `graphs`. Each becomes an assistant, addressed by its `graph_id`.
- **Standalone → embedded.** Move from a dedicated `create*Server` to the embed entry for your
framework (`skeinRouter` / `skeinPlugin` / `SkeinModule.forRoot` / `createSkein*Handlers`) — see the
adapter table above — to serve the protocol next to your existing routes, under a prefix if you want.
- **Go durable / scale out.** Swap the in-memory `deps` for `embedPostgresGraphs(...)` or
`buildRuntime({ store: "postgres", queue: "redis" })`. Add Redis to run more than one instance. See
[Go to production](#go-to-production-postgres--redis).
- **Drain more background runs at once.** Each instance executes 10 queued runs concurrently by
default — tune it with `skein dev --concurrency 4`, `SKEIN_RUN_CONCURRENCY=4`, or
`worker: { maxConcurrency: 4 }` on any adapter. See
[run concurrency](./runs-and-redis.md#run-concurrency).
- **Add auth, memory, HITL, webhooks.** These are drop-in — see the [recipes](./recipes/) (custom
auth, `getStore()` long-term memory, interrupt/resume, run-completion webhooks).
- **A framework we don't ship.** The adapters are thin shims over one transport-neutral handler table
(`createProtocolRuntime` + `skeinRoutes`); put skein on any Node HTTP framework by writing ~40 lines
of request/response mapping. See [building-an-adapter.md](./building-an-adapter.md).
## Gotchas
- **Auth is off by default.** No `auth` block / no `auth` dep → the server is fully open, exactly like
`langgraph dev`. Turn it on with a `@langchain/langgraph-sdk/auth` `Auth` instance — see the
[provider recipes](./recipes/authentication.md) and [production recipe](./recipes/production.md#custom-auth).
- **CORS is off by default.** Browser clients on another origin need `http.cors` in `langgraph.json`
(or the `cors` option). Same-origin (e.g. Next.js) needs nothing.
- **A long-lived Node process** is required for the background run worker and in-memory drivers — fine
on a normal server / `next start`; for serverless, use Postgres + Redis.
- **`useStream` needs an absolute URL** — pass `` `${window.location.origin}/api` ``, not a bare
`/api`.
- **Bundling skein yourself** (rspack/webpack/esbuild, or an unusual Next.js config)? skein is
ESM-only but `require()`-resolvable, and `@langchain/langgraph-api` + `@typescript/vfs` must stay
external. See [bundling.md](./bundling.md).
- **404s on every protocol path?** You're almost certainly pointing at the wrong root — the protocol
lives at your **mount path**, not the server root (see
[Where to point your client](#where-to-point-your-client-when-embedding)). On NestJS that means
`app.setGlobalPrefix("api")` moves it to `/api/threads`. Two red herrings worth ruling out: an
`Unsupported route path: "/api/*"` warning in a NestJS boot log is Nest auto-converting the
adapter's catch-all and is harmless, and `/info` isn't part of the surface — a 404 there is correct.
## Go deeper
- [Getting started](./getting-started.md) — the guided, end-to-end walkthrough.
- [Recipes](./recipes/) — auth, human-in-the-loop, long-term memory, CORS, background runs, deploy.
- [Overview & architecture](./index.md) · [Agent Protocol surface](./agent-protocol.md) ·
[Embedding](./embedding.md) · [Building a custom adapter](./building-an-adapter.md)
---
<!-- source: docs/adapters.md -->
# Framework adapters
An adapter is a thin transport shim: it takes the framework-agnostic handler table from
`@skein-js/agent-protocol` and puts it on Express, Fastify, NestJS, Next.js, or the Web Fetch API.
The engine, the endpoints and the wire format are identical on all five — what differs is how you
mount it, and a handful of host-framework details this page exists to spell out.
Every adapter takes the **same options bag**: `{ config }` to build a runtime from a
`langgraph.json`, or `{ deps }` to bring [your own assembled `ProtocolDeps`](./embedding.md). That
seam is the whole API. Picking a different adapter changes neither.
## Pick one
Each adapter has a **standalone** entry — a dedicated agent server — and an **embed** entry that
mounts the protocol inside an app you already run:
| Adapter | Standalone | Mount into your app | Example |
| ------------------- | ------------------------ | ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `@skein-js/express` | `createExpressServer` | `skeinRouter` → `app.use()` | [express-basic](https://github.com/skein-js/skein-js/tree/main/examples/express-basic) |
| `@skein-js/fastify` | `createFastifyServer` | `skeinPlugin` under a `prefix` | [fastify-basic](https://github.com/skein-js/skein-js/tree/main/examples/fastify-basic) · [fastify-app](https://github.com/skein-js/skein-js/tree/main/examples/fastify-app) |
| `@skein-js/nestjs` | `createNestServer` | `SkeinModule.forRoot()` | [nestjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-basic) · [nestjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-app) |
| `@skein-js/nextjs` | — (the routes _are_ it) | `createSkeinRouteHandlers` / Pages handler | [nextjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app) · [nextjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-basic) |
| `@skein-js/fetch` | `createSkeinFetchServer` | — (no mount export) | — (covered by CI's runtime matrix) |
Express is the default: it is what `skein dev` and the production image run, and what the other
docs use in examples. Reach for another when you already have an app on it.
## What is identical everywhere
The full endpoint surface (`/threads`, `/assistants`, `/runs`, `/store`, `/crons`), SSE streaming
with the same frames, human-in-the-loop, auth, and every client — `useStream`, Agent Chat UI, the
LangGraph SDK — work against any of them with only a URL change. A `ProtocolDeps` you assembled for
one adapter mounts on any other unchanged.
## What differs
This is the part worth reading before you pick:
| | Express | Fastify | NestJS | Next.js | Fetch |
| ------------------ | --------------------------- | ------------------------------ | ------------------------------------- | --------------------------- | ---------------------------------- |
| **Mount path** | `app.use("/agent", …)` | `register({ prefix })` | `app.setGlobalPrefix()` only | `basePath` (default `/api`) | `basePath` (default `""`) |
| **CORS** | bundled `cors` package | optional peer `@fastify/cors` | skein's middleware, skein routes only | built in | built in |
| **JSON body cap** | 100 kb, via `json.limit` | Fastify's `bodyLimit` (1 MB) | the host's parser, else unbounded | unbounded | 100 kb, via `maxBodyBytes` |
| **Logger default** | none — stays silent | `fastify.log` (pino) | Nest's `Logger` | none — stays silent | none — stays silent |
| **`/ok` route** | standalone only | standalone only | standalone only | **never** | always |
| **Shutdown** | `close()` drains the worker | `close()`, or plugin `onClose` | needs `enableShutdownHooks()` | **none** — nothing drains | `close()` after the listener stops |
Two of these bite most often. **The body cap is not uniform** — only Express and Fetch impose one of
their own, so on NestJS with `bodyParser: false`, and on Next.js, a large body is read into memory
before auth runs. And **only Express and Fetch drain cleanly by default**: a Next.js deploy has no
shutdown path at all, because the runtime lives on `globalThis` for the life of the process.
## Express
The default, and the reference implementation every other adapter is checked against.
```ts
import { createExpressServer, skeinRouter } from "@skein-js/express";
// Standalone — a dedicated agent server with a /ok probe:
const server = await createExpressServer({ config: "./langgraph.json" });
await server.listen(2024);
// Or mounted on an app you already run. `skeinRouter` is async — it seeds assistants
// and starts the run worker before returning, so the router is ready to serve:
const { router } = await skeinRouter({ deps });
app.use("/agent", router);
```
Express strips the mount path for you, so nothing needs to be kept in sync. Two options no other
adapter has: `json.limit` (default 100 kb — a limit the parser can't understand is rejected at mount
time rather than silently becoming unlimited) and `requestLog`.
**It logs nothing by default.** Express owns no logger, and a library shouldn't decide to start
writing to its host's stdout — pass `createConsoleLogger()` if you want output.
## Fastify
```ts
import { createFastifyServer, skeinPlugin } from "@skein-js/fastify";
// Standalone:
const server = await createFastifyServer({ config: "./langgraph.json" });
await server.listen(2024);
// Embedded — encapsulated, so skein's hooks don't leak into your app:
await app.register(skeinPlugin, { prefix: "/agent", config: "./langgraph.json" });
```
Three things to know. **CORS needs an optional peer**: `@fastify/cors` is imported lazily and only
when CORS is on, so enabling it without installing the package throws at registration rather than
failing a request later. The plugin **replaces the JSON content-type parser** so an empty body sent
with `Content-Type: application/json` is read as `{}` — Fastify's default parser 400s on it where
Express does not, and the adapters have to agree. And SSE **hijacks the reply** and writes to the raw
socket, which is why a client disconnect is detected there rather than on the request.
There is no `json.limit`; Fastify's own `bodyLimit` applies.
> [!WARNING]
> **`createFastifyServer` ignores `http.disable_*`.** The standalone server does not forward the
> route table that a `langgraph.json`'s disable flags produce, so it mounts the full protocol
> surface even when the config asks for less. `skeinPlugin` honours them correctly, as do the other
> four adapters. This is a bug, not a design choice — until it is fixed, use the plugin if you rely
> on those flags.
## NestJS
```ts
import { SkeinModule } from "@skein-js/nestjs";
@Module({ imports: [SkeinModule.forRoot({ config: "./langgraph.json" })] })
export class AppModule {}
```
NestJS is the odd one out on mounting: it reads its prefix from the framework via
`app.setGlobalPrefix()` rather than from an argument you pass, so there is no skein-side option that
can drift out of sync — but also no way to mount it somewhere else.
Three consequences that look like bugs and aren't:
- **`GET /api` 404s**, and so does `/info`. Nest never routes the bare prefix root to middleware, and
no skein route lives at `/`.
- **`SkeinModule` registers no `/ok`.** Only `createNestServer` does. Point health checks at a route
you own, or use the standalone server.
- **A log line about an unsupported route path** during boot is harmless.
**Call `app.enableShutdownHooks()`** when embedding. skein stops the run worker on
`beforeApplicationShutdown` so in-flight SSE streams settle before the server closes, and without the
hooks that never fires. The standalone server does it for you.
CORS is applied by skein's own middleware, scoped to skein's routes, so it behaves identically
standalone and embedded — `app.enableCors()` governs your routes, not these.
## Next.js
There is no standalone entry: the route handlers **are** the server, same-origin with your UI.
```ts
// app/api/[...path]/route.ts — App Router
import { createSkeinRouteHandlers } from "@skein-js/nextjs";
export const runtime = "nodejs"; // the run worker needs a long-lived process, not the edge
export const { GET, POST, PUT, PATCH, DELETE, OPTIONS } = createSkeinRouteHandlers({ deps });
```
```ts
// pages/api/[...path].ts — Pages Router
import { createSkeinPagesHandler } from "@skein-js/nextjs";
// externalResolver tells Next this route settles the response itself, which silences the
// "API resolved without sending a response" warning that SSE streams otherwise trigger.
export const config = { api: { bodyParser: true, externalResolver: true } };
export default createSkeinPagesHandler({ deps });
```
`basePath` defaults to `/api` and must match where you put the catch-all. The runtime is memoized on
`globalThis`, keyed by config path or `deps` identity, so two route files sharing a config share one
runtime — which is why a handler's own `logger` option outranks the shared runtime's, rather than
whichever handler served the first request deciding for both.
**This needs a warm process.** The in-memory drivers and the background run worker don't survive a
function that scales to zero — a serverless deploy needs the Postgres store and Redis queue. See
[deploy-serverless.md](./deploy-serverless.md).
## Fetch (Bun / Deno)
The web-standard transport, and the production path on Bun and Deno. `skein build --runtime bun|deno`
selects it for you.
```ts
import { createSkeinFetchServer, startBunServer } from "@skein-js/fetch";
const skein = await createSkeinFetchServer({ config: "./langgraph.json" });
const listener = startBunServer(skein, { port: 2024 });
// Stop accepting connections first, then drain:
await listener.stop();
await skein.close();
```
It is the only adapter that caps request bodies itself — `maxBodyBytes`, default 100 kb — and it has
to: `Bun.serve` defaults to 128 MB and `Deno.serve` has no limit at all, and the body is read before
auth runs, so an unbounded read is an unauthenticated way to exhaust memory.
`/ok` is always served, `basePath` defaults to `""`, and there is **no mount export and no invoke
surface** — the handler is a whole-server `fetch` function. Composing it into a host router is yours
to do.
## What you must get right
- **`skeinRouter` is async.** `skeinRouter({ deps }).router` is `undefined`; await it first. It seeds
assistants and starts the run worker before returning.
- **Auth handlers that match on a path are not portable between adapters.** Express and NestJS report
the full mount-inclusive URL, while Next.js and Fetch report it with the mount prefix stripped — so
a handler matching `/threads` sees that on one and `/api/threads` on another. Match on the
protocol-relative suffix, or keep the handler adapter-specific.
- **Auth is off by default on the `{ deps }` path**, whichever adapter you mount it on. See the
warning in [embedding.md](./embedding.md).
- **Only the standalone servers add `/ok`** — except Next.js, which never does, and Fetch, which
always does. Check the table above before pointing a platform health check at it.
- **No adapter serves the console.** `http.console` is honoured by `skein start`'s Node runtime,
which mounts it for you; on Bun/Deno the flag warns and does nothing. Mount it into your own app
yourself, whichever adapter you are on — see [console.md](./console.md#mounting-it-yourself).
## See also
- [Using skein-js](./using-skein.md) — the terse cheat-sheet version of this page
- [Embedding a graph](./embedding.md) — the `{ deps }` seam every adapter accepts
- [Errors & logging](./errors-and-logging.md#what-each-adapter-does-by-default) — logger defaults in full
- [A graph as a plain endpoint](./serving-a-single-graph.md) — the invoke surface, per adapter
- [Building an adapter](./building-an-adapter.md) — putting the handler table on a framework we don't ship
---
<!-- source: docs/langgraph-cli-compat.md -->
# LangGraph CLI compatibility
The `skein` CLI is a **drop-in replacement for the LangGraph CLI** — the migration path onto
skein-js for anyone already using it. That means two things: read the same `langgraph.json`, and
mirror the same command surface.
## Command mapping
| LangGraph CLI | skein-js | Behavior |
| ---------------------- | ------------------ | -------------------------------------------------------------- |
| `langgraph dev` | `skein dev` | In-process dev server, hot reload, **no Docker**. Local state. |
| `langgraph up` | `skein up` | Docker Compose stack (app + **Postgres + Redis**). |
| `langgraph build` | `skein build` | Build a deployable Docker image from the config. |
| `langgraph dockerfile` | `skein dockerfile` | Emit a standalone Dockerfile from the config. |
| `langgraph deploy` | — | Out of scope (hosted-platform push). |
Shared flags where sensible: `--port`, `--host`, `--no-reload`, `--config`,
`-n, --n-jobs-per-worker`.
Production commands (`up`, `build`, `dockerfile`, and the generated `start`) also accept
`--runtime node|bun|deno` and `--runtime-version`. Precedence is CLI flag, then
`skein.runtime` in config, then Node 24 LTS. Legacy `node_version` remains the Node version fallback.
**skein-only:** `skein start` serves a pre-built `.skein/build` artifact — plain compiled JS, no vite,
no reload. It is the production image's entrypoint, and it is also how you run skein **without a
container**: `skein build --artifact-only` writes the artifact on any machine with Node, and
`skein start` serves it from a systemd unit or any process manager. See
[Without Docker](./deploy.md#without-docker). The LangGraph CLI has no equivalent — it transforms
TypeScript at runtime inside the image.
**skein-only dev flags** (beyond the LangGraph CLI) — run `skein dev` against production-shaped
storage without Docker:
| Flag | Values | Default | Notes |
| ---------------------- | ------------------------------ | --------------------- | ----------------------------------------------------------------------------------- |
| `--store <driver>` | `memory`, `postgres`, `sqlite` | `memory` | Postgres selects `PostgresSaver`; SQLite selects the upstream SQLite saver. |
| `--queue <driver>` | `memory`, `redis`, `sqlite` | `memory` | Redis provides the shared queue/bus; SQLite persists jobs in the resource database. |
| `--sqlite-path <path>` | File path | `.skein/state.sqlite` | Resolved against the working directory; also accepted by `start`. |
Graph hot-reload works with any driver; the `.skein/` snapshot is skipped for durable drivers
(they persist inherently).
SQLite is a skein deployment option for a desktop app, CLI, daemon, or one Node owner. Install
`@skein-js/local` and `@langchain/langgraph-checkpoint-sqlite@~1.0.4`, then select both drivers:
```bash
skein dev --store sqlite --queue sqlite
skein start -c .skein/build/langgraph.json --store sqlite --queue sqlite
```
Mixed SQLite/non-SQLite pairs are rejected. `dev` still defaults to memory/memory and `start` to
Postgres/Redis. SQLite requires Node 22/24/26, does not import memory snapshots, and keeps its file
outside the build artifact. Models and tools remain in the graph. See
[local deployment](./local-deployment.md) for ownership, crash behavior, and native packaging.
**Background-run concurrency.** `-n, --n-jobs-per-worker <count>` is honored on `skein dev` **and**
`skein start`, with the same default of **10** the LangGraph CLI uses — so an existing
`langgraph dev -n 4` command line ports over unchanged. `--concurrency <count>` is the same knob
under a self-evident name, and wins if both are passed. Without a flag, skein reads
`SKEIN_RUN_CONCURRENCY`, then the LangGraph-compatible `N_JOBS_PER_WORKER` — which is how the setting
reaches a container. See [runs-and-redis.md](./runs-and-redis.md#run-concurrency).
One deliberate cosmetic difference: skein's banner prints `Starting 1 worker, up to 10 concurrent
runs` where `langgraph dev` prints `Starting 10 workers`. LangGraph really does spawn 10 loops;
skein runs one worker whose consumer executes 10 runs at a time. Same throughput, honest wording. `skein up`/`build`/`dockerfile` accept `--tag` (build) and `--output`
(dockerfile) in addition to `--config`.
**`skein build --artifact-only`** (skein-only) writes `.skein/build` — the bundled graphs, the pinned
`package.json`, the baked `schemas.json`, and the generated Dockerfile — and stops without invoking
Docker. Use it when something else builds the image: Kaniko or Cloud Build in CI, a platform that
builds from a committed Dockerfile, or any machine that has Node but no Docker daemon.
**Private/authenticated npm registries.** When your production dependencies include private scoped
packages (e.g. `@myorg/*` behind a token), pass an `.npmrc` so the image's dependency install can
authenticate. `skein build --npmrc <path>` and `skein up --npmrc <path>` mount it as a BuildKit
**secret** — it authenticates the install without ever landing in an image layer or build history.
The generated Dockerfile always declares this secret mount (`id=npmrc`); it is optional, so
public-registry builds that pass no `.npmrc` are unaffected. Building the standalone `skein dockerfile`
output by hand? Supply the same secret directly:
```bash
docker build --secret id=npmrc,src=$HOME/.npmrc -t my-app .
```
`skein build`/`up` also emit a `.dockerignore` that excludes `.env*`, `node_modules`, `.git`, and
`.skein` — so secrets are **never baked into image layers**. Runtime configuration reaches the
container through the environment instead: `skein up` sets `POSTGRES_URI`/`REDIS_URI` in the compose
`environment:` block, and you add any provider keys (e.g. `OPENAI_API_KEY` for semantic search) there
too. Generated Docker assets carry a `# Generated by skein` marker; skein regenerates its own files
but leaves a hand-edited `Dockerfile`/`compose.yaml` untouched.
References:
- LangGraph CLI docs — <https://docs.langchain.com/langsmith/cli>
- `@langchain/langgraph-cli` source — <https://github.com/langchain-ai/langgraphjs/tree/main/libs/langgraph-cli>
## Under the hood: what skein-js changes (transparently)
skein-js keeps the **contract** identical — same `langgraph.json`, same graph code, same Agent
Protocol on the wire, same clients — while re-implementing the runtime underneath with a different,
open, self-hostable set of building blocks. None of this requires a change on your side; it's the
"drop-in" promise honored at the implementation level.
**Same graph code, same config, same wire protocol.** skein-js reuses the LangGraph runtime,
checkpointers, the `langgraph.json` parser/schemas, and the SDK/types from the open
`@langchain/*` packages (see [reuse.md](https://github.com/skein-js/skein-js/blob/main/docs/reuse.md)). It only rebuilds the durable-production,
multi-framework, drop-in-CLI layer that isn't open — so your project moves over untouched.
**What we implement, and the tools we use:**
| Concern | skein-js implementation |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CLI | [commander](https://github.com/tj/commander.js) — the `skein dev`/`up`/`build`/`dockerfile` command surface, plus `skein start` (below). |
| Dev graph loading | [vite](https://vitejs.dev) loads your TypeScript graphs **in-process** — no separate build step. tsconfig `paths` aliases resolve, so graphs in an Nx/Turborepo/pnpm-workspace monorepo import shared workspace packages unchanged. |
| Prod image build | `skein build`/`up` bundle graphs (+ auth/embed) to plain JS with `vite.build()` at build time — same alias resolution as dev, anchored at the workspace root — into a self-contained `.skein/build` artifact. The slim image installs prod deps only and runs the compiled output via `skein start` (no vite, no runtime TypeScript transform). |
| Dev hot reload | **State-preserving** reload on source change: your threads, runs, and memory survive the reload. `.env` is watched too, so filling in a missing model key reloads the graphs instead of needing a restart. |
| Dev persistence | Dev state is snapshotted to `.skein/` so it survives restarts (opt out with `--no-persist`). |
| Run queue (prod) | [BullMQ](https://docs.bullmq.io) on Redis — background runs, retries, backoff, and crash recovery. |
| Cross-instance streaming | [ioredis](https://github.com/redis/ioredis) + **Redis Streams**/pub-sub — join a run's SSE stream from any instance. |
| Postgres store (prod) | [pg](https://node-postgres.com) + compiled-in schema migrations + **pgvector** semantic search. |
| Checkpoints (prod) | LangGraph-native [`PostgresSaver`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/checkpoint-postgres) — reused, not reinvented. |
**Transparent improvements over `langgraph dev`.** Because the storage/queue are pluggable drivers,
`skein dev` can run against **production-shaped** Postgres/Redis **without Docker** (`--store
postgres --queue redis`), and dev state persists across restarts — both beyond the stock LangGraph
CLI dev server, and both fully opt-in. The wire behavior your clients see is unchanged either way.
If any of this ever _does_ change observable behavior versus the LangGraph CLI, that's a
compatibility bug — please [report it](https://github.com/skein-js/skein-js/issues).
## `langgraph.json` — fields we honor
skein-js parses an existing `langgraph.json` **unchanged**. A `skein.json` may extend/override
it but is never required.
```jsonc
{
// REQUIRED: map of graph id -> "path:export"
"graphs": {
"agent": "./src/agent.ts:graph", // exported compiled graph instance
"chat": "./src/chat.ts:makeGraph", // or a factory function
},
// JS/Node runtime pin (20 | 22 | 24)
"node_version": "24",
// Native production runtime (skein extension). Omit for Node.
"skein": {
"runtime": { "name": "bun", "version": "1.3.14" },
// Retention for `Idempotency-Key` records (skein extension; see below). Tuning only —
// omitting the block does NOT disable the header.
"idempotency": { "retention_hours": 24, "in_flight_minutes": 15 },
// Delivery policy for run-completion callbacks (skein extension; see below). Tuning only —
// omitting the block does NOT make callbacks fire-once.
"webhooks": { "retries": { "max_attempts": 12 }, "allowed_hosts": ["hooks.example.com"] },
},
// .env path OR inline map
"env": ".env",
// long-term memory store; semantic search config drives our pgvector index
"store": {
"index": { "embed": "openai:text-embedding-3-small", "dims": 1536, "fields": ["$"] },
// your own LangGraph BaseStore instead of skein's driver (skein extension)
"adapter": "./src/my-store.ts:store",
},
// checkpointer backend; "default" == Postgres (via PostgresSaver)
"checkpointer": { "type": "default" },
// server customization — `cors` plus LangGraph's route toggles
"http": {
"cors": { "allow_origins": ["*"] },
"disable_assistants": false,
"disable_threads": false,
"disable_runs": false,
"disable_crons": false, // skein-only: also stops the scheduler, not just the routes
"disable_store": false,
"disable_docs": false, // hide /docs and /openapi.json while keeping /info and /metrics
"disable_meta": false, // turns off /info, /openapi.json, /docs and /metrics; never /ok
"console": false, // skein-only: serve the console at /console (or a path). Off unless set;
// `skein dev` serves it anyway. See docs/console.md.
"app": "./src/http-app.ts:app", // Fetch function or exported Hono/Fetch application
"disable_mcp": false,
"disable_a2a": false,
},
// custom authentication + authorization (see below)
"auth": {
"path": "./src/auth.ts:auth", // a "@langchain/langgraph-sdk/auth" `Auth` instance
"disable_studio_auth": true, // default; CLI always requires credentials for Studio
"dev_ui_access": false, // skein-only; dev key for console + Studio when true
},
// where runs report themselves — traces + lifecycle events (a skein extension; see below)
"telemetry": {
"langsmith": true,
"posthog": { "host": "https://eu.i.posthog.com" },
"otel": true,
"paths": ["./src/my-telemetry.ts:sink"], // your own TelemetrySink
},
// packages your code loads BY NAME at runtime, which no bundler can discover (skein extension)
"dependencies": ["@langchain/openai"],
// extra Dockerfile lines appended after the base image
"dockerfile_lines": [],
}
```
### How each field maps into skein-js
| `langgraph.json` field | skein-js wiring |
| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `graphs` | [`@skein-js/config`](./storage.md) resolves each `path:export`, loading a compiled graph or `makeGraph` factory. Drives `/agents` introspection + run execution. |
| `node_version` | Used by `skein build` / `skein dockerfile` base image selection. Defaults to Node 24 LTS when omitted; an explicit value is honoured verbatim, including an older one. |
| `skein.runtime` | Native production server and pinned official image: `node`, `bun`, or `deno`. Bun/Deno use `@skein-js/fetch` with `Bun.serve`/`Deno.serve`, never Express compatibility. |
| `skein.idempotency` | Retention for `Idempotency-Key` records. Tuning only — the header is honoured either way. See [below](#idempotency-skeinidempotency). |
| `skein.webhooks` | How run-completion callbacks are retried and bounded. Tuning only — a run carrying a `webhook` owes a callback either way. See [below](#webhooks-skeinwebhooks). |
| `env` | Loaded into `process.env` at boot (dev) / baked into the image (build). |
| `store` | `store.index.{embed,dims,fields,hnsw}` configures pgvector semantic search on the Postgres driver (`hnsw: true` opts into the approximate index); `store.ttl.{default_ttl,refresh_on_read,sweep_interval_minutes}` expires items. `store.adapter` is a **skein extension** — `path:export` to your own LangGraph `BaseStore` (e.g. `PostgresStore`, `MongoDBStore`), which then serves the whole `/store` surface. See [storage.md](./storage.md). |
| `checkpointer` | `"default"` → `PostgresSaver`; dev falls back to an in-memory `MemorySaver`. |
| `http` | `http.cors` maps to the adapter's CORS options; the `disable_*` flags remove that resource's routes before mounting, so it 404s from the host app as under `langgraph dev`. `http.console` is skein-only — it serves the [console](./console.md) at `/console` (off unless set; `skein dev` serves it regardless). `http.app` loads an exported Fetch/Hono application before protocol dispatch; a 404 falls through. Authentication runs before custom application dispatch. |
| `auth` | `auth.path` loads an `Auth` from `@langchain/langgraph-sdk/auth`; every request is authenticated + authorized. The CLI requires credentials for Studio regardless of `disable_studio_auth`; `auth.dev_ui_access: true` or loopback `skein dev --dev-ui-access` provides a temporary key for Studio and the console. The setting is stripped from production builds. |
| `telemetry` | **skein extension.** Builds the telemetry sinks runs report to — see [observability.md](./observability.md). Unknown to `langgraph dev`, which ignores it. |
| `dependencies` | **skein extension on the JS side** (LangGraph's schema has it for Python only). Extra packages `skein build` pins into the artifact — see below. |
| `dockerfile_lines` | Appended by `skein dockerfile` / `skein build`. |
**`dependencies` means something different here.** In LangGraph it is a Python-only field, required
and used to drive `pip install`; the JS schema has no such field, because `langgraph build` copies
your project into the image and runs your package manager over it. skein bundles instead, so it
derives the image's dependency list from what the bundle still imports — and `dependencies` is the
escape hatch for the packages that list cannot contain: the ones you load **by name at runtime**
(`import("@langchain/" + provider)`), which never appear in any module graph. Local-path entries
(`"."`, `"./pkg"`) are accepted and ignored — that source is already bundled. See
[bundling.md](./bundling.md#what-skein-build-inlines-vs-externalizes).
## Graph loading (`path:export` notation)
`@skein-js/config` resolves entries exactly like the LangGraph CLI:
- `"./src/agent.ts:graph"` — imports the module and reads the `graph` export, which must be
a `CompiledStateGraph`.
- `"./src/agent.ts:makeGraph"` — reads a factory export and calls it (optionally with
config) to obtain a `CompiledStateGraph`.
This is the same contract LangGraph.js users already write against, so **no code changes
are required** to move a project onto skein-js.
## Idempotency (`skein.idempotency`)
A skein extension with no LangGraph counterpart, so it lives under the reserved `skein` namespace
rather than at the top level — a top-level key would collide if LangGraph ever claimed the same name.
```jsonc
{
"skein": {
"idempotency": {
"retention_hours": 24, // how long a recorded response stays replayable
"in_flight_minutes": 15, // how long an unfinished create blocks a retry
"sweep_interval_minutes": 60, // how often expired records are reclaimed
},
},
}
```
**This block is tuning, not an on/off switch.** Omitting it does not disable `Idempotency-Key`: a
caller who sends one has been promised their retry will not start a second run, and honouring that is
not a deployment opinion. The defaults are the values shown above.
What each one is for:
- `retention_hours` — the dedup window. Long enough to cover any provider's retry schedule, short
enough that the table doesn't grow without bound. Raise it if you integrate a provider that retries
over days.
- `in_flight_minutes` — how long a create that hasn't finished holds its key before a retry may take
it over. Keep it above your slowest `POST /runs/wait`, or a retry could be admitted alongside the
original; keep it well under `retention_hours`, or an instance killed mid-request leaves its keys
409-ing until the retention expires.
- `sweep_interval_minutes` — cadence of the background reclaim. Purely housekeeping: an expired
record is taken over by the next claim whether or not the sweeper has run, so this affects disk,
never correctness.
Honoured identically on every driver combination, including `skein dev` with no Docker. See
[agent-protocol.md](./agent-protocol.md#idempotent-run-creation-idempotency-key) for the request
semantics and the replay table.
## Webhooks (`skein.webhooks`)
LangGraph's `webhook` field is a bare URL with no delivery policy at all, so this block is a skein
extension under the reserved namespace, for the same reason `skein.idempotency` is. The **payload
shape is unchanged** — this configures delivery, not the body.
```jsonc
{
"skein": {
"webhooks": {
"retries": {
"max_attempts": 12, // attempts before a callback is dead, counting the inline first one
"initial_delay_ms": 1000, // the first retry's delay; the rest double from it
},
// The signing key. Prefer SKEIN_WEBHOOK_SECRET, which wins over this — see below.
"secret": "whsec_…",
"max_payload_bytes": 262144, // cap on the stored body; over it, `values` is truncated
"retain_hours": 24, // how long a settled delivery is kept before it is reclaimed
"allowed_hosts": ["hooks.example.com"], // absent = no restriction (today's behaviour)
"require_https": true, // refuse plaintext callbacks; absent permits them
},
},
}
```
**This block is tuning, not an on/off switch.** Omitting it does not make callbacks fire-once: a run
that carries a `webhook` owes a callback, and how hard the server tries is a deployment decision
rather than permission to try at all. `retries.max_attempts: 1` is how you ask for one shot.
What each one is for:
- `max_attempts` — read it as a **time horizon, not a count**. The delays double, so 12 attempts is
1+2+4+…+1024 seconds ≈ **34 minutes**, which rides out a rolling deploy with room to spare. Six is
not "half as patient": it is ~31 seconds, which does not survive one redeploy. There is deliberately
no ceiling knob — the doubling tops out around 17 minutes at any sane attempt count, and on Redis
the schedule is BullMQ's own exponential backoff rather than ours.
- `initial_delay_ms` — the first retry's delay and the base the rest double from. Raise it for a
receiver you know is slow to come back; lower it only if you also lower `max_attempts`.
- `max_payload_bytes` — the body is **stored** so a retry has something to send, so an unbounded
payload is unbounded rows. Over the cap, `values` is replaced by a truncation marker inside the
signed body; everything a receiver needs to fetch the state itself survives.
- `retain_hours` — how long a delivered or dead row is kept. Disk only; nothing is incorrect if the
sweep never runs.
- `secret` — the signing key, or a list during a rotation (the first signs). **Prefer the
`SKEIN_WEBHOOK_SECRET` environment variable**, which takes precedence: a key here is a key in
version control, and skein warns about it once at startup. Note that skein does **not** expand
`${VAR}` anywhere in `langgraph.json`, so `"secret": "${SKEIN_WEBHOOK_SECRET}"` would sign every
callback with that literal 23-character string. Absent on both → callbacks are unsigned, which is
today's behaviour.
- `require_https` — refuse a callback whose URL is not `https:`. Off by default, because an internal
receiver on a trusted network is legitimate and turning it on for everyone would break those
deployments on upgrade. Turn it on whenever callbacks leave your network: the body carries the run's
final state, and retries mean it crosses the wire up to `max_attempts` times rather than once.
- `allowed_hosts` — an exact-hostname allowlist. Set it if you accept run creates from untrusted
callers: `webhook` is a caller-supplied URL, so it is a server-side request to a target they chose,
and retrying it turns a one-shot SSRF probe into a repeated one. **Off by default**, because turning
it on for everyone would make an upgrade start dropping deployments' own callbacks. A refused host
is recorded `dead` with the reason rather than silently skipped.
**Where the retries actually run depends on your queue driver.** With Redis, the whole schedule is
BullMQ's — delayed jobs, exponential backoff with jitter, and re-delivery of a job whose worker was
killed — so a retry still waiting survives a restart. Without it, skein polls the outbox instead:
the stored due time and delivery lease survive restart whenever the store is durable. SQLite and
Postgres therefore retain waiting retries without a separate delivery queue.
With a durable store, the delivery and its due time are recorded in the same transaction as the
run's terminal status. Waiting retries therefore survive restart without a separate queue. See
[webhooks.md](./webhooks.md).
## Authentication + authorization (`auth`)
skein-js implements LangGraph's **custom-auth** model. Point `auth.path` at a module exporting a
[`@langchain/langgraph-sdk/auth`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk) `Auth`
instance — the same class LangGraph Platform uses, so an existing auth file is drop-in:
```ts
import { Auth, HTTPException } from "@langchain/langgraph-sdk/auth";
export const auth = new Auth()
.authenticate(async (request) => {
const token = request.headers.get("authorization")?.replace(/^Bearer /, "");
const user = token ? await verify(token) : undefined;
if (!user) throw new HTTPException(401, { message: "Unauthorized" });
return { identity: user.id, permissions: user.scopes }; // becomes ctx.user
})
.on("threads", ({ user }) => ({ owner: user.identity })); // scope threads (and their runs) by owner
```
- `.authenticate(...)` runs on every request; throwing an `HTTPException` (or returning nothing) is a
`401`. Its return becomes the `user` passed to the `.on(...)` handlers.
- `.on("<resource>:<action>", ...)` (or a broader `"threads"` / `"*"`) authorizes each operation:
return `false` for a `403`, or a **filter** object to scope the resource. A returned filter both
hides other owners' rows on reads and stamps ownership onto new rows. Runs authorize through their
owning thread (there is no separate `runs` resource), matching LangGraph.
- **No `auth` block** → the server is fully open (unauthenticated), exactly as before.
- Studio's `x-auth-scheme: langsmith` header is caller-controlled, so the CLI requires credentials for
Studio even when `disable_studio_auth` is `false`. `skein dev --dev-ui-access` or
`auth.dev_ui_access: true` prints a temporary API key accepted by Studio and the console on a
loopback server. `skein start` never enables it, and production builds strip the setting.
Direct config loaders also default `disable_studio_auth` to `true`; an explicitly injected engine
controls its own policy.
See [agent-protocol.md](./agent-protocol.md#authentication--authorization) for the full request
lifecycle and the route → resource/action map.
## `dev` vs `up`
- **`skein dev`** — a single Node app process with hot reload. It defaults to in-memory
(optionally file-backed) state, so Docker is not required; pass
`--store postgres --queue redis` to develop against the production drivers, including Postgres
and Redis running in Docker. Fast feedback, the exact `langgraph dev` niche.
- **`skein up`** — Docker Compose bringing up the app plus Postgres (checkpoints + protocol
resources + pgvector) and Redis (queue + cross-instance streaming). Mirrors production —
see [runs-and-redis.md](./runs-and-redis.md).
## Migrating an existing `langgraph dev` state
When you run `langgraph dev`, LangGraph persists all local state — assistants, threads, runs,
store items, and graph **checkpoints** (state + full history) — under a `.langgraph_api/`
directory. skein reads that format and carries everything over, so switching loses nothing.
**Automatic (the common case).** The first time you run `skein dev` in a project that has a
`.langgraph_api/` directory but no skein state yet (`.skein/dev-state.json`), skein imports it on
boot and logs what it brought over. From then on it persists to `.skein/` as usual, so the import
runs exactly once. Nothing to do — your old threads and their history are just there.
**Explicit — `skein import-langgraph`.** For re-running, previewing counts, or importing into a
durable database:
| Flag | Values | Default | Notes |
| ------------------ | -------------------- | -------------------------- | ------------------------------------------------------------------------------------- |
| `-c, --config` | path | `langgraph.json` | Resolves the project directory. |
| `--store <driver>` | `memory`, `postgres` | `memory` | `memory` writes `.skein/dev-state.json`; `postgres` loads a live DB (`POSTGRES_URI`). |
| `--from <dir>` | path | `<project>/.langgraph_api` | Source directory to read. |
| `--force` | — | off | Overwrite an existing `.skein/dev-state.json` (memory target). |
```bash
skein import-langgraph # → .skein/dev-state.json, then `skein dev`
skein import-langgraph --store postgres # → your POSTGRES_URI (checkpoints + resource rows)
```
**What carries over, and what doesn't.** Assistants (with their **version history**), threads (with
their latest state), runs, store items, and the **full checkpoint history** are all migrated. Because
skein uses LangGraph's own `MemorySaver`/`PostgresSaver`, the checkpoint format is identical — history
is preserved exactly. Minor, deliberate drops: the store's derived **embedding index** (`vectors`) is
not copied (skein re-indexes on write / a configured `store.index` re-embeds on Postgres import), and
LangGraph **retry counters** aren't part of skein's model. Run **webhooks** _are_ carried through, so
an imported run still fires its completion webhook.
> The `.langgraph_api/` layout is a `@langchain/langgraph-api` internal (stable across 1.2.x–1.4.x),
> not a public API. The importer is best-effort and guarded — a format it can't read never blocks
> `skein dev` from starting.
>
> **Already on Postgres?** If your LangGraph deployment already uses a Postgres checkpointer, you
> don't need to migrate checkpoints at all: point skein at the **same `POSTGRES_URI`** and it shares
> the exact same checkpoint tables. `skein import-langgraph --store postgres` is for moving an
> in-memory `langgraph dev` (including one run in production) onto a durable skein deployment.
## Graph UI and custom HTTP applications
Standard `ui` and `ui_config` use the public `@langchain/langgraph-ui` build/watch API. For example:
```json
{
"graphs": { "agent": "./src/agent.ts:graph" },
"ui": { "agent": "./src/ui.tsx" },
"ui_config": { "shared": ["react", "react-dom"] },
"http": { "app": "./src/http-app.ts:app" }
}
```
A UI source exports a default component registry. Development watches the sources; `skein build`
compiles assets, schemas and the custom application into the artifact. Production serves
`POST /ui/{agent}` bootstrap HTML and `GET /ui/{agent}/{asset}` from those compiled assets without
loading the UI compiler or project sources. Application exports may be a Fetch function or an object
with `fetch(request)`, including Hono. They return a standard `Response`; a 404 continues to UI and
Agent Protocol routes, with the original request body available even if the app read its copy.
All five adapters share this behavior, including mounted path prefixes.
Deployment authentication runs before custom routes and UI dispatch, preserving the mounted
protocol routes’ existing exemptions: the `/info` handshake is public, and channel callbacks use
provider-signature verification. When rendering external UI
scripts or styles in a browser, use credentials the browser can supply on asset requests (such as
same-origin cookies), or explicitly allow those asset paths in the deployment's auth policy.
`auth.openapi` may describe that existing policy using standard `securitySchemes` and `security`;
it does not change authentication or authorization.
---
<!-- source: docs/embedding.md -->
# Embedding a graph you already have
> **User guide.** skein-js has **two on-ramps**. If you already run the LangGraph CLI, the
> [drop-in](./langgraph-cli-compat.md) (`langgraph dev` → `skein dev`, unchanged `langgraph.json`) is
> for you. This doc is the **other** on-ramp: you have a LangGraph.js graph in your own app and never
> adopted the LangGraph Platform's project shape — no `langgraph.json`, no CLI. Bring the compiled
> graph in code and get the same Agent Protocol server in a few lines.
## Two on-ramps
| | Drop-in CLI (`{ config }`) | In-code embedding (`{ deps }`) |
| -------------------- | ----------------------------------------------------- | --------------------------------------------------------------- |
| You start from | a `langgraph.json` + the `skein` CLI | a **compiled graph object** in your own code |
| Wiring | `createExpressServer({ config: "./langgraph.json" })` | `createExpressServer({ deps: embedInMemoryGraphs({ graph }) })` |
| Graph loading | `path:export` resolved from disk (vite/TS loader) | you already hold the graph — nothing is loaded from disk |
| Best for | migrating off / comparing against the LangGraph CLI | greenfield apps, or anyone who never used the Platform |
| Static graph schemas | ✅ extracted from source | 🟡 stubbed (see [trade-off](#the-one-trade-off-schemas)) |
Both produce the **exact same** Agent Protocol server — same threads/runs/streaming/HITL/persistence,
same `useStream` / Agent Chat UI / LangGraph SDK compatibility. The only difference is how graphs get
in and how `ProtocolDeps` is assembled. Everything downstream is identical.
### Choose an assembly for your host
| Helper | Storage | Choose it for |
| ------------------------------------------------- | ------------------------------------------------- | ----------------------------------------------------------- |
| `embedInMemoryGraphs` from `@skein-js/server-kit` | Memory | Development, tests, or state owned elsewhere |
| `embedSqliteGraphs` from `@skein-js/local` | SQLite resources, queue, and upstream checkpoints | A local app, CLI, daemon, or one Node process |
| `embedPostgresGraphs` from `@skein-js/runtime` | Postgres, optionally Redis | A durable server, with Redis for workers spanning instances |
The memory example below is the smallest server. For the local option, see
[SQLite embedding and host lifecycle](./local-deployment.md). Its helper starts no worker or
listener; your host owns startup and stops its runtime before disposing storage.
## The whole thing
For receipt-based access without HTTP, use the runtime's supported
[native client](./native-access.md). It adds authorized atomic admission, strict resume targets and
retained results for embedded callers. The optional BullMQ access adapter uses the same boundary.
```ts
import { createExpressServer } from "@skein-js/express";
import { embedInMemoryGraphs } from "@skein-js/server-kit";
import { graph } from "./my-graph.js"; // ← your existing `new StateGraph(...).compile()`
const server = await createExpressServer({ deps: embedInMemoryGraphs({ agent: graph }) });
await server.listen(2024);
```
That's a full server. Point any Agent Protocol client at `http://localhost:2024`:
```ts
import { Client } from "@langchain/langgraph-sdk";
const client = new Client({ apiUrl: "http://localhost:2024" });
const thread = await client.threads.create();
await client.runs.wait(thread.thread_id, "agent", {
input: { messages: [{ role: "user", content: "hello" }] },
});
```
`embedInMemoryGraphs` ([`@skein-js/server-kit`](https://github.com/skein-js/skein-js/tree/main/packages/server-kit)) turns a **graph map** into a
`ProtocolDeps` backed by in-process drivers — the store, run queue, event bus, and checkpointer. No
config file, and nothing to import from a storage package. `{ deps }` is the seam **every** adapter
accepts, so the same `deps` mounts on Express, Fastify, NestJS, or Next.js unchanged.
Runnable version: [`examples/embed-graph`](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph).
> **⚠️ Auth is off by default.** `embedInMemoryGraphs` sets no `auth`, so the server it produces
> **authenticates nothing** — every request is allowed (the same default as a `langgraph.json` with no
> `auth` block). That's fine behind your own middleware or on a private network, but mounting `{ deps }`
> on a **public** app exposes `/threads`, `/runs`, and `/store` to anyone — including running your graph
> (spending model tokens) and reading/writing the long-term store. Add an `auth` engine before you go
> public — see [Bring your own drivers, auth, logger](#bring-your-own-drivers-auth-logger).
## The graph map
Keys become graph ids (one auto-registered assistant each). Values are either a **compiled graph** or a
**factory** — a function that builds one, called with the run's `configurable`. Factories are how you
defer expensive or key-requiring construction until a graph is actually run:
```ts
embedInMemoryGraphs({
echo, // a compiled graph, imported eagerly
// built lazily on first use — keeps a keyless boot when the model needs an API key:
agent: async () => (await import("./agent-graph.js")).graph,
// or per-run config: (config) => buildGraph(config.configurable?.model),
});
```
A concretely-typed `.compile()` result (e.g. from `MessagesAnnotation`) is accepted **without a cast** —
the [`EmbeddableGraph`](#api-reference) type leaves the graph's generics open on purpose.
## Standalone or embedded
`{ deps }` works with every adapter, in both its standalone and embedded form:
```ts
const deps = embedInMemoryGraphs({ agent: graph });
// Express — standalone server, or mounted on your existing app.
// `skeinRouter` is async: it seeds assistants and starts the run worker before returning.
await (await createExpressServer({ deps })).listen(2024);
app.use((await skeinRouter({ deps })).router);
// Fastify — plugin under a prefix:
await app.register(skeinPlugin, { prefix: "/agent", deps });
// NestJS — dynamic module:
@Module({ imports: [SkeinModule.forRoot({ deps })] })
// Next.js — App Router catch-all (same-origin, no second server):
export const { GET, POST, PUT, PATCH, DELETE, OPTIONS } = createSkeinRouteHandlers({ deps });
```
The Next.js App Router case is the lightest full-stack story — an 11-line `route.ts` serving the
protocol same-origin behind a `useStream` UI. See [`examples/nextjs-app`](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app).
Each adapter mounts a little differently — prefixes, CORS, body limits, `/ok` and shutdown all vary.
[adapters.md](./adapters.md) covers each one in full.
## Bring your own drivers, auth, logger
`embedInMemoryGraphs(graphs, overrides)` takes a second argument that replaces any field of
`ProtocolDeps` except `graphs` (the first argument is the single source of graphs) — a driver, an
`auth` engine, a `logger`. Supplying `auth` is how you close the open-by-default surface from the
warning above:
```ts
import { loadAuthEngine } from "@skein-js/config";
embedInMemoryGraphs({ agent: graph }, { auth: await loadAuthEngine(/* … */), logger: myLogger });
```
A `logger` set here is used unless the adapter is given an explicit `logger` option, which wins. Under
NestJS and Fastify, leaving both unset falls back to the host framework's own logger rather than to
silence — see [errors-and-logging.md](./errors-and-logging.md#what-each-adapter-does-by-default).
## Going to production
The in-memory drivers are ideal for a single long-lived process (dev, tests, a small app). For durable,
horizontally-scalable state, use **`embedPostgresGraphs`** — the persistent sibling of
`embedInMemoryGraphs`. Same graph-in-code call, but it assembles a Postgres store + `PostgresSaver`
checkpointer and (when a Redis URL is present) a Redis run queue + event bus, reading `POSTGRES_URI` /
`REDIS_URI` from the environment:
```ts
import { createExpressServer } from "@skein-js/express";
import { embedPostgresGraphs } from "@skein-js/runtime";
import { graph } from "./my-graph.js";
const { deps, dispose } = await embedPostgresGraphs({ agent: graph }); // reads POSTGRES_URI / REDIS_URI
const server = await createExpressServer({ deps });
await server.listen(2024);
// It owns pools/connections, so release them on shutdown (embedInMemoryGraphs has nothing to release):
process.on("SIGTERM", () => dispose().finally(() => process.exit(0)));
```
> **⚠️ Auth is still off by default — and this is the production path.** Like `embedInMemoryGraphs`,
> `embedPostgresGraphs` sets no `auth`, so the server **authenticates nothing**. That's easy to miss
> here precisely because you reach for this helper to _deploy_: shipping it public with no `auth`
> exposes `/threads`, `/runs`, and `/store` — and running your graph (spending model tokens) — to
> anyone. Pass an `auth` engine via `overrides` before you go public (see below).
It lives in [`@skein-js/runtime`](https://github.com/skein-js/skein-js/tree/main/packages/runtime), not `@skein-js/server-kit` — a persistent helper
pulls in the Postgres/Redis drivers that `server-kit` deliberately avoids. Pass explicit
`postgresUri` / `redisUri` (and `index` for pgvector, `ttl`, `poolMax`, `sslNoVerify`) instead of env
vars if you prefer, and `overrides` for `auth` / `logger` / etc. — see the [API reference](#api-reference):
```ts
import { loadAuthEngine } from "@skein-js/config";
const { deps, dispose } = await embedPostgresGraphs(
{ agent: graph },
{ overrides: { auth: await loadAuthEngine(/* … */) } },
);
```
> **Redis is optional, but then you're single-instance.** With no `redisUri` / `REDIS_URI`, the run
> queue + event bus fall back to in-memory: state still survives a restart (it's in Postgres), but the
> run queue is process-local and streaming isn't fanned across instances, so you **can't run more than
> one instance**. Set a Redis URL to scale horizontally.
**Sizing the in-memory bus.** On the Redis-less path the event bus holds run frames in the process,
bounded by `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN` (10000) and `SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS` (50)
rather than by a Redis TTL. The worst case is roughly `MAX_FRAMES_PER_RUN × (concurrent runs +
MAX_RETAINED_RUNS)` — half a million frames at the defaults, which is hundreds of MB. Both are read from
the environment, so they reach an embedded host without a code change. Size them against what your graph
actually emits, and note that a far-behind subscriber loses the oldest frames here where Redis would
still have them: [performance.md](./performance.md) has the sizing table and the triage symptoms.
**Signing run-completion callbacks.** `SKEIN_WEBHOOK_SECRET` is read on the embed paths too, so
setting it is all an embedded host has to do to get signed callbacks — there is no `langgraph.json`
here to carry a `skein.webhooks` block. Pass `overrides.webhooks` to configure the rest of the
delivery policy (retries, `allowed_hosts`, `max_payload_bytes`); it is spread last, so an explicit
value wins over the environment. See [webhooks.md](./webhooks.md).
> Before this was wired up, an embedded host that exported `SKEIN_WEBHOOK_SECRET` sent **unsigned**
> callbacks with no warning — the env var was only read on the `langgraph.json` path. If you embed and
> rely on signatures, check you are on a version that includes this.
Prefer to assemble the drivers yourself (e.g. a Postgres store with an in-memory queue, or your own
pool)? Pass them through `embedInMemoryGraphs`' `overrides`:
```ts
import { PostgresSaver } from "@langchain/langgraph-checkpoint-postgres";
import { RedisRunEventBus, RedisRunQueue } from "@skein-js/redis";
import { createPostgresPool, PostgresSkeinStore } from "@skein-js/storage-postgres";
const checkpointer = new PostgresSaver(createPostgresPool(process.env.POSTGRES_URI!));
await checkpointer.setup();
const deps = embedInMemoryGraphs(
{ agent: graph },
{
store: await PostgresSkeinStore.connect(process.env.POSTGRES_URI!),
checkpointer,
queue: new RedisRunQueue(process.env.REDIS_URI!),
bus: new RedisRunEventBus(process.env.REDIS_URI!),
},
);
```
If you _do_ have a `langgraph.json`, [`@skein-js/runtime`](https://github.com/skein-js/skein-js/tree/main/packages/runtime)'s
`buildRuntime({ configPath, store: "postgres", queue: "redis" })` assembles all of these for you —
`embedPostgresGraphs` is the same assembly for graphs you hold in code.
Serverless/edge deploys need durable drivers: the in-memory drivers (and the background run worker) assume
one warm process, so they don't survive a function that scales to zero. See
[storage.md](./storage.md) and [runs-and-redis.md](./runs-and-redis.md).
Bundling the result yourself? The in-code path is the friendly one — it never reaches the
`langgraph.json` graph loader, so everything on it bundles cleanly, `@skein-js/storage-postgres`
included. See [bundling.md](./bundling.md).
## The one trade-off: schemas
A **compiled** graph no longer carries its TypeScript source, so the in-code path can't extract real
input/output/state JSON schemas — `embedInMemoryGraphs` returns a minimal `{ graph_id }` stub for the
assistants introspection endpoints. This is enough for **everything `useStream` and Agent Chat UI
render**; the only thing that degrades is **LangGraph Studio's** schema-driven forms and its graph/step
views. If you need full static schemas, use the [`{ config }` path](./langgraph-cli-compat.md) — the
`langgraph.json` loader runs `getStaticGraphSchema` over the graph source at build time.
## API reference
From [`@skein-js/server-kit`](https://github.com/skein-js/skein-js/tree/main/packages/server-kit):
```ts
// Build a ProtocolDeps around in-process drivers. Pass a graph map OR a ready GraphResolver.
function embedInMemoryGraphs(
graphs: GraphResolver | Record<string, EmbeddableGraph>,
overrides?: Omit<Partial<ProtocolDeps>, "graphs">, // every driver/auth/logger except `graphs`
): ProtocolDeps;
// Turn just the graph map into a GraphResolver (the ids/load/schemas seam the engine consumes).
function graphMapToResolver(graphs: Record<string, EmbeddableGraph>): GraphResolver;
// A graph you can embed: any compiled LangGraph.js graph, or a factory that builds one per run.
type EmbeddableGraph = CompiledGraph<any> | ((config: { configurable?: Record<string, unknown> }) => …);
```
> `embedInMemoryGraphs` was previously named `createInMemoryDeps`. The old name is still exported as a
> deprecated alias, so existing imports keep working — prefer `embedInMemoryGraphs` in new code.
`graphMapToResolver` is useful on its own when you want the resolver but your **own** `ProtocolDeps`.
`normalizeEmbeddableGraphs(graphs)` accepts either a graph map or a ready `GraphResolver` and returns a
`GraphResolver` — the same normalization both embed helpers apply.
### Assembling `ProtocolDeps` by hand
The engine drives an **`AgentGraph`** (`stream` + `getState` required, the rest optional), so
`@skein-js/agent-protocol` installs with no graph runtime. Running **LangGraph.js** graphs therefore
needs the binding wired in — four fields the embed helpers set for you, and which a hand-assembled
`ProtocolDeps` must set itself:
```ts
import { cloneLangGraphCheckpoint, langGraphResolver, SkeinBaseStore } from "@skein-js/langgraph";
import { MemorySaver } from "@langchain/langgraph";
const deps: ProtocolDeps = {
store,
queue,
bus,
checkpointer,
// Translates the engine's command envelope into a LangGraph `Command`.
// Without it, human-in-the-loop resume silently no-ops.
graphs: langGraphResolver(graphMapToResolver({ agent })),
// Bridges long-term memory in, so nodes reach it via `getStore()`.
storeBridge: (repo) => new SkeinBaseStore(repo),
// The throwaway saver `POST /invoke/:graph_id` runs against, so nothing persists.
ephemeralCheckpointer: () => new MemorySaver(),
// Clones a checkpoint when it is re-put under a new thread id (copy / prune / rollback).
cloneCheckpoint: cloneLangGraphCheckpoint,
};
```
`langGraphResolver` is safe to apply to any resolver: it binds LangGraph compiled graphs and returns
anything else untouched, so a resolver fronting your own `AgentGraph` still receives the envelope.
From [`@skein-js/runtime`](https://github.com/skein-js/skein-js/tree/main/packages/runtime) (the durable path — see
[Going to production](#going-to-production)):
```ts
// Build a durable ProtocolDeps (Postgres store + PostgresSaver, Redis queue/bus when configured) and a
// dispose() to release the pools/connections it owns. Async, because it connects + migrates on the way up.
function embedPostgresGraphs(
graphs: GraphResolver | Record<string, EmbeddableGraph>,
options?: {
postgresUri?: string; // default process.env.POSTGRES_URI (required — one of the two)
redisUri?: string; // default process.env.REDIS_URI (absent → in-memory queue/bus, single instance)
index?: StoreIndexConfig; // pgvector semantic search (a resolved embedder)
ttl?: StoreTtl; // store-item expiry + background sweep
threadTtl?: ThreadTtl; // thread expiry — the in-code `checkpointer.ttl`
poolMax?: number; // default env PG_POOL_MAX
sslNoVerify?: boolean; // default env DATABASE_SSL_NO_VERIFY
connectionTimeoutMs?: number; // default env PG_CONNECTION_TIMEOUT_MS, else 30s (0 = wait forever)
idleTimeoutMs?: number; // default env PG_IDLE_TIMEOUT_MS
statementTimeoutMs?: number; // default env PG_STATEMENT_TIMEOUT_MS, else 30s (0 = no limit)
maxPageSize?: number; // default env SKEIN_MAX_PAGE_SIZE, else 1000 (list/search bound)
overrides?: Omit<Partial<ProtocolDeps>, "graphs" | "store" | "queue" | "bus" | "checkpointer">;
},
): Promise<{ deps: ProtocolDeps; dispose(): Promise<void> }>;
```
Unlike `embedInMemoryGraphs` (which returns a plain `ProtocolDeps`), `embedPostgresGraphs` is **async** and
returns `{ deps, dispose }` — it owns Postgres pools and Redis connections, so you must `dispose()` them on
shutdown.
## See also
- [langgraph-cli-compat.md](./langgraph-cli-compat.md) — the other on-ramp (drop-in CLI + `langgraph.json`)
- [agent-protocol.md](./agent-protocol.md) — the endpoints you get either way
- [building-an-adapter.md](./building-an-adapter.md) — putting the engine on any HTTP framework
- [storage.md](./storage.md) · [runs-and-redis.md](./runs-and-redis.md) — swapping in production drivers
- [`examples/embed-graph`](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) · [`examples/nextjs-app`](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app)
For durable local embedding, use [`embedSqliteGraphs`](./local-deployment.md) from `@skein-js/local`.
It uses the same protocol and graph bindings with one owned SQLite file.
---
<!-- source: docs/local-deployment.md -->
# Durable local deployment
`@skein-js/local` assembles the existing Agent Protocol engine with SQLite persistence and LangGraph's
SQLite checkpointer. Use the same graphs in a desktop application, terminal tool, separate daemon,
or server. The application chooses models and tools, including any connection to a local model.
Choose this deployment when one Node process can own the database file. For a shared service with
multiple instances, use [Postgres and Redis](./deploy.md). The default development server continues
to use memory drivers; SQLite is selected explicitly and does not change existing defaults.
## Embed agents
```sh
npm install @skein-js/local @langchain/langgraph @langchain/langgraph-checkpoint-sqlite@~1.0.4
```
```ts
import { embedSqliteGraphs } from "@skein-js/local";
import { createProtocolRuntime } from "@skein-js/agent-protocol";
const local = await embedSqliteGraphs(
{ assistant: graph },
{
sqlitePath: "/absolute/application-data/state.sqlite",
// maxPageSize, ttl, threadTtl and non-owned ProtocolDeps overrides are optional.
},
);
const runtime = createProtocolRuntime(local.deps);
await runtime.service.assistants.registerGraphAssistants();
runtime.worker.start();
// For authorized receipt-based work, use runtime.createNativeClient({sourceId, allowedAssistants}).
// Use runtime.service for trusted administration, or give local.deps to a framework adapter.
// The host stops its worker/adapter before disposing owned resources.
await runtime.worker.stop();
await local.dispose();
```
The helper starts no assistants, workers, listener, application process or global signal handler.
See [native access](./native-access.md) for durable receipts, retained results and strict resume
without HTTP. Native process-loss outcomes use the existing outbox; SQLite still never automatically
replays an abandoned execution.
It owns storage and a small item-expiry timer. The existing protocol runtime owns the scheduler,
thread-retention sweep, idempotency sweep and delivery worker. An adapter manages that runtime's
lifecycle through its existing APIs. `dispose()` is idempotent.
For direct storage use, `openSqliteStorage` from `@skein-js/storage-sqlite` returns the existing
`store` and `queue` contracts, the borrowed native `database`, and `dispose()`. The helper's
`sqlitePath` and driver's `filename` must be absolute. Both create the requested parent directory.
Do not close or change ownership settings on the borrowed connection.
`overrides` retains existing auth, logger, clock, telemetry, webhook and external `storeItems`
injection. The owned graph/storage/queue/bus/checkpointer bindings cannot be replaced there.
The two libraries do not install server-kit, schema/config loading, Postgres, Redis, BullMQ or HTTP
adapters. Add an existing adapter when a transport is needed.
## CLI and scaffolding
```sh
npm create skein-js@latest my-agent -- --storage sqlite
cd my-agent
npm run dev
npm run build
npm start
```
Or select the SQLite pair on an existing project:
```sh
skein dev --store sqlite --queue sqlite
skein start -c .skein/build/langgraph.json --store sqlite --queue sqlite
```
Install `@skein-js/local` and the saver in the project first. Both commands accept `--sqlite-path`;
the CLI resolves it against the working directory and defaults to `.skein/state.sqlite` there.
The file remains outside the build artifact. SQLite does not import memory development snapshots.
Mixed SQLite/non-SQLite store and queue pairs are refused. Existing command defaults remain unchanged.
The LangGraph import command retains its memory and Postgres targets.
The config path keeps auth, channels, telemetry, retention and external-store configuration. SQLite
provides text/filter search and rejects `store.index`; configure an external semantic store through
its existing adapter seam. SQLite scaffolds omit Compose and external-service environment settings
and include `dev:memory` as an alternative. The scaffolder adds no desktop-framework or model axis.
## What survives a process ending
| Driver | Unclaimed queued work | Abandoned executing attempt | Stream frames |
| ------ | ----------------------------------- | ---------------------------------------------------------- | ---------------------------- |
| Memory | Lost with the in-memory queue | No durable failure accounting | Process-local |
| SQLite | Persists with its original due time | Existing `error` status with `ProcessLostError`; no replay | Process-local |
| Redis | Persists in Redis | Existing BullMQ stalled-job/redelivery behavior | Existing Redis bus retention |
SQLite holds exclusive file ownership from an initialization write until disposal. A second owner
is refused; an OS process exit releases the lock. Startup examines abandoned runs and queue claims
in internal batches without the public page cap. Pending inline requests also become failures.
Terminal outcomes, including real approval interruptions, retain their results. Saved values,
checkpoints and interrupt payloads remain available through existing state/history APIs.
Run creation records server-owned background intent and its absolute due time. SQLite commits the
run, kwargs and queue job together, including guarded admission and cron claims. Enqueue deduplicates
without shortening delays. Cron jobs remain immediately eligible; cron `after_seconds` remains ignored.
Failure accounting does not retry input, resume a checkpoint, cancel a thread's pending backlog,
or reconstruct a successful outcome. Applications use existing run/state APIs to make those choices.
Reconciliation does not synthesize completion callbacks, telemetry or application cleanup. Previously
committed delivery records remain eligible for the existing polling retry worker after restart.
## Desktop and daemon hosts
The maintained consumers are distinct applications:
| Consumer | Application | Model tools and local actions |
| ------------------------------------------------------------------------------------ | ------------------------------------ | ------------------------------------------------------------------------------------- |
| [Electron](https://github.com/skein-js/skein-js/tree/main/examples/desktop-electron) | **Daybook**, a meeting workspace | Read project context, search earlier meetings, review and save follow-ups |
| [Tauri](https://github.com/skein-js/skein-js/tree/main/examples/desktop-tauri) | **Reading Room**, a document library | Read a source document, search the library, review a brief with exact source excerpts |
| [CLI](https://github.com/skein-js/skein-js/tree/main/examples/local-cli) | **Dayplan**, a daily planner | Read a selected task file and prior plans, review focus blocks, save a Markdown plan |
From the repository on Node 24, install dependencies and choose a provider in desktop Settings or the CLI environment:
```sh
pnpm install
pnpm nx run example-desktop-electron:start
# Or:
pnpm nx run example-desktop-tauri:start
pnpm nx run example-local-cli:start
```
Desktop start commands package and open the applications; their first run requires native build
prerequisites and can take several minutes. The CLI starts directly. The examples offer **Vertex AI**, **Gemini API**, **OpenAI**, **Anthropic**, and **Ollama**. Desktop Settings configures the
provider, model and connection. The CLI uses environment variables:
| Provider | Configuration |
| ---------- | ------------------------------------------------------------------------------------------------ |
| Vertex AI | `MODEL_PROVIDER=vertex`, Google ADC, optional `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` |
| Gemini API | `MODEL_PROVIDER=gemini`, `GOOGLE_API_KEY` (or `GEMINI_API_KEY`) |
| OpenAI | `MODEL_PROVIDER=openai`, `OPENAI_API_KEY` |
| Anthropic | `MODEL_PROVIDER=anthropic`, `ANTHROPIC_API_KEY` |
| Ollama | `MODEL_PROVIDER=ollama`, optional `OLLAMA_BASE_URL` and `OLLAMA_MODEL` |
For Vertex, run `gcloud auth application-default login`; the app discovers a project from environment
or gcloud settings, or accepts one in Settings. No project name/ID is hardcoded. `MODEL_NAME` selects
a model; defaults are `gemini-3-flash-preview` for Google and `gpt-5-mini` for OpenAI. Provider-specific
`GOOGLE_MODEL`, `OPENAI_MODEL`, `ANTHROPIC_MODEL` and `OLLAMA_MODEL` also work. The Vertex region defaults to `global`. Cloud providers require valid access and available billing/quota. Explicit environment preferences override saved
UI preferences. UI-entered keys live only in backend session memory; only non-secret preferences
are persisted. Disconnecting stays in effect across restarts until the user reconnects, even when
environment credentials remain available. Selected source material and tool results are sent to the selected provider. Anthropic defaults to `claude-sonnet-5-5`; Ollama defaults to `qwen3:8b` at
`http://127.0.0.1:11434`. Ollama needs an installed tool-capable model and no API key. The app never
downloads models automatically. All model names and the Ollama endpoint are editable. This is application code: Skein adds no
provider configuration or credential store. Model integration remains application-owned.
Desktop UIs use `@langchain/langgraph-sdk` over the existing HTTP/SSE adapter. Background
`runs.create` and `runs.joinStream` let preparation continue while the user navigates. Actual tool
activity and generated answers stream through `custom` and `messages` modes. Drafts autosave,
conversations persist in separate threads, and pending reviews reopen from SQLite checkpoints.
Approval sends `command.resume`; the host-selected local file action runs only after review.
**History and versions** shows actual requests, run status and checkpoints. **Create alternative**
uses thread copy and state-update APIs without replacing the original. Saved output is searchable
through Skein's shared store. These application features require no additional library API.
Normal launches use the real model. Keyless deterministic fixtures are explicitly selected only
by tests and packaging gates.
Hosts choose application-data paths, launch and shutdown behavior. A separate daemon uses the same
assembly; its host owns supervision, endpoint discovery and credential exchange. No CLI daemon
lifecycle or desktop-specific protocol is introduced. The examples bind loopback on an OS-assigned
port and use a per-launch bearer through existing auth, with `studioAuthDisabled: true`. Credentials
are passed through environment/private bridges, rather than command-line arguments or saved files.
The target matrix is Node 22/24/26, Electron 44 and Tauri 2 on Linux x64, Windows x64, and macOS ARM64/x64.
Packaged consumer gates install tarballs outside the checkout and exercise native addon loading,
writable data paths, streaming, approval resume and credential rejection. Platform support requires
those gates to pass on the corresponding runner; a source build alone supplies no packaging evidence.
Tauri packages an official portable Node 24 distribution, verified against its SHA-256 manifest,
plus the backend directory and native assets. Electron rebuilds the SQLite addon against its own ABI.
See [Tauri resources](https://tauri.app/develop/resources/) for the bundle mapping.
This release does not support SQLite in browsers, Bun/Deno or distributed workers, or promise a
single executable containing every native dependency. Size/startup measurements belong to the
specific packed consumer and platform; the smoke prints package bytes, assembly time and process RSS.
See the [validation record](./local-deployment-validation.md) for measured sizes and completed versus
pending platform gates.
---
<!-- source: docs/local-deployment-validation.md -->
# Local deployment validation
Recorded on 2026-10-19. Local measurements were taken on macOS ARM64; the CI platform results below
come from their corresponding runners. Measurements are samples for this implementation, not
throughput or footprint guarantees.
## Completed checks
- Shared SQLite resource/queue conformance and driver regressions: 218 passed, one existing shared
case skipped. Covers real files, retained delays, deduplication, guarded/CAS admission, migrations,
exclusive ownership, configured retention, bounded reconciliation and disposal.
- Local assembly/checkpointer regressions: six passed. Covers actual approval resume after reopen,
state/history, copy, fork, prune, rollback, deletion, retention cleanup and durable outbox polling.
- Separate subprocess kills before enqueue, before/after commit, after claim, and during an actual
LangGraph run. The real graph retained its checkpoint and its original input was not replayed.
- Affected adapter suites, including terminal stream replay with lost completion state, all five
transports and both Next.js boundaries.
- Existing Postgres/Redis integration suites: 491 passed, two existing cases skipped.
- Existing Node/Bun/Deno server conformance matrix: 14 checks per runtime, all passed. SQLite remains
Node-only.
- Clean packed-library and generated SQLite project installs outside the checkout on Node 22.23.3
and 24.15.0. Declaration consumption, built import/require, dependency closure, scaffold tests,
artifact build, dev and start all passed. Packed-consumer gates use npm 11 for file overrides.
- Packaged Electron 44.9.0 utility-process and Tauri 2/portable Node 24 backend launches passed on
macOS ARM64. Both streamed progress, resumed approval, saved application Markdown output under isolated host-data
paths, and rejected missing credentials and forged Studio headers.
- Affected lint, typecheck and build passed across 47 projects. Next.js builds use `NODE_ENV=production`.
- Affected ordinary tests passed: 2,307 cases across 16 projects. Nx formatting checks passed.
- Local application tests: 18 passed. Covers meeting/reading/planner reviews across SQLite reopening,
edited approval, immutable recap revisions and identical-content reuse, dismissal without output,
source preservation, and credential-safe provider settings.
- Live Gemini 3 Flash verification passed with automatic Google Cloud project discovery: all three
applications called real tools, generated reviews and saved approved output. Desktop assistant
conversations streamed and persisted through the existing SDK. No project ID is hardcoded.
- Vertex AI, Gemini API, OpenAI, Anthropic and Ollama are selectable through UI/environment settings.
SDK request tests cover API-key routing, GPT-5 token parameters, Anthropic headers, Ollama endpoint
selection/tool probes, rejected credentials and non-secret preference restoration. Live OpenAI and
Anthropic generation was not checked because corresponding funded keys were unavailable.
- The installed Ollama models reported completion/vision capabilities without tools. A real local
server check confirmed the UI rejects a model without tool support; no model was downloaded.
Full local generation requires installing a compatible model such as `qwen3:8b`.
- Both desktop interfaces were exercised against real SQLite/Vertex backends: autosave, actual tool
activity, edited review/save, restored chat, independent checkpoint alternatives, provider switching
and dynamic project selection. Normal apps use real models; deterministic fixtures are explicit.
- Updated packed Electron 44.9.0 and Tauri 2 launches passed on macOS ARM64 with all five provider
integrations installed from declared dependencies outside the checkout. The reading proof covers
source excerpts, library search and chat; both hosts reject missing/forged credentials on model
settings routes as well as their ordinary authenticated protocol workflows.
Test-only build dependencies ensure package/export checks inspect both built libraries rather than
skipping absent output.
## Measurements
One clean packed-library sample per Node version, after imports and graph construction:
| Measurement | Node 22.23.3 | Node 24.15.0 |
| ---------------------------------- | ----------------- | ----------------- |
| Local runtime assembly | 9.73 ms | 8.75 ms |
| Process RSS at assembly completion | 123,551,744 bytes | 159,350,784 bytes |
Assembly timing excludes process/module startup and the first graph run. RSS includes Node and
loaded dependencies, and is not incremental library memory. These single samples do not establish
a performance comparison.
Packed npm artifacts measured 6,500 bytes for `@skein-js/local` and 13,894 bytes for
`@skein-js/storage-sqlite`. Those figures exclude dependencies and native assets. The macOS Tauri
bundle measured about 327.86 MiB for Reading Room with five model integrations; a small library
tarball does not imply a comparably small desktop application.
## Completed CI platform gates
[CI run 39261927104](https://github.com/skein-js/skein-js/actions/runs/39261927104) passed all 27 jobs
at commit `a2abf18`, including the following local-deployment matrix:
| Runner | SQLite consumers on Node 22/24 | Packaged Electron 44 | Packaged Tauri 2 |
| ----------- | ------------------------------ | -------------------- | ---------------- |
| Linux x64 | Passed | Passed | Passed |
| Windows x64 | Passed | Passed | Passed |
| macOS ARM64 | Passed | Passed | Passed |
| macOS x64 | Passed | Passed | Passed |
The consumer checks install packed artifacts outside the checkout, exercise native SQLite loading,
and run authenticated protocol workflows. Windows gates verify literal file paths, an explicit
npm 11 CLI, and the installed Tauri backend with normalized Windows paths. These are recorded
launch proofs; shipping applications still own their installer, signing, and update behavior.
---
<!-- source: docs/native-access.md -->
# Native access and BullMQ
Use `runtime.createNativeClient` when a trusted host admits work without HTTP and needs a receipt
that survives retries. It uses the same assistants, threads, authorization, run engine and storage
as HTTP. The optional `@skein-js/bullmq` package connects external command and outcome queues to
that client. It does not replace Skein's internal execution or delivery queues.
The [runnable approval example](https://github.com/skein-js/skein-js/tree/main/examples/native-transports)
includes an embedded SQLite client and an HTTP/BullMQ backend with a durable outcome inbox.
Read [security and operations](./native-access-security.md) before exposing a producer to real data.
## Embedded applications
```ts
import { embedSqliteGraphs } from "@skein-js/local";
import { createProtocolRuntime } from "@skein-js/agent-protocol";
const local = await embedSqliteGraphs(
{ agent: graph },
{
sqlitePath: "/absolute/application-data/agents.sqlite",
},
);
const runtime = createProtocolRuntime(local.deps);
await runtime.service.assistants.registerGraphAssistants();
const client = runtime.createNativeClient({
sourceId: "desktop-approval-worker", // stable across restarts
allowedAssistants: ["agent"], // assistant IDs or graph IDs; required, empty permits none
});
runtime.worker.start();
const receipt = await client.start(
{ assistant_id: "agent", input: {} },
{
idempotencyKey: "approval-42",
correlationId: "request-42",
},
);
// Admission has committed; execution may still be pending.
const run = await client.getRun(receipt.runId);
const recovered = await client.getReceipt("approval-42");
// Once terminal: client.getResult(receipt.runId) returns the whole retained RunResult.
await runtime.worker.stop();
await local.dispose();
```
The constructor is guaranteed on `createProtocolRuntime`'s concrete return and optional on the
`ProtocolRuntime` interface for compatibility with third-party implementations. There is no separate
native constructor. The existing `runtime.service` remains useful for trusted administrative work;
it does not add native authorization, receipts or admission guarantees to independently composed calls.
Sources and caller identity are host configuration. When authentication is enabled, every native call
requires the host's verified `authContext`. The client calls the existing authorizer with canonical
resource IDs and applies its ownership filters. It does not authenticate arbitrary broker claims or
turn graph metadata into authority. HTTP and native sources can use the same authorized thread when
the host maps their credentials to the same principal.
Keys are scoped by stable source and principal. Omit a key to generate one and receive it in
`{ idempotencyKey, threadId, runId }`. Identical retries return the original receipt; changed input
returns `idempotency_key_reused`. Receipt replay and reads recheck current authorization. Access
revocation takes effect immediately. Retention, expiry and resource deletion bound replay guarantees.
Keep a returned generated key before retrying; a new key means a new mutation.
Trusted adapters that derive a run from a command may set `idempotencyInput` to the **original**
command, including all caller-controlled meaning. This keeps retry identity stable if host routing
or start/resume selection changes. Do not set it to a constant or let a caller choose a reduced
fingerprint. The first admission retains its route, execution identity and deadline.
## Interrupts, cancellation and deadlines
After a run reaches `interrupted`, read its state and submit an answer with that exact checkpoint:
```ts
const state = await client.getState(receipt.threadId);
const answer = await client.resume(
{
thread_id: receipt.threadId,
assistant_id: "agent",
expected: {
runId: receipt.runId,
checkpoint: state.checkpoint,
// interruptIds: ["expected-interrupt-id"], // optional additional precondition
},
resume: "approved", // also accepts LangGraph's resume map
},
{ idempotencyKey: "answer-42", authContext: verifiedContext },
);
const cancelled = await client.cancel(answer.runId, {
idempotencyKey: "cancel-42",
authContext: verifiedContext,
wait: true,
// action: "rollback", // existing cancellation semantics
});
```
Only one admitted resume can consume an expected interrupted run/checkpoint. Admission and invocation
validate it under the thread execution lease. A stale target returns `stale_resume`; fetch current
state before submitting a new answer. Checkpoint updates use the same lease. Resuming an already
accepted key recovers its receipt even after execution advanced the checkpoint.
Separate checkpoint, status and concurrency overrides are rejected at runtime. Strict resume
validates every supplied `checkpoint_map` namespace against its current checkpoint before admission
and invocation; a current root ID cannot authorize an older nested checkpoint. Native state reads
hold the execution lease and can wait behind active work. Retry `native_state_changed` if a target
changes during authorization or lease acquisition.
Cancellation returns `{ receipt, status, stopped }`. A terminal status records the authoritative
decision; `stopped` separately confirms execution has stopped. Without waiting, a remote execution
may still be unwinding. Repeated cancellation with `wait: true` still waits for a cancelled execution.
Graphs and tools must honor cancellation signals. Lease loss aborts protected execution after it is
observed; this does not fence arbitrary external effects from tools that ignore their signal.
`deadlineAt` is a trusted absolute ISO timestamp in call options. Expired commands are rejected before
admission. An accepted command keeps its receipt and uses the earlier of its remaining absolute budget
and `runTimeoutMs`; retries cannot extend it. `after_seconds` retains its first absolute eligibility
time. Result reads return `run_result_pending` while active and `run_result_unavailable` if no output
was retained, including cancellation before execution and unserializable output. Serialization failure
does not rewrite a successful execution as a failed run or invent replacement output.
## Durable outcomes
Bind an optional trusted `outcomeTarget` on the source and compose `ProtocolDeps.webhookDispatcher`
to handle it. This opts in to durable terminal accounting for success, error, timeout, interruption,
pending cancellation/displacement and SQLite process loss. Without it native results remain available,
but no callback is owed. Legacy HTTP/channel callback defaults are unchanged.
The existing outbox records the outcome atomically with terminal status and optional retained result.
The dispatcher receives the existing `deliveryId` and `attempt`; channel and routed destination
callbacks now also receive those fields as optional context. Receiver deduplication must persist
`deliveryId` before acknowledging. Delivery is at least once, including a crash after publication
but before outbox acknowledgement. A removed BullMQ job cannot serve as a durable deduplication record.
Outcomes use existing payload/retention limits. Large outcomes carry run references and a truncation
marker. **Stored results remain whole**, independent of delivery truncation and checkpoint pruning.
Results follow existing run/thread deletion. Snapshot import and thread copy retain results but
exclude live native receipt, dispatch and delivery obligations.
Memory guarantees last only for that process. SQLite keeps atomic run/job admission and marks
abandoned attempts `ProcessLostError` on reopen; it does not automatically replay their input.
Postgres recovery drains bounded leased dispatch intents for explicitly admitted background work.
Acknowledging enqueue keeps ownership until run deletion: an active run becomes eligible again when
its lease expires, so a lost queue job can be reconstructed with the same run ID. Terminal runs are
skipped under the execution lease before checkpoint effects. Ordinary inline runs and unrelated
producers' rows are not adopted. Explicit cancellation, displacement, deletion and shutdown grace
expiry broadcast abort requests even when a local registration is only waiting for a remote executor.
Failed cancellation publication remains recoverable. Eligible execution owners also observe durable
terminal status to stop after a missed pub/sub notification. Native rollback preserves the admitted
run and receipt; explicit resource deletion still removes them.
## BullMQ access transport
```sh
npm install @skein-js/bullmq bullmq@^5 ioredis@^5
```
```ts
import { Queue } from "bullmq";
import { createBullMqTransport } from "@skein-js/bullmq";
import { createProtocolRuntime } from "@skein-js/agent-protocol";
const outcomes = new Queue("agent-outcomes", { connection });
const access = createBullMqTransport({
commandQueue: "agent-commands",
outcomeQueue: outcomes, // borrowed
connection, // options or borrowed supported ioredis connection
authenticate: async (job) => ({ authContext: await verifyQueueCaller(job) }),
// route: command => mapToAllowedGraph(command),
// projectResult: (result, runId) => applicationProjection(result, runId),
});
const runtime = createProtocolRuntime({
...deps,
webhookDispatcher: access.wrapDispatcher(existingDispatcher),
});
await runtime.service.assistants.registerGraphAssistants();
const native = runtime.createNativeClient({
sourceId: "backend-queue",
allowedAssistants: ["agent"],
outcomeTarget: access.outcomeTarget,
});
runtime.worker.start();
await access.start(native); // creating the transport consumes nothing
// Shutdown consumption before the runtime and storage.
await access.stop();
await runtime.worker.stop();
await outcomes.close(); // host owns this queue/connection
```
Workers own their blocking connection and are closed by `stop`; borrowed queues/connections remain
open. Keep the outcome queue name and Redis prefix stable across restart. Host-controlled
authentication, routing, deadlines, destinations and output projection are functions/configuration,
not authority accepted from message fields. HTTP remains available independently.
`projectResult` covers outcomes and `result.get`; `state.get` and `run.get` remain full authorized
reads. Deny raw operations in the host's existing callbacks if a producer should see only projections.
Redis credentials also grant access to stored job payloads/return values; configure ACLs, TLS and
separate retention for those copies. See [the security guide](./native-access-security.md).
The adapter owns the following versioned wire format, exposed as `BullMqCommand`:
```ts
await commands.add(
"skein.command.v1",
{
version: 1,
commandId: "work-42",
operation: "start",
input: { assistant_id: "agent", input: {} },
},
{ attempts: 5, backoff: { type: "exponential", delay: 1000 } },
);
```
| Operation | Fields beyond `version`, `commandId`, optional `correlationId` | Job return value |
| ------------- | -------------------------------------------------------------- | ---------------------------- |
| `start` | `input: NativeStartInput` | Admission receipt |
| `resume` | `input: NativeResumeInput` | Admission receipt |
| `cancel` | `runId`, optional `action`, `wait` | Cancellation response |
| `run.get` | `runId`, optional `offset`, `digest` | Bounded chunk of current run |
| `state.get` | `threadId`, optional `offset`, `digest` | Bounded chunk |
| `result.get` | `runId`, optional `offset`, `digest` | Bounded chunk |
| `receipt.get` | `idempotencyKey` | Receipt |
`commandId` becomes the native idempotency key. Mutations acknowledge durable acceptance. The first
admission keeps its diagnostic correlation ID; changing only `correlationId` on a retry does not
change command meaning. Execution
completion arrives as a `BullMqOutcome` (`version: 1`, `operation: "run.settled"`) on the outcome queue.
`eventId` is the outbox delivery ID, with `commandId`, resource IDs, status and a `resultRef`. Small
outcomes may include projected output; oversized ones omit it and set `truncated: true`.
One run has one authoritative terminal outcome; outcomes for different runs may arrive out of order.
Persist receiver deduplication by `eventId` and compare `runId` with the receipt for the turn your
application is displaying before changing that turn's state. Reconcile through `run.get`/`state.get`
when needed; checkpoint coordinates and chunk digests identify the snapshot being read. Arrival order
does not establish thread order.
BullMQ handles retries according to the producer's bounded `attempts`/`backoff`. Validation,
authorization, stale answers and unsupported capabilities fail with `UnrecoverableError`; its JSON
message includes the stable code and status. Admission contention, pending results and infrastructure
failures use normal retry behavior. Default BullMQ attempts remain one unless the producer sets them.
For command acknowledgement waiters, start `QueueEvents` with a retained cursor such as
`lastEventId: "0-0"`, as the runnable example does. Connection readiness alone does not prove that
event consumption has begun; a fast completion can otherwise fall between the waiter's status check
and its first stream read. If acknowledgement times out, recover admission by its stable command key.
Each run/state/result request reauthorizes and returns a base64 byte chunk with SHA-256 `digest`, `offset`,
`nextOffset` and `totalBytes`. Start at offset zero, then supply the returned digest and next offset.
Concatenate decoded bytes and verify the digest before parsing JSON. `result_digest_mismatch` means
the value/projection changed; restart retrieval. No HTTP is needed to reconcile missed or truncated
outcomes. Native `getResult` always returns the whole result. `maxMessageBytes` defaults to the
existing 256 KiB payload cap and can be reduced to 1 KiB. The supported dependency majors are BullMQ 5/6 and ioredis 5/6. The
[validation record](./native-access-validation.md#bullmq-and-ioredis-compatibility) describes the installed
consumer matrix. In mixed BullMQ 5/ioredis 6 installations, pass connection options to BullMQ queues
and workers so BullMQ retains its own client; the Skein transport also accepts borrowed ioredis 6
`Redis` and `Cluster` clients. No casts are needed in application code.
The adapter uses the borrowed outcome queue's publishing operations, so an application's BullMQ
major upgrade can retain a different supported version inside the adapter. For explicitly typed
options, use `BullMqTransportOptions<typeof outcomes>` to retain the application's full Queue type;
existing `BullMqTransportOptions` annotations keep their original default.
## Custom drivers and verification
Native writes explicitly require optional `SkeinStore.admission` and `RunRepo.settle`/`getResult`;
missing guarantees return `native_admission_unsupported` instead of emulating transactions with CRUD.
`admitRun` commits the scoped key/fingerprint, guarded run/displacements, receipt, resume reservation
and dispatch intent together. `admitCancel` does the corresponding cancellation transaction.
`claimDispatches` leases bounded batches; `markDispatched` only acknowledges the matching token.
Run intents remain recoverable while pending/running after acknowledgement; cancel intents stop after
acknowledgement. Retain admission ownership until run deletion, and atomically refuse a transition
from terminal to running for owned runs, returning the effective row unchanged.
`settle` atomically records the first terminal status, first matching result and optional identified
delivery, returning the effective rows. Existing `recordResult`, `getResult`, `finalizeWithDelivery`
and third-party-driver paths remain supported. Postgres leases may provide an optional loss signal.
New `settle` deliveries start pending at attempt zero; the delivery worker starts attempt one.
Legacy `finalizeWithDelivery` keeps its inline attempt and existing defaults.
`RunAdmission.signal` is an optional ephemeral `AbortSignal`. Drivers observe it before mutations and
at their pre-COMMIT boundary and roll back when it is aborted before COMMIT dispatch. It is never
stored in kwargs, keys or snapshots. Abort can race a COMMIT already sent: a committed receipt remains
authoritative and is recovered by key. This is cooperative admission cancellation, not database fencing.
The packed consumers run outside the workspace with package-root imports:
`pnpm exec nx run test-support:test-native-consumer`. They include an existing-API gap reproduction,
an HTTP/BullMQ approval worker on Postgres/Redis, durable receiver deduplication after job removal,
and a SQLite daemon killed and reopened without HTTP. See [the implementation evidence](./native-access-validation.md).
Existing MCP and A2A endpoints keep their protocol-specific identity, state and transport behavior.
This boundary adds no protocol endpoints, SDK dependency, workflow engine, artifact service or
exactly-once external-effect guarantee.
---
<!-- source: docs/native-access-security.md -->
# Native access security and operations
Native and BullMQ access compose existing authorization, admission, execution and outbox guarantees.
The host supplies verified identity and policy. Start with the
[runnable approval example](https://github.com/skein-js/skein-js/tree/main/examples/native-transports)
and the [API guide](./native-access.md).
## Authority at the boundary
Construct native clients in trusted code. Keep `sourceId`, `allowedAssistants` and `outcomeTarget`
stable and outside caller-controlled input. With authentication enabled, supply verified
`authContext` on every operation. A principal, scopes or destination in metadata or a queue payload
does not authenticate anyone. Receipt replay and reads recheck current authorization and ownership.
For BullMQ, `authenticate(job)` must verify the producer using the host's credential arrangement or
bind a queue restricted to one trusted backend to a server-owned principal. Trusting `job.data.userId`
would let a producer impersonate another user. The example uses one trusted backend and loopback HTTP
with a demonstration token. Replace that identity policy for deployment. Authentication-disabled
deployments preserve existing unauthenticated behavior.
An allowlist accepts assistant or graph IDs and limits operations for that source. It does **not**
partition data inside a shared thread: other authorized assistants/transports may have written its
checkpoints. Use separate threads and ownership rules for data isolation. Revocation prevents future
authorized reads; it cannot retract values already returned or copied to Redis.
Strict resume accepts the expected interrupted run/checkpoint, optional interrupt IDs and answer.
Runtime validation rejects separate checkpoint, status or concurrency overrides, including JSON
fields TypeScript excludes. The execution lease protects validation and invocation; atomic reservation
prevents replicas consuming the same interrupt.
Each supplied `checkpoint_map` entry must also name the current checkpoint in that namespace for the
authorized thread. A valid root ID cannot redirect a child to an older checkpoint. This check runs
at admission and before invocation, using the existing checkpointer contract without interpreting
LangGraph's internal metadata. Valid nested coordinates retain LangGraph's own execution semantics.
Native state reads authorize a resolved target, then hold the execution lease while constructing
its graph and reading checkpoints. They can wait behind execution. `native_state_changed` asks the
caller to retry if the target changes during authorization or lease acquisition. Pending admissions
cannot switch the graph mid-read. Factories retain the selected run's sanitized configuration and
use the **requesting** principal.
## Projection and broker storage
`projectResult` applies to outcomes and `result.get`. It does not redact `state.get` snapshots or
`run.get` metadata. Reject raw query operations in `authenticate` or `route` when a producer must
see only projected output; both existing callbacks can throw `SkeinHttpError`. Apply corresponding
policy to HTTP and other sources available to that principal. Native `getResult` returns the whole
retained result and belongs behind the host's trust boundary.
BullMQ stores inputs and return values in Redis. Queue names/prefixes organize keys; they are not
tenant access controls. Redis readers can inspect values without Skein authorization. Restrict
credentials, network access and ACLs, and configure TLS. Avoid secrets in commands/results; use
application-managed encryption when needed. Follow
[BullMQ's production guidance](https://docs.bullmq.io/guide/going-to-production) and
[Redis security guidance](https://redis.io/docs/latest/operate/oss_and_stack/management/security/).
Set retention for completed/failed commands, query return values, outcomes and receiver inboxes.
Skein run/thread deletion and outbox expiry do **not** erase external copies. Automatic job removal
does not replace receiver deduplication: persist `eventId` with application database effects before
acknowledging. Keep destination queue identity stable across restart so existing outbox obligations
continue reaching it.
Chunk digests detect changes between reads; they do not authenticate the broker or encrypt output.
Each chunk request reauthorizes. Bound total reassembly size for your application, verify offsets,
size and digest, and restart retrieval on a changed digest. Truncated outcomes leave whole retained
results available through reconciliation.
## Cancellation, recovery and resource use
Acknowledgement, terminal status and confirmed execution stop are separate facts. `wait: true`
reports `stopped: true` after acquiring valid execution rights. A failed stop check returns retryable
`execution_stop_unconfirmed` (503) while
the accepted receipt remains recoverable. Waiting can queue behind other work on that thread.
If an abort channel is configured without a cross-instance execution gate, waiting cancellation is
rejected before admission with `execution_stop_unsupported` (501); a local mutex cannot prove remote
execution stopped. Supported Postgres assembly supplies both capabilities.
Tools must honor abort signals; status changes cannot undo arbitrary external effects.
Redis pub/sub has [at-most-once delivery](https://redis.io/docs/latest/develop/pubsub/). Recovery
retries failed cancellation publication before acknowledging its intent. Successful publication
cannot prove reception. Native executions, and executions with both an abort channel and atomic
admission capability, also observe durable status once per second with one read in flight per
execution. For N active eligible executions, budget approximately N reads/second. Read failures are
logged and retried. Observed terminal status aborts local control without rebroadcasting. Ordinary
single-process runs without these capabilities retain their existing path.
Native rollback retains the admitted run, receipt and stored result for reconciliation. Explicit
deletion still removes them. Cleanup does not revert a later run that advanced the thread. Legacy
runs retain established rollback deletion behavior.
Expired trusted `deadlineAt` is rejected after authorization and immediately before admission. Once
accepted, the original deadline/route remain authoritative. Execution uses the earlier of remaining
budget and configured timeout. This is cooperative admission cancellation, not a database clock
fence or a promise to stop external effects at an exact instant.
Memory is process-local. SQLite requires one file owner, records process loss as failure and never
automatically replays abandoned input. Postgres recovery leases bounded batches of owned dispatches
and reconstructs lost queue jobs with original run IDs. It does not adopt unrelated runs. Observe
dispatch warnings, outbox depth, failed jobs, capacity and receiver lag; reconcile by receipt/run ID.
## Dependency review
The October 25, 2026 review updated compatible Vite, body-parser, follow-redirects and multer patches.
The production dependency audit fell from ten advisories to one high advisory:
[Zod collection validation resource consumption](https://github.com/advisories/GHSA-mrhx-c7xc-m3w5).
The published fix is Zod 4.6.6; this workspace and LangChain dependencies use Zod 3. A forced
transitive major replacement is not a compatible fix. This PR does not claim a clean dependency audit.
Enforce request byte limits, **pre-validation** collection limits and rate/concurrency budgets at
host boundaries, including user-facing bridges into native calls. Array `.max()` inside an affected
validator is not sufficient early rejection. BullMQ's message cap helps bound input but is not a
general resource-exhaustion guarantee. A compatible upstream fix or separately reviewed Zod migration
remains required; audit the actual deployed lockfile again.
---
<!-- source: docs/native-access-validation.md -->
# Native access validation
Implementation base: refreshed `main` at `a2e1a8e` (PR #63), in managed worktree
`issue-64-native-transports`. Existing result storage, migrations through Postgres `0014_run_results`
and SQLite version 3, copy/snapshot behavior and A2A/MCP implementations are reused.
## Existing public API gap
`scripts/fixtures/native-consumers/baseline.mjs` uses only the preserved baseline APIs, through packed
packages and package-root imports in a temporary directory outside the workspace. It calls:
1. `store.idempotency.claim(...)`.
2. `runtime.service.runs.createBackground(threadId, input)`.
3. The missing next operation would be `store.idempotency.record(...)`.
Immediately after step 2, SQLite has a committed run/job but the claim has no linked run. The fixture
models a caller lost at that boundary. Once the original claim expires, its retry creates another
run, even after the first interrupted successfully. These existing methods are intentionally still
available; the fixture bypasses the new admission capability to reproduce their separate-commit
behavior. It is an existing-surface reproduction against the implementation's packed artifacts,
not a claim that those artifacts were built from the untouched baseline commit.
The library addition is the missing transaction and supported native composition. Business approval,
credential verification, routing policy and durable receiver deduplication remain application code.
The result model, protocol registry, workflow engine and artifact service were not duplicated.
## External consumers
Run `pnpm exec nx run test-support:test-native-consumer` with Docker available. The target builds
packages, packs them, installs an npm consumer outside the workspace, typechecks package-root imports
without workspace aliases, and runs:
| Consumer | Contract exercised |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Existing-surface baseline | The exact gap between run/job admission and receipt recording |
| SQLite daemon | Native admission, SIGKILL during execution, reopen, receipt/status/result reconciliation without HTTP; no replay of abandoned input; fresh result survives another reopen |
| Backend approval worker | One graph over actual HTTP and BullMQ, shared authorized thread, strict expected checkpoint, duplicate receipt after broker job removal, stale answer rejection, pending cancellation, chunk offsets/digest and complete output |
| Outcome receiver | SQLite unique event IDs persisted across receiver reopen; publication-before-outbox-ack retry after broker job removal retains the same event ID |
The shared native input/client types are compiled in `types.ts`. Broker-specific schemas live only
in `@skein-js/bullmq`. The transport's async permanent-rejection handling was corrected after the
packed consumer exposed a missing `await` inside its classification boundary.
## Storage and compatibility checks
The shared store conformance suite now exercises concurrent key admission, changed-input conflicts,
leased dispatch reclamation after acknowledgement and queue loss, stale-token acknowledgement,
terminal-to-running guards, expected-target reservation independent
of receipt expiry, atomic terminal/result/delivery settlement and malformed-payload rollback.
Memory, SQLite and real PostgreSQL run the same cases.
Real runtime integration uses separate Postgres/Redis assemblies to race two answers to one expected
interrupt and recover the original admission after producer enqueue failure. Another case abandons an
acknowledged in-memory queue job and recovers it from Postgres without adopting unrelated pending work.
Repeated execution checks terminal status before rollback, and observed lease loss prevents subsequent
checkpoint effects. Local recovery waiters cannot suppress cancellation broadcasts to a remote owner.
Competing state writes and resume admission also share the real Postgres execution gate; an answer
blocked behind a checkpoint update is rejected before admission. Postgres backend
termination checks that a lost execution lease raises its signal and another connection acquires the
thread. SQLite subprocess tests inspect persisted receipts and process-loss outbox obligations.
The optional admission signal was separately audited as a cooperative pre-COMMIT guard. Tests abort
before admission, during authorization and before Postgres COMMIT dispatch; a deferred trigger also
checks that abort racing an already-dispatched COMMIT preserves the accepted receipt.
Existing HTTP, official SDK, browser, MCP and A2A regressions remain required. Shared validators stay
below adapter-specific mapping. Native additions do not require inline/waiting/streaming callers to
accept queued receipts, and optional capabilities preserve existing third-party-driver behavior.
These consumers demonstrate correctness and API usability for two concrete hosts. They do not
establish broad adoption, global ordering or exactly-once external effects. Memory is process-local;
retention/deletion limit deduplication, and tool cancellation remains cooperative.
## Pre-PR code review
The review covered the public exports and optional driver capabilities, per-operation authorization,
receipt replay, dispatch leases, strict resume, atomic result/outbox settlement, snapshot restoration,
broker chunking and connection ownership. An additional lease-loss gap was found in rollback
cancellation: it held an execution claim but did not pass its loss signal to checkpoint replacement.
The cancellation path now passes that signal and retains the run row when rollback cannot complete;
its regression checks that observed lease loss causes no checkpoint erasure or replay.
The A2A wrapper also now maps only stale-target conflicts from shared validation, preserving
infrastructure failures rather than misclassifying storage outages as stale answers. Both cases have
regression coverage.
## Goal and security audit follow-up
The later audit reproduced five gaps with executable probes: a runtime resume checkpoint override,
state target changes during authorization, stop confirmation after swallowed rollback failure,
cancel dispatch acknowledgement after failed publication, and expiration during authorization.
Permanent native regressions now reject the override, freeze state resolution under the execution
lease, require positive stop proof, retain failed publication for retry and recheck admission deadlines.
They also exercise missed successful publications and native rollback receipt retention.
An additional probe reproduced a forged child `checkpoint_map` that replayed a preparation node
despite a correct root checkpoint. Native validation now checks every supplied namespace's current
tip at both boundaries. Memory tests include two levels of subgraphs, valid official nested pointers,
missing/foreign/stale coordinates and a map changed after admission. SQLite reopen and real Postgres
tests also reject the older child pointer and resume the original root without replaying preparation.
The real Postgres/Redis test cancels a service/HTTP-origin run from another replica while the owner's
pub/sub subscriber misses publication. Durable status observation stops execution, then the remote
execution gate confirms it. Existing third-party capability and ordinary cancellation behavior remain
covered. No new public method, interface requirement or configuration key was needed for these fixes.
The maintained `examples/native-transports` project is generated through Nx and imports public package
roots. Its integration target runs built entrypoints as separate processes, reopens SQLite, makes a
real HTTP request, exercises BullMQ start/resume/result retrieval, restarts both backend and receiver,
and verifies receipts/results and the durable event inbox. It complements the externally installed
packed consumers; it does not establish external adoption.
See [security and operations](./native-access-security.md) for the authority and data-retention review,
projection limits, cancellation observation cost and the remaining Zod dependency advisory. Compatible
dependency patches reduce the production audit from ten advisories to one high advisory; no claim of
a clean audit is made.
The original native-access PR addresses #64 and workstreams #65–#70. The compatibility follow-up
for #47 is recorded below.
## BullMQ and ioredis compatibility
On 26 October 2026, the following uncached installations passed locally on Node 24.15.0 and
Node 26.12.1. Node 26 ran in an isolated checkout with its own native dependencies. The
[raw dependency-resolution records](https://github.com/skein-js/skein-js/blob/main/packages/runtime-redis/compatibility-results.json)
are committed alongside the implementation; CI repeats the same nine legs on Linux.
| BullMQ | Direct ioredis | Node 24 | Node 26 |
| ------ | -------------- | ------- | ----------------------------------- |
| 5.80.2 | 5.10.1 | Pass | Pass |
| 5.80.2 | 6.0.0 | Pass | Pass |
| 6.4.1 | 5.10.1 | Pass | Pass |
| 6.4.1 | 6.0.0 | Pass | Pass |
| 6.0.0 | 6.0.0 | Pass | Minimum-release leg runs on Node 24 |
Each leg selects only the three direct dependency owners: Redis, the native BullMQ adapter and its
maintained example. BullMQ 5 retains its own ioredis 5.10.1 dependency even when the application's
direct client is ioredis 6. The runner asserts those resolved versions and restores manifests, the
lockfile and installed links after each selection.
The gates include lint/typecheck/unit/build, Redis and runtime integration, external command/outcome
behavior, native example subprocesses and an externally installed npm consumer. The consumer
typechecks a borrowed client without application casts, performs HTTP/BullMQ admission, strict resume,
receipt/result reconciliation, cancellation and digest-checked retrieval, and confirms that stopping
the adapter leaves its borrowed client and outcome queue usable. Redis coverage includes delivery
retries, pub/sub replay, thread events, abort publication and unused/startup/shutdown ownership.
Two compatibility adaptations are exercised: sorted-set indexes use string wire arguments accepted
by both ioredis majors, and the adapter's existing connection input additively accepts the directly
installed Redis client type. BullMQ 5's nominal connector types are adapted at the vendor boundary;
the application does not need a cast. The deterministic acknowledgement-startup regression uses
BullMQ 6's public backend status operation and retains its v5 barrier.
Run a selection with:
```sh
pnpm exec nx run test-support:test-redis-compatibility --bullmq=6.4.1 --ioredis=6.0.0
```
See [the testing guide](./testing.md#redis-dependency-compatibility) for matrix lifecycle and artifacts.
### Review regressions
The follow-up review reproduced a scheduling error with an existing BullMQ job: a 2,500 ms native
retry backoff was shortened to about 1,000 ms by partition reconciliation. Promotion now consumes a
marker for a specific Skein-parked dequeue attempt, leaving genuine retries to BullMQ. The regression
checks the original backoff floor and successor order against real Redis.
A fresh packed pnpm consumer also reproduced a declaration failure for a borrowed ioredis 6 `Cluster`
with BullMQ 5. The additive connection input now covers both `Redis` and `Cluster`. Packed declaration
checks and a real single-node Redis Cluster test cover command admission and borrowed-client/queue
ownership after shutdown; the latter does not claim cluster failover coverage.
A host-only BullMQ major upgrade exposed another declaration problem: pnpm can retain the adapter's
v5 dependency while the application's borrowed outcome queue is v6. The adapter now requires only
the queue's publishing operations, while `BullMqTransportOptions` keeps its original complete Queue
type as the default. A packed consumer retains its pnpm lockfile through the major upgrade, then
typechecks and exercises command acknowledgements, outcome publication and borrowed-client ownership
across the installed versions.
Additional regressions exercise two consumers sharing a partition, legacy queue fallback,
cancellation during claim acquisition, both execution paths losing ownership during checkpoint
reads/graph loading, and permanent rejection of oversized commands or caller-supplied authority
before native admission. LangGraph still owns graph execution, commands and checkpoint handling;
the review adds no graph runtime or wire fields. The production dependency audit still reports the
previously documented Zod advisory, so this is not a clean dependency-security audit.
Review of the modified Postgres gate also reproduced a pre-existing process-crash path: a TCP drop
during an unlock query emitted an unhandled client error after the gate had detached its listener.
The listener now stays attached through cleanup and is removed immediately before pool handoff or
discard. Subprocess tests fault the actual TCP connection during release for both claim methods;
both exited unsuccessfully before the fix and cleanly afterward.
---
<!-- source: docs/serving-a-single-graph.md -->
# Serving a graph as a plain endpoint — no threads, no runs
> **User guide.** skein's main surface is the full [Agent Protocol](./agent-protocol.md) — threads,
> assistants, runs, streaming, history, human-in-the-loop. That's what a **chat** app needs. Plenty of
> LangGraph work isn't chat: a classifier, an extractor, an enrichment step another service calls. For
> those, this page describes a smaller surface — **one graph, one endpoint, called like a function**.
## Which surface do I want?
| | Full Agent Protocol | Single-graph invoke |
| ------------------ | -------------------------------------------------------------- | -------------------------------------------- |
| Mount | `skeinRouter` / `skeinPlugin` / `SkeinModule` / route handlers | `skeinInvoke*` / `SkeinInvokeModule` |
| Endpoint | `/threads`, `/assistants`, `/runs`, `/store`, … | `POST /invoke/:graph_id` |
| Request | `{ input, config, stream_mode, … }` on a thread | the graph input, raw |
| Response | a run object (or an SSE run stream) | the final graph state |
| Conversation state | persisted per thread (resume, history, time travel) | **none** — each call is independent |
| Best for | chat, HITL, anything resumable | classification, extraction, batch, workflows |
| Clients | `useStream`, Agent Chat UI, LangGraph SDK | `fetch`, `curl`, any HTTP client |
Both run the _same_ graph through the same LangGraph machinery. The difference is how much protocol
sits in front of it. Mounting both is supported and safe: each call attaches its checkpointer and
store to a per-call clone of the compiled graph, so an invoke never disturbs a concurrent protocol
run's durable state.
## The whole thing
```ts
import { skeinInvokeRouter } from "@skein-js/express";
import { embedInMemoryGraphs } from "@skein-js/server-kit";
import express from "express";
import { graph as triage } from "./triage-graph.js";
const app = express();
const { router } = await skeinInvokeRouter({ deps: embedInMemoryGraphs({ triage }) });
app.use(router);
app.listen(2024);
```
```bash
curl -sX POST localhost:2024/invoke/triage \
-H 'content-type: application/json' \
-d '{"text":"Refund charge failed — urgent!"}'
# {"text":"Refund charge failed — urgent!","category":"billing","priority":"P1"}
```
Runnable version: [`examples/invoke-endpoint`](https://github.com/skein-js/skein-js/tree/main/examples/invoke-endpoint).
## The contract
**The request body is the graph input. The response is the graph's final state.** No envelope in
either direction — the graph behaves like a function over HTTP.
- **One endpoint per graph.** Every id in the graph map (or in `langgraph.json`) is mounted at
`POST <prefix>/<graph_id>`. The prefix defaults to `/invoke`.
- **An unregistered id is a 404**, with `code: "graph_not_found"`.
- **An empty body runs the graph with its state defaults** rather than surfacing LangGraph's opaque
`EmptyInputError` — useful for graphs that take no input at all.
- **Responses are serialized with the same wire encoder** the protocol uses, so LangChain messages in
a final state flatten to the `{ type, content }` shape clients expect.
- **A graph that throws is an error response**, mapped through the adapter's normal error path.
## Streaming
The default is a single JSON response. Send `Accept: text/event-stream` and the _same_ endpoint
streams the graph's steps as SSE instead, ending with a terminal `end` (or `error`) event:
```bash
curl -NsX POST localhost:2024/invoke/triage \
-H 'content-type: application/json' -H 'accept: text/event-stream' \
-d '{"text":"Outage — everything is down"}'
```
```text
id: 1
event: values
data: {"text":"Outage — everything is down","category":"","priority":""}
id: 2
event: values
data: {"text":"Outage — everything is down","category":"general","priority":""}
event: end
data: {"status":"success"}
```
The mode defaults to `values` (each frame is the full state after a step, ending at the value the JSON
response would have returned). Override it per mount with `streamMode`, or per request with
`?stream_mode=updates` (comma-separate for several). Modes are validated at the boundary, so an
unknown one is a 400 rather than an opaque failure deeper in. If the graph throws mid-stream the
failure arrives as an `error` frame — headers are already sent, so it can't become an HTTP status.
`stream_mode=events` is **not** available here: it is not a Pregel stream mode (the run engine serves
it from `graph.streamEvents`, while this surface drives `graph.stream`), so it is rejected with a 400.
Use the Agent Protocol run endpoints for token-level events.
## On every adapter
The surface is the same everywhere; only the mount idiom differs.
```ts
const deps = embedInMemoryGraphs({ triage, extract });
// Express — a Router you mount
const { router } = await skeinInvokeRouter({ deps });
app.use(router);
// Fastify — a plugin, encapsulated under its prefix
await app.register(skeinInvokePlugin, { prefix: "/agent", deps }); // → POST /agent/invoke/:graph_id
// NestJS — a dynamic module, alongside your controllers
@Module({ imports: [SkeinInvokeModule.forRoot({ deps })] })
export class AppModule {}
// Next.js App Router — app/api/invoke/[graph_id]/route.ts
export const runtime = "nodejs";
export const { POST } = createSkeinInvokeRouteHandlers({ deps, basePath: "/api/invoke" });
```
Each accepts the same `{ config } | { deps }` seam as the full protocol, so
`skeinInvokeRouter({ config: "./langgraph.json" })` works too — see [embedding.md](./embedding.md).
The Express/Fastify/NestJS mounts claim only their own path and pass everything else through, so the
host app's routes are untouched.
## Auth
`deps.auth`, when configured, is enforced here exactly as on a run — invoking a graph _runs_ it
(spending model tokens), so this is not a way around the gate. The caller is authenticated (401 on
failure) and authorized against `threads:create_run` (403 on deny), and the authenticated principal is
stamped into the graph's `configurable` as `langgraph_auth_user`, just like a protocol run.
Auth runs _before_ the graph-exists check, so an unknown id also returns 401 rather than 404 — an
anonymous caller can't enumerate which graphs you serve by telling the two apart.
> **⚠️ With no `auth` configured, this endpoint is open** — the same default as the full protocol
> (and as `langgraph dev`). That's fine behind your own middleware or on a private network; anywhere
> public, wire an `auth` engine before you ship. See [embedding.md](./embedding.md#bring-your-own-drivers-auth-logger).
## What it deliberately doesn't do
Each call is **independent**: no thread is created, nothing is persisted between calls, and the run
never appears in `/runs`. Concretely, this surface has no:
- **conversation state** — no thread id, no history, no time travel;
- **interrupts / human-in-the-loop** — there is no thread to resume into;
- **background runs, run rows, or webhooks** — the call is inline; there is no run to cancel by id;
- **assistants** — you address the `graph_id` directly, with no assistant config layer.
The **long-term store is still injected**, so nodes reach cross-thread memory via `getStore()` as
usual. If you need any of the above, use the full protocol — that's what it's for.
The call's lifetime really is the request: the graph runs under an `AbortSignal` that fires when the
client disconnects (all four adapters wire this) and when `deps.runTimeoutMs` elapses — the same
budget the run engine applies — so a disconnected or hung call doesn't keep burning model tokens.
## API reference
From [`@skein-js/agent-protocol`](https://github.com/skein-js/skein-js/tree/main/packages/agent-protocol) (the shared handler every adapter wraps):
```ts
// A ProtocolHandler for `POST <prefix>/:graph_id`. Resolves the graph, injects the store, invokes.
function createGraphInvokeHandler(
deps: ProtocolDeps,
options?: { streamMode?: StreamMode | StreamMode[] }, // SSE modes; default "values"
): ProtocolHandler;
// The one-route table, shaped like `skeinRoutes` so catch-all adapters can match it identically.
function graphInvokeRoutes(prefix?: string): RouteBinding[]; // default prefix "/invoke"
// Build a matcher over any route table (used by the NestJS/Next.js catch-all mounts).
function createRouteMatcher(bindings: readonly RouteBinding[]): RouteMatcher;
```
Per adapter — each takes `SkeinRuntimeOptions` (`{ config } | { deps }`, plus `logger`/`cors`) and the
`streamMode` option above:
| Adapter | Entry point | Path option |
| ------- | ----------------------------------------- | -------------------- |
| Express | `skeinInvokeRouter(options)` | `prefix` (`/invoke`) |
| Fastify | `skeinInvokePlugin` | `invokePrefix` |
| NestJS | `SkeinInvokeModule.forRoot(options)` | `prefix` |
| Next.js | `createSkeinInvokeRouteHandlers(options)` | `basePath` |
From [`@skein-js/server-kit`](https://github.com/skein-js/skein-js/tree/main/packages/server-kit):
```ts
// Resolve `{ config } | { deps }` to just the deps — no assistants seeded, no run worker started.
function resolveRuntimeDeps(options: SkeinRuntimeOptions): Promise<{ deps: ProtocolDeps; cors? }>;
```
## See also
- [embedding.md](./embedding.md) — bringing a graph in code (`{ deps }`), the on-ramp this builds on
- [agent-protocol.md](./agent-protocol.md) — the full surface, and when you want it instead
- [streaming.md](./streaming.md) — how skein maps run frames onto SSE
- [`examples/invoke-endpoint`](https://github.com/skein-js/skein-js/tree/main/examples/invoke-endpoint) · [`examples/embed-graph`](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph)
---
<!-- source: docs/agent-protocol.md -->
# Agent Protocol surface
skein-js implements LangChain's [**Agent Protocol**](https://github.com/langchain-ai/agent-protocol),
an OpenAPI-specified, framework-agnostic HTTP + streaming contract for serving LLM agents.
**What this gives you:** a standard REST + SSE API your client already speaks — assistants, threads,
runs (wait / stream / background), streaming, interrupts, and a long-term store. Because it's the same
contract LangGraph Platform serves, your existing [`@langchain/langgraph-sdk`](./react-sdk.md) and
[`useStream`](./react-sdk.md) code works against a skein-js server by changing only the URL. You
almost never call these endpoints by hand — the SDK does — but this page is the map of what's
available and what ships in the MVP. For the streaming wire format, see [streaming.md](./streaming.md);
for building a frontend on top, see [react-sdk.md](./react-sdk.md).
## Core resources
| Resource | Description |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Assistants / agents** | A served graph plus its introspectable input/output/state/config schemas. |
| **Threads** | Multi-turn conversation containers with persistent state and history; track status (`idle`, `busy`, `interrupted`, `error`). |
| **Runs** | Atomic executions of a graph — stateless (ephemeral), streaming, or background. |
| **Store** | Long-term memory organized by namespace + key, with CRUD and (semantic) search. Also injected into graph runs as a LangGraph `BaseStore` — see [storage.md](./storage.md#long-term-memory-in-the-graph-getstore). |
| **Crons** | Schedules that fire a run on a cadence. A LangGraph Platform extension, not part of the open spec — see [crons.md](./crons.md). |
| **Messages** | First-class primitives aligned with OpenAI/Anthropic formats. |
## Endpoint inventory
Every endpoint below is implemented (✅). The route table in
[`packages/agent-protocol/src/http/routes.ts`](https://github.com/skein-js/skein-js/blob/main/packages/agent-protocol/src/http/routes.ts) is the
source of truth — paths mirror the `@langchain/langgraph-sdk` client, so runs are addressed
thread-scoped (`/threads/{thread_id}/runs/{run_id}`).
### Assistants
Full CRUD + version history (LangGraph parity). Assistants are auto-registered one-per-graph at
startup (`assistant_id` defaults to `graph_id`), and can also be created/updated/deleted over the
API. Every `PATCH` mints a new **immutable version**; the live row tracks the currently-active
version and mirrors its fields, and `POST .../latest` rolls back to any past version. (Routes use the
`/assistants/...` spelling the `@langchain/langgraph-sdk` client sends — not `/agents/...`.)
| Method | Path | Notes |
| -------- | --------------------------------------------- | --------------------------------------------- |
| `POST` | `/assistants` | Create; `if_exists: "raise" \| "do_nothing"` |
| `GET` | `/assistants/{assistant_id}` | |
| `PATCH` | `/assistants/{assistant_id}` | Update — mints a new version |
| `DELETE` | `/assistants/{assistant_id}` | `?delete_threads=true` cascades owned threads |
| `POST` | `/assistants/search` | Filter by graph_id/name/metadata; sort + page |
| `POST` | `/assistants/count` | Count matching the search filters |
| `GET` | `/assistants/{assistant_id}/schemas` | Input/output/state/config schemas |
| `GET` | `/assistants/{assistant_id}/graph` | Drawable graph JSON (`?xray`) |
| `GET` | `/assistants/{assistant_id}/subgraphs[/{ns}]` | Subgraph schemas by namespace (`?recurse`) |
| `POST` | `/assistants/{assistant_id}/versions` | Version history, newest-first (filter + page) |
| `POST` | `/assistants/{assistant_id}/latest` | Roll back to an existing version |
### Threads
| Method | Path | Notes |
| -------- | -------------------------------------------- | --------------------------------------------- |
| `POST` | `/threads` | Create; `if_exists`, `supersteps` seed state |
| `GET` | `/threads/{thread_id}` | |
| `POST` | `/threads/search` | |
| `POST` | `/threads/count` | How many match the filters (no pagination) |
| `POST` | `/threads/prune` | Bulk `delete`, or `keep_latest` history trim |
| `GET` | `/threads/{thread_id}/state` | Current state snapshot (`useStream` hydrates) |
| `POST` | `/threads/{thread_id}/state` | Time travel: fork state at a checkpoint |
| `GET` | `/threads/{thread_id}/state/{checkpoint_id}` | Time travel: state at a checkpoint |
| `POST` | `/threads/{thread_id}/state/checkpoint` | Same read, checkpoint given as an object |
| `POST` | `/threads/{thread_id}/history` | Checkpoint history, newest-first (paged) |
| `GET` | `/threads/{thread_id}/history` | Same, with `?limit` (the SDK sends `POST`) |
| `PATCH` | `/threads/{thread_id}` | |
| `POST` | `/threads/{thread_id}/copy` | Duplicates the thread + its history |
| `DELETE` | `/threads/{thread_id}` | |
**`if_exists` on thread creation.** `POST /threads` with a `thread_id` you choose defaults to
`if_exists: "raise"` — a 409 when that id is taken. `"do_nothing"` returns the **existing** thread
untouched, which makes `client.threads.create({ threadId: stableKey, ifExists: "do_nothing" })` a
get-or-create: the idiom for addressing a conversation by an external identity (a phone number, an email
thread, a ticket id) without tracking skein's own ids.
Uniqueness is enforced in the storage driver, not by a read-then-write in the service, so two instances
racing the same id cannot both win.
**Seeding a thread with `supersteps`.** `POST /threads` accepts a `supersteps` array — updates written
straight into the thread's checkpoint history, so you can **import** an existing conversation rather
than replaying it through the graph. Each superstep is one tick and becomes one checkpoint; each update
carries `values` (or a `command`) and a required `as_node` saying which node to attribute it to.
Writing state needs a graph, and a thread this new has no run to infer one from — so pass `graphId`,
which the SDK folds into `metadata.graph_id`. Without it this is a 400. That graph id is also what lets
the seeded state be **read back**: `GET /threads/{id}/state` and the history routes fall back to
`metadata.graph_id` when a thread has never run.
Bounded at 100 supersteps of 100 updates: each update is a graph-state write to the checkpointer.
Note supersteps are applied to whatever the create returned — including a thread that
`if_exists: "do_nothing"` merely _found_, in which case they append to it. That matches
`@langchain/langgraph-api`, which runs its bulk write on the result of the create either way.
**Filtering threads by graph.** `POST /threads/search` matches on a metadata subset. When a run is
created, skein stamps the run's `graph_id` and `assistant_id` into the thread's metadata (matching
LangGraph), so listing the threads for a graph is just:
```jsonc
// POST /threads/search
{ "metadata": { "graph_id": "my_graph" } }
```
The stamp reflects the thread's most recent run; a thread that has never run carries no `graph_id`.
**Paging checkpoint history.** `POST /threads/{id}/history` takes its options in the **body** (as the
LangGraph SDK sends them): `{ limit?, before?, metadata? }`. It returns at most **100** checkpoints when
`limit` is omitted, and rejects a `limit` above 1000 — each element is a checkpoint's whole graph state,
so a long thread's full history is one of the largest single responses skein can produce. Page back
through it with `before` (a checkpoint config carrying `checkpoint_id`, or a bare `checkpoint_id`) — the
bound is exclusive, so pass the last checkpoint you received — and narrow it with `metadata`, which
becomes the checkpointer's filter. Checkpoint IDs, namespaces and checkpoint maps reach the checkpointer; the thread scope remains
server-owned, so a `thread_id` in `before.configurable` is dropped.
A `?limit=` query param is still accepted for hand-rolled callers, but it is _clamped_ to 1000 rather
than rejected (a query string has no schema to 400 from). The body wins if both are present.
`useStream` sends `limit: 10`, so 10 checkpoints come back rather than every one, matching LangGraph
Platform. The rendered transcript is unaffected (the newest checkpoint's
`values` carry the whole message list); what shrinks is how far back the branch/edit tree reaches. Raise
it by passing your own `limit` if you need deeper history.
The 100-checkpoint default is independent of `SKEIN_MAX_PAGE_SIZE` — history is read from the
checkpointer, not from the store, so the store's page bound does not apply to it.
**Time travel (fork from a checkpoint).** `POST /threads/{id}/history` is read-only, but you can also
_branch_ from any past checkpoint:
- `POST /threads/{id}/state` with `{ values, as_node?, checkpoint_id? }` calls `graph.updateState` to
write a **new checkpoint** that forks history at `checkpoint_id` (or the tip). It returns the new
checkpoint pointer, `{ "checkpoint": { "thread_id", "checkpoint_ns", "checkpoint_id" } }`, and mirrors
the forked values onto the thread row. Rejected with `409` while a run is in flight on the thread.
- `GET /threads/{id}/state/{checkpoint_id}` reads the state snapshot at a specific checkpoint, and
`POST /threads/{id}/state/checkpoint` reads the same thing with the pointer in the body. Both exist
because the SDK picks between them by argument _type_: `threads.getState(id, "ckpt-1")` takes the
`GET`, while `threads.getState(id, { checkpoint_id, checkpoint_ns })` takes the `POST`. The body pointer preserves `checkpoint_id`, `checkpoint_ns` and `checkpoint_map`. A namespace-only
pointer reads that namespace’s current state. `subgraphs: true` includes recursive task state;
the GET routes accept `?subgraphs=true`.
- Run creation accepts a top-level **`checkpoint_id`** to start a run from a chosen checkpoint instead
of the thread tip. A full `checkpoint` pointer or validated `config.configurable` checkpoint coordinates also work.
An explicit top-level ID wins. The server always supplies the owning thread ID. Coordinates
survive queued execution, so the worker branches from the same checkpoint the caller selected.
### Runs — stateless / ephemeral
Each of these creates its own thread. `on_completion` decides what happens to it (below).
| Method | Path | Notes |
| ------ | -------------- | ------------------------------------------------ |
| `POST` | `/runs/wait` | Run to completion, answer with the final values |
| `POST` | `/runs/stream` | Run and stream frames as SSE |
| `POST` | `/runs` | Queue a background run, answer with the `Run` |
| `POST` | `/runs/batch` | An **array** of run-creates; max 100 per request |
| `POST` | `/runs/cancel` | `cancelMany` — see below |
### Runs — background (thread-scoped)
| Method | Path | Notes |
| -------- | -------------------------------------------------- | --------------------------------- |
| `POST` | `/threads/{thread_id}/runs` | Start a background run |
| `GET` | `/threads/{thread_id}/runs` | List a thread's runs (paginated) |
| `GET` | `/threads/{thread_id}/runs/{run_id}` | Fetch one run |
| `GET` | `/threads/{thread_id}/runs/{run_id}/stream` (join) | Join a run's stream |
| `GET` | `/threads/{thread_id}/runs/{run_id}/join` | Block until it settles, then JSON |
| `POST` | `/threads/{thread_id}/runs/{run_id}/cancel` | Cancel a run (`?action`, `?wait`) |
| `DELETE` | `/threads/{thread_id}/runs/{run_id}` | Delete a run |
| `GET` | `/runs/{run_id}/stream` (join) | Join by run id (thread-agnostic) |
**Run-completion deliveries.** A run created with a `webhook` records its callback in an outbox, so
the attempts are inspectable and a failed one can be re-sent:
| Method | Path | Notes |
| ------ | -------------------------------------------------------------------- | -------------------------------------- |
| `GET` | `/threads/{thread_id}/runs/{run_id}/deliveries` | `?status=` · `?limit=` · `?offset=` |
| `POST` | `/threads/{thread_id}/runs/{run_id}/deliveries/{delivery_id}/replay` | Makes a delivery due again immediately |
The list strips the stored `payload` — up to 256 KiB of the run's final state per row — and reports a
boolean `replayable` in its place; read the state from the run. Replay `409`s on a delivery that
already succeeded (its payload was cleared on success, so there is nothing left to send) and `404`s on
a delivery id that does not belong to the run in the path.
Both sit in the **`runs` route group** deliberately: a run is already an ownership-scoped resource, so
`http.disable_runs` and an existing `@auth.on.threads` handler cover them with no new switch and no new
`RouteGroup` — which, once shipped, could never be withdrawn. Delivery semantics, signing and
retention are in [webhooks.md](./webhooks.md).
**Configured-channel inventory.** When a deployment configures `skein.channels`, it also mounts
`GET /channels` for operator tooling. The response is intentionally small and sanitized:
```json
{
"channels": [
{
"route_name": "support-whatsapp",
"assistant": "support",
"allowed_assistants": ["refunds"],
"channel_name": "twilio",
"delivery_supported": true
}
]
}
```
The inbound path is derived as `/channels/{route_name}`. The endpoint never returns module paths,
`public_url`, raw configuration, credentials or reply destinations. It is absent when no channel is
configured and, under custom auth, requires `assistants:read`. This is a skein operator extension,
not part of the Agent Protocol or the upstream SDK resource clients; the console reuses the SDK's base
transport for it.
**Joining a run: two shapes.** `.../runs/{run_id}/stream` tails a run as SSE (`client.runs.joinStream()`,
resumable with `Last-Event-ID`). `.../runs/{run_id}/join` is the blocking form (`client.runs.join()`): it
waits for the run to settle and answers the thread's final `values` as plain JSON, or
`{ "__error__": ... }` for a failed one — the same envelope `POST /runs/wait` uses. Joining a run that has
_already_ settled returns immediately, including long after its frames have aged out of the event bus,
because the wait is decided by the run row rather than by the bus. `?cancel_on_disconnect` is honoured on
the **streaming** form and accepted-but-ignored on the blocking one, matching
`@langchain/langgraph-api` — see `on_disconnect` under the run endpoints below.
**Cancelling in bulk.** `POST /runs/cancel` takes `{ thread_id?, run_ids?, status? }`, narrowest
selector first: explicit `run_ids`, else one thread's inflight runs, else **every** inflight run on the
server. `status` is `pending` / `running` / `all` (the default). An unknown — or non-owned — run id is
skipped rather than failing the sweep, so the response reports what actually happened:
`{ cancelled_count, cancelled_run_ids, truncated }`. `truncated: true` means the whole-server sweep filled
the store's page bound and should be repeated; the SDK types this call as returning `void` and ignores the
body, so it is skein's to shape.
`truncated` is deliberately a boolean about the caller's own page rather than a count of what is left.
The per-thread concurrency guard the sweep reads through is **not** ownership-scoped — it has to see every
inflight run on a thread whoever started it — so a total would tell an authenticated caller how much work
every other principal has in flight.
**`?action` and `?wait` on a cancel.** Both are sent by every `client.runs.cancel(...)`.
`action=interrupt` (the default) settles the run `cancelled` and keeps whatever it
wrote; `action=rollback` additionally discards its checkpoint writes and deletes the run row, so the turn
reads as never having happened. `wait=1` returns only once the run has actually stopped executing rather
than as soon as it is marked.
**`on_completion` on a stateless run.** `"delete"` removes the server-created thread once the run
settles; `"keep"` leaves it. **skein defaults to `keep`, LangGraph to `delete`** — a deliberate
divergence, so a stateless run stays inspectable and so adding the field did not silently change what
`/runs/wait` and `/runs/stream` already did. Pass `"delete"` for LangGraph's behaviour. An `interrupted`
run keeps its thread either way: it has yielded to a human, and its checkpoint is the whole value of the
turn.
**`if_not_exists` on run creation.** Naming a thread that does not exist is a **404** by default
(`if_not_exists: "reject"`, matching LangGraph). Pass `"create"` to have the run bring the thread into
existence instead — the other half of addressing a conversation by an external key, so an inbound event
can start a run without a round trip to create the thread first.
All three thread-scoped run routes agree, and the default is the safe one — a mistyped thread id fails loudly rather than running against a fresh empty thread with none
of the history the caller expected. Pass `if_not_exists: "create"` to restore the old behaviour.
**`if_thread_status` on run creation.** A skein extension, and the counterpart to `multitask_strategy`:
that one arbitrates `pending`/`running`, this one guards every other thread status. It matters most for
`interrupted`, which is a **terminal** run status — a thread waiting on a human holds no inflight run,
so no multitask strategy protects it and a plain start discards the pending interrupt. Send
`if_thread_status: ["idle", "error"]` and an interrupted thread answers **409 `thread_status_mismatch`**
with the observed status in `details`. Settled inside the driver's atomic create, so it holds across
replicas. A storage driver that does not implement the precondition answers
`501 if_thread_status_unsupported` rather than degrading to a non-atomic check.
A thread created this way still belongs to the **caller**, not to the run: `on_completion: "delete"`
never removes it, because the caller named it. Only a thread the server minted (no `thread_id` at all)
is the run's to delete.
The field is inert on `POST /runs` and `/runs/batch`, which strip `thread_id` outright — the server owns
a stateless run's thread, so nothing can be missing.
**`after_seconds` on run creation.** Holds the run for that many seconds before it starts — the SDK's
"schedule a future run". Capped at **86400** (a day); anything longer is a [cron](./crons.md).
A **background** run (`POST /runs`, `/runs/batch`, `POST /threads/{id}/runs`) is held by the queue
itself. Redis keeps delayed jobs in BullMQ's delayed set; SQLite persists unclaimed jobs with their
original due times. Both retain waiting jobs across a process restart. On the in-memory queue the
delay is a timer, lost on restart along with the other queued jobs. Abandoned active attempts follow
each [queue driver's crash behavior](./runs-and-redis.md#queue-drivers).
An **inline** run (`/runs/wait`, `/runs/stream`) has no queue to hold it, so the server waits with your
connection open. A stream still responds immediately — the run row, its `Content-Location`, and the
SSE heartbeats all come first, and only execution is deferred — but `/runs/wait` sends nothing until
the run finishes, so a long `after_seconds` there will hit a proxy's idle timeout. Prefer a background
run for anything more than a few seconds.
Two consequences worth knowing. The run row exists for the whole delay with status `pending`, so it
counts as **inflight**: a second run on that thread under the default `multitask_strategy: "reject"` is
refused until the delayed one starts. That is the honest reading — work _is_ scheduled on the thread —
but it surprises people. And a delayed run is cancellable like any other; `POST .../cancel` settles it
before it ever runs.
**`on_disconnect` on run creation.** `"cancel"` settles the run when the caller's connection drops;
`"continue"` lets it finish. Only the inline routes (`/runs/wait`, `/runs/stream`) hold a connection to
drop, which is why the SDK does not send it on a background create.
**skein defaults to `"continue"`.** A proxy idle timeout or a load-balancer reset is indistinguishable
from a real hang-up at this layer, so defaulting to cancel would let routine infrastructure kill a
healthy run. Note `useStream` sends `"cancel"` on every submit unless the stream is resumable — so with
a browser client, closing the tab stops the run. That is LangGraph's behaviour.
The related query flag `?cancel_on_disconnect` is honoured on `GET .../runs/{run_id}/stream`
(`client.runs.joinStream()`), matching `@langchain/langgraph-api`. It stays **accepted and ignored** on
the blocking JSON `.../join`, because the reference server does not read it there either — a blocking
join should not behave differently against the two servers the same client talks to.
Adapters supply the disconnect signal from their own transport (`res` close, or the Web `Request`'s
`signal`). An adapter that cannot observe disconnects simply omits it, and `"cancel"` degrades to
`"continue"` rather than failing.
**`Content-Location` on run creation.** Every run-create response (and `POST /threads/{id}/stream`)
carries `Content-Location: /threads/{thread_id}/runs/{run_id}`. The `@langchain/langgraph-sdk` client
parses it to fire `onRunCreated`, which is what `useStream` stores to rejoin a stream after a remount —
so without it that callback never fires and `reconnectOnMount` cannot work. It is also the only way a
caller learns the thread id of a stateless `/runs/wait`, whose body is the graph's state.
### Crons (LangGraph Platform extension)
| Method | Path | Notes |
| -------- | --------------------------------- | ------------------------------------------------ |
| `POST` | `/runs/crons` | Stateless cron — a fresh thread per fire |
| `POST` | `/threads/{thread_id}/runs/crons` | Thread cron — reuses the named thread |
| `POST` | `/runs/crons/search` | Filter + sort + page; `x-pagination-total` |
| `POST` | `/runs/crons/count` | Returns a **bare integer** |
| `GET` | `/runs/crons/{cron_id}` | |
| `PATCH` | `/runs/crons/{cron_id}` | Tri-state `end_time`/`timezone`; metadata merges |
| `DELETE` | `/runs/crons/{cron_id}` | **200 with a JSON body**, not 204 |
These are **not** in the open Agent Protocol spec — its `openapi.json` has no cron paths, and the OSS
`@langchain/langgraph-api` throws `500 Not implemented` on all of them. skein serves them against the
LangSmith Deployment OpenAPI spec plus the SDK's types. Two response shapes are deliberately unusual
because the official client requires them: `count` answers a bare integer, and `DELETE` answers 200
with a body (the SDK skips `response.json()` only for 202 and 204). Full semantics — schedule format,
catch-up, driver support, the scheduler — are in [crons.md](./crons.md).
### Meta
| Method | Path | Notes |
| ------ | ------- | -------------------------------------------------------------------- |
| `GET` | `/info` | Version + `flags` capability handshake (Studio reads it) |
| `GET` | `/ok` | Liveness probe — served by each adapter, **outside** the route table |
`/ok` sits outside the protocol table on purpose. LangGraph groups it with `/info` under one
`disable_meta` flag; in skein it is the container health check (see the generated Dockerfile), so no
config flag may be able to make a healthy instance read as dead.
**`/info` is served unauthenticated**, even with an `auth` block configured — matching
`@langchain/langgraph-api`, whose auth middleware skips it explicitly. It is a handshake: Studio and
monitoring clients probe it before they have credentials, so requiring auth would break connecting to a
server `langgraph dev` would have answered. It exposes only versions and which resources are served.
### Store (long-term memory)
| Method | Path | Notes |
| -------- | --------------------- | ---------------------------------------------- |
| `PUT` | `/store/items` | Upsert an item (optional `ttl`) |
| `GET` | `/store/items` | Fetch by `?namespace=a.b&key=…` |
| `DELETE` | `/store/items` | `{ namespace, key }` in the body, or the query |
| `POST` | `/store/items/search` | pgvector semantic search, `filter` → `{items}` |
| `POST` | `/store/namespaces` | Prefix/suffix/depth → `{namespaces}` |
**`DELETE` takes a body.** The SDK's two single-item methods use different transports on the same path:
`store.getItem` sends `?namespace=a.b&key=…`, while `store.deleteItem` sends a JSON body with
`namespace` already an array. skein accepts **either** on both, body first. Reading only the query made
every SDK `deleteItem` a silent no-op — empty namespace, empty key, nothing deleted, `204`.
**Response shapes.** Search returns `{ "items": [...] }` and namespaces `{ "namespaces": [...] }` —
the envelopes the SDK's `store.searchItems` / `store.listNamespaces` read. Store items carry
`created_at`/`updated_at`, like every other resource here.
**`filter` on search** narrows by the **top-level** keys of an item's `value` — `"a.b"` is the literal
key `"a.b"`, not a path into a nested object. Keys are ANDed, and so are multiple operators on one key.
The operator set is LangGraph's: `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`; a bare
scalar means equality. Filtering happens before paging, so a page is a page of matches. An unknown
operator or a non-scalar filter value is a **400**, not a silent narrowing — see
[storage.md](./storage.md) for the full semantics, including the one place skein's ordering operators
deliberately differ from LangGraph's.
**Namespace matching** takes `prefix` and `suffix` (ANDed) and `max_depth`. `"*"` matches exactly one
segment, positionally — so `["users","*"]` selects the `users` subtree, matching `["users","1"]` and
`["users","1","memories"]` alike. Before 0.14 a wildcard was dropped and read as _no prefix_, so that
same request returned **every namespace in the store**. `max_depth` truncates each match and
de-duplicates, before paging.
**Pagination on these two.** `GET /threads/{thread_id}/runs` takes `?limit`/`?offset` and
`POST /store/namespaces` takes `limit`/`offset` in the body; both default to a **100**-row page, the
same default the SDK sends for `store.listNamespaces`. A query `limit` above 1000 is clamped rather than
rejected, matching every other query-string limit here. Truncation is not signalled on the response
(only assistant search carries `x-pagination-total`), so page until you receive fewer rows than you
asked for. `status` on `runs.list` **is** honoured, filtered in the driver so it pages the filtered set;
`select` on `runs.list` remains accepted without projection. Store reads and searches honor
`refresh_ttl` as a per-request override of `store.ttl.refresh_on_read`.
### Thread streaming (SSE)
| Method | Path | MVP |
| ------ | ------------------------------------ | --- |
| `POST` | `/threads/{thread_id}/stream` | ✅ |
| `GET` | `/threads/{thread_id}/stream` | ✅ |
| `GET` | `/threads/{thread_id}/stream/events` | ✅ |
| `POST` | `/threads/{thread_id}/stream/events` | ✅ |
| `POST` | `/threads/{thread_id}/commands` | ✅ |
`POST /threads/{thread_id}/stream/events` subscribes to the thread event protocol. Its body
contains `channels`, optional namespace prefixes, `depth`, and an optional numeric `since` cursor.
It never creates a thread or run. Without a cursor it sends a current state baseline followed by live
events; with a cursor it replays retained events after that sequence and then follows live events.
An expired or future cursor reports `thread_cursor_expired`; restore state and subscribe afresh.
`POST /commands` recognizes `{ id, method, params }` envelopes and answers correlated JSON
success/error responses. Implemented methods are `run.start`, `input.respond`, `agent.getTree`,
`state.get`, `state.listCheckpoints`, and `state.fork`. Run commands enqueue through the existing
worker; a subscription or reconnect never invokes the graph. Existing resume/goto/update bodies
continue to return legacy SSE. State reads require read permission, run starts require `create_run`,
and forks also require state-write and destination-creation permission.
`state.fork` verifies the source checkpoint in the configured saver before copying; custom-agent
snapshots do not need a `createdAt` timestamp to identify a valid stored checkpoint.
`GET /stream/events` remains an alias for `GET /stream`. Without parameters these join one run
and finish with it, preserving Skein's established default. Explicit `stream_mode` (repeated or
comma-separated; what the SDK's `threads.joinStream` sends) opts into a thread stream that stays
open across turns. `run_modes` includes every run frame and completion metadata. `lifecycle`
emits only `metadata` events: `{ run_id, attempt: 1 }` when execution starts and
`{ status: "run_done", run_id }` when it ends. `state_update` emits
`{ thread_id, state }`, where `state` is the full checkpoint snapshot captured when the run ends
or state is updated directly. Replay preserves each historical snapshot. Individual frame modes
and the Skein-specific `thread_state` thread-row events remain supported. SSE event IDs are thread-wide cursors, rather than
per-run frame IDs. Without `Last-Event-ID` it starts with live events; with one it replays after it.
Thread subscriptions require `ProtocolDeps.threadBus`; the built-in runtime assemblers provide it.
Memory replay is process-local. Redis replay is shared between API replicas and workers. Neither
publisher stores subscriber credentials or subscription filters: authorization and filtering happen
for each reader during replay and live delivery. See the [tested compatibility matrix](./langgraph-compatibility.md).
> The protocol also describes a WebSocket upgrade for bidirectional streaming. That is
> **post-MVP** — SSE covers the full client UX (see [streaming.md](./streaming.md)).
## Request/response conventions
- JSON for all non-streaming payloads.
- Request bodies carry `input`, optional `metadata`, optional `config`.
- Responses carry status (`pending` / `success` / `error`), timestamps, and resource IDs.
- A failed run also carries `error` — a skein extension over the SDK's `Run`, which records only
_that_ a run failed. See [errors-and-logging.md](./errors-and-logging.md).
- Schemas use JSON Schema for interoperability.
## Idempotent run creation (`Idempotency-Key`)
A skein extension — LangGraph Platform has no equivalent. Send an `Idempotency-Key` header on a run
create and a retry of that request returns the **original response** instead of starting a second
run.
This exists because every webhook provider retries: Twilio on timeout or 5xx, Stripe, GitHub, Slack,
SendGrid. Without it, a retried delivery means a second reply to the end user or two agents acting on
one event — and the failure is silent. The only other defence is a dedup table re-implemented inside
every caller's application.
**Supported on** `POST /threads/{thread_id}/runs`, `POST /threads/{thread_id}/runs/wait`,
`POST /runs`, `POST /runs/wait`, and `POST /runs/batch`. Omitting the header is exactly today's
behaviour, so this is purely additive.
**Not supported on the streaming creates** (`POST /runs/stream`,
`POST /threads/{thread_id}/runs/stream`, `POST /threads/{thread_id}/stream`,
`POST /threads/{thread_id}/commands`), which answer with a live SSE stream: there is no body to
record, and consuming the stream to make one would break it for the caller who asked. Sending the
header there is a **422** rather than being ignored — a silently-dropped key would leave you
believing your retries are deduplicated while every one starts another run.
| Case | Response |
| ------------------------------------------------ | -------------------------------------------------------------- |
| Key unseen | Executes normally; the response is recorded |
| Key seen, same request, original done | The recorded response verbatim, plus `Idempotent-Replay: true` |
| Key seen, same request, original still in flight | `409` `idempotency_key_in_flight` — retry shortly |
| Key seen, **different** request body | `422` `idempotency_key_reused` — a caller bug; do not retry |
| Key over 255 chars, or not printable ASCII | `422` `invalid_idempotency_key` |
The `code` carries the meaning on the 422s: these routes already answer 422 for a _transient_ reason
(`thread_busy`, under the `reject` multitask strategy), and the two demand opposite client behaviour.
Keys follow Stripe's 255-character limit.
`Content-Location` is replayed with the rest of the response, so the SDK's `onRunCreated` still fires
and `useStream`'s `reconnectOnMount` still works on a retry.
A few properties worth knowing:
- **The claim is atomic across instances.** The record is inserted first and the store's uniqueness
constraint arbitrates, so two provider retries landing on two pods milliseconds apart still produce
exactly one run. Held to that by the shared `SkeinStore` conformance suite, on every driver.
- **Keys are scoped per principal** when auth is configured, so one caller cannot replay another's
response by guessing their key.
- **A replay is re-authorized.** It answers from the record without reaching the handler table, so
the route's `@auth.on.*` check runs again first — and when it returns ownership filters, the
recorded thread is re-read through the scoped store. Revoking a caller's access stops their replays
at once rather than at the end of the retention window.
- **Failures are never recorded.** A create that throws or answers non-2xx releases its key, so the
next retry really runs — pinning a transient 503 for the retention would make a momentary outage
permanent for that key.
- **At-least-once, not exactly-once.** The guarantee is that a retry of the _same_ request does not
create a second run. Choose keys that are stable across retries (the upstream provider's message id
is usually right).
- **Deleting a thread or a run erases its recorded responses.** A recorded response has to outlive
its run for a replay to mean anything, and for `POST /runs/wait` that response _is_ the graph's
final state — so `DELETE /threads/{thread_id}` takes the matching records with it, as do
`DELETE /threads/{thread_id}/runs/{run_id}`, an assistant delete, and thread-TTL expiry. Erasure
means erasure; a retry afterwards executes again rather than replaying, and on a deleted thread
that is a 404.
The one deletion that does **not** erase is a stateless run tidying up its own server-created thread
(`on_completion: "delete"`). That thread is an implementation detail of `POST /runs`, disposed of on
every such run — scrubbing there would destroy the record moments after writing it and break
retries for exactly the case the header exists for. Those records expire on `retention_hours` like
any other. Lower it if you want that window shorter.
`POST /runs/batch` records carry no thread: their runs may span several, and the body is a list of
run rows rather than conversation content.
Retention is tunable under `skein.idempotency` in `langgraph.json` — see
[langgraph-cli-compat.md](./langgraph-cli-compat.md#idempotency-skeinidempotency).
## Authentication + authorization
Auth follows LangGraph's [custom-auth model](https://docs.langchain.com/langsmith/custom-auth) and is
**transport-neutral** — it wraps the handler table, so every adapter inherits it identically. Active
only when an `Auth` engine is configured (a `langgraph.json` `auth` block or an injected `auth` dep);
otherwise the server is unauthenticated.
Per request:
1. **Authenticate** — run the user's `authenticate` handler → an `AuthContext`, or `401`. The CLI
requires credentials for Studio, regardless of its self-declared `x-auth-scheme: langsmith` header.
In loopback `skein dev`, `--dev-ui-access` or `auth.dev_ui_access: true` creates a temporary API
key for both Studio and the console. A directly injected `AuthEngine` may choose its own Studio
policy.
2. **Authorize** — run the matching `@auth.on.*` handler (priority `resource:action` → `resource` →
`*:action` → `*`) → `403` on `false`, else an ownership **filter**.
3. **Dispatch** — with the authenticated `user`. Ownership scoping applies to the `threads` family
(threads + their runs): a non-owned row reads as absent (`404`, never `403`), and the filter's values
are stamped onto rows it creates. `crons` scope the same way and **fall back to the `threads`
handler when no `@auth.on.crons` callback is registered**, since callbacks match by exact event key
and a deployment that scoped only threads would otherwise serve crons unscoped. `assistants` is
**gate-only** — a handler can deny, but no filter applies, because auto-registered graph assistants
have no owner and must stay runnable. Store scoping works through the namespace instead:
### Scoping the store
A store item carries no metadata, so an ownership filter has nothing to match on. LangGraph's idiom is
to **rewrite** `value.namespace`, and skein honours it on all five store routes:
```ts
auth.on("store", ({ user, value }) => {
value.namespace = [user.identity, ...(value.namespace ?? []).slice(1)];
});
```
In-place mutation counts (`value.namespace[0] = …`), and the body namespace wins over query params on
`GET`/`DELETE`. `value.key` is honoured the same way on `put`/`get`/`delete`. A rewrite that is not a
`string[]` is a **500** (`store_namespace_rewrite_invalid`), not a silent fall-through — a handler that
meant to scope and got the shape wrong is exactly the failure this prevents. Assigning `value.key` on
`search` or `list_namespaces` is likewise a 500 (`store_key_rewrite_invalid`): there is nowhere for it
to land. If you supply your own `AuthEngine`, its `authorize` must return the **same** `value` object
for a rewrite to be observable.
> **With no store handler registered, nothing narrows a store read.** The namespace is a request
> parameter, so any authenticated caller can send `{"namespace_prefix": ["memories"]}` — or omit the
> prefix entirely — and read **every** tenant's items. `POST /store/namespaces` likewise returns every
> tenant's namespace names.
Three traps if you validate rather than rewrite:
```ts
// A namespace label is one segment: it must contain neither `.` (the query separator) nor `*` (the
// positional wildcard). encodeURIComponent escapes neither, so do it explicitly.
const tenantLabel = (identity: string) =>
encodeURIComponent(identity).replace(/\./g, "%2E").replace(/\*/g, "%2A");
auth.on("store", ({ user, value }) => {
const namespace = value.namespace; // server-derived; absent means "every namespace" on search
if (!namespace?.length || namespace[0] !== tenantLabel(user.identity)) {
throw new HTTPException(403, { message: "Out of scope." });
}
});
```
- **Require a namespace; don't just reject suspicious ones.** A deny handler can refuse a request but
never filter the result. An absent prefix reads every tenant, and a `"*"` wildcard matches a strict
**subset** of what the shorter literal prefix already returns — so blocking wildcards buys nothing.
- **Encode the identity.** `GET /store/items` takes a dot-joined query string, so `alice@corp.com`
writes to `["alice@corp.com", …]` but reads back as `["alice@corp", "com", …]`. Emails and OIDC
subjects hit this constantly.
- **A handler cannot cover `getStore()` inside a graph** — that is not an HTTP request and has no
principal. Build the namespace from `config.configurable.langgraph_auth_user_id`, never from model
output.
`value.namespace` is **server-derived** — set after merging the body, from the field the endpoint will
really use. Store bodies are `passthrough`, so otherwise a caller could send a decoy `namespace` that
satisfied the handler while the endpoint searched from an absent `namespace_prefix`.
### Where scoping runs
**In the database.** An ownership-filtered thread search becomes a metadata containment clause
(`metadata @> …` in Postgres, hitting `threads_metadata_idx`), so `limit`/`offset` page owned rows
directly rather than reading everything and filtering in JS.
The in-process `matchesFilters` check still runs and is what actually enforces ownership. The
translation errs **broad**, omitting any clause it cannot express exactly, because a too-strict clause
would silently hide rows a caller owns. A **custom** `AuthEngine` whose `matchesFilters` is stricter
than its own filters is the exception — pages then come back short; keep the two consistent.
**Principal in the run config.** Nodes read `config.configurable.langgraph_auth_user`,
`langgraph_auth_user_id` and `langgraph_auth_permissions`. Server-owned and reserved, so a client
cannot spoof them, and persisted on the run so a background run resumed elsewhere injects the same
principal. With no `auth` configured, no keys are added.
Route → resource/action (runs authorize through their owning thread — there is no `runs` resource):
| Endpoint(s) | resource\:action |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- |
| `GET /assistants/{id}`, `/assistants/{id}/schemas` | `assistants:read` |
| `POST /assistants/search` | `assistants:search` |
| `POST /threads` | `threads:create` |
| `GET /threads/{id}`, `/state`, `/state/{checkpoint_id}`; `POST /state/checkpoint`, `/history`; `GET /history`, `.../runs`, `.../runs/{run_id}`, run join (stream or blocking) | `threads:read` |
| `POST /threads/search`, `POST /threads/count` | `threads:search` |
| `PATCH /threads/{id}`; `POST /threads/{id}/state` (state fork); run cancel; `POST /runs/cancel` | `threads:update` |
| `DELETE /threads/{id}`; run delete; `POST /threads/prune` | `threads:delete` |
| run create (wait/stream/background/stateless/batch), thread stream / commands | `threads:create_run` |
| `GET /info` | _(unauthenticated — see the Meta section)_ |
| `PUT/GET/DELETE /store/items`, `/store/items/search`, `/store/namespaces` | `store:{put,get,delete,search,list_namespaces}` |
`POST /runs/cancel` sweeps broadly but authorizes narrowly — every run goes back through the
ownership-filtered `get`, so a non-owned run reads as absent and is skipped. The per-thread concurrency
guard is deliberately **not** ownership-filtered: it must see every in-flight run on a thread whoever
started it, or two could execute at once and interleave checkpoint writes. Nothing leaks, since the
thread itself is gated.
## References
- Agent Protocol repo + OpenAPI — <https://github.com/langchain-ai/agent-protocol>
- aegra's Agent Protocol implementation (Python prior art) — <https://github.com/aegra/aegra>
### Execution, search and retention options
Run creation accepts optional `timeout_ms`: integer milliseconds from **1** through **2,147,483,647**.
It is persisted with the run, including queued, native start/resume, and BullMQ command inputs.
The duration begins when execution starts, so queue delay does not consume it. The effective budget
is the minimum of this duration, the deployment's run limit, and any remaining native deadline.
Timeouts follow the same cancellation and terminal `timeout` settlement path as deployment limits.
`GET /info` optionally reports `flags.per_run_timeout: true` when supported. No new configuration
key or endpoint is needed; omission preserves existing behavior.
Thread responses optionally include `ttl`, `expires_at`, and `ttl_strategy`; store item/search
responses optionally include `ttl` and `expires_at`. Durations are minutes. Missing means unavailable;
`null` means known no expiry. Refresh-enabled store reads return the updated expiry. These are
optional projections, preserving compatibility with custom drivers.
Run requests preserve `durability: "sync" | "async" | "exit"` through the queue. It takes precedence
over deprecated `checkpoint_during`; explicit `true` maps to `async`, explicit `false` to `exit`, and
omission keeps LangGraph’s default. Legacy SSE also recognizes `tasks` and `checkpoints` modes.
Thread search supports `exclude_metadata` (one nonempty object or a nonempty list, excluding any
matching subset), `sort_by: "state_updated_at"`, `select`, and up to ten named `extract` paths.
Extraction accepts roots `values`, `metadata`, `config`, and `interrupts`, dot properties and array
indices such as `values.messages[-1].content`; absent values become `null`. Extracted aliases are
returned in an `extracted` object, alongside selected fields, and cannot overwrite thread identity.
Filters run before
pagination; projection and extraction run after authorization. Assistant and cron search support
`select`. HTTP assistant name matching is case-insensitive substring matching; direct repository
calls retain exact matching unless `nameMatch: "substring"` is requested.
Store writes accept `index: false | string[]` and `ttl: null` for an explicit no-expiry item.
`GET /store/items?refresh_ttl=false` and store search `refresh_ttl` control sliding expiry per read.
Custom stores that cannot honor requested indexing or TTL controls fail explicitly.
Thread TTL objects accept `strategy: "keep_latest"` with their TTL minutes. The sweeper retains the
thread and current checkpoint in every retained namespace, including referenced child checkpoints
and pending writes. State updates refresh that window; active runs are skipped. `delete` remains
the default strategy and explicit `ttl: null` pins a thread. See [storage](./storage.md).
### MCP and A2A
`/mcp` uses the official MCP SDK’s stateless Streamable HTTP transport. Tools list authorized
assistants and their input schemas. Calling a tool creates a run through the same authorized run
service; cancellation uses the existing run cancellation machinery.
`POST /a2a/{assistant_id}` uses the official A2A SDK’s JSON-RPC handlers and legacy codecs.
Discovery is `GET /.well-known/agent-card.json?assistant_id=...`; `DEFAULT_A2A_ASSISTANT_ID` supplies
an omitted assistant and `LANGGRAPH_API_URL` supplies the public origin. Supported method families
are `SendMessage`, `SendStreamingMessage`, `GetTask`, `CancelTask`, `ListTasks`,
`GetExtendedAgentCard` and the four legacy send/stream/get/cancel names.
An A2A context maps to a thread. Each new task starts a queued run; a continuation keeps the first
run’s task ID. Final output is captured before terminal status on existing run rows, independently
of checkpoint pruning and event replay. Optional `RunRepo.recordResult/getResult` support is
required before A2A admits work; ordinary Agent Protocol runs remain usable with older drivers.
Resume checks the saved checkpoint under the existing execution lease and rejects stale tasks.
Cancellation status takes precedence over a computed output.
History defaults to the context. The advertised history-scope extension accepts `task` in
`configuration.historyScope` for sends, in `params.historyScope` for reads/listing, or through
`LangGraph-A2A-History-Scope`. Length is bounded to ten, after scope. Streaming omits response
history. Assistant `metadata.a2a.input_modes`/`output_modes` customize nonempty advertised MIME
lists. File parts preserve bytes or URLs, MIME and filenames through standard content blocks.
Skein advertises HTTP/SSE, history scope and streaming. It does not advertise A2UI, task push
notifications or subscription-to-task support. These unsupported methods fail explicitly. Dedicated
frontend browser and packed-package gates validate clients separately from these transport tests.
`http.disable_mcp` and `http.disable_a2a` remove their route groups.
### Discovery and storage diagnostics
`GET /openapi.json` and `/docs` are generated on each request from the mounted routes and the
same internal operation contracts used by request validators and typed response constructors.
They describe request fields, response schemas, status codes, content types, and deployment-supplied
authentication. There is no separate OpenAPI JSON file to update. Graph-defined payloads remain open
and are described by the assistant schema endpoints.
Set `http.disable_docs: true` to hide `/docs` and `/openapi.json` independently.
`http.disable_meta: true` retains its broader meaning: hide docs, `/info`, and `/metrics`.
Only literal `true` disables a group; `/ok` remains available. Settings take effect on server startup. `/metrics` exposes
host-owned OpenTelemetry collection (501 when unavailable). Authorized
`GET /threads/{thread_id}/storage` returns concrete saver payload counts and bytes, or 501 for a
custom saver without an inspector. These routes use the common adapter/runtime paths; see
[diagnostics](./observability.md#http-diagnostics) for measurement boundaries.
---
<!-- source: docs/building-blocks.md -->
# Building blocks
Every agent starts as a model, a prompt and a loop. That version works for about a day. What follows
is what reality adds — each piece arriving because something broke without it, and each one either
yours to write, LangGraph's to define, or skein's to run.
```mermaid
flowchart LR
Y["<b>Yours</b><br/>model · prompt · tools"]
L["<b>LangGraph</b><br/>state · nodes · edges · interrupt"]
S["<b>skein</b><br/>runs · threads · memory<br/>schedules · callbacks · streaming"]
Y --> L --> S
class Y accent
```
Read it through once; after that it is a map — find the pressure you are under and follow the link.
## The model and the prompt — yours
skein never sees your model choice. You construct it in your graph, from any LangChain provider, and
your key lives in `.env`.
One habit worth forming early: keep raw data in state and format the prompt inside the node. State is
checkpointed every step, so a rendered prompt in a channel is a copy you pay for on every turn.
→ [Your first agent](./your-first-agent.md#5-give-it-a-real-model)
## Tools — yours
A function plus the metadata the model reads to decide when to call it. That metadata is the model's
_only_ context for the decision, so write it for someone who knows nothing else.
**The trap:** a failing tool does not fail the run. The error goes back to the model, which will
often try again. Bound that in the graph, not in hope.
→ [LangGraph essentials](./langgraph-essentials.md#tools)
## Control flow — LangGraph's
Nodes and edges: what runs, and what runs next. Cycles are expected — an agent loop _is_ a node that
routes back to the model. skein serves whatever you compile and wraps none of it, which is why the
same graph runs on LangGraph Platform.
→ [LangGraph essentials](./langgraph-essentials.md)
## A run is one turn
One input, one execution of your graph. What differs is how you wait: hold the connection, stream, or
take an id and let it work.
Two surprises. A run is not tied to the connection that started it — drop the stream and it keeps
going. And a second message mid-run does not queue: the default strategy is `reject`, which fails the
_second_ run with a 422 and leaves the first alone.
→ [Runs](./runs.md)
## Checkpoints, and everything they buy
This is where homegrown agent servers start hurting, because "the conversation is still there" is not
something a request handler can give you.
LangGraph saves state at every node boundary; skein persists it. One mechanism, four consequences:
```mermaid
flowchart LR
CP[checkpoint at each<br/>node boundary]
CP --> A[multi-turn<br/>conversations]
CP --> B[pause for<br/>a human]
CP --> C[time travel<br/>fork a past turn]
CP --> D[crash<br/>recovery]
class CP accent
```
> [!WARNING]
> **No checkpointer, none of the above** — and nothing warns you. Runs start empty and resume quietly
> does nothing. `skein dev`, `embedInMemoryGraphs` and `embedPostgresGraphs` all configure one; the
> mistake is constructing your own and handing it to `.compile()`.
A related trap arrives with it. A run carries four bags — `input`, `context`, `config.configurable`,
`metadata` — and only `input` becomes state the agent remembers.
→ [State, context & persistence](./state-and-context.md)
## Threads: one conversation
A thread is the state plus every checkpoint behind it. Address one by a key you already have — a
ticket number, a phone number — and your system needs no mapping table.
→ [Threads](./threads.md)
## Memory: what outlives the conversation
A thread remembers a conversation, not a person. Two stores, two lifetimes:
| | Scope | Written by |
| ---------------- | ------------------------------------------ | ---------------------- |
| **Checkpointer** | One thread | skein, every step |
| **Store** | Namespaces you choose, across every thread | Your nodes, explicitly |
**The trap:** a user's name in the thread. They start a new conversation and the agent has forgotten.
Facts about a person go in the store; facts about this conversation stay in state.
→ [Memory](./memory.md) · [Storage](./storage.md)
## Pausing for a human
The moment your agent can spend money or send email, someone will want to be asked first. `interrupt()`
_ends the run_ on a checkpoint — no connection held, no timer, nothing billed while it waits. Approval
comes an hour or a week later and the graph resumes where it stopped.
A pause that cannot survive a redeploy is not worth much, which is why this is the feature that most
often justifies durable storage on its own.
→ [Human-in-the-loop](./human-in-the-loop.md)
## Watching it think
Runs stream over server-sent events, and LangGraph's stream modes map onto the wire unchanged — which
is why `useStream` works against a skein server with a URL change and nothing else. A dropped stream
reconnects and joins the same run.
→ [Streaming](./streaming.md) · [Frontend SDKs](./react-sdk.md)
## Work that outlives the request
Not everything is a chat turn.
- [Background jobs](./background-jobs.md) — hand it over, get an id
- [Crons](./crons.md) — a schedule that fires exactly once across every instance
- [Webhooks](./webhooks.md) — your service told when a run settles
The webhook contract is worth reading twice: delivery is durable, committed in the same transaction
as the run's terminal status, but **at-least-once**. Dedupe on `X-Skein-Delivery-Id` and answer `2xx`
only once your own work is committed.
## Config without a redeploy
An **assistant** is a named, versioned configuration of one graph. Ship a prompt change as a version,
roll it back in one call, graph untouched.
→ [Assistants](./assistants.md)
## Knowing what it did
An agent that works in a demo and misbehaves in production is the normal case. skein emits run and
model-call telemetry through a sink interface, with LangSmith, PostHog and OpenTelemetry
implementations, so traces land in a backend you already run.
What skein does not do is judge the output — that is LangSmith or your own harness. Its job is making
every run recorded and replayable.
→ [Observability](./observability.md) · [Errors & logging](./errors-and-logging.md)
## Where this lives in production
The default development server uses memory drivers. Durable deployment is a choice: SQLite for
one local Node owner, or Postgres/Redis for a shared server. They provide different capabilities:
| | Buys you |
| ------------ | -------------------------------------------------------------------------------------------------- |
| **Postgres** | State, checkpoints, threads, runs, memories and pending webhooks across a restart |
| **Redis** | A shared durable run queue and streaming across instances |
| **SQLite** | Local resources, checkpoints, unclaimed queued jobs, and delivery outbox records in one owned file |
Your graph code is identical either way — drivers are injected, not imported.
SQLite records abandoned attempts as failures; applications choose whether to retry or resume.
Redis retains its existing stalled-job redelivery behavior. Delivery retries poll the store's
outbox and do not require Redis.
→ [Storage](./storage.md) · [Local deployment](./local-deployment.md) · [Runs & Redis](./runs-and-redis.md) · [Deploy](./deploy.md)
## What was yours all along
The model, the prompt and the tools — the reason your agent is _your_ agent. The graph is
LangGraph's. Everything else is plumbing skein exists so you do not write.
Want it as a lookup table instead of a story? [Features](./features.md). Want the graph model itself?
[LangGraph essentials](./langgraph-essentials.md).
---
<!-- source: docs/langgraph-essentials.md -->
# LangGraph essentials
skein-js serves LangGraph.js graphs **unchanged**. It wraps none of the API below, adds no dialect
of its own, and the graph you write here runs on LangGraph Platform without an edit. This page is
the shallow dive: the parts of LangGraph you need to read and modify your own agent, each with a
link to the LangChain docs that own it in full.
It is not a LangGraph tutorial. When a section is the one you actually need,
[Thinking in LangGraph](https://docs.langchain.com/oss/javascript/langgraph/thinking-in-langgraph)
and the reference behind each **Go deeper** link are where to spend the time.
## The smallest graph
State, a node, edges and `.compile()` — the first four concepts below, all in the graph
[`npm create skein-js`](./scaffolding.md) scaffolds for you:
```ts
import { AIMessage, type BaseMessage } from "@langchain/core/messages";
import { MessagesAnnotation, StateGraph } from "@langchain/langgraph";
function echo(state: typeof MessagesAnnotation.State): { messages: BaseMessage[] } {
const last = state.messages.at(-1);
const text = typeof last?.content === "string" ? last.content : "";
return { messages: [new AIMessage(`echo: ${text}`)] };
}
export const graph = new StateGraph(MessagesAnnotation)
.addNode("echo", echo)
.addEdge("__start__", "echo")
.addEdge("echo", "__end__")
.compile();
```
## State and channels
State is the data flowing through your agent, declared as **channels**. `MessagesAnnotation` is the
ready-made one for chat — a list of messages that appends. When you need your own shape, declare it:
```ts
import { Annotation, MessagesAnnotation } from "@langchain/langgraph";
const State = Annotation.Root({
...MessagesAnnotation.spec,
draft: Annotation<string>(),
});
```
Keep **raw data in state and format prompts inside nodes**. Everything in state is checkpointed on
every step, so a formatted prompt stored in a channel is a copy you pay for forever.
**Go deeper →** [Graph API](https://docs.langchain.com/oss/javascript/langgraph/graph-api)
## Reducers
A channel's reducer decides what happens when a node returns a value for it: replace, or combine.
The `messages` reducer appends, which is why returning one message adds it rather than truncating
the conversation to a single entry.
```ts
// appends to messages — it does not overwrite the list
return { messages: [new AIMessage("done")] };
```
This bites in one specific place: **editing state by hand**, from the API or the
[console](./console.md). You write through the same reducers, so patching `{ messages: [...] }` to
fix a transcript appends to it. Use `asNode` to attribute the write to a node whose reducer does
what you meant. See [state & context](./state-and-context.md#state-what-the-agent-knows-right-now).
**Go deeper →** [Graph API](https://docs.langchain.com/oss/javascript/langgraph/graph-api)
## Nodes
A node is a plain function. It receives the current state and returns **only the part that
changed** — never the whole state. Because it is a plain function, you can import it in a test,
call it with a literal, and step through it in a debugger without a server anywhere.
Nodes may be async, and they may reach for injected resources — see
[persistence](#persistence-the-checkpointer-and-the-store) below.
**Go deeper →** [Graph API](https://docs.langchain.com/oss/javascript/langgraph/graph-api)
## Edges and conditional edges
Edges say what runs next. `__start__` and `__end__` are the built-in entry and exit points. A
straight line needs only `addEdge`; branching — the reason it is a graph at all — uses
`addConditionalEdges` with a function that returns the name of the next node:
```ts
const graph = new StateGraph(State)
.addNode("classify", classify)
.addNode("approve", approve)
.addNode("send", send)
.addEdge("__start__", "classify")
.addConditionalEdges("classify", (state) => (state.needsApproval ? "approve" : "send"))
.compile();
```
Cycles are allowed and expected: an agent loop is a node that routes back to the model until there
is nothing left to call.
**Go deeper →**
[Workflows and agents](https://docs.langchain.com/oss/javascript/langgraph/workflows-agents)
## `.compile()`
`.compile()` turns the definition into something runnable, and **what it returns is what skein
serves** — the value your `langgraph.json` points at:
```json
{ "graphs": { "agent": "./src/agent-graph.ts:graph" } }
```
Do not pass a `checkpointer` or `store` to `.compile()` yourself. skein injects both per run; see
below.
**Go deeper →** [Graph API](https://docs.langchain.com/oss/javascript/langgraph/graph-api)
## Prebuilt agents
Most agents are the same loop: call the model, run the tool it asked for, call the model again.
`createAgent` is that loop, already written:
```ts
import { createAgent } from "langchain";
export const graph = createAgent({
model: "anthropic:claude-sonnet-5",
tools: [getWeather],
});
```
It returns a compiled graph like any other, so everything skein does — threads, streaming,
interrupts, memory — works against it unchanged.
> [!NOTE]
> You will meet **`createReactAgent`** from `@langchain/langgraph/prebuilt` in older code. It still
> works, but it is `@deprecated` as of `@langchain/langgraph` 1.4: it moved to the `langchain` package
> and was renamed. The parameter changed too — `llm` became `model`, which also accepts a
> `"provider:model"` string — and a dynamic `prompt` function becomes `dynamicSystemPromptMiddleware`.
> skein's examples and scaffolder are all on `createAgent`.
**Go deeper →** [Agents](https://docs.langchain.com/oss/javascript/langchain/agents)
## Tools
A tool is a function plus the metadata a model needs to decide when to call it:
```ts
import { tool } from "@langchain/core/tools";
import { z } from "zod";
const getWeather = tool(async ({ city }: { city: string }) => `It's 21°C in ${city}.`, {
name: "get_weather",
description: "Get the current weather for a city.",
schema: z.object({ city: z.string().describe("City name, e.g. 'Nairobi'") }),
});
```
The `description` and the schema's `.describe()` calls are the model's **only** context for the
tool. Write them for a reader who knows nothing else, because that is exactly the situation.
**Go deeper →** [Tools](https://docs.langchain.com/oss/javascript/langchain/tools)
## `interrupt()` and commands
`interrupt()` pauses the graph from inside a node. The run **ends** on a checkpoint — no connection
held, no timer running — and resuming later returns your supplied value from the `interrupt()`
call:
```ts
import { interrupt } from "@langchain/langgraph";
const answer = interrupt({ question: "Send this email?", draft: state.draft });
```
This is the concept most dependent on the checkpointer: without one, `interrupt()` has nowhere to
park and resume silently no-ops. skein exposes resuming as `command: { resume }` on a normal run
create, plus `resume` / `update` / `goto`.
**Go deeper →** [Interrupts](https://docs.langchain.com/oss/javascript/langgraph/interrupts) ·
skein side: [human-in-the-loop](./human-in-the-loop.md)
## Subgraphs
A compiled graph can be a node in another graph. That is how you keep a large agent readable — a
research step, a drafting step, an approval step, each its own graph with its own state, composed
at the top. skein serves the outer graph; the nesting is invisible to the protocol.
**Go deeper →** [Subgraphs](https://docs.langchain.com/oss/javascript/langgraph/use-subgraphs)
## Stream modes
`graph.stream()` takes a `streamMode` — `values`, `updates`, `messages`, `custom`, `events`,
`debug` — and you may request several at once. skein maps each of them onto Agent Protocol SSE
frames without translation, which is why the LangChain SDKs and `useStream` work against a skein
server with only a URL change.
**Go deeper →** [Streaming](https://docs.langchain.com/oss/javascript/langgraph/streaming) ·
skein side: [streaming](./streaming.md)
## Persistence: the checkpointer and the store
LangGraph defines two persistence interfaces, and **skein supplies both** — a checkpointer bound to
the thread, and a `BaseStore` bridged from whichever [storage driver](./storage.md) you configured.
Your nodes reach the store the usual LangGraph way:
```ts
import { getStore, type LangGraphRunnableConfig } from "@langchain/langgraph";
// In a node, the store arrives on the config…
async function remember(state: State, config: LangGraphRunnableConfig) {
await config.store?.put(["users", userId], "profile", { name: state.name });
}
// …and inside a tool, where there's no config argument, reach for getStore().
async function saveName(name: string) {
await getStore().put(["users", userId], "profile", { name });
}
```
`getStore()` reads the run currently executing, so call it **inside** the function. At module scope
there is no run yet and it throws on import.
> [!WARNING]
> **Do not construct your own checkpointer or store and pass them to `.compile()`.** It is the
> single most common way this breaks: the graph then persists somewhere skein does not know about,
> so threads, time travel and interrupt-resume all read the wrong state. Let the injection happen.
**Go deeper →** [Persistence](https://docs.langchain.com/oss/javascript/langgraph/persistence) ·
skein side: [state & context](./state-and-context.md), [storage](./storage.md)
## Who owns what
| LangGraph owns | skein-js owns |
| ------------------------------------- | --------------------------------------------------- |
| State, channels, reducers | Threads, runs, assistants and their versions |
| Nodes, edges, subgraphs, `.compile()` | The HTTP surface — Agent Protocol, SSE, the console |
| Tools and the agent loop | The run queue, multitask strategies, cancellation |
| `interrupt()` and commands | Crons, run-completion webhooks, idempotency |
| Stream modes | Storage drivers, and injecting them per run |
The line matters when something goes wrong: if it is about what your graph _computed_, it is a
LangGraph question. If it is about what got _served, stored or scheduled_, it is ours.
## See also
- [Building blocks](./building-blocks.md) — the skein-side map of the same territory
- [Your first agent](./your-first-agent.md) — build one from an empty directory
- [LangGraph CLI compatibility](./langgraph-cli-compat.md) — what `langgraph.json` supports
- [Building a runner](./building-a-runner.md) — serving the protocol from something that isn't
LangGraph
---
<!-- source: docs/features.md -->
# Features
What skein-js can do, in one page. Find the row that matches what you're trying to build, then follow
the link. ✅ ships today · ⚠️ preview · 🧪 experimental (works, may change in a minor release) · 🗺️ planned.
Comparing against LangGraph Platform specifically? [roadmap.md](./roadmap.md) has the same ground
organised by parity instead.
## Run agents
| Capability | Status | What it gets you |
| --------------------------------------------------------------------------------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| [Background jobs](./background-jobs.md) | ✅ | Fire-and-forget work: hand skein a job, get an id back, hear the result by webhook |
| [Background runs](./runs.md) | ✅ | Return immediately, let the graph keep working, join the stream later |
| [Wait / stream runs](./runs.md) | ✅ | Hold the connection for the answer, or stream tokens as they're produced |
| [Stateless & batch runs](./runs.md) | ✅ | One-shot calls with no conversation to keep; up to 100 runs per request |
| [Multitask / double-texting](./runs.md#multitask-what-happens-to-the-run-already-going) | ✅ | Decide what a second message does to the run already going — all four strategies |
| [Cancel & rollback](./runs.md#cancelling) | ✅ | Stop a run, keeping its writes or discarding them |
| [Run timeouts](./runs.md#bound-a-runaway-run) | ✅ | Bound a graph that hangs, instead of losing a worker slot |
| [Idempotent run creation](./agent-protocol.md#idempotent-run-creation-idempotency-key) | ✅ | A retrying caller can't start the same run twice. No LangGraph Platform equivalent |
| [Cron schedules](./crons.md) | ✅ | Fire a graph on a schedule, exactly once across instances, no leader election |
| [A graph as a plain endpoint](./serving-a-single-graph.md) | ✅ | `POST /invoke/:graph_id` — body in, final state out, no threads or runs |
| [Deep Agents](./recipes/deep-agents.md) | 🧪 experimental | Serve a LangChain `createDeepAgent` graph — files, per-user memory, approval, and subagents streamed live or run in the background |
## Hold a conversation
| Capability | Status | What it gets you |
| --------------------------------------------------------------------- | ------ | -------------------------------------------------------------------------------------- |
| [State, context & persistence](./state-and-context.md) | ✅ | What the agent knows, what's only for this run, and what survives a restart |
| [Threads](./threads.md) | ✅ | Conversations that persist, addressable by your own key (a ticket id, a phone number) |
| [Streaming (SSE)](./streaming.md) | ✅ | Tokens, tool calls and reasoning as they happen — including a true `events` mode |
| [Reconnect & join](./streaming.md) | ✅ | Resume a dropped stream, or tail the same run from a second client |
| [Subgraph streaming](./streaming.md#subgraphs-and-subagents) | ✅ | `stream_subgraphs` streams nested graphs and subagents under `mode\|<namespace>` names |
| [Human-in-the-loop](./human-in-the-loop.md) | ✅ | Pause for approval and resume hours later, from anywhere |
| [Time travel](./threads.md#time-travel-re-run-a-turn-a-different-way) | ✅ | Fork from any past checkpoint and run forward — "edit and resubmit" |
| [Thread state & history](./threads.md#read-the-state) | ✅ | Read, patch, page, copy or prune a conversation's state |
| [Thread TTL](./storage.md#thread-ttl) | ✅ | Expire conversations automatically. LangGraph OSS ignores `ttl`; skein doesn't |
| [Assistants & versioning](./assistants.md) | ✅ | Ship a prompt change without redeploying, and roll it back in one call |
| [`useStream` and friends](./react-sdk.md) | ✅ | The LangChain client ecosystem works unchanged — React, Vue, Svelte, Angular |
## Remember things
| Capability | Status | What it gets you |
| ------------------------------------------------------------------------------- | ------ | ----------------------------------------------------------------------- |
| [The store, via `getStore()`](./memory.md) | ✅ | Memory that outlives a conversation, reachable from inside a graph node |
| [Semantic search](./storage.md) | ✅ | pgvector-backed recall, with optional HNSW indexing |
| [Filters & namespace traversal](./storage.md#filtering-and-namespace-traversal) | ✅ | Query memory by value, and walk namespaces with wildcards |
| [Store TTL](./storage.md#store-item-ttl) | ✅ | Expire memories, optionally refreshing on read |
| [Bring your own store](./storage.md#bringing-your-own-store-storeadapter) | ✅ | Point skein at a LangGraph `BaseStore` or your own implementation |
| [Memory patterns](./memory.md) | ✅ | Profile vs collection shapes, the dedup trap, recall, background writes |
## Build workflows across your systems
LangGraph owns workflow orchestration; it doesn't own provider integrations. Skein channels supply
the authenticated sources and durable destinations around the graph.
| Capability | Status | What it gets you |
| -------------------------------------------------------------------------- | ------ | ---------------------------------------------------------------------------------- |
| [Workflows & channels](./channels.md) | ✅ | Turn provider events into real processes with durable outcomes |
| [Run-completion webhooks](./webhooks.md) | ✅ | Be told a run finished — durably, so a receiver's redeploy doesn't lose the news |
| [Signed callbacks](./webhooks.md#verify-a-callback-is-really-from-you) | ✅ | Receivers can prove a callback is yours and reject replays. The verifier ships too |
| [Deliveries & replay](./webhooks.md#see-what-a-callback-did-and-replay-it) | ✅ | See every attempt, and re-send one by hand when it never landed |
| [Custom auth](./agent-protocol.md#authentication--authorization) | ✅ | LangGraph's `Auth` model, drop-in, with ownership filters pushed into the query |
| [Telemetry sinks](./observability.md) | ✅ | LangSmith, PostHog, OpenTelemetry — or your own `TelemetrySink` |
| [MCP endpoint](./roadmap.md) | 🗺️ | Expose your graphs as MCP tools |
## Operate it
| Capability | Status | What it gets you |
| ------------------------------------------------------- | ---------- | ------------------------------------------------------------------------------------ |
| [The console](./console.md) | ✅ | Test graphs and track channel wiring, interrupts and deliveries at `/console` |
| [Postgres + pgvector](./storage.md) | ✅ | Durable server storage with semantic search and automatic migrations |
| [Local apps with SQLite](./local-deployment.md) | ✅ | Desktop, CLI, and daemon embedding with persistent resources, queue, and checkpoints |
| [Redis queue & pub/sub](./runs-and-redis.md) | ✅ | Durable run queue and cross-instance streaming |
| [Multi-instance](./deploy.md#scaling-past-one-instance) | ✅ | Atomic create guard, cross-instance cancel, per-thread execution claim |
| [Errors & logging](./errors-and-logging.md) | ✅ | What a failed run reports, and where — wire, log, and callback |
| [Deploy anywhere](./deploy.md) | ✅ | One image; guides for Cloud Run, Fly, Railway, Render, AWS, Kubernetes, a VPS |
| [Performance tuning](./performance.md) | ✅ | Every knob in one table, plus symptom → knob triage |
| [Bun / Deno runtimes](./deploy.md) | ⚠️ preview | Fetch launchers and images ship; the runtime matrices must graduate each |
## Build on it
| Capability | Status | What it gets you |
| -------------------------------------------------- | ------ | ------------------------------------------------------------------------ |
| [Framework adapters](./adapters.md) | ✅ | Express, Fastify, NestJS, Next.js, native Fetch — standalone or embedded |
| [Embed a graph in code](./embedding.md) | ✅ | No `langgraph.json`, no CLI — bring a compiled graph and mount it |
| [Drop-in LangGraph CLI](./langgraph-cli-compat.md) | ✅ | `skein dev` for `langgraph dev`, with your `langgraph.json` unchanged |
| [Scaffolding](./scaffolding.md) | ✅ | `npm create skein-js@latest` — a working project, no API key needed |
| [Your own adapter](./building-an-adapter.md) | ✅ | Put the handler table on any HTTP framework |
| [Your own agent runtime](./building-a-runner.md) | ✅ | Serve the Agent Protocol from something that isn't LangGraph |
## Deliberately not
- **WebSocket transport** — SSE covers the client UX and doesn't affect the React SDK.
- **`skein deploy` to a hosted platform** — self-hosted by design; there's no managed target.
- **Sub-minute cron schedules**, and **backfilling missed occurrences** — see
[crons.md](./crons.md#semantics).
- **Exactly-once webhook delivery** — delivery is durable and retried, but at-least-once, so every
attempt carries a stable dedup key. See
[webhooks.md](./webhooks.md#at-least-once-and-what-your-receiver-owes).
Something missing? [File an issue](https://github.com/skein-js/skein-js/issues) — compatibility
reports are the most useful feedback we get.
---
<!-- source: docs/assistants.md -->
# Assistants
An assistant is a named, versioned configuration of one graph. It's how you ship a prompt change or a
model swap without redeploying — and how you roll it back when the change was wrong.
## You already have one
Every graph in your `langgraph.json` is registered as an assistant at startup, with
`assistant_id` equal to `graph_id`. So this works on a fresh server with nothing configured:
```ts
await client.runs.wait(threadId, "agent", { input }); // "agent" is the graph id *and* an assistant id
```
If you never need more than one configuration per graph, you can stop reading here. Assistants become
useful when you want **several** configurations of the same graph.
## Several configurations of one graph
```ts
const support = await client.assistants.create({
graphId: "agent",
name: "support-tone",
config: { configurable: { system_prompt: "Be concise and formal." } },
});
await client.runs.wait(threadId, support.assistant_id, { input });
```
Whatever you put in `config.configurable` arrives in your graph as `config.configurable`. That's the
seam: the graph stays one deployment, and the assistant decides how it behaves.
This is what makes per-tenant or per-plan behaviour tractable — one assistant per tenant, rather than
branching inside the graph on metadata you have to thread through every node.
## Versioning and rollback
**Every `PATCH` mints a new immutable version.** The live assistant tracks whichever version is
current, so changing a prompt is a `PATCH`, and undoing it is one call:
```ts
await client.assistants.update(assistantId, {
config: { configurable: { system_prompt: "Be warm and brief." } },
}); // now version 2
await client.assistants.getVersions(assistantId); // newest first
await client.assistants.setLatest(assistantId, 1); // back to version 1
```
Rollback points the assistant at an existing version — it doesn't delete the newer one, so you can go
forward again. Nothing is lost by trying a change.
**Runs record which assistant they used**, so a thread tells you what configuration produced it. What
a version does _not_ capture is your graph's code: rolling back an assistant restores config, not the
deployment. If the bad change was in a node, roll back the deploy.
## Inspect what a graph expects
Useful when you're building a UI against a graph you didn't write, or checking what a config change is
allowed to set:
| You want | Call |
| --------------------------------- | --------------------------------------------- |
| Input/output/state/config schemas | `client.assistants.getSchemas(assistantId)` |
| The drawable graph | `client.assistants.getGraph(assistantId)` |
| Subgraph schemas by namespace | `client.assistants.getSubgraphs(assistantId)` |
The [console](./console.md) renders all three — schemas, graph shape, and version history — so it's
usually faster to look there than to call them.
## Housekeeping
```ts
await client.assistants.search({ graphId: "agent" }); // filter by graph, name or metadata
await client.assistants.delete(assistantId);
```
Two things to know before deleting:
- **`delete(id, { deleteThreads: true })` cascades** to the threads that assistant owns. Without it,
the threads stay and keep their history.
- **Deleting a graph-registered assistant doesn't stop the graph.** It's re-registered on the next
boot, since registration is keyed on the graph id.
## What you must get right
- **`assistant_id` is not a graph id once you create your own.** Runs take an assistant id; a graph id
only works because of the startup registration. Store the id you got back from `create`.
- **`config.configurable` is the contract with your graph.** Nothing validates that an assistant's
config matches what the graph reads — a typo'd key is silently ignored rather than rejected. Check
`getSchemas` if you're unsure of the shape.
- **`create` defaults to `if_exists: "raise"`.** Pass `"do_nothing"` if you're seeding assistants at
startup and want the call to be idempotent.
## Working example
There's no dedicated example yet. The [console](./console.md)'s Assistants view is the practical way
to see versioning: edit an assistant's config, run it from the playground, then roll back.
## See also
- [Runs](./runs.md) — passing an assistant id to a run
- [The console](./console.md) — schemas, graph shape, version history in a UI
- [LangGraph CLI compatibility](./langgraph-cli-compat.md) — how `graphs` entries become assistants
- [Agent Protocol](./agent-protocol.md#assistants) — every assistant endpoint and field
---
<!-- source: docs/state-and-context.md -->
# State, context & persistence
What your agent knows, where each piece of it lives, and what survives a restart. This is the
building block everything else rests on — interrupts, time travel, memory and durability are all
consequences of it.
## The four bags, and which one you want
This is the single most common source of confusion, so start here. A run carries four separate
things, and they behave completely differently:
| You pass | Goes to | Persisted? | Use it for |
| --------------------- | ------------------------------- | --------------------------------- | --------------------------------------------------------------------------- |
| `input` | Your graph's **state channels** | **Yes** — checkpointed per thread | The turn itself: the new message, the document |
| `context` | LangGraph's **runtime context** | Stored on the run, not the state | Per-run facts the graph reads but shouldn't remember: a locale, a tenant id |
| `config.configurable` | The graph's **config** | Stored on the run, not the state | Knobs: which model, which prompt, a feature flag |
| `metadata` | The **run/thread row** | Yes, as searchable labels | Finding things later — never read by the graph |
```ts
await client.runs.create(threadId, "agent", {
input: { messages: [msg] }, // becomes state
context: { locale: "en-GB" }, // available to nodes this run
config: { configurable: { model: "fast" } }, // knobs
metadata: { ticket: "T-1234" }, // searchable label
});
```
The rule of thumb: **if the agent should still know it next turn, it belongs in `input`.** Everything
else is per-run.
## State: what the agent knows right now
Your graph declares the shape — the channels — and each node returns a partial update. skein-js
doesn't define this; it's LangGraph's, unchanged. If the model is new to you,
[Thinking in LangGraph](https://docs.langchain.com/oss/javascript/langgraph/thinking-in-langgraph)
is the shortest path to it — and its advice to keep **raw data in state and format prompts inside
nodes** is worth following, because everything you put in state is what gets checkpointed.
The part that surprises people is that **updates go through reducers**. A channel that appends (the
usual `messages` reducer) _adds_ what you return rather than replacing it:
```ts
// appends to messages — it does not overwrite the list
return { messages: [new AIMessage("done")] };
```
That matters the moment you edit state by hand, from the API or the [console](./console.md): you're
writing through the same reducers, so writing `{ messages: [...] }` to fix a transcript will append,
not replace. Use `asNode` to attribute the write to a node whose reducer does what you intend.
## Checkpoints: what makes everything else work
LangGraph saves the state to a **checkpoint** at node boundaries — each superstep of the graph. That
single mechanism is what gives you:
- **Multi-turn conversations** — the next run on the thread resumes from the last checkpoint rather
than starting empty.
- **[Human-in-the-loop](./human-in-the-loop.md)** — `interrupt()` parks on a checkpoint. The run
ends; nothing holds a connection or a timer. Resuming reads the checkpoint back.
- **[Time travel](./threads.md#time-travel-re-run-a-turn-a-different-way)** — every checkpoint is
addressable, so you can read one, fork it, and run forward from the branch.
- **Crash recovery** — a process that dies mid-run leaves a committed checkpoint behind.
> [!WARNING]
> **No checkpointer, none of the above.** Without one, each run starts empty, `interrupt()` has
> nowhere to park, and resume silently no-ops — the single most common way this goes wrong. `skein dev`
> configures one for you; in code, `embedInMemoryGraphs` / `embedPostgresGraphs` do.
Checkpoint history is readable and pageable — see [threads.md](./threads.md#read-the-state). It's
also the largest thing you store, which is why `POST /threads/prune` has a `keep_latest` strategy
that drops history while keeping current state.
## Short-term vs long-term: two different stores
The distinction people miss:
<div class="skein-cards">
<div class="skein-card">
<span class="ico">🧵</span>
### Short-term — the checkpointer
Scoped to **one thread**. Holds the conversation's state and its history. Written automatically by
the graph on every step; you rarely touch it directly.
</div>
<div class="skein-card">
<span class="ico">🗄️</span>
### Long-term — the store
Scoped to **whatever namespace you choose**, across every thread. Read and written explicitly by your
nodes via `getStore()`, with semantic search and TTL.
</div>
</div>
A user's name belongs in the store, not the thread — otherwise the agent forgets it the moment they
start a new conversation:
```ts
import { getStore, type LangGraphRunnableConfig } from "@langchain/langgraph";
// In a node, the store arrives on the config…
async function remember(state: State, config: LangGraphRunnableConfig) {
await config.store?.put(["users", userId], "profile", { name: state.name });
}
// …and inside a tool, where there's no config argument, reach for getStore().
async function saveName(name: string) {
await getStore().put(["users", userId], "profile", { name });
}
```
`getStore()` reads the run currently executing, so it has to be called inside the function — at
module scope there is no run yet and it throws.
Both reach the same store — skein bridges its own into every run as a LangGraph `BaseStore`, so
nothing here is skein-specific and the same graph runs unchanged on LangGraph Platform.
Full patterns — profile vs collection shapes, the dedup trap, recall — are in
[memory.md](./memory.md); the mechanism and drivers are in [storage.md](./storage.md).
## What actually persists, and where
| Thing | Stored in | Survives a restart? |
| ------------------------- | -------------------------------- | -------------------------------------------------------------------------------------- |
| Graph state & checkpoints | The checkpointer | With SQLite/Postgres, yes. In memory, no |
| Threads, runs, assistants | The store | With SQLite/Postgres, yes. In memory, no |
| Long-term memories | The store | With SQLite/Postgres, yes. In memory, no |
| Queued & delayed runs | The queue | Unclaimed jobs persist with SQLite/Redis; active attempts follow driver crash behavior |
| Pending webhook retries | The delivery outbox in the store | With SQLite/Postgres; the delivery worker polls persisted records after restart |
| Live stream frames | The event bus | SQLite uses process-local frames; Redis frames remain replayable within retention |
**By default, `skein dev` uses memory drivers** and snapshots to `.skein/` so your threads survive a
restart while you work. **For a shared server**, use Postgres
and Redis. **For a local app or one Node owner**, SQLite persists resources and checkpoints in
one file. It records abandoned attempts as `ProcessLostError` failures without replaying input;
applications decide whether to retry or resume. See [storage.md](./storage.md),
[local deployment](./local-deployment.md), and [runs-and-redis.md](./runs-and-redis.md).
## Expiring what you don't need
State accumulates. Two TTLs bound it:
- **Thread TTL** (`checkpointer.ttl`) — expire whole conversations. A per-thread `ttl` overrides the
default, and `null` pins a thread forever. [Details](./storage.md#thread-ttl)
- **Store TTL** (`store.ttl`) — expire individual memories, optionally refreshing on read.
[Details](./storage.md#store-item-ttl)
Both are skein going past LangGraph OSS, which drops `ttl` on the floor.
## What you must get right
- **`input` is the only bag the agent remembers.** `context` and `configurable` are per-run. Putting
a fact in `context` and expecting it next turn is the classic mistake.
- **Reserved `configurable` keys are stripped.** `thread_id`, `run_id`, `checkpoint_id`,
`checkpoint_ns`, `langgraph_auth_user`, anything starting with `__` — the server owns these, so a
client can't redirect a run to another thread or spoof the authenticated caller. Your own keys pass
through untouched.
- **The authenticated user arrives in `configurable`**, not in `context` — as
`langgraph_auth_user`, `langgraph_auth_user_id` and `langgraph_auth_permissions`, stamped
server-side and unspoofable. Present only when [auth](./agent-protocol.md#authentication--authorization)
is configured.
- **The store is not ownership-scoped by default.** An authenticated caller can read every tenant's
items unless an `@auth.on.store` handler narrows it. See
[scoping the store](./agent-protocol.md#scoping-the-store).
- **Big state is expensive.** Checkpoints hold the whole state per superstep, so a graph that keeps
large blobs in a channel multiplies them. Keep artifacts in object storage and a reference in state.
## See also
- [Threads](./threads.md) — reading, editing, forking and expiring conversation state
- [Memory](./memory.md) — long-term memory patterns
- [Storage](./storage.md) — `SkeinStore`, drivers, TTL, bring-your-own
- [Human-in-the-loop](./human-in-the-loop.md) — what checkpoints make possible
- [Building a runner](./building-a-runner.md) — implementing state for a non-LangGraph runtime
---
<!-- source: docs/threads.md -->
# Threads
A thread is one conversation: its message history, its graph state, and every checkpoint along the
way. If you're building a chat product, a thread is what a user would call "a chat" — and it's the
thing you address instead of tracking state yourself.
## Start a conversation
```ts
const thread = await client.threads.create();
await client.runs.wait(thread.thread_id, "agent", { input: { messages: [msg] } });
```
You rarely need more than that. Threads persist, so the next run on the same id picks up where the
last one left off — the graph sees the accumulated state without you passing any of it back.
## Address a conversation by your own key
Most apps already have an identity for the conversation — a phone number, a support ticket, an email
thread. You don't need to store skein's thread id alongside it. Pass your own id and make the create
a **get-or-create**:
```ts
const thread = await client.threads.create({
threadId: ticketId,
ifExists: "do_nothing", // returns the existing thread instead of a 409
});
```
Uniqueness is enforced in storage, so two instances racing the same id cannot both win — no lock of
your own required. The default is `ifExists: "raise"`, a 409 on a taken id.
The other half of the idiom is on run creation: `if_not_exists: "create"` lets a run bring the thread
into existence, so an inbound webhook can start a run in one call. See [runs.md](./runs.md).
## Read the state
```ts
const state = await client.threads.getState(threadId); // current values, next nodes, tasks
const history = await client.threads.getHistory(threadId, { limit: 10 }); // newest first
```
`getState` is what a UI hydrates from on load. `getHistory` walks the checkpoints backwards — page
with `before`, passing the last checkpoint you received.
**History is capped, and the cap matters.** You get at most 100 checkpoints when `limit` is omitted,
and a `limit` above 1000 is rejected — each element is a whole graph state, so a long thread's full
history is one of the largest responses skein can produce. `useStream` asks for 10, which is enough to
render the transcript (the newest checkpoint carries the whole message list) and only limits how far
back the branch/edit tree reaches.
## Time travel: re-run a turn a different way
```mermaid
flowchart LR
C1[turn 1] --> C2[turn 2] --> C3[turn 3]
C2 --> F[edited copy of turn 2] --> F2[turn 3, differently]
class F accent
```
The original branch is left intact — a fork is a new checkpoint, not a rewrite.
Fork from any past checkpoint and run forward from there, leaving the original branch intact. This is
what "edit and resubmit" in a chat UI is built on, and it's also the fastest way to debug what a node
actually saw.
```ts
// Run again from an earlier checkpoint
await client.runs.create(threadId, "agent", { input, checkpointId });
```
To change the state before re-running, write a new checkpoint that forks history first:
```ts
const forked = await client.threads.updateState(threadId, {
values: { messages: editedMessages },
checkpointId, // omit to fork from the tip
});
// The SDK returns a checkpoint config; the new id is on `configurable`.
const forkedId = forked.configurable?.["checkpoint_id"] as string;
// `input: null` + a checkpoint resumes the graph from that point rather than starting over.
await client.runs.create(threadId, "agent", { input: null, checkpointId: forkedId });
```
Things to know:
- **The fork target is server-validated.** A client cannot redirect a run to an arbitrary checkpoint
through `config` — skein reads `checkpoint_id` only from the top-level field.
- **`updateState` 409s while a run is in flight** on that thread. Cancel or wait first.
- **It rides the checkpointer**, so forking costs no extra storage. If you want a whole independent
copy instead, `client.threads.copy(threadId)` duplicates the thread and its history.
- The [console](./console.md) does all of this without code — open a checkpoint, edit, fork, run
forward.
> [!WARNING]
> **Values you write go through the graph's reducers.** A channel that appends — a message list, say —
> will add what you write rather than replace it. This surprises everyone once. Use `asNode` to
> attribute the write to a node whose reducer does what you intend.
## Find threads
```ts
await client.threads.search({ metadata: { graph_id: "my_graph" }, status: "interrupted" });
```
skein stamps a run's `graph_id` and `assistant_id` onto its thread's metadata, so filtering by graph
is just a metadata match. The stamp reflects the thread's most recent run — a thread that has never
run carries no `graph_id`.
Status filters are how you build a "waiting for you" queue: `interrupted` threads are the ones
[paused for a human](./human-in-the-loop.md).
## Import an existing conversation
Migrating from another system, you usually want the history there without re-running the graph over
it. `POST /threads` accepts `supersteps` — updates written straight into the checkpoint history:
```ts
await client.threads.create({
graphId: "agent", // required — a brand-new thread has no run to infer the graph from
supersteps: [{ updates: [{ values: { messages: [past] }, asNode: "__start__" }] }],
});
```
Each superstep is one tick and becomes one checkpoint. Bounded at 100 supersteps of 100 updates.
Without `graphId` this is a 400 — and it's also what lets the seeded state be **read back**, since
`getState` falls back to `metadata.graph_id` for a thread that has never run.
## Clean up
| You want | Call |
| ---------------------------------------------- | ---------------------------------------------------- |
| One thread gone, with its runs and checkpoints | `client.threads.delete(threadId)` |
| Many gone, by filter | `POST /threads/prune` with `strategy: "delete"` |
| To keep the conversation but drop its history | `POST /threads/prune` with `strategy: "keep_latest"` |
| Threads to expire on their own | `checkpointer.ttl` — see below |
`keep_latest` is the one people miss: it keeps each thread's current state and discards only the
checkpoint history behind it. That's usually where the bytes are, and losing it costs you time travel,
not the conversation.
### Expiring threads automatically
```json
{
"checkpointer": {
"ttl": { "default_ttl": 43200, "strategy": "delete", "sweep_interval_minutes": 60 }
}
}
```
Durations are in **minutes**. `POST /threads` takes a per-thread `ttl` that overrides the default, and
an explicit `null` **pins** a thread so no TTL ever collects it.
**Expiry means "may be collected", not "gone".** An expired thread still reads normally until the
sweeper takes it — hiding it early would make a thread with an in-flight run vanish out from under
that run.
> [!WARNING]
> **Deleting a thread deletes any cron scheduled on it.** A thread-scoped [cron](./crons.md) on an
> expiring thread stops firing, silently. Pin such threads with `ttl: null`, or use a stateless cron,
> which owns no thread to lose.
LangGraph OSS drops `ttl` on the floor (it's a Platform feature there), so this is skein going past
what the open-source checkpointers do. Full reference: [storage.md](./storage.md#thread-ttl).
## Working example
[`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) puts each issue
on its own thread, keyed by the issue id, and re-sweeps without duplicating any of them.
## See also
- [Runs](./runs.md) — starting work on a thread
- [Human-in-the-loop](./human-in-the-loop.md) — the `interrupted` status and resuming
- [The console](./console.md) — browsing threads, checkpoints and forks in a UI
- [Storage](./storage.md) — where threads and checkpoints actually live
- [Agent Protocol](./agent-protocol.md#threads) — every thread endpoint and field
---
<!-- source: docs/runs.md -->
# Runs
A run is one execution of your graph. Starting one is the single most common thing you'll do against
skein, and the shape you pick — wait for it, stream it, or queue it — decides everything about how
your client has to behave.
## Pick a run mode
| You want | Use | You get |
| -------------------------------------------- | ------------------------------------------ | ----------------------------------------- |
| The answer, and you can hold a connection | `client.runs.wait(threadId, …)` | The final state, as JSON |
| Tokens as they're produced (a chat UI) | `client.runs.stream(threadId, …)` | An SSE stream |
| To return immediately and check back later | `client.runs.create(threadId, …)` | The `Run` row; work continues server-side |
| A one-shot call with no conversation to keep | The same three, with `null` for the thread | skein creates and owns the thread |
| To start many at once | `client.runs.createBatch([…])` | Up to 100 runs per request |
```ts
// Wait for it
const state = await client.runs.wait(threadId, "agent", { input });
// Stream it
for await (const chunk of client.runs.stream(threadId, "agent", {
input,
streamMode: "messages",
})) {
// …
}
// Queue it, come back later
const run = await client.runs.create(threadId, "agent", { input });
for await (const ev of client.runs.joinStream(threadId, run.run_id)) console.log(ev);
```
**Background runs need somewhere to run.** They execute on the server after your request returns, so
they need a process that stays alive — which is why they don't work on serverless platforms. See
[what doesn't work on serverless](./deploy-serverless.md).
Streaming is its own topic — the stream modes, reconnecting mid-stream, and joining from a second
client are in [streaming.md](./streaming.md).
## Multitask: what happens to the run already going
Send a second message before the first finishes ("double-texting") and something has to give. Pass
`multitask_strategy` on the second run to say what:
| Strategy | The run already going | The new run | Reach for it when |
| ----------- | --------------------------- | ------------------ | ------------------------------------------------------------------------------------------ |
| `reject` | Keeps going | **Fails with 422** | The default. A second message is a bug or a double-click |
| `interrupt` | Stops, keeps what it wrote | Starts now | The user changed their mind, and the partial answer is still worth keeping |
| `rollback` | Stops, its writes discarded | Starts now | The user is correcting themselves — the abandoned turn should read as if it never happened |
| `enqueue` | Runs to completion | Waits, then runs | Both messages matter and order is what you want |
```ts
await client.runs.create(threadId, "agent", { input, multitaskStrategy: "interrupt" });
```
Things worth knowing before you pick:
- **`reject` is the default, and a `pending` run counts as busy.** A run held by `afterSeconds`, or
one sitting in the queue, will reject the next one just as a running one does. That surprises
people — but work _is_ scheduled on the thread.
- **The 422 is `thread_busy`.** Handle it distinctly from an ordinary validation failure: the
client's move is to retry or to switch strategy, not to fix the request.
- **`enqueue` is not strict FIFO** unless you run at concurrency 1. Concurrent workers can observe
queued runs in different orders. See [thread dispatch](./runs-and-redis.md#dispatch-by-thread).
- **It's decided atomically**, so two instances racing the same thread cannot both win. You don't need
a lock of your own.
A displaced run settles `interrupted` (under `interrupt`) or `cancelled` (under `rollback`), and one
that never started executing sends no [webhook](./webhooks.md).
## Cancelling
```ts
await client.runs.cancel(threadId, runId); // action: "interrupt", the default
```
| `action` | Effect |
| ----------------------- | ----------------------------------------------------------------------------------------------------- |
| `interrupt` _(default)_ | Settles the run `cancelled` and **keeps** whatever it wrote |
| `rollback` | Also discards its checkpoint writes and deletes the run row — the turn reads as never having happened |
> [!WARNING]
> **`cancel` takes these positionally, not as an options object.** The signature is
> `cancel(threadId, runId, wait?, action?)`, so `cancel(tid, rid, { action: "rollback" })` binds your
> object to `wait` and silently performs an ordinary `interrupt` instead.
```ts
await client.runs.cancel(threadId, runId, true, "rollback"); // wait for it to stop, discard its writes
```
The third argument returns only once the run has actually stopped, rather than as soon as it's been
marked.
To cancel in bulk, `POST /runs/cancel` takes `{ thread_id?, run_ids?, status? }`, narrowest selector
first: explicit ids, else one thread's inflight runs, else **every** inflight run on the server.
Unknown ids are skipped rather than failing the sweep, and the response says what actually happened.
## Bound a runaway run
A graph that hangs — a model call with no timeout of its own, a node that loops — holds a worker slot
until the process restarts. Set `--run-timeout <ms>` or `SKEIN_RUN_TIMEOUT_MS`; the run aborts and
settles as `timeout`.
**Off by default, deliberately.** A legitimate research or multi-step tool run takes minutes, so a
default would turn slow-but-working into killed — the exact failure the timeout exists to prevent.
Pick a number from your own graphs' worst honest case.
Individual run creation requests can additionally supply `timeout_ms`, an integer from 1 through
2,147,483,647 milliseconds. The duration starts at execution, so a queued delay does not consume it.
The effective limit is the smallest of this duration, the deployment limit, and any remaining native
deadline. Native start/resume and BullMQ command inputs inherit the same field. Omit it for the
existing deployment behavior. `GET /info.flags.per_run_timeout: true` confirms support.
The installed LangGraph SDK does not serialize this additional field. Use a submission-scoped
`onRequest` hook to add it to the prepared run body, or send the wire request directly:
```ts
await fetch(`${url}/threads/${threadId}/runs`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ assistant_id: "research", input: {}, timeout_ms: 30_000 }),
});
```
The console exposes this control under **Advanced run options** after capability confirmation.
## Start a run later
`afterSeconds` holds a run before it starts — capped at 86400 (a day); for anything longer use a
[cron](./crons.md).
Use it on a **background** run. The queue holds it, so it costs nothing while it waits and, on Redis,
survives a restart. On an inline `wait`/`stream` run the server holds _your connection_ open instead,
so a long delay will hit a proxy's idle timeout.
A delayed run is cancellable like any other, and counts as inflight the whole time it waits.
## Fields where skein differs from LangGraph
Two defaults are deliberately not LangGraph's. If you're migrating, these are the ones to check:
| Field | skein | LangGraph | Why |
| --------------- | ---------- | --------- | ------------------------------------------------------------------------------------------ |
| `on_completion` | `keep` | `delete` | A stateless run's thread stays inspectable afterwards |
| `on_disconnect` | `continue` | `cancel` | A proxy timeout is indistinguishable from a real hang-up, and shouldn't kill a healthy run |
Pass the LangGraph value explicitly if you want its behaviour. Note `useStream` sends
`on_disconnect: "cancel"` on every submit unless the stream is resumable, so with a browser client
closing the tab does stop the run — skein's default only applies to callers that don't send the field.
One more worth knowing, though it matches LangGraph: **naming a thread that doesn't exist is a 404**
(`if_not_exists: "reject"`). Pass `"create"` to have the run bring the thread into existence instead —
which is how you start a run keyed on an external identity (a phone number, a ticket id) without a
round trip to create the thread first.
## Don't start a run on a thread that's waiting for a human
`interrupt()` leaves a thread parked until someone answers, and the run that parked it is **terminal**
— `interrupted` is a finished status, so the thread holds no inflight run. That means
`multitask_strategy` cannot protect it: every strategy, including the default `reject`, arbitrates
`pending` and `running` only. A plain start on an interrupted thread therefore **succeeds**, and the
pending question is discarded with no error and no log line.
`if_thread_status` is how you say "only if nobody is waiting":
```ts
await fetch(`${url}/threads/${threadId}/runs`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({
assistant_id: "support",
input: { messages: [{ role: "human", content: text }] },
if_thread_status: ["idle", "error"],
}),
});
// → 409 { code: "thread_status_mismatch", details: { status: "interrupted" } }
```
The check happens **inside the driver's atomic create**, alongside the one `multitask_strategy: "reject"`
already uses — so two replicas racing the same thread cannot both win. An in-process mutex cannot give
you this; the window is between processes.
The 409 carries the status it actually observed, so you can branch without a second read that might
see a third value. The usual branch for an async chat channel is: refused with `interrupted` → resume
the pending interrupt instead of starting a new run.
```ts
// The reply hours later is an answer, not a new conversation.
body: JSON.stringify({ assistant_id: "support", command: { resume: text } });
```
It composes with `multitask_strategy` rather than overlapping it — that one guards `pending`/`running`,
this one guards everything else. Omit the field and run creation behaves exactly as it always has.
A storage driver that does not implement the precondition answers `501 if_thread_status_unsupported`
rather than quietly falling back to a read-then-create, because a non-atomic fallback looks like it
works right up until two replicas race. Both first-party drivers implement it.
## Don't create the same run twice
A retrying caller — Stripe, GitHub, Twilio, or your own sweep — should not start a second run. Send an
`Idempotency-Key` header and the original response replays:
```ts
// `runs.create` has no `headers` option — a `headers` key in the payload is silently dropped and you
// get a brand-new run every retry. A per-key client is the only way to send it.
const client = new Client({ apiUrl, defaultHeaders: { "Idempotency-Key": issue.id } });
await client.runs.create(threadId, "agent", { input });
```
Streaming creates reject the header rather than ignoring it — an SSE response has no body to replay.
Details: [agent-protocol.md](./agent-protocol.md#idempotent-run-creation-idempotency-key).
## Reading a run back
| You want | Call |
| ---------------------------------------- | ----------------------------------------- |
| One run's row | `client.runs.get(threadId, runId)` |
| A thread's runs | `client.runs.list(threadId)` |
| To tail a run already in flight | `client.runs.joinStream(threadId, runId)` |
| To block until it settles, then get JSON | `client.runs.join(threadId, runId)` |
Joining a run that has **already** settled returns immediately, long after its frames have aged out —
the wait is decided by the run row, not by the event stream. So a client that reconnects late still
gets its answer.
To be told a run finished without holding anything open, use a [webhook](./webhooks.md).
## Working example
[`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) dispatches one
background run per issue, each on its own thread, with idempotent re-sweeps — and runs with no API key
and no network.
## See also
- [Streaming](./streaming.md) — stream modes, reconnecting, joining
- [Runs & Redis](./runs-and-redis.md) — run concurrency, the queue, scaling past one instance
- [Human-in-the-loop](./human-in-the-loop.md) — pausing a run for a person
- [Crons](./crons.md) — running a graph on a schedule
- [Agent Protocol](./agent-protocol.md#runs--stateless--ephemeral) — every run endpoint and field
---
<!-- source: docs/background-jobs.md -->
# Background jobs
Fire-and-forget work: hand skein a job, get an id back immediately, and hear about the result later.
No conversation to maintain, no connection to hold open. This is the shape you want when the caller is
another service rather than a person — classification, enrichment, document processing, a queue
drained by an agent.
There is no separate "tasks" API. A background job **is** a run — you just don't keep the thread.
## Enqueue and return
```ts
const run = await client.runs.create(null, "agent", {
input: { documentUrl },
onCompletion: "delete", // don't keep the thread once it's done
webhook: "https://example.com/hooks/done", // tell me when it lands
});
// returns in milliseconds; the graph runs server-side
```
`null` for the thread id means **stateless**: skein creates a thread for the run, and
`onCompletion: "delete"` removes it once the run settles. Nothing accumulates.
Leave `onCompletion` off (skein's default is `keep`) if you'd rather the job stay inspectable
afterwards — you can read its state and its checkpoints for as long as you keep it.
## Hearing the result
Three ways, in order of how much they cost you:
| Approach | Use when |
| ----------------------------------- | -------------------------------------------------------------- |
| A [webhook](./webhooks.md) | The default. Nothing to poll, and delivery survives a redeploy |
| `client.runs.join(threadId, runId)` | You can hold a connection and just want the answer |
| `client.runs.get(threadId, runId)` | You already have a polling loop |
The webhook is the one to reach for. It's recorded in the same transaction as the run's terminal
status, so a receiver that was restarting doesn't lose the news — and you can
[see and replay](./webhooks.md#see-what-a-callback-did-and-replay-it) a callback that never landed.
If you kept the thread (`onCompletion: "keep"`), joining works long after the run settled — the wait is
decided by the run row, not by a live stream. If you deleted it, the webhook payload is the record;
that's exactly why the payload is stored rather than re-rendered at send time.
## Don't run the same job twice
A retrying caller — a queue, a cron, a partner's webhook — will re-send. Send an `Idempotency-Key` and
the original response replays instead of starting a second run:
```ts
// `runs.create` has no `headers` option, so a per-key client is the only way to send it.
const client = new Client({ apiUrl, defaultHeaders: { "Idempotency-Key": documentId } });
await client.runs.create(null, "agent", { input, onCompletion: "delete" });
```
The claim is arbitrated by a uniqueness constraint, so 50 concurrent retries across two instances
still produce exactly one run. Details:
[agent-protocol.md](./agent-protocol.md#idempotent-run-creation-idempotency-key).
## Many, later, or on a schedule
```ts
// Up to 100 at once
await client.runs.createBatch(items.map((item) => ({ assistantId: "agent", input: item })));
// Start in 30 seconds (capped at a day)
await client.runs.create(null, "agent", { input, afterSeconds: 30 });
// Every five minutes, forever
await client.crons.create("agent", { schedule: "*/5 * * * *", input: { source: "queue" } });
```
A delayed background run is held by the queue, so it costs nothing while it waits and — on Redis —
survives a restart. For anything longer than a day, use a [cron](./crons.md).
## How many run at once
One instance executes **10** background runs concurrently by default, matching the LangGraph CLI's
`--n-jobs-per-worker`. Raise it with `--concurrency` or `SKEIN_RUN_CONCURRENCY` when your jobs are
I/O-bound (model calls); add instances when they're CPU-bound.
Concurrency is the knob with the widest blast radius — it multiplies memory and Postgres connections
at once. Size it against your pool: [runs-and-redis.md](./runs-and-redis.md#run-concurrency).
## What you must get right
- **A failed job is not retried.** skein retries webhook _delivery_, not the run. A graph that throws
settles `error` and stays there. If your work needs retries, either retry inside the graph, or have
your caller re-submit with a fresh idempotency key.
- **A crashed worker follows its queue's contract.** Redis can redeliver stalled work, skipping
already terminal runs. SQLite preserves unclaimed jobs but marks abandoned attempts as
`ProcessLostError` failures; applications choose whether to retry. Memory loses queued jobs.
- **Background runs need a process that stays alive.** They execute after your request returns, so
they don't work on serverless. See [deploy-serverless.md](./deploy-serverless.md).
- **Use durable storage.** Choose SQLite for one local owner, or Postgres/Redis for shared workers.
See [local deployment](./local-deployment.md) and [runs-and-redis.md](./runs-and-redis.md).
- **Bound a job that hangs.** A graph with no timeout of its own holds a worker slot until the process
restarts. Set `--run-timeout` / `SKEIN_RUN_TIMEOUT_MS`; it's off by default.
- **With Postgres, each job holds a connection** while it runs, and a second one on that driver for
its thread claim. Ten concurrent jobs is not one connection.
## When you want the answer inline instead
If the caller can wait and there's genuinely no conversation — a classifier, an extractor — skip runs
entirely:
```bash
curl -sX POST localhost:2024/invoke/triage \
-H 'content-type: application/json' \
-d '{"text":"Refund charge failed — urgent!"}'
```
`POST /invoke/:graph_id` takes the graph's input as the body and answers with its final state. No
threads, no run rows, no webhooks. See
[serving-a-single-graph.md](./serving-a-single-graph.md).
## Working example
[`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) is exactly this
shape: a cron sweep dispatches one durable background run per item, idempotently, so re-sweeping
creates nothing. It runs with no API key and no network.
## See also
- [Runs](./runs.md) — every run mode and field
- [Webhooks](./webhooks.md) — being told a run finished
- [Crons](./crons.md) — running a graph on a schedule
- [Runs & Redis](./runs-and-redis.md) — the queue, concurrency, scaling out
- [Serving a single graph](./serving-a-single-graph.md) — the synchronous non-chat surface
---
<!-- source: docs/human-in-the-loop.md -->
# Human-in-the-loop
Pause a run for a person to approve, edit, or answer something — and resume it later, possibly hours
later, from a different client. An interrupted run holds no connection and no timer, only a
checkpoint, so your process can restart while it waits.
```mermaid
flowchart LR
A[create run] --> B["interrupt()"] --> C["run ends<br/>nothing held open"]
C --> D["hours, or weeks"] --> E[resume] --> F[done]
class C accent
```
## Pause the graph
Call LangGraph's `interrupt()` from a node. Whatever you pass becomes the payload your UI renders:
```ts
import { interrupt } from "@langchain/langgraph";
function askForApproval(state: State) {
const answer = interrupt({
question: "Send this email?",
draft: state.draft,
});
return { sent: answer === true };
}
```
The run settles, the thread's status becomes `interrupted`, and the interrupt surfaces in the stream.
Nothing is holding anything open.
## Resume it
```ts
await client.runs.create(threadId, "agent", {
input: null, // not a new turn — pick up where the interrupt parked
command: { resume: true },
});
```
`input: null` plus a `command` is the whole idiom. The run resumes from the checkpoint the interrupt
parked on rather than starting over, and the value you pass in `resume` is what `interrupt()` returns
inside the node.
Three commands are available:
| Command | Does | Use for |
| -------- | ------------------------------------------- | ------------------------------------------ |
| `resume` | Returns a value from the `interrupt()` call | Approve / reject / supply an edited answer |
| `update` | Writes state before continuing | Correcting something the graph got wrong |
| `goto` | Continues at a named node | Sending the turn down a different path |
If you're building on `POST /threads/{id}/commands` directly, it infers the assistant from the
thread's last run, so you don't have to track it — and it **409s (`thread_not_interrupted`)** on a
thread that isn't paused, rather than quietly starting a new turn.
## In a React UI, this is free
[`useStream`](./react-sdk.md) renders the interrupt and resumes it for you — no extra endpoints, no
polling. The interrupt arrives on the stream, you render your approval card, and submitting a command
continues the same conversation.
## Find what's waiting
An approvals queue is a thread search:
```ts
const waiting = await client.threads.search({ status: "interrupted" });
```
The [console](./console.md) ships this view already — its Overview lists what's waiting for you, and
the Interrupts view approves, rejects, or resumes with any JSON, without you building a UI first.
## What you must get right
- **A checkpointer is required.** Without one there is no checkpoint to park on, and resume silently
no-ops — the single most common way this fails. `skein dev` gives you one; in code, use
`embedInMemoryGraphs`/`embedPostgresGraphs` or supply your own. See [embedding.md](./embedding.md).
- **Use durable storage for anything real.** With the in-memory driver an interrupted thread is gone
on restart, which defeats the point of pausing for a human. [Postgres](./storage.md) keeps it.
- **The pause has no timeout.** A thread waits indefinitely. If your product needs "auto-reject after
48 hours", that's a [cron](./crons.md) sweeping `status: "interrupted"` threads — skein won't do it
for you.
- **Resuming starts a new run.** The interrupted run is terminal; the resume is a fresh run on the
same thread. So a webhook fires per run, and the run id you resumed with is not the id you get back.
- **A plain start on a waiting thread is not refused for you.** Because the interrupted run is
_terminal_, the thread holds no inflight run — so `multitask_strategy`, which arbitrates
`pending`/`running`, never sees it, and a fresh start **succeeds** and discards the pending
question. Pass [`if_thread_status`](./runs.md#dont-start-a-run-on-a-thread-thats-waiting-for-a-human)
when the caller might not know the thread is waiting, which is every asynchronous channel:
```ts
// 409 `thread_status_mismatch` instead of silently trampling the interrupt.
{ assistant_id: "support", input, if_thread_status: ["idle", "error"] }
```
The check runs inside the driver's atomic create, so it holds across replicas — an in-process lock
cannot give you this.
- **Not available on `POST /invoke/:graph_id`**, which has no thread to park on. See
[serving-a-single-graph.md](./serving-a-single-graph.md).
## Interrupting from outside the graph
You can also pause without an `interrupt()` call, by naming nodes on run creation:
```ts
await client.runs.create(threadId, "agent", { input, interruptBefore: ["send_email"] });
```
`interruptBefore` / `interruptAfter` take a list of node names, or `"*"` for every node. This is a
debugging and stepping tool more than a product feature — `interrupt()` is what you want when the
pause is part of the design, because it carries a payload the UI can render.
## Working examples
- [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) — reached by
a conditional edge, so only items that need a human park. Runs with no API key and no network;
approve from the console.
- [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) — an approval card in
a real Next.js UI.
## See also
- [Threads](./threads.md) — the `interrupted` status, and editing state before you resume
- [Runs](./runs.md) — what a resume run is, and multitask strategies
- [Frontend SDKs & `useStream`](./react-sdk.md) — rendering and resuming in React
- [The console](./console.md) — approving without building a UI
- [Building a runner](./building-a-runner.md) — implementing interrupts for a non-LangGraph runtime
---
<!-- source: docs/webhooks.md -->
# Run-completion webhooks
Tell another service that a run finished, and have that news survive a receiver that was redeploying
when it happened. Pass a `webhook` URL on run creation and skein POSTs the settled run to it —
durably, with retries, optionally signed, and replayable by hand when it never lands.
## Quick start
```ts
await client.runs.create(threadId, "agent", {
input,
webhook: "https://example.com/hooks/run",
});
```
That is the whole opt-in. The field is LangGraph's, unchanged; everything below is delivery
semantics, not a different payload.
It works on every run-creation route that has a run to report on — background, `wait` and `stream`,
stateless, batch, and each firing of a [cron](./crons.md). It is not available on
[`POST /invoke/:graph_id`](./serving-a-single-graph.md), which has no run row.
## What the callback carries
The settled run, as a JSON body: the run's own fields (`run_id`, `thread_id`, `assistant_id`,
`metadata`, …) plus
| Field | What it is |
| ------------------------------- | ----------------------------------------------------------------------------------------- |
| `status` | The run's terminal status — `success` · `error` · `timeout` · `interrupted` · `cancelled` |
| `values` | The run's final state |
| `error` | The failure message, on a failed run only (a string, matching LangGraph) |
| `interrupts` | The questions the run is waiting on, on an `interrupted` run only — see below |
| `reply` | What the graph declared should be sent back, if it declared one — see below |
| `run_started_at`/`run_ended_at` | When the run ran |
| `webhook_sent_at` | When **this attempt** was sent — it changes between retries |
`status` is read from the delivery row rather than from the stored body, so a callback can never
disagree with the run you read back afterwards. A cancel that won the race is reported as the cancel.
### `interrupts` — the question a paused run is asking
A run that hits `interrupt()` settles as `interrupted` and waits for an answer. The question itself is
**not in `values`** — it is the interrupt's payload, which lives on the thread's pending tasks — so a
receiver holding only the callback could not render it, and a question nobody asks is a conversation
that waits forever.
`interrupts` carries it, keyed by the task that raised it, in the same shape
[`GET /threads/{id}`](./threads.md) returns:
```jsonc
{
"run_id": "…",
"status": "interrupted",
"values": { "…": "…" },
"interrupts": { "task-1": [{ "id": "…", "value": { "question": "Refund £40?" } }] },
}
```
The field is **absent entirely** on any other status, so a run that succeeded sends exactly the body it
sent before this existed. Both `values` and `interrupts` count toward
[`max_payload_bytes`](#what-this-stores-and-for-how-long); if the body is over the cap, `values` is replaced by a
truncation marker first and `interrupts` only if that was not enough — you can act on a question
without the state, but not on the state without the question.
**A run that never started executing sends no callback.** A run cancelled while still `pending`, or
one displaced by a [multitask strategy](./runs.md#multitask-what-happens-to-the-run-already-going)
before it began, has nothing to report — so do not treat "a run was created" as "a callback is
owed". Only a run that started does.
Every request also carries three headers, whenever the store records deliveries — which both bundled
drivers do:
| Header | Use |
| --------------------- | ----------------------------------------------------------- |
| `X-Skein-Delivery-Id` | Stable across every retry — **this is your dedup key** |
| `X-Skein-Attempt` | Which attempt this is, counting the first inline one as `1` |
| `X-Skein-Signature` | Present when a signing key is configured — see below |
The one exception is a custom store that doesn't record deliveries — see
[the guarantee](#the-guarantee-why-a-callback-survives-a-crash) below.
## The guarantee: why a callback survives a crash
```mermaid
flowchart LR
R[run settles] --> TX
subgraph TX["one transaction"]
ST[terminal status] --- D[delivery row]
end
TX --> I[inline attempt]
I -->|2xx| Done[done]
I -->|fails| Q[retry schedule]
Q --> Done
class TX accent
```
**The callback is recorded in the same transaction as the run's terminal status.** A crash between
"the run finished" and "someone was told" cannot lose it, because there is no instant where one is
committed and the other is not.
That guarantee comes from the store. It is not the retry policy, and it is not something you
configure — a run that carries a `webhook` owes a callback whether or not you have tuned anything.
skein attempts the first delivery inline, so a healthy receiver hears within milliseconds; only a
failure falls through to the retry schedule.
A [bring-your-own store](./storage.md) that does not implement the `deliveries` repo degrades to the
older best-effort path — one POST, failures logged and swallowed. Both bundled drivers implement it,
so this only bites a store you wrote yourself.
> [!WARNING]
> That fallback POST carries **no `X-Skein-Delivery-Id`, no `X-Skein-Attempt` and no
> `X-Skein-Signature`** — there is no delivery row for those to come from. A receiver that dedupes on
> the header, or rejects unsigned requests, will find nothing to work with. If you implement a custom
> store and want callbacks at all, implement the `deliveries` repo.
The body is **stored**, not re-rendered at send time. It has to be: a stateless `POST /runs` deletes
the thread the server made for it as soon as the run settles, so an hour later there is no run row
and no checkpoint left to render from — and that is exactly the "another service drives skein and
wants the answer back" case webhooks exist for.
Each POST is aborted after `SKEIN_WEBHOOK_TIMEOUT_MS` (default 5s), sized against the 8s shutdown
budget so a slow receiver cannot get the server killed mid-POST. Delivery happens after the thread's
execution lock is released, so a slow target never blocks other runs on that thread — which also
means two callbacks for one thread are not guaranteed to arrive in run order.
## At-least-once, and what your receiver owes
**Delivery is at-least-once, never exactly-once.** A POST that timed out _after_ your receiver
committed is indistinguishable from one that never arrived, so it is retried. Dedupe on
`X-Skein-Delivery-Id`, which is stable across every attempt.
Answer `2xx` only once the work is durably done. Answering early and then failing loses the callback
— the sender heard success and cleared the payload. A non-2xx is how you ask for a retry.
The API remains the source of truth for run state. A callback is a notification, not a ledger.
## Where the retries actually run
Durable stores retain both delivery payloads and scheduling timestamps. A separate delivery queue
changes how retries are scheduled, rather than whether those timestamps survive:
| Deployment | Retry scheduling | Restart behavior |
| ---------------------------- | ----------------------------------------------- | ---------------------------------------------------------------------------- |
| Postgres + Redis | BullMQ delayed jobs and recovery sweep | Waiting jobs survive in Redis; persisted outbox rows recover lost scheduling |
| Postgres or SQLite, no Redis | Polling persisted due times and delivery leases | Waiting retries and abandoned delivery leases survive in the database |
| Memory | Polling the in-memory outbox | Delivery rows and due times are lost with the process |
The polling worker needs no SQLite-specific delivery queue. The host starts and stops it through the
existing protocol runtime lifecycle.
A process killed at exactly the wrong moment can leave a committed delivery with no schedule attached.
Nothing is lost: a background sweep picks it up on its next pass. The practical consequence is that
such a callback arrives late rather than never — worth knowing if you alert on delivery latency.
The default is 12 attempts over roughly 34 minutes, which rides out a rolling deploy with room to
spare. Read that as a **time horizon, not a count** — the delays double, so dropping to 6 attempts is
~31 seconds, which does not survive one redeploy. Tune it under
[`skein.webhooks`](./langgraph-cli-compat.md#webhooks-skeinwebhooks).
## Verify a callback is really from you
Set `SKEIN_WEBHOOK_SECRET` and every callback carries `X-Skein-Signature` in Stripe's format —
`t=<unix seconds>,v1=<hex hmac-sha256>` over `"{t}.{raw body}"` — so a receiver can reuse
verification code it already has. skein ships the verifier too, from a package that installs with no
graph runtime, so a service whose only relationship with skein is _receiving_ callbacks can depend on
it for this and nothing else:
```ts
import { verifySkeinSignature } from "@skein-js/agent-protocol";
const result = verifySkeinSignature({
header: req.headers["x-skein-signature"],
body: rawBody, // the RAW bytes — see below
secrets: [process.env.SKEIN_WEBHOOK_SECRET!],
});
if (!result.ok) return res.status(401).end();
```
It refuses with a `reason` of `missing`, `malformed`, `stale` or `mismatch`, compares digests in
constant time, and tries every secret you hand it.
Because the signature covers the timestamp, a **replay is detectable with no state on your side**: a
captured callback stops verifying once it is outside the tolerance window (5 minutes by default,
`toleranceSeconds`). The check is absolute, so a callback timestamped in the future is refused too.
### `reply` — letting the graph say what to send back
Anything turning a run into an outbound message — a chat reply, a notification, an email — has to know
where the answer lives. Only two parties could decide: your receiver, or the graph. A receiver that
guesses (`state.answer`? `state.messages.at(-1)`? `state.draft`?) stops working the moment you change
the graph's state shape.
So let the graph declare it, on the custom stream:
```ts
import { replyWith } from "@skein-js/agent-protocol";
// inside a node, with LangGraph's StreamWriter in scope
writer(replyWith("Your order ships Tuesday."));
```
It arrives as `reply` in the callback body, and the run must request the custom stream
(`stream_mode: ["values", "custom"]`) for the writer's output to be seen. The field is absent unless
the graph declared one, so nothing changes for a graph that doesn't — and it **adds** to the body
rather than replacing `values`, so a receiver that already reads state keeps working.
Declaring it on the stream rather than in state is deliberate: the reply is a message _about_ the run,
not part of it, and putting it in state would make it a channel every reducer and checkpoint carries
forever. skein captures it into the stored delivery as the run settles, so — unlike anything read off
a live stream — it survives a crash and is replayed verbatim on a retry. Last write wins.
### Verifying somebody else's signature
`verifySkeinSignature` only understands skein's own format. If you are on the _other_ side — checking
a signature from Twilio, Slack, GitHub or Stripe as it arrives at a route of yours — the one piece
worth borrowing rather than writing is the comparison:
```ts
import { equalsConstantTime } from "@skein-js/agent-protocol";
const expected = createHmac("sha1", authToken).update(signable, "utf8").digest("base64");
if (!equalsConstantTime(expected, received)) return res.status(401).end();
```
`===` on a signature is a timing oracle — it returns at the first differing byte, so an attacker who
can send many requests and measure your response recovers a valid signature a byte at a time. The
length check inside is not decoration either: Node's `timingSafeEqual` **throws** on unequal lengths,
so the obvious hand-rolled version turns a forged short signature into a 500 instead of a 401.
### Three things receivers get wrong, in the order they get them wrong
1. **Verifying a re-serialized body.** `JSON.stringify(req.body)` does not reproduce what was signed
— key order, whitespace and number formatting are not guaranteed to round-trip — so _every_
genuine callback fails. In Express: `express.raw({ type: "application/json" })` on that route, or
`express.json({ verify })`.
2. **Not deduping.** See at-least-once above. `X-Skein-Delivery-Id` is the key; `X-Skein-Attempt`
tells you it is a retry.
3. **Answering 2xx before the work is durable.** The sender clears the payload on success. There is
nothing left to resend.
A runnable receiver doing all three correctly — accepting a genuine callback, refusing a forged one
and a replayed one — is `src/webhook-receiver.ts` in
[`examples/express-basic`](https://github.com/skein-js/skein-js/tree/main/examples/express-basic)
(`pnpm webhook-receiver`).
### Rotating the key
Set `SKEIN_WEBHOOK_SECRET` to a comma-separated list, **new key first**: skein signs with the first
and receivers accept any they hold. Remove the old key only once every receiver has the new one — the
other order refuses every callback until they catch up.
> [!WARNING]
> If you inject your own `webhookDispatcher`, it must forward `attempt.headers` and send
> `attempt.body` verbatim, or callbacks go out unsigned while your config says they are signed. skein
> warns at startup when it sees both a signing key and a custom dispatcher.
## See what a callback did, and replay it
When a receiver was down for longer than the retry horizon, the delivery is `dead` — and still on
disk, which is what makes a by-hand replay possible.
| Method | Path | Notes |
| ------ | -------------------------------------------------------------------- | ----------------------------------- |
| `GET` | `/threads/{thread_id}/runs/{run_id}/deliveries` | `?status=` · `?limit=` · `?offset=` |
| `POST` | `/threads/{thread_id}/runs/{run_id}/deliveries/{delivery_id}/replay` | Makes it due immediately |
```bash
curl "$API/threads/$TID/runs/$RID/deliveries?status=dead"
curl -X POST "$API/threads/$TID/runs/$RID/deliveries/$DID/replay"
```
Each row reports its `status` (`pending` · `delivering` · `delivered` · `dead`), `attempt`,
`last_error`, `next_attempt_at` and `payload_truncated`.
The list reports a boolean **`replayable`** rather than the payload itself. The payload is up to
256 KiB of the run's final state _per row_, and this is a list — returning it would make an
operator's "what failed?" the heaviest response the server can produce, for a field they did not ask
for. The question a replay turns on is only whether there is anything left to send. Read the state
from the run.
Replay makes a delivery due immediately. It **409s** on one that already succeeded (its payload was
cleared on delivery, so there is nothing to resend) and **404s** on a delivery id that does not belong
to the run in the path — so an id cannot be used to reach a callback on a run you have no access to.
Both endpoints sit in the `runs` route group, so `http.disable_runs` and your existing
`@auth.on.threads` handler already cover them; no new switch, no new resource.
The console shows deliveries on each run page, including failed attempts and a confirmed Replay action.
## What this stores, and for how long
A retry has to have something to send, so **the callback body is stored** — including the run's final
`values`. That is a new at-rest copy of whatever your graph produced, in the `deliveries` table, and
it is worth classifying deliberately if that state carries personal or regulated data:
- **On success it is cleared immediately**, so steady-state storage is in-flight deliveries, not every
delivery ever made.
- **A `dead` delivery keeps it** for `retain_hours` (default 24) — that is what makes replay possible,
and it means a callback that never landed leaves the run's final state on disk for a day.
- **It is not encrypted by skein.** Use your database's encryption at rest; the column is ordinary
`jsonb`.
- **Deleting a thread or run does not erase its deliveries** — unlike idempotency records, which are
erased explicitly. Lower `retain_hours` if that matters to you.
Payloads are capped at `max_payload_bytes` (default 256 KiB). Over the cap, `values` is replaced by a
truncation marker **inside the signed body** — so a receiver is told it is looking at a truncated
state rather than left to infer it — and the row's `payload_truncated` records it. Raise the cap, or
read the state back from the API using the `run_id` in the callback.
`values` and `interrupts` are the only fields ever replaced, because they are the only unbounded ones.
When both are present `values` goes first: an interrupted run's question is what makes the callback
actionable at all, so it is the last thing dropped.
skein **redacts the webhook URL's path in logs**, keeping only the scheme and host: for Slack, Discord
and Teams the path _is_ the credential, and a failing delivery would otherwise write it into your log
sink on every attempt.
## Accepting webhook URLs from untrusted callers
`webhook` is a caller-supplied URL, so it is a server-side request to a target they chose — and
retrying it turns a one-shot SSRF probe into a repeated one. Set
[`skein.webhooks.allowed_hosts`](./langgraph-cli-compat.md#webhooks-skeinwebhooks) if you accept run
creates from clients you do not control. It is off by default so that upgrading cannot start dropping
your own callbacks; a refused host is recorded `dead` with the reason rather than silently skipped.
Set `require_https: true` if callbacks leave your network. Plaintext is permitted by default (an
internal receiver is legitimate), but retries mean the body crosses the wire up to `max_attempts`
times rather than once — a dozen exposures rather than one.
## Configuration
Every knob — `retries.max_attempts`, `retries.initial_delay_ms`, `secret`, `max_payload_bytes`,
`retain_hours`, `allowed_hosts`, `require_https` — is documented once, with its default and its
reasoning, in [`skein.webhooks`](./langgraph-cli-compat.md#webhooks-skeinwebhooks).
Two environment variables sit outside that block: `SKEIN_WEBHOOK_SECRET` (which takes precedence over
a `secret` in the config file) and `SKEIN_WEBHOOK_TIMEOUT_MS`.
> [!WARNING]
> skein does **not** expand `${VAR}` anywhere in `langgraph.json`. `"secret": "${SKEIN_WEBHOOK_SECRET}"`
> signs every callback with that literal 23-character string. Use the environment variable.
## See also
- [Runs](./runs.md) — the run-creation fields a callback reports on
- [Errors & logging](./errors-and-logging.md) — what a failed run puts in the callback body
- [Agent Protocol](./agent-protocol.md) — the full endpoint inventory
- [Background jobs](./background-jobs.md) — the fire-and-forget shape webhooks were built for
---
<!-- source: docs/channels.md -->
# Workflows and channels — connect LangGraph to external systems
Workflows and channels solve different halves of the same problem. A **workflow** defines what should
happen after an event arrives. A **channel** is the provider integration: it defines how that event
enters the workflow and how an outcome reaches an external system. Together they form one reliable
path:
```text
channel source → LangGraph workflow → channel destination
```
Skein owns the channel boundary: authentication, deduplication, thread and run coordination, and
durable delivery. LangGraph owns the workflow in the middle: state, decisions, tools, branching, and
`interrupt()`.
LangGraph deliberately doesn't own integrations to WhatsApp, email, Slack, GitHub, or your internal
systems. It gives you the orchestration engine. Skein channels give that engine authenticated sources
and durable destinations. This is the part Skein adds: the integration lifecycle that turns graph
logic into a workflow people and systems can actually participate in.
## What “workflow” means here
A workflow is the business process expressed by your LangGraph graph and carried across its persisted
thread state. It is more than forwarding one message to another provider. It can inspect an event,
call tools, branch, fan out work, ask a person for a decision, pause for hours or days, resume from a
later event, and finally choose what should happen next.
For example, “send this email to WhatsApp” is message forwarding. “Validate a refund request, collect
the required approvals over WhatsApp, wait for every response, decide the refund, and notify the
customer by email” is a workflow.
Workflow is not a new Skein resource or API. It is the LangGraph graph you already write, plus the
state LangGraph persists in a thread. A channel is the adapter and Skein pipeline around that graph:
its source verifies and translates provider events; its destination translates and delivers graph
outcomes.
```text
channel source LangGraph workflow channel destination
email / WhatsApp / webhook → state + decisions → email / WhatsApp / provider API
tools + branching
interrupt / resume
```
## What this looks like in practice
The useful unit is the whole process, not a provider-to-provider pipe:
| Real-world workflow | Channel source | LangGraph does | Channel destination |
| ------------------------------ | ------------------------- | ------------------------------------------------------------- | --------------------------------------- |
| **Answer an order question** | Customer WhatsApp message | Looks up the order, decides whether to escalate, drafts reply | Reply to the same WhatsApp chat |
| **Approve a refund** | Customer email | Validates the claim, gathers approvals, pauses and resumes | WhatsApp approvers, then email customer |
| **Handle a failed deploy** | GitHub deployment webhook | Inspects the failure, classifies severity, chooses escalation | Alert the on-call team in Slack |
| **Resolve an order exception** | ERP order webhook | Checks inventory, branches by risk, waits for warehouse input | Notify sales or email the customer |
In every case, the channel handles provider truth—signatures, event IDs, identities, payloads, and
delivery—while the workflow handles business truth: what the event means and what should happen next.
## Why LangGraph and channels fit together
LangGraph doesn't try to own provider integrations, and external systems fail differently from
business logic. A provider webhook needs authentication, a fast acknowledgement, retry
deduplication, and stable identity mapping. The business process needs state, branching, tool calls,
and human-in-the-loop pauses. Outbound delivery needs recipient policy, provider-specific validation,
retries, and idempotency.
Keeping those responsibilities separate gives each layer one job:
| Layer | Owns |
| ----------------------- | -------------------------------------------------------------------- |
| **Channel source** | Verify, parse, deduplicate, identify the thread, and start or resume |
| **LangGraph workflow** | State, decisions, tools, branching, fan-out, and `interrupt()` |
| **Channel destination** | Validate and deliver the declared outcome through the durable outbox |
This keeps provider code thin and keeps transport concerns out of the graph. The graph declares an
outcome; the destination adapter performs the external side effect after the run settles.
## When to use it
Use a workflow with channels when an external provider event starts or resumes a process whose state
belongs in LangGraph, and the result must return to a provider reliably. The channel shape depends on
the workflow:
- Use a **coupled channel** when the source already determines where the outcome belongs, such as a
WhatsApp question followed by a WhatsApp answer.
- Use **decoupled channel delivery** when the workflow must choose another provider, recipient, or
route, such as an email request that triggers WhatsApp approvals and an email decision.
- Use the Agent Protocol run API directly when your own application already owns ingress and only
needs to invoke or stream a graph; a channel adds no value in that path.
In both shapes, the workflow stays provider-independent. It receives ordinary graph input and
declares an outcome; the surrounding channels translate between that data and provider payloads.
You write thin provider adapters. Skein handles retry deduplication, external-identity-to-thread
mapping, start-versus-resume decisions, and durable delivery consistently across channels.
> A chat channel is a `Channel` whose reply target happens to be the sender. A GitHub webhook and a
> Stripe dispute go through the identical pipeline without that property — which is why these types
> talk about _events_, and why you will not find `from`, `to`, `body` or `typing` in any of them.
Entirely optional. A deployment that configures no channel does not install the package, serves no
channel routes, and cannot tell the feature exists.
## What Skein removes from each workflow connection
Connecting a workflow to a phone number by hand means writing signature verification, deduplication
for retried deliveries, a mapping from `whatsapp:+254…` to a thread, a branch on whether that thread
is already waiting on a human, payload mapping in both directions, and a reply path that does not
double-send.
Only two of those are about the provider. The rest are identical for every integration anyone will
ever write — and their failures are silent ones: a double reply, a lost reply, a question nobody was
ever asked.
## Quick start
```bash
pnpm add @skein-js/channels
```
Write the provider adapter and bind its source route to a graph:
**`src/twilio-channel.ts`**
```ts
import { equalsConstantTime } from "@skein-js/agent-protocol";
import type { Channel } from "@skein-js/channels";
export const channel: Channel = {
name: "twilio",
verify(request) {
const expected = sign(process.env.TWILIO_AUTH_TOKEN!, request.url.href, request.form());
if (!equalsConstantTime(expected, request.headers["x-twilio-signature"] ?? "")) return false;
return { identity: `channel:twilio:${request.form()["From"]}` };
},
parseEvent(request) {
const message = request.form();
if (!message["Body"]) return { kind: "ignore" };
return {
kind: "event",
event: {
threadKey: message["From"]!,
idempotencyKey: message["MessageSid"]!,
replyTo: { to: message["From"] },
input: { messages: [{ role: "human", content: message["Body"] }] },
resumeWith: message["Body"],
},
};
},
async deliver(outcome, target) {
const { to } = target as { to: string };
if (outcome.reply !== undefined) await sendWhatsApp(to, String(outcome.reply));
},
};
```
**`langgraph.json`**
```jsonc
{
"graphs": { "support": "./src/support.ts:graph" },
"skein": {
"channels": {
"twilio": {
"path": "./src/twilio-channel.ts:channel",
"assistant": "support",
"public_url": "https://api.example.com",
},
},
},
}
```
That serves `POST /channels/twilio`. Point your provider's webhook at it.
A complete, runnable version — offline, no Twilio account —
is [`examples/whatsapp-agent`](https://github.com/skein-js/skein-js/tree/main/examples/whatsapp-agent).
### Let a workflow route between providers
The source that starts a workflow does not have to receive the result. The
[`decoupled-delivery`](https://github.com/skein-js/skein-js/tree/main/examples/decoupled-delivery)
example sends an email into one LangGraph workflow, routes approval requests to WhatsApp, collects
parallel authenticated approvals with LangGraph `interrupt()`, then sends the result by email. Its
runnable refund demo uses Skein's source, destination, auth, deduplication, thread/run and delivery
paths with offline provider fakes.
Skein exposes this as a small composition layer rather than separate provider hierarchies:
```ts
import {
composeRoutedChannel,
declareChannelDestinationDelivery,
type ChannelDestinationDelivery,
} from "@skein-js/channels";
const destinations = new Map([
[
"whatsapp",
async (delivery: ChannelDestinationDelivery) => {
const message = whatsappSchema.parse(delivery);
await sendWhatsApp(message, { idempotencyKey: delivery.runId });
},
],
]);
export const channel = composeRoutedChannel(emailSource, destinations);
```
### How LangGraph fills the middle and selects the destination
LangGraph passes a custom-stream writer to every node in `LangGraphRunnableConfig`. The Skein helper
uses that existing writer to declare the result of the workflow; the node does not call WhatsApp or
email itself:
```ts
import {
Annotation,
END,
START,
StateGraph,
type LangGraphRunnableConfig,
} from "@langchain/langgraph";
import { declareChannelDestinationDelivery } from "@skein-js/channels";
const RelayState = Annotation.Root({
source: Annotation<"email" | "whatsapp">,
sender: Annotation<string>,
whatsappNumber: Annotation<string | undefined>,
subject: Annotation<string>,
});
function routeMessage(
state: typeof RelayState.State,
config: LangGraphRunnableConfig,
): Partial<typeof RelayState.State> {
if (state.source === "email" && state.whatsappNumber) {
declareChannelDestinationDelivery(config.writer, {
destination: "whatsapp",
target: { to: state.whatsappNumber },
payload: { body: `Important email from ${state.sender}: ${state.subject}` },
});
} else {
declareChannelDestinationDelivery(config.writer, {
destination: "email",
target: { to: state.sender },
payload: { subject: "Assistant update", body: state.subject },
});
}
// The destination declaration is an output instruction, not graph state.
return {};
}
export const graph = new StateGraph(RelayState)
.addNode("route-message", routeMessage)
.addEdge(START, "route-message")
.addEdge("route-message", END)
.compile();
```
The workflow connection is deliberately small:
1. Skein's source converts email or WhatsApp into ordinary LangGraph input.
2. LangGraph performs extraction, classification, branching, `Send`, and `interrupt()` as usual.
3. The graph writes one explicit destination declaration through `config.writer`.
4. When the run settles, Skein resolves that declaration and invokes the allowlisted destination
through its durable outbox.
For approvals, keep using LangGraph's `interrupt()`; no Skein approval abstraction is needed. A node
can declare the WhatsApp approval request before it interrupts, and declare the final email result
after `interrupt()` returns on resume. The complete parallel HR/Manager/Finance implementation is
[`examples/decoupled-delivery/src/relay-graph.ts`](https://github.com/skein-js/skein-js/blob/main/examples/decoupled-delivery/src/relay-graph.ts).
The map allowlists destination adapters and remains application-owned, along with provider credentials
and routing policy. It does not authorize recipients or operations: derive those from trusted workflow
data and enforce tenant/recipient policy before sending, rather than treating an LLM-selected or
user-supplied target as permission. Only `declareChannelDestinationDelivery` triggers routing;
ordinary `replyWith` values and inferred interrupt/AI-message replies are not dispatched by a routed
channel. Construct declarations through the helper, never its internal persisted marker.
`target` and `payload` must be plain JSON values so the outbox can persist them. They remain `unknown`
to the destination, which must validate its provider boundary—Zod is used above for exactly that.
Unknown or malformed declarations and thrown callbacks fail the outbox attempt visibly and are
retried. Delivery is at-least-once, so pass a stable key such as `runId` to providers that support
idempotency. One run still owns one outbox retry unit; aggregate fan-out does not create independent
delivery rows.
This helper is for channel-ingested runs. A poller using the Agent Protocol run API still uses its own
HTTP callback receiver. Existing `Channel` implementations with `deliver` continue to work unchanged.
Configured route keys and explicit channel-name aliases must be unique; collisions now fail at boot
instead of sharing thread identity or misrouting a callback.
For copy-first workflow implementations, compare the practical
[WhatsApp → order lookup → WhatsApp recipe](./recipes/coupled-channel.md) with the
[customer refund email → Finance WhatsApp approval → customer email recipe](./recipes/decoupled-channel-delivery.md).
## The pipeline
```text
POST /channels/twilio
→ verify() your signature check → a principal, or 401
→ authorize() your Auth block, exactly as on every run-creating route
→ parseEvent() ignore → 204 · respond → that response · event → on
→ dedup the provider's event id, claimed over the raw bytes
→ thread a deterministic id, get-or-create
→ start | resume decided atomically against the thread's real status
→ 2xx after enqueueing, never after the run finishes
├→ onSignal() progress, best-effort
└→ deliver() the answer, durably
```
Two properties of that shape are worth internalising before you write anything.
**The acknowledgement is early, always.** Slack retries after three seconds and shows the user an
error; Twilio times out comparably. Your `parseEvent` returns, the run is enqueued, and skein answers
2xx — it never waits for the graph. That is why progress needs `onSignal` and why the answer needs
`deliver`: by the time either has something to say, the HTTP request is long gone.
**`verify` runs before anything is parsed.** A signature covers the bytes as sent, so any middleware
that parses first destroys the thing you need to check. skein hands you the body as text for exactly
this reason.
## Implementing `verify`
```ts
verify(request: InboundRequest): Promise<ChannelPrincipal | false> | ChannelPrincipal | false;
```
Return a **principal**, or `false` for a 401.
**Why a principal and not a boolean.** A provider's signature _is_ an authentication scheme; it just
is not a bearer token. Producing an identity lets the message flow through the `Auth` block you
already configured — `@auth.on.threads` handlers see it, ownership filters apply, multi-tenancy works
— instead of needing a route that bypasses authorization to create runs.
```ts
return {
identity: `channel:twilio:${from}`, // derived from provider-verified data only
permissions: ["threads:write"], // optional; becomes AuthContext.scopes
metadata: { tenant }, // optional; merged into the user object
};
```
Build `identity` **only from data the signature covered**. A value the caller could have set freely
is not an identity, and everything downstream — ownership filters, per-tenant scoping — inherits
whatever trust you put in it here.
**`verify` is required.** A provider with no signature scheme must still yield a principal some other
way: a secret path segment, a shared-secret query parameter, basic auth. Weaker is a decision your
deployment gets to make; absent is not, because the route creates runs.
### `InboundRequest`
| Member | Notes |
| --------- | --------------------------------------------------------------------------- |
| `method` | The HTTP method |
| `url` | A `URL`. The **public** one — see below |
| `headers` | Lower-cased keys, whatever the transport |
| `text()` | The body exactly as sent |
| `form()` | Decoded `application/x-www-form-urlencoded` |
| `json()` | Parsed JSON. Throws on a malformed body, so guard it if the provider varies |
The body views are lazy and cached, so reading `form()` in both `verify` and `parseEvent` parses once.
**`url` is the public URL, and that matters more than it looks.** Twilio signs the full request URL,
so verification depends on knowing what the provider saw — and behind a proxy the server cannot know
it. The only inputs are `Host` and `X-Forwarded-Proto`, both attacker-controlled. So skein takes the
origin from `public_url` in your config and keeps the router's own path. Set it in production, or
signature verification will fail in a way that looks like every request being forged.
### Comparing signatures
```ts
import { equalsConstantTime } from "@skein-js/agent-protocol";
```
Not `===`. It returns at the first differing byte, so an attacker who can send many requests and
measure your response recovers a valid signature a byte at a time. The length check inside is not
decoration either: Node's `timingSafeEqual` **throws** on unequal lengths, so the obvious hand-rolled
version turns a forged short signature into a 500 instead of a 401.
## Implementing `parseEvent`
```ts
parseEvent(request: InboundRequest): Promise<ChannelOutcome> | ChannelOutcome;
```
Three outcomes, and you will need all three.
### `{ kind: "event", event }` — do something
```ts
return {
kind: "event",
event: {
threadKey: message.From, // which conversation
input: { messages: [...] }, // what a fresh turn runs with
resumeWith: message.Body, // what an interrupt gets, if different
idempotencyKey: message.MessageSid, // the provider's own event id
replyTo: { to: message.From }, // opaque; handed back to deliver/onSignal
onExisting: "resume", // default
assistantId: "triage", // only if allowed_assistants lists it
metadata: { locale: message.Locale }, // stamped on the thread
},
};
```
| Field | Required | Notes |
| ---------------- | -------- | --------------------------------------------------------------------------- |
| `threadKey` | ✅ | Stable external identity. Hashed into a thread id |
| `input` | ✅ | Graph input for a fresh turn |
| `threadId` | | Choose the thread id yourself; skips the derivation entirely |
| `resumeWith` | | What `interrupt()` receives. Defaults to `input` — usually wrong, see below |
| `idempotencyKey` | | No key, no dedup. See [Retries](#retries-and-idempotency) |
| `replyTo` | | Opaque to skein. Omit it and no reply is delivered |
| `onExisting` | | `resume` (default) · `enqueue` · `interrupt` · `reject` |
| `assistantId` | | Rejected unless the deployment listed it in `allowed_assistants` |
| `metadata` | | Merged onto the thread's metadata |
### `{ kind: "ignore" }` — acknowledge, do nothing
Answers 204 and starts no run. **Use it liberally.** Slack redelivers your own bot's messages, and a
channel that cannot cheaply say "not interesting" ends up answering itself in a loop. Delivery
receipts, read receipts, reactions, media-only messages and bot echoes all belong here.
```ts
if (payload.bot_id) return { kind: "ignore" };
```
### `{ kind: "respond", status, body }` — answer this request directly
No run, your response. Required by real providers, not a convenience:
```ts
// Slack's very first request is a challenge that must be echoed back.
if (payload.type === "url_verification") {
return { kind: "respond", status: 200, body: { challenge: payload.challenge } };
}
// A slash command wants an immediate ephemeral acknowledgement.
return { kind: "respond", status: 200, body: { response_type: "ephemeral", text: "On it…" } };
```
## Retries and idempotency
Providers retry. Twilio retries on any non-2xx and on a timeout; Slack retries three times. Without
dedup the customer gets two answers.
Set `idempotencyKey` to **the provider's own event id** — `MessageSid`, Slack's `event_id`, GitHub's
`X-GitHub-Delivery`. A retry carries the same one, and skein replays the first response instead of
starting a second run.
Three things worth knowing about how it behaves:
- **The fingerprint is the raw inbound bytes**, not anything derived. A retry that correctly re-reads
the thread and resumes instead of starting sends the same key with a _different_ run body — and
fingerprinting that derived body would refuse the retry precisely because it did the right thing.
- **Same key, different bytes → 422.** Two genuinely different events sharing an id is a provider bug
or a forgery; replaying would tell the sender its message was handled while silently dropping it.
- **No key means no protection**, deliberately. Without a provider-assigned id nothing identifies two
deliveries as the same event, and deriving one from the body would collide when a customer
legitimately sends "yes" twice.
## Conversations that wait for a human
This is the case channels exist for, and the one every hand-written integration gets wrong.
`interrupt()` parks a thread until someone answers. The run that parked it settles as `interrupted`,
which is a **terminal** status — so the thread holds no inflight run, no `multitask_strategy` guards
it, and a plain start succeeds and discards the pending question. Silently.
A channel does the right thing by default: a message arriving on a paused thread **resumes** it. The
decision is a [precondition on the create](./runs.md#dont-start-a-run-on-a-thread-thats-waiting-for-a-human),
settled inside the driver, so two replicas cannot both resume.
### Set `resumeWith` — you almost certainly need it
`input` is what a _fresh turn_ takes. For a message-shaped graph that is an envelope:
`{ messages: [{ role: "human", content: "yes" }] }`. But `interrupt()` returns whatever the node asked
for — usually a scalar the graph author chose:
```ts
const answer = interrupt("Approve this refund?");
if (/^y/i.test(String(answer))) { … }
```
Hand that node the envelope and `String(...)` gives `"[object Object]"`, which does not match — so the
graph reads a "yes" as a "no", and nothing errors. skein will not guess between the two shapes, so
say which is which:
```ts
input: { messages: [{ role: "human", content: body }] },
resumeWith: body,
```
### Choosing `onExisting`
| Value | What happens | When |
| ----------- | -------------------------------------------------------- | --------------------------------------------------- |
| `resume` | Answers a pending interrupt; otherwise queues a new turn | **Default.** Chat, email, anything conversational |
| `enqueue` | Always a new turn, behind whatever is running | A feed of events that each deserve their own run |
| `interrupt` | Cancels the run in flight and starts this one | "Actually, stop — do this instead" |
| `reject` | 409 unless the thread is genuinely free | An event that is only valid on an idle conversation |
Under `resume`, a message arriving while the agent is still thinking **queues** rather than being
refused. The server's default `multitask_strategy` is `reject`, which for Twilio would render as a
failed message for having typed twice; a conversation wants ordering.
When the answer is not an answer — _"actually, never mind, what's my balance?"_ — skein still resumes
and the **graph** decides whether to re-ask, pivot or abandon, because the graph holds the question.
If you would rather have a blunter rule, use `enqueue`.
## Delivering the reply
```ts
deliver?(outcome: RunOutcomeForChannel, target: ReplyTarget): Promise<void>;
```
Optional — a GitHub channel that comments from inside the graph owes no callback. Omit it and no
reply is delivered.
**It runs inside skein's delivery outbox**, which is where all its guarantees come from. The delivery
row is written in the run's finalize transaction, so a crash between "the run succeeded" and "the
customer was told" cannot lose the answer. A **throw is a failed attempt**: retried with exponential
backoff, recorded, and replayable from the [delivery routes](./webhooks.md#see-what-a-callback-did-and-replay-it).
So let it throw — swallowing an error here is how an answer gets silently dropped.
```ts
async deliver(outcome, target) {
if (outcome.reply === undefined) return; // nothing to say
const { to } = target as { to: string };
await sendWhatsApp(to, String(outcome.reply)); // throws → retried
}
```
### What `outcome.reply` is
Resolved by skein, in order:
1. **What the graph declared**, via `replyWith` on the custom stream.
2. **The last AI message** in `values.messages` — LangGraph's `MessagesAnnotation` convention, so an
ordinary chat agent works with no graph changes at all.
3. **The interrupt's question**, when the run is `interrupted`.
4. Otherwise `undefined`.
Resolved centrally rather than per channel on purpose: a channel that guessed at `state.answer` versus
`state.draft` would stop being reusable across graphs, which is the premise the whole plugin surface
rests on. For a graph whose state is not message-shaped, declare it:
```ts
import { replyWith } from "@skein-js/agent-protocol";
// inside a node, with LangGraph's StreamWriter in scope
writer(replyWith("Your order ships Tuesday."));
```
**A failed run says nothing.** `error`, `timeout` and `cancelled` resolve to no reply, because whether
an end user hears "something went wrong" is a product decision and leaking internals to a phone number
is the wrong default. Send your own message from `deliver` if you want one.
## Progress signals
```ts
signals: { kinds: ["progress"], keepaliveMs: 10_000 },
async onSignal(signal, target) { … }
```
Because the acknowledgement is early, progress cannot ride the response. `onSignal` is how a typing
indicator gets sent while the graph works.
**Its guarantees are deliberately the opposite of `deliver`'s**: best-effort, at most once, never
retried, never blocking the run, dropped on any error. Retrying a "typing" indicator four minutes
late is nonsense, and a slow provider must never be why the acknowledgement times out. That is why
they are two methods rather than one with a flag.
- `progress` — fires immediately, then once per frame the run produces.
- `keepalive` — fires on `keepaliveMs`, for providers whose indicator expires (most do, in seconds).
Declaring a subscription is what lets skein pick the cheapest stream that satisfies it. A channel that
asks for nothing costs exactly what an API run costs; one that asks for `progress` gets node-level
updates and **never** pays for token streaming it would only throw away.
## Addressing a conversation from outside
Thread ids are derived, and the derivation is exported — which is what makes hashing acceptable rather
than merely private:
```ts
import { threadIdForChannelKey } from "@skein-js/channels";
const threadId = threadIdForChannelKey("twilio", "whatsapp:+254712345678");
await client.threads.get(threadId);
```
The phone number never reaches a primary key, an index or a backup. The raw key **is** stamped into
thread metadata, so a search answers the same question from the other side — which is the shape a
GDPR erasure request actually arrives in:
```ts
await client.threads.search({ metadata: { skein_thread_key: "whatsapp:+254712345678" } });
```
### Ending a conversation and starting a new one
Two different things people mean by this, and they need different answers.
**"Forget everything and start over."** Delete the thread. The next message from that number derives
the same id, finds nothing there, and begins fresh:
```ts
await client.threads.delete(threadIdForChannelKey("twilio", "whatsapp:+254712345678"));
```
**"Start a new conversation, but keep the old one."** Then the thread key is not just the phone number
— it is the phone number _and_ which conversation. `threadKey` is whatever your channel says, so put
the session in it:
```ts
// A ticket, a billing period, a counter you keep — whatever "a conversation" means to you.
threadKey: `${message.From}:${await currentSessionFor(message.From)}`;
```
Old conversations stay readable, each under its own thread, and a search on `skein_thread_key` still
finds them because the raw key is what gets stamped.
A channel that already has its own conversation ids returns `threadId` on the event instead, and skein
does no transformation at all.
## Configuration
| Key | Meaning |
| -------------------- | --------------------------------------------------------------------- |
| `path` | `path:export`, or a package name |
| `assistant` | **Required.** Which graph these events run |
| `allowed_assistants` | Graphs the channel may route to itself. Omitted means it cannot route |
| `public_url` | Your externally reachable origin, when signatures cover the URL |
**`assistant` is required and not defaultable.** The binding is deployment knowledge: a community
Twilio adapter has no business knowing you named your graph `support`. It takes a graph name or an
assistant UUID, resolving exactly as it does for [crons](./crons.md).
**`allowed_assistants` is opt-in and bounded** for a sharper reason. A channel is an npm package you
installed; without a bound, an `assistantId` derived from untrusted input could reach any graph you
serve. Omit the key and the channel cannot route at all.
Everything is validated **at boot**: a missing `assistant`, one naming a graph that does not exist, an
`allowed_assistants` entry outside your graphs, a `path` whose export is not a channel. Discovering
any of those when the first customer texts is the failure this avoids.
## Testing a channel
The quickest end-to-end check is the console's **Channels** tab. It shows the effective routes and
assistant allowlists that passed boot validation. Copy an inbound path, send a fixture or provider
event, then open the resulting thread and run: the run's **Deliveries** panel shows attempts, retry
timing, errors, and a confirmed replay action. Sensitive webhook paths and opaque reply targets are
redacted in the UI.
The same inventory is available as authenticated `GET /channels`; it is mounted only when at least
one channel is configured and exposes no module paths, public URLs, credentials or raw config.
A channel is a plain object, so `verify` and `parseEvent` are unit-testable with no server:
```ts
const request = buildInboundRequest({
method: "POST",
url: "https://api.example.com/channels/twilio",
headers: { "x-twilio-signature": signature },
text: "From=whatsapp%3A%2B254&Body=hi&MessageSid=SM-1",
});
expect(channel.verify(request)).toEqual({ identity: "channel:twilio:whatsapp:+254" });
```
For the whole path, boot a runtime from a `langgraph.json` and dispatch into the handler — no HTTP
server, no ports:
```ts
const resolved = await resolveProtocolRuntime({ config: "./langgraph.json" });
const response = await resolved.runtime.handlers.handleInboundEvent(request);
expect(response.status).toBe(202);
```
Worth covering, because each has shipped broken in real integrations: a forged signature is refused;
a retried delivery produces exactly one run; an interrupt is resumed rather than trampled; a delivery
receipt starts no run. [`examples/whatsapp-agent`](https://github.com/skein-js/skein-js/tree/main/examples/whatsapp-agent)
does all four offline.
## Things that will bite you
- **Mount order, if you also hand-roll routes.** `skeinRouter` installs a JSON body parser for
everything routed into it, so a raw-body route registered _after_ it sees an empty body and fails to
verify — a 401 that looks like a forged request. Channel routes are handled for you; your own are
not.
- **`public_url` behind a proxy.** Without it, URL-signing providers fail every request.
- **Forgetting `resumeWith`.** The graph reads a "yes" as a "no" and nothing errors.
- **Swallowing errors in `deliver`.** A throw is what buys you the retry.
- **Returning `false` from `verify` for a _parse_ failure.** That reports a forged request when the
truth is a malformed one; prefer `{ kind: "respond", status: 400 }` from `parseEvent`.
- **Sending the reply from `parseEvent`.** It will be sent again on every retry. The outbox exists so
that it is not your problem.
## What is not here yet
- **First-party channels.** There are none: you write the file. Shipping one would make skein
responsible for the correctness of someone else's signature scheme, so a flaw becomes a CVE here
rather than in your code — a real cost that has to be earned.
- **A channel conformance suite**, so a community channel can prove it rejects forged signatures,
stale timestamps and duplicate ids without hand review.
- **Chunked streaming of the answer** — send it in pieces, crash halfway, and the outbox replays a
message the user already partly received. Needs provider-side editing (Slack can, WhatsApp cannot).
- **Attachments.** Carry the provider's media URL in `metadata` and fetch it from the graph.
- **Polled sources** (IMAP, queue consumers). The pipeline is entered by an inbound HTTP request, and
there is nothing to enter it with when you poll — that is closer to [crons](./crons.md).
## See also
- [webhooks.md](./webhooks.md) — the durable delivery a channel's reply rides on
- [human-in-the-loop.md](./human-in-the-loop.md) — `interrupt()`, and why a reply hours later must
resume rather than restart
- [runs.md](./runs.md#dont-start-a-run-on-a-thread-thats-waiting-for-a-human) — `if_thread_status`,
the precondition underneath `onExisting`
- [agent-protocol.md](./agent-protocol.md#authentication--authorization) — the `Auth` block a
verified principal flows through
---
<!-- source: docs/streaming.md -->
# Streaming (SSE)
Streaming is what makes an agent UI feel alive: tokens appear as the model writes them, model
**thinking** streams into a collapsible panel, tool calls and their **structured results** show up as
they happen, and a paused run's **interrupt** surfaces for approval. skein-js delivers all of it over
one transport — **Server-Sent Events (SSE)** — so the standard clients ([`useStream`](./react-sdk.md),
the vanilla SDK, Agent Chat UI) render a rich conversation against a skein-js server with only a URL
change. The flagship [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) example wires the full experience end to end;
[`react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream) is the minimal harness.
Under the hood, skein-js maps LangGraph.js **stream modes** onto Agent Protocol SSE. This one
transport powers the `/runs/stream` endpoint, joining an in-flight run (`/runs/{id}/stream`), and
thread-scoped streaming (`/threads/{id}/stream`).
Reference: LangGraph streaming — <https://docs.langchain.com/oss/javascript/langgraph/streaming>
## LangGraph.js stream modes
A `CompiledStateGraph.stream(input, { streamMode })` can emit any combination of:
| Mode | Emits |
| ---------------- | ------------------------------------------------- |
| `values` | Full state after each step |
| `updates` | State deltas per node (**default**) |
| `messages` | Complete messages |
| `messages-tuple` | Message chunk + metadata tuples (token streaming) |
| `custom` | User-emitted custom events |
| `events` | Fine-grained execution events |
| `debug` | Detailed debug info |
Multiple modes can be requested at once; skein-js preserves that. When `events` is among the
requested modes the run engine drives the graph via LangGraph's `streamEvents` (v2) and emits each
event as an `events` frame (co-requested modes like `values` still stream alongside); otherwise it
uses `graph.stream`.
## Mapping to Agent Protocol SSE
Each LangGraph stream item becomes an SSE frame:
```
event: <mode> # e.g. messages, updates, values, custom
id: <monotonic-seq> # per-run sequence for replay/reconnect
data: <json payload>
```
- **Event id sequencing** — each run assigns monotonically increasing ids so a reconnecting
client can resume via `Last-Event-ID` (replay support; full replay buffering is iterative).
- **Terminal frames** — a final `event: end` (or `error`) closes the stream with the run's
status. An `error` frame's payload is a `RunError` — `{ error, message, name, cause?, errors? }`,
plus `stack` when the server sets `exposeErrorStacks`. See
[errors-and-logging.md](./errors-and-logging.md).
- **Transport ownership** — `@skein-js/core` produces an async iterator of normalized frames;
each framework adapter writes them as `text/event-stream` (Express `res.write`, Fastify
reply stream, etc.). The core stays framework-agnostic.
- **Idle heartbeats** — a stream that goes 15s without a frame gets a `: heartbeat` line. An SSE
comment is spec-defined as ignorable, so no client dispatches it as an event; it exists for the
machinery in between. Without it, a graph thinking for a minute before its first token looks
identical to a dead socket — nginx, ALB, and Cloud Run all default to closing an idle connection
around 60s, and the SDK's `stream_idle_reconnect: "auto"` has nothing to distinguish the two either.
Tune it with `toSseEvents(..., { heartbeatMs })`; `0` writes only real frames.
## Subgraphs and subagents
A graph that runs another graph — a compiled subgraph added as a node, or a graph invoked from a
tool, which is how [Deep Agents](./recipes/deep-agents.md) runs a subagent — streams only its own
frames by default. Set `stream_subgraphs: true` on the run (`streamSubgraphs: true` in the JS SDK) to
stream the inner graphs' frames too. Each is named after the mode and the namespace it came from:
```
event: updates # the run's own graph
event: updates|tools:6d1f… # a subagent run from a tool call
event: messages|tools:6d1f…|model_request:9a2c… # that subagent's model tokens
```
These are LangGraph Platform's names; `@langchain/langgraph-sdk` splits on `|` to tell supervisor
and subagent streams apart. Joining a run (`Last-Event-ID` included) replays them unchanged, and a
background or cron run keeps the flag.
Without the flag you are not blind to subgraphs: `messages` mode already carries their model tokens
(each tuple's metadata holds `langgraph_checkpoint_ns`), and `events` mode carries every nested event.
What the flag adds is their `updates`, `values` and `custom` frames, and names a client can route on.
With it on, the run's own model tokens are namespaced too (`messages|model_request:…`), as upstream —
the root graph's frames are the ones with no `|`.
Inside Skein, `RunFrame.event` remains the bare mode and `RunFrame.namespace` carries the optional
subgraph path. Root frames omit the namespace. Event buses preserve both fields; SSE serialization
combines them into the upstream event name shown above. Existing switches over `RunFrame.event`
continue to work.
Declared replies keep their existing last-write-wins behavior, including nested custom writes that
LangGraph propagates. The client's `stream_subgraphs` setting does not change which reply is delivered.
## Slow clients and backpressure
A stream is only as fast as the client reading it, and skein paces itself accordingly. Every adapter's
write loop honors the response stream's backpressure signal: when the socket's buffer is full it waits
for `drain` before pulling the next frame, rather than queueing whatever the graph produces.
That matters because the alternative is unbounded. A client on a bad connection — a phone on mobile
data, a buffering reverse proxy — that reads more slowly than the graph writes would otherwise be
served entirely out of the server's memory, one full copy of the stream per connection.
Measured in [`packages/bench`](https://github.com/skein-js/skein-js/tree/main/packages/bench) on the `slow-client` scenario (clients reading at
~25 fps against a 500 fps graph, ~2 MB per stream):
| Concurrent slow streams | Unflushed server-side buffer | Per streaming connection |
| ----------------------- | ---------------------------- | ------------------------ |
| 50, before | 62.9 MB | ~1.26 MB |
| 100, before | 125.5 MB | ~1.26 MB |
| 50, after | 3.3 MB | ~67 KB |
| 100, after | 6.5 MB | ~65 KB |
Before, the per-connection cost was the whole stream, so total memory grew with both the number of
clients **and** the length of each run. After, it is a constant close to the socket's own 64 KB
high-water mark: still linear in connection count, as it must be, but no longer proportional to how
much the graph produces.
Two consequences worth knowing:
- **The graph does not slow down.** The run engine publishes into the event bus and the write loop
reads from it, so pacing the reader changes how fast frames leave the bus, not how fast they enter
it. A slow client cannot stall the run, or any other client's stream.
- **Backpressure itself never discards.** It delays delivery; a slow client receives every frame, just
later. What it does is move the queue from the socket into the event bus — and the bus is bounded, so a
client slow enough to exceed that bound has its stream **ended** rather than being left hanging.
What happens next differs by bus. On **Redis** the frames are still in the durable stream, so
reconnecting with `Last-Event-ID` replays them: nothing is lost. On the **in-memory bus** the stream
ended precisely _because_ the frames were evicted to stay under
`SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN`, and that buffer is also the replay log — so a reconnect resumes
from what survives, with a gap. See
[performance.md](./performance.md#streaming-backpressure-drops-and-recovery) for the bounds, and why
that asymmetry is a reason to run Redis for unreliable clients.
The Next.js App Router adapter gets this for free: it maps frames onto a `ReadableStream` whose `pull`
is demand-driven by the platform.
## Joining and cross-instance fan-out
- `GET /runs/{run_id}/stream` lets a late client join a run already in progress.
- When a run executes on a **different** worker than the one holding the client connection,
[`@skein-js/redis`](./runs-and-redis.md) pub/sub fans the frames across instances so the join
still works. In single-process `skein dev`, an in-memory event bus is used instead.
- The [SQLite local assembly](./local-deployment.md) also uses a process-local event bus. Records
and checkpoints survive reopening, but live token frames do not. Use persisted state/history to
restore the UI after restart; SQLite does not provide cross-instance fan-out.
## Why SSE is enough (no WebSocket in v1)
The entire LangChain client surface — the vanilla SDK, the [`useStream`](./react-sdk.md)
React hook, and Agent Chat UI — consumes **SSE**. The protocol's optional WebSocket upgrade
buys bidirectional framing we don't need for v1, so it is deferred (see
[roadmap.md](./roadmap.md)). **Deferring WebSocket does not affect the React SDK.**
## Thread event subscriptions
The thread protocol separates execution from observation. Submit `{ id, method: "run.start",
params: { assistant_id, input } }` to `/threads/{id}/commands`, then observe
`POST /threads/{id}/stream/events` with `{ channels: ["values", "messages", "lifecycle", "input"] }`.
Reopening that subscription executes nothing. Commands return JSON envelopes; legacy command bodies
retain their SSE response. The LangGraph binding uses the public `streamEvents(..., { version: "v3" })`
path for protocol runs, including graph-defined transformers and custom channels.
Subscriptions stay open across turns and receive idle heartbeat comments. Channel, namespace-prefix
and depth filters are reader-specific; static namespace segments also match their runtime `:task-id`
suffixes. The bus assigns sequence numbers before filtering, so reconnect cursors work across runs.
Request `since` to replay after a cursor; an invalid retained-history boundary requires state
restoration and a fresh subscription. With no cursor, the current state is the initial baseline.
`MemoryThreadEventBus` bounds each payload history to 10,000 entries, retaining at most 50 payload
histories. `RedisThreadEventBus` uses capped streams (approximately 10,000 entries) with a one-hour
payload expiry. Sequence and deletion-generation records survive payload expiry. Deleting a thread
clears payloads and closes its readers; execution captures its generation so a late publisher cannot
repopulate history after deletion or ID reuse. Release the Redis bus with `dispose()` when assembling
infrastructure by hand; runtime assemblers own that cleanup automatically.
Legacy `GET /threads/{id}/stream` joins one run by default. Pass explicit `stream_mode` to follow
events across turns. The official categories are `run_modes` (run frames plus completion metadata),
`lifecycle` (start/end `metadata` events), and `state_update` (`{ thread_id, state }` checkpoint
snapshots after execution or direct state updates). Recorded snapshots are replayed without
substituting a later thread state. Individual frame modes and optional `thread_state` thread-row
events remain available. The stream starts with live events unless
`Last-Event-ID` resumes after an earlier one. This default differs from Agent Server;
see the [compatibility matrix](./langgraph-compatibility.md) for the verification scope.
A subscription without `since` replays retained events so late selectors can recover custom
channels and media, then receives a current state baseline and live events. An explicit `since`
resumes after that publication sequence. Sequences span runs; payload expiry does not reset them.
Expired or future explicit cursors return `thread_cursor_expired` and require state restoration.
Memory and Redis buses use independent reader cursors and reject publications from a deleted
thread incarnation, including after an explicit thread-ID reuse. SQLite's event bus is process-local;
its state and run records remain durable across restart.
---
<!-- source: docs/errors-and-logging.md -->
# Errors & logging
What happens when a graph throws — where the failure shows up, what a client sees, and what lands in
your logs. Errors and logging live in one doc because a failure goes to both, on deliberately
different terms: **the log gets everything; the wire gets what is safe to hand a caller.**
## When a local process disappears
Reopening a SQLite deployment records abandoned executing/claimed attempts and pending inline
requests as `error` with a `ProcessLostError` payload. Unclaimed queued jobs keep their due times;
terminal outcomes and real approval interruptions are preserved. This startup bookkeeping does not
replay input, resume checkpoints, or synthesize completion callbacks and telemetry. Applications
decide what to do next. See [local crash behavior](./local-deployment.md#what-survives-a-process-ending).
## When a graph throws
A node that throws does not crash the server. The run engine catches it, and five things happen:
| Surface | What it gets |
| --------------------- | ---------------------------------------------------------------------------------------------- |
| **The server log** | An `error`-level line with the full stack and `cause` chain. Always — see [Logging](#logging). |
| **The SSE stream** | A terminal `event: error` frame carrying a [`RunError`](#the-runerror-payload). |
| **The run row** | `status: "error"` plus `error` — the same `RunError`, durable. |
| **The thread row** | `status: "error"` plus `error` (the message). A _mirror of the latest turn_. |
| **`POST /runs/wait`** | `200` with `{ "__error__": { … } }` in place of the graph's values. |
| **The webhook** | `error` as a plain message string, alongside the settled run. |
The run row is the durable record. The thread's `status` and `error` are cleared as soon as a later
run on that thread succeeds — they describe the thread _now_, not its history. If you need to know
why a particular run failed, read the run.
```bash
curl localhost:2024/threads/$THREAD/runs/$RUN
```
```json
{
"run_id": "63086ee4-…",
"status": "error",
"error": {
"error": "Error",
"name": "Error",
"message": "model call failed",
"cause": { "error": "Error", "name": "Error", "message": "MISSING_KEY is undefined" }
}
}
```
### The `RunError` payload
One shape (`RunError`, from `@skein-js/core`) is used by the SSE frame, the persisted `Run.error`,
and the `__error__` wait body — so the stream and a later `GET` can never disagree.
| Field | |
| --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `error` | The error's constructor name. LangGraph Platform's name for this field. |
| `message` | The error's message. The one field every client renders. |
| `name` | Always identical to `error` — see [compatibility](#langgraph-platform-compatibility). |
| `cause` | The `Error.cause` chain, one `RunError` per link, up to five deep. |
| `errors` | An `AggregateError`'s members (up to ten) — LangGraph throws one when several nodes fail in the same superstep, and its envelope message alone tells you nothing. |
| `stack` | **Only when the server opts in.** See below. |
`toRunError(thrown, { includeStack })` builds one from anything thrown. It is cycle-safe, depth
capped, and never throws — it runs on the failure path, where a second failure would hide the first.
### `exposeErrorStacks`
A stack names server file paths, dependency versions, and sometimes argument values. So it goes to
your logs unconditionally, and to the **client** only when you ask:
```ts
const deps: ProtocolDeps = { /* … */ exposeErrorStacks: true };
```
- **`skein dev` turns it on**, unconditionally — not behind `--verbose`. Needing a flag to find out
why your graph crashed is the problem this exists to remove.
- **`skein start`, and every embedded server, leave it off.** LangGraph Platform never puts a stack
on the wire either, so with it off the frame is a strict superset of the platform's.
Turning it off never costs you information as an operator: the server log has the full stack either
way. It only affects what the _client_ can see.
## Logging
skein logs through a four-method interface — no framework, no transport, no color:
```ts
interface Logger {
debug(message: string, meta?: unknown): void;
info(message: string, meta?: unknown): void;
warn(message: string, meta?: unknown): void;
error(message: string, meta?: unknown): void;
}
```
Every adapter takes a `logger` option, and it reaches **the run engine** — not just the transport. It
is the one knob that decides whether a crashed graph is visible at all.
### What each adapter does by default
| | Default | Why |
| ------------------- | -------------------- | ------------------------------------------------------------------ |
| `@skein-js/nestjs` | Nest's own `Logger` | A facade over `app.useLogger()` / `NestFactory.create({ logger })` |
| `@skein-js/fastify` | `fastify.log` (pino) | The host's own instance, honoring its config |
| `@skein-js/express` | **nothing** | Express owns no logger — pass `createConsoleLogger()` |
| `@skein-js/nextjs` | **nothing** | Same |
The split is deliberate. Where the framework owns a logger the host has already configured, skein
_borrows_ that decision — including the decision to be silent (`NestFactory.create({ logger: false })`,
`Fastify({ logger: false })`). Where it doesn't, defaulting on would mean a library writing into
someone's stdout uninvited, so skein stays quiet until asked:
> **The two standalone servers are the exception, and differ from each other.** `createNestServer`
> owns its bootstrap and keeps Nest's banner off, which silences the global facade — so it defaults to
> its own `ConsoleLogger` and `app.useLogger()` does _not_ redirect it; pass `logger` instead.
> `createFastifyServer` leaves pino off exactly as Fastify does, so it stays silent until you enable
> it with `{ fastify: { logger: true } }` or pass a `logger`.
```ts
import { createConsoleLogger, createExpressServer } from "@skein-js/express";
createExpressServer({ config: "./langgraph.json", logger: createConsoleLogger() });
```
`createConsoleLogger({ level, prefix })` is plain uncolored output — `level: "warn"` suppresses the
per-run summaries while still reporting every failure. (The colored, code-framed failure block is
`skein dev`'s; it knows your source root and can safely read from it.)
### Precedence
| Situation | Result |
| ---------------------------- | ----------------------------------------------------- |
| `deps.logger` is set | that logger, always — the adapter never overwrites it |
| otherwise, `logger` a Logger | that logger |
| otherwise, `logger: false` | nothing — no default installed |
| otherwise | the adapter's default from the table above |
An injected `deps.logger` outranking the adapter option is deliberate, and it is what makes the rest
safe: the adapter only ever _fills_ `deps.logger`, never replaces it. That in turn lets it hand the
engine your actual deps object rather than a copy — which is what keeps post-mount configuration
(`deps.exposeErrorStacks = true` after mounting) working, since the invoke surface re-reads its deps
on every request.
### Bridging to your own logger
Each bridge ships with its adapter, so meta lands in the shape that logger actually wants:
```ts
import { createNestLogger } from "@skein-js/nestjs";
// Point skein at a specific LoggerService rather than the globally configured one.
SkeinModule.forRoot({ config, logger: createNestLogger({ logger: myPinoNestLogger }) });
import { createFastifyLogger } from "@skein-js/fastify";
// pino is object-first, so a failed run's ids become queryable fields and `err` gets pino's
// error serializer — not a pre-flattened string.
app.register(skeinPlugin, {
config,
logger: createFastifyLogger(app.log.child({ svc: "graphs" })),
});
```
Anything else is four lines against the interface above.
### What is always logged
A **failed run** is always reported at `error` level, whatever `logRunActivity` says. The `meta` is a
`RunFailureReport` — plain data plus the original `Error`, so a console logger can render a stack and
a code frame while a JSON logger can serialize the fields. Recognize it with `isRunFailureReport`.
Also always logged: background-run lifecycle summaries (at `error` level, with a `run_error` field,
when the run failed), webhook delivery failures, rollback failures, and queue-shutdown problems.
That is the whole steady-state volume — failures, plus one line per background run. The noisy
per-run chatter is behind `logRunActivity` and stays off.
One cost worth knowing: naming the node that threw takes a checkpointer read, so the engine skips
building the report when nothing is listening. With a logger configured — which, under NestJS and
Fastify, is now the default — a **failed** run pays that read. Successful runs are unaffected.
### Sending failures somewhere else
The log is one of **three** surfaces a failure reaches. The third is a telemetry sink — LangSmith,
PostHog, OpenTelemetry, or your own — which receives a `run.finished` event carrying the `RunError`,
the node that threw, and the original `Error`:
```ts
const sink: TelemetrySink = {
name: "sentry",
onRunEvent: (event) => {
if (event.type === "run.finished" && event.cause) Sentry.captureException(event.cause);
},
};
```
A sink is **server-side, like the log** — so it always gets the full `Error`, stack and `cause` chain
included, **regardless of `exposeErrorStacks`**. That flag governs only what reaches a client. See
[observability.md](./observability.md).
### What `--verbose` adds
`skein dev --verbose` sets `ProtocolDeps.logRunActivity`, which adds per-run _chatter_: run
start/finish with duration and frame count, each tool call and tool result, and interrupt prompts.
It costs nothing when off — the engine skips the stream inspection entirely.
It does **not** gate failures. A graph that throws is logged either way.
### The failure block
The CLI — `skein dev` and `skein start` alike — renders a graph failure as a fenced block naming the
run, the assistant, the thread, the node that threw, and a code frame pointing at the line:
```
error: Graph run failed: model call failed
──────────────────────── GRAPH RUN FAILED ────────────────────────
run 63086ee4-1729-4744-8c8e-2cbdd48aff80
assistant boom
thread 76258f06-a401-44c0-a6d7-7e189a3f8b36
node call_model
3 | function callModel(): never {
4 | const apiKey = process.env["MISSING_KEY"];
> 5 | throw new Error("model call failed", {
| ^
6 | cause: new Error(`MISSING_KEY is ${String(apiKey)}`),
Error: model call failed
at RunnableCallable.callModel (src/boom-graph.ts:5:9)
caused by: Error: MISSING_KEY is undefined
──────────────────────────────────────────────────────────────────
```
The rules and blank lines are plain text, not color — piped logs, CI output, and `NO_COLOR` all
disable color, and a crash needs to stand out precisely there.
Two parts are best-effort and are simply omitted when unavailable, never faked:
- **The `node` row.** LangGraph rethrows a node's error verbatim, so the error object never names the
node. skein reads it from the post-failure state snapshot instead, where the runner records an
`__error__` write against the failing task. A failure before the graph started yields nothing, and
a failure inside a subgraph names the _parent_ node.
- **The code frame.** It needs the stack to point at readable source inside the project. Under
`skein dev` it does — vite's module runner source-maps stacks back to your `.ts` files, and the
frame is bounded to the workspace root vite serves from. Under `skein start` the bundled artifact
usually ships without original sources, so the frame is normally absent and the stack still prints.
The frame is deliberately conservative about where it reads from, because an error _message_ is
frequently attacker-influenced — a raw model response, a fetched document, or simply
``throw new Error(`bad mode: ${input.mode}`)`` over a client-supplied input. Since `Error.stack`
is `${name}: ${message}` followed by the frames, a newline in a message produces a line that parses
exactly like a stack frame and sits ahead of every genuine one. skein parses frames only from the
region after the header, and reads only files resolving inside the project root — so a crafted
message cannot steer it into reading an unrelated file.
### The load-failure block
A graph can fail the other way too — never loading at all, because importing its module threw. That
is the failure a fresh project hits first, and almost always for one reason: the graph builds a model
client at module scope and the API key is not set.
`skein dev` and `skein start` import every declared graph once at startup, **after** the banner, and
report each failure as its own block:
```
error: graph "agent" failed to load
────────────────────── GRAPH FAILED TO LOAD ──────────────────────
graph agent
source src/agent-graph.ts:68
66 | const apiKey = process.env.GOOGLE_API_KEY;
67 | if (!apiKey) {
> 68 | throw new Error(
| ^
GOOGLE_API_KEY is not set — the "agent" graph needs it. Uncomment it in
.env (get a key at https://aistudio.google.com/apikey) and save; the dev
server picks it up on reload. The "echo" graph needs no key.
SkeinConfigError: Failed to import graph module ".../src/agent-graph.ts".
caused by: Error: GOOGLE_API_KEY is not set — the "agent" graph needs it. …
──────────────────────────────────────────────────────────────────
```
The headline is the **root** of the `cause` chain, not the wrapper: `SkeinConfigError` says _where_
the failure happened, and only its cause says _what_ went wrong. The code frame follows the root's
stack for the same reason — the wrapper's stack is nothing but skein frames. The full chain still
prints underneath.
One graph that cannot load never takes the server down; the rest keep serving. Under `skein dev` the
project's `.env` is watched, so filling in the missing key and saving reloads the graphs — a key that
is _newly added_ takes effect, while changing one that is already set still needs a restart (the
ambient environment outranks the file, by design).
## Errors at the edges
Two typed errors, both carrying a `cause` that the CLI prints:
- **`SkeinHttpError`** (`@skein-js/core`) — carries the HTTP status a handler wants, plus optional
`code` and `details`. Adapters map it to `{ status, message, code?, details? }`; anything else
becomes an opaque `500` with the real error sent to the logger. See
[building-an-adapter.md](./building-an-adapter.md).
- **`SkeinConfigError`** (`@skein-js/config`) — a bad `langgraph.json`, an unknown graph, or a graph
module that failed to import. Its `cause` is the actual import failure and its `details` are the
Zod issues, so read past the top-level message.
A graph that fails to _load_ during a **run** surfaces through the same failure path as one that
throws, with the config error's `cause` chain intact — unchanged.
On the surfaces that answer over HTTP directly — thread state and history, assistant introspection,
the single-graph invoke handler — it is mapped to a `SkeinHttpError` carrying `code:
"graph_load_failed"`, so it stops being an unhandled fault. How much of _why_ reaches the caller is
governed by [`exposeErrorStacks`](#exposeerrorstacks), the same switch that governs stacks:
| | Body |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| **`skein dev`** (on) | `{"status":500,"message":"Graph \"agent\" failed to load: GOOGLE_API_KEY is not set …","code":"graph_load_failed"}` |
| **`skein start`**, embedded (off) | `{"status":500,"message":"Graph \"agent\" failed to load.","code":"graph_load_failed"}` |
The reason is withheld in production because a load failure's message is not skein's: `Cannot find
module '/srv/app/dist/tools.js'` and `connect ECONNREFUSED 10.0.3.14:5432` are both ordinary ones,
and they name server paths and internal hosts. The stack never travels either way.
**The operator loses nothing.** Adapters log every `5xx` — including this one, typed or not — with
the full `cause` chain, so the reason is always in your logs even when it is not on the wire.
`rootCause(thrown)` / `rootCauseMessage(thrown)` (`@skein-js/core`) are what collapse a chain to one
sentence, if you want the same in your own handler. Cycle-safe and depth-capped, like `toRunError` —
which you want instead when a client should see _every_ layer.
## LangGraph Platform compatibility
Verified against `@langchain/langgraph-api` and `@langchain/langgraph-sdk`:
- The platform's error frame is `{ error, message }`, and the SDK's `ErrorStreamEvent` declares
exactly that. Its `StreamError` reads `data.name ?? data.error`. skein emits **both** `error` and
`name` with the same value, so the SDK, the platform's own clients, and older skein clients (which
saw only `name`) all agree. Extra keys are ignored by the SDK.
- `Run.error` is a skein extension — the SDK's `Run` records only _that_ a run failed. It is
optional, so a client that ignores it is unaffected. This and the `"cancelled"` run status are the
only two places skein deliberately steps outside the SDK's wire contract.
- `Thread.error` **is** an SDK field that the JS platform leaves empty; skein populates it (and keeps
the older `metadata.error` alongside it for existing readers).
- `ThreadTask.error` is a JSON string, because `useStream` reads thread history by `JSON.parse`ing
this field and rebuilding a `StreamError` from it.
- `POST /runs/wait` answers a failed run with `{ "__error__": … }`, the platform's key. skein does
not reproduce the platform's double-encoding of that payload.
- The webhook's `error` is a plain message string, matching the platform exactly.
---
<!-- source: docs/react-sdk.md -->
# Frontend SDKs / `useStream` compatibility
Skein supports the run-based JavaScript SDK, the legacy React hook from
`@langchain/langgraph-sdk/react`, and the dedicated framework packages. The `@langchain/react`, `@langchain/vue`,
`@langchain/svelte`, and `@langchain/angular` packages use a separate thread event protocol.
The browser matrix covers each framework at 1.2.6 and React at 1.2.7, with the media workarounds
described below. The React examples use `@langchain/react` 1.2.7 and the current JavaScript client
1.12.7; legacy SDK coverage remains separate. See the [compatibility matrix](./langgraph-compatibility.md).
**What this gives you:** point
[`useStream`](https://reference.langchain.com/javascript/langchain-react/useStream) at a
skein-js server and you get the whole rich chat UX for free — live token streaming, model
**thinking**, structured **tool-result cards**, and **human-in-the-loop** interrupt/resume — with no
custom SDK and no bespoke wire format. You send a turn with `thread.submit(...)`, read live state off
`thread.messages`, and when a graph node pauses with `interrupt()` you render an approval card off
`thread.interrupt` and resume with `thread.respond(...)`. The flagship [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app)
example builds all of this; [`react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream) is the minimal
copy-paste starting point.
```tsx
// send a turn — thread.messages updates live as tokens stream in
void thread.submit({ messages: [{ role: "user", content: input }] });
// a pending interrupt (e.g. "approve this booking?") surfaces here; resume with a command
if (thread.interrupt) {
void thread.respond({ approved: true });
}
```
Long-term memory is the one piece that lives on the server, not in the hook: a graph node calls
`getStore()` and skein-js persists it (see [storage.md](./storage.md)). The frontend just keeps
streaming.
## The clients skein-js must satisfy
| Client | Package | How it talks to skein-js |
| ---------------------- | --------------------------------------- | ------------------------------------------------------------------------------ |
| Vanilla JS SDK | `@langchain/langgraph-sdk` | `client.threads.*`, `client.runs.stream()`, `client.runs.wait()` |
| **React hook** | `@langchain/react` | **`useStream({ apiUrl, assistantId })`** over thread SSE and commands |
| Legacy React hook | `@langchain/langgraph-sdk/react` | Run-based SSE; retained compatibility coverage |
| New framework bindings | `@langchain/{react,vue,svelte,angular}` | Thread event subscriptions and command envelopes; see the compatibility matrix |
| Agent Chat UI | (built on `useStream`) | Same SSE path |
| LangGraph Studio | — | Agent Protocol HTTP |
## Not just React — Vue, Svelte and Angular
LangChain publishes first-party bindings for four frameworks. Their current releases subscribe to
`POST /threads/{thread_id}/stream/events` and send protocol command envelopes to `/commands`.
This differs from the run-based endpoints used by the legacy React hook. A subscription must never
start a run, including when the client reconnects.
| Framework | Package | Entry points | Peer range |
| --------- | ---------------------------------------------------------------------------------------------- | ----------------------------------------------- | ------------- |
| React | [`@langchain/react`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-react) | `useStream`, `StreamProvider` | React 18 · 19 |
| Vue | [`@langchain/vue`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-vue) | `useStream`, `provideStream`, `LangChainPlugin` | Vue 3 |
| Svelte | [`@langchain/svelte`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-svelte) | `useStream`, `provideStream`, `getStream` | Svelte 5 |
| Angular | [`@langchain/angular`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-angular) | `injectStream`, the `inject*` family | Angular 18–21 |
Each follows its own idiom — a React hook, a Vue composable, Svelte context, Angular DI — over a
shared streaming core. The React examples below use the dedicated package.
**Mind the package names.** These are `@langchain/<framework>` packages, _not_ subpaths of the SDK —
`@langchain/langgraph-sdk/vue` does not resolve, and the SDK's own README links to them by monorepo
path, which is easy to misread as a subpath. `@langchain/langgraph-sdk/react` still exists for legacy clients; new applications should use the
dedicated packages shown here.
skein-js ships no standalone Vue, Svelte or Angular example. The dedicated compatibility CI job
exercises all four framework packages in Chromium, separately from the runtime matrix and legacy
`useStream` coverage. The [compatibility matrix](./langgraph-compatibility.md) records tested versions
and features.
## `useStream` against skein-js
```tsx
"use client";
import { useStream } from "@langchain/react";
export function Chat() {
const thread = useStream({
apiUrl: process.env.NEXT_PUBLIC_SKEIN_URL!, // e.g. http://localhost:2024
assistantId: "agent", // a graph id from langgraph.json
});
return (
<div>
{thread.messages.map((m) => (
<div key={m.id}>{typeof m.content === "string" ? m.content : ""}</div>
))}
<button onClick={() => thread.submit({ messages: [{ role: "user", content: "hello" }] })}>
Send
</button>
</div>
);
}
```
The only difference from a LangGraph Platform setup is that `apiUrl` points at a skein-js
server. The dedicated hook subscribes to `POST /threads/{thread_id}/stream/events` and dispatches
turns through `POST /threads/{thread_id}/commands`. Subscriptions stay open across turns, and
reconnecting or restoring an existing thread never starts a run. The legacy hook instead uses
run-based streaming endpoints (see [streaming.md](./streaming.md)).
Call `thread.stop()` to cancel the active run. Unmounting or reopening a thread subscription is
separate from execution; use the explicit stop action when the user intends to cancel.
## Why it works over SSE
`useStream` is an SSE client. Because skein-js serves the Agent Protocol streaming endpoints as
`text/event-stream` with the same event names and payloads LangGraph emits, the hook cannot
tell the difference. **No WebSocket is required**, so deferring WebSocket transport in v1
does not affect the React SDK.
## Verification harness
[`examples/react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream)
is a minimal Next.js app using `@langchain/react` 1.2.7. The dedicated browser target exercises current
framework packages over real HTTP, and the packed consumer checks both current and legacy clients
outside the workspace. See [testing.md](./testing.md#compatibility-regression-gates).
## References
- LangGraph JS SDK — <https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk>
- `useStream` API reference — <https://reference.langchain.com/javascript/langchain-react/useStream>
- LangGraph streaming — <https://docs.langchain.com/oss/javascript/langgraph/streaming>
- Agent Chat UI — <https://github.com/langchain-ai/agent-chat-ui>
- React bindings — <https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-react>
- Vue bindings — <https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-vue>
- Svelte bindings — <https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-svelte>
- Angular bindings — <https://github.com/langchain-ai/langgraphjs/tree/main/libs/sdk-angular>
## Dedicated framework packages
The dedicated `@langchain/react`, `@langchain/angular`, `@langchain/vue`, and `@langchain/svelte`
packages use the thread event protocol. Versions **1.2.6** are exercised in Chromium against Skein
with their native HTTP transport, including multiple turns, restoration, interrupts, subgraphs,
custom transformer channels, scoped media, forced reconnect and cancellation. Reopening a
subscription never starts another graph execution. See the [compatibility matrix](./langgraph-compatibility.md)
for the exact server, SDK and backend evidence.
Svelte 1.2.6 has an upstream limitation: its root `useImages` selector returns an empty projection.
Scoped media selectors are covered; target the subgraph namespace containing the media. This is a
client selector limitation, even when the server sends valid root media events. React, Angular and
Vue root media selectors also pass the browser checks.
Vue 1.2.6 wraps media asset instances in reactive proxies. Reading an asset's `blob` directly can
throw a private-field error; unwrap it with Vue's public `toRaw` first. The browser gate verifies the
actual MIME type and bytes with this workaround:
```ts
import { toRaw } from "vue";
const blob = await toRaw(images.value[0]).blob;
```
---
<!-- source: docs/langgraph-compatibility.md -->
# LangGraph Agent Server compatibility
Baseline investigated on **22 October 2026**. A shared dependency or a route with a similar name
does not establish compatibility: the official client must exercise the behavior end to end.
| Surface | Baseline | Compatibility work |
| ------------------------------------------------- | --------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| Run-based JavaScript SDK and legacy React hook | Supported | Retain coverage for SDK 1.9.25 and 1.9.31 |
| Thread event protocol | Not yet supported | SDK 1.12.6; protocol 0.0.20; subscriptions, commands, reconnect and restoration |
| Dedicated React, Angular, Vue and Svelte bindings | Not yet verified | Test each package at 1.2.6 in a real browser |
| Thread-wide legacy stream | Joins one run | Add explicit `stream_mode` following without changing the existing default |
| Execution and checkpoint options | Partial | Durability, task/checkpoint modes, recursive and namespaced state/history |
| Search and retention | Partial | Projection, exclusion, extraction, state timestamps, per-operation store options and keep-latest TTL |
| MCP and A2A endpoints | Not implemented | Official transport SDKs over the existing authorized services |
| Graph UI assets and `http.app` | Accepted configuration without implementation | Development loading and standalone production artifacts |
| OpenAPI and `/docs` | Not implemented | Describe actual enabled routes and authentication |
| `/metrics` and thread storage reporting | Not implemented | Existing OTel instrumentation and exact saver-backed payload accounting |
Explicit legacy thread following supports the SDK's `run_modes`, `lifecycle`, and `state_update`
categories, including per-run completion metadata and immutable checkpoint snapshots on replay.
Search extraction returns aliases inside `extracted`; checkpoint forks validate saver coordinates
without requiring optional custom-agent timestamps.
The versions above are **implementation targets**, not a claim that all of their features have
passed. The implementation updates this matrix with the corresponding test evidence.
## Intentional differences
- HTTP/SSE is the supported transport target; WebSockets and gRPC remain outside the compatibility
program.
- Stateless runs keep their threads by default. Request `on_completion: "delete"` to remove them.
- Store ordering filters compare numbers only.
- Local SQLite persists resources and checkpoints; event replay is process-local and is lost on
restart.
- Skein adds idempotent run creation and durable, signed, retried webhook delivery. LangGraph also
supports ordinary run-completion webhooks.
- Managed hosting, billing, evaluations and application identity policy remain outside Skein.
See [the frontend guide](./react-sdk.md), [protocol reference](./agent-protocol.md),
[CLI compatibility](./langgraph-cli-compat.md), and
[issue #62](https://github.com/skein-js/skein-js/issues/62).
## Implementation evidence, 23 October 2026
The workspace exercises LangGraph.js **1.4.7**; the clean packed consumer pins **1.4.12**.
The thread-protocol binding uses public `streamEvents({ version: "v3" })` and requires that runtime
capability. The existing legacy execution path remains available for older graphs and custom agents.
| Scope | Versions and backends exercised | Evidence |
| -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| Thread subscriptions, commands, restoration, multiple turns and interrupts | JavaScript SDK 1.12.6 / protocol 0.0.20; Express + memory | Official client HTTP tests in `thread-protocol.test.ts` |
| Reconnect across API instances, one queued execution | SDK 1.12.6; Express + real Redis queue and thread buses | `thread-protocol.integration.test.ts`, including a proxy-forced reconnect |
| Legacy protocol and adapters | Packed SDK 1.9.25 / 1.9.31 clients and legacy protocol tests; Express, Fastify, Fetch, NestJS, Next.js | Existing adapter suites retained; dedicated framework browser coverage is listed separately below |
| Store indexing, nullable write TTL and per-read refresh | Memory, SQLite and real Postgres | Shared store-options conformance; real SQLite reopen |
| MCP discovery and execution | Official MCP SDK 1.32.1; stateless Streamable HTTP | Official client transport test; shared MCP/A2A HTTP checks across all five adapters |
| A2A output retention and interrupts | Official A2A SDK 1.3.0; memory and reopened SQLite | Official client over HTTP; first-task output survives another turn and checkpoint pruning |
| Dedicated frontend bindings | React/Angular/Vue/Svelte 1.2.6, SDK 1.12.6, Chromium; Express + memory | Native HTTP streaming, restoration, interrupts, cancellation, multiple turns, subgraphs, custom channels, scoped media and proxy-forced reconnect |
| Compiled UI and custom applications | Public UI compiler 1.4.2; generated Node 24 production image + SQLite | Real build → artifact → image; UI bootstrap/assets, Fetch app, OpenAPI and docs without sources or optional UI compiler |
| Diagnostics | Memory, SQLite and real Postgres saver payloads; standard OTel metric reader | Actual buffer/blob/datum accounting; Prometheus and OTLP JSON serializers; explicit unsupported-capability errors |
A separate packed consumer installs Skein tarballs outside the workspace and exercises SDK 1.12.6,
legacy SDKs 1.9.25 and 1.9.31, official MCP and A2A clients, and diagnostics over native HTTP with no
application transport shim. That gate passes. These checks establish the listed behavior only;
release also requires the full Nx quality gates. Svelte 1.2.6's root `useImages` selector returns an empty
projection in its own implementation; scoped media selectors pass. Root media from React, Angular
and Vue was also exercised while validating graph-scope message namespaces. Vue 1.2.6 media assets
require `toRaw(asset)` before accessing `blob`, because its reactive proxy breaks private fields.
The browser gate checks actual media bytes with that public Vue workaround. These client limitations
are tracked separately from server media delivery. Development UI watches currently follow the
upstream watcher’s process lifetime; applications own process shutdown.
## Frontend example refresh, 25 October 2026
The React examples (`chat-app`, `nextjs-app`, and `react-usestream`) use `@langchain/react` 1.2.7.
Example JavaScript clients require SDK 1.12.7. The browser matrix also runs the React package
installed by `react-usestream`, alongside the pinned 1.2.6 framework packages and SDK 1.12.6.
Legacy SDK 1.9.25 and 1.9.31 coverage remains in the packed consumer.
---
<!-- source: docs/storage.md -->
# Storage
**What this gives you:** durable agents and **long-term memory** that outlives a single
conversation, with zero setup in dev. Threads, runs, and stored memories survive restarts, and inside
a graph node you get a LangGraph-native store — `getStore()` — for cross-thread facts ("prefers window
seats") backed by **pgvector semantic search** when using Postgres with indexing. It's the same store LangGraph Platform
auto-provides, so a graph that calls `getStore()` runs unchanged on skein-js. The best part: you write
your graph once and choose a backend — **memory for the default `skein dev`, SQLite for local apps,
or Postgres + pgvector for a server** — no graph code change. SQLite provides text/filter search;
semantic indexing uses Postgres or an external store. The flagship [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) example uses
this to remember a user across sessions.
skein-js separates two kinds of persistence, and it is important not to conflate them:
1. **Graph checkpoints** — LangGraph's own state/history for a thread (this is what powers
**interrupt/resume** and history). **Reused, never reimplemented:** delegated to an existing
LangGraph checkpointer (`MemorySaver` for memory, `SqliteSaver` for the local assembly,
`PostgresSaver` for Postgres). See [reuse.md](https://github.com/skein-js/skein-js/blob/main/docs/reuse.md).
2. **Protocol resources** — assistants, thread metadata/status, run rows, and long-term
store items. These are the gap OSS keeps _in memory_, so skein-js owns them behind a single
`SkeinStore` interface with durable drivers.
## `SkeinStore` interface
A single interface, implemented by each driver, covering the protocol resources:
```ts
interface SkeinStore {
// assistants (derived from langgraph.json graphs, plus user-created)
assistants: AssistantRepo;
// threads: metadata + status (idle | busy | interrupted | error), plus the latest turn's error
threads: ThreadRepo;
// runs: status + queue rows (pending | running | success | error | cancelled | timeout),
// and — for a failed run — why (see errors-and-logging.md)
runs: RunRepo;
// schedules that fire runs on a cadence, plus the compare-and-swap claim the scheduler uses
crons: CronRepo;
// long-term memory: namespace/key items with optional semantic search
store: StoreRepo;
// recorded responses for `Idempotency-Key`, so a provider's retry replays instead of re-running
idempotency: IdempotencyRepo;
}
```
Each repo exposes CRUD + list/search shaped to the [Agent Protocol](./agent-protocol.md)
endpoints. All drivers are validated against **one shared conformance test suite**, so
memory, SQLite, and Postgres share the same resource behavior. Their queue ownership and crash
semantics differ; see [local deployment](./local-deployment.md#what-survives-a-process-ending).
### Page bound (`SKEIN_MAX_PAGE_SIZE`)
Every list and search path is **bounded — including when the caller passes no `limit` at all**. The
default is **1000 rows**. This is a memory bound: a thread row carries the thread's mirrored graph
state, so an unbounded `POST /threads/search` on a large deployment materializes the table twice over
(the rows, then the JSON response string) inside one request.
| Surface | Bound |
| --------------------------------------- | ----------------------------------------------------------- |
| `limit` on a search request | rejected above 1000 by the wire schema |
| `limit` omitted, or a `list()` call | the first `SKEIN_MAX_PAGE_SIZE` rows (default 1000) |
| `assistants.count()`, `threads.count()` | **not** bounded — they answer "how many match" in total |
| `runs.listByThread()` | **not** bounded — run rows carry no graph state |
| `runs.latestForThread()` | one row by construction — the thread state path uses this |
| `runs.listActiveRuns()` (all threads) | bounded — the whole-server sweep behind `POST /runs/cancel` |
| `POST /threads/{id}/history` | 100 checkpoints by default, 1000 max — a separate bound |
`runs.latestForThread()` exists because the unbounded `listByThread()` above used to be read on **every**
thread state, history, and state-update request: resolving which graph a thread belongs to means reading
its most recent run's `assistant_id`, and that was done by fetching the thread's entire run history and
sorting it. A driver must return the newest run by `created_at` descending, tie-broken on `run_id`
descending — the tie-break is a within-driver determinism contract (`created_at` ties at millisecond
resolution), not a cross-driver one.
Sizing guidance is in [performance.md](./performance.md#sizing). Set `SKEIN_MAX_PAGE_SIZE` to change the
driver bound (`maxPageSize` on the store constructor and on
`embedPostgresGraphs` do the same in code). Lowering it is the useful direction on a small container.
Raising it widens what an omitted `limit` returns, but **not** the wire cap: a client-supplied `limit`
is still rejected above 1000, deliberately, so a single request can't be made arbitrarily expensive
from outside.
Truncation is **not** signalled on the response today — a short page is indistinguishable from the end
of the results, so page with `offset` until you get fewer rows than you asked for. Lowering the bound
below 1000 clamps a client-supplied `limit` silently for the same reason, so page by what you _received_
rather than by what you requested.
### Server-enforced metadata (`enforcedMetadata`)
`ThreadSearchQuery` carries a second metadata subset alongside `metadata`, AND-ed with it. It is set by
the server, never read from a request body, and exists so the auth ownership filter is a `WHERE` clause
instead of a full read plus an in-process pass — see
[agent-protocol.md](./agent-protocol.md#authentication--authorization).
Two subsets rather than one merged object, because a merge would silently drop one side on a key
collision: a caller filtering `owner: "bob"` while the ownership filter requires `owner: "alice"` must
match **nothing**, not one or the other. Both drivers apply it with the same containment semantics as
`metadata`, and the shared conformance suite holds them to that.
### Assistant versioning
`AssistantRepo` carries full CRUD plus a **version history** (LangGraph parity — see the
[assistants endpoints](./agent-protocol.md#assistants)). The model is deliberately simple so nothing
else has to change:
- Each assistant keeps an append-only list of immutable **version snapshots**
(`graph_id`/`name`/`description`/`config`/`context`/`metadata` at that version).
- The **live assistant row mirrors the currently-active version** and carries its `version` number.
So every existing reader — `get`, `list`, the run engine's graph resolution, thread history —
keeps working unchanged; a run always resolves the assistant's _active_ version.
- `create` seeds version 1. `update` mints a new version (`max + 1`) and makes it active.
`setLatest(version)` rolls the live row back to an existing snapshot **without** minting a new one.
`listVersions` returns history newest-first (filterable by metadata, paginated). Deleting an
assistant cascades its versions.
The memory driver holds versions in a second map; Postgres uses an `assistant_versions` table
(`PRIMARY KEY (assistant_id, version)`, `ON DELETE CASCADE` from `assistants`) added in migration
`0003`. Both are exercised by the shared conformance suite.
### Long-term memory in the graph (`getStore()`)
The `store` repo isn't only reachable over the `/store/items` HTTP endpoints — it is also injected
into **every graph run** as a LangGraph
[`BaseStore`](https://docs.langchain.com/oss/javascript/langgraph/persistence),
alongside the checkpointer. A node reads and writes cross-thread memory the LangGraph-native way:
```ts
import { getStore } from "@langchain/langgraph";
async function remember(state) {
const store = getStore(); // the run's SkeinStore.store, as a BaseStore
await store.put(["memories", userId], "prefs", { units: "metric" });
const hits = await store.search(["memories", userId], { query: "units" }); // pgvector in Postgres
return { ... };
}
```
This is what makes skein a faithful drop-in: LangGraph Platform auto-provides a store to graphs, so
a graph that calls `getStore()` runs unchanged on skein. The bridge is `SkeinBaseStore` in
[`@skein-js/langgraph`](https://github.com/skein-js/skein-js/tree/main/packages/langgraph), attached in the run engine the same way the
checkpointer is. Semantic `search` uses pgvector on the Postgres driver and a naive scan on memory —
both come from the same `StoreRepo`, so behavior matches.
### Filtering and namespace traversal
Two ways to narrow a store read: by **content** (`search`'s `filter`) and by **namespace shape**
(`listNamespaces`' `prefix`/`suffix`/`maxDepth`). Both are honoured over HTTP and through
`getStore()`, and both are pinned for every driver by the shared conformance suite.
```ts
await store.search(["users", userId], { filter: { topic: "coffee", score: { $gte: 3 } } });
await store.listNamespaces({ prefix: ["users", "*"], suffix: ["facts"], maxDepth: 3 });
```
**`filter`** applies to the **top-level** keys of an item's `value`, never a nested JSON path — `"a.b"`
is the literal key `"a.b"`. Keys are ANDed, as are multiple operators on one key. The operators are
LangGraph's — `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin` — and a bare scalar means
equality. It is applied **before** paging, so a page is a page of matches.
| Case | Behaviour |
| ------------------------------- | ---------------------------------------------------------------------------------------------------- |
| Ordering (`$gt`/`$gte`/…) | **Numbers only** on both sides. A string, boolean, `null`, object, array or absent key never matches |
| Key absent from `value` | `$ne` and `$nin` match; `$eq`, a bare scalar, `$in` and every ordering operator do not |
| Present JSON `null` | Distinguishable from absent — `{ deleted: null }` means "present and null" |
| Empty operator bag `{}` | States no conditions, so it matches everything |
| Unknown `$op`, non-scalar value | **400** at the boundary, not a silent narrowing |
The numbers-only ordering rule is a **deliberate departure from LangGraph**, which coerces both sides
with `Number()` and so treats `"5" > 3` as true. Postgres cannot reproduce that (`'abc'::numeric`
throws), and faithful reproduction would also make `true > 0` and `null >= 0` true. The divergence is
limited to numeric strings.
**Namespace matching** is positional. `"*"` stands for exactly one segment, `prefix` anchors at the
first and `suffix` at the last, and a path longer than the namespace never matches. A path _shorter_
than the namespace still matches, which is what makes `prefix` select a subtree: `["users","*"]`
matches `["users","1"]` and `["users","1","memories"]` alike. `maxDepth` truncates each match to that
many segments and de-duplicates, before sorting and paging — so `{ maxDepth: 1, limit: 10 }` is ten
_roots_, not the roots of the first ten namespaces.
Namespaces come back in ascending, element-wise order, shorter first on a shared prefix. Per-segment
ordering follows the **driver's** collation (Postgres's database collation; UTF-16 code units in
memory), which agree for ordinary segments and can differ on exotic ones like `"a-b"` vs `"ab"`. The
contract is the element-wise order, not a byte-exact one.
> **Driver authors:** `StoreRepo.listNamespaces` takes a single `StoreNamespaceQuery`
> (`{ prefix, suffix, maxDepth, limit, offset }`) rather than the old `(prefix, pagination)` pair,
> which could not express a wildcard, a suffix or a depth. `listNamespaces(["users"])` becomes
> `listNamespaces({ prefix: ["users"] })`. Like every other list path, it now applies the driver's
> page bound when the caller names no `limit`.
#### Multi-tenancy is yours to define
> `prefix`, `suffix` and `filter` are _request parameters_, not access control. An omitted prefix
> matches every namespace, so **any caller who can reach `POST /store/items/search` can read every item
> in the store** — no wildcard required.
skein offers no store-scoping mechanism of its own, deliberately: who owns what, and how a namespace
encodes it, is the policy that varies most between deployments. Scope it with an `@auth.on.store`
handler that rewrites `value.namespace` — the pattern, and its two limits, are in
[agent-protocol.md](./agent-protocol.md#scoping-the-store).
### Store item TTL
Store items can expire, matching LangGraph's store TTL. Configure it in `langgraph.json` under
`store.ttl` (all durations in **minutes**):
```json
{
"store": { "ttl": { "default_ttl": 1440, "refresh_on_read": true, "sweep_interval_minutes": 60 } }
}
```
- `default_ttl` — lifetime applied to a `put` that doesn't pass its own `ttl`. A `PUT /store/items`
body may include a per-item `ttl` (minutes) that overrides the default for that item.
- `refresh_on_read` (default `true`) — a `get` extends a live item's expiry by its own TTL.
- `sweep_interval_minutes` (default `60`) — how often the background sweeper deletes expired rows.
Expiry is enforced two ways: **lazily** (an expired item reads as absent from `get`/`search`/
`listNamespaces` even before it's swept) and by the **sweeper** (a periodic `DELETE`). With no
`store.ttl` set, items never expire. The sweeper runs in the production runtime (`skein up`/`build`,
and `skein dev --store postgres`); pure in-memory `skein dev` still enforces expiry lazily on read.
### Thread TTL
Threads can expire too, configured under `checkpointer.ttl` — the shape LangGraph Platform documents,
so the same `langgraph.json` works under both. Durations in **minutes**:
```json
{
"checkpointer": {
"ttl": { "default_ttl": 43200, "strategy": "delete", "sweep_interval_minutes": 60 }
}
}
```
- `default_ttl` — lifetime applied to a thread created without its own `ttl`. `POST /threads` accepts a
per-thread `ttl` (minutes) that overrides it, and an explicit `null` **pins** the thread so no TTL
ever collects it. `PATCH /threads/{id}` can change or clear it later; a new value restarts the clock.
- `strategy` — `"delete"` (default) removes the expired thread; `"keep_latest"` retains current state, referenced namespace checkpoints and necessary ancestor writes.
- `sweep_interval_minutes` (default `60`) — how often the sweeper runs.
**Expiry means "may be collected", not "gone"** — the one place this deliberately differs from store
items. An expired thread still reads normally from `GET /threads/{id}` and search until the sweeper
takes it. Hiding it early would make a thread with an in-flight run vanish out from under that run.
The sweeper deletes through the **thread service**, not the driver: an expiring thread's in-flight run
is aborted and its event bus closed first, then its runs and checkpoints go with it. That is why it
lives beside the cron scheduler rather than with the store-item sweeper — a thread is a container, not
a row. It collects a bounded batch per tick and re-ticks immediately while the batch stays full, so a
backlog drains without waiting out the interval.
> **Deleting a thread deletes any cron scheduled on it.** Thread crons cascade with their thread (the
> same `ON DELETE CASCADE` a manual `DELETE /threads/{id}` triggers), so a **thread-scoped**
> [cron](./crons.md) on an expiring thread stops firing — silently, since nothing errors. Either pin
> such threads with `ttl: null` or use a stateless cron, which owns no thread to lose.
The sweeper runs whether or not `checkpointer.ttl` is set, because a per-thread `ttl` can arrive on any
`POST /threads`; the config block supplies the _default lifetime_ and the cadence, not permission to
collect. With neither configured nor requested, nothing has an expiry and each sweep is one indexed
read that finds nothing.
Threads without a configured or per-thread TTL live until explicitly deleted.
> Note this goes **past** LangGraph OSS rather than catching up to it: the open-source
> `@langchain/langgraph-api` accepts `ttl` on thread create and silently drops it — thread expiry is a
> LangGraph Platform feature there.
## Drivers
### `@skein-js/storage-memory` (dev/tests — and the Redis-less production path)
- In-process maps; zero external dependencies.
- Paired with an in-memory queue and a `MemorySaver` checkpointer for `skein dev`.
- `store` semantic search falls back to a naive scan/embedding compare.
- **Not only a dev driver.** `embedPostgresGraphs` uses this queue and event bus whenever no Redis URL
is configured, so its retention bounds apply to real traffic — see
[embedding.md](./embedding.md#going-to-production) for `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN` and
`SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS`.
### `@skein-js/storage-postgres` (prod)
- Backed by `pg`; owns tables for assistants (+ assistant_versions)/threads/runs/store items +
migrations. `store.migrate()` applies them on boot — idempotent, tracked in a `skein_migrations`
table, and serialized by an advisory lock so concurrent boots queue rather than collide. The SQL is
compiled into the package (no filesystem access at runtime), so the driver bundles with zero
externals — see [bundling.md](./bundling.md).
- Uses **`@langchain/langgraph-checkpoint-postgres`** (`PostgresSaver.fromConnString`) for
graph checkpoints — we wrap it rather than reimplement checkpointing.
<https://github.com/langchain-ai/langgraphjs/tree/main/libs/checkpoint-postgres>
- **pgvector** for semantic store search, configured from `langgraph.json`'s
`store.index.{embed, dims, fields}` (see [langgraph-cli-compat.md](./langgraph-cli-compat.md)).
pgvector is **opt-in**: the base schema needs no extension, so skein runs on a stock managed
Postgres out of the box. Only when `store.index` is set does `migrate()` run
`CREATE EXTENSION IF NOT EXISTS vector` and add the `embedding` column — which requires a Postgres
that ships pgvector (see the provider table in [deploy.md](./deploy.md#1-a-postgres)).
#### Indexes, and the one thing to watch on upgrade
Migration `0005_performance_indexes` adds the indexes the list/search paths need: composite
`(created_at, thread_id)` and `(updated_at, thread_id)` on `threads` matching the `ORDER BY … , <id>`
the queries actually emit, `(status, created_at)` for the status filter, GIN `jsonb_path_ops` on
`threads.metadata` and `assistants.metadata` for the `metadata @> …` containment the auth ownership
check performs on **every** request, `runs (thread_id, created_at)`, and `store_items (created_at, key)`.
`runs_thread_id_idx` is dropped, superseded by the composite with the same leading column.
They are built with **`CREATE INDEX CONCURRENTLY`**, so the boot migration does not hold a
write-blocking lock while indexing an existing table — a plain `CREATE INDEX` on a large `threads`
would stall writes for minutes, at boot, during a rolling deploy. Concurrency has one cost: such a
migration cannot be transactional, so a failure partway leaves some indexes created and the ledger row
unwritten, and the next boot retries the whole migration. Every statement is `IF NOT EXISTS` for that
reason.
**If an index build is interrupted** (the pod is killed mid-migration, say), Postgres leaves an
_invalid_ index behind — and `IF NOT EXISTS` will then skip it forever, so queries silently never use
it. Nothing breaks; it just stays slow. To check and fix:
```sql
-- Any invalid indexes?
SELECT c.relname FROM pg_class c JOIN pg_index i ON i.indexrelid = c.oid WHERE NOT i.indisvalid;
-- Drop each one; the next boot rebuilds it.
DROP INDEX CONCURRENTLY <name>;
```
#### Semantic search: exact by default, HNSW opt-in
With `store.index` configured, semantic search ranks by cosine distance over an **unindexed**
`embedding` column — an exact scan of every row. Correct, and fine until the store is large. Add
`"hnsw": true` to opt into an HNSW index:
```json
{
"store": {
"index": { "embed": "openai:text-embedding-3-small", "dims": 1536, "hnsw": true }
}
}
```
Off by default deliberately: HNSW is an **approximate** nearest-neighbour index, so turning it on
changes which rows a search returns. That is a semantic change, not something to inherit from an
upgrade.
Enabling it also **pins the column to `vector(dims)`** — pgvector cannot index a dimensionless
`vector`, which is how the column is created so the base schema works without knowing `dims`. If rows
already exist at a different dimensionality (an embedder or model change), boot fails with an error
saying so rather than a raw Postgres one; re-embed or clear those rows before enabling it.
Three things to know before turning it on:
- **The first boot is not free.** Pinning the column rewrites the table under `ACCESS EXCLUSIVE`, which
blocks reads and writes on `store_items` for the duration and queues behind any long-running query
already touching it. The index build that follows is concurrent and does _not_ block, but it does
hold boot until it finishes. If pgvector warns that the graph no longer fits in
`maintenance_work_mem`, raise it — that is the biggest lever on build time.
- **Namespace-filtered search needs pgvector ≥ 0.8.** HNSW selects a fixed candidate set
(`hnsw.ef_search`) and the namespace predicate is applied _after_ it, so a prefixed search can return
fewer rows than exist — or none. skein sets `hnsw.iterative_scan = strict_order` on every connection
to prevent that, which requires pgvector 0.8. On an older server it warns once at startup and the
post-filter behaviour stands; leave `hnsw` off there.
- **Turning it back off does not unpin the column.** `hnsw: false` skips the `ALTER`, it does not
reverse it, so a later `dims` change still fails on the pinned column. Undo it by hand:
`ALTER TABLE store_items ALTER COLUMN embedding TYPE vector;`
An interrupted build leaves an invalid index; the next boot detects it, drops it concurrently and
rebuilds. That check exists because pinning a column whose index is invalid rebuilds that index
_inline_ and non-concurrently, holding `ACCESS EXCLUSIVE` for the whole build.
```ts
import { PostgresSaver } from "@langchain/langgraph-checkpoint-postgres";
const checkpointer = PostgresSaver.fromConnString(process.env.POSTGRES_URI!);
await checkpointer.setup(); // idempotent migrations for checkpoint tables
```
### Bringing your own store (`store.adapter`)
Long-term memory is the one repo you can swap without implementing the other five. Point
`store.adapter` at a `"path:export"` and skein uses it for `/store/*` **and** for the `getStore()` handed
to every graph run; assistants, threads, runs, crons and idempotency keep using the configured driver.
```jsonc
{ "store": { "adapter": "./src/my-store.ts:store" } }
```
```ts
// src/my-store.ts — LangChain's own JS long-term-memory guide builds on this exact class.
import { PostgresStore } from "@langchain/langgraph-checkpoint-postgres/store";
export const store = PostgresStore.fromConnString(process.env.POSTGRES_URI!);
await store.setup(); // top-level await: `store.adapter` imports a *ready* store
```
The export may be **either** a LangGraph
[`BaseStore`](https://docs.langchain.com/oss/javascript/langgraph/persistence) — `PostgresStore`,
`InMemoryStore`, your own — or a skein `StoreRepo`. They are told apart structurally (`BaseStore` has
`batch`), and a mis-shaped export fails at **startup** naming the missing method, rather than at the first
request that reaches it. Prefer `BaseStore`: it is the wider ecosystem, and `PostgresStore` brings pgvector
with HNSW _and_ IVFFlat plus a `"text" | "vector" | "hybrid" | "auto"` search mode — hybrid search being a
capability skein's own driver does not have.
**skein re-applies its own semantics rather than forwarding them**, because forwarding would be wrong:
`InMemoryStore`'s `search` matches namespace prefixes as a raw _string_ (so `["users"]` also matches
`["users2", …]` — a cross-tenant read the moment a prefix is derived from a principal), ignores `query`
without a vector index, and coerces filter operands with `Number()`. So the adapted store supplies a
candidate set and its vector ranking; skein applies filtering, ordering and paging on top. The price is
over-fetching. The shared conformance suite runs against both an adapted `InMemoryStore` and a real
`PostgresStore`.
Four things worth knowing before you switch:
| | |
| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`put` does a read-back** | `StoreRepo.put` returns the stored item; `BaseStore.put` returns `void`. So the adapter writes then reads, costing a round trip — and it is **racy**: a concurrent write to the same key returns the other writer's item. |
| **TTL is partly expressible** | `store.ttl.default_ttl` works — the adapter stamps it onto every write. But `refresh_on_read` cannot be expressed through `BaseStore.get`, and it **defaults to enabled**, so `store.ttl` is refused unless you set `refresh_on_read: false` and accept write-based expiry. A store with no `sweepExpiredItems()` (`InMemoryStore`) refuses `store.ttl` outright; `PostgresStore` has it. A silently ignored retention policy is worse than a boot error. |
| **A prefix-less search fans out** | Some stores reject an empty namespace prefix (`PostgresStore` does), so `search({})` enumerates namespaces and searches each. Scores are re-sorted globally afterwards, so semantic ranking still holds — but give a prefix when you can. |
| **The store may have its own rules** | `PostgresStore` rejects namespace labels containing `.`, `%`, `_` or `\`, and a root label of `"langgraph"`. Items written under such a namespace by another driver are not reachable through it. |
**Your store owns its items.** skein imports _into_ an adapter — `skein import-langgraph`, a restored
`.skein/dev-state.json`, the one-time `langgraph dev` auto-import — but never snapshots back out of it,
because writing whatever store you brought into `.skein/dev-state.json` every few seconds is not something
to do to somebody's database. So an adapted `InMemoryStore` loses its items across a `skein dev` restart
while a `PostgresStore` keeps them. `skein dev` prints this at startup rather than letting you find it by
losing items.
**`store.index` is refused alongside an adapter.** It configures pgvector on the store the adapter
replaces, so it could only ever have no effect on search. Configure the index on your own store instead —
`PostgresStore` and `InMemoryStore` both take one in their constructor.
In code (no `langgraph.json`), the same seam is `ProtocolDeps.storeItems` — wrap a `BaseStore` with
`fromBaseStore` first. Pass it as its own field rather than composing `{ ...store, store: mine }`: the
bundled drivers expose `maxPageSize` and `durable` as class **getters** over **private** fields, so a
spread silently loses them and a prototype clone throws on read. `withStoreItems` is the supported way to
do it by hand.
## Checkpointer selection
| Storage selection | skein-js uses |
| -------------------------------- | --------------------------------------------------- |
| Memory (the default `skein dev`) | `MemorySaver` |
| Postgres | `PostgresSaver` |
| SQLite store/queue pair | `SqliteSaver` sharing the owned resource connection |
You rarely wire these drivers by hand. [`@skein-js/runtime`](https://github.com/skein-js/skein-js/tree/main/packages/runtime) assembles the
`PostgresSkeinStore` + `PostgresSaver` + Redis queue/bus (and their `dispose()`) for you, two ways:
**`buildRuntime`** from a `langgraph.json` (the `skein dev`/`skein up` path), and
**`embedPostgresGraphs`** from a graph you hold in code (the durable sibling of `embedInMemoryGraphs` —
see [embedding.md](./embedding.md#going-to-production)).
Protocol resources (`SkeinStore`) stay separate from LangGraph checkpoints, which is what allows an
in-memory dev experience with no database while the checkpoint format stays 100% LangGraph-native — so
history endpoints and interrupt/resume behave exactly as LangGraph expects.
For SQLite, use `embedSqliteGraphs` from `@skein-js/local`, or the CLI's explicit SQLite pair.
It assembles resources, the queue, and the upstream saver with one file owner. See
[local deployment](./local-deployment.md) for lifecycle and native packaging.
## Local SQLite storage
[`@skein-js/storage-sqlite`](./local-deployment.md) implements the full resource store and a persisted
run queue in one exclusively owned file. `@skein-js/local` assembles it with LangGraph's SQLite saver.
Unclaimed work persists; abandoned attempts become failures without automatic input replay.
## Per-operation indexing and retention
The bundled memory, SQLite and Postgres stores implement optional `StoreRepo.putWithOptions`:
```ts
await store.putWithOptions!(namespace, key, value, { index: ["title"], ttl: null });
await store.get(namespace, key, { refreshTtl: false });
await store.search({ prefix: namespace, query: "meeting", refreshTtl: true });
```
`index: false` stores the item without making it a text/vector search candidate. A field list limits
indexed content; `$` includes the whole value. Exact key reads still return the complete item.
An explicit nullable TTL disables the configured default for that write. Existing `put` and its
`ttl?: number` options remain source compatible. Unsupported custom stores reject requested
advanced controls. The LangGraph bridge forwards indexing, write TTL and read/search refresh
options, including batch operations. An external BaseStore must expose the corresponding controls;
Skein does not emulate persistent expiry with process-local timers.
Thread TTL retains deletion as the default. An opt-in `keep_latest` strategy prunes old history
without deleting the thread, root checkpoint, referenced child state or pending writes. The retention
window slides when state changes; the sweep skips an active run and rearms its next check after
pruning. A null TTL pins the thread. The SQLite and Postgres migrations preserve existing pins:
older rows without a recorded TTL remain pinned rather than acquiring an unexpected expiry.
Pruning and rollback use LangGraph’s public saver methods. A scratch copy is written under a random,
operation-specific ID, checked for existing checkpoints, before erasure. Fixed names such as
`<thread>::skein-rollback` and `<thread>::skein-prune` are never used as scratch IDs because they may
belong to real threads. If replay fails, the backup's actual ID is included in the error for operator
recovery; retain that copy until recovery is complete. These operations require the same
thread execution gate as state writes. They preserve namespace coordinates when copying history.
Checkpoint pruning retains current root/child namespaces, explicit checkpoint references and
pending writes. Delta channels may also require ancestor writes until a full snapshot; those
necessary ancestors remain readable after prune and copy. `keep_latest` removes expendable history,
not checkpoint data required to reconstruct current state. Exact payload accounting is available at
`GET /threads/{thread_id}/storage`; see [diagnostics](./observability.md#http-diagnostics).
### Retention readback
Bundled memory, SQLite, and Postgres drivers enrich thread resources with optional `ttl` (minutes),
`expires_at`, and `ttl_strategy`. Store `Item` and `SearchItem` resources expose optional `ttl` and
`expiresAt` in repository results; HTTP responses serialize expiry as `expires_at`. An absent field
means the driver cannot report it; `null` means known no expiry. Thread expiry means eligible for
sweeping, rather than a promise of immediate deletion. Refresh-enabled reads return the new expiry.
These fields reuse existing storage columns, require no migration, and add no required custom-driver
methods. Passive tools can use `refreshTtl: false` to inspect without extending a sliding lifetime.
---
<!-- source: docs/memory.md -->
# Agent memory
**What this gives you:** the patterns for giving an agent memory on skein, and the traps that are not
obvious until they bite. There is no `@skein-js/memory` package to install — memory is something you build
in your graph out of pieces that already exist, and this page is the map of which pieces, and why.
That is a deliberate choice. Memory is _agent behaviour_, and skein's job is durable persistence, the
queue, the adapters and the CLI. Everything below is ~100 lines in your own graph, portable to LangGraph
Platform because it uses `getStore()` and nothing skein-specific. Where skein does contribute — durable
storage, semantic search, schedules — it is called out. Per-owner isolation is **not** on that list: it is a
policy your `@auth.on.store` handler decides, and [Multi-tenant memory](#multi-tenant-memory) is how.
## What you already have
| Need | Use | From |
| ------------------------------- | ---------------------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| Durable thread state | the upstream checkpointer — `SqliteSaver`, `PostgresSaver`, or `MemorySaver` | skein selects it for the chosen storage ([storage.md](./storage.md)) |
| Trimming a long conversation | `trimMessages`, `filterMessages`, `RemoveMessage` | `@langchain/core` |
| Summarizing a long conversation | `summarizationMiddleware`, `contextEditingMiddleware` | the `langchain` v1 package |
| Cross-thread memory store | `getStore()` — namespace/key items, vector search | skein injects it into every run |
| Semantic search over memories | `store.index` (pgvector, HNSW optional) | skein ([storage.md](./storage.md#drivers)) |
| Per-owner isolation | an `@auth.on.store` handler that roots the namespace | you ([agent-protocol.md](./agent-protocol.md#authentication--authorization)) |
| Scheduled background extraction | a cron firing an extractor graph | skein ([crons.md](./crons.md)) |
| Memory _shapes_, tools, dedup | this page | you, in ~100 lines |
## Short-term memory is already solved
Worth stating plainly, because "durable short-term memory" sounds like a gap and is not one.
**Durability** is the checkpointer. `SqliteSaver` or `PostgresSaver` persists a thread's state and history, so a
conversation survives restarts and resumes from an interrupt; thread TTL collects the ones that go stale.
skein selects it for you — nothing to build.
Choose [SQLite](./local-deployment.md) for one local Node owner, or Postgres for a server. SQLite
supplies text/filter search; semantic recall requires Postgres indexing or an external semantic store.
**Context management** — keeping a long thread inside the model's window — is upstream too:
```ts
import { createAgent, summarizationMiddleware } from "langchain";
const agent = createAgent({
llm,
tools,
middleware: [summarizationMiddleware({ model: llm, trigger: { tokens: 4000 }, keep: 20 })],
});
```
Do **not** write your own trimming loop. If you keep a running summary yourself, keep it in **graph
state** rather than the store: state is checkpointed, thread-scoped, and rolls back with the checkpoint,
none of which is true of a store item.
## Long-term memory: profile and collection
Cross-thread memory is `getStore()` — namespace/key JSON documents, the same store LangGraph Platform
auto-provides. Two shapes cover almost everything, and the choice is really "one document or many":
**Profile** — one continuously-updated document. Good for stable facts with one current value: timezone,
tone, home airport. Read it whole, merge a patch, write it back.
**Collection** — many documents accumulated over time. Good for open-ended facts: "prefers morning
meetings", "allergic to shellfish". Append, and retrieve by relevance rather than reading all of them.
```ts
const NAMESPACE = (userId: string) => [userId, "memories"] as const;
// Profile: read-modify-write one document.
async function mergeProfile(store: BaseStore, userId: string, patch: Record<string, unknown>) {
const current = (await store.get([userId], "profile"))?.value ?? {};
await store.put([userId], "profile", { ...current, ...patch });
}
```
**Pick one merge rule and write it down.** The rule matters less than having one; without it the profile
becomes unpredictable, which is the failure LangChain's own memory guide warns about. A rule that works:
_shallow, patch wins, arrays unioned, `null` deletes, nested objects replaced._ Replace rather than
deep-merge — deep merge makes deletion inexpressible.
Two things to know:
- **There is no compare-and-swap.** `BaseStore` has none, so read-modify-write is last-writer-wins. On
skein, runs on one thread are serialized by the execution lock, so concurrent merges only arise across
threads for the same user. Do not build an optimistic-retry counter — without CAS it does not close the
race and implies a guarantee that is not there.
- **Keep collection fields top-level and scalar.** `search`'s `filter` reads the _top-level_ keys of an
item's `value` and takes scalars, so `{ content, kind, createdAt }` filters and `{ meta: { kind } }`
does not. `tags: string[]` is storable but not filterable (`$in` asks the opposite question). Full
operator semantics in [storage.md](./storage.md#filtering-and-namespace-traversal).
## The dedup trap
**The most important paragraph on this page.** Without dedup, an agent that saves a memory each turn
accumulates the same fact dozens of times and recall degrades into noise. The obvious fix — semantic
search for a near-duplicate before writing — **inverts** on most substrates:
| Substrate | `score` on a text search |
| ------------------------------------------------------ | ------------------------ |
| `@skein-js/storage-memory` (dev) | **`1` for every hit** |
| `@skein-js/storage-postgres` **without** `store.index` | **`1` for every hit** |
| `@skein-js/storage-postgres` **with** `store.index` | real cosine similarity |
| LangGraph `InMemoryStore` without an index | no `score` field at all |
So `score >= 0.9 means duplicate` classifies _everything_ as a duplicate on two of three substrates, and
your agent **silently stops recording memories**. It will not error. You will notice weeks later.
Two rules:
**1. Dedup on content, not similarity.** A content-addressed key makes exact dedup free, with no read at
all — saving the same sentence twice upserts one row instead of appending a second:
```ts
async function memoryKey(content: string): Promise<string> {
const normalized = content.trim().toLowerCase().replace(/\s+/g, " ");
const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(normalized));
return `mem-${[...new Uint8Array(digest).slice(0, 8)].map((b) => b.toString(16).padStart(2, "0")).join("")}`;
}
```
Use `globalThis.crypto.subtle`, not `node:crypto` — it is present on Node ≥20, Bun, Deno and workers, so
the same graph still runs on an edge runtime.
**2. Treat semantic dedup as opt-in, requiring `store.index`.** And check the scores are meaningful before
trusting them — at least one candidate scoring **strictly below 1** — falling back to exact dedup
otherwise. State the safe direction out loud: **dedup errs toward writing.** A duplicate is recoverable;
a dropped memory is not.
One more, on cost: on Postgres _without_ `store.index`, a `query` search is an unbounded full-table read
filtered in JS. Semantic dedup on every write there reads the whole `store_items` table each time.
## Recall
Retrieval is `store.search` with the namespace and filter already right:
```ts
const relevant = await store.search([userId, "memories"], { query: latestUserMessage, limit: 5 });
```
**When to recall is your graph's decision, and skein has no opinion.** Two patterns, both fine:
- **Auto-inject** — fetch relevant memories before each model call and fold them into the system message.
Personalization does not depend on the model remembering to ask. This is what
[`examples/chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) does.
- **A recall tool** — the model calls `search_memory` when it decides it needs to. Fewer tokens per turn,
but it only works when the model chooses well.
If you expose memory as a **tool**, bind the namespace server-side and give the model no namespace
parameter at all:
```ts
// The model may choose *which collection*, from an enum — never which namespace.
const userId = config.configurable?.langgraph_auth_user_id; // never model output
```
That is the pattern people get wrong: a namespace the model can name is a namespace the model can escape.
## Multi-tenant memory
Memory is per-user by definition, so this matters more here than anywhere else in skein.
`prefix`, `suffix` and `filter` are **request parameters, not access control**. An omitted prefix matches
every namespace. The control is an **`@auth.on.store` handler** that rewrites `value.namespace` to root the
caller — LangGraph's own idiom, and skein honours it. See
[agent-protocol.md](./agent-protocol.md#authentication--authorization).
It covers the HTTP surface. **`getStore()` inside a graph has no request and no principal**, so nothing
guards it for you. Build its namespace from `config.configurable.langgraph_auth_user_id` (server-injected and
unspoofable), never from model output, use the same encoding your handler uses — and encode it: an identity
containing `.` splits into two namespace segments on `GET /store/items`, and `PostgresStore` rejects `.`,
`%`, `_` and `\` in a label outright.
## Writing memories in the background
Extracting memories inline costs the user's turn a model call. Two ways to move it off the hot path, both
working today with no new skein surface:
**A cron.** Point a [schedule](./crons.md) at an extractor graph — an ordinary graph you write, with your
model and your prompt — and it runs on a cadence, durably, across restarts and instances:
```jsonc
{ "graphs": { "agent": "./src/agent.ts:graph", "extract-memories": "./src/extract.ts:graph" } }
```
**A debounced follow-up run.** Closer to real-time: when a turn finishes, create a _stateless background
run_ of the extractor with `after_seconds`, and cancel this thread's pending one first so extraction fires
once when the conversation pauses rather than once per turn. `after_seconds` is backed by the queue, so a
pending extraction is a **row in the store** and survives the process that scheduled it — which is the part
an in-process scheduler (Python `langmem`'s `ReflectionExecutor`, say) cannot give you without a server
behind it.
```ts
// Cancel the pending extractor for this thread, then schedule a fresh one.
const inflight = await client.runs.list(threadId);
for (const run of inflight.filter((r) => r.metadata?.extractor && r.status === "pending")) {
await client.runs.cancel(threadId, run.run_id);
}
await client.runs.create(null, "extract-memories", {
afterSeconds: 30,
input: { thread_id: threadId },
metadata: { extractor: true },
});
```
The one thing neither expresses: a graph node cannot know its own run _settled_, so "extract only from
successful runs, reliably even on error and cancellation paths" is not reachable from inside the graph. A
server-side settled-run trigger would cover it; it is
[deferred](./roadmap.md), because everything above works without it and the gap is narrow.
## See also
- [storage.md](./storage.md) — the store itself: drivers, `getStore()`, filters, TTL, scoping, BYO store
- [crons.md](./crons.md) — schedules, for the cron extraction pattern
- [`examples/chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) — a working research assistant with memory and recall
- [LangChain's memory guide](https://docs.langchain.com/oss/javascript/concepts/memory) — the
profile/collection framing this page builds on
---
<!-- source: docs/runs-and-redis.md -->
# Runs & Redis
This doc covers how skein-js executes runs and how it scales horizontally — modeled on
[aegra](https://github.com/aegra/aegra)'s worker + Redis architecture, adapted to Node.
> **Reuse note:** `@skein-js/redis` is the run **queue + pub/sub** — the piece LangGraph OSS
> does not provide (the open [`@langchain/langgraph-api`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/langgraph-api)
> server runs runs in-process, in-memory). It is _not_ a checkpointer; for Redis-backed
> checkpoints use `@langchain/langgraph-checkpoint-redis`. See [reuse.md](https://github.com/skein-js/skein-js/blob/main/docs/reuse.md).
## Run modes
The [Agent Protocol](./agent-protocol.md) defines three ways to execute a graph:
| Mode | Endpoint | Behavior |
| -------------- | -------------------------------------------- | ---------------------------------------------------------- |
| **wait** | `POST /runs/wait`, `GET /runs/{id}/wait` | Run to completion, return final output. |
| **stream** | `POST /runs/stream`, `GET /runs/{id}/stream` | [SSE](./streaming.md) as output is produced. |
| **background** | `POST /threads/{id}/runs` | Enqueue; poll (`GET /runs/{id}`) or join its stream later. |
A **per-thread concurrency guard** prevents two active runs on the same thread (the protocol's
concurrency-control requirement). That is a different thing from
[run concurrency](#run-concurrency) below, which is how many runs on _different_ threads one
instance executes at once.
## Run engine
`@skein-js/agent-protocol` owns a run engine that:
1. Resolves the target graph via [`@skein-js/config`](./langgraph-cli-compat.md).
2. Persists a run row through [`SkeinStore`](./storage.md) (`pending → running → success/error`).
A failed run also records _why_ on the row, so `GET /threads/{tid}/runs/{rid}` can still explain
it afterwards — see [errors and logging](./errors-and-logging.md).
3. Invokes the graph (`invoke` for wait, `stream` for streaming), threading the LangGraph
**checkpointer** so state/history persist and **interrupt/resume** (human-in-the-loop)
works.
4. Publishes stream frames to subscribers (local bus or Redis pub/sub).
## Queue drivers
The engine talks to small `RunQueue` / `RunEventBus` interfaces (`@skein-js/core`) with queue
drivers for memory, SQLite, and Redis. `RunQueue` is **processor-driven**: `enqueue(run)` adds a job and
`consume(process)` registers a worker that drains the queue — so the same run worker code drives
each driver. Crash behavior depends on the queue: Redis can redeliver stalled work; SQLite records
abandoned attempts as failures; memory loses queued work on process exit. The worker skips runs
already terminal in the store.
`enqueue` takes an optional `{ delayMs }` — the run-create
[`after_seconds`](./agent-protocol.md). The driver holds the run until it comes due, which is why a
delayed run cannot be picked up early. The drivers dedupe a
re-enqueue of a run that is still waiting out its delay, so the cron scheduler's outbox sweep — which
re-enqueues anything it cannot prove reached the queue — can never cut a delay short or schedule the
same run twice. A delayed run is durable exactly as far as its driver is: see below.
### In-memory (dev)
- Single-process queue + event bus. No external services.
- Used by `skein dev` so nothing beyond Node is required locally.
- An `after_seconds` delay is a local timer, so it is lost on restart — like every other run already
sitting in this queue, which is in-process too.
### SQLite (local applications)
- `@skein-js/storage-sqlite` persists the queue in the same file as protocol resources, owned by
one process. `@skein-js/local` adds the upstream checkpointer and process-local event bus.
- Run creation and background-job admission commit together. Unclaimed jobs retain their original
due times across reopening; duplicate enqueue does not shorten a delay.
- Claimed or executing attempts abandoned by a process become `ProcessLostError` failures. Skein
does not replay their input or resume checkpoints automatically. The application decides.
- Live stream frames stay in memory. This is a Node 22/24/26 option for local apps and single-owner
services, with no distributed worker support. See [local deployment](./local-deployment.md).
### `@skein-js/redis` (shared servers)
- **Job queue ([BullMQ](https://docs.bullmq.io))** — background runs are enqueued in Redis; worker
processes across instances consume and execute them. BullMQ provides retries, backoff, and
concurrency out of the box.
- **Crash recovery** — a stalled job (its worker died mid-run) is moved back to the queue by
BullMQ's stalled-job check and retried, so runs survive restarts.
- **Delayed runs** — an `after_seconds` delay is handed to BullMQ's own delayed set, which lives in
Redis and promotes the job when it comes due, so a scheduled run outlives the process that created it.
- **Cross-instance pub/sub** — run stream frames are published to a Redis Stream + channel so a
client connected to instance B can join a run executing on instance A (see [streaming.md](./streaming.md)).
- **Cross-instance cancellation** — a `RunAbortChannel` over Redis pub/sub carries the _stop now_ signal
to whichever instance is executing a run, so a cancel routed to the wrong replica still stops the
graph. The cancel itself is durable in the run row before the message is published, so a dropped
message costs promptness rather than correctness. See
[deploy.md](./deploy.md#scaling-past-one-instance).
### What a frame costs
Publishing sits inside the graph's own loop — the engine awaits it per chunk — so at token
granularity its cost is paid per token. It is **one pipelined round trip**: `XADD` and `PUBLISH`
batched together, with the frame serialized once for both. The stream's TTL is refreshed every 256
frames and on close, rather than on every frame, since the window only has to outlive the run.
All in-flight subscribers share **one** pub/sub connection, with a `SUBSCRIBE` per run rather than a
connection per stream — an instance serving 500 SSE streams holds two Redis sockets, not 501.
Replay is paged (`XRANGE … COUNT`), and a reconnecting subscriber resumes from the last stream id it
read rather than re-reading the whole stream.
Two knobs bound what this costs:
| Variable | Default | Purpose |
| ---------------------------- | ------- | --------------------------------------------------------------------------------------- |
| `SKEIN_REDIS_STREAM_MAXLEN` | 10000 | Approximate cap on a run's stream, in frames. `0` disables trimming (TTL only). |
| `SKEIN_STREAM_BUFFER_FRAMES` | 512 | Frames one subscriber may queue before it is judged too far behind and its stream ends. |
`MAXLEN` exists because the 1-hour TTL bounds a stream in _time_ but not in size, and a chatty graph
can produce a very large one well inside an hour. Trimming is approximate (`~`), which lets Redis trim
whole nodes instead of walking the stream on every append.
A subscriber whose buffer overflows has its stream ended rather than being allowed to grow without
bound behind a reader that cannot drain it; the client reconnects with `Last-Event-ID` and replays from
the stream, which is what the stream is for.
This is the same shape aegra uses (Redis job queue + pub/sub, crash recovery, Postgres
checkpoints) — <https://github.com/aegra/aegra>.
## Run concurrency
How many **queued** (background) runs one instance executes at once. It defaults to **10**, matching
the LangGraph CLI's `--n-jobs-per-worker`, so a project moving off `langgraph dev` keeps its
throughput. Inline `wait`/`stream` runs never touch the queue and are unaffected.
Concurrency is the knob with the widest blast radius: it multiplies memory, Postgres connections, and
in-flight graph state at once. Size it together with `PG_POOL_MAX` — see
[performance.md](./performance.md#sizing), and note `skein start` warns at boot when the two disagree.
**One worker, N concurrent runs.** skein runs a _single_ background worker whose consumer executes up
to N runs at a time — which is why the startup banner says `Starting 1 worker, up to 10 concurrent
runs` rather than `langgraph dev`'s `Starting 10 workers` (it really does spawn 10 loops; we don't).
The observable behavior is the same.
> **Upgrading from ≤ 0.9.0?** This default changed: background runs used to execute strictly one at a
> time. Nothing about your code or config needs to change, but each instance now does up to 10 runs
> concurrently — so check that your Postgres pool has headroom (see
> [pool sizing](./deploy.md#connection-budget)), and note that background runs on one thread
> using `multitask_strategy: "enqueue"` no longer execute in strict enqueue order. Set
> `SKEIN_RUN_CONCURRENCY=1` (or `--concurrency 1`) to restore the previous behavior exactly.
Three ways to set it, highest precedence first:
| Surface | How |
| -------------- | ---------------------------------------------------------------------------- |
| CLI flag | `skein dev --concurrency 4` / `skein start -n 4` (`--n-jobs-per-worker` too) |
| Environment | `SKEIN_RUN_CONCURRENCY=4`, or the LangGraph-compatible `N_JOBS_PER_WORKER=4` |
| Adapter option | `createExpressServer({ deps, worker: { maxConcurrency: 4 } })` |
An explicit value wins, but the environment is still validated — so a typo'd
`SKEIN_RUN_CONCURRENCY` fails loudly at boot instead of being silently ignored. The environment is
the path that reaches a container: add it to the `skein up` compose `environment:` block or your
PaaS config.
**Per-thread ordering is unaffected.** Two runs on the same thread are serialized by the engine's
execution lock at _every_ concurrency, so the per-thread guard above holds regardless.
<span id="head-of-line-blocking"></span>
### Dispatch by thread
Bundled memory, SQLite and Redis queues retain a busy thread's pending head while dispatching work
for independent threads. A busy execution claim does not hold a worker execution slot or spend a
processor-failure attempt. The execution claim remains responsible for serialization, including
against inline runs and other instances. The default `multitask_strategy` remains `"reject"`;
`"enqueue"` is the strategy that builds a thread backlog.
The worker uses the optional `RunQueue.consumePartitioned` and, when a backend gate is configured,
`ThreadExecutionGate.tryAcquire` capabilities. Existing third-party queues or gates without either
capability retain ordinary blocking dispatch. `tryAcquire` returns a lease or `undefined` for a busy
thread/capacity; backend errors still reject. A partitioned processor returns `true` only after handling
a run and `false` when execution never started. A false result retains identity and ordering position.
Redis continues using BullMQ jobs, delay floors, retries and stalled-job recovery. Internal partition
metadata registers jobs when first eligible, parks followers, and promotes the next head after
settlement. Promotion applies only to Skein-parked attempts; native BullMQ retry backoff remains in
force. Busy heads retry after 50 ms; followers have a 60-second parking backstop. A bounded sweep
reconciles up to 100 partitions per second after crashes or removals. Registration never promotes a
run before its original `after_seconds` delay. Empty metadata is removed. Existing queued jobs register
lazily, so an upgrade does not require flushing the queue. Coordinate worker upgrades to obtain the
new dispatch behavior throughout the deployment; the execution claim remains the safety boundary.
SQLite keeps the original job sequence and due time, with no schema migration. Abandoned attempts
still fail without automatic replay. Deferred work is not a new recovery policy.
Relatedly, **ordering across background `"enqueue"` runs is not guaranteed above concurrency 1**:
several are dequeued at once and race for the thread's claim. This matches LangGraph at
`N_JOBS_PER_WORKER=10`, whose N worker loops have no cross-loop ordering guarantee either. If you
need strict FIFO across background runs on one thread, set concurrency to 1.
**Concurrency vs. replicas.** Raise concurrency when you have many independent threads and runs are
I/O-bound (model calls). Add instances when runs are CPU-bound, or when threads are long-lived and
serialized. Note each concurrent run holds a Postgres connection — see the pool-sizing note in
[deploy.md](./deploy.md#connection-budget) — and, on the Postgres driver, a second one for its
per-thread execution claim, held for the whole run.
## Deployment topology (`skein up`)
```mermaid
flowchart TB
C(["clients · SSE"]) --> A["instance A"]
C --> B["instance B"]
A --> R[("Redis<br/>queue + pub/sub")]
B --> R
A --> P[("Postgres<br/>checkpoints · resources<br/>pgvector")]
B --> P
class R accent
```
`skein up` brings this stack up via Docker Compose. Horizontal scaling is verified by
starting a run on instance A and joining its SSE stream from instance B through Redis (see
[testing.md](https://github.com/skein-js/skein-js/blob/main/docs/testing.md)).
To run the same topology on a hosted platform, the generated image is PaaS-friendly (binds the
injected `$PORT`, non-root, `/ok` health probe, graceful `SIGTERM`) — see
[deploy.md](./deploy.md) for what every platform needs, plus per-platform guides for Cloud Run,
Railway, Fly.io, Render, AWS, Kubernetes and a plain VPS.
---
<!-- source: docs/crons.md -->
# Cron / scheduled runs
Schedules that fire a run on a cadence — LangGraph Platform's **Crons** resource, served by skein-js
and driven by the official `@langchain/langgraph-sdk` client.
> **Compatibility note:** crons are **not** part of the open
> [Agent Protocol](https://github.com/langchain-ai/agent-protocol) spec — its `openapi.json` has no
> cron paths. They are a LangGraph Platform / LangSmith Deployment extension, and a paid-tier one.
> The OSS [`@langchain/langgraph-api`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/langgraph-api) server
> registers the routes but every handler throws `500 Not implemented`, and its `GET /info` reports
> `crons: false`. skein implements them against the LangSmith Deployment OpenAPI spec plus the SDK's
> TypeScript types — the same oracle the rest of the wire surface uses ([reuse.md](https://github.com/skein-js/skein-js/blob/main/docs/reuse.md)).
**A cron is a _cadence_, not a delay.** To run something once, a little later, pass
[`after_seconds`](./agent-protocol.md) on an ordinary run create — no cron row, nothing to clean up
afterwards. Reach for a cron when the run should keep happening.
## Quick start
```ts
import { Client } from "@langchain/langgraph-sdk";
const client = new Client({ apiUrl: "http://localhost:2024" });
// Every weekday at 09:00 New York time, on a fresh thread each fire.
const cron = await client.crons.create("agent", {
schedule: "0 9 * * 1-5",
timezone: "America/New_York",
input: { messages: [{ role: "user", content: "Summarize yesterday's issues." }] },
});
// What has it produced? A stateless cron makes a thread per fire, each tagged with the cron id.
const threads = await client.threads.search({ metadata: { cron_id: cron.cron_id } });
const runs = await client.runs.list(threads[0].thread_id);
await client.crons.update(cron.cron_id, { enabled: false }); // pause
await client.crons.delete(cron.cron_id);
```
`assistant_id` accepts a UUID **or a graph name** from your `langgraph.json` — a graph name resolves
to the assistant skein auto-registers for it.
## Stateless vs thread crons
| | Stateless (`crons.create`) | Thread (`crons.createForThread`) |
| --------------- | --------------------------------------------- | ------------------------------------------ |
| Thread | A fresh one per fire | The one you named, reused every fire |
| State | Starts clean each time | Accumulates across fires |
| Default cleanup | `on_run_completed: "delete"` | n/a — the thread is yours |
| Concurrency | Cannot collide (each fire has its own thread) | `multitask_strategy` defaults to `enqueue` |
Use a **stateless** cron for independent jobs (a nightly digest). Use a **thread** cron when each run
should see what the previous ones did (a long-running monitor).
> **`on_run_completed` defaults to `"delete"`**, matching LangGraph — and deliberately _unlike_
> skein's own `on_completion` default of `"keep"` for one-off stateless runs. A schedule is the case
> where the difference bites: a five-minutely cron keeping every thread accrues over a hundred
> thousand of them a year, with nobody watching. Pass `on_run_completed: "keep"` if you want them,
> and give the checkpointer a TTL.
## Schedule format
**Standard 5-field cron only** — minute, hour, day-of-month, month, day-of-week:
```
┌───────── minute (0-59)
│ ┌─────── hour (0-23)
│ │ ┌───── day of month (1-31)
│ │ │ ┌─── month (1-12)
│ │ │ │ ┌─ day of week (0-6, Sunday = 0)
│ │ │ │ │
0 9 * * 1-5
```
Six-field (seconds-first) expressions, `@daily`-style nicknames, and `L`/`?` are **rejected with a
422**, matching LangGraph's _"cron must be a standard 5-field expression"_. skein's parser
([croner](https://github.com/hexagon/croner)) would accept them, so the five-field rule is enforced
before the expression reaches it: silently running `0 0 9 * * *` at a different time than the author's
crontab says is worse than refusing it.
`timezone` is an optional IANA name (`America/New_York`); absent or `null` means UTC. DST is handled:
an occurrence in a spring-forward gap shifts forward into the same day rather than being skipped, and
a repeated fall-back hour fires once.
The finest resolution is **one minute**. Sub-minute schedules are not expressible and are a
[non-goal](./roadmap.md).
## Endpoints
| Method | Path | Notes |
| -------- | --------------------------------- | ----------------------------------- |
| `POST` | `/runs/crons` | Stateless cron → `Cron` |
| `POST` | `/threads/{thread_id}/runs/crons` | Thread cron → `Cron` |
| `POST` | `/runs/crons/search` | → `Cron[]`, `x-pagination-total` |
| `POST` | `/runs/crons/count` | → a **bare integer** |
| `GET` | `/runs/crons/{cron_id}` | → `Cron` |
| `PATCH` | `/runs/crons/{cron_id}` | → `Cron` |
| `DELETE` | `/runs/crons/{cron_id}` | → **200** with a JSON body, not 204 |
All seven answer the full `Cron` object. Two response shapes are deliberately unusual because the
official SDK requires them: `count` returns a bare integer rather than `{ count }`, and `DELETE`
returns 200 with a body (the SDK skips `response.json()` only for 202 and 204, so an empty 200 makes
it throw).
`search` accepts `assistant_id`, `thread_id`, `enabled`, `metadata` (subset match), `limit`, `offset`,
`sort_by`, `sort_order`, and `select`. `select` is validated and then ignored — skein always returns
the whole row.
Set `"http": { "disable_crons": true }` in `langgraph.json` to remove the resource. That also stops
the scheduler, so a disabled deployment does not keep firing schedules nobody can see or cancel.
## Semantics
**Pausing.** `enabled: false` stops a cron without deleting it; the row stays readable and
`next_run_date` becomes `null`. Re-enabling recomputes it.
**`next_run_date === null` means dormant** — disabled, past `end_time`, or an unreachable expression
(`0 0 30 2 *`). It is the same field the scheduler indexes on, so the wire answer and the firing
behaviour can never disagree.
**`end_time`** is inclusive: an occurrence falling exactly on it still fires. On `PATCH` it is
tri-state — omit to leave it, send `null` to clear it, send a value to set it. Same for `timezone`.
**Metadata merges** on `PATCH` rather than being replaced. The run **payload** is replaced wholesale
when a patch touches any run field, because a half-replaced request (new `input`, stale `config`) is
not a request anyone asked for.
**Catch-up.** If the server is down when occurrences pass, the cron fires **once** on return and
resyncs to the next future occurrence. It does not backfill: replaying an hour of a five-minutely
schedule is twelve times the model spend on near-identical inputs, and on a thread cron they serialize
so the catch-up becomes the outage. This matches APScheduler's `coalesce` default; there is no knob.
**Traceability.** Every fired run — and every thread a stateless cron creates — carries `cron_id` in
its metadata. For a **stateless** cron that makes `client.threads.search({ metadata: { cron_id } })`
the way in, since each fire has its own thread; for a **thread** cron the thread is already known, so
`client.runs.list(thread_id)` lists the runs and each carries the `cron_id` that produced it. (The SDK
has no `runs.search` — runs are always addressed through their thread.)
**Failures.** If a fire fails (a deleted assistant, a store error) the cron is logged at `error`,
counted, and **left enabled** — the next occurrence is the retry. skein never auto-disables a
schedule: turning a thirty-second rolling-deploy blip into a silently stopped cron needing a human is
the worse failure.
**Deleting a thread deletes its thread crons**, by foreign key. A cron cannot outlive the thread it
runs on.
## Driver requirements
Crons work on every supported store/queue combination, including `skein dev` on memory with no Docker.
| Store | Queue | Crons | Posture |
| -------- | ------ | ------------------------------------------------- | ------------------------- |
| memory | memory | Work; snapshotted to `.skein/` by `skein dev` | Development |
| postgres | memory | Durable; single-instance delivery | Fine for small production |
| postgres | redis | Durable; at-least-once delivery; multi-instance | Shared servers |
| memory | redis | Work, but **lost on restart** outside `skein dev` | Warned about at startup |
| sqlite | sqlite | Durable schedules and unclaimed queued jobs | One local Node owner |
SQLite requires the store/queue pair. Claimed or executing attempts abandoned by a process become
failures without replaying their input; the schedule remains enabled for future occurrences. See
[local crash behavior](./local-deployment.md#what-survives-a-process-ending).
The scheduler logs one warning at startup on a non-durable store. A schedule that quietly stops is
worse than one that was never created, and an embedded memory store in production loses every cron on
every deploy.
## How it works
Every instance runs a ticker. Every 30 seconds (`SKEIN_CRON_TICK_MS`) it asks the store for enabled
crons whose `next_run_date` has arrived, and for each one:
1. Computes the **next** occurrence, from the later of the stored date and now — so a cron that fell
behind lands in the future in one step rather than replaying the backlog.
2. **Claims** it with a single-row conditional `UPDATE` on the primary key, guarded by a claim token
the store bumps on every write. If three instances see the same occurrence, exactly one wins; the
others skip. No leader election, no distributed lock, no Redis.
3. Commits the claim **and the `pending` run row in one transaction**, then enqueues the run.
That third step is a transactional outbox, and it is what makes delivery durable. Advancing without
the run would silently skip the occurrence if the instance died in between; creating the run first
would re-fire it. Committed together, the worst case is a `pending` run that never reached the queue —
which the next tick re-enqueues. Because enqueue is idempotent (keyed on the run id) and the worker
skips runs already terminal, delivery is **at-least-once** and execution **exactly-once**.
The recovery sweep looks only at runs a cron produced. A bare `pending` is not enough to conclude a
run is waiting for a worker: an inline `wait`/`stream` run is written `pending` too and only becomes
`running` once it acquires its thread's lock, so one queued behind a long peer would look identical
while its caller waits to execute it in-process. Handing that to a worker would run the graph twice.
The run itself then goes through the ordinary run queue — on Redis that is BullMQ, so cron-fired runs
get its retries, backoff, and stalled-job crash recovery like any other background run.
Schedule state lives in the store rather than in the queue, because `enabled`, `sort_by=next_run_date`,
filtered `count`, and metadata search are all things a queue cannot serve — and because a schedule
that a `FLUSHALL` can erase while the API still reports it as enabled is worse than no schedule.
## Operating a scheduler
**The one metric to alert on is cron lag** — how overdue the most overdue enabled cron is. Healthy, it
stays below one tick; if the scheduler dies it climbs without bound. It is the only signal that
distinguishes "nothing is due" from "nothing is running the crons". Each tick reports it, along with
counts of fired / claims lost / failed / re-enqueued runs.
Claims lost are **not** errors: on N instances, N−1 lose every occurrence by design.
| Setting | Default | Notes |
| -------------------- | ------- | ------------------------------------------------------------- |
| `SKEIN_CRON_TICK_MS` | `30000` | Lower does not help below one minute; higher trades lateness. |
**Serverless caveat.** Nothing fires if no process is running. On Cloud Run that means
`--no-cpu-throttling` and `--min-instances=1` — see [deploy-cloud-run.md](./deploy-cloud-run.md).
Scale-to-zero and cron are incompatible, and no queue changes that.
## Authentication
Crons are a first-class auth resource, so `@auth.on.crons.create`, `.read`, `.update`, `.delete`, and
`.search` all work — matching the SDK's own `Auth` class.
**If you have not registered a cron handler, crons inherit your `threads` scoping.** This matters,
because `@auth.on` callbacks are matched by exact event key: a deployment that wrote
`.on("threads", …)` has no callback named `crons`, and without the fallback the whole resource would
be served unscoped. Since a schedule is really "runs on a thread, later", `threads` is both the safe
default and the honest one — cron routes fall back to it, and `create` specifically falls back to
`threads:create_run`.
**Attaching a schedule to a thread is always authorized as `threads:create_run`**, separately from
the cron handler. A cron writes into a thread indefinitely, so it is gated by thread ownership even
when your cron handler is more permissive than your thread handler. A thread you cannot read answers
404, exactly as it does everywhere else.
A cron fires with no HTTP request behind it, so skein remembers the **creating principal** (beside the
row, never on the wire) and replays it into every run. The ownership filters are re-derived from your
`@auth.on` handler at fire time rather than frozen at create time, which means:
- Editing your auth handler applies to existing crons immediately.
- Revoking a principal stops their crons producing runs; re-granting resumes them.
- A stateless cron's per-fire thread is owner-stamped, so the user who created the cron can actually
find its results in their own `POST /threads/search`.
See [agent-protocol.md](./agent-protocol.md#authentication--authorization) for the request lifecycle.
---
<!-- source: docs/console.md -->
# The skein console
> A web UI for a running skein server: assistants, threads, live runs, channel wiring and deliveries,
> interrupts, time travel, the store, and crons. Served by the server itself, at `/console`.
## What it is
The console is a **client**; the server owns resource data and execution. A browser-local workspace
store keeps unsent drafts and navigation positions for the current tab, including across reloads.
Most screens use the standard Agent Protocol through the real
[`@langchain/langgraph-sdk`](./react-sdk.md). Two small skein extensions
cover operational state the upstream SDK does not model: a sanitized channel inventory and a run's
durable delivery attempts. The console subclasses the SDK's transport for these calls, so connection,
authentication and errors use the SDK transport. Run submissions disable automatic retries. A narrow
batch serializer normalizes options the installed SDK does not serialize; a submission-scoped request
hook supplies `timeout_ms` without changing the shared SDK client.
That is a deliberate constraint, not a coincidence. If a view cannot be built, it means the API is
missing something, and we would rather feel that here than paper over it with a bespoke endpoint. (One
gap found exactly this way: there is no cross-thread run search — runs list per thread — so "recent
activity" fans out over threads. See [roadmap](./roadmap.md).)
It ships as [`@skein-js/console`](https://github.com/skein-js/skein-js/tree/main/packages/console): the compiled UI plus a resolver, with **no
runtime dependencies**.
## Watch the console walkthrough
This silent recording follows a message from Playground into thread inspection, returns with an
unsent draft intact, and reopens the conversation from Threads and Recent threads. It uses the
offline demo graphs. Use the player's fullscreen control for a closer view.
<video controls playsinline preload="none" aria-label="Console walkthrough: run an Assistant, inspect its thread, and return to an unsent draft" poster="/images/console/playground-light.png" style="width: 100%; border-radius: 12px">
<source src="./videos/console-walkthrough.webm" type="video/webm">
Your browser cannot play this video.
</video>
## Running it
`skein dev` serves it by default and prints the URL:
```
skein · Agent Protocol dev server
API http://127.0.0.1:2024
Console http://127.0.0.1:2024/console/
Docs https://github.com/skein-js/skein-js/tree/main/docs
```
Pass `--no-console` to leave it out.
With custom auth, run `skein dev --dev-ui-access` or enable it in `langgraph.json`:
```jsonc
{
"auth": {
"path": "./src/auth.ts:auth",
"dev_ui_access": true, // skein dev only; stripped from production builds
},
}
```
Both forms print a temporary API key shared by the console and LangGraph Studio. Enter it in the
console's connection control or add `x-api-key` under Studio's **Custom Headers**. The key skips your
custom `authenticate` callback; your `@auth.on.*` authorization callbacks still run with the
`skein-dev-ui-user` identity. It rotates every 15 minutes, with a one-minute grace period for the
previous key. The new key appears in the server output; enter it again in each UI. It works only when
the dev server binds to loopback (`127.0.0.1`, `::1`, or `localhost`). Without either opt-in, both
UIs need normal credentials. Dev UI access allows Studio's `https://smith.langchain.com` origin when
no `http.cors` policy is configured; an explicit policy must allow that origin itself. Keep the
printed key private; anyone holding it can call the API as that dev identity.
Because it is served by the server, it is **same origin**: no CORS to configure, no second process, no
account, and it works with no internet connection.
## Turning it on in production
**Off by default** under `skein start` and the production image. The console can read every thread,
memory and schedule on the server, and delete runs, memories and schedules, so enabling it is a
decision you make:
```jsonc
{
"http": {
"console": true, // serve at /console
// or: "console": "/admin/console" — any path but "/"
},
}
```
Requests from the console go through the **same** [`auth`](./agent-protocol.md#authentication--authorization)
path as any other client. The temporary UI key is unavailable under `skein start`. On a server with
custom auth, use the console's connection control (top right) to supply an API key — it is sent as
`x-api-key` and held in
`localStorage`, since a static bundle has no server of its own to set a cookie.
Two things worth knowing before you enable it on a public host:
- The **assets** are served unauthenticated (they are a UI shell; every byte of data behind them is
authorized). If that is not acceptable, put the mount path behind your ingress' own auth.
- `http.console` is honoured by the **Express** transport, which is `skein start`'s Node runtime. On
Bun/Deno the flag logs a warning and does nothing — mount it yourself (below).
## What it shows
The sidebar groups work by purpose: **Build** (Playground, Assistants), **Inspect** (Threads), and
**Operate** (Overview, Channels, Store, Schedules). Playground remains the landing page and returns
to this tab's last workspace. **New thread** starts fresh. Existing
hash links, including `#/crons`, still work. Navigation becomes a drawer on phones. The Playground
inspector becomes a drawer below 1024px. Light, dark, and system themes share the same labeled status
colors; teal marks primary actions and selection.
The shell fills the browser window. Navigation stays in place while resource pages scroll in the
main pane. In Playground, conversation output and interrupt requests scroll together above a docked
composer; expanding advanced options scrolls within the composer. The inspector has its own scroll
area. This keeps the input controls reachable in short desktop windows and on phones.
### The playground
<div class="light-only">
</div>
<div class="dark-only">
</div>
Choose an **Assistant**, a saved graph configuration, then use **Chat** for a graph with a `messages`
channel or **JSON** for structured input. The input schema explains the selected mode and supplies a
JSON example. Each mode keeps its own draft. Threads are created only on submission; validation and
request failures preserve the draft. If schema discovery fails, its error remains visible and you
can select Chat or JSON explicitly or retry discovery.
**Recent threads** opens a paged list ordered by last update, with an exact-ID opener and a link to
all threads. Conversation previews use available human-message text; structured or unavailable state
falls back to a graph name and thread ID. **Open in Playground** is available directly on thread rows
and details, and on Assistant details for a fresh workspace with that configuration selected.
<div class="light-only">
</div>
<div class="dark-only">
</div>
Chat and JSON drafts stay separate for each thread and Assistant. A Zustand workspace store saves
only drafts, input mode, last workspace, and navigation positions to `sessionStorage`. It does not
persist fetched transcripts, resource responses, or stream events. Workspace state is scoped to the
API base and reset when the console's connection or API key changes, including changes from another
tab. Server authorization still controls access; account changes outside the connection control are
not detected automatically. If browser storage fails, the console keeps working in memory and shows
that drafts may not survive a reload.
The execution summary explains what was observed and what to do next. Reopening a thread restores
its latest run's retained activity through an observer that never cancels execution. **Activity** shows reported
node output, tool calls and results, pauses, and failures. **Graph** highlights nodes that reported
output; this does not establish which node is executing now. **State** shows the latest reported
values. The connection badge describes server reachability separately from execution. Markdown is
rendered without raw HTML, and remote images open only after an explicit click. Reading earlier
output keeps your scroll position.
**Stop run** sends server cancellation. An observer, such as the run detail page, closes its stream
without cancelling execution. An interrupted run displays its originating task and requested value;
**Send true**, **Send false**, and the editable response submit their literal values. A resumed run
is followed until its terminal event.
Leaving or reloading Playground also closes observation without cancelling its run. Return to the
same thread to observe its latest run; opening a thread never resubmits input. Use **Stop run** to
cancel explicitly.
**Advanced run options** are collapsed initially:
- Run on this thread, an explicit existing thread, a fresh stateless thread, or a batch of **1–100
JSON objects** for the selected Assistant.
- Delay execution in seconds; choose reject, enqueue, interrupt, or rollback when the thread is busy.
- Set an execution timeout in integer milliseconds, subject to the deployment limit. This control
requires positive confirmation of `/info.flags.per_run_timeout`; discovery failure leaves ordinary
operations available.
- Supply an idempotency key, choose completion retention for generated threads, and select captured
stream modes.
Delayed and keyed submissions are created in the background, then observed. Closing observation
leaves these runs running. A disconnect or ambiguous failure never automatically resubmits input.
Inspect the thread before trying again. A failed batch may already have created some runs.
### Overview
<div class="light-only">
</div>
<div class="dark-only">
</div>
Start with threads **waiting for you** or reporting errors, then inspect counts and recent activity.
Freshness labels describe the last successful read. Lists refresh explicitly; only observed runs
stream live. Unavailable counts and failed reads remain visible. Recent activity is bounded to
recent threads because the protocol has no global run search.
### Assistants
<div class="light-only">
</div>
<div class="dark-only">
</div>
Create an Assistant from a registered graph, edit its name/config/context/metadata, preview a saved
version, or restore it using `setLatest`. Deletion keeps threads by default; cascading deletion is an
explicit choice. Assistants registered from graph configuration return at server startup. Deleting
one does not remove the graph from `langgraph.json`.
<div class="light-only">
</div>
<div class="dark-only">
</div>
### Threads, and what is waiting on you
<div class="light-only">
</div>
<div class="dark-only">
</div>
Filter by status or recorded graph metadata, open an exact ID, and page through results. Older
threads without a recorded graph may be absent from a graph-filtered result. A thread has focused
**Conversation**, **Runs**, **State**, **Checkpoints**, and **Details** tabs. Structured graphs default
to State. The Assistant is resolved from the latest run, then metadata, then explicit selection.
<div class="light-only">
</div>
<div class="dark-only">
</div>
Copy or delete a thread, trim its checkpoints with `keep_latest`, edit metadata, and inspect/change
retention. Select threads on the current page to prune or delete them. Results report affected and
unchanged/skipped counts rather than assuming every selection changed. Retention is in minutes;
`null` means known no expiry, while missing readback means unavailable. Thread expiry marks
eligibility for background cleanup, which can be deferred by active work.
Thread and run inspection tabs are shareable through the hash URL's `tab` query. Existing tab names
are accepted; unknown names fall back safely. Returning from inspection restores the prior thread
list filters and page. A direct run link returns to the thread's Runs tab. Explicit fresh-workspace
links use `#/playground?new=1`, optionally with `assistant=<id>`; existing
`#/playground/<threadId>` links continue the named conversation.
<div class="light-only">
</div>
<div class="dark-only">
</div>
### Runs
<div class="light-only">
</div>
<div class="dark-only">
</div>
Run details separate **Activity**, **Raw events**, **Deliveries**, and **Details**. The event buffer
retains 500 frames and reports dropped history. Server execution state and observer connection state
are labeled separately. Closing the observer never cancels the run. Explicit cancel, rollback, and
delete actions explain their effects in keyboard-accessible confirmation dialogs.
Deliveries show lifecycle status, attempts, last errors, next retries, and server timestamps. Dead
deliveries can be replayed after confirmation. Delivery is at-least-once: receivers must deduplicate
by the stable delivery ID. Destinations show only the host/path presence or channel name; credentials,
queries, paths, and opaque reply targets remain redacted.
### Channels
<div class="light-only">
</div>
<div class="dark-only">
</div>
Channels is a read-only inventory of effective inbound routes, provider identity, default Assistant,
allowed Assistants, and outbound delivery support. It uses `GET /channels`, which is absent when no
channels are configured. Code and `langgraph.json` remain the source of truth; changes require a
restart. Provider credentials and module configuration are not returned. Custom authorization uses
`assistants:read`; inbound provider events still use channel signature verification.
### Time travel
Checkpoints support **Fork here** (write edited values), **Fork and run**, and **Run from here
unchanged**. Edits go through graph reducers, so an append channel adds values instead of replacing
its history. Review the displayed checkpoint and reducer warning before running forward.
### The store
<div class="light-only">
</div>
<div class="dark-only">
</div>
Browse namespaces, filter item values with JSON, and use semantic queries when the deployment
supports them. Open individual items, create values, edit JSON, or delete items. Saving is an
**upsert**: it replaces any value at the same namespace/key and resets expiry using the chosen
lifetime or deployment default. Retention and expiry are shown when the store reports them.
Passive inspection sends `refreshTtl: false` so browsing does not extend sliding expiry. If a custom
adapter rejects per-read TTL controls, the console explains this and offers **Load using normal
reads**, which may refresh expiry. It never silently falls back.
### Crons
<div class="light-only">
</div>
<div class="dark-only">
</div>
The navigation label is **Schedules**; the URL remains `#/crons`. Create stateless or existing-thread
schedules with five-field cron expressions, a timezone, optional end time, and applicable completion
retention. For example, `0 9 * * 1-5` runs at 09:00 on weekdays in the chosen timezone. Scheduling
resolution is one minute. An enabled schedule with no future match is dormant and shows **no
upcoming run**. Pause/resume without deleting. Editing scheduling fields preserves the saved run
payload; it does not replace input or Assistant configuration.
## Boundaries
Lists are paged and refresh on demand. Global run search, graph editing, and separate authentication
or permission management are outside this console. Every operation uses the server's existing
authorization. Destructive resource changes and delivery replay use accessible dialogs with focus
restoration. The foreign-server confirmation at boot remains a browser confirmation, before any
application data is requested; saved API keys stay bound to the server where they were entered.
## Mounting it yourself
The console is route bindings and bytes, so any adapter can serve it. It is not a dependency of the
adapters on purpose — the compiled UI stays under 1 MiB, and mounting the protocol should not cost that.
```ts
import { consoleAssetHeaders, resolveConsoleRequest } from "@skein-js/console";
// Express, Fastify, Hono, a Fetch handler — the shape is the same.
app.use((req, res, next) => {
if (req.method !== "GET" && req.method !== "HEAD") return next();
const resolution = resolveConsoleRequest(req.path, { mountPath: "/console" });
if (resolution.kind === "miss") return next();
if (resolution.kind === "redirect") return res.redirect(302, resolution.location);
res.set(consoleAssetHeaders(resolution.asset));
return res.status(200).send(Buffer.from(resolution.asset.bytes));
});
```
Two rules the resolver enforces so you do not have to:
- **A bare mount path redirects to its slashed form.** Assets are referenced relatively (so one build
works at any mount point); at `/console` the browser would resolve them against the parent directory.
- **An unknown path is a `miss`, not a fallback to `index.html`.** Hash routing means a real deep link
never arrives, so answering 200-with-HTML would only turn a broken asset reference into a blank page.
The SPA derives the API base by dropping the **last segment** of its own path: `/console/` → `/`,
`/api/console/` → `/api`. Mount it at `<api-base>/console` and it finds the server with no
configuration. To point it somewhere else entirely, append `?baseUrl=https://your-deployment` (the
choice is remembered), which is also how you would host the bundle statically.
---
<!-- source: docs/observability.md -->
# Observability
How to see what your agents are doing in production — traces in LangSmith or Langfuse, metrics and
spans in OpenTelemetry, product analytics in PostHog — and how to write a sink for anything else.
For **logs** and what happens when a graph throws, see
[errors-and-logging.md](./errors-and-logging.md). This doc is about the other two surfaces.
## Three surfaces
skein reports what a run did in three places, deliberately carrying different amounts of detail:
| Surface | Who reads it | What it gets |
| ------------ | ----------------- | ------------------------------------------------------------- |
| **The wire** | your API client | `RunError` — a stack only when you set `exposeErrorStacks` |
| **The log** | you, the operator | everything: the original `Error`, its stack and `cause` chain |
| **A sink** | you, the operator | everything, same as the log — it's server-side too |
That last row is the rule worth remembering: **`exposeErrorStacks` governs the wire, not telemetry.**
A sink always receives the real `Error`. See [errors-and-logging.md](./errors-and-logging.md).
Telemetry itself splits in two, and one interface covers both:
- **Traces** — spans for what happens _inside_ a graph: each LLM call, tool, and chain. These come
from LangChain's callback system, so any callback-based tracer works (LangSmith, Langfuse,
Braintrust, OpenLLMetry).
- **Events** — the run's own lifecycle: started, settled, how long, how many frames, what failed.
skein emits these itself, from the one code path every run mode goes through.
## Turning it on
Three ways, highest precedence first.
**1. In code** — pass a sink as `ProtocolDeps.telemetry`. Works everywhere, CLI or not:
```ts
import { createPostHogTelemetry } from "@skein-js/posthog";
import { embedInMemoryGraphs } from "@skein-js/server-kit";
const telemetry = createPostHogTelemetry();
const deps = embedInMemoryGraphs({ agent }, { overrides: telemetry ? { telemetry } : {} });
```
Each `create*Telemetry()` returns `undefined` when its backend isn't configured, so that ternary is
the whole "is it enabled" story. Pass an **array** to feed several backends at once.
**2. In `langgraph.json`** — for `skein dev` / `skein start`:
```json
{
"graphs": { "agent": "./src/agent.ts:graph" },
"telemetry": {
"langsmith": true,
"posthog": { "host": "https://eu.i.posthog.com" },
"otel": true,
"paths": ["./src/my-telemetry.ts:sink"]
}
}
```
`true` enables with defaults, `false` **hard-disables** even when the environment says otherwise, and
an object is passed through to the adapter. This is a skein extension; it's additive, so a config
carrying it still loads under `langgraph dev`.
**3. From the environment** — a provider the config doesn't mention turns itself on when its
variables are present:
| Provider | Detected from |
| ------------- | ------------------------------------------------------------------------------------------ |
| LangSmith | `LANGSMITH_TRACING=true` **and** `LANGSMITH_API_KEY` (or the `LANGCHAIN_` equivalents) |
| PostHog | `POSTHOG_API_KEY` |
| OpenTelemetry | `OTEL_EXPORTER_OTLP_ENDPOINT` · `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` · `OTEL_SERVICE_NAME` |
With nothing configured there is no telemetry and no cost — the engine skips building any context at
all. A provider that is only _detected_ stays quietly off when it can't be built — its package isn't
installed, or its API key is absent. One you **declared** in `langgraph.json` instead **fails at
startup**, naming what is missing, because a silent no-op for something you explicitly asked for is
worse: the first you'd hear of it is an incident with no traces to investigate. Remove the entry to run
without it.
> **Why LangSmith needs two variables.** Turning tracing on uploads every prompt, tool argument, and
> model response — your users' messages — to a third party. A stray `LANGSMITH_API_KEY` (common in
> environments that use LangSmith for something else) is not consent for that. `LANGSMITH_TRACING=true`
> is LangChain's own opt-in, so that is the signal skein takes; declaring `telemetry.langsmith` in
> `langgraph.json` is the other way to say yes.
> **Building an image?** `skein build` can't see a dynamically imported adapter, so declare the
> provider in `langgraph.json` and it gets pinned into the image. Environment-detected providers are
> deliberately **not** pinned: the build machine's environment isn't the runtime's.
## LangSmith
```bash
pnpm add @skein-js/langsmith langsmith
export LANGSMITH_API_KEY=lsv2_pt_...
export LANGSMITH_TRACING=true
```
Both variables, deliberately — see the note above. Declaring `{"telemetry": {"langsmith": true}}` in
`langgraph.json` switches tracing on for you, so the key alone is enough on that path.
**What this actually fixes.** LangSmith's tracer already instruments everything inside your graph.
What it can't know is what a "run" is on your server, so without help every run lands as an anonymous
root trace: no thread, no assistant, no user. This adapter supplies that identity — above all
**`session_id`**, the key LangSmith's **Threads** view groups on, so a conversation reads as one
thread there just as it does through the Agent Protocol.
It deliberately attaches **no tracer of its own**. `@langchain/core` installs a global tracer when
`LANGSMITH_TRACING` is on; a second one would send every span twice — you'd pay double and couldn't
trust the numbers. The adapter enriches that one tracer instead.
Credentials are **read from the environment, never written to it**: skein will not copy an API key
into `process.env`, where every module and every child process would inherit it. Set
`LANGSMITH_API_KEY` (and `LANGSMITH_ENDPOINT` for self-hosted) in the environment — that is where
LangChain's tracer reads them from.
| Set on every trace | |
| ------------------ | --------------------------------------------------------------------------- |
| `metadata` | `session_id` (thread id), `run_id`, `thread_id`, `assistant_id`, `graph_id` |
| `metadata` | `ls_user_id` when auth is on, `skein_trigger` |
| `tags` | `skein`, `graph:<id>`, `trigger:<how>`, `assistant:<id>` |
| run name | the graph id |
Full options in the [package README](https://github.com/skein-js/skein-js/blob/main/packages/telemetry-langsmith/README.md).
## PostHog
```bash
pnpm add @skein-js/posthog posthog-node
export POSTHOG_API_KEY=phc_...
```
Two layers, correlated by `$ai_trace_id` (= skein's run id):
- **`skein_run_started` / `skein_run_finished`** — status, duration, queue wait, frame count, and the
graph, assistant, and thread. The operational picture.
- **`$ai_generation`** — PostHog's LLM Analytics schema, per model call: `$ai_model`, `$ai_provider`,
input/output/total tokens, `$ai_latency`, `$ai_is_error`. This is what fills PostHog's LLM
dashboards — cost per user, tokens per conversation.
Token counts come from the message's `usage_metadata` where the provider supplies it, falling back to
the older `llmOutput.tokenUsage`. When neither is present the token fields are **omitted** rather
than reported as zero — a missing number is more honest than a wrong one.
Runs are attributed to the authenticated user's identity, falling back to the thread id so anonymous
traffic groups per conversation instead of collapsing into one distinct id. Override with
`distinctId`. Turn the LLM layer off with `captureGenerations: false`. See the
[package README](https://github.com/skein-js/skein-js/blob/main/packages/telemetry-posthog/README.md).
## OpenTelemetry
```bash
pnpm add @skein-js/otel @opentelemetry/api
```
`@skein-js/otel` records through the OTel **API** without creating an SDK or exporter. Optional HTTP metric collection uses official serializers. Your app owns
the SDK and decides where data goes, exactly as it already does for the rest of your service. That's
why one small package covers every OTLP backend. If no SDK is registered, the API's no-op
implementation takes over and this costs almost nothing.
**Span** `skein.run <graph_id>`, one per run, with `skein.run.id` / `.thread.id` / `.assistant.id` /
`.graph.id` / `.trigger` / `.status` / `.queue_ms` / `.frames` / `.failing_nodes`, plus the
`gen_ai.*` semconv equivalents. A failure records the exception and sets an `ERROR` status.
**Metrics** `skein.runs` (counter), `skein.run.duration` (histogram, ms),
`skein.run.queue.duration` (histogram, ms), and `skein.run.frames` (histogram), dimensioned by graph,
trigger, and status — deliberately **low-cardinality**, with no run, thread, or user id, so they stay
cheap in Prometheus and friends.
### Spans inside the graph
skein reports the _run_. For LLM- and tool-level spans, add a LangChain instrumentation —
[`@traceloop/node-server-sdk`](https://github.com/traceloop/openllmetry-js) or
[`@arizeai/openinference-instrumentation-langchain`](https://github.com/Arize-ai/openinference).
The OTel sink makes the Skein run span active while LangGraph executes. Instrumentation that respects
the OTel context therefore nests model, tool, and chain spans under the run span, including across
awaits; the run and thread attributes remain available for correlation and filtering too.
### Datadog, Grafana, Honeycomb, Jaeger, New Relic, Sentry
All of these are the OTel adapter plus their own exporter — no skein-specific code. Configure the SDK
the way that vendor documents, then:
```json
{ "telemetry": { "otel": true } }
```
For most, exporting `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` at their collector
is the entire integration.
## Other backends
### Langfuse
Langfuse ships a LangChain callback handler, so it needs only a small sink:
```ts
import { CallbackHandler } from "langfuse-langchain";
import type { TelemetrySink } from "@skein-js/core";
export const sink: TelemetrySink = {
name: "langfuse",
callbacks: (context) => [
new CallbackHandler({
sessionId: context.threadId, // groups a conversation in Langfuse
userId: context.userId,
metadata: { run_id: context.runId, graph_id: context.graphId },
}),
],
};
```
```json
{ "telemetry": { "paths": ["./src/langfuse.ts:sink"] } }
```
### Braintrust
Same shape — Braintrust's LangChain handler in `callbacks`, with the thread id as the span's parent
identifier.
### Sentry
Sentry reads OTel spans, so `@skein-js/otel` plus Sentry's SDK covers tracing. For error _events_
specifically, a lifecycle sink is more direct:
```ts
import * as Sentry from "@sentry/node";
import type { TelemetrySink } from "@skein-js/core";
export const sink: TelemetrySink = {
name: "sentry",
onRunEvent: (event) => {
if (event.type !== "run.finished" || !event.cause) return;
// `event.cause` is the original Error — stack and cause chain intact.
Sentry.captureException(event.cause, {
tags: { graph_id: event.context.graphId, status: event.status },
contexts: { skein: { run_id: event.context.runId, thread_id: event.context.threadId } },
});
},
};
```
### Your existing logger
For skein's **own** reports — failed runs, webhook failures — the seam is `ProtocolDeps.logger`, already
wired to the host's logger under NestJS and Fastify. See
[errors-and-logging.md](./errors-and-logging.md#logging).
For **run lifecycle events** in your structured logs, a sink is a ten-line bridge over `onRunEvent`,
the same hook the Sentry example uses.
## Writing your own sink
The whole interface, from `@skein-js/core`. Every method is optional — implement the half you need:
```ts
export interface TelemetrySink {
name: string;
/** Run lifecycle. Fire-and-forget: never awaited. */
onRunEvent?(event: RunTelemetryEvent): void;
/** LangChain callback handlers, so a tracer's spans nest under the run. */
callbacks?(context: RunTelemetryContext): unknown[];
/** Extra metadata / tags stamped on the graph call. */
traceMetadata?(context: RunTelemetryContext): Record<string, unknown>;
traceTags?(context: RunTelemetryContext): string[];
/** Make a backend context active while the graph executes. */
withRunContext?<T>(context: RunTelemetryContext, body: () => Promise<T>): Promise<T>;
/** Drain buffered data — called on shutdown. */
flush?(): Promise<void>;
shutdown?(): Promise<void>;
}
```
`RunTelemetryContext` carries `runId`, `threadId`, `assistantId`, `graphId`, `userId`, `trigger`
(`wait` / `stream` / `background` / `invoke`), `streamModes`, and the run's `metadata`.
`run.finished` adds `status`, `durationMs`, `frameCount`, and on failure `error` (the JSON-safe
`RunError`), `failingNodes` (which graph node threw), and `cause` (the original `Error`).
Two things to know:
- **`assistantId` is absent for `trigger: "invoke"`.** The `POST /invoke/:graph_id` surface addresses
a graph directly, with no assistant in between. It emits lifecycle telemetry using a synthetic run
identity for correlation, but deliberately creates no persistent run row.
- **Sinks may not throw or block.** Every call is guarded, so a throw is logged and swallowed rather
than failing a run — but a sink doing inline I/O still slows every run it observes. Buffer, and
implement `flush()`; skein calls it on shutdown so buffering is the safe default.
Point `langgraph.json` at it:
```json
{ "telemetry": { "paths": ["./src/my-telemetry.ts:sink"] } }
```
The export may be the sink itself or a function returning one, so your module can read its own
configuration.
## Cost and safety
- **Off by default, free when off.** With no sink configured the engine builds no context and reads
no metadata. There is no "disabled telemetry" tax.
- **A broken sink can't break a run.** Every method is wrapped; a throw is logged through
`ProtocolDeps.logger` and dropped. One sink failing doesn't stop the others.
- **Flushed on shutdown.** The run worker and the runtime both drain sinks when stopping, so batching
exporters don't lose the tail of a process — which is exactly the telemetry you want after a crash.
- **Metric cardinality.** skein's own OTel metrics carry no run, thread, or user id. If you write a
sink, resist adding them as metric dimensions — they belong on spans and events, not counters.
- **What leaves your server.** Traces carry your graph's inputs and outputs, which for an agent means
user messages. Check that against your data-handling obligations before pointing them at a hosted
backend; self-hosted LangSmith, Langfuse, and any OTLP collector are all supported.
## Learn more
- [Errors & logging](./errors-and-logging.md) — the other two reporting surfaces
- [Runs & Redis](./runs-and-redis.md) — the run engine these events come from
- [Deploy](./deploy.md) — running skein in production
## HTTP diagnostics
`GET /openapi.json` describes the actual mounted routes and request validators. `GET /docs` serves
an offline API reference using that document. Disabled route groups are omitted, and disabling the
meta group removes both endpoints and `/metrics`. Deployment-provided `auth.openapi` security
schemes describe existing authentication; custom policy remains in the host.
`GET /metrics` collects an explicitly supplied host metric reader. With `createOtelTelemetry`, pass
`metricReader` already registered on your `MeterProvider`; install the optional official
`@opentelemetry/exporter-prometheus@^0.222.0` and/or
`@opentelemetry/otlp-transformer@^0.222.0` serializers for the formats you expose. The default response
is Prometheus text; `?format=json` (or an Accept header for JSON) returns standard OTLP JSON. Skein
owns neither provider lifetime nor exporter/backend selection. A deployment without a collector
returns 501; collection failures return 503.
Alongside existing run/queue instruments, the sink records `skein.worker.active_runs`,
`skein.http.requests`, and `skein.http.duration`. HTTP duration measures handler response latency;
stream consumption may continue afterwards. HTTP dimensions contain method, handler and status,
without thread IDs, caller IDs or credentials. Internal MCP/A2A service calls do not count as
additional HTTP requests.
Authorized `GET /threads/{thread_id}/storage` reports exact stored saver payload sizes: checkpoints,
channel versions, pending writes and their total. Memory sums encoded buffers, SQLite uses stored
blob lengths, and Postgres measures stored JSONB and BYTEA datums, including database compression,
in one snapshot. Inline channel values count in checkpoint bytes. Counts exclude indexes, table/page overhead and filesystem
allocation; this is payload accounting rather than a disk-space estimate. Custom savers without an
inspector return 501.
---
<!-- source: docs/recipes/index.md -->
# Recipes
Task-oriented pages: a problem and the smallest code that solves it. Runnable examples named here are
in the repo and exercised by CI; provider integration snippets are checked against the linked official
SDK documentation. For a terse API reference see [using-skein.md](../using-skein.md).
## Read working code first
[`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) covers crons,
background runs, idempotency, human-in-the-loop, long-term memory and time travel in **one graph** — and
runs with **no API key and no network** (bundled fixtures, deterministic fallback classifier).
```bash
pnpm --filter @skein-js/example-triage-agent dev # console at http://127.0.0.1:2024/console/
pnpm --filter @skein-js/example-triage-agent seed # register the schedule + sweep once
```
## The recipes
| Page | Covers |
| ---------------------------------------------------------- | --------------------------------------------------------------------------- |
| [Serving](./serving.md) | Pick an adapter, serve a graph as a plain endpoint, CORS for a browser |
| [Running agents](./running-agents.md) | Background runs, crons, idempotency, human-in-the-loop, run timeouts |
| [Coupled WhatsApp workflow](./coupled-channel.md) | A workflow with the same provider as source and destination |
| [Cross-provider workflow](./decoupled-channel-delivery.md) | Email source → LangGraph workflow → allowlisted WhatsApp/email destinations |
| [Memory](./memory.md) | `getStore()`, semantic search, and the dedup trap |
| [Deep Agents](./deep-agents.md) _(experimental)_ | Files, per-user memory, streamed subagents, approval and async subagents |
| [Authenticating requests](./authentication.md) | Better Auth, Clerk, Supabase, Firebase, OIDC/JWT, and authorization policy |
| [Production](./production.md) | Auth, run-completion webhooks, durable storage and deploying |
| [Local apps and SQLite](../local-deployment.md) | Desktop, CLI, and daemon hosts with one owned database file |
## Which example shows what
| Example | Shows |
| -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) | Crons, background runs, idempotency, HITL, memory, time travel |
| [`decoupled-delivery`](https://github.com/skein-js/skein-js/tree/main/examples/decoupled-delivery) | Routed channels: email source → approval workflow → WhatsApp/email destinations |
| [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) | Full-stack chat: streaming, thinking, tool cards, HITL, memory |
| [`deep-agent`](https://github.com/skein-js/skein-js/tree/main/examples/deep-agent) | Research briefing assistant on Deep Agents _(experimental)_: memory, subagents, approval |
| [`invoke-endpoint`](https://github.com/skein-js/skein-js/tree/main/examples/invoke-endpoint) | Non-chat graphs as plain HTTP endpoints |
| [`embed-graph`](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) | A graph you already have, served with no `langgraph.json` |
| [`migrated-langgraph`](https://github.com/skein-js/skein-js/tree/main/examples/migrated-langgraph) | The drop-in proof — a stock LangGraph project under `skein dev` |
| [`react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream) | A minimal `useStream` frontend against any skein server |
| [Daybook — Electron](https://github.com/skein-js/skein-js/tree/main/examples/desktop-electron) | Meeting context tools, streaming, recap review, and Markdown revisions |
| [Reading Room — Tauri](https://github.com/skein-js/skein-js/tree/main/examples/desktop-tauri) | Local documents, library tools, source excerpts, and reviewed reading briefs |
| [Dayplan — CLI](https://github.com/skein-js/skein-js/tree/main/examples/local-cli) | Local task files, previous plans, and approval before saving |
---
<!-- source: docs/recipes/serving.md -->
# Serving
Getting the protocol in front of clients: which adapter, the non-chat surface, and browser access.
## Pick a framework adapter
Every adapter serves the identical protocol and takes the same `{ config } | { deps }` seam — pick the
framework you already run. Each has a **standalone** entry (a dedicated server) and an **embedded** one
(mount beside your existing routes).
| Framework | Package | Standalone | Embedded | Examples |
| --------- | ------------------- | --------------------- | -------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Express | `@skein-js/express` | `createExpressServer` | `skeinRouter` | [express-basic](https://github.com/skein-js/skein-js/tree/main/examples/express-basic), [embed-graph](https://github.com/skein-js/skein-js/tree/main/examples/embed-graph) |
| Fastify | `@skein-js/fastify` | `createFastifyServer` | `skeinPlugin` | [fastify-basic](https://github.com/skein-js/skein-js/tree/main/examples/fastify-basic), [fastify-app](https://github.com/skein-js/skein-js/tree/main/examples/fastify-app) |
| NestJS | `@skein-js/nestjs` | `createNestServer` | `SkeinModule.forRoot` | [nestjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-basic), [nestjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nestjs-app) |
| Next.js | `@skein-js/nextjs` | route handlers | `createSkeinRouteHandlers` (App) · `createSkeinPagesHandler` (Pages) | [nextjs-app](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app), [nextjs-basic](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-basic) |
Bun and Deno use [`@skein-js/fetch`](../deploy.md), selected with `skein build --runtime`. For a
framework skein doesn't ship, the adapters are ~40-line shims over one handler table — see
[building-an-adapter.md](../building-an-adapter.md). Mount snippets:
[using-skein.md](../using-skein.md#mount-it-on-your-framework).
## Serve a graph as a plain endpoint
For a classifier, extractor, or a workflow another service calls — no threads, no runs. The request body
**is** the graph input; the response **is** its final state.
```ts
const { router } = await skeinInvokeRouter({ deps: embedInMemoryGraphs({ triage }) });
app.use(router);
// curl -X POST localhost:2024/invoke/triage -d '{"text":"…"}'
```
Working version:
[`invoke-endpoint`](https://github.com/skein-js/skein-js/tree/main/examples/invoke-endpoint) — two
non-chat graphs, no model, no API key. Details:
[serving-a-single-graph.md](../serving-a-single-graph.md).
## CORS for a browser client
CORS is **off by default**. Same-origin needs nothing — see
[`nextjs-app`](https://github.com/skein-js/skein-js/tree/main/examples/nextjs-app), which serves the
protocol and the UI from one app.
```jsonc
// langgraph.json — matches the LangGraph CLI
{ "http": { "cors": { "allow_origins": ["http://localhost:3000"] } } }
```
Or pass `cors` to any adapter (`true` for permissive dev, `false` to force off). Cross-origin example:
[`react-usestream`](https://github.com/skein-js/skein-js/tree/main/examples/react-usestream).
---
<!-- source: docs/recipes/running-agents.md -->
# Running agents
Getting work started, keeping it from duplicating, pausing it for a human, and bounding it.
## Background runs, join and cancel
Kick off a long run, return immediately, stream it from anywhere.
```ts
const run = await client.runs.create(threadId, "agent", { input }); // returns immediately
for await (const ev of client.runs.joinStream(threadId, run.run_id)) console.log(ev);
await client.runs.cancel(threadId, run.run_id);
```
Cross-instance join needs the Redis queue + bus ([runs-and-redis.md](../runs-and-redis.md)). Concurrent
runs on **one** thread follow `multitask_strategy`; how many run at once across **different** threads is
[run concurrency](../runs-and-redis.md#run-concurrency) (default 10).
Working version: [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent)
— one background run per issue, each on its own thread. Details: [runs.md](../runs.md); for
fire-and-forget work with no conversation to keep, [background-jobs.md](../background-jobs.md).
## Scheduled runs (crons)
Fire a graph on a schedule. Schedules live in the store, so they survive restarts and fire exactly once
across instances — no leader election.
```ts
await client.crons.create("agent", { schedule: "*/5 * * * *", input: { source: "queue" } });
```
Cron delivery is **at-least-once**: an occurrence committed but never queued is re-enqueued by the
scheduler's sweep. Working version:
[`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) — a sweep that
dispatches one run per issue. Details: [crons.md](../crons.md).
## Don't create the same run twice
A retrying caller — Twilio, Stripe, GitHub, or your own sweep — should not start a second run. Send an
`Idempotency-Key` and the original response replays.
```ts
// `runs.create` has no `headers` option — a `headers` key in the payload is silently dropped and you
// get a brand-new run every retry. A per-key client is the only way to send it.
const client = new Client({ apiUrl, defaultHeaders: { "Idempotency-Key": issue.id } });
await client.runs.create(threadId, "agent", { input });
```
The claim is an insert arbitrated by a uniqueness constraint, so 50 concurrent retries across two
instances still produce exactly one run. Keys are scoped per principal, failures are never recorded, and
the **streaming** creates reject the header rather than ignoring it — an SSE response has no body to
replay. **LangGraph Platform has no equivalent.**
Working version: [`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent)
— re-sweeping creates nothing. Details:
[agent-protocol.md](../agent-protocol.md#idempotent-run-creation-idempotency-key).
## Human-in-the-loop (interrupt / resume)
Pause for approval, resume later — possibly hours later, from a different client. An interrupted run
holds no connection and no timer, only a checkpoint.
```ts
import { interrupt } from "@langchain/langgraph";
async function approve(state) {
const decision = interrupt({ question: "Send this email?", draft: state.draft });
return { sent: decision === "yes" };
}
```
The thread's status becomes `interrupted` and the interrupt surfaces in the stream; resume by submitting
a `command`. [`useStream`](../react-sdk.md) renders and resumes it for free.
Working versions:
[`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) (reached by a
conditional edge) and [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app)
(approval card in the UI). Details: [human-in-the-loop.md](../human-in-the-loop.md).
## Bound a runaway run
A graph that hangs — a model call with no timeout of its own, a node that loops — holds a worker slot
until the process restarts. Set `--run-timeout <ms>` or `SKEIN_RUN_TIMEOUT_MS`; the run aborts and
settles as `timeout`.
**Off by default, deliberately.** A legitimate research or multi-step tool run takes minutes, so a
default would turn slow-but-working into killed — the exact failure the timeout exists to prevent. Pick a
number from your own graphs' worst honest case.
## Re-run a turn a different way
Fork from any past checkpoint and run from there, leaving the original branch intact — useful for
retrying a decision with different input, or debugging what a node saw.
```ts
await client.runs.create(threadId, "agent", { input, checkpointId });
```
The fork target is server-validated, so a client cannot redirect a run to an arbitrary checkpoint through
config. Visible in the [console](../console.md); details:
[threads.md](../threads.md#time-travel-re-run-a-turn-a-different-way).
---
<!-- source: docs/recipes/coupled-channel.md -->
# Build a LangGraph WhatsApp workflow with a coupled channel
Use a coupled channel when a workflow receives messages from a provider and its outcome naturally goes
back through the same provider: WhatsApp → LangGraph → WhatsApp, Slack → LangGraph → Slack, or email
→ LangGraph → email. One `Channel` owns both translations, while Skein supplies authentication,
deduplication, thread resolution, interrupt/resume, and durable delivery.
Consider a customer who sends **“Where is order GT-1042?”** to a shop's WhatsApp number. The graph
looks up the shipment and replies **“GT-1042 left our Nairobi warehouse and arrives tomorrow.”** The
sender, conversation, and expected return provider are already the same, so splitting the source and
destination would add ceremony without adding control.
## When coupling is the simpler design
Coupling is useful, not a limitation, when the inbound event already contains the correct reply
address. A WhatsApp sender expects a WhatsApp response, and keeping both provider mappings together
means fewer names, maps, and failure paths.
Use a [cross-provider workflow](./decoupled-channel-delivery.md) when LangGraph must choose a
different destination—for example, an email source that should alert a manager over WhatsApp.
## Implement the WhatsApp channel
The event's `replyTo` is opaque to Skein. It is stored with the run and passed back to the same
channel's `deliver` method after the run settles.
```ts
import type { Channel } from "@skein-js/channels";
import { z } from "zod";
const whatsappReplyTargetSchema = z.object({ to: z.string().min(1) });
export const channel: Channel = {
name: "whatsapp",
verify(request) {
if (!verifyWhatsAppSignature(request)) return false;
const message = request.form();
return { identity: `channel:whatsapp:${message["From"]}` };
},
parseEvent(request) {
const message = request.form();
if (!message["Body"] || !message["From"]) return { kind: "ignore" };
return {
kind: "event",
event: {
threadKey: message["From"],
idempotencyKey: message["MessageSid"],
input: { customerMessage: message["Body"] },
resumeWith: message["Body"],
replyTo: { to: message["From"] },
},
};
},
async deliver(outcome, target) {
if (outcome.reply === undefined) return;
const { to } = whatsappReplyTargetSchema.parse(target);
await sendWhatsAppMessage({ to, body: String(outcome.reply) }, { key: outcome.runId });
},
};
```
Provider retries cannot create duplicate runs when `idempotencyKey` is stable. Delivery is
at-least-once, so use `outcome.runId` as a provider idempotency key when the provider supports one.
## Return a reply from LangGraph
A LangGraph node can declare the exact value delivered by the channel with Skein's existing
`replyWith` helper. If it does not, Skein can fall back to the last AI message in message-shaped
state.
```ts
import {
Annotation,
END,
START,
StateGraph,
type LangGraphRunnableConfig,
} from "@langchain/langgraph";
import { replyWith } from "@skein-js/agent-protocol";
const State = Annotation.Root({ customerMessage: Annotation<string> });
async function answerOrderQuestion(state: typeof State.State, config: LangGraphRunnableConfig) {
const orderId = /GT-\d+/.exec(state.customerMessage)?.[0];
const shipment = orderId ? await orderSystem.findShipment(orderId) : undefined;
const answer = shipment
? `${orderId} ${shipment.status} and arrives ${shipment.estimatedArrival}.`
: "I could not find that order. Please send the order number, for example GT-1042.";
config.writer?.(replyWith(answer));
return {};
}
export const graph = new StateGraph(State)
.addNode("answer-order-question", answerOrderQuestion)
.addEdge(START, "answer-order-question")
.addEdge("answer-order-question", END)
.compile();
```
Bind the channel to that graph:
```jsonc
{
"graphs": { "order-support": "./src/graph.ts:graph" },
"skein": {
"channels": {
"whatsapp": {
"path": "./src/whatsapp-channel.ts:channel",
"assistant": "order-support",
"public_url": "https://api.example.com",
},
},
},
}
```
The result follows one durable path:
```text
WhatsApp webhook → channel.parseEvent → LangGraph → channel.deliver → WhatsApp
```
For a runnable implementation with `interrupt()` and resume, see
[`examples/whatsapp-agent`](https://github.com/skein-js/skein-js/tree/main/examples/whatsapp-agent).
---
<!-- source: docs/recipes/decoupled-channel-delivery.md -->
# Build a cross-provider workflow with Skein channels
Use a routed workflow when a source starts work in one system and LangGraph must choose a different
destination: email → LangGraph → WhatsApp, WhatsApp → LangGraph → email, or one source routed to
several application-owned provider adapters. The technical mechanism is decoupled channel delivery:
Skein supplies the source and destination, while LangGraph fills the middle with the workflow.
Here, a workflow means the complete business process, not merely forwarding a message: validate the
request, collect approvals, pause and resume, decide the result, and notify the customer. The
[workflows and channels guide](../channels.md#what-workflow-means-here) explains why these
responsibilities are split across Skein and LangGraph.
The source still uses Skein's normal channel pipeline. The only extra layer is an allowlisted
destination map plus one explicit instruction written by the graph.
The example below handles a concrete request: a customer emails **“Please refund KES 27,500 for
order GT-1042; I was charged twice.”** LangGraph asks Finance for approval over WhatsApp, the Finance
reply resumes the interrupted graph, and the customer receives the decision by email.
## See the refund approval workflow
The recording follows the complete example: an email starts the workflow, LangGraph pauses for the
three required approvals, each authenticated WhatsApp response resumes its own interrupt, and the
approved result is delivered back to the customer by email.
<video controls playsinline preload="metadata" style="width: 100%; border-radius: 12px">
<source src="../videos/decoupled-refund-approval.mp4" type="video/mp4">
Your browser cannot play this video.
</video>
## When a workflow should use separate source and destination
Separate source and destination providers are appropriate when routing is a workflow decision rather
than an inherent response:
- an ERP order event alerts a sales team over WhatsApp;
- an email refund request asks HR, a manager, and Finance for approval;
- a personal assistant reads calendar or email events and sends a WhatsApp briefing.
If a WhatsApp message simply needs a WhatsApp reply, use the smaller
[coupled WhatsApp workflow recipe](./coupled-channel.md). Both forms use the same `Channel`, run,
thread, LangGraph, and outbox primitives; decoupled delivery is an additional routing shape, not a
replacement.
## 1. Define the workflow's inbound source
A source verifies and parses only. `composeRoutedChannel` arms the existing durable callback path, so
the source does not need a fake email `deliver` method. Both inbound providers use one trusted,
tenant-scoped workflow ID when they intentionally converge on the same thread:
```ts
const tenantId = "acme"; // Deployment configuration, never inbound request data.
export function workflowThreadId(refundId: string): string {
return `relay:${tenantId}:${refundId}`;
}
```
Here `refundId` is issued by the application or verified provider metadata; do not derive an
authoritative thread ID directly from email text or another user-controlled field.
```ts
import type { Channel } from "@skein-js/channels";
import { z } from "zod";
import { workflowThreadId } from "./workflow-thread-id.js";
const refundEmailSchema = z.object({
messageId: z.string().min(1),
refundId: z.string().min(1),
from: z.string().email(),
orderId: z.string().min(1),
amountKes: z.number().positive(),
reason: z.string().min(1),
});
export const emailSource = {
name: "email-source",
verify(request) {
if (!verifyEmailWebhook(request)) return false;
return { identity: "channel:email:inbound" };
},
parseEvent(request) {
const email = refundEmailSchema.parse(request.json());
return {
kind: "event" as const,
event: {
threadKey: email.refundId,
threadId: workflowThreadId(email.refundId),
idempotencyKey: email.messageId,
input: {
source: "email",
refundId: email.refundId,
customerEmail: email.from,
orderId: email.orderId,
amountKes: email.amountKes,
reason: email.reason,
financeWhatsapp: "whatsapp:+254700000013",
},
},
};
},
} satisfies Pick<Channel, "name" | "verify" | "parseEvent">;
```
## 2. Allowlist workflow destinations
Destination callbacks own credentials, provider validation, authorization, and idempotency. Names
are explicit and local to this composed channel. The map allowlists provider adapters, not recipients:
authorize every target from trusted application data rather than treating an LLM-selected or
user-supplied phone number as permission to send.
```ts
import { composeRoutedChannel, type ChannelDestinationDelivery } from "@skein-js/channels";
import { z } from "zod";
const whatsappDeliverySchema = z.object({
target: z.object({ to: z.string().min(1) }),
payload: z.object({ body: z.string().min(1) }),
});
const emailDeliverySchema = z.object({
target: z.object({ to: z.string().email() }),
payload: z.object({ subject: z.string(), body: z.string() }),
});
export const destinations = new Map([
[
"whatsapp",
async (delivery: ChannelDestinationDelivery) => {
const message = whatsappDeliverySchema.parse(delivery);
await assertAuthorizedRecipient(delivery.threadId, "whatsapp", message.target.to);
await sendWhatsAppMessage(
{ to: message.target.to, body: message.payload.body },
{ key: delivery.runId },
);
},
],
[
"email",
async (delivery: ChannelDestinationDelivery) => {
const message = emailDeliverySchema.parse(delivery);
await assertAuthorizedRecipient(delivery.threadId, "email", message.target.to);
await sendEmail(
{
to: message.target.to,
subject: message.payload.subject,
body: message.payload.body,
},
{ key: delivery.runId },
);
},
],
]);
export const channel = composeRoutedChannel(emailSource, destinations);
```
The map is copied at construction and is the complete adapter allowlist. Unknown destination names
fail the durable outbox attempt instead of silently dropping or guessing a route. Recipient and
operation policy remains application-owned; `assertAuthorizedRecipient` represents a lookup against
trusted workflow or tenant data, not another check of the recipient's string shape.
## 3. Let LangGraph run the workflow and choose WhatsApp
LangGraph passes `config.writer` to the node. The graph declares data; the destination callback
performs the external side effect only after the run settles.
```ts
import {
Annotation,
END,
interrupt,
START,
StateGraph,
type LangGraphRunnableConfig,
} from "@langchain/langgraph";
import { declareChannelDestinationDelivery } from "@skein-js/channels";
const State = Annotation.Root({
source: Annotation<"email">,
refundId: Annotation<string>,
customerEmail: Annotation<string>,
orderId: Annotation<string>,
amountKes: Annotation<number>,
reason: Annotation<string>,
financeWhatsapp: Annotation<string>,
financeDecision: Annotation<"approve" | "reject" | undefined>,
});
function notifyFinance(state: typeof State.State, config: LangGraphRunnableConfig) {
declareChannelDestinationDelivery(config.writer, {
destination: "whatsapp",
target: { to: state.financeWhatsapp },
payload: {
body:
`Approve KES ${state.amountKes.toLocaleString()} refund for ${state.orderId}? ` +
`Reason: ${state.reason}. Reply APPROVE or REJECT.`,
},
});
return {};
}
function awaitFinance(state: typeof State.State) {
const decision = interrupt({
kind: "refund-approval",
refundId: state.refundId,
assignedTo: state.financeWhatsapp,
});
return { financeDecision: decision as "approve" | "reject" };
}
function emailCustomer(state: typeof State.State, config: LangGraphRunnableConfig) {
const approved = state.financeDecision === "approve";
declareChannelDestinationDelivery(config.writer, {
destination: "email",
target: { to: state.customerEmail },
payload: {
subject: approved
? `Refund approved for ${state.orderId}`
: `Refund update for ${state.orderId}`,
body: approved
? `Finance approved your KES ${state.amountKes.toLocaleString()} refund.`
: "Finance could not approve this refund. Our support team will contact you.",
},
});
return {};
}
export const graph = new StateGraph(State)
.addNode("notify-finance", notifyFinance)
.addNode("await-finance", awaitFinance)
.addNode("email-customer", emailCustomer)
.addEdge(START, "notify-finance")
.addEdge("notify-finance", "await-finance")
.addEdge("await-finance", "email-customer")
.addEdge("email-customer", END)
.compile();
```
On the first run, `notify-finance` supplies the durable WhatsApp delivery and `await-finance` parks
the thread. The WhatsApp approval webhook needs its own verified source channel, must address the same
trusted workflow thread, and resumes the interrupt:
```ts
import { composeRoutedChannel, type Channel } from "@skein-js/channels";
import { z } from "zod";
import { destinations } from "./destinations.js";
import { workflowThreadId } from "./workflow-thread-id.js";
const financeReplySchema = z.object({
messageId: z.string().min(1),
refundId: z.string().min(1),
interruptId: z.string().min(1),
from: z.string().min(1),
decision: z.enum(["approve", "reject"]),
});
const whatsappSource = {
name: "whatsapp-source",
verify(request) {
if (!verifyWhatsAppWebhook(request)) return false;
const message = financeReplySchema.parse(request.json());
return { identity: `channel:whatsapp:${message.from}` };
},
parseEvent(request) {
const message = financeReplySchema.parse(request.json());
return {
kind: "event" as const,
event: {
threadKey: message.refundId,
threadId: workflowThreadId(message.refundId),
idempotencyKey: message.messageId,
input: { source: "whatsapp", ...message },
resumeWith: { [message.interruptId]: message.decision },
},
};
},
} satisfies Pick<Channel, "name" | "verify" | "parseEvent">;
export const channel = composeRoutedChannel(whatsappSource, destinations);
```
`workflowThreadId` must derive a tenant-scoped ID from trusted workflow data, and both sources must use
the same function. Derive the Finance principal from the verified WhatsApp sender and validate that
the principal is assigned to this approval before accepting the decision. The runnable example
performs both checks and keeps the actor, provider event ID, and timestamp in graph state as an audit
trail.
Only `declareChannelDestinationDelivery` triggers a routed destination. Ordinary `replyWith` output
and inferred AI replies are ignored here, preventing an accidental chat response from becoming an
external action. `target` and `payload` must be JSON-persistable; the destination validates their
provider-specific shape.
## 4. Configure the workflow's source routes
```jsonc
{
"graphs": { "relay": "./src/relay-graph.ts:graph" },
"skein": {
"channels": {
"email": {
"path": "./src/email-channel.ts:channel",
"assistant": "relay",
"public_url": "https://api.example.com",
},
"whatsapp": {
"path": "./src/whatsapp-channel.ts:channel",
"assistant": "relay",
"public_url": "https://api.example.com",
},
},
},
}
```
Configured route keys and explicit source names must be unique. Skein rejects those collisions at
boot so a delivery alias cannot resolve to the wrong channel. That does not namespace an explicit
`threadId`: this recipe intentionally shares one between email and WhatsApp. Build explicit IDs from
trusted, tenant-scoped workflow identifiers; when `threadId` is omitted, Skein safely namespaces the
derived ID by channel name instead.
The complete workflow is:
```text
Customer email → LangGraph → Finance WhatsApp → interrupt/resume → LangGraph → customer email
```
For conditional email/WhatsApp routing plus parallel LangGraph `interrupt()` approvals, run
[`examples/decoupled-delivery`](https://github.com/skein-js/skein-js/tree/main/examples/decoupled-delivery).
---
<!-- source: docs/recipes/memory.md -->
# Memory
Remembering things across threads and sessions.
## Long-term memory (`getStore()`)
The store is injected into every run as a LangGraph `BaseStore`, so a node reads and writes it the native
way — with memory for the default `skein dev`, SQLite for local apps, or Postgres for a shared
server. The graph code stays the same.
```ts
import { getStore } from "@langchain/langgraph";
async function remember(state, config) {
const store = getStore();
const userId = config.configurable.langgraph_auth_user_id ?? "anon";
await store.put(["memories", userId], "prefs", { units: "metric" });
const hits = await store.search(["memories", userId], { query: "units" });
return { known: hits.map((h) => h.value) };
}
```
The same items are reachable over the `/store/items` HTTP endpoints.
Working versions: [`chat-app`](https://github.com/skein-js/skein-js/tree/main/examples/chat-app) (recalls
a user across sessions) and
[`triage-agent`](https://github.com/skein-js/skein-js/tree/main/examples/triage-agent) (reads your
conventions back into the prompt), and
[`deep-agent`](https://github.com/skein-js/skein-js/tree/main/examples/deep-agent) (a Deep Agent's
`/memories/` files, one namespace per user).
> [!WARNING]
> **Read this before writing a dedup rule**
>
> Memory, SQLite, and Postgres **without** `store.index` return `score: 1` for every text hit — so
> the obvious "score >= 0.9 means duplicate" rule classifies everything as a duplicate and the agent
> **silently stops recording memories**. It does not error. [memory.md](../memory.md) covers this and the
> other shapes that bite.
## Rank by meaning, not substring
On Postgres, configure an embedder and `store.search({ query })` uses pgvector. In-memory falls back to a
naive scan, so dev behaviour matches.
```jsonc
// langgraph.json
{
"store": { "index": { "embed": "openai:text-embedding-3-small", "dims": 1536, "fields": ["$"] } },
}
```
`embed` takes a `provider:model` string or a function path; `dims` is required with it.
## Expire what you don't need
`store.ttl` (minutes) expires items on a background sweep, with `refresh_on_read` to keep active ones
alive. Threads have their own `checkpointer.ttl`. Both in [storage.md](../storage.md#store-item-ttl).
## Bring your own store
Long-term memory is the one repo you can swap without implementing the other five — point `store.adapter`
at a LangGraph `BaseStore` (including `PostgresStore`, which brings hybrid text+vector search skein's own
driver lacks) or a skein `StoreRepo`. Details:
[storage.md](../storage.md#bringing-your-own-store-storeadapter).
---
<!-- source: docs/recipes/deep-agents.md -->
# Deep Agents
> [!WARNING]
> **Experimental.** Deep Agents support is tested end to end against `deepagents` 1.15, but Deep
> Agents itself moves quickly and this integration may change in a minor release. See
> [what is covered](#what-is-covered) before relying on it in production.
Serving a LangChain [Deep Agent](https://docs.langchain.com/oss/javascript/deepagents/overview) —
planning, files, subagents, human approval — with skein.
`createDeepAgent()` returns a compiled LangGraph graph, so there is nothing to adapt: export it, name it
in `langgraph.json`, and skein supplies the checkpointer, the store, streaming and interrupt/resume
around it. There is no `@skein-js/deepagents` package; depend on `deepagents` directly.
```ts
// src/agent.ts
import {
CompositeBackend,
createDeepAgent,
StateBackend,
StoreBackend,
type StoreBackendContext,
} from "deepagents";
function memoryNamespace({ config }: StoreBackendContext): string[] {
const userId = config?.configurable?.langgraph_auth_user_id;
if (typeof userId !== "string" || userId.length === 0) {
throw new Error("Durable memory requires an authenticated user.");
}
const component = userId.replace(
/[^A-Za-z0-9\-_.@+:]/g,
(character) => `~${character.codePointAt(0)!.toString(16)}~`,
);
return ["memories", component];
}
export const graph = createDeepAgent({
model: "google-genai:gemini-2.5-flash",
systemPrompt: "You are a careful research assistant.",
backend: new CompositeBackend(new StateBackend(), {
"/memories/": new StoreBackend({
namespace: memoryNamespace,
}),
}),
subagents: [
{ name: "researcher", description: "Answers one question.", systemPrompt: "Be brief." },
],
});
```
```json
{ "graphs": { "agent": "./src/agent.ts:graph" }, "auth": { "path": "./src/auth.ts:auth" } }
```
`skein dev` serves it, with the same `langgraph.json` `langgraph dev` reads. A working version is
[`examples/deep-agent`](https://github.com/skein-js/skein-js/tree/main/examples/deep-agent): a research
briefing assistant that plans, delegates research to a subagent, drafts briefings, remembers each
user's preferences, asks before emailing, and runs deep dives in the background. It runs offline with
a scripted model, so it needs no API key.
## What is covered
Tested end to end in the example, through the official `@langchain/langgraph-sdk` client:
- a `createDeepAgent()` export loaded from an unchanged `langgraph.json`, including as a graph factory;
- planning with `write_todos`, and files in thread state that persist across a thread's turns;
- `StoreBackend` files shared across a user's threads and isolated between users;
- a synchronous subagent streamed under its own namespace with `stream_subgraphs`;
- approve / edit / reject on an `interruptOn` tool;
- an async subagent started, checked, updated and cancelled over HTTP on the same server.
Not covered, or deliberately out of scope:
- **The URL-less async transport.** LangGraph Platform runs co-deployed async subagents in process;
skein needs an explicit `url`.
- **Subgraph streaming on `/invoke`.** Use the run endpoints.
- **Python Deep Agents.** skein loads TypeScript graphs; a Python agent can only be reached as a
separate Agent Protocol service.
- **Version range.** Tested with `deepagents` 1.15 and `@langchain/langgraph` 1.4.10 or later. Pin
`deepagents` in your application.
## Files: thread state by default, the store for what outlives a thread
- **`StateBackend`** keeps files in the thread's state. They survive every turn of that thread and
vanish with it. Deep Agents also writes its own scratch (offloaded tool results, history) to the
default backend, so state is the right default.
- **`StoreBackend`** keeps files in skein's store — memory under `skein dev`, SQLite or Postgres in
production — so they are shared across threads. Mount it on a prefix with `CompositeBackend`.
**Give `StoreBackend` an explicit `namespace` on a shared server.** Its default uses assistant
identity when available and otherwise a shared `filesystem` namespace; neither isolates users.
Build it from `langgraph_auth_user_id`: skein stamps that from your [`auth`](./authentication.md) handler and strips it from anything a client sends, so it cannot
be spoofed. With no `auth` configured it is absent — fail rather than pool everyone together. Namespace
components accept only letters, digits and `-_.@+:~`, so encode identities that may contain anything
else. [Memory](./memory.md) covers the store itself.
Secure the HTTP store routes separately: graph backend namespacing does not authorize `/store/items`.
The example denies HTTP store access with `.on("store", () => false)` while retaining the graph's
injected store. Applications needing that HTTP API must scope every store action through `auth`.
Legacy `assistant_id` and `graph_id` configurable values remain application-owned; do not use them as
authenticated identity.
## Streaming subagents
A synchronous subagent runs inside the `task` tool. Ask for subgraph frames and its progress streams
alongside the main agent's, under the tool call's namespace:
```ts
for await (const chunk of client.runs.stream(threadId, "agent", {
input,
streamMode: ["updates", "messages-tuple"],
streamSubgraphs: true,
})) {
chunk.event; // "updates" from the main agent, "updates|tools:<id>" from the subagent
}
```
The SDK's UI helpers route on these names. See [subgraphs and subagents](../streaming.md#subgraphs-and-subagents).
## Planning
`write_todos` comes from LangChain's todo-list middleware. Deep Agents adds it only for some model
profiles, so pass it yourself when you rely on planning:
```ts
import { todoListMiddleware } from "langchain";
createDeepAgent({ /* … */ middleware: [todoListMiddleware()] });
```
## Human approval
`interruptOn` pauses before a tool runs; the run ends `interrupted`. Resume it with a decision per
pending action:
```ts
createDeepAgent({
/* … */ interruptOn: { send_briefing: { allowedDecisions: ["approve", "edit", "reject"] } },
});
await client.runs.wait(threadId, "agent", {
command: { resume: { decisions: [{ type: "approve" }] } },
// or { type: "edit", editedAction: { name: "send_briefing", args: { … } } }
// or { type: "reject", message: "not this week" }
});
```
See [human-in-the-loop](../human-in-the-loop.md).
## Async subagents
An async subagent is another graph, started and polled over the Agent Protocol. Give it an explicit
`url` and credentials scoped to the originating caller. Build the main agent as a graph factory so
upstream's async client is constructed with that caller's credentials. For the example's demo auth:
```ts
import type { RunnableConfig } from "@langchain/core/runnables";
import { createDeepAgent } from "deepagents";
export function graph(config: RunnableConfig) {
const userId = config.configurable?.langgraph_auth_user_id;
// An empty identity is rejected by the demo auth handler, including on an unbound factory.
const headers = { "x-user": typeof userId === "string" ? userId : "" };
return createDeepAgent({
subagents: [
{
name: "deep-researcher",
description: "Runs a longer research investigation in the background.",
graphId: "researcher",
url: process.env.SKEIN_URL ?? "http://127.0.0.1:2024",
headers,
},
],
});
}
```
In production, mint or forward a caller-scoped credential your auth handler verifies. A shared
service credential would give user-supplied `asyncTasks` records authority over other callers' child
runs. Keep receiving thread ownership checks in place; task identifiers are not authorization.
Every call is an ordinary route — thread create, run create, run get, thread state, a new run with
`multitask_strategy: "interrupt"` to update, and cancel — so it works against any skein deployment. The
URL-less, in-process transport LangGraph Platform offers is not recreated. Each started task is a
background run, so give the server enough [run concurrency](../runs-and-redis.md#run-concurrency) for the
main agent and its children at once.
## Shell and code execution
Use a remote sandbox provider for anything that executes commands. **Never use `LocalShellBackend` or
`FilesystemBackend` on a shared server**: both act directly on the host skein runs on, with that
process's permissions, for whoever can reach the agent. State and store backends need no sandbox.
## Dependency security
This workspace applies a local patch for the unpatched `braces` nesting advisory in Deep Agents'
glob dependency. Applications copying this example must audit their own lockfiles and apply the
patch in their own package-manager configuration; workspace patches do not propagate through a
published package or an exported artifact. See the [dependency audit](../audits/dependency-advisories-2026-10-20.md)
for the patch, compatibility details, and current scan results.
---
<!-- source: docs/recipes/authentication.md -->
# Authenticating requests
Keep the login flow you already have. Skein only needs the server-side step that turns an incoming
request into a stable principal. Its `auth.path` uses LangGraph's `Auth` class, so the adapter is one
function:
```text
cookie or bearer token → your provider's server SDK → { identity, permissions }
```
The examples below were checked against the providers' official server documentation on 2026-09-17
(Better Auth 1.7, Auth.js `next-auth` 5.0 beta 32, `@clerk/backend` 3.16,
`@supabase/supabase-js` 2.116, `firebase-admin` 14.3, and `jose` 6.2). Provider APIs move
independently of Skein, so follow the linked provider documentation if your installed major differs.
| If your app already uses… | Start here |
| ----------------------------------------------- | ------------------------------------------------------------------ |
| Better Auth sessions | [Better Auth](#better-auth) — pass Skein's request headers through |
| Auth.js / NextAuth sessions | [Auth.js](#authjs--nextauth) — wrap your session reader |
| Clerk users and organizations | [Clerk](#clerk) — authenticate the whole request |
| Supabase Auth | [Supabase](#supabase-auth) — verify its access token |
| Firebase Auth | [Firebase](#firebase-auth) — verify its ID token |
| Auth0, Okta, Cognito, WorkOS, or another issuer | [OIDC/JWT](#auth0-okta-cognito-workos-or-another-oidc-issuer) |
Nothing here replaces your sign-in page, callbacks, or session storage. Those stay with the provider
you already chose; this page is only the bridge at the API boundary.
## Wire the provider into Skein
Choose one `authenticate-request.ts` implementation below, then apply your authorization policy in
`auth.ts`. The Better Auth section shows a co-located version instead, because its adapter must reuse
the exact Better Auth instance your application already exports.
```ts
// auth.ts
import { Auth } from "@langchain/langgraph-sdk/auth";
import { authenticateRequest } from "./authenticate-request.js";
const tenantLabel = (identity: string) =>
encodeURIComponent(identity).replace(/\./g, "%2E").replace(/\*/g, "%2A");
export const auth = new Auth()
.authenticate(authenticateRequest)
// A filter hides other owners' rows on reads and stamps ownership onto writes. Runs authorize
// through their thread, and crons fall back to this handler when no crons handler is registered.
.on("threads", ({ user }) => ({ owner: user.identity }))
// Store items have no metadata to filter. Root their namespace instead.
.on("store", ({ user, value }) => {
value.namespace = [tenantLabel(user.identity), ...(value.namespace ?? []).slice(1)];
});
```
Point `langgraph.json` at that export:
```jsonc
{
"auth": { "path": "./src/auth.ts:auth", "disable_studio_auth": true },
}
```
The CLI requires credentials for Studio even when `disable_studio_auth` is `false`: a caller can forge
Studio's identifying header. For local development, `skein dev --dev-ui-access` or
`auth.dev_ui_access: true` in `langgraph.json` prints a temporary API key accepted by both Studio and
the console on a loopback server. Authentication answers _who is calling_; the `.on(...)` handlers
answer _what they may access_. Keep organization membership, roles, and other application
policy in those handlers rather than in Skein config.
Bearer-token providers can share this small helper:
```ts
// auth-helpers.ts
import { HTTPException } from "@langchain/langgraph-sdk/auth";
export function requireBearerToken(request: Request): string {
const match = request.headers.get("authorization")?.match(/^Bearer\s+(.+)$/i);
if (!match?.[1]) throw new HTTPException(401, { message: "Missing bearer token" });
return match[1];
}
```
## Better Auth
Better Auth's server API accepts the same `Headers` object Skein gives the authentication callback.
This works with its normal session cookie; it also accepts `Authorization: Bearer ...` when you have
enabled Better Auth's [Bearer plugin](https://better-auth.com/docs/plugins/bearer).
```ts
// src/auth.ts — add `skeinAuth` beside the Better Auth instance your app already uses
import { Auth as LangGraphAuth, HTTPException } from "@langchain/langgraph-sdk/auth";
import { betterAuth } from "better-auth";
export const auth = betterAuth({
// Keep your existing database, providers, plugins, and session options here.
});
const tenantLabel = (identity: string) =>
encodeURIComponent(identity).replace(/\./g, "%2E").replace(/\*/g, "%2A");
export const skeinAuth = new LangGraphAuth()
.authenticate(async (request) => {
const session = await auth.api.getSession({ headers: request.headers });
if (!session) throw new HTTPException(401, { message: "Unauthorized" });
return {
identity: session.user.id,
display_name: session.user.name,
email: session.user.email,
permissions: [],
};
})
.on("threads", ({ user }) => ({ owner: user.identity }))
.on("store", ({ user, value }) => {
value.namespace = [tenantLabel(user.identity), ...(value.namespace ?? []).slice(1)];
});
```
This reuses your existing Better Auth instance and session store; do not create a second auth
database for Skein. Let Better Auth's own client and endpoints continue to handle login, logout, and
cookie refresh—Skein only reads the session presented on each request. See Better Auth's
[server-side session API](https://better-auth.com/docs/basic-usage#get-session).
For this variant, point `langgraph.json` at `"./src/auth.ts:skeinAuth"` so it does not confuse the
Better Auth instance with LangGraph's `Auth` instance.
## Clerk
Use Clerk's request-level verifier rather than decoding the session JWT yourself. Set
`authorizedParties` to the origins that are allowed to send Clerk credentials to this server.
```ts
// authenticate-request.ts
import { createClerkClient } from "@clerk/backend";
import { HTTPException } from "@langchain/langgraph-sdk/auth";
const clerk = createClerkClient({
secretKey: process.env.CLERK_SECRET_KEY,
publishableKey: process.env.CLERK_PUBLISHABLE_KEY,
});
export async function authenticateRequest(request: Request) {
const requestState = await clerk.authenticateRequest(request, {
authorizedParties: [process.env.APP_ORIGIN!],
});
if (!requestState.isAuthenticated) {
throw new HTTPException(401, { message: "Unauthorized" });
}
const principal = requestState.toAuth();
if (!principal.userId) throw new HTTPException(401, { message: "Unauthorized" });
return {
identity: principal.userId,
org_id: principal.orgId,
permissions: principal.orgPermissions ?? [],
};
}
```
Clerk documents the full [`authenticateRequest()`
contract](https://clerk.com/docs/reference/backend/authenticate-request), including networkless
verification with `CLERK_JWT_KEY` and separate machine-token modes.
## Supabase Auth
For an API server, accept the user's Supabase access token as a bearer token and verify its claims.
`getClaims(token)` uses the project's cached JWKS when possible; unlike `getSession()`, it does not
trust client-side session storage.
```ts
// authenticate-request.ts
import { HTTPException } from "@langchain/langgraph-sdk/auth";
import { createClient } from "@supabase/supabase-js";
import { requireBearerToken } from "./auth-helpers.js";
const supabase = createClient(process.env.SUPABASE_URL!, process.env.SUPABASE_PUBLISHABLE_KEY!);
export async function authenticateRequest(request: Request) {
const { data, error } = await supabase.auth.getClaims(requireBearerToken(request));
if (error || !data || typeof data.claims.sub !== "string") {
throw new HTTPException(401, { message: "Unauthorized" });
}
const scope = data.claims.scope;
return {
identity: data.claims.sub,
permissions: typeof scope === "string" ? scope.split(" ").filter(Boolean) : [],
};
}
```
See Supabase's [`getClaims()` reference](https://supabase.com/docs/reference/javascript/auth-getclaims).
If your project still uses an HS256 signing secret, Supabase may call its Auth server to verify each
token; do not decode the JWT without verifying it.
## Firebase Auth
The browser obtains a Firebase **ID token** after sign-in and sends it as a bearer token. Verify that
ID token with the Admin SDK; do not send or accept a Firebase custom token here.
```ts
// authenticate-request.ts
import { applicationDefault, getApps, initializeApp } from "firebase-admin/app";
import { getAuth } from "firebase-admin/auth";
import { HTTPException } from "@langchain/langgraph-sdk/auth";
import { requireBearerToken } from "./auth-helpers.js";
const firebaseApp = getApps()[0] ?? initializeApp({ credential: applicationDefault() });
const firebaseAuth = getAuth(firebaseApp);
export async function authenticateRequest(request: Request) {
try {
const token = await firebaseAuth.verifyIdToken(requireBearerToken(request));
return { identity: token.uid, permissions: [] };
} catch {
throw new HTTPException(401, { message: "Unauthorized" });
}
}
```
Firebase's [ID-token verification guide](https://firebase.google.com/docs/auth/admin/verify-id-tokens)
explains service-account setup and the optional revocation check. Map only custom claims you created
for authorization; do not turn every JWT claim into a permission.
## Auth0, Okta, Cognito, WorkOS, or another OIDC issuer
For a standards-based issuer, verify the access-token signature, issuer, audience, and expiry from
its JWKS. The `jose` package works across Node, Bun, and Deno:
```ts
// authenticate-request.ts
import { HTTPException } from "@langchain/langgraph-sdk/auth";
import { createRemoteJWKSet, errors, jwtVerify } from "jose";
import { requireBearerToken } from "./auth-helpers.js";
const issuer = process.env.OIDC_ISSUER!; // exact `iss`
const audience = process.env.OIDC_AUDIENCE!; // this Skein API's identifier
const issuerBase = issuer.endsWith("/") ? issuer : `${issuer}/`;
const jwks = createRemoteJWKSet(new URL(".well-known/jwks.json", issuerBase));
export async function authenticateRequest(request: Request) {
try {
const { payload } = await jwtVerify(requireBearerToken(request), jwks, {
issuer,
audience,
});
if (!payload.sub) throw new HTTPException(401, { message: "Unauthorized" });
return {
identity: payload.sub,
permissions:
typeof payload.scope === "string" ? payload.scope.split(" ").filter(Boolean) : [],
};
} catch (error) {
if (error instanceof HTTPException) throw error;
if (error instanceof errors.JOSEError) {
throw new HTTPException(401, { message: "Unauthorized" });
}
throw error;
}
}
```
Use an **access token whose audience is this API**, not an ID token meant for a browser client. Auth0,
for example, makes that distinction explicit in its [token guide](https://auth0.com/docs/secure/tokens).
Some issuers publish a discovery document at `/.well-known/openid-configuration`; use its `jwks_uri`
when it differs from the conventional path above.
## Authorization: decide what callers can access
Once a provider has proved who the caller is, stop thinking about Clerk, Firebase, or JWTs. Every
provider now reaches the same LangGraph authorization handlers with a normalized `user`:
- `user.identity` is the stable user ID.
- `user.permissions` is the trusted permission list you returned during authentication.
- Extra verified fields such as Clerk's `org_id` remain available on `user`.
An authorization handler has three useful answers:
| Return value | Meaning |
| ---------------------------- | ----------------------------------------------------------------------- |
| `false` | Deny with `403` |
| `true`, `null`, or no return | Allow without an ownership filter |
| `{ owner: user.identity }` | Allow and scope the resource; the filter also stamps newly created rows |
| Rewrite `value.namespace` | Scope long-term store access |
Handlers match from most specific to broadest: `threads:delete` → `threads` → `*:delete` → `*`.
That lets one sensitive action be stricter without repeating the policy for every route.
### A practical per-user policy
This is a good default for an app where every person owns their own conversations:
```ts
export const auth = new Auth()
.authenticate(authenticateRequest)
.on("threads:delete", ({ user }) =>
user.permissions.includes("threads:delete") ? { owner: user.identity } : false,
)
.on("threads", ({ user }) =>
user.permissions.includes("skein:admin") ? true : { owner: user.identity },
)
.on("assistants", ({ user }) => user.permissions.includes("assistants:read"))
.on("store", ({ user, value }) => {
value.namespace = [tenantLabel(user.identity), ...(value.namespace ?? []).slice(1)];
});
```
Here an administrator can see all threads, ordinary users see only their own, deletion needs an
additional permission, and assistant discovery is explicitly gated. Returning an ownership filter
for `threads:delete` is important: returning only `true` would let any caller with that permission
delete any user's thread.
Thread policy automatically covers runs because runs belong to a thread. Crons have their own
resource, but fall back to the `threads` handler when you do not register a `crons` handler. Add an
explicit handler when scheduling is more privileged than chatting:
```ts
auth.on("crons", ({ user }) =>
user.permissions.includes("crons:manage") ? { owner: user.identity } : false,
);
```
### Share data inside an organization
If threads belong to a Clerk organization, Auth0 organization, or your own workspace rather than one
person, authenticate the organization ID as a trusted field and filter on it consistently:
```ts
const organizationId = (user: { org_id?: unknown }) =>
typeof user.org_id === "string" ? user.org_id : undefined;
auth
.on("threads", ({ user }) => {
const orgId = organizationId(user);
return orgId ? { org_id: orgId } : false;
})
.on("store", ({ user, value }) => {
const orgId = organizationId(user);
if (!orgId) throw new HTTPException(403, { message: "Choose an organization" });
value.namespace = [tenantLabel(orgId), ...(value.namespace ?? []).slice(1)];
});
```
Do not accept `org_id`, `owner`, or permissions from request metadata or graph input. They must come
from provider-verified claims. Decide explicitly what a user with no active organization should do;
the example denies access instead of silently falling back to a personal tenant.
### Know which resources can be filtered
| Resource | What authorization can do |
| ------------ | --------------------------------------------------------------------------------------- |
| `threads` | Deny or return metadata filters; the same policy protects their runs |
| `crons` | Deny or return metadata filters; falls back to `threads` when no cron handler exists |
| `assistants` | Gate access only; graph-backed assistants are shared, so ownership filters do not apply |
| `store` | Rewrite `value.namespace`; returning a metadata filter does not isolate store items |
The last row is the easy one to miss: authentication alone does **not** make long-term memory
multi-tenant. Keep the namespace rewrite even if every other resource is owner-filtered. Calls to
`getStore()` from inside a graph do not pass through HTTP authorization, so build their namespace
from `config.configurable.langgraph_auth_user_id` (or another server-injected verified field), never
from model output.
## Auth.js / NextAuth
Auth.js already knows how to verify its own session. You only need a small wrapper that accepts two
functions:
- `readVerifiedSession(request)` returns your provider's verified session, or `null`.
- `toPrincipal(session)` returns the user shape Skein needs. `identity` must be a stable user ID;
everything else is application-defined context for authorization handlers.
The wrapper returns exactly the function accepted by `.authenticate(...)`:
`(request: Request) => Promise<SkeinPrincipal>`.
```ts
import { HTTPException } from "@langchain/langgraph-sdk/auth";
type SkeinPrincipal = {
identity: string;
permissions: string[];
[attribute: string]: unknown;
};
type ReadVerifiedSession<TSession> = (request: Request) => Promise<TSession | null>;
type ToPrincipal<TSession> = (session: TSession) => SkeinPrincipal;
function createSessionAuthenticator<TSession>(
readVerifiedSession: ReadVerifiedSession<TSession>,
toPrincipal: ToPrincipal<TSession>,
): (request: Request) => Promise<SkeinPrincipal> {
return async (request) => {
const session = await readVerifiedSession(request);
if (!session) throw new HTTPException(401, { message: "Unauthorized" });
const principal = toPrincipal(session);
if (!principal.identity) throw new HTTPException(401, { message: "Unauthorized" });
return principal;
};
}
```
For example, in the same module where a Next.js app already configures Auth.js, use the exported
`auth()` session reader and map your session fields:
```ts
import { Auth as LangGraphAuth } from "@langchain/langgraph-sdk/auth";
import NextAuth, { type DefaultSession } from "next-auth";
import GitHub from "next-auth/providers/github";
declare module "next-auth" {
interface Session {
user: {
id: string;
permissions?: string[];
} & DefaultSession["user"];
}
}
const { auth: readAuthJsSession, handlers } = NextAuth({
providers: [GitHub],
callbacks: {
session({ session, token }) {
// This example uses Auth.js's JWT session strategy. For database sessions, use `user.id`.
if (!token.sub) throw new Error("Auth.js token is missing a user ID");
session.user.id = token.sub;
return session;
},
},
});
export { handlers };
export const skeinAuth = new LangGraphAuth()
.authenticate(
createSessionAuthenticator(
async () => readAuthJsSession(),
(session) => ({
identity: session.user.id,
display_name: session.user.name,
email: session.user.email,
permissions: session.user.permissions ?? [],
}),
),
)
.on("threads", ({ user }) => ({ owner: user.identity }));
```
This is intentionally code you own: change only the session reader and field mapping when your
framework or session shape differs. Auth.js documents how to expose a stable user ID for [JWT and
database sessions](https://authjs.dev/guides/extending-the-session) and how to keep custom session
fields type-safe with [module augmentation](https://authjs.dev/getting-started/typescript).
If Skein runs as a separate server, the same wrapper still applies, but
`readVerifiedSession(request)` should call a trusted session-validation endpoint. Alternatively,
exchange the application session for a short-lived signed access token and use the [OIDC/JWT
recipe](#auth0-okta-cognito-workos-or-another-oidc-issuer). Do not feed an Auth.js database-session
cookie to a JWT decoder: that cookie is a lookup key, not a JWT.
## Browser cookies and CORS
Same-origin cookies need no CORS configuration. Across origins, all three pieces must agree:
1. The browser request includes credentials.
2. The auth provider issued a cookie valid for the Skein origin/domain and appropriate `SameSite` and
`Secure` attributes.
3. Skein allows the exact frontend origin and credentials—never `*` with credentials:
```jsonc
{
"http": {
"cors": {
"allow_origins": ["https://app.example.com"],
"allow_credentials": true,
},
},
}
```
Bearer tokens do not require credentialed cookies, but the browser still needs the `authorization`
header allowed by its CORS preflight. Prefer same-origin mounting when your framework already hosts
the frontend.
## What Skein does after authentication
The returned object becomes `user` in every `.on(...)` callback. Skein also stamps it into graph run
config as `langgraph_auth_user`, `langgraph_auth_user_id`, and `langgraph_auth_permissions`; clients
cannot spoof those keys. Thread filters apply to runs as well. Long-term store access is separate and
must be rooted by namespace, as in the shared policy above.
For the full request lifecycle, route-to-permission map, and store-scoping traps, see
[Authentication + authorization](../agent-protocol.md#authentication--authorization).
---
<!-- source: docs/recipes/production.md -->
# Production
Locking it down, being told when work finishes, and running it durably.
## Custom auth
The server is open by default. skein implements
[LangGraph's custom-auth model](https://docs.langchain.com/langsmith/custom-auth), so an existing `Auth`
file is drop-in. Using Better Auth, Clerk, Supabase, Firebase, Auth0, or another OIDC provider? Start
with the [authentication provider recipes](./authentication.md).
```ts
// auth.ts
import { Auth, HTTPException } from "@langchain/langgraph-sdk/auth";
export const auth = new Auth()
.authenticate(async (request) => {
const token = request.headers.get("authorization")?.replace(/^Bearer /, "");
const user = token ? await verify(token) : undefined;
if (!user) throw new HTTPException(401, { message: "Unauthorized" });
return { identity: user.id, permissions: user.scopes };
})
.on("threads", ({ user }) => ({ owner: user.identity })); // scopes threads and their runs
```
Wire it with `"auth": { "path": "./auth.ts:auth" }` in `langgraph.json`, or in code as
`embedInMemoryGraphs(graphs, { auth })` — note the second argument **is** the overrides bag here, while
`embedPostgresGraphs` nests them under `overrides`. A returned filter both hides other owners' rows
and stamps ownership onto new ones.
**Inside a graph**, the principal arrives on the run config as
`config.configurable.langgraph_auth_user`, `…_user_id` and `…_permissions` — server-owned and
unspoofable, present only when auth is configured.
> [!WARNING]
> **The store is not scoped by an ownership filter**
>
> A store item carries no metadata to filter on, so **an authenticated caller can read every tenant's
> items** unless an `@auth.on.store` handler narrows it. Scope it by rewriting `value.namespace` — the
> pattern and its traps are in [agent-protocol.md](../agent-protocol.md#scoping-the-store).
Full route → permission map:
[agent-protocol.md](../agent-protocol.md#authentication--authorization).
## Get notified when a run finishes
Pass a `webhook` URL on run creation; skein POSTs the settled run to it.
```ts
await client.runs.create(threadId, "agent", { input, webhook: "https://example.com/hooks/run" });
```
**The callback is recorded in the same transaction as the run's terminal status**, so a crash between
"the run finished" and "someone was told" cannot lose it. skein attempts the first delivery inline, so
a healthy receiver hears within milliseconds; a failure is recorded and retried — for about 34 minutes
by default, which rides out a rolling deploy.
**It is at-least-once, not exactly-once.** A retry can duplicate a callback your receiver already
processed — after a network timeout that actually succeeded, say. Every attempt carries the same
`X-Skein-Delivery-Id`; dedupe on it. The API remains the source of truth for run state.
Set `SKEIN_WEBHOOK_SECRET` and every callback is signed, so a receiver can tell a real one from
anybody who guessed the URL:
```ts
import { verifySkeinSignature } from "@skein-js/agent-protocol";
const result = verifySkeinSignature({
header: req.headers["x-skein-signature"],
body: rawBody, // the RAW bytes, not JSON.stringify(req.body)
secrets: [process.env.SKEIN_WEBHOOK_SECRET!],
});
if (!result.ok) return res.status(401).end();
```
Working receiver: `src/webhook-receiver.ts` in
[`examples/express-basic`](https://github.com/skein-js/skein-js/tree/main/examples/express-basic)
(`pnpm webhook-receiver`) — accepts a genuine callback, refuses a forged and a replayed one.
**Persist the delivery outbox** with Postgres or SQLite. The delivery worker polls pending records
after restart; Redis supplies scheduled delivery when configured, but is not required for recovery.
Details — the retry tiers, signature verification and key rotation, listing and replaying failed
deliveries, and what gets stored and for how long: [webhooks.md](../webhooks.md).
## Go durable and scale out
Back skein with Postgres (state + checkpoints) and Redis (queue + cross-instance streaming). Your server
code doesn't change — only how `deps` is built.
```ts
import { buildRuntime } from "@skein-js/runtime";
const rt = await buildRuntime({
configPath: "./langgraph.json",
store: "postgres",
queue: "redis",
});
// pass { deps: rt.deps, channels: rt.channels } to an adapter; call rt.dispose() on shutdown
```
Or use the CLI: `skein build` produces a deployable image, `skein up` brings up app + Postgres + Redis via
Compose, and `skein dev --store postgres --queue redis` runs the durable stack locally. Redis is optional
for one instance and **required for more than one**.
## Deploy it
For a local app or one Node owner, SQLite is another durable option. It needs no Postgres/Redis
services and uses the same protocol; the host owns lifetime and recovery decisions. Follow
[local deployment](../local-deployment.md) for CLI selection or `embedSqliteGraphs`.
That image runs anywhere you can run a container. [deploy.md](../deploy.md) covers what every platform
needs — Postgres, Redis, the port, the `/ok` probe, pool sizing, SIGTERM draining, SSE through proxies —
with guides for [Cloud Run](../deploy-cloud-run.md), [Railway](../deploy-railway.md),
[Fly.io](../deploy-fly.md), [Render](../deploy-render.md), [AWS](../deploy-aws.md),
[Kubernetes](../deploy-kubernetes.md), [a VPS](../deploy-vps.md), and
[what doesn't work on serverless](../deploy-serverless.md).
## Watch it run
The [console](../console.md) at `/console` shows threads, live run tails, interrupt approvals, time
travel, the store browser and crons — served by your own server, no account and no tunnel. It is **off by
default** in production; opt in with `{"http": {"console": true}}`.
For tracing and metrics, [observability.md](../observability.md) covers the `TelemetrySink` seam and the
LangSmith, PostHog and OpenTelemetry adapters.
---
<!-- source: docs/building-an-adapter.md -->
# Building your own adapter
skein-js ships adapters for [Express](https://github.com/skein-js/skein-js/tree/main/packages/server-express),
[Fastify](https://github.com/skein-js/skein-js/tree/main/packages/server-fastify), [NestJS](https://github.com/skein-js/skein-js/tree/main/packages/server-nestjs), and
[Next.js](https://github.com/skein-js/skein-js/tree/main/packages/server-nextjs) (App + Pages Router). If your stack isn't one of those — a raw
Node `http` server, [Hono](https://hono.dev), Koa, an existing app on some other framework — you can
write your own adapter in a few dozen lines. This guide shows how; the four shipped adapters are all
built exactly this way.
## Why this is easy
All of skein-js's protocol logic lives in [`@skein-js/agent-protocol`](https://github.com/skein-js/skein-js/tree/main/packages/agent-protocol)
behind a **transport-neutral handler table**. An adapter adds _no protocol logic_ — it only does
shape translation: turn your framework's request into a normalized `ProtocolRequest`, call the right
handler, and write the returned `ProtocolResponse` back out. The shipped
[Express adapter](https://github.com/skein-js/skein-js/tree/main/packages/server-express) is exactly this and nothing more; yours will mirror it.
```text
your framework request ──▶ ProtocolRequest ──▶ handler ──▶ ProtocolResponse ──▶ your framework response
(Step 3) (Step 2) (Step 4)
```
## The contract
Three types from `@skein-js/agent-protocol` are all you touch:
```ts
interface ProtocolRequest {
method: string; // "POST"
url: string; // absolute URL (path + query) — an auth handler may read it
params: Record<string, string>; // path params, e.g. { thread_id }
query: Record<string, string | string[] | undefined>;
body: unknown; // parsed JSON body
headers: Record<string, string | undefined>; // lowercased names
}
type ProtocolResponse =
| { kind: "json"; status: number; body: unknown; headers?: Record<string, string> }
| { kind: "empty"; status: number; headers?: Record<string, string> }
| {
kind: "sse";
status: number;
events: AsyncIterable<string>;
headers?: Record<string, string>;
};
type ProtocolHandler = (req: ProtocolRequest) => Promise<ProtocolResponse>;
```
`ProtocolHandlers` is a table of named handlers (`createThread`, `createStreamRun`, `joinRunStream`,
`putStoreItem`, …). Each validates the request (with Zod), calls the typed service, and returns a
`ProtocolResponse`. You dispatch to them by name.
For a worked example, `@skein-js/fetch` is the shortest one to read: a single file over WHATWG
`Request`/`Response`, so it shows the whole contract — request mapping, the headers channel, a
pull-driven SSE body, and a bounded request read — without a framework's conventions in the way.
## Step 1 — assemble a runtime
Build a `ProtocolRuntime` from a `ProtocolDeps` (the injected storage/queue/graph bundle). The
easiest way to get production `deps` from a `langgraph.json` is
[`@skein-js/runtime`](https://github.com/skein-js/skein-js/tree/main/packages/runtime)'s `buildRuntime`:
```ts
import { buildRuntime } from "@skein-js/runtime";
import { resolveProtocolRuntime } from "@skein-js/server-kit";
const assembled = await buildRuntime({
configPath: "./langgraph.json",
store: "memory", // or "postgres"
queue: "memory", // or "redis"
});
const resolved = await resolveProtocolRuntime({
deps: assembled.deps,
channels: assembled.channels,
});
// resolved.runtime is running; resolved.routes includes optional channel routes.
```
> You can also construct `deps` by hand (your own `SkeinStore`, `RunQueue`, `RunEventBus`, and a
> `GraphResolver`) — see [`@skein-js/core`](https://github.com/skein-js/skein-js/tree/main/packages/core) for the interfaces and
> [storage.md](./storage.md) / [runs-and-redis.md](./runs-and-redis.md) for the drivers.
## Step 2 — the route table
The paths mirror the `@langchain/langgraph-sdk` client exactly (that's the conformance oracle — don't
invent your own spelling). Bind each `method + path` to a handler name. The canonical table is
exported from `@skein-js/agent-protocol` as `skeinRoutes` (re-exported from `@skein-js/express` too,
for back-compat). Mount `resolved.routes` when using `resolveProtocolRuntime`: it starts with that
canonical table and appends configured channel routes without changing the core protocol surface.
```ts
import { skeinRoutes, copyThreadIdIntoBody, matchSkeinRoute } from "@skein-js/agent-protocol";
// skeinRoutes: { method, path, handler, foldThreadIdIntoBody? }[]
// resolved.routes: the same table plus deployment-specific channel bindings
// e.g. { method: "post", path: "/threads/:thread_id/runs/stream",
// handler: "createStreamRun", foldThreadIdIntoBody: true }
//
// copyThreadIdIntoBody(request) — the body-fold rule for foldThreadIdIntoBody routes
// matchSkeinRoute(method, pathname) — match a catch-all path → { binding, params }
// (handy for adapters that dispatch from one route, like the NestJS + Next.js adapters)
```
Three things to get right:
- **Order most-specific first** within each method so literals win over params (e.g.
`/threads/search` before `/threads/:thread_id`).
- **`foldThreadIdIntoBody`** — the SDK addresses a thread-scoped run by its path
(`POST /threads/{id}/runs/stream`) but the stateless run handlers read `thread_id` from the body.
For those routes, copy the path `thread_id` into the body before dispatch (see the worked example).
- **Strip your mount prefix before matching.** `skeinRoutes` paths are anchored at the protocol root,
so if your adapter mounts a **catch-all** and matches by hand, the framework hands it the full
external path (`/api/threads`) which will never match `^/threads$`. Use `stripBasePath` from
`@skein-js/server-kit`, and treat its `null` as "not ours" — pass the request through untouched so
the host app's own routes still resolve:
```ts
import { stripBasePath } from "@skein-js/server-kit";
const pathname = stripBasePath(url.pathname, mountPrefix);
if (pathname === null) return next(); // not under our mount — the host app's problem
const match = matchSkeinRoute(method, pathname);
```
Adapters that mount each route **explicitly** (Express's `Router`, Fastify's plugin `prefix`) get
this from their router for free and can skip it. Where `mountPrefix` comes from is
framework-specific: the NestJS adapter reads Nest's own `app.setGlobalPrefix(...)` via
`ApplicationConfig`, while the Next.js adapters take an explicit `basePath` option.
## Step 3 — map your request onto `ProtocolRequest`
Pure shape translation. Header names must be **lowercased** (handlers look up `last-event-id`), and
array-valued headers flattened to a single value:
```ts
function toProtocolRequest(req /* your framework request */, params): ProtocolRequest {
return {
method: req.method,
url: absoluteUrl(req), // e.g. `http://${host}${originalUrl}` — must include the query string
params, // from your router match, e.g. { thread_id }
query: parsedQuery(req),
body: parsedJsonBody(req),
headers: lowercasedSingleValueHeaders(req),
};
}
```
If you're on Express specifically, `@skein-js/express` exports `toProtocolRequest` so you don't have
to write this.
## Step 4 — serialize the `ProtocolResponse`
Switch on `response.kind`:
- **`json`** — **serialize with `serializeWireJson` from `@skein-js/core`, not your framework's
`res.json`.** Bodies may contain LangChain messages (thread state, history, `runs.wait` values)
that must be flattened to the wire shape clients expect. Send with
`Content-Type: application/json`.
- **`headers`** — present on every kind, and **optional to you but not invisible to clients**: this is
how the engine returns response metadata it cannot put in the body, such as `x-pagination-total` on
assistant search. Set them all before writing the body. Dropping them costs no test failure and no
error — the client just silently loses the metadata — so it is worth wiring in from the start.
- **`empty`** — just write the status and end.
- **`sse`** — set the SSE headers (`SSE_HEADERS`), flush them, then write each string from
`response.events` as-is. **Do not re-encode** — the core already produced complete frames (each
ends in `\n\n`). When the client disconnects, call the iterator's `return()` so the run's frame
subscription is torn down.
The [complete adapter](#a-complete-minimal-adapter) below shows this switch assembled.
## Step 5 — handle errors
Handlers throw `SkeinHttpError` for client-visible faults (it carries the intended `status`,
`message`, and optional `code`/`details`). Anything else is an unexpected `500`. Once SSE headers are
flushed you can no longer set a status — just end the stream.
`SkeinHttpError` carries `code` and `details` alongside `status`/`message`; forward them when
present. Log anything that is _not_ a `SkeinHttpError` — that is a bug, not a client fault.
## Step 6 — worker lifecycle & CORS
- **Worker** — `runtime.worker.start()` drains the run queue (background runs). Call
`runtime.worker.stop()` on shutdown so in-flight runs drain cleanly. If you wire
`createProtocolRuntime` by hand rather than using the shortcut below, pass
`{ worker: { maxConcurrency: resolveRunConcurrency(options.worker?.maxConcurrency) } }` — otherwise
your adapter silently ignores `--concurrency` and `SKEIN_RUN_CONCURRENCY`, which is exactly the bug
`resolveProtocolRuntime` centralizes away. See
[run concurrency](./runs-and-redis.md#run-concurrency).
- **CORS** — browser clients (Agent Chat UI, React `useStream`) run on a different origin than your
server, so you must send `Access-Control-Allow-*` headers (and answer preflight `OPTIONS`) on every
route, including the SSE streams. [`@skein-js/server-kit`](https://github.com/skein-js/skein-js/tree/main/packages/server-kit) exports
`corsFromHttpConfig` / `toCorsOptions` (the shared, framework-agnostic home; also re-exported from
`@skein-js/express`) to derive `cors`-style options from the `langgraph.json` `http.cors` block; on
another framework, apply the equivalent middleware.
- **Logging** — `resolveProtocolRuntime(options, frameworkLogger?)` takes an optional second argument:
your framework's own logger, used only when the caller supplied neither `options.logger` nor
`deps.logger`. Pass one **only if your framework owns a logger the host has already configured**
(NestJS's `Logger`, `fastify.log`) — then defaulting it on borrows the host's decision, silence
included. If it doesn't, pass nothing; a library should not decide on its host's behalf to start
writing to stdout. Either way, read the resolved logger back off the result and use _that_ for your
transport-fault logging, so the engine and the transport can't disagree about where output goes:
```ts
const { runtime, cors, logger } = await resolveProtocolRuntime(options, myFrameworkLogger);
// …later, in your error path:
sendError(error, res, logger); // the try/catch shape shown in the complete adapter below
```
> **Shortcut:** [`@skein-js/server-kit`](https://github.com/skein-js/skein-js/tree/main/packages/server-kit)'s `resolveProtocolRuntime(options)`
> does Steps 1 + the worker lifecycle in one call — resolve `{ config } | { deps }` into a running
> runtime (assistants seeded, worker started) plus any CORS from the config and the resolved logger.
> It's what the Express, Fastify, NestJS, and Next.js adapters all use.
## A complete minimal adapter
A dependency-free adapter over Node's built-in `http` server — no Express, no framework:
```ts
import { createServer } from "node:http";
import { SSE_HEADERS } from "@skein-js/agent-protocol";
import { isSkeinHttpError, serializeWireJson } from "@skein-js/core";
import { buildRuntime } from "@skein-js/runtime";
import { resolveProtocolRuntime } from "@skein-js/server-kit";
const assembled = await buildRuntime({
configPath: "./langgraph.json",
store: "memory",
queue: "memory",
});
const { runtime, routes: routeBindings } = await resolveProtocolRuntime({
deps: assembled.deps,
channels: assembled.channels,
});
// Compile the resolved protocol + channel route patterns to matchers once.
const routes = routeBindings.map((r) => ({
...r,
regex: new RegExp("^" + r.path.replace(/:(\w+)/g, "(?<$1>[^/]+)") + "$"),
}));
const server = createServer(async (req, res) => {
try {
const url = new URL(req.url ?? "/", `http://${req.headers.host ?? "localhost"}`);
const match = routes.find(
(r) => r.method === req.method?.toLowerCase() && r.regex.test(url.pathname),
);
if (!match) {
res.writeHead(404).end();
return;
}
const params = match.regex.exec(url.pathname)?.groups ?? {};
const body = await readJson(req); // parse the JSON body (omitted for brevity)
let protocolRequest = {
method: req.method,
url: url.href,
params,
query: Object.fromEntries(url.searchParams),
body,
headers: req.headers, // already lowercased by Node
};
// Thread-scoped run routes: fold the path thread_id into the body.
if (match.foldThreadIdIntoBody && params.thread_id) {
protocolRequest = {
...protocolRequest,
body: { ...(body ?? {}), thread_id: params.thread_id },
};
}
const response = await runtime.handlers[match.handler](protocolRequest);
if (response.kind === "json") {
res.writeHead(response.status, { "content-type": "application/json" });
res.end(serializeWireJson(response.body));
} else if (response.kind === "empty") {
res.writeHead(response.status).end();
} else {
res.writeHead(response.status, SSE_HEADERS);
const it = response.events[Symbol.asyncIterator]();
res.on("close", () => void it.return?.(undefined));
for (let n = await it.next(); !n.done; n = await it.next()) res.write(n.value);
if (!res.writableEnded) res.end();
}
} catch (error) {
if (res.headersSent) {
if (!res.writableEnded) res.end();
} else if (isSkeinHttpError(error)) {
res.writeHead(error.status, { "content-type": "application/json" });
res.end(JSON.stringify({ status: error.status, message: error.message }));
} else {
res.writeHead(500, { "content-type": "application/json" });
res.end(JSON.stringify({ status: 500, message: "Internal Server Error" }));
}
}
});
server.listen(2024);
// On shutdown: await runtime.worker.stop(); server.close(); await assembled.dispose();
```
Point the official SDK at `http://localhost:2024` and it just works — because the wire format is
produced by the same handler table the Express adapter uses.
## Checklist
- [ ] Assembled a `ProtocolRuntime`; called `registerGraphAssistants()` and `worker.start()`.
- [ ] Bound every route in `resolved.routes`, most-specific-first, with `foldThreadIdIntoBody` honored.
- [ ] `ProtocolRequest` has lowercased single-value headers and an absolute `url` (with query).
- [ ] JSON responses serialized with `serializeWireJson` (not a plain `JSON.stringify`/`res.json`).
- [ ] `response.headers` forwarded on all three kinds (silently lost otherwise — see step 4).
- [ ] SSE responses stream frames unmodified, set `SSE_HEADERS`, and tear down on client close.
- [ ] `SkeinHttpError` mapped to its status; everything else → `500`; no status changes mid-stream.
- [ ] CORS applied for browser clients; `worker.stop()` on shutdown.
- [ ] Verified with the real `@langchain/langgraph-sdk` client (see [testing.md](https://github.com/skein-js/skein-js/blob/main/docs/testing.md)).
Reference implementation: [`@skein-js/express`](https://github.com/skein-js/skein-js/tree/main/packages/server-express) —
[`routes.ts`](https://github.com/skein-js/skein-js/blob/main/packages/server-express/src/routes.ts),
[`to-protocol-request.ts`](https://github.com/skein-js/skein-js/blob/main/packages/server-express/src/to-protocol-request.ts),
[`send-protocol-response.ts`](https://github.com/skein-js/skein-js/blob/main/packages/server-express/src/send-protocol-response.ts),
[`error-response.ts`](https://github.com/skein-js/skein-js/blob/main/packages/server-express/src/error-response.ts).
Built an adapter for a framework we don't ship? We'd love a PR — see
[CONTRIBUTING.md](https://github.com/skein-js/skein-js/blob/main/CONTRIBUTING.md).
---
<!-- source: docs/building-a-runner.md -->
# Serving the Agent Protocol with your own agent
skein-js is built for [LangGraph.js](https://github.com/langchain-ai/langgraphjs), but the protocol
engine is not. `@skein-js/agent-protocol` installs with **no graph runtime** — no
`@langchain/langgraph`, no `@langchain/langgraph-checkpoint`, in the emitted JavaScript _or_ the
generated type declarations. Both are asserted in CI by
[`static-imports.test.ts`](https://github.com/skein-js/skein-js/blob/main/packages/test-support/src/static-imports.test.ts),
as two separate checks, because they fail independently.
So if you have an agent of your own — an AI SDK loop, a Mastra workflow, a hand-rolled tool loop —
you can serve the Agent Protocol with it, and every LangGraph client
([`useStream`](./react-sdk.md), Agent Chat UI, LangGraph Studio) works against it unchanged.
This page is the runtime seam. [Building your own adapter](./building-an-adapter.md) is the transport
seam — one page per seam.
## What you implement
The engine drives an **`AgentGraph`**. Two methods are required; the rest are optional, and the split
is empirical rather than designed — it is what a measured, working server actually needed.
```ts
interface AgentGraph {
// REQUIRED — run the agent, yielding one chunk per step.
stream(
input: unknown,
options?: unknown,
): Promise<AsyncIterable<unknown>> | AsyncIterable<unknown>;
// REQUIRED — the thread's authoritative state after a run.
getState(config: { configurable?: Record<string, unknown> }): Promise<AgentStateSnapshot>;
// Optional. Each maps to endpoints; see the capability table below.
getStateHistory?(config, options?): AsyncIterable<AgentStateSnapshot>;
updateState?(config, values, asNode?): Promise<{ configurable?: Record<string, unknown> }>;
bulkUpdateState?(config, supersteps): Promise<{ configurable?: Record<string, unknown> }>;
streamEvents?(input, options?): AsyncIterable<unknown>;
invoke?(input, options?): Promise<unknown>;
getGraphAsync?(options?): Promise<{ toJSON(): unknown }>;
getSubgraphsAsync?(namespace?, recurse?): AsyncIterable<[string, unknown]>;
}
```
A LangGraph `CompiledGraph` satisfies this by construction — that is what keeps the type additive.
### Streaming: the one trick worth knowing
`stream` yields whatever you like, but a **`[mode, data]` tuple** is unwrapped into that stream mode.
`StreamMode` is a plain string union from `@skein-js/core`; nothing LangGraph participates. So a
plain async generator produces correct SSE:
```ts
async *stream(input, options) {
const threadId = String(options?.configurable?.thread_id ?? "");
for await (const step of myAgent(input)) {
yield ["values", step]; // -> event: values
}
}
```
If your agent runs nested agents, a client can ask for their frames with `stream_subgraphs`. Your
`stream` then receives `options.subgraphs === true`, and may yield a **`[namespace, mode, data]`
triple** — `namespace` a string array, one segment per nesting level, empty for your own frames:
```ts
function* streamNestedStep(step: unknown) {
yield [["researcher:1"], "updates", step]; // -> event: updates|researcher:1
}
```
Ignore the option and keep yielding pairs if you have nothing nested; see
[subgraphs and subagents](./streaming.md#subgraphs-and-subagents).
Anything that is _not_ such a tuple is published as an `updates` payload.
### State: seven fields
`getState` returns an `AgentStateSnapshot`. skein projects it onto the wire `ThreadState`, and reads
exactly these:
| Field | Meaning |
| -------------- | ---------------------------------------------------------------------------------------------- |
| `values` | current state — what `GET /threads/{id}` mirrors |
| `next` | nodes still to run. **Non-empty ⇒ the run is reported `interrupted`** |
| `tasks` | `{ id, name, error?, interrupts, result? }` — carries pending interrupts and per-node failures |
| `config` | its `configurable` carries the checkpoint coordinates |
| `parentConfig` | the parent snapshot's config, if any |
| `metadata` | passed through untouched |
| `createdAt` | ISO timestamp |
Only `values`, `next` and `tasks` are required. `{ values: state, next: [], tasks: [] }` is a valid
snapshot for an agent that does not pause.
## Capabilities: what an absent method does
An optional method you do not implement is a **handled 422**, not a crash:
```json
{
"status": 422,
"message": "This agent does not implement \"updateState\", which this endpoint requires.",
"code": "agent_capability_missing",
"details": { "capability": "updateState" }
}
```
| Method | Endpoints it serves |
| ------------------- | ---------------------------------------- |
| `getStateHistory` | `GET`/`POST /threads/{id}/history` |
| `updateState` | `POST /threads/{id}/state` (time travel) |
| `bulkUpdateState` | `POST /threads` carrying `supersteps` |
| `streamEvents` | the `events` stream mode |
| `invoke` | `POST /invoke/{graph_id}` |
| `getGraphAsync` | `GET /assistants/{id}/graph` |
| `getSubgraphsAsync` | `GET /assistants/{id}/subgraphs` |
**Why 422 and not 501.** `@langchain/langgraph-sdk`'s `AsyncCaller` retries any status outside
`STATUS_NO_RETRY = [400,401,402,403,404,405,406,407,408,409,422]`. A 501 would cost the official
client five requests and exponential backoff to learn a fact that cannot change on retry. 422 is in
that list, and "the request cannot be processed as sent" is honest: the route exists, the body is
fine, this agent cannot serve it.
`GET /threads/{id}/state` is deliberately **not** in that table: it falls back to `getState` when
`getStateHistory` is absent, so the required tier alone serves it.
## A complete server
No `langgraph.json`, no CLI, no `@langchain/*` in your file:
```ts
import type { AgentGraph, AgentStateSnapshot } from "@skein-js/agent-protocol";
import { createExpressServer } from "@skein-js/express";
import { embedInMemoryGraphs } from "@skein-js/server-kit";
const stateByThread = new Map<string, unknown>();
const threadId = (c?: { configurable?: Record<string, unknown> }) =>
String(c?.configurable?.["thread_id"] ?? "");
const agent: AgentGraph = {
stream(input, options) {
const id = threadId(options as { configurable?: Record<string, unknown> });
return (async function* () {
const reply = { messages: [{ role: "ai", content: "hello" }] };
stateByThread.set(id, reply);
yield ["values", reply];
})();
},
async getState(config): Promise<AgentStateSnapshot> {
return { values: stateByThread.get(threadId(config)) ?? {}, next: [], tasks: [] };
},
};
// Pass a `GraphResolver` — not a graph map. A map's values are LangGraph compiled graphs by type.
const deps = embedInMemoryGraphs({
ids: ["chat"],
load: async () => agent,
schemas: async (id) => ({ [id]: { graph_id: id } }) as never,
});
await (await createExpressServer({ deps })).listen(2024);
```
That serves `POST /threads`, `GET /threads/{id}`, `POST /runs/wait`, `POST /runs/stream` (SSE with
replay), `GET /threads/{id}/state`, `GET /info`, plus auth, `Idempotency-Key`, crons, the store, and
multitask strategies — none of which you implement.
`storeBridge`, `ephemeralCheckpointer` and `cloneCheckpoint` on `ProtocolDeps` are all **optional**
and runtime-specific; omit them. `checkpointer` is a structural `ThreadCheckpointer` (five methods:
`getTuple`, `list`, `put`, `putWrites`, `deleteThread`) — supply a stub if your agent keeps its own
state, or a real one if you want thread copy / prune / rollback.
## Interrupts and resume
`POST /runs` with a `command` body resumes an interrupted run. The engine does **not** construct a
runtime's command type — that would be a runtime import. It hands your agent a branded envelope:
```ts
import { isAgentCommand, agentCommandPayload } from "@skein-js/agent-protocol";
stream(input, options) {
if (isAgentCommand(input)) {
const { resume, update, goto } = agentCommandPayload(input);
// …resume your agent with `resume`
}
}
```
Use `agentCommandPayload` rather than reading fields off the envelope: the wire schema passes unknown
fields through, and the payload is the complete, unbranded object.
To report a pause, return a snapshot with non-empty `next` (or tasks carrying `interrupts`).
## What you cannot change
- **The wire types.** They are `@langchain/langgraph-sdk`'s, deliberately — that is why every
LangGraph client works against skein by construction. See [reuse](./reuse.md).
- **The run lifecycle** — statuses, multitask strategies, the SSE frame envelope.
- **The auth model** — see [the Agent Protocol reference](./agent-protocol.md). Auth is the
deployment's concern, not the agent's; a runner never sees it.
## Known limits
- **`events` mode is effectively LangChain-only.** Its demux keys on `on_chain_stream`, root
`run_id`, and `langsmith:hidden` tags. Your agent may yield `mode: "events"` chunks, but their
payload is your contract with your own clients — `useStream` will not render them as token events.
- **`useStream` hydration** calls `GET /threads/{id}/state`. Implement `getState` well (it is
required anyway) or hydration is empty.
- **Time travel** needs `updateState` + `getStateHistory`. Without them, `checkpoint_id` on a run is
ignored rather than honoured.
## The reference implementation
[`@skein-js/langgraph`](https://github.com/skein-js/skein-js/tree/main/packages/langgraph) is one
implementation of this seam, and it imports only `@skein-js/agent-protocol`'s public entry point —
no privileged access. If it ever needs an internal, the internal is missing from the public API.
Reading it is the fastest way to see what a complete binding looks like.
---
<!-- source: docs/deploy.md -->
# Deploy anywhere
## Choose a deployment
| Option | Use it for | Storage and host |
| --------------------- | ----------------------------------------------------- | -------------------------------------------------------------------------- |
| **Shared server** | Web/API clients and workers spanning instances | Postgres for resources/checkpoints; Redis for a shared queue and streaming |
| **Local application** | Desktop apps, CLI tools, daemons, or one Node process | One exclusively owned SQLite file through `@skein-js/local` |
For the SQLite option, follow [local deployment](./local-deployment.md). It supports Node 22/24/26;
Electron and Tauri hosts package the same engine and use the existing protocol. Models and tools
remain application choices. The default `skein dev` continues to use memory drivers.
The rest of this page covers the **server image and shared-service deployment**.
**Deploy your LangGraph.js graphs anywhere you can run a container — or just Node.** skein has no `skein deploy` and
no control plane — that's the point ([roadmap.md](./roadmap.md) lists it as an explicit non-goal).
What it ships instead is an ordinary OCI image: `skein build` bundles your TypeScript graphs to plain
JS and produces a Docker image that needs a Postgres, a Redis, and two environment variables. Nothing
about it is specific to any host.
Production artifacts can run on Node, Bun, or Deno. Node 24 LTS is the default and uses Express. Bun and
Deno use the Web-standard `@skein-js/fetch` adapter and their native HTTP servers. Select one in
`langgraph.json` or at build time:
Node is the graduated production fallback. Bun and Deno are preview targets until each complete
clean-image conformance matrix (real SDK, Postgres/Redis, multi-instance streaming, slow clients,
telemetry parenting, and PID-1 shutdown) is green; the native launchers themselves are tested.
```json
{ "skein": { "runtime": { "name": "deno", "version": "2.9.4" } } }
```
```bash
skein build --runtime bun --runtime-version 1.3.14 -t my-agent
```
CLI flags override config. The image pins the official runtime image, imports every graph under that
runtime during the build, starts the runtime directly as PID 1, and uses a non-root user. Deno gets
explicit network, environment, artifact-read, system, and native-library permissions. Skein cannot
make an arbitrary Node-native graph dependency portable; the compatibility probe fails that image
build so the dependency can be replaced or isolated before deployment.
The server image contract below applies across the listed hosting platforms. The per-platform guides are the
dashboard and CLI steps on top of it.
## Pick a platform
**Background runs is the deciding column.** A platform that throttles or suspends your container
between requests will stall a queued run until the next request wakes it — which is fine for a
request/response graph and wrong for anything scheduled or fired-and-forgotten.
:::tabs
== Cloud Run
- **Deploy from** — push image to Artifact Registry
- **Postgres + Redis** — Cloud SQL + Memorystore, or any hosted
- **Background runs** — ⚠️ needs `--no-cpu-throttling` + `--min-instances=1`
- **Scales to zero** — yes — queued runs stall until a request
- **Stop-signal window** — 10s default, configurable
→ [Google Cloud Run guide](./deploy-cloud-run.md)
== Railway
- **Deploy from** — Dockerfile in repo
- **Postgres + Redis** — Railway plugins
- **Background runs** — ✅ default
- **Scales to zero** — no
- **Stop-signal window** — ~30s
→ [Railway guide](./deploy-railway.md)
== Fly.io
- **Deploy from** — Dockerfile in repo
- **Postgres + Redis** — Fly Postgres / Upstash Redis
- **Background runs** — ⚠️ needs `min_machines_running = 1`
- **Scales to zero** — yes, if auto-stop is on
- **Stop-signal window** — `kill_timeout` (5s default)
→ [Fly.io guide](./deploy-fly.md)
== Render
- **Deploy from** — Dockerfile or image
- **Postgres + Redis** — Render Postgres + Key Value
- **Background runs** — ⚠️ paid instance (free ones spin down)
- **Scales to zero** — free tier only
- **Stop-signal window** — ~30s
→ [Render guide](./deploy-render.md)
== AWS App Runner
- **Deploy from** — push image to ECR
- **Postgres + Redis** — RDS + ElastiCache
- **Background runs** — ⚠️ CPU throttled between requests
- **Scales to zero** — no (min 1 instance)
- **Stop-signal window** — ~30s
→ [AWS App Runner guide](./deploy-aws.md#app-runner)
== AWS Fargate
- **Deploy from** — push image to ECR
- **Postgres + Redis** — RDS + ElastiCache
- **Background runs** — ✅ default
- **Scales to zero** — no
- **Stop-signal window** — `stopTimeout` (30s default)
→ [AWS ECS Fargate guide](./deploy-aws.md#ecs-fargate)
== Kubernetes
- **Deploy from** — push image to any registry
- **Postgres + Redis** — whatever you run
- **Background runs** — ✅ default
- **Scales to zero** — no (unless KEDA/Knative)
- **Stop-signal window** — `terminationGracePeriodSeconds` 30s
→ [Kubernetes guide](./deploy-kubernetes.md)
== VPS
- **Deploy from** — build on the box or pull
- **Postgres + Redis** — containers or managed
- **Background runs** — ✅ default
- **Scales to zero** — no
- **Stop-signal window** — `docker stop -t` (10s default)
→ [VPS / plain Docker guide](./deploy-vps.md)
== Serverless
- **Deploy from** — n/a — not a container
- **Postgres + Redis** — any hosted
- **Background runs** — ❌ not supported
- **Scales to zero** — yes
- **Stop-signal window** — none
→ [Vercel & serverless guide](./deploy-serverless.md)
:::
> **How these were verified.** The image contract itself — port binding, the `/ok` probe, `SIGTERM`
> draining in-flight runs to a terminal status, Postgres migrations on boot — was exercised
> end-to-end against real Postgres and Redis containers, and the behaviors described below are what
> was actually observed. The individual platform guides apply that same contract using each
> platform's own documentation; they have not each been deployed for real. If a step is wrong on your
> platform, please [open an issue](https://github.com/skein-js/skein-js/issues) — corrections to
> these are very welcome.
## What the image already does for you
`skein build` produces the image; `skein dockerfile` prints the same Dockerfile if you'd rather commit
it and let your platform build it. Either way:
- **Binds the port the platform gives it.** The CMD passes no `--port`, so the server binds `$PORT`
when one is injected (Railway, Render, Cloud Run and AWS App Runner all do). When nothing is
injected it falls
back to **8123** — the same port the image `EXPOSE`s and health-checks — so a bare
`docker run -p 8123:8123` works, as do platforms that route to a port you declare without setting
`PORT` for you (Fly.io, ECS, Kubernetes).
- **Handles `SIGTERM` properly.** The selected runtime is PID 1 (the CMD invokes it directly, not through
`npx` — under `npx`, PID 1 is npm, which exits on `SIGTERM` without waiting for the server). On
signal, skein stops accepting queued runs, gives in-flight runs
[a grace window](#graceful-shutdown) to finish, aborts whatever is left so it lands in a **terminal**
status rather than stranded as `running`, then closes the pools.
- **Serves a health probe** at `GET /ok` → `200 {"ok":true}`, and declares a Docker `HEALTHCHECK`
against it with a 20s start period.
- **Runs unprivileged** as the runtime image's `node`, `bun`, or `deno` user.
- **Reads config from the environment only** — `POSTGRES_URI` and `REDIS_URI`. The generated
`.dockerignore` excludes `.env*` and `.npmrc*`, so secrets are never baked into a layer.
- **Installs pinned production dependencies only.** No vite/tsx, no devDependencies, no runtime
TypeScript transform: `skein build` resolved your tsconfig `paths` and workspace aliases once, on the
host. Source maps stay on, so stack traces still point at your TypeScript.
The pinned set is **derived from the bundle**: every published package your graphs still import
after bundling is recorded at the exact version installed on the build host, and `skein build` fails
on the host if the two ever disagree. Packages you load **by name at runtime** are the exception a
bundler cannot see — declare those under `dependencies` in `langgraph.json`
([what `skein build` bundles](./bundling.md#what-skein-build-inlines-vs-externalizes)).
- **Caches dependency installs** via a BuildKit cache mount, and accepts an optional `id=npmrc` build
secret for private scoped packages — `skein build --npmrc <path>`, or
`docker build --secret id=npmrc,src=$HOME/.npmrc` for the standalone Dockerfile. See
[langgraph-cli-compat.md](./langgraph-cli-compat.md).
**Migrations run automatically on boot.** There is no `skein migrate` step. On startup skein applies
its schema (tracked in a `skein_migrations` table), sets up LangGraph's checkpoint tables, registers
one assistant per declared graph, and — because `skein start` warms graphs — imports every graph
module. All of that finishes _before_ the server starts listening. Migrations take their own advisory
lock, so several instances booting at once during a rolling deploy is safe — the ones that don't win
the lock **wait**, then find nothing to apply.
That covers **both** schemas, on two separate lock keys: skein's own, and LangGraph's checkpoint
tables. The second one is skein's lock around someone else's migration —
`PostgresSaver.setup()` in `@langchain/langgraph-checkpoint-postgres` reads its version ledger and
then creates types and inserts rows with no exclusion of its own, so concurrent boots against a
database with pending migrations collide on `checkpoint_migrations_pkey` or `pg_type`. Separate keys
because the two schemas are independent: sharing one would make every boot wait on a migration it
does not depend on. It only bites when migrations are genuinely pending — a first deploy, or an
upgrade that bumps that package's schema — which is exactly when replicas start together.
> **Building on Apple Silicon?** `skein build` doesn't pass `--platform`, so you'll get an arm64 image
> that most hosts reject. Export `DOCKER_DEFAULT_PLATFORM=linux/amd64` before building.
## Without Docker
A container is the default, not a requirement. `skein build --artifact-only` stops before invoking
Docker and writes `.skein/build` — your graphs bundled to plain JS, a rewritten `langgraph.json`, the
baked `schemas.json`, and a `package.json` whose dependencies are **exact-pinned**, so the install is
deterministic without a lockfile. Ship that directory to a machine with Node and run it:
```bash
skein build --artifact-only # on any machine with Node; no Docker daemon needed
rsync -a .skein/build/ server:/srv/skein/
# on the server
cd /srv/skein && npm install --omit=dev
export POSTGRES_URI=postgres://… REDIS_URI=redis://…
npx skein start # or: node node_modules/skein-js/dist/index.js start
```
The bare command above runs inside the copied artifact directory. To serve a local build from the
project root, select the artifact's config explicitly:
```bash
skein build --artifact-only
skein start -c .skein/build/langgraph.json
```
`skein start` defaults to `langgraph.json` in the working directory. When that config has no adjacent
schemas and a build exists in its directory's `.skein/build`, the error suggests a `-c` argument to
use in your original start command. Keep its other options, including storage drivers and any
custom `--sqlite-path`. Artifact selection stays explicit.
`skein start` also reads a conventional `.env` from the directory you run it in, as well as one beside
the config it loads — so running your own build from the project root
(`skein start -c .skein/build/langgraph.json`) picks up that project's `.env` without exporting
anything. An artifact never carries a `.env` of its own: it is the Docker build context, and `skein
build` drops a file `env` from the config for that reason. The ambient environment still wins over
both, which is what the `export` above relies on.
`skein start` serves the artifact and is the same entrypoint the image uses — the container's `CMD` is
literally `node /app/node_modules/skein-js/dist/index.js start --store postgres --queue redis --host
0.0.0.0`. `skein-js` is pinned into the artifact's own dependencies, so the install above puts the CLI
on the box with it.
Four differences from the containerised path:
- **It binds `127.0.0.1` by default**, where the image passes `--host 0.0.0.0`. That default is right
behind a reverse proxy on the same machine and wrong if something else must reach it directly — the
image overrides it because a container is already isolated.
- **`--store postgres --queue redis` are the defaults**, so a bare `skein start` reaches for both and
fails with an actionable error if `POSTGRES_URI` / `REDIS_URI` are missing. There is no in-memory
production mode.
- **The port is 8123**, not `dev`'s 2024, and `PORT` from the environment wins over the default.
- **Bun and Deno need `--runtime`.** The artifact records which runtime it was built for and refuses
to start under a different one. Node needs no flag.
Everything else on this page still applies — migrations run on boot, `/ok` is the probe, `SIGTERM`
drains in-flight runs. What you give up is the pinned base image and the process supervision the
platform would have done, which on a VM is a
[systemd unit](./deploy-vps.md#run-it-without-docker).
## What every deployment needs
### 1. A Postgres
Set `POSTGRES_URI`. It holds protocol resources (assistants, threads, runs, store items) and
LangGraph checkpoints. The base schema needs **no extensions**.
pgvector is needed **only if you set `store.index` in `langgraph.json`** for semantic search. skein
runs `CREATE EXTENSION IF NOT EXISTS vector` on boot, which can only enable an extension the server
already has — it cannot install one. If it's missing, boot fails with an error telling you so.
| Provider | pgvector available |
| ------------------- | ---------------------------------------------------------- |
| Cloud SQL (PG 13+) | ✅ (`vector` is a supported extension) |
| AWS RDS (PG 15+) | ✅ |
| Neon | ✅ |
| Supabase | ✅ (enabled by default) |
| Render Postgres | ✅ |
| Railway | ⚠️ use the **pgvector template**, not the default Postgres |
| `postgres:16` image | ❌ — use `pgvector/pgvector:pg16` |
### 2. A Redis
Set `REDIS_URI`. skein uses it for the run queue (BullMQ) and for cross-instance stream pub/sub with
replay. Configure the instance with `maxmemory-policy noeviction` — BullMQ's job data must not be
evicted.
**The image requires it, and so does the entrypoint.** Its CMD runs
`skein start --store postgres --queue redis` — but those are also `skein start`'s own defaults now, and
it _rejects_ `--store memory` / `--queue memory` outright. Overriding the CMD, or running the binary by
hand, can no longer produce a production server whose queue is process-local and whose state disappears
on restart. The redis
queue driver fails the boot if `REDIS_URI` is unset. (Redis is only _optional_ on the in-code
embedding path — `embedPostgresGraphs` falls back to an in-memory queue and bus when no Redis URL is
given, which keeps state durable but limits you to a single instance. See
[embedding.md](./embedding.md#going-to-production).)
### 3. The port
Nothing to do if your platform injects `PORT`. If it asks you which port the container listens on,
answer **8123**.
### 4. A health probe
Point it at **`/ok`**. It's a dependency-free liveness check that deliberately does _not_ touch
Postgres or Redis, so a transient database blip can't flap an otherwise healthy instance.
It also works as a **startup/readiness probe**: migrations, assistant registration and graph warming
all complete before the server binds, so a responding `/ok` genuinely means "fully booted". Budget
your startup probe accordingly — the image's own estimate is 20 seconds.
### 5. Auth — read this before you expose it
> ⚠️ **skein's auth is off by default, and this is the production path.** With no `auth.path`
> configured in `langgraph.json`, every protocol endpoint is open: anyone who can reach the URL can
> create threads, run your graphs, and spend your model-provider tokens. Either configure auth (see
> [agent-protocol.md](./agent-protocol.md) for the route→permission map) or keep the service private —
> behind your platform's authenticated ingress, a VPC, or an authenticating proxy.
>
> `/ok` is registered ahead of the auth engine, so health probes keep working either way.
## Sizing & tuning
### Connection budget
skein opens **three** Postgres pools per instance: one for protocol resources, one for LangGraph's
`PostgresSaver`, and one for the per-thread execution claim (see
[Scaling past one instance](#scaling-past-one-instance)). All three are capped by `PG_POOL_MAX`, so plan
for:
```text
max connections ≈ 3 × PG_POOL_MAX × instances
```
Check that against your database's limit — this is the most common way to exhaust a small managed
Postgres once autoscaling kicks in. `PG_POOL_MAX=5` is a sane starting point.
The execution-claim pool behaves differently from the other two, and it is worth knowing how: a
connection is held for the **whole duration** of an executing run, not for the length of a query. So its
working size is your run concurrency, not your request rate — and it is sized from
`SKEIN_RUN_CONCURRENCY` plus headroom for the inline run modes (`/runs/wait`, `/runs/stream`,
`/threads/{id}/stream`, `/threads/{id}/commands`), which execute without consuming a worker slot and are
therefore bounded by request arrival rather than by run concurrency.
Past that headroom an inline run waits for a free claim connection and then **fails** rather than
executing unguarded — the safe direction, since executing without the claim is what interleaves two runs'
checkpoint writes. Keep `PG_POOL_MAX` at or above run concurrency (`skein start` warns at boot when it is
not), and raise it if you serve heavy concurrent streaming alongside a saturated background worker.
### When the database stops answering
`pg` waits for a pool connection **forever** by default, which turns an unreachable database into a
**hang** rather than an error: no status code, no log line, just a socket the client eventually
abandons. skein applies a 30s `PG_CONNECTION_TIMEOUT_MS` so the fault surfaces.
Thirty rather than something tighter, because `pg` uses that one timer for two different waits — the
connection handshake **and** waiting for a free client when the pool is already at `PG_POOL_MAX`. A
tight bound therefore fails two ordinary situations: a burst of slow-but-working queries against a
small pool, and an autosuspended serverless Postgres (Neon, Supabase) waking up, which regularly takes
longer than ten seconds and happens on the boot path. Set `PG_CONNECTION_TIMEOUT_MS=0` for `pg`'s
original wait-forever behaviour.
`PG_STATEMENT_TIMEOUT_MS` bounds a single statement server-side — the last line of defence against one
pathological query pinning a pool connection. **On by default at 30s**; `0` disables it. It is per
_statement_, not per request, so a legitimately long sequence of quick queries is unaffected; what it
catches is one query that is stuck or scanning something it shouldn't. Suggested 15000 on a small
instance, 60000 on a large one.
It became a default only once the list/search paths were page-bounded and indexed — before that it would
have turned slow-but-working queries into errors. If your deployment has a query that genuinely runs
longer, raise it or set `0`; a cancelled statement surfaces as a `57014` error naming the statement,
rather than as a hang. The shapes that can still take a while, all of them either bounded by you or
inherently large: a deep `OFFSET`, an unindexed `values` filter, `POST /store/namespaces`
(`SELECT DISTINCT` over the store), store search by text with no `store.index` configured (a whole-table
read by design), and anything that walks a very large thread's whole checkpoint history — copying a
thread, or a `multitask_strategy: "rollback"` on one. Rollback in particular reports a failure as a
warning and continues, so on very large threads either raise the timeout or prefer another strategy.
Schema DDL is exempt, and deliberately so: our migrations and the pgvector setup lift the timeout on
their own connection, and the checkpointer's `setup()` runs on a separate untimed pool. Index builds are
legitimately slow, a cancelled `CREATE INDEX CONCURRENTLY` leaves an _invalid_ index that the retry skips
by name (recording the migration as applied while the index goes permanently unused), and a cancelled
boot migration is a boot _loop_ rather than a slow boot. Those connections are destroyed rather than
returned to the pool, so the lifted timeout can't leak into a later query that happens to reuse them.
It is applied with a `SET` on each new connection rather than as a startup parameter, because
PgBouncer and Supabase's pooler reject unrecognised startup parameters outright. Under **transaction**
pooling a `SET` does not persist, so the timeout silently does not apply there — it is a backstop, not
a guarantee.
**Full sizing guidance, and every tuning knob in one table, is in
[performance.md](./performance.md).** This section covers only what is specific to running the
container.
### Heap size vs. the container's memory limit
The image bakes **no** `--max-old-space-size`, because a Dockerfile cannot know the limit the container
will be given. Instead `skein start` compares the two at boot and warns when V8's ceiling is above ~75% of
the container's limit, naming the flag and a computed value.
Node **is** cgroup-aware — since v12 it sizes the heap from the container's limit, not the host's — so the
usual advice to set `--max-old-space-size` on every container is wrong here, and following it can make
things worse. Measured on `node:22-slim`, V8's ceiling tracks about half the limit: 512Mi → 259MB,
1Gi → 524MB, 2Gi → 1048MB. Setting it to "75% of the limit" on a 512Mi container would _raise_ the heap
from 259MB to 432MB and make an OOM kill more likely, not less.
What the automatic sizing does not do is go below a floor of about **259MB**. So under roughly 345Mi —
256Mi and 128Mi are ordinary Cloud Run and Kubernetes settings — V8's ceiling meets or exceeds the whole
container, it never feels the pressure that would trigger a full GC, and the kernel kills the process
first. That appears as a restart with no stack, no log line, and nothing in your metrics.
**Only act on this when the warning fires**, and then lower the heap rather than raising it. The warning
names a computed value. Include the image's own flag when you set it — `NODE_OPTIONS` replaces the
image's value rather than adding to it, so omitting `--enable-source-maps` silently costs you TypeScript
stack traces:
```
NODE_OPTIONS="--enable-source-maps --max-old-space-size=153"
```
`skein start` also warns when run concurrency exceeds `PG_POOL_MAX`, for the same reason: runs then queue
waiting for a connection rather than executing, which looks like flat throughput rather than like a pool
limit. Budget **two** pools per instance (store + checkpointer).
### Heap pressure while running
Separately from the boot check, skein samples heap usage every 30s and warns once when it passes 85% of
the limit — then stays quiet until it drops back below 70%, so a sustained episode is one log line rather
than one every 30 seconds. It runs for the life of the background worker and needs a `logger` to be
configured; `SKEIN_HEAP_WARN_PERCENT=0` turns it off.
The warning carries what makes it actionable, because the percentage alone does not:
| What the line shows | What it means |
| -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| High `runs in flight`, at or near your concurrency | Too much work at once — lower `SKEIN_RUN_CONCURRENCY` or scale out. |
| High `buffered frames` | A slow SSE consumer is holding a run's frames in memory — see `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN`, or move to Redis. |
| Neither, and the heap stays high across episodes | A genuine leak. Capture a heap snapshot. |
On Redis there is no `buffered frames` figure, because the frames are not in this process's heap — which
is itself part of the answer.
Two limits worth knowing. It watches the **JS heap only**: `used_heap_size` excludes external memory, so
a process killed for RSS — large `Buffer`s, many sockets — never trips it. That is the right scope for
what skein itself retains (buffered frames, mirrored graph state), but it does not make the boot check
above redundant. And the re-arm band sits 15 points under whatever threshold you set, so
`SKEIN_HEAP_WARN_PERCENT=60` re-arms at 45%; a value at or below 15 latches after its first warning for
the life of the process, which is the safe direction rather than a flood.
### Run concurrency
Each instance executes up to **10** queued runs at once. Set `SKEIN_RUN_CONCURRENCY` (or the
LangGraph-compatible `N_JOBS_PER_WORKER`) to change it. Every in-flight run draws from both pools
above, so `concurrency × instances` is the number to budget against the connection cap. Prefer more
instances over higher concurrency when runs are CPU-bound. See
[runs-and-redis.md](./runs-and-redis.md#dispatch-by-thread) for thread partitioning and custom-driver fallback.
The environment variable is validated even when the flag is also passed, so a typo fails the boot
loudly instead of silently reverting to the default.
### TLS to the database
A URL with `?sslmode=require` and a real CA chain needs nothing extra — `pg` honors `sslmode`. For a
database presenting a self-signed certificate, set `DATABASE_SSL_NO_VERIFY=true`. Over a private
network (Railway's `*.railway.internal`, a VPC, a Unix socket) you need neither.
If your provider offers a **pooled** connection endpoint (PgBouncer and friends), use the **direct**
endpoint instead. Boot migrations take a session-level advisory lock, which transaction-mode pooling
does not preserve.
### Graceful shutdown
On `SIGTERM` skein stops pulling from the queue, waits `SKEIN_SHUTDOWN_GRACE_MS` (default **5000**)
for in-flight runs to finish, then aborts the stragglers so they settle to a terminal status, closes
the pools, and exits. If that whole sequence hasn't finished 3 seconds after the grace window, the
process force-exits anyway — so the default worst case is ~8s, which fits inside the tightest common
kill window.
Raise the grace window where the platform allows a longer one, and keep it below what the platform
actually grants — past that, you're just being SIGKILLed mid-drain:
| Platform | Window between SIGTERM and SIGKILL |
| ------------- | ----------------------------------------- |
| Cloud Run | 10s default, configurable |
| Railway | ~30s |
| Fly.io | `kill_timeout` in `fly.toml` (5s default) |
| Render | ~30s |
| ECS | `stopTimeout` (30s default, 120s max) |
| Kubernetes | `terminationGracePeriodSeconds` (30s) |
| `docker stop` | `-t` (10s default) |
Runs that get aborted are marked terminal, not lost: with Redis, BullMQ's stalled-job recovery
re-delivers a job whose worker died, and the worker skips any run already in a terminal state.
**Embedding skein in your own server instead of using `skein start`?** You get no signal handling —
wire it yourself: `process.on("SIGTERM", …)` → `runtime.worker.stop()` → dispose. Drain first, dispose
second; disposing while runs are still draining pulls the store out from under them.
### Cold starts
Boot does real work: schema migrations, `PostgresSaver` setup, one get-or-create per declared graph,
and eager-loading every graph module. Twenty seconds is a reasonable startup-probe budget. On
platforms that scale to zero, this is paid on the first request after an idle period.
## Scaling past one instance
With Postgres and Redis, replicas share state and streams — a client can join a run executing on
another instance, rolling deploys are safe, **and the run semantics hold across instances**. No session
affinity or single-instance restriction is needed for any of it:
- **Cancellation** crosses instances. `POST …/runs/{id}/cancel` routed to instance B stops a run
executing on instance A: the run row is settled immediately (which is what makes the cancel durable),
and the _signal to stop now_ travels over a Redis pub/sub channel to whichever instance is executing
it. Delivery is best-effort by design — a dropped message costs promptness, never correctness.
- **One-active-run-per-thread** is decided by Postgres, not by a lock in one process:
`multitask_strategy: "reject"` is an atomic check-and-insert, so two instances racing the same thread
cannot both win.
- **`multitask_strategy: "enqueue"`, `"interrupt"` and `"rollback"`** all hold. A run claims its thread
with a Postgres **session advisory lock** held for the run's duration, so a queued run waits for the
active one wherever that one is running. The displaced run's base checkpoint and the displacing run's
rollback plan are persisted on the runs themselves, so any instance can apply them — and a run
recovered after a crash still cleans up what it displaced.
**Why a Postgres lock and not a Redis lease.** A run holds its claim for its whole execution, which can
be minutes. A TTL lease has to be renewed for all of it, and a late renewal — a blocked event loop, a GC
pause — expires the lease while the run is still writing, putting two instances on one thread's
checkpoint history. A session lock has no TTL: Postgres holds it until the session releases it _or the
connection dies_, so a crashed instance frees its threads at once and a slow one keeps them. This is the
same split LangGraph Platform makes (Postgres for rows and exclusivity, Redis for ephemeral pub/sub).
**Budget for it.** Each _concurrently-executing_ run holds one connection from a dedicated pool for the
run's duration — the same trade LangGraph Platform makes. See [Connection budget](#connection-budget).
The one remaining caveat is utilization, not correctness: a queued run waiting for a busy thread still
occupies a worker slot, so a burst of `enqueue` runs on one thread can crowd out other threads' work.
Tracked as per-thread partitioned dispatch on the [roadmap](./roadmap.md). See
[runs-and-redis.md](./runs-and-redis.md).
## Streaming through proxies (SSE)
skein sends `text/event-stream` with `cache-control: no-cache, no-transform` and flushes headers
immediately. Streams are **back-pressured**: a client that reads slowly is paced rather than buffered
in the server's memory, so a few hundred slow connections cost a bounded ~65 KB each instead of a full
copy of each stream — see [streaming.md](./streaming.md#slow-clients-and-backpressure). Two things it
does _not_ do, which matter in front of a proxy:
- It sends **no `X-Accel-Buffering: no` header**. A buffering reverse proxy will hold the stream
until the run finishes, which looks exactly like a hang. Turn buffering off: nginx
`proxy_buffering off;`, ingress-nginx `nginx.ingress.kubernetes.io/proxy-buffering: "off"`, Caddy
`flush_interval -1`. Don't put a caching CDN in front of the stream routes.
- It sends **no heartbeat frame**. A stream that produces no tokens for a while can be culled by an
idle timeout, so raise the proxy's read timeout to cover your longest quiet stretch.
There is also **no server-side run timeout** under `skein start` — the engine supports one
(`runTimeoutMs`), but no CLI flag or environment variable exposes it, so it is only reachable when you
[embed skein in your own server](./embedding.md). With the image, the platform's request timeout is
the only ceiling on a streaming run; set it generously (Cloud Run allows up to 60 minutes).
Clients reconnect with `Last-Event-ID` and skein replays missed frames from a Redis stream (kept for
an hour), so a dropped connection is recoverable. See [streaming.md](./streaming.md).
## Verify a deployment
Every platform guide points here. Substitute your service's base URL.
```bash
BASE=https://your-service.example.com
# 1. Liveness — the same probe your platform uses.
curl -s $BASE/ok # {"ok":true}
# 2. Your graphs registered as assistants at boot.
curl -s -X POST $BASE/assistants/search \
-H 'content-type: application/json' -d '{"limit":10}'
# 3. An inline streaming run — exercises SSE and Postgres checkpointing end to end.
THREAD=$(curl -s -X POST $BASE/threads -H 'content-type: application/json' -d '{}' \
| node -pe 'JSON.parse(require("fs").readFileSync(0)).thread_id')
curl -N -X POST $BASE/threads/$THREAD/runs/stream \
-H 'content-type: application/json' \
-d '{"assistant_id":"agent","input":{"messages":[{"role":"user","content":"hi"}]}}'
# 4. A BACKGROUND run — the one that catches CPU-throttling platforms. It returns immediately;
# the work happens after the request ends, which is exactly when a throttled instance freezes.
RUN=$(curl -s -X POST $BASE/threads/$THREAD/runs \
-H 'content-type: application/json' \
-d '{"assistant_id":"agent","input":{"messages":[{"role":"user","content":"hi"}]}}' \
| node -pe 'JSON.parse(require("fs").readFileSync(0)).run_id')
# 5. Join it mid-flight (replayed from Redis), then confirm it finished.
curl -N $BASE/threads/$THREAD/runs/$RUN/stream
curl -s $BASE/threads/$THREAD/runs/$RUN | node -pe 'JSON.parse(require("fs").readFileSync(0)).status'
```
Step 4 is the one worth actually running. A background run that never leaves `pending`/`running` while
the service is idle means the platform is suspending your instance between requests — see that
platform's guide.
## Environment variables
skein reads these and nothing else. Note there is **no `DATABASE_URL` or `REDIS_URL`** — those are
platform names; map them onto skein's.
| Variable | Required | Purpose |
| ------------------------ | -------------------- | ----------------------------------------------------------------- |
| `POSTGRES_URI` | yes (postgres store) | Postgres connection string (resources + checkpoints). |
| `REDIS_URI` | yes (redis queue) | Redis connection string (run queue + stream pub/sub). |
| `PORT` | usually injected | Port to bind. Defaults to 8123 — the port the image exposes. |
| `HOST` | no | Host to bind. The image already passes `--host 0.0.0.0`. |
| `DATABASE_SSL_NO_VERIFY` | no | `true` to skip TLS cert verification (self-signed database cert). |
Everything else is **tuning**, and lives in one place so the numbers can't drift apart:
**[performance.md](./performance.md#every-knob)** has every knob with its default and a suggested value
for a small (256–512Mi) and a large (1–4Gi) deployment — run concurrency, the shutdown drain, page and
stream bounds, the pool and statement timeouts, the heap monitor, and request logging.
Two notes specific to the container:
- The in-memory bus knobs (`SKEIN_MEMORY_BUS_*`) are **not** reachable from this image — `skein start`
rejects `--queue memory`. They apply on the embedded path (`embedPostgresGraphs` with no `REDIS_URI`)
and under `skein dev`; see [embedding.md](./embedding.md#going-to-production).
- `PG_POOL_MAX` is per pool and skein opens **three** per instance, so budget three times your setting against
the database's own connection cap — per replica.
## Compatibility assets and diagnostics
`skein build` includes configured graph UI assets and `http.app` exports in the production artifact.
Ship the whole artifact; production does not need TypeScript sources or the development UI compiler.
The build-image gate runs the generated image and checks custom routes, UI bootstrap/assets,
OpenAPI and docs. See [configuration compatibility](./langgraph-cli-compat.md#graph-ui-and-custom-http-applications).
Use Redis thread events for reconnect across API instances. SQLite and memory event replay are
process-local, even when SQLite resource rows and checkpoints persist. Monitoring remains host-owned:
configure an OpenTelemetry reader before exposing `/metrics`, and use the storage endpoint for exact
checkpoint payload accounting. See [diagnostics](./observability.md#http-diagnostics).
---
<!-- source: docs/performance.md -->
# Performance & memory
How skein behaves under load, what bounds it, and which knob to reach for when something looks wrong.
The short version: **every buffer in skein is bounded, and every bound has a knob.** The defaults are
chosen so an ordinary deployment never loses a frame or truncates a page, which means they are not
chosen to fit a small container. If you run on 256–512Mi, read [Sizing](#sizing) — a handful of knobs do
almost all the work.
## What actually uses memory
Four things, in the order they matter:
1. **Run frames in flight.** A streaming run produces frames faster than a client consumes them. Those
frames live in the event bus until every subscriber has read them, and under `stream_mode: "values"`
each frame is a copy of the whole graph state. This is the biggest and the most variable.
2. **Socket write buffers.** Every SSE connection whose client is slower than the graph. Bounded to the
socket's own high-water mark (~64KB) — see [backpressure](#streaming-backpressure-drops-and-recovery).
3. **Rows read by one request.** A thread row carries the thread's mirrored graph state, so a search
that returns 1000 threads is 1000 graph states, twice over (the rows, then the serialized response).
4. **The runtime itself.** ~130MB RSS to import an adapter and start a server, dominated by
`@langchain/core` and `@langchain/langgraph`. This is a floor, not a variable — see
[bundling.md](./bundling.md).
Runs themselves are cheap when idle: a queued run is a row, not a process.
## Sizing
### Small: 256–512Mi, many replicas
The shape most platforms default to. The defaults do **not** fit here — they are sized so a long,
chatty run never loses a frame, and the worst case is roughly
`MAX_FRAMES_PER_RUN × (concurrent runs + MAX_RETAINED_RUNS)`. At the defaults that ceiling is half a
million frames.
```bash
SKEIN_RUN_CONCURRENCY=3
SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN=2000 # only when queue=memory
SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS=20 # only when queue=memory
SKEIN_STREAM_BUFFER_FRAMES=128 # only when queue=redis
SKEIN_MAX_PAGE_SIZE=200
PG_POOL_MAX=5
PG_STATEMENT_TIMEOUT_MS=15000
```
Worked example, 512Mi with Redis: the runtime floor is ~130MB. Three concurrent runs, each with one SSE
subscriber, buffer at most `3 × 128` frames in this process; at ~2KB per frame that is under a megabyte.
The store pool holds 5 connections, the checkpointer pool another 5. A `POST /threads/search` returns at
most 200 threads. That leaves the bulk of the container for the graph's own working set — which is where
you actually want it, and which skein cannot size for you.
Below ~345Mi also check [the heap-limit warning](./deploy.md#heap-size-vs-the-containers-memory-limit):
Node's automatic heap sizing has a floor and stops adapting there.
### Large: 1–4Gi, high concurrency
The defaults are close to right; raise concurrency to match the database.
```bash
SKEIN_RUN_CONCURRENCY=25
PG_POOL_MAX=30 # ≥ concurrency, and skein opens THREE pools per instance
SKEIN_REDIS_STREAM_MAXLEN=50000
SKEIN_STREAM_BUFFER_FRAMES=2048
PG_STATEMENT_TIMEOUT_MS=60000
```
**Budget three pools per instance** (store + checkpointer + the per-thread execution claim), so 25
concurrency at `PG_POOL_MAX=30` is 90 connections against your database's cap, per replica. `skein start`
warns at boot when concurrency exceeds `PG_POOL_MAX`, because the symptom otherwise is flat throughput
with nothing pointing at the pool.
The claim pool is the one that makes `PG_POOL_MAX ≥ concurrency` a hard requirement rather than advice:
it holds one connection per _executing_ run for that run's whole duration (see
[deploy.md](./deploy.md#connection-budget)), so a pool smaller than the concurrency simply caps how many
runs can execute at once.
### The one that isn't about size
`SKEIN_RUN_TIMEOUT_MS` is off by default and stays off unless you set it. A legitimate agent run takes
minutes; a research or multi-step tool graph takes longer. A default here would turn slow-but-working
into killed. Set it from your own graphs' worst honest case, not from a round number.
## Every knob
Defaults are what skein uses when the variable is unset. "Small" and "Large" are the starting points
above, not requirements.
### Runs
| Variable | Default | Small | Large | What it bounds |
| --------------------------------------------- | ------- | ----- | ----- | ----------------------------------------------------------------- |
| `SKEIN_RUN_CONCURRENCY` / `N_JOBS_PER_WORKER` | 10 | 3 | 25 | Queued runs one instance executes at once (`--concurrency`, `-n`) |
| `SKEIN_RUN_TIMEOUT_MS` | off | off | off | Abort a run executing longer than this (`--run-timeout`) |
| `SKEIN_SHUTDOWN_GRACE_MS` | 5000 | 5000 | 15000 | How long `SIGTERM` lets in-flight runs finish before aborting |
| `SKEIN_WEBHOOK_TIMEOUT_MS` | 5000 | 5000 | 5000 | One webhook POST. Sized to the shutdown budget — raise both |
### Streaming
| Variable | Default | Small | Large | What it bounds |
| ------------------------------------- | ------- | ----- | ----- | --------------------------------------------------------- |
| `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN` | 10000 | 2000 | 10000 | Frames one run may buffer — in-memory bus only |
| `SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS` | 50 | 20 | 200 | Finished runs still replayable to a late `join` |
| `SKEIN_REDIS_STREAM_MAXLEN` | 10000 | 2000 | 50000 | Approximate cap on a run's Redis stream (`0` = TTL only) |
| `SKEIN_STREAM_BUFFER_FRAMES` | 512 | 128 | 2048 | Frames one slow subscriber may queue before being dropped |
The socket write buffer has no knob: the socket's own high-water mark sets it, and nobody could size a
second one sensibly.
### Storage
| Variable | Default | Small | Large | What it bounds |
| -------------------------- | ---------- | ----- | ----- | ------------------------------------------------------ |
| `SKEIN_MAX_PAGE_SIZE` | 1000 | 200 | 1000 | Largest page any list/search returns |
| `PG_POOL_MAX` | pg's 10 | 5 | 30 | Connections per pool — skein opens three per instance |
| `PG_CONNECTION_TIMEOUT_MS` | 30000 | 30000 | 30000 | Waiting for a free connection (`0` = wait forever) |
| `PG_IDLE_TIMEOUT_MS` | pg's 10000 | 10000 | 30000 | How long an unused pooled client is kept |
| `PG_STATEMENT_TIMEOUT_MS` | 30000 | 15000 | 60000 | One statement server-side (`0` = off). Not per request |
### Diagnostics
| Variable | Default | What it does |
| ------------------------- | ------- | ------------------------------------------------------------------ |
| `SKEIN_HEAP_WARN_PERCENT` | 85 | Warn above this % of the heap limit (`0` disables the monitor) |
| `SKEIN_HEAP_SAMPLE_MS` | 30000 | How often the heap monitor samples |
| `SKEIN_REQUEST_LOG` | see doc | A line per HTTP request. On for `skein dev`, off for `skein start` |
Most of these can also be set in code — `worker: { maxConcurrency, shutdownGraceMs }` on any adapter,
`embedPostgresGraphs`' options, `runTimeoutMs` on the deps, or a store constructor. Where a code path
exists, the environment is still read **and validated** even when you pass the option, so a typo fails at
boot rather than sitting unnoticed in a deployment that also passes the value.
Four are environment-only: `SKEIN_WEBHOOK_TIMEOUT_MS`, `SKEIN_REQUEST_LOG` (or the CLI flag), and the two
Redis stream bounds — setting those in code means constructing `RedisRunEventBus` yourself. The webhook
timeout and the request-log switch also _fall back_ on a malformed value rather than throwing: both sit on
paths where refusing to boot would be the more damaging failure.
## Streaming: backpressure, drops, and recovery
**Backpressure.** The SSE write loop waits for the socket to drain before pulling the next frame, so a
slow client's unwritten bytes stay in the socket's own buffer (~64KB) instead of accumulating in the
process. Measured: a slow consumer's per-connection buffer went from ~1.26MB (the whole stream) to a
constant ~67KB; 100 slow streams went from 125.5MB to 6.5MB, with no change to throughput or p99 and no
frames lost. The graph does not slow down — the bus decouples it from the socket.
**Drops.** Backpressure moves the problem rather than solving it: the frames queue in the bus instead.
So the bus is bounded too, and a subscriber that falls behind the bound has its stream **ended** rather
than being handed a silent gap.
**Recovery.** An ended stream is not a lost run. Reconnect with `Last-Event-ID` and skein replays from
that point — from the in-memory buffer, or from the Redis stream, whichever bus you run. The LangGraph
SDK's `joinStream` does this for you. What you cannot recover is a frame that has aged out of
`SKEIN_REDIS_STREAM_MAXLEN` or been evicted from the in-memory buffer, which is why those bounds are the
ones to raise if reconnecting clients see gaps.
**Retention after a run ends.** A finished run's frames stay replayable so a late `join` still works —
for `SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS` runs on the in-memory bus, or until the stream's TTL on Redis.
Beyond that, joining a finished run completes immediately with no frames rather than hanging.
See [streaming.md](./streaming.md) for the wire format and [runs-and-redis.md](./runs-and-redis.md) for
what a frame costs in Redis.
## Query bounds
Every list and search path is bounded — **including when the caller passes no `limit` at all**. Before
that, one `POST /threads/search` with an empty body pulled an entire table into the heap and then
serialized it into a single response string.
- A client-supplied `limit` above 1000 is rejected.
- An absent `limit` means the first `SKEIN_MAX_PAGE_SIZE` rows, not all of them.
- `POST /threads/{id}/history` returns 100 checkpoints by default, capped at 1000, and pages with
`before`. Each element is a checkpoint's whole graph state, so this is bounded harder than a row-based
page — and separately from `SKEIN_MAX_PAGE_SIZE`, since history comes from the checkpointer rather
than the store.
- `GET /threads/{thread_id}/runs` and `POST /store/namespaces` page too, defaulting to **100** rows
(`limit`/`offset`; a query `limit` above 1000 is clamped, not rejected). 100 matches what the
LangGraph SDK sends for `store.listNamespaces`. `POST /store/namespaces` also applies the driver's
`SKEIN_MAX_PAGE_SIZE` bound — it was
the one list path that escaped it, and an `offset` with no `limit` used to be ignored outright on
Postgres, silently answering from row 0.
- A **wildcard** namespace prefix or a `suffix` cannot use the slice-equality form and is matched
per-position instead. Neither form is index-backed: `store_items`' only relevant index is its
`(namespace, key)` primary key, and Postgres will not derive a range scan from a slice expression, so
both were already sequential scans. `filter` is pushed into the `WHERE` clause on both search paths
(including the pgvector one), so the page is the top-_n_ of the **matched** set rather than the
matched subset of a page.
- Assistant search reports the unpaginated match count in `x-pagination-total`. Other collections do
not: a total costs a second query, and for those it would be a count over exactly the rows the bound
exists to avoid touching. Page them until they return fewer rows than you asked for.
With auth configured, the ownership filter is pushed into the driver query, so a tenant's search is an
indexed lookup rather than a full read filtered in JS. Details in [storage.md](./storage.md) and
[agent-protocol.md](./agent-protocol.md#authentication--authorization).
## Triage: symptom → knob
| Symptom | Likely cause | What to do |
| -------------------------------------------------------- | ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Restarts with no stack, no log line, nothing in metrics | OOM kill | Check the boot warning; below ~345Mi see [heap sizing](./deploy.md#heap-size-vs-the-containers-memory-limit). Then lower `SKEIN_RUN_CONCURRENCY` and the bus bounds. |
| `heap pressure` warning, high `runs in flight` | Too much work at once | Lower `SKEIN_RUN_CONCURRENCY`, or scale out. |
| `heap pressure` warning, high `buffered frames` | A slow SSE consumer | Lower `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN`, or move to Redis so frames live outside the process. |
| `heap pressure` warning, neither, across episodes | A genuine leak | Capture a heap snapshot. Please open an issue. |
| Throughput flat, adding concurrency does nothing | Runs queuing on the Postgres pool | Raise `PG_POOL_MAX` to ≥ concurrency (×2 for both pools). `skein start` warns about this at boot. |
| Requests hang, then fail late | Unreachable or wedged database | `PG_CONNECTION_TIMEOUT_MS` bounds the wait; `PG_STATEMENT_TIMEOUT_MS` bounds one query. |
| A query errors with `57014` | Statement timeout | The query is scanning something. Deep `OFFSET`, an unindexed `values` filter, or a very large checkpoint history — see [deploy.md](./deploy.md). Raise the timeout only after checking which. |
| Streaming clients see gaps after reconnecting | Frames aged out before the reconnect | Raise `SKEIN_REDIS_STREAM_MAXLEN` (or the memory bus's frame bound). |
| A stream ends early under load, on **Redis** | The subscriber's mailbox overflowed | Raise `SKEIN_STREAM_BUFFER_FRAMES`, or fix the consumer. `Last-Event-ID` recovers it from the durable stream. |
| A stream ends early under load, on the **in-memory bus** | Frames were evicted out from under the subscriber | Raise `SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN`. Reconnecting resumes with a gap — the buffer _is_ the replay log. `SKEIN_STREAM_BUFFER_FRAMES` does nothing here. |
| Searches return fewer rows than expected | The page bound | Expected — page with `offset`. Raise `SKEIN_MAX_PAGE_SIZE` only if you know the rows are small. |
| One thread's runs all stall behind each other | A run holding the thread lock | Runs on a thread are serialized by design. If it is a hung graph, set `SKEIN_RUN_TIMEOUT_MS`. |
| Shutdown takes ~8s and kills in-flight work | The drain window | Raise `SKEIN_SHUTDOWN_GRACE_MS`, and your platform's termination grace with it. |
## Measuring it yourself
`packages/bench` boots a real skein server in-process, opens real SSE clients against it, and samples
while they stream. It is deliberately not part of `pnpm test` — it takes minutes and needs Docker for the
Postgres/Redis driver.
```bash
nx build bench && nx bench bench # all scenarios, in-memory drivers, no Docker
nx bench bench -- --driver postgres-redis # needs Docker
nx bench bench -- --scenario slow-client --streams 500
```
The measurement that matters is **RSS after idle + a forced GC**, not peak RSS: peak tells you about
churn, retained tells you what is actually held. A slow-client run at increasing `--streams` should
plateau, not climb — that plateau is the whole point of the bounds above.
Alongside it, the counters are deterministic and make a regression provable rather than plausible:
buffered frames, Redis commands per frame, socket `writableLength`, iterator pulls, connections opened.
Those are integers with no timing in them, which is why they **do** gate CI where throughput and latency
cannot. See [testing.md](https://github.com/skein-js/skein-js/blob/main/docs/testing.md) for how the same idea is applied in the unit tests.
The run therefore ends in `all bounds hold.` or a non-zero exit listing what no longer does — the
per-stream SSE buffer ceiling, the bus's retained-run and per-run frame caps, and that backpressure
delayed frames rather than dropping them. CI's `perf bounds (bench)` job runs exactly this on the
in-memory driver and publishes the numbers to the run summary without asserting any of them. So if you
are reading this page because a bound above stopped holding for you, that job is what should have caught
it first; if it was green, the shape you hit is one the scenarios do not cover, and it is worth adding
one (see [`packages/bench/README.md`](https://github.com/skein-js/skein-js/blob/main/packages/bench/README.md)).
## Thread dispatch measurements
`pnpm exec nx run bench:dispatch --args="--output results/dispatch.json"` runs two Node processes
against shared Postgres/Redis, with four execution slots each. The hot-thread scenario enqueues
32 runs taking 250 ms on one thread, then 32 independent runs taking 25 ms. The control uses
64 independent threads. Reported cold-start latency runs from persisted run creation to graph-node
start, including the initial enqueue phase. Only exclusivity and concurrency bounds are asserted.
On 26 October 2026, Node 24.15.0 with BullMQ 5.80.2, ioredis 5.10.1, Redis 7 and Postgres 16 produced
these medians of five repetitions on the same macOS host. The benchmark harness was added before
changing dispatch. Raw samples are committed under `packages/bench/results/dispatch-baseline.json`
and `dispatch-after.json`. The latter was refreshed after review corrections; its dirty-worktree
marker distinguishes the measured source from the recorded parent commit.
| Scenario / metric | Before | Partitioned dispatch |
| ----------------------------------- | -------: | -------------------: |
| Hot backlog: cold start p50 | 7,624 ms | 191 ms |
| Hot backlog: cold start p95 | 7,792 ms | 248 ms |
| Hot backlog: cold start p99 | 7,792 ms | 250 ms |
| Hot backlog: total runs/second | 7.11 | 7.06 |
| Independent control: start p95 | 384 ms | 382 ms |
| Independent control: runs/second | 164.52 | 166.67 |
| Peak occupied slots, both scenarios | 8 | 8 |
Cold-thread p95 improves about 31-fold. Total throughput in the hot scenario remains constrained
by the same serialized chain of 32 hot runs; partitioned dispatch lets independent work start while
that chain proceeds. Both workloads fill the configured capacity at their peak. These measurements
are evidence for this workload, not throughput or latency guarantees for other machines or graphs.
---
<!-- source: docs/profiling.md -->
# Profiling and performance tuning Skein
This guide is the practical bridge between “the server feels slow” and an evidence-backed change.
It assumes no profiling experience.
## The four tools and the tuning loop
- A **benchmark** tells you how much: latency, throughput, memory, boot time.
- A **profiler** tells you where: hot functions, allocation sites, retained objects, garbage collection.
- **Telemetry** tells you when and why in production: graph, runtime, queue delay, failures, saturation.
- A **correctness test** tells you whether a faster result is still the same product.
Use the same loop every time:
1. Write an SLO, such as “p99 time to first frame below 250 ms at 100 streams.”
2. Reproduce it with a deterministic graph and fixed container CPU/memory.
3. Warm the runtime, then record a baseline and raw artifacts.
4. Profile the failing interval, not server startup or an idle process.
5. Change one variable.
6. Run protocol/conformance tests and a negative test designed to expose the suspected failure.
7. Repeat the identical benchmark and compare confidence intervals, not one lucky run.
Create a directory per experiment; generated profiles are ignored by Git and production images:
```bash
mkdir -p .profiles/2026-07-31-slow-sse/{before,after}
nx build bench
```
## Reading the numbers
Latency percentiles answer different questions. p50 is the ordinary request. p95/p99 are the tail
your users notice during queueing, GC, pool waits, or a noisy dependency. For streams, record both
**time to first frame** and total duration; an agent that starts responding in 150 ms and finishes in
20 seconds feels different from one silent for 10 seconds.
Throughput is completed requests/second or frames/second. It means little without concurrency and
saturation. Increase concurrency until throughput stops rising; then inspect CPU, event-loop delay,
Postgres pool waits, Redis, and the load generator. The first saturated component is the limit.
Memory has several layers:
- **JS heap used** is live/recent JavaScript data.
- **Heap capacity/limit** is what the collector/runtime reserved or may grow to; it is not usage.
- **RSS** is resident process memory: JS heap plus runtime, native clients, stacks, code, and buffers.
- **Native/external memory** includes socket buffers and runtime allocations outside the JS heap.
- A healthy GC graph is a sawtooth. A leak has a rising _post-GC floor_ after the same work becomes
unreachable. A bounded cache rises and then plateaus.
For CPU profiles, **self time** is time in the function itself. **Total time** includes callees. High
self time points at the hot implementation; high total but low self time says to descend into its
children. Wide flame-graph boxes consumed more samples. GC width means allocation pressure, not
necessarily that the collector itself is defective.
## Skein’s cross-runtime benchmark
Use deterministic graphs first—no model calls—then repeat with a separate realistic model workload.
The existing harness exercises real HTTP/SSE and can use real Postgres and Redis:
```bash
nx bench bench -- --scenario fast-client
nx bench bench -- --scenario slow-client --streams 1
nx bench bench -- --scenario slow-client --streams 50
nx bench bench -- --scenario slow-client --streams 100
nx bench bench -- --scenario slow-client --streams 500
nx bench bench -- --driver postgres-redis
```
Do not model a slow client with `fetch()` plus delayed reads: the client runtime may eagerly drain the
socket into its own heap. Skein’s slow-client harness pauses a raw socket so the TCP receive window
closes and backpressure reaches the server.
For Node/Bun/Deno comparisons there is **no harness support yet** — `packages/bench` starts the server
inside its own process, so it cannot measure another runtime, and adding an external-target mode is
tracked follow-up work. Today the comparison is a manual procedure: build each production artifact and
run the load generator outside the measured container. Keep identical CPU/memory limits, graph bundle, payloads, Postgres/Redis versions,
telemetry setting, warm-up, sample duration, and runtime image architecture. Record:
- cold boot and readiness;
- p50/p95/p99 and time to first frame;
- requests/s and frames/s;
- RSS, runtime heap values, and settled post-GC memory;
- memory slope at 1/50/100/500 slow streams;
- queue delay and Postgres pool saturation;
- Redis commands, connections, and retained frames;
- telemetry-off versus telemetry-on overhead;
- SIGTERM drain duration and lost terminal states/events;
- a sustained mixed-load soak.
Run at least five measured repetitions after warm-up and report the median plus a 95% confidence
interval. Save raw JSON beside runtime, image digest, CPU model, memory limit, OS/kernel, and commit.
## Node recipes
Profile the built production entry, send representative traffic from another process, then terminate
it cleanly so profiles flush:
```bash
mkdir -p .profiles/node/cpu .profiles/node/heap
node --cpu-prof --cpu-prof-dir=.profiles/node/cpu \
packages/cli/dist/index.js start --runtime node
node --heap-prof --heap-prof-dir=.profiles/node/heap \
packages/cli/dist/index.js start --runtime node
node --trace-gc packages/cli/dist/index.js start --runtime node
node --inspect packages/cli/dist/index.js start --runtime node
```
Load `.cpuprofile` in Chrome DevTools’ Performance panel. Load `.heapprofile` in Memory. In an
inspector session, take two heap snapshots only after the same forced-idle/GC point, repeat the
workload between them, and compare retained size and retaining paths. Snapshots pause the process and
can temporarily require roughly another heap’s worth of memory; capture them on a replica you can
remove from traffic.
## Bun recipes
```bash
mkdir -p .profiles/bun/cpu .profiles/bun/heap
bun --cpu-prof --cpu-prof-dir=.profiles/bun/cpu \
packages/cli/dist/index.js start --runtime bun
bun --heap-prof --heap-prof-dir=.profiles/bun/heap \
packages/cli/dist/index.js start --runtime bun
```
Bun has separate JavaScriptCore and native heaps. For an instrumented experiment, record
`heapStats()` from `bun:jsc` for JS objects, `Bun.memoryUsage()` for Bun/native allocations, and RSS.
A flat JS heap with rising RSS points away from retained JS objects and toward native/socket buffers.
Bun’s heap profiler can also emit Markdown summaries; follow the options supported by the pinned Bun
version used by the artifact.
## Deno recipes
The production launcher needs explicit access to the artifact, environment, network, system data, and
native dependencies. Deno’s profiler can produce the raw profile, a Markdown report, and an
interactive flamegraph together:
```bash
mkdir -p .profiles/deno/cpu
deno run --allow-net --allow-env --allow-read=. --allow-sys --allow-ffi \
--cpu-prof --cpu-prof-dir=.profiles/deno/cpu \
--cpu-prof-md --cpu-prof-flamegraph \
packages/cli/dist/index.js start --runtime deno
deno run --inspect --allow-net --allow-env --allow-read=. --allow-sys --allow-ffi \
packages/cli/dist/index.js start --runtime deno
```
Open the SVG directly or load `.cpuprofile` in DevTools. Deno reports transpiled JavaScript line
numbers in CPU profiles, so use function names and sourcemaps to get back to TypeScript.
## Five diagnostic exercises
### 1. SSE backpressure
Run `slow-client` at 1, 50, 100, and 500 streams. Watch server RSS, socket buffered bytes, and bus
frames. Expected: socket bytes and bus retention plateau at configured bounds. Negative verification:
temporarily replace the raw paused client with a fast client—the pressure disappears, proving a fast
client cannot validate slow-client safety.
### 2. Event-bus retention
Run `long-run`, let all clients disconnect, force idle+GC, and inspect retained objects. Expected:
completed-run retention stops at `SKEIN_MEMORY_BUS_MAX_RETAINED_RUNS`, and frames per run stop at
`SKEIN_MEMORY_BUS_MAX_FRAMES_PER_RUN`. A retaining path through a subscription after cancellation is
a leak; a bounded map at its configured ceiling is not.
### 3. Serialization cost
Compare `stream_mode: "values"` with `"updates"` using the same graph. If CPU self time concentrates
in wire serialization and GC grows, reduce full-state frames or payload size. Verify final state and
frame order before accepting the change.
### 4. Postgres pool saturation
Sweep run concurrency through 1/5/10/20 while holding `PG_POOL_MAX=5`, then repeat at 20. Throughput
should flatten and queue time rise when the pool saturates. Raising worker concurrency alone cannot
fix a five-connection bottleneck; remember Skein uses separate store and checkpointer pools.
### 5. Telemetry shutdown leak
Create a test sink that buffers events and whose `flush()` rejects. Send SIGTERM. The test passes only
if `shutdown()` still runs, the other sinks flush, terminal run state is persisted, and the process
exits within the configured grace. This distinguishes exporter failure from lifecycle loss.
## Benchmark traps
- Warm-up/JIT: keep cold start as its own metric; do not mix it into steady state.
- Coordinated omission: a closed-loop generator waits during stalls and under-samples the stall.
- Noisy neighbours and CPU turbo: pin container resources and repeat.
- A weak load generator: if its CPU is full, you measured the client. This is especially easy with a
fast native Bun server.
- Model/network variance: deterministic graphs establish runtime overhead; model-backed tests answer a
different question.
- Different hardware, regions, images, database sizes, or TLS paths: not a runtime comparison.
- Average-only reporting: averages hide queue and GC tails.
- Optimizing without a negative verification: a benchmark win can be dropped frames or skipped work.
## Comparing Skein with LangGraph Platform
Treat Platform as a black box. Use the same graph, SDK client, client region, payloads, concurrency,
warm-up, and observation window. Publish raw results and methodology. A “better” claim requires full
protocol correctness plus a reproducible win in p99, memory, throughput, cold start, or operating cost,
with no material regression in the others. If infrastructure cannot be made identical, label the
result as an end-to-end deployment comparison rather than a runtime benchmark.
Official references: [Node CLI profiling](https://nodejs.org/api/cli.html#--cpu-prof),
[Bun benchmarking and profiling](https://bun.sh/docs/project/benchmarking), and
[Deno CPU profiling](https://docs.deno.com/runtime/fundamentals/cpu_profiling/).
---
<!-- source: docs/deploy-cloud-run.md -->
# Deploy on Google Cloud Run
Cloud Run runs your container, scales it, and bills per request — a good fit for skein, with **one
sharp edge**: by default Cloud Run throttles a container's CPU to near-zero between requests, and
skein's background runs do their work _after_ the request that created them has returned. Get that
setting wrong and inline runs work perfectly while background runs mysteriously never finish.
Everything platform-agnostic — environment variables, pool sizing, probes, scaling caveats — is in
[deploy.md](./deploy.md).
## Before you start
You need the `gcloud` CLI authenticated, a project with billing enabled, and Docker. Enable the APIs
once:
```bash
export PROJECT_ID=your-project REGION=us-central1 REPO=skein SERVICE=skein-app
gcloud config set project $PROJECT_ID
gcloud services enable run.googleapis.com artifactregistry.googleapis.com secretmanager.googleapis.com
```
## 1. Build and push the image
```bash
# Cloud Run runs x86. skein build doesn't pass --platform, so set this on Apple Silicon.
export DOCKER_DEFAULT_PLATFORM=linux/amd64
export IMAGE=$REGION-docker.pkg.dev/$PROJECT_ID/$REPO/$SERVICE:v1
gcloud artifacts repositories create $REPO --repository-format=docker --location=$REGION
gcloud auth configure-docker $REGION-docker.pkg.dev
skein build -t $SERVICE
docker tag $SERVICE $IMAGE
docker push $IMAGE
```
## 2. Provision Postgres + Redis
Anything reachable works. Two shapes are common:
**Managed outside GCP** (Neon, Supabase, Upstash, Redis Cloud) — simplest, no VPC needed, and the
connection strings work as-is. Use the **direct** Postgres endpoint, not a pooled one
([why](./deploy.md#tls-to-the-database)).
**GCP-native** — Cloud SQL and Memorystore:
- **Cloud SQL** attaches over a Unix socket, which needs no TLS configuration. `POSTGRES_URI` is
handed straight to `pg`, so the socket form works:
```text
postgresql://USER:PASSWORD@/DATABASE?host=/cloudsql/PROJECT:REGION:INSTANCE
```
Add `--add-cloudsql-instances=PROJECT:REGION:INSTANCE` to the deploy below.
- **Memorystore** is only reachable from inside your VPC, so the service needs Direct VPC egress or a
Serverless VPC Access connector (`--network`/`--subnet`, or `--vpc-connector`).
Store both URIs in Secret Manager rather than passing them as plain environment variables —
`gcloud run services describe` prints env vars in cleartext:
```bash
printf '%s' "$POSTGRES_URI" | gcloud secrets create skein-postgres-uri --data-file=-
printf '%s' "$REDIS_URI" | gcloud secrets create skein-redis-uri --data-file=-
```
Grant the service's runtime service account `roles/secretmanager.secretAccessor`.
## 3. Deploy
```bash
gcloud run deploy $SERVICE \
--image=$IMAGE \
--region=$REGION \
--port=8123 \
--set-secrets=POSTGRES_URI=skein-postgres-uri:latest,REDIS_URI=skein-redis-uri:latest \
--set-env-vars=PG_POOL_MAX=5,SKEIN_RUN_CONCURRENCY=5 \
--no-cpu-throttling \
--min-instances=1 \
--max-instances=3 \
--cpu=1 --memory=1Gi \
--timeout=3600 \
--no-allow-unauthenticated
```
The flags that matter, and why:
| Flag | Why |
| ------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| `--port=8123` | Cloud Run injects `PORT`, and skein binds it. This tells Cloud Run which port to _send_ traffic to. |
| `--no-cpu-throttling` + `--min-instances=1` | **Required for background runs.** See [below](#background-runs-need-cpu-outside-requests). |
| `--timeout=3600` | The ceiling on a streaming SSE run; skein imposes none of its own. |
| `--no-allow-unauthenticated` | skein's auth is off by default — see [the warning](./deploy.md#5-auth--read-this-before-you-expose-it). |
| `PG_POOL_MAX=5` | Two pools per instance × instances vs. your database's cap ([budget](./deploy.md#connection-budget)). |
Cloud Run's built-in health checking uses the container port; you can also declare an explicit
startup probe on `GET /ok` with a ~30s failure budget, since boot runs migrations before listening.
## 4. Verify
With `--no-allow-unauthenticated`, reach the service through an authenticated tunnel rather than
opening it up:
```bash
gcloud run services proxy $SERVICE --region=$REGION --port=8123
```
Then run the [verification sequence](./deploy.md#verify-a-deployment) against `http://localhost:8123`.
Step 4 — the background run — is the one that proves the CPU settings are right.
## Cloud Run caveats
### Background runs need CPU outside requests
This is the big one. `POST /threads/{id}/runs` enqueues a run and responds immediately; the
worker executes it afterwards. Under Cloud Run's default _CPU-allocated-during-requests_ model, the
instance's CPU is throttled to near-zero the moment the response is sent, so that run stops making
progress — and with `--min-instances=0` the instance may be shut down entirely.
Set **`--no-cpu-throttling`** (CPU always allocated) and **`--min-instances=1`**. This costs more —
you're paying for an always-on vCPU rather than per-request — but it's what makes background runs,
webhooks and cron-shaped work behave.
If you only use inline runs (`/runs/wait`, `/runs/stream`), the work happens _during_ the request and
the defaults are fine. Scale to zero freely.
### Request timeout vs. streaming
Default 300s, max 3600s. A long SSE run is a long request. skein sends no heartbeat frame, so a
stream that goes quiet still counts against the timeout — set `--timeout` to cover your worst case.
Don't put Cloud CDN in front of the streaming routes; it buffers.
### Connections vs. autoscaling
Each instance opens two Postgres pools, so a burst to `--max-instances=10` with `PG_POOL_MAX=10`
wants 200 connections. Cap `--max-instances`, keep `PG_POOL_MAX` small, or front Cloud SQL with the
Auth Proxy or a pooler.
### Shutdown
Cloud Run sends `SIGTERM` and SIGKILLs 10 seconds later by default. skein drains in-flight runs for
5s, aborts the rest so they land in a terminal status, and exits — comfortably inside that window. If
you raise `SKEIN_SHUTDOWN_GRACE_MS`, raise Cloud Run's termination grace period to match, or you'll
be killed mid-drain.
Note that Cloud Run has no `init` process. The image is fine as-is, but if your graphs spawn child
processes you'll want `tini` — add it via `dockerfile_lines` in `langgraph.json`.
### Multi-instance semantics
With `--max-instances` above 1 and Postgres + Redis configured, cross-instance cancellation and the
one-run-per-thread guard both hold — no session affinity needed. See
[Scaling past one instance](./deploy.md#scaling-past-one-instance) for what that costs in Postgres
connections.
---
<!-- source: docs/bundling.md -->
# Bundling skein
Most people never read this page: if you deploy the `skein build` image, or run `next start` with
skein mounted as a route handler, bundling is already handled. You need this page when **you** own
the bundler — an rspack/webpack server build, an esbuild bundle for Lambda, a Next.js app with an
unusual config — and skein is inside its module graph.
The short version: **skein is ESM-only, and everything except the CLI bundles cleanly.** Two packages
must stay external, and only if you use the `langgraph.json` on-ramp.
## skein is ESM-only
Every `@skein-js/*` package is `"type": "module"` and ships a single ESM entry. There is no CommonJS
build and there won't be one.
That does **not** mean you can't `require()` it. Each library package exposes a `default` export
condition, so Node's `require(esm)` resolves it:
```js
const { embedPostgresGraphs } = require("@skein-js/runtime"); // works on Node 20.19+ / 22.12+
```
`require(esm)` landed unflagged in **Node 20.19** and **22.12**. On an older Node you get
`ERR_REQUIRE_ESM` telling you to use `import()` instead — which is the honest answer, and what a
CJS-emitting bundler should be configured to do. Plain `import` works on any Node ≥ 20.
> Before 0.10.0 the packages declared only `types` + `import` conditions, so `require()` failed with
> `ERR_PACKAGE_PATH_NOT_EXPORTED` — which reads like the package is broken rather than like a module
> format mismatch. If you're pinned below 0.10.0, that's the fix.
>
> The `skein-js` CLI keeps the old shape deliberately: it has no exports, and its entry point runs a
> command as a side effect. `require()`-ing it should fail.
**`require()` does not tree-shake.** `require("@skein-js/runtime")` eagerly loads
`@skein-js/config` and therefore `@langchain/langgraph-api`, even if you only use
`embedPostgresGraphs` and never touch a `langgraph.json`. It works, it just costs ~0.5s of cold
start. Prefer `import` where your toolchain allows it.
## What must stay external
| Package | Bundle it? | Why |
| -------------------------- | ------------------------- | --------------------------------------------------------------------------------------- |
| `@langchain/langgraph-api` | **No** — mark external | skein's graph loader `import()`s a path computed at runtime; no bundler can follow that |
| `@typescript/vfs` | **No** — mark external | pulled in by the same loader |
| `skein-js` (the CLI) | **Never** | a `bin` with no exports and top-level await; importing it runs a command |
| every other `@skein-js/*` | **Yes** — bundles cleanly | including `@skein-js/storage-postgres` as of 0.10.0 |
The first two only matter if you point skein at a `langgraph.json` (`buildRuntime`, the CLI, the
`{ config }` form of any adapter). If you embed graphs in code — `embedPostgresGraphs`,
`embedInMemoryGraphs`, the `{ deps }` form — nothing reaches them: `@langchain/langgraph-api` is loaded
with `await import()` at the two points that genuinely need it (analysing a graph's schema, and adapting
a user's `Auth` instance), and the in-memory runtime loader is behind a dynamic import on the `{ config }`
branch. So on an embedded path they are never in the module graph at all, rather than being present and
merely tree-shakeable.
That is asserted, not asserted-by-comment: `packages/test-support/src/static-imports.test.ts` walks each
adapter's built output — following `@skein-js/*` edges into their own `dist` — and fails if
`@langchain/langgraph-api`, `@typescript/vfs`, or `superjson` is statically reachable. The walk is
transitive because the regression it caught was: no adapter imported `@langchain/langgraph-api`, but
every adapter imported `@skein-js/server-kit`, which imported the `@skein-js/config` barrel for one error
class, and that barrel imported `@langchain/langgraph-api`.
One trade-off worth knowing: because `@langchain/langgraph-api` is now loaded on demand, a bundling
mistake around it (the `serverExternalPackages` config below) surfaces when something first asks for a
graph schema rather than at startup. The container boots and passes its probes, and
`GET /assistants/{id}/schemas` returns a 500. Bake your schemas at build time (`skein build` does) and
the path is never taken at all.
Two consequences for the public API, both of which exist to keep that graph clean:
- `SkeinConfigError` is importable from `@skein-js/config/errors` as well as the root, and internal code
uses the subpath. The root barrel is the `langgraph.json` loader.
- `readLanggraphDevState` / `loadSnapshotIntoStore` / `describeSnapshot` live at
`@skein-js/server-kit/dev`, not on the root barrel — they carry `superjson` and `node:fs/promises`, and
only `skein dev` / `skein import` call them. They are deliberately **not** re-exported from the root or
from `@skein-js/express`: a re-export is still a static import, which would undo the split.
## Copy-paste configs
**Next.js** (`next.config.mjs`):
```js
export default {
serverExternalPackages: ["@langchain/langgraph-api", "@typescript/vfs"],
};
```
**webpack / rspack** (server build):
```js
export default {
target: "node",
externals: [
{ "@langchain/langgraph-api": "commonjs @langchain/langgraph-api" },
{ "@typescript/vfs": "commonjs @typescript/vfs" },
],
};
```
**esbuild**:
```bash
esbuild server.ts --bundle --platform=node --format=esm \
--external:@langchain/langgraph-api --external:@typescript/vfs
```
Prefer `--format=esm` if you can. With `--format=cjs`, anything the bundle `require()`s at runtime
still needs Node 20.19+, per above.
## The "Critical dependency" warning
webpack and rspack emit this when they meet skein's graph loader:
```text
Critical dependency: the request of a dependency is an expression
```
It's expected and harmless — that expression is the `import()` of your graph module, resolved from
`langgraph.json` at runtime. Externalizing `@langchain/langgraph-api` removes most of it; to silence
the rest:
```js
// next.config.mjs
export default {
serverExternalPackages: ["@langchain/langgraph-api", "@typescript/vfs"],
webpack: (config) => {
config.ignoreWarnings = [
...(config.ignoreWarnings ?? []),
{ message: /Critical dependency: the request of a dependency is an expression/ },
];
return config;
},
};
```
## Postgres migrations are compiled in
`@skein-js/storage-postgres` needs **no skein-side externals**. Its schema migrations ship as string
constants inside `dist/index.js`, so the package makes no filesystem access at runtime — it imports
only `node:crypto`, `pg`, and `@skein-js/core`. (`pg` itself has one wrinkle — see below.)
Before 0.10.0 it located its `migrations/` directory with `new URL("../migrations", import.meta.url)`
and handed it to `node-pg-migrate`. Bundlers rewrite `import.meta.url` to the **output** location, so
a bundled build looked fine until boot, then failed to find its own SQL against a real database. If
you hit that on an older version, externalize `@skein-js/storage-postgres` (which then has to be
present in `node_modules` at runtime) or upgrade.
Migrations still run automatically on boot, tracked in a `skein_migrations` table and serialized by a
Postgres advisory lock — see [deploy.md](./deploy.md) and [storage.md](./storage.md).
### One caveat, from `pg` rather than skein
`pg` has an optional native binding it reaches for at runtime: `pg/lib/native/client.js` does
`require('pg-native')`, and `pg-native` is an optional peer that is normally not installed. pg wraps
that call in a `try`/`catch` specifically so bundlers tolerate it, and **esbuild does** — it leaves a
runtime `require` and emits nothing. **webpack and rspack are stricter** and report:
```text
Module not found: Can't resolve 'pg-native'
```
You do not want the native binding; tell the bundler to ignore it:
```js
// webpack / rspack
import webpack from "webpack";
export default {
plugins: [new webpack.IgnorePlugin({ resourceRegExp: /^pg-native$/ })],
};
```
Next.js keeps `pg` on its built-in server-externals list, so this never surfaces there.
## What `skein build` inlines vs. externalizes
The section above is about bundling **skein**. This one is about the bundler skein itself runs:
`skein build` compiles your graphs (plus auth, custom embed, custom telemetry sinks) into
`.skein/build`, and the split it makes there is the reason the production image is small and the
monorepo case works at all.
**Inlined into the artifact** — your own source, including anything reached through a `tsconfig`
`paths` alias or a workspace link (`@myorg/js`, the Nx/Turborepo/pnpm-workspace pattern). Those files
exist nowhere a package manager could install them from, so resolving them once on the build host is
what dissolves the "my Docker build context doesn't contain my monorepo" problem.
**Left external and pinned** — every published `node_modules` package. `skein build` records each one
at the exact version installed on the build host and writes it into the artifact's `package.json`,
which the image installs with `npm install --omit=dev`. Externalizing is not a limitation to work
around; it is load-bearing:
- **One copy of each library.** The image installs `skein-js`, which brings `@langchain/langgraph` and
`@langchain/core`, and that runtime is what imports your graph bundle. Inline `@langchain/core` into
the graph and there are two copies: `instanceof BaseMessage` starts failing, and config/callbacks
propagate through a different `AsyncLocalStorage` than the one the runtime reads.
- **Native addons can't be inlined.** `pg-native`, `sharp`, `better-sqlite3` and friends are platform
binaries; they have to be installed for the image's platform.
- **Package-relative asset reads survive.** A bundler rewrites `import.meta.url`/`__dirname` to the
output location, which breaks packages that load workers, wasm, or data files from beside
themselves (pdfjs, `tiktoken`, …) — the same hazard that made skein
[compile its own SQL in](#postgres-migrations-are-compiled-in).
- **Cheaper rebuilds.** `COPY package.json` + install is a cached Docker layer; a graph edit re-ships
only the bundle.
The one thing a bundler structurally cannot see is a package imported **by name at runtime** —
`initChatModel` doing `import("@langchain/" + provider)`, a plugin loaded from config. Those never
appear in the module graph, so declare them under `dependencies` in `langgraph.json`:
```json
{
"graphs": { "agent": "./src/graph.ts:graph" },
"dependencies": ["@langchain/openai"]
}
```
(skein pins the packages behind a declared `store.index.embed` provider and a declared `telemetry`
provider for you — the field is for the ones only your code knows about.)
`skein build` fails on the **host** if the artifact would ship an import it does not install, so a
missing pin is a build-time error with a package name in it rather than an `ERR_MODULE_NOT_FOUND` from
inside `docker build`.
## See also
- [deploy.md](./deploy.md) — deploying the built image, env vars, probes, scaling
- [embedding.md](./embedding.md) — the in-code on-ramp, which avoids the graph loader entirely
- [storage.md](./storage.md) — the Postgres driver and its schema
---
<!-- source: docs/reuse.md -->
# Reuse-first architecture
> **Contributor / design doc.** This explains the reuse-first architecture that shapes skein-js — what
> it reuses from `@langchain/*` versus what it rebuilds. If you just want to _use_ skein-js, start with
> the [README](https://github.com/skein-js/skein-js/blob/main/README.md) and [docs index](./index.md). To contribute, see
> [CONTRIBUTING.md](https://github.com/skein-js/skein-js/blob/main/CONTRIBUTING.md) and [AGENTS.md](https://github.com/skein-js/skein-js/blob/main/AGENTS.md).
<!-- -->
> **Principle:** Reuse as much of the LangGraph open source as possible. Build only what
> LangGraph OSS does not already give us — and build that well.
LangChain publishes a large, **MIT-licensed** JavaScript ecosystem under
[`langchain-ai/langgraphjs`](https://github.com/langchain-ai/langgraphjs). Crucially — and
unlike the Python side, where the equivalent is not open — the **JS Agent Protocol dev server
itself is open source**
([`@langchain/langgraph-api`](https://github.com/langchain-ai/langgraphjs/tree/main/libs/langgraph-api), MIT).
So skein-js is deliberately _thin_. We stand on the OSS runtime, checkpointers, parser,
schemas, and SDK, and add only the **durable-production, multi-framework, drop-in-CLI**
layer that OSS does not provide.
## What skein-js reuses (dependencies, all MIT)
| Concern | LangGraph OSS package | How skein-js uses it |
| ----------------------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Graph runtime | `@langchain/langgraph` | Run graphs via `CompiledStateGraph.invoke` / `.stream`; interrupts + resume for human-in-the-loop. Never reimplemented. Reached through `@skein-js/langgraph`, which is the only package that imports it at runtime. |
| Checkpoint base + dev saver | `@langchain/langgraph-checkpoint` | `BaseCheckpointSaver`, `MemorySaver` for dev. |
| Postgres checkpoints | `@langchain/langgraph-checkpoint-postgres` | `PostgresSaver` for graph state in prod. Its `./store` `PostgresStore` is also usable as a drop-in long-term-memory store via `store.adapter` — see [storage.md](./storage.md#bringing-your-own-store-storeadapter) — which brings hybrid `text \| vector` search skein's own driver lacks. |
| Long-term store interface | `@langchain/langgraph-checkpoint` | `BaseStore` in both directions: `SkeinBaseStore` (in `@skein-js/langgraph`, because it _extends_ `BaseStore`) exposes skein's store to graphs as one, and `fromBaseStore` (in `@skein-js/agent-protocol`, whose import is type-only) accepts anyone's as skein's store. `InMemoryStore` is the conformance target for that adapter. |
| Redis checkpoints | `@langchain/langgraph-checkpoint-redis` | Optional Redis-backed checkpointer. (Note: distinct from `@skein-js/redis`, which is the run **queue**.) |
| SQLite checkpoints | `@langchain/langgraph-checkpoint-sqlite` | `SqliteSaver` in `@skein-js/local`, sharing the SQLite resource driver's borrowed connection. LangGraph owns checkpoint tables and migrations. |
| Checkpointer conformance | `@langchain/langgraph-checkpoint-validation` | Reused as-is in our test suite to validate any checkpointer wiring. |
| **Agent Protocol dev server** | **`@langchain/langgraph-api`** (MIT) | Reuse its public exports: `./schema` (the `langgraph.json` parser), `./auth`'s pure `isAuthMatching` (so our ownership-filter `$eq`/`$contains` semantics match), `./experimental/embed` (store-search embeddings). Its Zod schemas + in-memory handler logic (MIT) are the reference we adapt for the durable drivers. |
| CLI + config semantics | `@langchain/langgraph-cli` | Reference for `langgraph.json` fields and `dev/up/build/dockerfile` behavior we mirror. |
| Wire types + JS client | `@langchain/langgraph-sdk` | **Reuse the SDK's TypeScript types** for Thread/Run/Assistant/etc. as our wire contract instead of regenerating; also our conformance oracle. |
| React streaming | `@langchain/react` | `useStream` — a target client we satisfy, not something we build. The dedicated Vue, Svelte and Angular packages share the thread event protocol; legacy React remains tested. |
| Chat UI | `langgraph-ui` / Agent Chat UI | Interop target for smoke tests. |
**Rule of thumb:** if a `@langchain/*` package already does it, we depend on it (as a
`peerDependency` where the consumer should own the version). We do not fork or vendor it.
Subgraph SSE event naming (`mode|<namespace>`) follows `@langchain/langgraph-api`'s `streamState`.
That helper is not a public export, so Skein mirrors only its wire encoding. Deep Agents itself
remains an upstream dependency used directly by application graphs.
## What skein-js rebuilds (the gap)
`@langchain/langgraph-api` is explicitly an **in-memory dev server** ("in-memory mode,
suitable for development and testing"), built on Hono and oriented around the CLI. It does
not aim to provide durable production infrastructure. That gap — the same one aegra fills
for Python — is skein-js's actual product:
| Gap in OSS | skein-js package | Why it's needed |
| ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Durable persistence of **protocol resources** (assistants / threads / runs / store rows) | `@skein-js/storage-postgres` or `@skein-js/storage-sqlite` + `SkeinStore` | Protocol resources need durable storage for server or local deployment. Graph checkpoints reuse the upstream Postgres/SQLite saver. |
| Local resource and job admission | `@skein-js/storage-sqlite` | One owned file commits protocol resources and queued jobs together; LangGraph continues to own execution and checkpoints. |
| Durable **background-run queue** + **cross-instance pub/sub** streaming | `@skein-js/redis` | The OSS server runs runs in-process; horizontal scaling needs a real queue and fan-out. |
| **Framework-native adapters** | `@skein-js/express` (· `@skein-js/fastify` · `@skein-js/nestjs`) | The OSS server is Hono-only; teams want to mount the protocol into their existing Express/Fastify/Nest app. |
| **Normalized protocol core** tying runtime + checkpointer + store + queue together | `@skein-js/core` | Adapter- and driver-agnostic handlers so behavior is identical everywhere. |
| **Drop-in production CLI** | `skein-js` | `skein dev/up/build/dockerfile` reading an unchanged `langgraph.json`, wiring the durable drivers. |
| **`langgraph.json` loading orchestration** | `@skein-js/config` | Thin wrapper over `@langchain/langgraph-api`'s `./schema` parser, adding `skein.json` overrides and driver selection. |
| **Channels for cross-provider workflows** | `@skein-js/channels` | Skein supplies durable sources and destinations; LangGraph owns the workflow and writes an explicit destination intent to the existing callback/outbox. |
| **Native and BullMQ access** | `@skein-js/agent-protocol` · optional `@skein-js/bullmq` | Compose existing authorization, run services, result storage and outbox around atomic admission/receipts. LangGraph still owns execution and checkpoints. |
## Consequences
- **Small surface, few bugs.** Most agent behavior lives in battle-tested LangChain code;
skein-js's own code is persistence, transport, and wiring.
- **Version alignment.** Reused packages are `peerDependencies` so apps control the exact
LangGraph version and avoid duplicate installs.
- **Upgrades are cheap.** When LangGraph ships new stream modes or schema fields, skein-js
inherits them through the shared runtime/types instead of chasing them by hand.
- **Honest positioning.** skein-js is an open alternative to LangGraph Platform — "aegra for
TypeScript" — with a narrower job to do, because on JS the server internals are already open. We
add production durability, not a second server.
See [code-practices.md](./code-practices.md) for how we keep the code we _do_ write small
and neat, and [storage.md](./storage.md) / [runs-and-redis.md](./runs-and-redis.md) for the
rebuilt pieces in detail.
Thread protocol execution delegates to LangGraph’s public `streamEvents({version: "v3"})`;
subscription and reconnect do not invoke a graph. The legacy graph stream remains available for
run-based clients. MCP and A2A use their official SDKs for transport and canonical serialization,
while Skein’s existing authorization, run admission, queue, persistence and cancellation remain the
execution authority. A2A results live on run rows rather than in a second task database.