langsmith · diff
v2.0.0 to v2.1.0
193 added, 221 removed. Audit A to A.
---
name: langsmith
description: >
- Instrument, trace, evaluate, and review LLM applications with LangSmith by choosing the
- right workflow first: debugging traces, offline evals, online feedback/review queues,
- prompt-registry work, or multi-service trace propagation. Use when the user needs
- LangSmith for LLM observability, LangChain tracing, `@traceable` / `traceable`,
- provider wrappers like `wrap_openai` / `wrapOpenAI`, dataset-backed regression checks,
- annotation queues, experiment comparison, or prompt/version review; even if they only
- mention trace IDs, LLM judge scores, human feedback, run trees, prompt hub, or
- production confidence for an AI feature. Not for generic alerting/SLO design,
- deployment orchestration, or runtime guardrails outside LangSmith.
+ Route LangSmith work into one workflow packet before touching SDK code.
+ Use when the user needs LangSmith tracing, offline evals, annotation/review queues,
+ prompt-registry decisions, audit/gap review, or cross-service trace propagation for
+ an LLM app or agent workflow. Choose one packet: trace-debug, eval, review,
+ prompt-registry, propagation, or audit. Triggers on: LangSmith, LangChain tracing,
+ `@traceable` / `traceable`, `wrap_openai` / `wrapOpenAI`, datasets, experiments,
+ annotation queues, feedback criteria, Prompt Hub, run trees, trace IDs, or
+ production confidence for an AI feature. Not for generic SLO/alert design,
+ non-LangSmith deployment orchestration, or runtime guardrails outside LangSmith.
allowed-tools: Bash Read Write Edit Glob Grep WebFetch
license: MIT
compatibility: >
- Works best when LangSmith is already the chosen tracing/eval substrate or the user needs
- help deciding how to instrument/debug/evaluate an LLM app. Best with Python or
- TypeScript projects plus access to `LANGSMITH_API_KEY`; self-hosted setups may also need
- `LANGSMITH_ENDPOINT` and `LANGSMITH_WORKSPACE_ID`.
+ Works best when LangSmith is already the chosen tracing/eval substrate or the user
+ needs help deciding how to instrument, debug, evaluate, or review an LLM app.
+ Best with Python or TypeScript projects plus access to `LANGSMITH_API_KEY`;
+ self-hosted setups may also need `LANGSMITH_ENDPOINT` and `LANGSMITH_WORKSPACE_ID`.
metadata:
tags: langsmith, langchain, tracing, observability, evaluation, llm-monitoring, prompt-hub, datasets, openevals
- version: "2.0.0"
+ version: "2.1.0"
source: https://docs.langchain.com/langsmith/home
---
# LangSmith
- Use this skill to turn a vague “set up LangSmith” request into a **mode-specific operator packet**.
-
- The goal is **not** to dump SDK trivia.
- The goal is to decide:
- 1. what LangSmith workflow the user actually needs,
- 2. how much instrumentation is required,
- 3. what should stay inside LangSmith versus route out to adjacent skills,
- 4. what evidence packet to produce before implementation or rollout.
-
- Read [references/modes-and-routing.md](references/modes-and-routing.md) before handling mixed requests that blur tracing, evals, feedback workflows, prompt registry work, and generic observability.
- Read [references/python-sdk.md](references/python-sdk.md) for Python-specific API details.
- Read [references/typescript-sdk.md](references/typescript-sdk.md) for TypeScript-specific API details.
- Read [references/cli.md](references/cli.md) when the user needs CLI export/list/get workflows rather than SDK code.
-
## When to use this skill
- - Add or review LangSmith tracing for a chat app, RAG pipeline, tool-calling backend, or agent workflow
- - Choose between quick wrapper-based tracing, targeted decorators, or more explicit manual tracing
- - Build offline evals on a dataset before a prompt/model/app change ships
- - Add online feedback, annotation queues, or review rubrics for production traces
- - Compare prompt or model variants with experiment runs and judge/evaluator packets
- - Decide whether Prompt Hub should own the prompt lifecycle for this workflow
- - Correlate multi-service traces by propagating parent context across jobs, workers, or microservices
- - Audit an existing LangSmith setup that “collects traces” but still does not give trustworthy debugging or launch confidence
+ - The user already chose **LangSmith** and needs the smallest useful packet for tracing, evals, review, prompt ownership, or trace propagation.
+ - A request mentions **LangChain tracing**, **`@traceable` / `traceable`**, **`wrap_openai` / `wrapOpenAI`**, **annotation queues**, **run trees**, **trace IDs**, **datasets**, or **experiment comparison**.
+ - The team needs to decide whether a change needs **debugging traces**, an **offline eval gate**, an **online review queue**, **Prompt Hub / prompt-registry** work, or **multi-service lineage**.
+ - The user has some LangSmith setup already, but confidence is still low and they need an **audit / gap review**.
- ## When not to use this skill
- - **Generic metrics / logs / dashboards / SLO / alert design** → use `monitoring-observability`
- - **Root-cause triage on existing logs or stack traces** → use `log-analysis`
- - **Code-level reproduction and bug isolation** → use `debugging`
- - **Performance bottleneck diagnosis after measurements exist** → use `performance-optimization`
- - **Deployment / rollout orchestration** → use `deployment-automation`
- - **Runtime guardrails / approval policies / safety middleware** → use the appropriate runtime-policy or security skill instead of overclaiming LangSmith as the control plane
+ Do **not** use this skill as the main workflow when:
+ - The real job is **generic dashboards / alerts / SLOs / telemetry ownership** → `monitoring-observability`
+ - The real job is **existing-log root-cause triage** → `log-analysis`
+ - The real job is **code-level reproduction / bug isolation** → `debugging`
+ - The real job is **deployment / rollout choreography** → `deployment-automation`
+ - The real job is **generic exported-metrics interpretation or KPI synthesis** → `data-analysis`
+ - The real job is **runtime guardrails / approvals / policy middleware** → the relevant security or policy skill
- ## Mode selection
- Choose one primary mode before suggesting code or CLI commands.
+ ## Core idea
+ `langsmith` should behave like a **workflow packet router**, not a giant SDK encyclopedia.
- | Mode | Use when | Main output |
- |------|----------|-------------|
- | Debugging traces | The team cannot explain what the app/agent actually did | trace-debug packet |
- | Offline evals | A change needs regression confidence before shipping | eval packet |
- | Online feedback & review | Production traces need human or automated quality scoring | review packet |
- | Prompt registry & experiment comparison | Prompt/version choice is the bottleneck | prompt packet |
- | Multi-service / agent propagation | Runs span services, jobs, or agent/tool boundaries | propagation packet |
- | Audit / gap review | LangSmith exists already but confidence is still low | audit packet |
+ 1. Label the request before naming APIs.
+ 2. Choose **one primary packet**.
+ 3. Pick the smallest instrumentation / eval / review surface that answers the question.
+ 4. Return a compact operator brief.
+ 5. Route adjacent observability, debugging, deployment, and policy work outward fast.
- If multiple modes appear, pick the bottleneck first and list the others as follow-ups.
+ Read these support docs before choosing the packet:
+ - [references/intake-packets-and-route-outs.md](references/intake-packets-and-route-outs.md)
+ - [references/modes-and-routing.md](references/modes-and-routing.md)
+ - [references/python-sdk.md](references/python-sdk.md)
+ - [references/typescript-sdk.md](references/typescript-sdk.md)
+ - [references/cli.md](references/cli.md)
## Instructions
- ### Step 1: Label the workflow before choosing APIs
- Capture the minimum facts first.
-
- Record:
- - app shape: chat app | RAG | tool-calling backend | agent | batch job | multi-service system | mixed
- - language/runtime: Python | TypeScript | mixed | unknown
- - request type: new setup | debug issue | eval rollout | prompt comparison | feedback queue | audit
- - current evidence: traces | datasets | evaluators | feedback configs | prompt versions | none
- - confidence problem: missing traces | noisy traces | no regression gate | no review rubric | prompt drift | missing cross-service correlation
- - environment: local | staging | preview | prod | mixed
-
- Do **not** start with “should I use `@traceable`?” Start by labeling the workflow.
-
- ### Step 2: Choose the primary LangSmith mode
-
- #### Mode A — Debugging traces
- Use when the real question is “what happened?”
+ ### Step 1: Normalize the request
+ Convert the prompt into this intake packet first:
- Focus on:
- - trace tree coverage across app, LLM, retrieval, tools, and downstream services
- - project/tag/metadata standards so traces are queryable later
- - whether provider wrappers are enough or explicit decorators/contexts are needed
- - explicit flush behavior for short-lived scripts/jobs
+ ```yaml
+ langsmith_packet:
+ primary_packet: trace-debug | eval | review | prompt-registry | propagation | audit
+ app_shape: chat | rag | tool-calling | agent | batch-job | multi-service | mixed | unknown
+ runtime: python | typescript | mixed | cli-only | unknown
+ request_type: setup | debug | pre-ship gate | production review | prompt decision | cross-service tracing | audit
+ current_evidence: traces | datasets | evaluators | feedback queues | prompt versions | exported runs | none
+ confidence_problem: missing traces | noisy traces | no regression gate | weak review rubric | prompt drift | broken lineage | mixed | unknown
+ route_after: stay-here | monitoring-observability | log-analysis | debugging | deployment-automation | data-analysis
+ ```
- Return:
- - tracing surface to instrument first
- - minimum metadata/tags to attach
- - wrapper vs decorator vs manual trace choice
- - route-out if the real problem is generic infra observability or log triage
+ Choose **one** primary packet for the run. If two seem plausible, pick the one that reduces uncertainty fastest.
- #### Mode B — Offline evals
- Use when the team needs a regression gate before shipping a change.
+ ### Step 2: Choose the packet
- Focus on:
- - dataset scope and example shape
- - deterministic evaluator vs LLM-as-judge vs pairwise comparison
- - experiment naming and baseline/candidate comparison
- - concurrency / cost awareness
+ | Packet | Use when | Main output | Typical signals |
+ |---|---|---|---|
+ | `trace-debug` | The real question is “what happened?” | trace-debug packet | missing visibility, wrappers/decorators, trace coverage, metadata discipline |
+ | `eval` | A change needs pre-ship or regression confidence | eval packet | dataset creation, evaluators, experiments, baseline/candidate comparison |
+ | `review` | Traces exist but trust is weak in production | review packet | annotation queues, feedback criteria, human scoring, inline review |
+ | `prompt-registry` | Prompt/version ownership is the bottleneck | prompt packet | Prompt Hub, prompt versions, Git sync, compare prompt variants |
+ | `propagation` | One request spans services, jobs, tools, or agents | propagation packet | parent/child trace IDs, cross-service headers, multi-hop lineage |
+ | `audit` | LangSmith is present but still not useful | audit packet | noisy traces, missing IDs, absent evals, unclear prompt ownership |
- Return:
- - dataset packet
- - evaluator packet
- - experiment packet
- - “ship / not yet” confidence note
+ Packet rules:
+ - Prefer `trace-debug` when the team cannot explain the run tree.
+ - Prefer `eval` when the next risk is shipping without a regression gate.
+ - Prefer `review` when real-user output needs human or structured quality feedback.
+ - Prefer `prompt-registry` only when prompt/version choice is the actual bottleneck.
+ - Prefer `propagation` when one request fans out beyond a single service boundary.
+ - Prefer `audit` when the system already “has LangSmith” but confidence is still low.
- #### Mode C — Online feedback & review
- Use when traces exist but quality review is weak.
+ ### Step 3: Pick the smallest useful surface
- Focus on:
- - feedback schema and score semantics
- - annotation queue design and required rubric items
- - who will review and when
- - how run IDs / trace IDs get preserved for later feedback attachment
+ | Surface | Use when | Strengths | Risks |
+ |---|---|---|---|
+ | Provider wrappers (`wrap_openai`, `wrap_anthropic`, `wrapOpenAI`) | Need quick model-call coverage | Fastest path to visible traces | Misses higher-level workflow context if used alone |
+ | Decorators / `traceable()` | Need named spans around app functions | Clear span boundaries, reusable | Still needs deliberate placement |
+ | Manual trace blocks / tracing context | Need scoped or temporary visibility | Good for audits and suspicious blocks | Easy to under-instrument |
+ | Annotation queues + feedback criteria | Need structured human review | Preserves rubric and reviewer flow | Review ownership still has to be designed |
+ | Dataset + experiment workflow | Need benchmark or regression checks | Gives pre-ship comparison | Can drift into generic eval theory if not bounded |
+ | CLI export/list/get flows | Need audit/export operations more than instrumentation | Good for scripts and inspection | Not a substitute for instrumentation design |
+ | Cross-service propagation headers / metadata | Need end-to-end lineage | Preserves parent/child context | Easy to forget at service boundaries |
- Return:
- - feedback keys + scale definitions
- - queue/rubric packet
- - reviewer workflow
- - route-out if the user really needs product analytics, support ops, or moderation tooling instead
+ Rules:
+ - Prefer the **smallest surface** that answers the present question.
+ - Always name project/workspace/environment grouping.
+ - Preserve `run_id`, `trace_id`, and metadata if humans or downstream tools will review runs later.
+ - Include shutdown flush / wait behavior for CLIs, workers, and short-lived jobs.
- #### Mode D — Prompt registry & experiment comparison
- Use when prompt/version choice is the bottleneck.
+ ### Step 4: Choose the eval or review layer only when needed
- Focus on:
- - whether Prompt Hub should own the prompt or the team should stay in Git/app config
- - version pinning and experiment comparison
- - pull/push workflow boundaries
- - how prompts connect to datasets/evals
+ | Layer | Use when | Notes |
+ |---|---|---|
+ | Deterministic / code evaluator | Output can be checked mechanically | Best first choice for structure/factual rules |
+ | LLM-as-judge (`openevals`) | Quality is subjective or reference-light | Useful, but not ground truth |
+ | Human review | Stakes are high or nuance dominates | Needs rubric, reviewer, and cadence |
+ | Pairwise comparison | Two prompts/models/app versions are being compared | Good for baseline vs candidate decisions |
+ | Online evaluator | Production traffic needs ongoing scoring | Pair with explicit alert/review semantics |
- Return:
- - prompt registry decision
- - baseline vs candidate comparison packet
- - explicit note if Prompt Hub is optional rather than required
+ Do **not** pretend one layer replaces the others.
- #### Mode E — Multi-service / agent propagation
- Use when one user request becomes multiple runs across services, jobs, or agents.
+ ### Step 5: Return one compact LangSmith brief
+ Default response shape:
- Focus on:
- - parent/child context propagation
- - shared trace identifiers and metadata
- - service boundary handoffs
- - what still belongs in generic observability/cost dashboards outside LangSmith
+ ```markdown
+ ## LangSmith brief
+ - Packet: trace-debug | eval | review | prompt-registry | propagation | audit
+ - App shape: ...
+ - Runtime: ...
+ - Current evidence: ...
- Return:
- - propagation packet
- - required headers/IDs
- - missing correlation risks
- - route-out if the main job is distributed-systems observability rather than LangSmith instrumentation
+ ## Chosen surface
+ - wrappers / decorators / manual trace / dataset+experiment / review queue / prompt registry / propagation headers
- #### Mode F — Audit / gap review
- Use when LangSmith is already installed but trust is low.
+ ## Required config
+ - `LANGSMITH_API_KEY`
+ - project/workspace/environment
+ - endpoint/workspace overrides if self-hosted
- Focus on:
- - missing instrumentation coverage
- - empty or noisy metadata
- - absent evals / weak review rubrics
- - prompt/version ambiguity
- - missing trace flush or run-ID preservation
+ ## Required IDs / metadata
+ - `run_id`, `trace_id`, tags, metadata fields, user/session identifiers as needed
- Return:
- - gap list by severity
- - first ratchet to add next
- - route-outs to adjacent skills
+ ## Verification step
+ - one trace query, one experiment comparison, one queue review check, or one prompt-version sanity check
- ### Step 3: Choose the instrumentation surface
+ ## Route-out
+ - next neighboring skill if the bottleneck moves
+ ```
- | Choice | Use when | Strengths | Risks |
- |---|---|---|---|
- | Provider wrappers (`wrap_openai`, `wrap_anthropic`, `wrapOpenAI`) | Need fast coverage around model calls | Quickest adoption, low touch | Misses non-model workflow context unless paired with higher-level traces |
- | Decorators / `traceable()` | Need named workflow spans around application functions | Clear trace boundaries, reusable | Still requires explicit placement |
- | `tracing_context(...)` / manual trace blocks | Need scoped or temporary tracing | Good for audits, tests, suspicious blocks | Easy to under-instrument if used as the only approach |
- | Cross-service header propagation | Request spans multiple services/jobs | Preserves trace lineage | Manual glue; easy to forget |
- | CLI export/list/get flows | Need read/export/audit operations more than instrumentation | Useful for scripts and audits | Not a substitute for instrumentation design |
+ Keep it compact. The point is to leave the user with one smallest next artifact, not a wall of SDK notes.
- Rules:
- - Prefer the **smallest useful coverage** first instead of tracing everything blindly.
- - Always set a clear project/workspace/environment grouping.
- - Use tags/metadata that match the question you will ask later.
- - If traces will be reviewed after the fact, preserve `run_id` / `trace_id` explicitly.
- - If the process is short-lived, include a flush/wait step before exit.
+ ### Step 6: Packet-specific heuristics
- ### Step 4: Choose the evaluation packet
+ #### For `trace-debug`
+ - Start from the missing visibility surface: app span, tool calls, retrieval, provider call, or worker boundary.
+ - Prefer wrappers plus a few high-value decorators over tracing everything blindly.
+ - Return the first trace query/filter the team should run once instrumentation lands.
- | Eval type | Use when | Notes |
- |---|---|---|
- | Heuristic / code evaluator | Output can be checked deterministically | Best first choice for factual or structural checks |
- | LLM-as-judge (`openevals`) | Quality is subjective or reference-light | Good for style/helpfulness/safety packets, but not ground truth |
- | Human review | Stakes are high or judgments are nuanced | Needs rubric and reviewer workflow |
- | Pairwise comparison | Comparing baseline vs candidate | Good for prompt/model swaps |
- | Online evaluator | Production traces need ongoing scoring | Pair with explicit review/alert semantics |
+ #### For `eval`
+ - Define the smallest dataset that reflects the risky change.
+ - Separate deterministic checks from LLM-as-judge or human review.
+ - Name baseline/candidate comparison, experiment naming, concurrency/cost limits, and the ship/no-ship decision point.
- Do **not** claim one eval type replaces the others. Mixed stacks are normal.
+ #### For `review`
+ - Define feedback keys, scale semantics, reviewer ownership, and queue cadence.
+ - Preserve `run_id` / `trace_id` so feedback attaches to the right artifact later.
+ - Route broader support-ops workflows out if the issue is bigger than output review.
- ### Step 5: Produce a concrete LangSmith packet
+ #### For `prompt-registry`
+ - Explicitly decide whether Prompt Hub / LangSmith should own the prompt or whether Git/app config should stay canonical.
+ - Connect prompt versions to evals or experiments; do not treat registry as a separate island.
+ - Be explicit when public hub browsing is reference-only rather than the team's source of truth.
- Return a compact plan with:
- - selected mode
- - chosen instrumentation surface
- - required env/config (`LANGSMITH_API_KEY`, project/workspace/env, endpoint if self-hosted)
- - required metadata / IDs / tags
- - dataset/evaluator/review design if applicable
- - route-outs to adjacent skills
- - verification step
+ #### For `propagation`
+ - Name parent/child headers, metadata handoff rules, and which services must preserve trace context.
+ - Keep distributed-systems reliability and alerting concerns routed to generic observability where appropriate.
- ### Step 6: Use route-outs aggressively
+ #### For `audit`
+ - Rank gaps by severity: missing coverage, bad metadata, absent evals, weak review flow, prompt/version ambiguity, broken flush or lineage.
+ - Recommend the **first ratchet**, not a giant reinstall plan.
- Route out when:
- - the main issue is generic dashboards / alerts / telemetry ownership → `monitoring-observability`
- - the main issue is reproducing a bug rather than collecting traces → `debugging`
- - the main issue is runtime guardrails or approval logic → the relevant policy/security skill
- - the main issue is rollout, release, or deployment procedure → `deployment-automation`
- - the main issue is generic data interpretation after exports exist → `data-analysis`
+ ### Step 7: Route out aggressively
+ Switch as soon as LangSmith is no longer the main bottleneck:
+ - **Generic telemetry / dashboards / alerts / SLOs** → `monitoring-observability`
+ - **Existing-log root-cause triage** → `log-analysis`
+ - **Reproduction and bug isolation** → `debugging`
+ - **Rollout / deploy / environment promotion** → `deployment-automation`
+ - **Exported metrics, experiments, or KPI interpretation** → `data-analysis`
+ - **Policy / approval / runtime guardrails** → the relevant security or policy skill
## Examples
- ### Example 1: Quick tracing packet
- **Prompt:** “Add LangSmith tracing to this FastAPI RAG app.”
+ ### Example 1: Trace-debug packet
+ **Prompt:**
+ > Add LangSmith to this FastAPI RAG service so we can debug why answer quality regressed after a retriever change.
- Use this skill to:
- - label it as **debugging traces**
- - choose wrappers plus higher-level decorators if retrieval/tools must be visible
- - define project/tags/metadata and a flush step for short-lived scripts
- - route out generic infra dashboards to `monitoring-observability`
+ **Good response shape:**
+ - choose `trace-debug`
+ - recommend wrappers plus targeted higher-level spans if retrieval/tools must be visible
+ - define project/tags/metadata and a flush step for short-lived runs
+ - route generic service-health observability outward
- ### Example 2: Eval rollout packet
- **Prompt:** “Before we ship this prompt change, build a LangSmith eval gate.”
+ ### Example 2: Eval packet
+ **Prompt:**
+ > Before we ship this prompt rewrite, build a LangSmith eval workflow with a baseline, a candidate, and one judge for subjective quality.
- Use this skill to:
- - label it as **offline evals**
- - define dataset shape, baseline/candidate comparison, and evaluator mix
+ **Good response shape:**
+ - choose `eval`
+ - define dataset, evaluator mix, experiment naming, and verification step
- separate deterministic checks from LLM-as-judge and human review
- - return a compact experiment packet instead of raw SDK trivia
+ - keep rollout orchestration out of scope
- ### Example 3: Human review queue packet
- **Prompt:** “Our support copilot has traces, but nobody trusts the responses. Set up review.”
+ ### Example 3: Review packet
+ **Prompt:**
+ > We already have LangSmith traces, but support leads still do not trust the AI assistant. Set up a review workflow with human scoring.
- Use this skill to:
- - label it as **online feedback & review**
- - define feedback keys, rubric items, queue flow, and run-ID preservation
- - clarify what belongs in LangSmith versus product analytics or support ops tooling
+ **Good response shape:**
+ - choose `review`
+ - define feedback keys, rubric, queue ownership, and run-ID preservation
+ - keep the answer grounded in review workflow rather than generic prompt management
- ### Example 4: Prompt Hub decision
- **Prompt:** “Should we store these prompts in LangSmith or keep them in Git?”
+ ### Example 4: Prompt-registry packet
+ **Prompt:**
+ > Should these prompts live in LangSmith Prompt Hub or stay in Git?
- Use this skill to:
- - label it as **prompt registry & experiment comparison**
- - compare Prompt Hub against Git/app-config ownership
- - define how prompt versions connect to experiments and rollout confidence
+ **Good response shape:**
+ - choose `prompt-registry`
+ - compare platform-owned vs Git-owned prompt source of truth
+ - connect the choice to experiments and release confidence
+ - stay honest when Prompt Hub is optional
- ### Example 5: Multi-service trace propagation
- **Prompt:** “One chat request fans out to workers and tools — how do we keep one trace?”
+ ### Example 5: Propagation packet
+ **Prompt:**
+ > One chat request fans out to workers and tools. How do we keep one trace through LangSmith?
- Use this skill to:
- - label it as **multi-service / agent propagation**
- - define parent/child propagation and shared metadata requirements
- - route out generic distributed-systems observability concerns when LangSmith alone is not enough
+ **Good response shape:**
+ - choose `propagation`
+ - define parent/child IDs, metadata propagation, and service boundaries
+ - route generic distributed-systems observability outward when needed
## Best practices
- 1. Start with the **workflow mode**, not the SDK call.
+ 1. Start with the **packet**, not the SDK call.
2. Treat LangSmith as a **trace/eval/review substrate**, not the entire production control plane.
- 3. Preserve `run_id`, `trace_id`, tags, and metadata deliberately; they are the handle for later debugging and feedback.
- 4. Prefer the smallest instrumentation surface that answers the current question.
- 5. Separate deterministic evaluators, LLM judges, and human review instead of pretending one replaces the others.
- 6. Be explicit when Prompt Hub is optional.
- 7. Include shutdown flush behavior for CLIs, scripts, workers, and cron-style jobs.
- 8. Route generic observability, deployment, and runtime-policy requests elsewhere instead of overextending this skill.
+ 3. Preserve `run_id`, `trace_id`, tags, and metadata deliberately.
+ 4. Prefer the smallest useful surface that answers the present question.
+ 5. Keep deterministic checks, LLM judges, and human review distinct.
+ 6. Be explicit when Prompt Hub is optional or reference-only.
+ 7. Include flush/wait behavior for scripts, workers, and cron-style jobs.
+ 8. End with one verification step and one route-out.
## References
- - `references/modes-and-routing.md` — workflow modes, operator packets, and route-outs
- - `references/python-sdk.md` — Python client, decorators, evaluators, feedback, annotation queues
- - `references/typescript-sdk.md` — TypeScript wrappers, evaluators, feedback, LangChain callbacks
- - `references/cli.md` — CLI list/get/export flows for traces, runs, datasets, experiments
+ - `references/intake-packets-and-route-outs.md`
+ - `references/modes-and-routing.md`
+ - `references/python-sdk.md`
+ - `references/typescript-sdk.md`
+ - `references/cli.md`
- Official docs: https://docs.langchain.com/langsmith
- SDK repo: https://github.com/langchain-ai/langsmith-sdk
- OpenEvals: https://github.com/langchain-ai/openevals