instrument-llm-observability SKILL.md skillA

instrument-llm-observability from contextjet-ai/awesome-llm-observability is an agent skill: a SKILL.md folder an AI agent loads when a task calls for it.

Install it, pinned
mdr add contextjet-ai/awesome-llm-observability/instrument-llm-observability@git:20260705.23078be

Pins git:20260705.23078be in mdr.lock. Pin the fingerprint instead.

Audit grade
A, 17 of 17 checks passed
Kind
SKILL.md skill
Repository
contextjet-ai/awesome-llm-observability
Path
skills/instrument-llm-observability/SKILL.md
Versions recorded
2
Latest version
git:20260705.23078be, committed Jul 5, 2026
First version recorded
Jul 5, 2026
Size
3,582 bytes
Repository stars
45
License
a license file GitHub does not identify

Use this when adding tracing/observability to an LLM or AI-agent application - capturing prompts, tool calls, token usage, latency, and cost per step. Trigger whenever someone wants to "add tracing", "instrument", "monitor", "see what my agent is doing", or debug an LLM app in production. Prefer vendor-neutral OpenTelemetry unless a specific platform is already in use.

Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.

What the file says

# Instrument an LLM app for observability

Add production-grade tracing to an LLM/agent app so every run is inspectable: prompts, tool calls, retrievals, token usage, latency, and cost per span.

## Decide the approach first (2 questions)

1. **Is a backend already chosen?** (Langfuse, Arize Phoenix, Comet Opik, Datadog, Grafana…). If yes, use its SDK/auto-instrumentation. If no, default to **OpenTelemetry GenAI semantic conventions** so traces export to *any* OTel backend and you avoid lock-in.
2. **Lowest-friction path available?** If the app calls LLMs through a **gateway/proxy** (LiteLLM, Helicone, Portkey), you may get tracing + cost with *zero code* by changing the base URL. Check that before adding an SDK.

## Vendor-neutral path (recommended default): OpenTelemetry

Emit `gen_ai.*` spans via an OTel instrumentation library, then point at any collector.

- Use **OpenLLMetry** (`traceloop-sdk`) or **OpenInference** (Arize) - both emit OTel-compatible LLM spans.
- Minimal shape:
  1. Install the instrumentation lib + an OTLP exporter.
  2. Initialize once at startup (set service name + OTLP endpoint via `OTEL_EXPORTER_OTLP_ENDPOINT`).
…

Read the whole file at its exact version.

How to install

The exact contents, by fingerprint
mdr add contextjet-ai/awesome-llm-observability/instrument-llm-observability@sha256:44496e083bae81ca

The line at the top of this page pins the label git:20260705.23078be, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.

Badge

mdr badge

[![mdr](https://markdownregistry.com/badge/art_zfjztwujoud6dtug.svg)](https://markdownregistry.com/a/art_zfjztwujoud6dtug)

1 badge image fetches in 30 days. What the badge shows.

Versions

The registry has recorded 2 versions of this file, all committed on Jul 5, 2026.

versioncommittedcommitsizeaudit
git:20260705.23078be latest2026-07-05 23078be 3,582 BA view · diff
git:20260705.23977fd2026-07-05 23977fd 3,592 BA view

Audit of the latest version

A  17 of 17 checks passed. Fixed automatic checks with no AI involved, so the same checks give the same answer every run.
  • pass: Frontmatter block present
  • pass: Frontmatter declares a name
  • pass: Frontmatter declares a description
  • pass: Size between 200 bytes and 200 KB (3582 bytes)
  • pass: No zero-width or bidi control characters
  • pass: No instruction hidden inside an HTML comment
  • pass: No link to an exfiltration or paste host
  • pass: No credential-shaped string
  • pass: No instruction to send local credentials anywhere
  • pass: No text hidden with inline styles
  • pass: No prompt-injection phrasing
  • pass: No curl or wget piped into a shell
  • pass: No recursive delete of root, home or parent
  • pass: No instruction to read or print local credentials
  • pass: No base64 blob over 200 characters
  • pass: No link to a raw IP address
  • pass: No script tag

Source

GitHub

contextjet-ai/awesome-llm-observability · 45 stars · license a license file GitHub does not identify · pushed 2026-11-02 · branch main

API

GET https://markdownregistry.com/api/v1/artifacts/art_zfjztwujoud6dtug
GET https://markdownregistry.com/api/v1/resolve?ref=contextjet-ai/awesome-llm-observability/instrument-llm-observability
GET https://markdownregistry.com/api/v1/blob/44496e083bae81ca27e4e5b6b2e3f383d22088c1f50bb2cbbbc871f42ba7b5a5

Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.

Guides: the SKILL.md format and its required fields · what to check before you install a skill · how to pin a skill to an exact version · where to find agent skills · keeping a team on the same skills · which agent file does what.

More from contextjet-ai/awesome-llm-observability

add-llm-evals SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/add-llm-evals/SKILL.md · Use this when adding evaluation to an LLM/agent app - measuring output quality (correctness, faithfulness, relevance…
git:20260705.23078be · audit A · 45 stars
add-llm-fallbacks SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/add-llm-fallbacks/SKILL.md · Use this to make an LLM app resilient to provider failures, rate limits, timeouts, and outages. Trigger on "handle LLM…
git:20260705.6b497b1 · audit A · 45 stars
add-llm-guardrails SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/add-llm-guardrails/SKILL.md · Use this to add safety and security guardrails to an LLM/agent app - blocking prompt injection, PII leakage…
git:20261015.76d5de4 · audit A · 45 stars
annotate-traces-for-review SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/annotate-traces-for-review/SKILL.md · Use this to set up human review and annotation of LLM traces, so people (often domain experts) can label outputs, do…
git:20260705.95f5cea · audit A · 45 stars
build-eval-dataset SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/build-eval-dataset/SKILL.md · Use this to build a good evaluation dataset for an LLM app, the part everyone underestimates. Trigger on "make an eval…
git:20260705.4a58130 · audit A · 45 stars
check-answer-consistency SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/check-answer-consistency/SKILL.md · Use this to get a cheap, reference-free signal that an LLM answer might be made up, by sampling the same prompt a few…
git:20260705.4a31848 · audit A · 45 stars
choose-observability-stack SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/choose-observability-stack/SKILL.md · Use this to recommend an LLM observability / evaluation tool or stack for a specific situation. Trigger on "which…
git:20261015.76d5de4 · audit A · 45 stars
collect-user-feedback SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/collect-user-feedback/SKILL.md · Use this to capture user feedback on LLM outputs (thumbs up/down, edits, corrections, implicit signals) and feed it…
git:20260705.6b497b1 · audit A · 45 stars
compare-llm-models SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/compare-llm-models/SKILL.md · Use this to pick or switch the LLM behind a feature, based on evidence instead of hype or the newest release. Trigger…
git:20260705.4a58130 · audit A · 45 stars
debug-agent-from-traces SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/debug-agent-from-traces/SKILL.md · Use this to diagnose WHY an LLM agent or chain produced a wrong, empty, slow, or expensive result, by reading its…
git:20260705.23078be · audit A · 45 stars
detect-hallucinations SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/detect-hallucinations/SKILL.md · Use this to detect when an LLM is making things up, so you can flag or block confident-but-wrong answers before users…
git:20260705.2cc0ac5 · audit A · 45 stars
estimate-llm-cost SKILL.md skill
contextjet-ai/awesome-llm-observability · skills/estimate-llm-cost/SKILL.md · Use this to estimate what an LLM call or feature will cost, and to compare models on price, before or after shipping…
git:20261020.4f317d3 · audit A · 45 stars

Every file in contextjet-ai/awesome-llm-observability

Browse by kind, by grade A, or by owner.