evo2-nim ยท diff
git:20260825.0e67a61 to git:20260929.ac632e9
63 added, 65 removed. Audit A to A.
---
name: evo2-nim
description: >
Generate and analyze DNA sequences using NVIDIA's Evo 2 BioNeMo NIM microservice. Use for Evo2/Evo 2, DNA generation, genomic sequence generation, hosted generation, local Docker deployment, local forward passes, layer outputs, logits, sampled probabilities, and BioNeMo NIM workflows.
license: Apache-2.0 AND CC-BY-4.0
compatibility: "requests>=2.28; numpy>=1.24"
allowed-tools: Bash, Read, Write, AskUserQuestion
---
# Evo 2 NIM
- Use Evo 2 for DNA generation and, locally, layer-output extraction. Use this
- `SKILL.md` for basic hosted/local use; load supplemental files only when needed:
+ Use Evo 2 for DNA generation and, locally, layer-output extraction. Load
+ supplemental files only when needed:
- `references/api.md`: exact schemas, layer names, Docker flags, hardware notes.
- `references/science.md`: genomic use cases, limits, and interpretation.
- `references/parameters.md`: generation/forward parameter effects.
- `references/validation.md`: DNA, probability, timing, and tensor checks.
- `references/examples.md`: compact hosted/local request patterns.
+ ## Instructions
+
+ For generation, use `scripts/generate.py` to execute the request, validate the
+ response, and save its artifacts. Resolve the script path relative to this
+ skill's directory and choose an output directory in the user's workspace.
+ Use the user's sequence and requested parameters; the example below is only
+ a smoke test.
+
+ 1. Select the requested mode. For hosted generation, go directly to the
+ generation example; Docker setup and local forward passes are separate tasks.
+ 2. When the user asks to run generation, execute the client and inspect its
+ exit status and result. Writing a script alone does not complete that request.
+ 3. Report the generated DNA (or its file for long sequences), actual
+ `elapsed_ms`, sampled-probability summary, seed, and artifact paths from the
+ successful run. Read the saved response or metrics if any result is unclear.
+
+ If the request or validation fails, report the actual failure and any diagnostic
+ files. Do not replace an unavailable API response with example values. For a
+ code-only request, provide the command without making an inference call.
+
## Choose Mode
Honor `NIM_API_MODE` when it is set. Accepted values are `hosted` and `local`.
If it is unset, treat an explicit `EVO2_NIM_URL` as local; otherwise ask when
the requested mode is unclear:
> Hosted NVIDIA API or local Docker Evo 2 NIM?
- Hosted generation: `https://health.api.nvidia.com/v1/biology/arc/evo2-40b/generate`
- Local base URL: `$EVO2_NIM_URL`, falling back to `http://localhost:8000`
- Local generation: `$EVO2_NIM_URL/biology/arc/evo2/generate`
- Local forward/layer outputs: `$EVO2_NIM_URL/biology/arc/evo2/forward`
Always resolve local health and inference routes from `EVO2_NIM_URL` when it
is present. `localhost` works only when the caller and NIM share a network
namespace; a caller in a separate container usually needs a service URL such
as `http://evo2-nim:8000`. Do not silently switch modes when the selected
endpoint is unavailable. Report the failed endpoint and fix its configuration.
The hosted docs expose generation. `/forward` is documented for local Docker;
do not invent a hosted `/forward` endpoint. Hosted requests use `Authorization: Bearer $NGC_API_KEY`. Supported local Docker
startup uses `NGC_API_KEY` (or `NVIDIA_API_KEY` via the preflight) for
registry login, entitlement checks, and first-run model downloads; pass it
into the container with `-e NGC_API_KEY`. Local inference requests use no
auth header after readiness. Warm-cache key-free startup varies by
image/version and should not be assumed.
+ ## Examples
+
+ Normalize prompts before sending. Use A/C/G/T unless ambiguous bases are a
+ deliberate modeling choice and clearly reported.
+
+ For a hosted generation request, run the bundled client with the user's inputs
+ (the script path below is relative to the skill directory):
+
+ ```bash
+ python scripts/generate.py \
+ --mode hosted \
+ --sequence ACTGACTGACTGACTG \
+ --num-tokens 64 --seed 1 \
+ --temperature 0.7 --top-k 3 --top-p 0.0 \
+ --output-dir /path/to/workspace/evo2-output
+ ```
+
+ For an already-ready local NIM, use `--mode local`; the client resolves
+ `EVO2_NIM_URL` and sends no Authorization header. It never switches endpoints
+ after a failed request. Set `--timeout` for a longer read if the user requests
+ a larger generation; failed requests are not automatically resubmitted.
+
+ The client saves `request.json`, the actual `response.json`, `generated.fasta`,
+ and `metrics.json` in the chosen output directory. It validates the requested
+ number of generated bases, A/C/G/T alphabet, finite sampled probabilities in
+ `[0, 1]`, and nonnegative timing before printing a successful summary. Existing
+ outputs are not overwritten; choose a new output directory for each run.
+ The FASTA contains generated bases only, not the input prompt prepended again.
+
+ `sampled_probs` is requested by the client and summarized with count/min/max/mean;
+ the full values stay in the saved response. A missing or malformed probability
+ array is a validation failure, not permission to invent confidence values.
+ Only request `enable_logits` in a custom request when needed; logits can make
+ responses large. See `references/api.md` for custom payloads.
+ `random_seed` supports development reproducibility, not biological certainty.
+
## Local Docker Requirements
Evo 2 local deployment requires FP8-capable GPUs. Do not present A100 as
compatible; A100 can pull the image but fails warmup because FP8 requires
compute capability 8.9 or higher.
- Default 40B: 2x H100 80 GB or 1x H200 141 GB. Use `NIM_TEST_GPUS=0,1` for
2x H100, or `NIM_TEST_GPUS=0` for one H200.
- 7B fallback: set `NIM_VARIANT=7b`; supported GPUs include H100, H200,
RTX 6000 Ada, and L40S.
- Approximate disk: 110 GB for 40B, 50 GB for 7B.
Use shell env first; source repo-root `.env` only if present. Do not invent a
cache default or drop the `NVIDIA_API_KEY` fallback.
```bash
set -a
[ -f .env ] && . ./.env
set +a
if [ -z "${NGC_API_KEY:-}" ] && [ -n "${NVIDIA_API_KEY:-}" ]; then
export NGC_API_KEY="$NVIDIA_API_KEY"
fi
: "${NGC_API_KEY:?Set NGC_API_KEY or NVIDIA_API_KEY}"
: "${LOCAL_NIM_CACHE:?Set LOCAL_NIM_CACHE}"
echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
# 40B default: 0,1 for 2x H100; set 0 for a single H200.
export NIM_TEST_GPUS="${NIM_TEST_GPUS:-0,1}"
mkdir -p "${LOCAL_NIM_CACHE}"
chmod 700 "${LOCAL_NIM_CACHE}" # owner-only; if the NIM runs as a different UID, add -u "$(id -u)" to docker run
# For 7B: export NIM_VARIANT=7b; export NIM_TEST_GPUS="${NIM_TEST_GPUS:-0}"
docker run --rm -it --name evo2-nim \
--runtime=nvidia \
--gpus "\"device=${NIM_TEST_GPUS}\"" \
-e NGC_API_KEY \
-e NIM_VARIANT \
-v "${LOCAL_NIM_CACHE}:/opt/nim/.cache" \
-p 8000:8000 \
nvcr.io/nim/arc/evo2:2
```
Readiness:
```bash
evo2_nim_url="${EVO2_NIM_URL:-http://localhost:8000}"
until curl -sf "${evo2_nim_url%/}/v1/health/ready"; do sleep 10; done
```
If RTX PRO 6000 Blackwell Workstation fails with no Transformer Engine
attention backend, treat it as outside the current validated matrix and rerun
on a documented GPU/runtime.
- ## DNA Generation
-
- Normalize prompts before sending. Use A/C/G/T unless ambiguous bases are a
- deliberate modeling choice and clearly reported.
-
- ```python
- import json
- import os
- from pathlib import Path
- import requests
-
- def clean_dna(value: str) -> str:
- seq = "".join(value.upper().split())
- invalid = sorted(set(seq) - set("ACGT"))
- if invalid:
- raise ValueError(f"Unexpected DNA characters: {''.join(invalid)}")
- return seq
-
- prompt = clean_dna("ACTGACTGACTGACTG")
- mode = os.getenv("NIM_API_MODE")
- if mode is None:
- mode = "local" if os.getenv("EVO2_NIM_URL") else "hosted"
- if mode not in {"hosted", "local"}:
- raise ValueError("NIM_API_MODE must be 'hosted' or 'local'")
-
- nim_url = os.getenv("EVO2_NIM_URL", "http://localhost:8000").rstrip("/")
- url = (
- "https://health.api.nvidia.com/v1/biology/arc/evo2-40b/generate"
- if mode == "hosted" else f"{nim_url}/biology/arc/evo2/generate"
- )
- headers = {"Content-Type": "application/json"}
- if mode == "hosted":
- api_key = os.getenv("NGC_API_KEY")
- if not api_key:
- raise RuntimeError("Set NGC_API_KEY for hosted Evo 2")
- headers["Authorization"] = f"Bearer {api_key}"
-
- payload = {
- "sequence": prompt,
- "num_tokens": 64,
- "temperature": 0.7,
- "top_k": 3,
- "top_p": 0.0,
- "random_seed": 1,
- "enable_sampled_probs": True,
- "enable_elapsed_ms_per_token": True,
- }
- response = requests.post(url, headers=headers, json=payload, timeout=180)
- response.raise_for_status()
- result = response.json()
- seq = result["sequence"]
- if sorted(set(seq.upper()) - set("ACGT")):
- raise ValueError("Generated sequence contains unexpected non-ACGT bases")
-
- Path("evo2_generation.json").write_text(json.dumps(result, indent=2) + "\n")
- Path("evo2_generated.fa").write_text(f">evo2_generated\n{seq}\n")
- print(f"Generated {len(seq)} bases in {result.get('elapsed_ms')} ms")
- ```
-
- Only request `enable_logits` when needed; logits can make responses large.
- `random_seed` supports development reproducibility, not biological certainty.
-
## Local Forward Pass
Forward returns base64-encoded NPZ tensors.
```python
import base64
import io
import os
import numpy as np
import requests
mode = os.getenv("NIM_API_MODE", "local")
if mode != "local":
raise RuntimeError("Evo 2 /forward is available only in local mode")
nim_url = os.getenv("EVO2_NIM_URL", "http://localhost:8000").rstrip("/")
+ sequence = "ACTGACTGACTG" # Replace with the user's DNA sequence.
+ sequence = "".join(sequence.upper().split())
+ if not sequence or set(sequence) - set("ACGT"):
+ raise ValueError("Expected nonempty A/C/G/T DNA")
payload = {
- "sequence": clean_dna("ACTGACTGACTG"),
+ "sequence": sequence,
"output_layers": ["output_layer", "decoder.layers.3.self_attention"],
}
response = requests.post(
f"{nim_url}/biology/arc/evo2/forward",
headers={"Content-Type": "application/json"},
json=payload,
timeout=300,
)
response.raise_for_status()
npz_bytes = base64.b64decode(response.json()["data"])
with open("evo2_forward_outputs.npz", "wb") as handle:
handle.write(npz_bytes)
arrays = np.load(io.BytesIO(npz_bytes), allow_pickle=False)
for name in arrays.files:
arr = arrays[name]
print(name, arr.shape, arr.dtype, bool(np.isfinite(arr).all()), float(arr.mean()))
```
## Validate And Report
Save request/response JSON, generated FASTA, and a metrics JSON with sequence
length, GC fraction, ambiguous-base fraction, homopolymer length, sampled-prob
checks, and elapsed timing. Treat invalid schema or alphabet as hard failures;
treat extreme GC, low complexity, duplicates, and missing motifs as warnings.
For deeper checks, read `references/validation.md`.
Key fields: `sequence`, `num_tokens`, `temperature`, `top_k` (0-6), `top_p`
(0-1), `random_seed`, `enable_sampled_probs`, `enable_elapsed_ms_per_token`,
and optional `enable_logits`.
## Troubleshooting
- `401/403`: hosted key missing/expired or not sent as Bearer token.
- `422`: wrong field names such as `max_tokens` instead of `num_tokens`.
- Local endpoint confusion: print `NIM_API_MODE` and `EVO2_NIM_URL`; do not
replace a configured service URL with `localhost`.
- Local auth confusion: do not send `Authorization` to local inference.
- Local startup: first run downloads model assets; poll
`$EVO2_NIM_URL/v1/health/ready` before inference.
- FP8 failure: use hosted, 7B on a supported FP8 GPU, or documented 40B GPUs.