git:20260906.525ee24 to git:20260906.2c32550

---

name: detection-training

description: Use to train and evaluate Faster R-CNN detectors from LanceDB materialized views (BDD100K failure-mode slices) — direct versus deployed-service execution, the mandatory label map for string categories, and checkpoint discovery at eval time.

---

# Detection training (LanceDB view → Faster R-CNN)

This tool trains an object detector on a **slice** of a dataset rather than the

whole thing. The input is a LanceDB materialized view, which is what makes the

failure-mode workflow possible: curate a view of nighttime frames, or distant

objects, or riders, then train and evaluate only on that slice.

Upstream is `skills/tools/lancedb/SKILL.md` — create the view with

`npa workbench lancedb create-mv` / `refresh-mv` before anything here. The end-to-end

pipeline is `workflows/testing/bdd100k-pipeline.yaml`, with a

walkthrough in `docs/workbench/cookbooks/bdd100k-pipeline.md`.

## Two execution modes

Direct (the default) runs the work from your CLI invocation. Service mode calls a

deployed Kubernetes endpoint:

```bash

npa workbench detection-training train --view <mv-name> --output-uri s3://<bucket>/runs/<id>/

npa workbench detection-training train --view <mv-name> --service --endpoint <url>

```

Deploy the service only when you want a persistent endpoint several runs share:

```bash

npa workbench detection-training deploy \

--project <alias> --cluster-name <name> \

--input-path s3://<bucket>/lancedb/<db>/ \

--output-path s3://<bucket>/detection/ \

--gpu-type h100 --namespace default \

--dry-run # prints the manifest without applying

npa workbench detection-training deploy --project <alias> --destroy

```

the variable named by `--token-env`, default `DETECTION_TRAINING_TOKEN`);

`--insecure-no-auth` exists but should not be used. Official NPA GHCR images

pull anonymously, so `--image-pull-secret` defaults to empty. For an optional

operator-controlled private registry, pass the name of an existing standard

Kubernetes pull secret explicitly. Always `--dry-run` first and read the

manifest.

## Train

```bash

npa workbench detection-training train \

--view <materialized-view> \

--output-uri s3://<bucket>/detection/runs/<id>/ \

--label-map '{"person":0,"rider":1,"car":2}' \

--epochs 10 --batch-size 8 --learning-rate 0.005 \

--wait --poll-seconds 30 --timeout-seconds 21600 \

--output json

```

`--view` is the only required flag. `--data-path` (alias `--lance-uri` /

`--input-path`) overrides which LanceDB the view is read from. `--override

KEY=VALUE` sets request fields the flags do not expose. Weights and Biases

logging is opt-in and defaults to `offline` mode.

`--wait` polls `/status` until the run completes **and fails if it does not** —

without it, the command returns as soon as the run is accepted, which is easy to

misread as success. Default output format for `train` and `eval` is `json`.

## The label map is not optional for BDD100K

**`--label-map` is required whenever the dataset stores string categories, which

BDD100K does.** Without it, category names reach `int()` and the run dies on:

```

invalid literal for int() with base 10: 'train'

```

`train` is a real BDD100K category — the vehicle. Reading that traceback as a

bug in the training code, rather than a missing label map, costs more time than

it should. Accepted formats are JSON (`{"person":0,...}`) or comma-separated

## Evaluate

```bash

npa workbench detection-training eval \

--checkpoint-uri s3://<bucket>/detection/runs/<id>/ \

--eval-view <validation-mv> \

--output-uri s3://<bucket>/detection/eval/<id>/ \

--label-map '{"person":0,"rider":1,"car":2}' \

--discover-checkpoint \

--write-canonical-metrics

```

`--eval-view` and `--output-uri` are required. `--discover-checkpoint` changes

what `--checkpoint-uri` means: it becomes the training **output prefix** to

checkpoint filename from an epoch count you assumed.

`--write-canonical-metrics` publishes the response under the output prefix so

## In workflows

toolRefs are per-slice pairs rather than one generic tool:

`workbench.detection_training.train_nighttime` / `.eval_nighttime`,

`.train_distant` / `.eval_distant`, `.train_rider` / `.eval_rider`. They appear

together in `bdd100k-pipeline.yaml`, which builds the views, trains each slice,

and evaluates each against its validation view.

## Gotchas

- **Without `--wait`, "started" is not "succeeded".** Check `status` before

reporting a result.

- **A view is a slice, so the metric is about that slice.** Nighttime mAP is not

overall mAP; label it accordingly when reporting.

- **`--service` needs both a reachable `--endpoint` and the token variable set.**

A missing token presents as an auth failure from the endpoint, not as a CLI

validation error.

- **Refresh the view after the underlying table changes.** `refresh-mv` is not

automatic, and training a stale view silently trains on old rows.

## Verify

```bash

```