cli git:20260712.453b9edA

Immutable. This exact content is served forever at /api/v1/blob/69be8759f4ede72a.

---
name: cli
description: Use when you need to drive the sparq RDF/SPARQL engine from the command line — load a Turtle/N-Triples/N-Quads/TriG (or HDT) file and run a SPARQL query, compare RDF triple sets, build/query memory-mapped on-disk indexes for datasets larger than RAM, materialize RDFS/OWL-RL/N3 reasoning closures, stream-ingest huge gzip/bzip2/zstd dumps, import CSV as RDF (direct mapping or R2RML), or benchmark query suites. Covers the actual `sparq-cli` subcommands, positional argument order, and cargo feature flags.
---

# sparq-cli

`sparq-cli` is the command-line front-end to the `sparq` RDF triplestore + SPARQL engine. It loads RDF files (with transparent gzip/bzip2/zstd decompression), runs SPARQL, builds and queries out-of-core memory-mapped indexes, and materializes reasoning closures (RDFS / OWL-RL / N3).

**Argument style (important):** the CLI uses a hand-rolled positional parser — there is *no* clap, no `--help`, and **no GNU-style flags except `--reason`/`--proof`, `query`'s `--format`/`--count`, and `diff`'s `--exact`**. The first token is the subcommand; the rest are positional and order matters. An unknown/missing subcommand prints a short usage block and exits with code 2.

## Quickstart

Run via cargo (the binary is `sparq-cli`; build with `--release` — debug builds are far slower):

```bash
# Load a Turtle file and run one query — prints the RESULTS (a readable table by default).
cargo run --release -p sparq-cli -- \
  query data.ttl turtle 'SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 10'
# stderr: loaded N triples in 0.123s (...)
# stdout: a table of the solution bindings + a "(K row(s))" footer
```

`query` emits real results by FORM: SELECT → the solution bindings, ASK → a boolean (`true`/`false`), CONSTRUCT/DESCRIBE → the resulting triples as N-Triples. Pick the SELECT/ASK serialisation with `--format <table|tsv|csv|xml|json|ntriples>` (default `table`); add `--count` to restore the old count-only line (`<n> solutions/triples in <ms>ms`). See the `query` entry under Key APIs for the full matrix.

`format` is one of `turtle | ntriples | nquads | trig` (aliases: `n-triples`, `n-quads`, `application/trig`). `nquads`/`trig` are loaded as a dataset so `GRAPH {}` works. Compressed inputs are auto-detected by extension (`.gz`, `.bz2`, `.zst`/`.zstd`) and streamed.

## Key APIs (subcommands)

All invoked as `sparq-cli <subcommand> <args...>`:

- `query <data-file> <format> <sparql> [--format <out>] [--count] [--reason <rdfs|owl|n3>]` — load file, run one query, print its **results** to stdout, dispatched by query form:
  - **SELECT** → the solution bindings. `--format` chooses the serialisation: `table` (default, a readable fixed-width ASCII table with a `(K row(s))` footer), `tsv` / `csv` / `xml` (W3C SPARQL Results, reusing `sparq-server`'s serialisers), or `json` (SPARQL 1.1 Results JSON, the engine's direct serialiser). `ntriples` is not meaningful for bindings and falls back to `tsv`.
  - **ASK** → a boolean: `true` / `false`. `--format json` / `--format xml` emit the W3C boolean documents (`{"head":{},"boolean":…}` / `<sparql>…<boolean>…</boolean></sparql>`); other formats print the bare token.
  - **CONSTRUCT / DESCRIBE** → the resulting triples serialised as **N-Triples** (always; `--format` is a SELECT/ASK selector and is ignored for the graph forms).
  - `--count` restores the historical count-only output (`<n> solutions in <ms>ms` for SELECT/ASK, `<n> triples in <ms>ms` for the graph forms) — the backward-compatible escape hatch for scripts that scraped the count.
  - An unknown `--format` value is a usage error (exit 2); a query/runtime error exits 1.
- `diff <file-a> <file-b> [--exact]` *(opt-in `diff` feature; [GPT-5.6] sq-lsp7k.28)* — auto-detect each RDF format from `.nt`, `.ttl`, `.nq`, `.trig`, or `.jsonld` (before an optional compression extension), compare the documents as triple sets, and emit a deterministic N-Triples patch. Lines prefixed by `-` form the first lexicographically sorted block; lines prefixed by `+` form the second. Exit 0 means identical sets and empty stdout; exit 1 means different sets. Named-graph names are discarded, duplicate triples collapse, and blank-node labels compare exactly as loaded. `--exact` is currently a compatibility alias for that same behavior.
- `reason <data-file> <format> <rdfs|owl|n3> [out.nt]` — materialize the entailed closure; print closure triple count; with `out.nt`, write the full closure as N-Triples. Add `--proof` (N3 only) to print each derivation step.
- `build <file[.gz|.bz2|.zst]> <format> <dir> [chunk_millions=16]` — EXTERNAL-MEMORY build: stream the (compressed) document straight to on-disk memory-mapped indexes via disk-backed sort/merge. For datasets whose indexes exceed RAM. `chunk_millions` sets the in-memory run size. Writes RAW perms by default; set `SPARQ_BUILD_COMPRESSED=1` to emit block-compressed (`SPQCPRM1`) perms directly from the merge tail, skipping a later `recompress` (byte-identical to build-then-recompress).
- `save <data-file> <format> <dir> [compressed]` — load into RAM then persist the six permutation indexes to `<dir>`. Add the literal word `compressed` for block-compressed permutations.
- `query-mmap <dir> <sparql> [--format <out>] [--count]` — open a saved/built dir with indexes MEMORY-MAPPED (out-of-core) and run a query, printing its **results**. Output is at **parity with `query`**: SELECT → bindings (default a readable table; `--format <table|tsv|csv|xml|json|ntriples>` selects the serialisation), ASK → a boolean (`--format json|xml` → the W3C boolean documents), CONSTRUCT/DESCRIBE → the resulting triples as N-Triples; `--count` restores the legacy count-only line (`<n> solutions/triples in <ms>ms`). The only difference from `query` is the data source — an mmap-backed `Graph::open` instead of an in-RAM load (permutations stay in the OS page cache, not the process heap). An unknown `--format` is a usage error (exit 2); a query/runtime error exits 1.
- `recompress <src-dir> <dst-dir>` — re-persist a saved dir with block-compressed permutations without re-parsing (dirs must differ).
- `compact <persist-dir>` — **WAL compaction / vacuum for erasure-completeness** (`sq-x32t`). OFFLINE operator command: stop a `--persist` server, run this on its directory, restart. Opens the dir (replaying its WAL into the live overlay), then **physically rewrites** the store to only the current live triples with a **re-interned (purged) dictionary**, and **atomically swaps** the directory (rollback-safe two-rename + WAL truncate; an interrupted swap is healed on the next open). So a logically-`DELETE`d / `DROP`ped triple's data — including an orphaned **literal value** — is gone from disk, not just hidden. The live triple set is preserved exactly (round-trip). The **online** equivalent is `POST /admin/compact` on a running server (see the `http-server` skill). **Honest scope:** scrubs the engine's own on-disk segments + dictionary; it cannot reach off-box copies (filesystem snapshots, COW history, external backups) — see `compliance/privacy/retention-erasure-runbook.md` §7a/§7b.
- `dump <file[.gz|.bz2|.zst]> <in-format> <out-format>` — load an RDF document and re-serialize the whole graph (default + named graphs) to stdout in the RDF **writer matrix**. `out-format` ∈ `turtle | turtle-pretty | trig | trig-pretty | nquads | ntriples | jsonld[-expanded|-flattened|-compacted] | jsonld-pretty[-expanded|-flattened|-compacted]` (Turtle emits the default graph only; trig/nquads/jsonld emit the full dataset; bare `jsonld` == `jsonld-expanded`, bare `jsonld-pretty` == `jsonld-pretty-expanded`; the `turtle-pretty`/`trig-pretty` forms emit deterministic, idiomatic Turtle/TriG — sorted, blank-line-separated subject blocks, the engine home for the site's pretty-Turtle reshaper; the `jsonld-pretty*` forms emit indented JSON-LD — a whitespace-only re-indent of the minified document, so same ordering, same RDF). The writer matrix is `sparq-engine/serialize-rdf` (zero new deps — the JSON-LD writer is a native, hand-rolled emitter with no json-ld/serde crate), pulled into the **default build** by the default-on `jsonld` feature (see "Default cargo features" below). `dump` also **reads** a JSON-LD `<in-format>` (`jsonld` / `json-ld` / `application/ld+json`) in the default build — JSON-LD is default-on ([OPUS-4.8] sq-oy1f.4). A `--no-default-features` build drops both the `oxjsonld` parser and the writer matrix (`dump` then errors on a `jsonld` out-format, and a `jsonld` in-format → exit 2). Unknown out-format → exit 2.
- `ingest <file[.gz|.bz2|.zst]> [parse|intern|full] [max_millions]` — streaming-throughput experiment over N-Triples: `parse` (decompress+parse+count), `intern` (+dictionary), `full` (+build indexes). Reports triples/s.
- `bench <data-file> <format> <queries-dir> [iters=5] [count|materialize|json]` — load once, run every `*.rq` in the dir (sorted) `iters` times, print TSV `<name>\t<rows>\t<min_micros>`. Mode default `materialize`.
- `memstat <data-file> <format> [compressed]` *([FABLE-5] sq-7d3dj.32)* — load a document and print a **deterministic memory-composition breakdown** as `name<TAB>value` lines on stdout: triple/term counts, the self-accounted heap total **decomposed** into dictionary / six-permutation store / numeric+temporal caches (bytes and B-per-triple, plus dict B-per-term), and the kernel's `VmRSS`/`VmHWM` (post-load resident + peak during load; Linux `/proc/self/status`, `0` elsewhere). Trailing literal `compressed` (or `SPARQ_STORE_PROFILE=compressed`, OR'd, applied once) re-encodes into the memory-bound in-RAM mode (`Graph::into_compressed`: block-compressed permutations + blob dictionary) before reporting, so both in-memory framings come from one instrument (the `mode` line says which). The at-scale extension of the CI `store_bytes_per_triple` metric — driven by `scripts/bench/bytes-per-triple.sh` (bench id `bytes-per-triple`) for the in-memory-vs-external bytes/triple envelope. Numbers are host-reported and non-canonical off the dedicated bench box; never commit them into docs.
- `bench-mmap <index-dir> <queries-dir> [iters=5] [count|materialize|json] [decompress]` — same as `bench` but opens the dataset out-of-core; trailing literal `decompress` decodes compressed perms to RAM first. Mode default `count`.
- `scaling <data-file> <format> <queries-dir> [threads=1,2,4,8,…] [iters=3]` — parallel-efficiency sweep across rayon pool sizes; TSV `subsystem\tthreads\tbest_ms\tspeedup\tefficiency`.
- `probe-compress <perm-file>` / `compare-compress <data-file> <format> [<sparql>]` / `bench-remap [n] [dict] [iters]` — measurement/instrumentation probes.
- `tabular <csv[.gz|.zst|.bz2]> [<name>=<csv> …] [flags]` *(opt-in `tabular` feature; [FABLE-5] sq-lsp7k.8)* — **materializing tabular→RDF import**, streaming end-to-end (CSV rows → first-party RFC-4180 reader → per-row N-Triples chunks → the parallel NT ingest; no whole-file buffering; compressed inputs auto-detected by extension). Two modes:
  - **Direct mapping** (default): subject = `--template` IRI template with `{col}` + `{_row}` (1-based data-row number) placeholders, default `<base><table>/row/{_row}`; predicate = `<base><table>#<column>`; object = the cell with **datatype inference** (`xsd:integer`/`xsd:decimal`/`xsd:double`/`xsd:boolean`, else plain string; `--no-infer` disables); each row typed `rdf:type <base><table>` (`--class <iri|none>` overrides). `--base` defaults to `http://example.com/`; table = file stem; template-substituted values are IRI-safe percent-encoded. An EMPTY cell is NULL → no triple (a NULL in the subject template skips the row).
  - **R2RML** (`--mapping <r2rml.ttl>`): the materializing subset over **CSV logical tables** (`rr:tableName` binds to a CSV by file stem, or explicitly via a `<name>=<path>` positional). Supports `rr:subjectMap`/`rr:subject`, `rr:class`, `rr:predicateObjectMap`, `rr:predicateMap`/`rr:predicate`, `rr:objectMap`/`rr:object`, `rr:template`/`rr:column`/`rr:constant`, `rr:termType` (IRI/Literal/BlankNode), `rr:datatype`, `rr:language`. **Fail-closed:** any other `rr:` construct — `rr:sqlQuery`, `rr:parentTriplesMap`/`rr:joinCondition`, `rr:graphMap`/`rr:graph`, `rr:inverseExpression` — is a loud exit-1 error, never a silent skip. No datatype inference here (CSV's natural datatype is string, per the spec).
  - **Output:** default = load the graph and print the summary line; add `--query <sparql>` (+ `--format`/`--count` as in `query`) to query it in the same shot; or `--out <file.nt[.gz|.zst]>` to stream N-Triples out **without building a graph**. `--sep <char|tab>` sets the separator. Usage errors exit 2; data/mapping errors exit 1. Fixture suite: `crates/sparq-cli/tests/fixtures/r2rml/` (W3C-R2RML-adapted cases); perf smoke: bench id `tabular-import-smoke`.
- `terse <terse-query | ->` *(opt-in `terse` feature; [OPUS-4.8] sq-vczh2)* — transpile a **terse** query (the `K:<name>` keyword layer over canonical SPARQL) into the **canonical, conformant SPARQL** it expands to, printing the verifiable JSON contract `{ "canonical_sparql", "keywords": [{ "keyword", "iri", "legendVersion" }], "resolutions": [], "warnings": [], "legendVersion" }` (the SAME shape the server's `POST /terse/transpile` returns). Pass `-` to read the query from stdin. It does **not** execute the query — pipe `canonical_sparql` into `query`. The `K:<name>` legend maps the hot PKG predicates/classes (e.g. `K:derivedFrom` → `<http://www.w3.org/ns/prov#wasDerivedFrom>`) so an agent need not emit a `PREFIX` line. **Loud-fail**, never a silent guess: an unknown `K:<name>`, a `PREFIX K:` collision, or non-conformant input (the silent-rewrite canary) exits 2 with a message on stderr. `resolutions` is always empty in this build — `V("phrase")` concept resolution needs the crate's `vectors` feature (a graph-bound resolver + embedder), a future extension, so a `V(...)` construct exits 2 rather than guessing (caveat `sq-26fdp`). Off by default; build with `--features terse`.

Underlying engine entry points the CLI calls (for reference): `sparq_engine::query(&Graph, &str) -> Result<QueryResult, String>`, `::ask(...) -> Result<bool, String>`, `::count(...) -> Result<usize, String>`, `::query_json(...) -> Result<String, String>`, `::construct_ntriples(...) -> Result<String, String>`, and `PreparedQuery::{parse, is_graph_form}` to classify the form. The SELECT TSV/CSV/XML serialisers are reused from `sparq_server::results::{select_to_tsv, select_to_csv, select_to_xml, ask_to_json, ask_to_xml}` (the pure serialiser library — the CLI depends on `sparq-server` with `default-features = false`, so none of the async HTTP stack is pulled in). Loading goes through `sparq_core::Graph::{load_str, load_dataset, load_reader_parallel, build_external, open, save, save_compressed}`.

## Common recipes

**Build out-of-core indexes from a compressed dump, then query without loading into RAM:**
```bash
cargo run --release -p sparq-cli -- build dump.nt.zst ntriples ./idx
cargo run --release -p sparq-cli -- query-mmap ./idx \
  'SELECT (COUNT(*) AS ?n) WHERE { ?s ?p ?o }'
```

**Materialize an RDFS closure and write it out:**
```bash
cargo run --release -p sparq-cli -- reason ontology.ttl turtle rdfs closure.nt
```

**N3 forward-chaining with a proof trace:**
```bash
cargo run --release -p sparq-cli -- reason rules.n3 turtle n3 --proof
```
(For `n3` the `<format>` argument is ignored — the file is parsed as Notation3 facts+rules.)

**Query with reasoning applied first (OWL-RL); inconsistencies print to stderr):**
```bash
cargo run --release -p sparq-cli -- query data.ttl turtle \
  'SELECT ?s WHERE { ?s a ?c }' --reason owl
```

**Get serialised SELECT bindings (CSV/TSV/XML/JSON) or a CONSTRUCT graph:**
```bash
# SELECT as CSV (W3C SPARQL Results CSV)
cargo run --release -p sparq-cli -- query data.ttl turtle \
  'SELECT ?s ?o WHERE { ?s <http://ex/age> ?o }' --format csv
# ASK as a boolean
cargo run --release -p sparq-cli -- query data.ttl turtle 'ASK { ?s ?p ?o }'   # -> true
# CONSTRUCT prints the resulting triples as N-Triples
cargo run --release -p sparq-cli -- query data.ttl turtle \
  'CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }'
```

**Compare two RDF files as triple sets (requires the `diff` feature):**
```sh
cargo run --release -p sparq-cli --features diff -- diff old.ttl new.ttl
```

**Benchmark a query suite, JSON-serialization mode, 10 iterations:**
```bash
cargo run --release -p sparq-cli -- bench data.nt ntriples ./queries 10 json
```

**Load an HDT archive (requires the `hdt` feature):**
```bash
cargo run --release -p sparq-cli --features hdt -- \
  query graph.hdt hdt 'SELECT ?s ?p ?o WHERE { ?s ?p ?o }'
# .hdt / .hdt.gz extensions are auto-detected even if you pass a wrong format arg.
```

**Export a document AS an HDT archive (requires the `hdt-write` feature; sq-8ju74):**
```sh
cargo run --release -p sparq-cli --features hdt-write -- \
  to-hdt data.ttl turtle out.hdt      # out.hdt.gz/.zst/.bz2 compress by output extension
# HDT holds a single default graph: named graphs are dropped LOUDLY (stderr warning).
```

## Gotchas / feature flags / prerequisites

- **`query` and `query-mmap` print RESULTS** (a table by default for SELECT; a boolean for ASK; N-Triples for CONSTRUCT/DESCRIBE) on stdout — load stats and reasoning/inconsistency reports go to stderr. Use `--format <table|tsv|csv|xml|json|ntriples>` to pick the SELECT/ASK serialisation, or `--count` for the legacy count-only line (`<n> solutions/triples in <ms>ms`). The two are at output parity (they share one emission core; `query-mmap` only differs in opening the indexes memory-mapped instead of loading into RAM). `bench`/`bench-mmap` are unchanged and emit TSV.
- **No `--help`.** Run a subcommand with too few args to see its one-line usage; an unknown subcommand prints the top-level usage and exits 2. Query errors exit 1.
- **`--reason <profile>` is a flag on `query`** (scanned anywhere in argv); the standalone `reason` subcommand instead takes the profile as the 3rd positional. Profiles: `rdfs`, `owl`, `n3`.
- **Default cargo features:** `mmap` (out-of-core), `mimalloc` (global allocator — `--no-default-features --features mmap` falls back to the system allocator for A/B), `dict-spill`, and — [OPUS-4.8] sq-oy1f.4 (user-prioritised epic sq-oy1f) — `jsonld` (JSON-LD parse via `oxjsonld` + the `serialize-rdf` writer matrix). `jsonld` being default-on is a deliberate maintainer-directed exception to sparq's opt-in-by-default principle; it stays toggleable (`--no-default-features --features mmap,mimalloc,dict-spill` drops the JSON-LD parser + writers).
- **Engine features every CLI build lights (dep features, so on even under `--no-default-features`):** the CLI's `sparq-engine` dependency enables `dp-planner` (DPccp cost-optimal join ordering, [SONNET-4.6] sq-7d3dj.30.5) and `algebra-rewrite` (the result-equivalent pre-execution rewrite of #1735 — `FILTER(?v = <iri>)` IRI-constant folding + `FILTER(!bound)` anti-join; [FABLE-5] sq-7d3dj.30.13). The shipped binary and every canonical benchmark therefore run the rewritten, DP-ordered plans; the sparq-engine LIBRARY defaults keep both OFF for lean library consumers.
- **HDT is opt-in:** `--features hdt` (loader) / `--features hdt-write` (adds the `to-hdt` exporter, implies `hdt`). OFF by default partly because the wrapped `hdt` crate declares MSRV **1.87** and its decode stack is dead weight on the common paths.
- **Tabular import is opt-in:** `--features tabular` adds the `tabular` subcommand (CSV direct mapping + the R2RML materializing subset; zero new deps). SQL-connection R2RML and R2RML joins (`rr:parentTriplesMap`) are out of scope and fail loudly.
- **Terse transpiler is opt-in:** `--features terse` adds the `terse` subcommand (the `K:<name>` keyword layer → canonical SPARQL, verifiable JSON expansion). OFF by default; the lean `sparq-terse` build depends only on `spargebra` (already in the CLI's tree), so it adds no heavy dep. See the `terse` subcommand above and the `http-server` skill for the matching `POST /terse/transpile` endpoint.
- **RDF diff is opt-in:** `--features diff` adds the `diff` subcommand. It uses the existing `sparq-core` load path and adds no dependency; feature-OFF builds contain no diff dispatch or implementation.
- **In-RAM store profile (native, no feature needed; [FABLE-5] sq-7d3dj.32.2.1):** `SPARQ_STORE_PROFILE` selects the in-memory store layout on the **shared load path**, so `query` / `bench` / `reason` / `scaling` inherit it uniformly from one env var. Unset or `raw` → the default six-permutation layout, **byte-identical** to before this hook existed; `compressed` → re-encode into the block-compressed in-RAM mode (`Graph::into_compressed`: block-compressed permutations + blob dictionary) after load; **any other value is a hard error (exit 2, fail-closed)** — a typo never silently falls through to raw. Query solutions are **identical** across profiles (a memory trade, never a semantic change). **Honest scope: this reduces steady-state RESIDENT footprint only — it does NOT reduce load-time peak RSS,** because `into_compressed` builds the raw perms first and encodes from them, so the peak still includes the raw build. For a genuinely peak-bound load, use the external build (`SPARQ_BUILD_COMPRESSED=1` + `open`) instead. Measure the trade with `memstat [compressed]` (see above). Not a cargo feature — the compressed code paths are compiled unconditionally; the choice is per-workload, not per-build (see `research/compressed-memory-profile.md` §2).
- **External-build env vars (native, `dict-spill` feature):** `SPARQ_DICT_SPILL=1` spills the term dictionary during `build` (N-Triples only) to bound peak RSS; tune with `SPARQ_DICT_SPILL_BUDGET_MB` (default ¼ RAM) and `SPARQ_DICT_SPILL_DISK_FLOOR_MB` (default 1024, aborts before filling disk). Output is byte-identical. `SPARQ_BUILD_COMPRESSED=1` makes `build` emit block-compressed (`SPQCPRM1`) perms in one pass (no later `recompress`; raw is the default, output byte-identical to build-then-recompress). Also `SPARQ_NO_PREFETCH=1` for the `bench-remap` probe.
- **Format ↔ ingest path:** N-Triples streams block-by-block (parallel parse, no full decompressed copy in RAM); Turtle/N-Quads/TriG are buffered whole for the parallel statement-splitter. zstd decompresses much faster than bzip2 (see the `fused-decompress-parse` skill for measured numbers) — recompress `.bz2` sources once with `zstd -9 -T0` for big ingests.
- **Compressed-perm dirs** written by `save ... compressed` / `recompress` are auto-detected by `query-mmap`/`bench-mmap`; `bench-mmap ... decompress` decodes them to RAM first.

## See also

- `core` — the `sparq_core::Graph` library API (load/save/open, the store, the Dict) used under the hood.
- `engine` — `sparq_engine::{query, count, query_json}` and query budgets / prepared queries.
- `reason` — RDFS/OWL-RL/N3 materialization (`sparq_reason`) invoked by `--reason` and the `reason` subcommand.
- `hdt` — the opt-in HDT loader (`sparq_hdt::load`).
- `hdt-write` — the opt-in `to-hdt` HDT exporter (`sparq_hdt::save`); implies `hdt`.
- `server` — the HTTP SPARQL endpoint (`sparq-server`) for serving instead of one-shot CLI queries.