arrow-columnar git:20260712.6ac4789A

Immutable. This exact content is served forever at /api/v1/blob/c46b95621e43675f.

---
name: arrow-columnar
description: Export and import sparq SPARQL SELECT QueryResult values through Apache Arrow RecordBatch or in-memory Parquet bytes with the opt-in sparq-arrow crate. Use when moving query results into dataframe, analytics, or storage tooling while preserving RDF term kinds, datatype and language metadata, RDF 1.2 triple terms, empty literals, and unbound cells.
---

# sparq-arrow — Arrow and Parquet interop

Use `sparq-arrow` when a Rust application needs a faithful columnar representation of a
`sparq_engine::QueryResult`. The crate maps every SELECT variable to one nullable Arrow
`Struct` column with five nullable UTF-8 children: `kind`, `value`, `datatype`,
`language`, and `direction`.

## Choose a feature

- Enable `arrow` for `to_record_batch`, `from_record_batch`, `term_schema`, and
  `term_struct_type`.
- Enable `parquet` for `to_parquet_bytes` and `from_parquet_bytes`; it implies `arrow`.
- Leave both features disabled to retain only the dependency-free field-name constants.

Both features are default-OFF. The crate is a leaf capability crate, so neither Arrow
nor Parquet enters `sparq-core`, `sparq-engine`, or the WebAssembly bundle.

## Use a RecordBatch

```rust,ignore
use sparq_arrow::{from_record_batch, to_record_batch};

let batch = to_record_batch(&result)?;
let restored = from_record_batch(&batch)?;
assert_eq!(restored.vars, result.vars);
assert_eq!(restored.rows, result.rows);
# Ok::<(), Box<dyn std::error::Error>>(())
```

Add the feature with `cargo add sparq-arrow --features arrow`.

## Use Parquet bytes

```rust,ignore
use sparq_arrow::{from_parquet_bytes, to_parquet_bytes};

let bytes: Vec<u8> = to_parquet_bytes(&result)?;
let restored = from_parquet_bytes(&bytes)?;
assert_eq!(restored.vars, result.vars);
assert_eq!(restored.rows, result.rows);
# Ok::<(), Box<dyn std::error::Error>>(())
```

Add the feature with `cargo add sparq-arrow --features parquet`. Parquet is only a
serialization of the same RecordBatch projection; it does not use a second term
encoding. `from_parquet_bytes` rejects unreadable files, a schema that deviates from the
five-field term struct, and invalid RDF lexical components.

## Preserve the mapping

- A null outer struct is an unbound cell. It is distinct from a bound empty-string
  literal, whose `kind` is `"literal"` and `value` is `""`.
- Plain string literals carry the explicit
  `http://www.w3.org/2001/XMLSchema#string` datatype.
- Language-tagged literals use `language`; RDF 1.2 directional literals additionally
  use `direction` with `"ltr"` or `"rtl"`.
- RDF 1.2 triple terms use `kind = "triple"` and store their N-Triples lexical form in
  `value`.
- Numeric and temporal literals remain lexical strings plus datatype IRIs. This surface
  does not narrow them into native Arrow numeric or temporal columns.

Treat the Arrow batch or Parquet bytes as a transport projection, not as a canonical RDF
serialization or an RDF document.

[GPT-5.6] Verified against `sparq-arrow` for bead `sq-lsp7k.21`.