git:20260519.287447c to git:20260519.a957b43

1 added, 168 removed. Audit A to B.

What this update changes in the checks

No check changed: every fixed check gives the same answer before and after this update.

A tripped check is a label, not a verdict: read the matched text and the lines below. This week's updates that newly trip a check.

- # Copilot Instructions for `oxo-flow`
-
- ## ⚠️ Mandatory pre-commit CI gate
-
- **Before every call to `report_progress`, ALL of the following checks MUST pass locally with zero errors.**
- Pushing code that fails any of these checks will break the CI "Test" job and is not acceptable.
-
- ```bash
- # Option A – run everything in one command (preferred):
- make ci
-
- # Option B – run each step individually:
- cargo fmt -- --check # formatting (MUST pass – most commonly forgotten)
- cargo clippy -- -D warnings # zero lint warnings allowed
- cargo build # must compile
- cargo test # all unit + integration tests must pass
- ```
-
- If `cargo fmt -- --check` reports diff output, fix it first with `cargo fmt` and re-run the check.
- **Never call `report_progress` until `make ci` (or all four individual commands) exits with code 0.**
-
- ## Project overview
-
- **oxo-flow** is a Rust-based bioinformatics pipeline engine built from first principles for performance, reproducibility, and clinical-grade rigor.
- It provides a core library, CLI, and web interface for building, running, and managing
- reproducible bioinformatics workflows with first-class support for multiple software
- environment managers (conda, pixi, venv, docker, singularity, etc.).
-
- Licensed under Apache 2.0 — fully open source and free.
-
- ## Workspace layout
-
- ```
- oxo-flow/ # Cargo workspace root
- ├── Cargo.toml # workspace manifest
- ├── crates/
- │ ├── oxo-flow-core/ # Core library: DAG engine, environment, config, reporting
- │ │ ├── Cargo.toml
- │ │ └── src/
- │ │ ├── lib.rs # Public API surface
- │ │ ├── dag.rs # DAG construction, validation, topological sort
- │ │ ├── executor.rs # Task execution engine (local, cluster, cloud)
- │ │ ├── environment.rs # Environment management (conda, pixi, docker, singularity, venv)
- │ │ ├── config.rs # Workflow configuration and .oxoflow file parsing
- │ │ ├── rule.rs # Rule/step definitions with inputs, outputs, shell, resources
- │ │ ├── scheduler.rs # Job scheduling with resource constraints
- │ │ ├── report.rs # Modular report generation system
- │ │ ├── wildcard.rs # Wildcard pattern expansion ({sample}, {chr}, etc.)
- │ │ ├── error.rs # Unified error types
- │ │ └── container.rs # Container build & packaging utilities
- │ ├── oxo-flow-cli/ # CLI binary crate
- │ │ ├── Cargo.toml
- │ │ └── src/
- │ │ └── main.rs # Clap-based CLI entry point
- │ ├── oxo-flow-web/ # Web interface crate (axum-based API + frontend)
- │ │ ├── Cargo.toml
- │ │ └── src/
- │ │ └── lib.rs # Web server with REST API
- ├── examples/ # Example .oxoflow files
- ├── docs/ # Documentation (MkDocs)
- │ └── guide/
- │ └── src/
- └── tests/ # Integration tests
- ```
-
- ## Build, test, lint (individual command reference)
-
- ```bash
- cargo build # build all workspace members
- cargo test # run all tests (unit + integration)
- cargo fmt -- --check # check formatting
- cargo clippy -- -D warnings # lint (zero warnings allowed)
- ```
-
- Integration tests should test the CLI binary and core library public API.
-
- ## Key design principles
-
- 1. **DAG-first execution** — All workflows are compiled into a Directed Acyclic Graph before execution. The DAG engine handles dependency resolution, topological ordering, and parallel execution.
-
- 2. **Environment-agnostic** — Every rule/step can declare its software environment (conda, pixi, docker, singularity, venv). The executor resolves and activates the correct environment before running each task.
-
- 3. **Wildcard expansion** — oxo-flow supports `{sample}`, `{chr}` style wildcards in file paths. The engine expands wildcards based on input file discovery or explicit configuration.
-
- 4. **Modular reporting** — The report system generates structured HTML/PDF/JSON reports from templates, suitable for patient-facing clinical reports.
-
- 5. **Container-first reproducibility** — Workflows can be packaged into self-contained containers with all dependencies, data references, and configuration baked in.
-
- 6. **Resource-aware scheduling** — Jobs declare CPU, memory, GPU, and disk requirements. The scheduler respects resource constraints and supports cluster backends (SLURM, PBS, SGE).
-
- ## Critical conventions
-
- **Error handling** — Use `thiserror` for library error types, `anyhow` for CLI/binary error handling. Every public API function returns `Result<T, OxoFlowError>`.
-
- **Serialization** — Use `serde` with TOML as the primary config format. The `.oxoflow` file format is TOML-based.
-
- **Async runtime** — Use `tokio` for async execution. The DAG executor runs tasks concurrently via tokio tasks.
-
- **Logging** — Use `tracing` for structured logging throughout the codebase.
-
- **CLI structure** — Use `clap` with derive macros. Subcommands mirror the web API endpoints for consistency:
- - `oxo-flow run` — Execute a workflow
- - `oxo-flow dry-run` — Simulate execution, show DAG
- - `oxo-flow validate` — Validate .oxoflow file
- - `oxo-flow graph` — Visualize DAG
- - `oxo-flow report` — Generate reports
- - `oxo-flow env` — Manage environments
- - `oxo-flow package` — Package workflow into container
- - `oxo-flow serve` — Start web interface
-
- ## Adding / editing things
-
- **New rule type or feature in core:**
- 1. `crates/oxo-flow-core/src/` — add/modify the relevant module
- 2. Update `lib.rs` — export new public types
- 3. Add unit tests in the same file or `tests/` directory
- 4. Update documentation
-
- **New CLI subcommand:**
- 1. `crates/oxo-flow-cli/src/main.rs` — add Clap subcommand variant
- 2. Implement handler function
- 3. Add integration test
- 4. Update CLI docs
-
- **New pipeline:**
- 1. Create `.oxoflow` files in `pipelines/<name>/`
- 2. Add environment specs in `pipelines/<name>/envs/`
- 3. Add report templates if applicable
- 4. Document in `docs/`
-
- ## .oxoflow file format
-
- The `.oxoflow` file format is TOML-based with the following top-level sections:
-
- ```toml
- [workflow]
- name = "my-pipeline"
- version = "1.0.0"
-
- [config]
- samples = "samples.csv"
- reference = "/path/to/ref.fa"
-
- [[rules]]
- name = "fastqc"
- input = ["{sample}_R1.fastq.gz", "{sample}_R2.fastq.gz"]
- output = ["qc/{sample}_R1_fastqc.html"]
- threads = 4
- memory = "8G"
- environment = { conda = "envs/qc.yaml" }
- shell = "fastqc {input} -o qc/"
-
- [[rules]]
- name = "bwa_align"
- input = ["trimmed/{sample}_R1.fastq.gz"]
- output = ["aligned/{sample}.bam"]
- threads = 16
- memory = "32G"
- environment = { docker = "biocontainers/bwa:0.7.17" }
- shell = "bwa mem -t {threads} {config.reference} {input} | samtools sort -o {output}"
- ```
-
- ## Documentation
-
- MkDocs source lives under `docs/guide/src/`.
-
- When CLI behavior changes, update docs accordingly.
- When architecture changes, update this file and relevant docs.
+ Please read and strictly follow the instructions in AGENTS.md at the root of the repository.