AGENTS.md · diff
git:20260810.58cbef2 to git:20261021.bf52c45
44 added, 190 removed. Audit A to A.
- # Rust Workspace Agent Instructions
-
- ## Scope
-
- - This template targets Rust workspaces only.
- - `bin/` contains CLI binary crates.
- - `crates/` contains reusable library crates.
- - No frontend/web-framework-specific assumptions.
-
- ## Execution Strategy
-
- - Maximize parallelism by dispatching subagents aggressively and consuming tokens freely to complete tasks faster.
-
- ## Tool Usage & Commands
-
- - **NEVER execute `cargo` commands in parallel.** Rust's cargo uses strict file locks on the `target/` directory.
- - ALWAYS run `cargo check`, `cargo build`, or `cargo test` sequentially. Wait for one to finish before starting the next.
- - When fixing errors, execute the file `Write`/`Edit` tool FIRST, wait for it to succeed, and only THEN run `cargo` commands to verify. Do not parallelize file edits with cargo builds.
-
- ## Build Configuration
-
- - Ensure `.cargo/config.toml` contains:
- ```toml
- [build]
- rustc-wrapper = "kache"
- ```
- This wraps `rustc` with [kache](https://github.com/nicholasgasior/kache) to cache compilation artifacts and significantly speed up incremental builds.
-
- ## Cargo Workspace Rules (Critical)
-
- 1. Never manually type dependency versions in `Cargo.toml`; use `cargo add`.
- 2. Add workspace-level dependencies with:
-
- ```bash
- cargo add <crate> --workspace
- ```
-
- 3. Add sub-crate dependencies with:
-
- ```bash
- cargo add <crate> -p <crate-name> --workspace
- ```
-
- 4. Root `[workspace.dependencies]` must use numeric versions only.
- 5. Root `[workspace.dependencies]` must not carry features by default.
- 6. Sub-crates must use `workspace = true` for `version`, `edition`, and shared dependencies.
-
- ## Preferred Dependencies
-
- When introducing new dependencies, prefer these crates unless compatibility requires a different choice:
-
- ```
- clap, config, eyre, proptest, serde, thiserror, tokio, tracing, tracing-subscriber,
- tracing-opentelemetry, opentelemetry, opentelemetry-otlp, sqlx, utoipa, utoipa-swagger-ui,
- arc-swap, hpx, scc, winnow, shadow-rs, ecdysis
- ```
-
- Always use the latest stable version available on crates.io. Run `cargo add <crate> --workspace` to add dependencies — this automatically resolves to the latest version.
-
- ## Dependency Priority and Forbidden Choices
-
- - HTTP client preference: `hpx` (with `rustls`) over `reqwest`.
- - Concurrent map/set preference: `scc` over `dashmap` and `RwLock<HashMap<...>>`.
- - Parsing preference: `winnow` or `pest` over ad-hoc manual parsing.
- - Read-heavy shared state: `arc-swap` over `RwLock`.
- - Forbidden by default: `anyhow`, `log`, `reqwest`, `dashmap`.
-
- ## Engineering Principles
-
- ### Rust Implementation Guidelines
-
- 1. Error handling:
- - Application layer: `eyre`.
- - Library layer: `thiserror`.
- 2. Database (`sqlx`):
- - Prefer runtime queries (`sqlx::query_as`).
- - DB structs should derive `sqlx::FromRow`.
- - Avoid compile-time `sqlx::query!` macros by default.
- 3. Concurrency:
- - Prefer lock-free/container-first approaches (`scc`, `ArcSwap`).
- - Avoid `Arc<Mutex<T>>` when better alternatives are available.
- 4. Observability:
- - Logging: `tracing` only.
- - Metrics/traces: OpenTelemetry OTLP gRPC.
- - Prometheus should not be the default instrumentation path.
- 5. API docs:
- - Generate OpenAPI with `utoipa` when exposing HTTP APIs.
- 6. Configuration:
- - Use the `config` crate and external configuration files (prefer TOML).
- 7. Binaries:
- - Use `ecdysis` for graceful restart/reload flows in daemon/server binaries.
- 8. Safety:
- - Use `unsafe` only when strictly necessary and document the safety invariants.
-
- ### Key Design Principles
-
- - Modularity: Design each crate so it can be used as a standalone library with clear boundaries and minimal hidden coupling.
- - Performance: Prefer architectures that support parallelism, memory-mapped I/O for large read-heavy workloads, optimized data structures, and lock-free data types.
- - Extensibility: Use traits and generic types to support multiple implementations without invasive refactors.
- - Type Safety: Maintain strong static typing across interfaces and internals, with minimal use of dynamic dispatch.
-
- ### Performance Considerations
-
- - Avoid allocations in hot paths; prefer references and borrowing to reduce allocation and copy overhead.
- - Use `rayon` for CPU-bound parallel processing.
- - Use `tokio` async/await for I/O-bound concurrency.
-
- ### Concurrency and Async Execution
-
- - Prefer atomic types (`AtomicUsize`, `AtomicBool`, etc.) with explicit `Ordering` for simple shared state.
- - Use `scc` for highly concurrent maps/sets; avoid `Arc<RwLock<HashMap<...>>>` and `Arc<Mutex<HashMap<...>>>` on hot paths.
- - Use `moka` for concurrent caches instead of custom LRU implementations.
- - Prefer `parking_lot::{Mutex, RwLock}` over `std::sync` locks for synchronous locking.
- - Release `std::sync::Mutex` and `parking_lot::Mutex` guards before hitting any `.await` point.
- - Use `tokio::sync::Mutex` for locks that span across `.await` points.
- - Use `tokio::task::spawn_blocking` for CPU-bound work and blocking I/O.
- - Batch work or use bounded worker patterns instead of spawning massive volumes of tiny Tokio tasks.
- - Channel selection:
- - Async-to-Async: `tokio::sync::mpsc` / `tokio::sync::broadcast`
- - Sync/MPMC: `crossbeam-channel` or `flume`
- - Avoid `std::sync::mpsc`
-
- ### Memory and Allocation
-
- - For binary server applications, configure `tikv-jemallocator` or `mimalloc`.
- - For trusted internal hash keys, prefer `ahash` or `rustc-hash` over default SipHash-based maps.
- - Use `compact_str` or `smol_str` for small-string-heavy paths.
- - Prefer `beef::Cow` over `std::borrow::Cow` when minimizing footprint.
- - Use `bytes::Bytes` / `bytes::BytesMut` for network buffers; pass `Bytes` instead of cloning `Vec<u8>`.
- - For critical serialization hot paths, prefer `rkyv` or `zerocopy`; reserve `serde_json` for config and non-critical APIs.
-
- ### Type and Layout
-
- - Order struct fields from largest to smallest unless stronger semantic grouping is required.
- - Use `#[repr(C)]` / `#[repr(packed)]` only for FFI or fixed protocol layout requirements.
- - Keep error types compact on hot paths; box large error payloads when needed to reduce `Result<T, E>` size.
- - Prefer typestate-style APIs for compile-time state transitions instead of runtime state checks.
-
- ### Tooling and Hot Paths
-
- - Keep code clean under `clippy::pedantic`, `clippy::nursery`, and `clippy::cargo` (allow `missing_errors_doc` for non-public APIs when needed).
- - Use `#[inline]` for tiny methods called in hot loops or on every request, especially across crate boundaries.
- - Mark cold error paths with `#[cold]` and `#[inline(never)]` when it improves hot-path instruction locality.
-
- ### Common Pitfalls
-
- - Keep async tasks non-blocking; offload CPU-bound work to `spawn_blocking` or `rayon`.
- - Handle errors explicitly and consistently with the `?` operator and concrete error types.
-
- ### What to Avoid
-
- - Incomplete implementations: finish features before submitting.
- - Large, sweeping changes: keep changes focused and reviewable.
- - Mixing unrelated changes: keep one logical change per commit.
-
- ## Foundry Rules (If Solidity Exists)
+ # shellflow Agent Instructions
- - Use `soldeer` for dependencies; do not use git submodules.
- - Required commands:
- - `forge soldeer install`
- - `forge soldeer update`
- - `forge build`
- - `forge test`
+ ## Non-negotiables
- ## Development Workflow
+ - Surgical edits only: every changed line must trace to the request. No drive-by refactors, no reformatting untouched code, no deleting pre-existing dead code (mention it instead).
+ - Fix root causes. Never silence a warning, weaken a check, or patch a symptom to make a gate pass.
+ - Never report completion while any `just` gate below fails. Report the failure verbatim and stop.
+ - Stop and ask when a request is ambiguous, conflicts with existing conventions, or requires credentials, publishing, or destructive operations.
+ - When a correction reveals a missing rule, add one specific rule here ("always X for Y", never "be careful with Y"); delete rules that never catch real errors.
- When fixing failures, identify root cause first, then apply idiomatic fixes instead of suppressing warnings or patching symptoms.
+ ## Scope
- ## Parallelization and Resource Utilization
+ - `shellflow` is a shell-native, agentless automation CLI: it drives a plain Bash script whose blocks are delimited by `# @` comment directives, over the system `ssh`/`rsync`/`scp`/`bash`. Targets need only `bash`; there is no agent, no SSH library and no YAML.
+ - `crates/shellflow-dsl` owns the directive scanner and the plan model, `crates/shellflow-core` the templating and exported-variable handling, `crates/shellflow-secrets` the `age`-encrypted secret store, and `bin/shellflow` the CLI, executors and service management.
+ - `examples/env-app` and `examples/secret-app` are `publish = false` demo targets; `playbooks/deploy.sh` exercises every DSL feature against the `trade`/`api` hosts from `~/.ssh/config`; `SKILL.md` is the agent-facing skill; `docs/design.md` holds the architecture (Part I DSL/engine, Part II `S1`-`S12` secrets).
+ - English only in code, comments, docs, and commit messages.
- - **Use as many subagents and as much token budget as needed** to complete tasks efficiently. Parallelize independent work aggressively and maximize context utilization. Spawn subagents for independent research, code exploration, and implementation tasks to reduce latency and improve throughput.
+ ## Execution
- Use test-driven development for behavior changes:
+ - Parallelize aggressively: dispatch subagents for independent research, exploration, and implementation.
+ - NEVER run two `cargo` commands concurrently, and never overlap file edits with a running `cargo` command (`target/` file lock): finish edits, wait, then run.
+ - NEVER use `git worktree`. Edit the current branch in place.
+ - TDD for behavior changes: failing test first, smallest change to pass, then run `just mutation` and fix surviving mutants.
- - **Git Restrictions:** NEVER use `git worktree`. All code modifications MUST be made directly on the current branch in the existing working directory.
- - drive implementation with failing crate-local unit tests and `proptest` properties in the affected crate,
- - keep `proptest` in the normal `cargo test` loop instead of creating a separate property-test command,
- - treat `cargo-fuzz` as conditional planning work rather than baseline template setup,
- - after the inner loop is green, run `just mutation` (cargo-mutants) and fix any surviving mutants.
+ ## Verification loop
- After each feature or bug fix, run:
+ After each feature or bug fix, in order; any failure means the task is not done:
```bash
just format
just lint
just test
just mutation
```
- If any command fails, report the failure and do not claim completion.
+ `just lint` is the CI gate: `typos`, `rumdl`, `cargo sort --check`, `cargo +nightly fmt --check`, `cargo +nightly clippy -D warnings`, `cargo shear`. `just fuzz-check` type-checks the fuzz targets; `just fuzz SECS=60` runs them.
- ## Testing Requirements
+ ## Rust workspace rules
- - Unit tests: colocate with implementation (`#[cfg(test)]`).
- - Use `cargo test` as the TDD inner loop; write a failing test first, then implement the smallest change that makes it pass.
- - Prefer example-based unit tests for named business cases and edge cases, and reserve `proptest` for invariants that should hold across many generated inputs.
- - Property tests: colocate `proptest` coverage with the crate logic it exercises so it runs through the ordinary `cargo test` path.
- - Cover boundary conditions (empty inputs, zero values, maximum sizes) with explicit assertions.
- - Mutation testing: run `just mutation` (cargo-mutants) and fix any surviving mutants to keep the mutation score high.
- - Benchmarks: only plan or add Criterion when the scope includes an explicit latency SLA, throughput target, or known hot path in a specific crate.
- - Benchmark workflow: when benchmarking is justified, add Criterion only in the affected crate instead of pre-seeding benchmark scaffolding across the workspace.
- - Fuzz tests: only plan or add `cargo-fuzz` when a crate parses hostile input, implements protocols, decodes binary formats, or contains meaningful `unsafe` code.
- - Fuzz workflow: when fuzzing is justified, generate the standard layout in the affected crate with `cargo fuzz init`, then run targets with `cargo fuzz run <target>` instead of pre-seeding a `fuzz/` directory in every starter.
- - For `/pb-plan` work, mark benchmarking as `conditional` or `N/A` unless the scope explicitly includes a performance requirement or hot path, and mark fuzzing as `conditional` or `N/A` unless the scope explicitly includes parser-like, protocol, binary-decoding, or `unsafe`-heavy code.
- - Integration tests: place in crate-level `tests/`.
- - Add tests for behavioral changes and public API changes.
+ - Edition 2024, resolver 3. Members are `crates/*`, `bin/*` and `examples/*`; the fuzz crates under `crates/*/fuzz` are detached workspaces and are excluded from the root workspace on purpose.
+ - Because the fuzz crates are detached, `cargo fmt --all` skips them: `just format` formats their `fuzz_targets/*.rs` explicitly with `rustfmt --edition 2024`. Keep it that way when a fuzz target is added.
+ - `unwrap_used`, `expect_used`, `panic`, `todo`, `unimplemented`, `print_stdout`, `print_stderr` and `allow_attributes` are `deny`. The only escape hatch is `#[expect(lint, reason = "...")]`.
+ - `.cargo/config.toml` sets `rustc-wrapper = "kache"`; clippy invocations must clear it (`RUSTC_WRAPPER=`) exactly as the Justfile does.
+ - There is no `rust-toolchain.toml`: CI installs nightly, so use `cargo +nightly fmt` and `cargo +nightly clippy` exactly as the Justfile does.
+ - `.cargo/mutants.toml` excludes `bin/shellflow/src/**` and `examples/*/src/**` because cargo-mutants only runs each crate's own test target; the orchestration logic is covered by `bin/shellflow/tests/integration.rs` instead.
+ - Never hand-type dependency versions. Shared crates go into root `[workspace.dependencies]` and members reference them with `workspace = true`.
- ## Language Requirement
+ ## Dependency choices
- - Documentation, comments, and commit messages must be English only.
+ - The DSL is Bash, not a parser library: `shellflow-dsl` scans `# @` directives itself. Do not introduce a YAML/TOML script format.
+ - Remote execution shells out to the system `ssh`/`rsync`/`scp` through `tokio::process::Command`; never add an SSH client crate.
+ - Secrets use `age` for encryption and `zeroize` for in-memory wiping. Secret values are masked in every preview, including local blocks.
+ - `nix` covers process/signal/user/fs handling, `sd-notify` the systemd readiness protocol, `zbus` the optional D-Bus path, `tokio` (`full`) the async runtime, `clap` (derive) the CLI, `colored` and `tracing-subscriber` (env-filter) the output.
+ - Errors: `thiserror` in the library crates, `eyre` in `bin/shellflow`. Concrete error types through `?`; never string errors.
+ - Forbidden here: `anyhow`, `log`, `reqwest`, `dashmap`.
+
+ ## Code conventions
+
+ - A deploy script must stay 100% valid Bash: `bash deploy.sh` has to run it unchanged. Never emit syntax that only the shellflow parser accepts.
+ - Directives are `# @server`, `# @group`, `# @local`, `# @remote`, `# @copy`, `# @export`, `# @env`, `# @only_if`, `# @timeout` and `# @name`; add new ones to `shellflow-dsl` with a parse test, never ad hoc in the executors.
+ - Multi-host fan-out goes through `JoinSet` plus a semaphore; never spawn one task per host without the bound.
+ - The `dbus` feature (systemd unit state over D-Bus) is off by default so the standard build stays a single static binary; the CLI polls instead. Keep D-Bus code behind the feature.
+ - `unsafe` appears only in `shellflow-secrets` test helpers that call `std::env::set_var`/`remove_var`; they hold `ENV_LOCK` and carry a `// SAFETY:` comment. Keep that pattern.
+ - `proptest` properties live next to the crate logic they exercise and run in the ordinary `cargo test` loop; the fuzz targets cover the hostile-input parsers (`fuzz_parse_script`, `fuzz_interpolate`, `fuzz_parse_env_file`).