AGENTS.md · diff

git:20260910.77cf2f6 to git:20260917.ec0f6a6

82 added, 18 removed. Audit A to A.

# AGENTS.md

Guidance for AI coding agents working on the Soliplex project. Human

contributors should read [DEVELOPMENT.md](DEVELOPMENT.md), which covers the

same ground in prose. (`CLAUDE.md` is a thin stub that imports this file so

Claude Code loads it automatically.)

## Project Overview

Soliplex is an AI-powered RAG system with a FastAPI backend, Flutter web

frontend, and terminal UI. It provides semantic document retrieval,

multi-room chat, and multi-provider LLM support.

**This repository contains the Python backend and TUI only.** The Flutter

frontend lives in a sibling repo at <https://github.com/soliplex/frontend>

with its own tooling.

## Build and Test

```bash

# Install dependencies (use uv, not pip)

uv sync --group dev

# Run unit tests with 100% coverage requirement

uv run pytest

# Run a specific test file / test. '--no-cov' is required: 'addopts'

# always applies the 100% gate, which a partial run cannot satisfy.

uv run pytest --no-cov tests/unit/test_agents.py

uv run pytest --no-cov tests/unit/test_agents.py::test_name

# Run the unit tests in parallel (pytest-xdist); see below for '-n'

uv run pytest -n 8

# Run functional tests (require a running LLM); never pass '-n' here

uv run pytest --no-cov tests/functional/ -m needs_llm

# Lint and format

uv run ruff check

uv run ruff format --check

# Auto-fix lint and format issues

uv run ruff check --fix

uv run ruff format

# Start a dev server (no auth)

uv run soliplex-cli serve example/minimal.yaml --no-auth-mode

# Validate a configuration

uv run soliplex-cli audit example/minimal.yaml

```

## Code Style

- Line length: 79 characters

- Single-line imports enforced (isort via ruff)

- Ruff rule sets: F, E, B, U, I, PD, TRY, PT

- Target version: Python 3.13

- Use `uv run` to execute all Python commands

## Pre-commit Hooks

Optional: `uv run pre-commit install` automates the CI checks before each

commit (`uv run pre-commit run --all-files` runs them once against the whole

tree). The configured hooks (see `.pre-commit-config.yaml`) enforce:

- `ruff-check` / `ruff-format` -- lint and format Python sources

- `pymarkdown` -- lint Markdown files

- `agentskills-validate` -- validate the published `soliplex-docs` agent

skill; runs only when something under `skills/soliplex-docs/` changes

- `lint-textio` -- reject text file IO in `src/soliplex/` without an

explicit `encoding=` (falls back to the host locale encoding, `cp1252` on

Windows); `scripts/lint_textio.py`, stdlib-only

- `lint-textio-self-test` -- run `scripts/lint_textio.py --self-test` when

that script itself changes, so a check gone blind fails loudly instead of

passing by finding nothing

- `actionlint` -- lint GitHub Actions workflow files

- `check-toml` / `check-yaml` -- validate TOML and YAML syntax

- `gitleaks` -- scan for committed secrets

- `pip-audit` -- scan dependencies for known vulnerabilities (runs when

`pyproject.toml`, `uv.lock`, or `.pre-commit-config.yaml` changes)

- `debug-statements` -- reject leftover `pdb` / `breakpoint()` calls

- `trailing-whitespace` / `end-of-file-fixer` -- normalize whitespace

- `check-merge-conflict` -- reject unresolved merge-conflict markers

- `no-commit-to-branch` -- block direct commits to `main` / `master`

## Testing Requirements

- Unit tests live in `tests/unit/`, mirroring the `src/soliplex/` structure

- 100% branch coverage is enforced via pytest-cov (`--cov-fail-under=100`)

- Coverage measures four targets (see `addopts` in `pyproject.toml`):

`src/soliplex`, `tests/unit`, `scripts`, and

`skills/soliplex-docs/scripts` -- the test suite and the helper scripts

are held to the same 100% bar as `src/`

- Those targets are *paths*, not importable names, and must stay that way.

`soliplex` is a namespace package (there is no

`src/soliplex/__init__.py`); coverage cannot enumerate one by walking the

filesystem, so `--cov=soliplex` would measure only the modules some test

happened to import, and a module nobody imports would pass unnoticed

- `[tool.coverage.run] omit` is therefore the single place that decides

what is exempt: `scripts/lint_textio.py` (it runs its own `--self-test`

during lint) and `src/soliplex/tui/*` (the TUI is deliberately

untested). Everything else under `src/soliplex/` must reach 100%

- Use pytest-asyncio for async tests

- Functional tests (`tests/functional/`) require a running LLM and are

skipped by default (marker: `needs_llm`)

- `--cov-fail-under=100` lives in `addopts`, so it applies to *every*

`pytest` invocation, not just full runs. Pass `--no-cov` when running a

subset -- a single file, a single test, or the functional suite --

otherwise the run fails on the threshold no matter how the tests

themselves fared

- `pytest-xdist` is a dev dependency, but no `-n` appears in `addopts`, so

runs are serial unless you ask for parallelism

- **Unit tests are parallel-safe.** Choosing `-n`: start from `nproc` and

take about half the logical cores (so `-n 8` on a 16-core box). Each

worker pays a fixed startup and collection cost, so the gain flattens

well before one-worker-per-core, and leaving half the cores free keeps

the machine usable while the suite runs. Tune from there if a run feels

slow -- the best value is machine-specific

- `-n auto` means one worker per logical core. It is the right choice only

where the core count is not known ahead of time, which is why CI uses it

and you generally should not

- **Functional tests must stay serial -- never pass `-n` to them.** They

share on-disk state and module-scoped app fixtures: e.g.

`tests/functional/test_sandbox_workdirs.py` creates, and on teardown

`rmtree`s, a workdir tree keyed by a constant `ROOM_ID`, so parallel

workers delete each other's fixtures. CI runs them in a separate,

`-n`-less step

- Coverage and xdist compose (pytest-cov merges the workers' data), so

`-n` does not weaken the 100% gate

- **The suite runs on Windows and macOS too.** The one exception is

the bubblewrap sandbox, which is Linux-only, and so are the symlinks,

FIFOs and `O_NOFOLLOW` opens its tests set up. The two modules

covering it (`test_bwrap_sandbox.py` under `tests/unit/skills/`, and

`test_sandbox_workdirs.py` under `tests/unit/views/`) therefore carry

`pytestmark = _platform.requires_posix_sandbox` and skip whole off

POSIX

- The 100% gate still applies off POSIX. `tests/conftest.py` drops the

two skipped modules, and the two they cover, from the coverage

*report* (`tests/_platform.py: POSIX_ONLY_COVERAGE_OMIT`), warning

that it has -- so everything the host can measure is still held to

100%, and only the sandbox needs re-checking on Linux

- CI checks that: `.github/workflows/python-test.yaml` runs the unit

suite on `windows-latest` as well, on Python 3.13 only, with the

functional step skipped there. The 100% gate applies to that job like

any other

- Never loosen an assertion to make a platform pass. An assertion on a

rendered path should build its expectation with `pathlib` so it holds

on either separator; a test that needs a POSIX-only primitive and is

*not* about the sandbox needs a gate of its own rather than a weaker

check

## Repository Structure

Non-obvious modules and directories (the rest are self-explanatory from

their filenames -- `ls src/soliplex/` for the full layout):

- `agui/` -- AG-UI protocol (threads, runs, persistence)

- `authz/` -- authorization policy engine

- `config/` -- YAML config parsing (16 modules; see `installation.py` for

the top-level entry)

- `tools/` -- agent tools (RAG, feedback, file uploads)

- `agents.py` -- Pydantic AI agent creation

- `completions.py` -- OpenAI-compatible streaming endpoint (not just

LLM-level completions)

- `installation.py` -- installation lifespan, admin bootstrap, and global

state management

- `main.py` -- FastAPI app factory (`create_app`)

- `tests/unit/` -- 100% coverage required; mirrors `src/soliplex/`

- `tests/functional/` -- tests requiring an LLM (marked `needs_llm`) are

skipped by default; other functional tests run

- `example/` -- sample configs (rooms, completions, oidc, quizzes, skills)

- `schemas/` -- AG-UI feature JSON schemas

Key files:

- `pyproject.toml` -- dependencies, scripts, tool config

- `src/soliplex/config/installation.py` -- master config parsing

- `src/soliplex/main.py` -- FastAPI app factory

- `example/installation.yaml` -- full config example

- `example/minimal.yaml` -- minimal config for development

- `.env.example` -- environment variable reference

## Configuration System

- YAML-based hierarchical config in `src/soliplex/config/` (16 modules)

- Top-level entry: `InstallationConfig` in `config/installation.py`

- Config classes use dataclasses with a `from_yaml` classmethod

- Private fields `_installation_config` and `_config_path` carry context so

nested configs can resolve env vars, secrets, and paths relative to the

config file without threading them through every `from_yaml` call

- Environment variables resolved via `Installation.get_environment()`

- Secrets resolved via a configurable source chain (env vars, files,

subprocess, random generation) in `config/secrets.py`

## Database Migrations

- Alembic drives **two** databases from one revision tree (`alembic.ini`:

- `databases = agui, authz`):

+ The operator- and developer-facing reference is

+ [docs/server/migrations.md](docs/server/migrations.md) (published as

+ "Database Migrations"); what follows is the part that constrains how code

+ in this repository is written.

+ Alembic drives **two** databases from one revision tree, which lives in the

+ package -- `src/soliplex/alembic_migrations/` -- and therefore ships in the

+ wheel, so a deployment can migrate from its own image:

- `agui` -- `thread_persistence_dburi`, schema `soliplex.agui.schema`

- `authz` -- `authorization_dburi`, schema `soliplex.authz.schema`

- Every revision therefore has `upgrade_agui()` / `upgrade_authz()` pairs

- behind an `upgrade(engine_name)` dispatcher.

+ Every revision has `upgrade_agui()` / `upgrade_authz()` pairs behind an

+ `upgrade(engine_name)` dispatcher.

+ **The revisions are the schema.** a writable open brings its database to the

+ alembic head, which creates it when it does not exist and leaves it

+ stamped. That is only safe because migrating an empty

+ database from base reproduces the models exactly --

+ `scripts/lint_alembic_chain.py` proves it on every CI run, and its

+ `--self-test` proves the comparison is not blind.

+ `soliplex.alembic_migrations` is the API: `ensure_current_engine(engine,

+ database, *, sole_writer)` for an async engine, the

+ `ensure_current_connection()` it runs on the caller's connection, plus

+ `head_revision()`, `database_state()` and `upgrade()`. One database at a

+ time, and three situations:

+ - at head -- returns, having written nothing (one `SELECT`);

+ - empty, or behind head -- migrates, **but only when this process is the sole

+ writer**. `serve --workers N` (N > 1) sets `_SOLIPLEX_MULTIPLE_WRITERS` in

+ the private env-var contract, so no worker migrates and the app raises

+ `MigrationRequired` instead; migrate ahead of time with every writer

+ stopped;

+ - tables but no `alembic_version` row -- raises `UnstampedDatabase`, naming

+ the one-off bootstrap script (issue #1367). Such a database was built by

+ soliplex <= 0.81, when nothing stamped; see #1368 for the promise that no

+ later release leaves one that way.

+ Every writable open goes through one of two helpers, which own the

+ migration and the engine's lifetime: `installation.open_engines` (for

+ `lifespan`) and `cli_util.open_db` (the pre-flight for every `admin-users`

+ / `room-authz` command, and for `cli/ask.py`). `open_db` also owns the CLI

+ policy an in-memory DBURI needs -- `alembic_migrations` knows nothing about

+ RAM databases. `audit` is a reader: it passes `must_exist=True`, so nothing

+ is created or migrated, and reports an uncreated database as "nothing

+ configured".

```bash

# Upgrade both databases

uv run alembic -x soliplex.installation_path=<dir> upgrade head

- # Emit SQL instead: writes 'agui.sql' and 'authz.sql' into the cwd

+ # Emit SQL instead: writes 'agui.sql' and 'authz.sql' into the cwd, and

+ # never connects

uv run alembic -x soliplex.installation_path=<dir> upgrade head --sql

- # New revision (per-database stubs from 'alembic/script.py.mako')

+ # New revision (per-database stubs from the tree's 'script.py.mako')

uv run alembic -x soliplex.installation_path=<dir> revision -m "soliplex-vX.Y"

```

- - The DB URIs come from the installation config, not `alembic.ini`, so

- `-x soliplex.installation_path=` is mandatory; without it `env.py` prints

- its usage and exits 2

- - `env.py` builds both engines at import with `init_schema=True`, so *every*

- command -- `current` and `history` and `--sql` included -- connects and

- runs `metadata.create_all()`. There is no read-only command; use a

- throwaway installation when experimenting

+ **Those are checkout-only.** `script_location` lives in `[tool.alembic]` in

+ `pyproject.toml`, which no deployment image carries, so the bare `alembic`

+ CLI there fails with `No 'script_location' key found in configuration`. A

+ deployment migrates through soliplex's own writable open, or by calling

+ `alembic_migrations.upgrade()`, both of which set `script_location` from the

+ package directory.

+ - The DB URIs come from the installation config rather than from any alembic

+ config file, so `-x soliplex.installation_path=` is mandatory; without it

+ `env.py` prints its usage and exits 2. `resolve_dburis()` reads that

+ config with `config.installation.load_installation()` plus

+ `resolve_environment()` -- all the DBURIs depend on, and nothing more, so

+ an unrelated broken room or OIDC config cannot block a migration

+ - In-process callers pass either an explicit `{name: dburi}` mapping or a

+ live `connection` and the `database` it belongs to through

+ `config.attributes`. The connection form is what lets an in-memory

+ database be migrated at all: it lives inside one engine, so the migration

+ has to run on the caller's own connection

+ - `env.py` is a four-line shim over `alembic_migrations.run()`, and builds

+ its engines without creating anything, so `current`, `history`, `check`

+ and `--sql` leave the schema alone. That is what makes `alembic check`

+ meaningful: it compares the models against the migrated database

- Revisions must not import live app code: freeze any borrowed helper into

the revision, so replaying history never depends on the current tree

- Mode-dependent logic splits into `_x_online()` / `_x_offline()` behind a

`context.is_offline_mode()` dispatcher; offline has no bind, so express it

as set-based `op.execute()` SQL

- Branch on `op.get_context().dialect.name` where SQLite and PostgreSQL

differ

- - Config is split by what Alembic can load from where, not by accident: the

- logging sections and `databases` must stay in `alembic.ini`

- (`fileConfig()` needs an ini; `databases` is read via `get_main_option()`,

- which ignores `pyproject.toml`), while `prepend_sys_path` and

- `post_write_hooks` live in `[tool.alembic]`

+ - `DATABASE_NAMES` in the package names the two databases; alembic has no

+ `databases` option of its own to read. `script.py.mako` imports it to

+ emit one `upgrade_<name>()` / `downgrade_<name>()` pair per database, and

+ the chain lint builds both databases from it, while `env.py` iterates

+ whatever DBURIs `resolve_dburis()` hands it. There is no `alembic.ini`

+ at all:

+ `script_location`, `prepend_sys_path` and the `ruff` `post_write_hooks`

+ entry live in `[tool.alembic]` in `pyproject.toml`, and

+ `configure_logging()` skips `fileConfig()` unless the file alembic names

+ exists

+ - Only `src/soliplex/alembic_migrations/versions/*` is exempt from the 100%

+ coverage bar. The package's `__init__.py` *and* `env.py` are held to it,

+ which is why the tests drive alembic for real against throwaway SQLite

+ files rather than mocking it

## Adding a New Tool

1. Create or modify a tool module in `src/soliplex/tools/`

2. Tool functions are async and accept `RunContext[AgentDependencies]`

3. If the tool needs configuration, add a `ToolConfig` subclass in

`config/tools.py`

4. Register it in `TOOL_CONFIG_CLASSES_BY_TOOL_NAME` (found in

`config/tools.py` and `config/meta.py`)

5. Reference the tool in room configuration under `agent.tools`

## Adding a New Room

1. Create `example/rooms/<room_id>/room_config.yaml`

2. Required fields: `id`, `name`, `description`, `agent`

3. Optionally add `prompt.txt` for an external system prompt

## Adding API Endpoints

1. Create or modify a router in `src/soliplex/views/`

2. Register the router in `main.py` with the appropriate prefix

3. Add unit tests achieving 100% branch coverage

## Key Architecture

- FastAPI app created via `create_app()` in `main.py`

- Rooms contain agents, each with tools, skills, and an LLM provider config

- AG-UI protocol handles thread/run lifecycle with SSE event streaming

- Authorization via a policy engine in `authz/`

- MCP server exposes Soliplex tools; MCP client consumes external tool servers

- Authentication via OIDC/JWT in `authn.py`

- Public API models defined in `models.py`

## Key Dependencies

See `pyproject.toml` for authoritative version constraints.

- FastAPI / Uvicorn -- REST API and ASGI server

- pydantic-ai-slim[google] -- agent framework

- haiku.rag-slim -- RAG functionality

- FastMCP -- Model Context Protocol

- ag-ui-protocol -- AG-UI event protocol

- SQLModel / aiosqlite -- database ORM

- haiku-skills -- Haiku skills framework

## Entry Points

- `soliplex-cli` -- backend CLI; run `soliplex-cli --help` for the full

command list

- `soliplex-tui` -- terminal UI client

- `soliplex-tui-serve` -- TUI server

## Environment Variables

See `.env.example` for the full reference. Key variables:

- `OLLAMA_BASE_URL` -- Ollama server URL (without `/v1` suffix)

- `OPENAI_API_KEY` / `GEMINI_API_KEY` -- LLM provider keys

- `SOLIPLEX_URL_SAFE_TOKEN_SECRET` -- MCP token secret (auto-generated if

unset)

- `LOGFIRE_TOKEN` -- Pydantic Logfire token (optional)

- `SOLIPLEX_CLI_LOG_CONFIG` -- path to a Python logging-config YAML enabling

audit logging for privileged CLI commands (also the `--cli-log-config`

group option on `admin-users` / `room-authz` / `audit`); unset means CLI

audit records are suppressed (see `docs/config/logging.md`)

## Documentation

Detailed configuration and usage docs are in [docs/](docs/) (served via

Zensical). Example configurations are in [example/](example/).