AGENTS.md git:20260823.0b2b664A

Immutable. This exact content is served forever at /api/v1/blob/68c453708c8c00e5.

# Sidemantic

Universal SQL-first semantic layer in Python. Imports from Cube/LookML/dbt/Hex/Rill/Malloy, queries against DuckDB/Postgres/BigQuery/Snowflake/etc.

## Usage Preference (Important)

- Sidemantic usage is overwhelmingly CLI-first in practice (roughly 95% CLI usage).
- Default to `sidemantic` CLI workflows (`validate`, `info`, `query`, `migrator`, etc.).
- Treat Python API examples as secondary, advanced automation paths, not the default onboarding path.

## Codebase Structure

**Main Python package:** `sidemantic/`
- `core/` - SemanticLayer, Model, Dimension, Metric, SemanticGraph
- `adapters/` - Format parsers (Cube, LookML, MetricFlow, Hex, Rill, Malloy, etc.)
- `db/` - Database adapters
- `sql/` - SQL generation and query rewriting
- `widget/`, `server/`, `workbench/`, `mcp_server.py` - Optional features with lazy imports

**Separate experimental implementations (not the main codebase):**
- `sidemantic-rs/` - Rust rewrite (WIP)
- `sidemantic-duckdb/` - DuckDB extension wrapping sidemantic-rs

These don't share code with Python. CI runs them only when their dirs change.

## Version Management

Update BOTH when releasing:
- `pyproject.toml`: `version = "X.Y.Z"`
- `sidemantic/__init__.py`: `__version__ = "X.Y.Z"`

## Validation Before Commits

Match validation scope to the change instead of running the entire suite automatically.

- For small, localized changes, run ruff on the changed Python files and the narrowest relevant tests. This is the default.
- Expand to the affected package or subsystem when shared code, planners, adapters, or fixtures may have broader impact.
- Run the full CI-equivalent sequence only when explicitly requested, preparing a release, changing dependencies/CI/tooling, making a broad cross-cutting refactor, or when targeted checks cannot bound the blast radius.
- Do not rerun the full suite for a small follow-up on a branch that already had a green full run when targeted tests cover the follow-up.
- Before handoff, report exactly what ran and what was left to CI.

When full CI parity is warranted, run these commands in order:

```bash
# Run these in order:
uv sync --extra dev
uv run python - <<'PY'
import sys

from sidemantic import Dimension, Metric, Model

assert Model.__name__ == "Model"
assert Dimension.__name__ == "Dimension"
assert Metric.__name__ == "Metric"
assert "sidemantic_dax" not in sys.modules
PY
uv sync --extra dev --extra dax
uv run ruff check . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
uv run ruff format --check . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
uv run python -m playwright install --with-deps chromium
uv run pytest -v
```

If any fail, fix them:
```bash
# Fix ruff issues
uv run ruff check --fix . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
uv run ruff format . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
```

**PRs:** Do not include test commands in PR bodies unless explicitly requested.

**Why this matters:**
- Targeted checks keep small changes fast while still covering the affected behavior
- CI runs the full matrix and remains the final integration gate
- Ruff must be in `[project.optional-dependencies] dev` for CI
- NOT in `[dependency-groups]` (that's uv-specific, CI uses optional-dependencies)

## Dependency Management

- Use `uv` for all Python package management
- Ruff should be in dev dependencies (`[dependency-groups] dev`)
- DO NOT add dev tools to main dependencies unless explicitly requested
- Optional features use `[project.optional-dependencies]`:
  - `workbench` - textual, plotext, textual-plotext (for TUI, NOT Pyodide compatible)
  - `serve` - mcp[cli], riffq, pyarrow (for PostgreSQL server, NOT Pyodide compatible)
  - `dev` - pytest, ruff, pandas, numpy (for development)

**CRITICAL: Pyodide Compatibility Rules**
- **Core dependencies** (in main `dependencies` list) MUST work in Pyodide/WASM
- **Optional dependencies** can use packages incompatible with Pyodide (textual, riffq, pyarrow, mcp)
- **ALL imports** of optional deps MUST be lazy (inside functions, NOT at module level)
- **Test**: `from sidemantic import Model, Dimension, Metric` must work without any optional deps

## Pyodide Compatibility

**Import Structure:**
- `sidemantic/__init__.py` - Only imports core classes (Model, Dimension, etc.)
- `sidemantic/cli.py` - Imports typer (core dep) at top, workbench/server imports inside command functions
- `sidemantic/workbench/__init__.py` - Lazy imports textual inside `run_workbench()` function
- `sidemantic/server/` - Never imported unless `sidemantic serve` command is run

**Pyodide typing-extensions issue:**
- Pyodide has typing-extensions==4.11.0
- Some deps (pydantic>=2.10, inflect>=7.2) require typing-extensions>=4.12+
- Dashboard handles this by installing pydantic<2.10 with deps=False
- inflect<7.2 constraint in core deps marked with `# PYODIDE:` comment

**Version constraints with "PYODIDE:" comments are REQUIRED:**
- Heavy deps (textual, riffq, mcp) are optional to avoid Pyodide conflicts
- Pyodide CI builds local wheel and installs it with `deps=False`
- This ensures code changes are tested in Pyodide before publish
- If adding new core deps, check they work in Pyodide or make them optional

**If Pyodide CI fails:**
- Check if a dep version changed that requires newer typing-extensions
- Either downgrade that dep OR add workaround in dashboard.qmd install
- CI installs local wheel to test current code, not PyPI version

## Testing

Use the validation scope above. Prefer focused test paths or test names for localized changes; reserve `uv run pytest -v` for cases that warrant full CI parity.

## Publishing to PyPI

```bash
gh workflow run publish.yml
```

## Common Mistakes to Avoid

1. **Breaking ruff** - Always ensure ruff is installed in dev dependencies
2. **Pyodide compatibility** - Keep heavy deps (textual, pygments) optional
3. **Not linting** - Format and lint EVERY TIME before commit