AGENTS.md ยท diff
git:20260221.a5e90db to git:20260612.9439c6c
13 added, 0 removed. Audit A to A.
# Sidemantic
Universal SQL-first semantic layer in Python. Imports from Cube/LookML/dbt/Hex/Rill/Malloy, queries against DuckDB/Postgres/BigQuery/Snowflake/etc.
## Usage Preference (Important)
- Sidemantic usage is overwhelmingly CLI-first in practice (roughly 95% CLI usage).
- Default to `sidemantic` CLI workflows (`validate`, `info`, `query`, `migrator`, etc.).
- Treat Python API examples as secondary, advanced automation paths, not the default onboarding path.
## Codebase Structure
**Main Python package:** `sidemantic/`
- `core/` - SemanticLayer, Model, Dimension, Metric, SemanticGraph
- `adapters/` - Format parsers (Cube, LookML, MetricFlow, Hex, Rill, Malloy, etc.)
- `db/` - Database adapters
- `sql/` - SQL generation and query rewriting
- `widget/`, `server/`, `workbench/`, `mcp_server.py` - Optional features with lazy imports
**Separate experimental implementations (not the main codebase):**
- `sidemantic-rs/` - Rust rewrite (WIP)
- `sidemantic-duckdb/` - DuckDB extension wrapping sidemantic-rs
These don't share code with Python. CI runs them only when their dirs change.
## Version Management
Update BOTH when releasing:
- `pyproject.toml`: `version = "X.Y.Z"`
- `sidemantic/__init__.py`: `__version__ = "X.Y.Z"`
## CRITICAL: Before Every Commit
**ALWAYS run the EXACT same commands CI runs before committing:**
```bash
# Run these in order:
+ uv sync --extra dev
+ uv run python - <<'PY'
+ import sys
+
+ from sidemantic import Dimension, Metric, Model
+
+ assert Model.__name__ == "Model"
+ assert Dimension.__name__ == "Dimension"
+ assert Metric.__name__ == "Metric"
+ assert "sidemantic_dax" not in sys.modules
+ PY
+ uv sync --extra dev --extra dax
uv run ruff check . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
uv run ruff format --check . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
+ uv run python -m playwright install --with-deps chromium
uv run pytest -v
```
If any fail, fix them:
```bash
# Fix ruff issues
uv run ruff check --fix . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
uv run ruff format . --exclude docs/_extensions --exclude sidemantic-duckdb/extension-ci-tools --exclude sidemantic-duckdb/scripts --exclude sidemantic-duckdb/duckdb --exclude sidemantic/adapters/malloy_grammar --exclude sidemantic/adapters/holistics_grammar
```
This is NON-NEGOTIABLE. You MUST run these BEFORE every commit.
**PRs:** Do not include test commands in PR bodies unless explicitly requested.
**Why this matters:**
- CI runs these exact commands and will fail if they don't pass
- You keep pushing broken code because you don't run these locally
- Ruff must be in `[project.optional-dependencies] dev` for CI
- NOT in `[dependency-groups]` (that's uv-specific, CI uses optional-dependencies)
## Dependency Management
- Use `uv` for all Python package management
- Ruff should be in dev dependencies (`[dependency-groups] dev`)
- DO NOT add dev tools to main dependencies unless explicitly requested
- Optional features use `[project.optional-dependencies]`:
- `workbench` - textual, plotext, textual-plotext (for TUI, NOT Pyodide compatible)
- `serve` - mcp[cli], riffq, pyarrow (for PostgreSQL server, NOT Pyodide compatible)
- `dev` - pytest, ruff, pandas, numpy (for development)
**CRITICAL: Pyodide Compatibility Rules**
- **Core dependencies** (in main `dependencies` list) MUST work in Pyodide/WASM
- **Optional dependencies** can use packages incompatible with Pyodide (textual, riffq, pyarrow, mcp)
- **ALL imports** of optional deps MUST be lazy (inside functions, NOT at module level)
- **Test**: `from sidemantic import Model, Dimension, Metric` must work without any optional deps
## Pyodide Compatibility
**Import Structure:**
- `sidemantic/__init__.py` - Only imports core classes (Model, Dimension, etc.)
- `sidemantic/cli.py` - Imports typer (core dep) at top, workbench/server imports inside command functions
- `sidemantic/workbench/__init__.py` - Lazy imports textual inside `run_workbench()` function
- `sidemantic/server/` - Never imported unless `sidemantic serve` command is run
**Pyodide typing-extensions issue:**
- Pyodide has typing-extensions==4.11.0
- Some deps (pydantic>=2.10, inflect>=7.2) require typing-extensions>=4.12+
- Dashboard handles this by installing pydantic<2.10 with deps=False
- inflect<7.2 constraint in core deps marked with `# PYODIDE:` comment
**Version constraints with "PYODIDE:" comments are REQUIRED:**
- Heavy deps (textual, riffq, mcp) are optional to avoid Pyodide conflicts
- Pyodide CI builds local wheel and installs it with `deps=False`
- This ensures code changes are tested in Pyodide before publish
- If adding new core deps, check they work in Pyodide or make them optional
**If Pyodide CI fails:**
- Check if a dep version changed that requires newer typing-extensions
- Either downgrade that dep OR add workaround in dashboard.qmd install
- CI installs local wheel to test current code, not PyPI version
## Testing
Run tests before committing significant changes:
```bash
uv run pytest -v
```
## Publishing to PyPI
```bash
gh workflow run publish.yml
```
## Common Mistakes to Avoid
1. **Breaking ruff** - Always ensure ruff is installed in dev dependencies
2. **Pyodide compatibility** - Keep heavy deps (textual, pygments) optional
3. **Not linting** - Format and lint EVERY TIME before commit