CLAUDE.md@test · diff
git:20261031.ad87b87 to git:20261031.cead4b3
4 added, 0 removed. Audit A to A.
What this update changes in the checks
No check changed: every fixed check gives the same answer before and after this update.
A tripped check is a label, not a verdict: read the matched text and the lines below. This week's updates that newly trip a check.
# test/ — rules
Mirrors the `codenib/` package layout. Loads on top of the project-wide
[`.claude/CLAUDE.md`](../.claude/CLAUDE.md).
## Marker tiers
| Marker | Scope | Duration |
|--------|-------|----------|
| _(none)_ | Unit — pure logic, mocks only | ~1 min |
| `integration` | External repos, **read-only / parallel-safe**: chunkers, fixture-based SCIP | ~2 min |
| `integration_serial` | **Mutates shared repos** (SCIP indexing, `process_instance`, `git checkout`/`apply`) — must run sequentially | ~25 min |
| `integration_serial_consumer` | Consumes the `graph.pkl` written by `integration_serial` (runs in a separate downstream job) | ~5 min |
| `slow` | LLM API calls, GPU embeddings | ~15 min |
```bash
make test # unit only; safe local default
pytest -m "not slow and not integration and not integration_serial and not integration_serial_consumer" -x # unit only
pytest -m "integration and not slow" # parallel-safe integration
pytest -m integration_serial # serial (repo-mutating)
pytest -m integration_serial_consumer # graph.pkl consumers
make test-slow # slow only; may make billed calls
```
A new test defaults to the unit tier — only add a marker if it actually needs
the heavier deps, and pick the right tier: anything that *mutates* a shared
repo must be `integration_serial`, never plain `integration` — the
`integration` job runs parallel under pytest-xdist and a mutating test breaks
it. The CI jobs key off exactly these markers (`unit`, `integration`,
`integration-serial`, `graph-consumer`, `slow`; `scip-core` runs the C++
executables and SCIP/Fact/clangd Python gates through `make core-test`) — see
[`docs/ci_cd.md`](../docs/ci_cd.md) for the job chain.
## Fixtures & caches
- Repo fixtures cache clones under `${CODENIB_TEMP_DIR}/gt-test/`.
- HuggingFace dataset cache lives at `~/.codenib/`.
- Git fixtures that copy or swap `.git` must disable automatic maintenance
before committing, so transient maintenance files cannot race the snapshot.
+ - When executing captured Python source with `-I`, also pass `-B` and verify
+ `sys.flags.dont_write_bytecode`. Isolated mode ignores `PYTHON*` environment
+ settings; an environment-only bytecode setting can add files and invalidate
+ the source receipt. Retain failed source-preservation attempts separately.
- SCIP identity captures use filename-driven parametrization with a fixed
single-file shape. Put independently shaped multi-file captures in their own
fixture directory so existing globs do not collect them as compatible cases.
- CodeNib's managed loggers do not propagate to the root logger. Attach
`caplog.handler` to the actual logger and use `caplog.set_level` for the
expected severity when asserting its records. Console setup can change
capture-handler thresholds; attaching the handler alone still depends on
another test's logging setup.
## Gotchas
- Module-local `subprocess` test doubles must preserve the constants and
exceptions the exercised path reads, including `PIPE` and `TimeoutExpired`.
Reproduce against unchanged code before counting fixture failures as product
regressions.
- Keep regression watchdogs independent of runner timeouts. Unit CI uses
`pytest-timeout`, so its `SIGALRM` may already be armed.
- For a module-local no-I/O check, replace that module's `os` binding with a
guarded proxy. Do not replace process-wide `os.open`, `os.read`, or
`os.stat`: pytest-xdist control threads and reporting still need them, and
can exit the worker instead of producing a test failure.
- **Package shadowing**: a `test/<name>/__init__.py` whose `<name>` matches a
top-level package (e.g. `scripts/`) shadows that real package under pytest's
rootdir import mode. Two independently-green PRs can break once merged — when a
test dir mirrors a top-level package name, revalidate the *merged* tree, not
just per-PR CI.
- The `slow` job needs LLM API keys + a GPU; a red `slow` run is usually
pre-existing infra flake, not a regression in your change — don't block merges
on it without checking it failed for the same reason before your PR.