test-microflows · git:20260922.4954f84 · 2026-09-22 · sha256 718d1593457f4b08
test-microflows git:20260922.4954f84A
Immutable. This exact content is served forever at /api/v1/blob/718d1593457f4b08.
---
name: test-microflows
description: "Write and run MDL-based microflow tests with `mxcli test` — annotations, file formats, and the warm local test loop. Use when testing microflow logic rather than the UI: return values, entity changes, control flow."
---
# Test Microflows Skill
This skill guides you through writing and running MDL-based microflow tests using `mxcli test`.
## Reference files
- [`reference/annotations.md`](reference/annotations.md) — every test annotation:
`@test`, `@setup`, `@teardown`, `@cleanup`, `@skip`, parameters and expected
values, with what each one does and the exact spelling. **Check an annotation
here rather than guessing** — an unknown one is a parse error, not a warning.
## When to Use This Skill
Use this when:
- The user asks to test microflow logic (not UI/pages)
- The user wants to verify that microflows return correct values
- The user wants to validate entity creation, updates, or control flow
- You have generated MDL microflows and want to verify they work at runtime
- The user asks for unit tests or integration tests on business logic
For **UI/page testing** (widget rendering, form interactions, browser tests), see the `test-app` skill instead.
## Prerequisites
- Mendix project with microflows to test
- A way to run the app — **either** of:
- `--local` (no Docker): mxcli boots the runtime itself, the same way
`mxcli run --local` does. This is the only option in a container without a
Docker daemon, which includes Claude Code web sessions.
- Docker: stack initialized (`mxcli docker init -p app.mpr`) and the app
buildable (`mxcli docker build -p app.mpr`).
```bash
mxcli test tests/ -p app.mpr --local # no daemon needed
mxcli test tests/ -p app.mpr # Docker
```
`--local` uses its own ports (app 8081, admin 8091) and its own `<project>_test`
database, so a `mxcli run --local` dev loop can keep serving the same project
while the tests run — the tests never write into the database you are looking at
in the browser. The database is created on first use.
The **deployment directory is shared**, and not by choice: mxbuild always writes
it to `<app dir>/deployment` and has no option to move it. That used to blank the
running app — a headless test boot does not bundle the web client, and the boot's
packaging pass deletes `deployment/web/dist`, so the app answered **HTTP 200 with
a blank page** (Mendix's SPA shell over a 404 for `/dist/index.js`) while the
tests passed and nothing was reported at either end. The bundle is now copied
aside before the boot and put back after, so the dev loop keeps the exact bundle
it built.
**The compiled Java is shared too, and that one is not fixable — only reportable.**
A test run recompiles the project into `deployment/run/bin`, the classpath the
running app's JVM is holding open. Measured: every class file is rewritten (new
inode, identical content). A JVM loads classes lazily, so one the app has not
reached yet can afterwards fail with `NoClassDefFoundError` — and the microflows
behind it then answer **HTTP 200 with an empty body** rather than an error, so the
app looks half-working. Only Java-backed resources are affected, which is why it
does not look like the test run did it.
mxcli warns when it sees a dev loop serving the same project (`run --local`
records itself in `.mxcli/run-local.json`, removed on exit). **If something the
app serves stops returning data after a test run, restart that app.** That is the
whole remedy.
One thing left that has not been measured: both runtimes share `deployment/data/`.
No damage observed; if you see something odd, run them one at a time and say so.
### Constants
A `--local` run boots the app with the **same constant values `mxcli run --local`
uses**: the project configuration's shared overrides, layered over each
constant's default. It prints what it applied before the run:
```
Applying 1 constant value(s):
MyModule.ApiKey configuration "Default"
```
Pass `--configuration <name>` to pick one when the project has several and none
is called `Default` (it refuses to guess rather than run production's values by
accident). `--attach` takes neither flag: it runs against an app someone else
booted and inherits **that app's** constants.
To set a value for one run without touching the project, use `--constant`
(repeatable). It wins over the configuration and is never written to the model:
```bash
mxcli test tests/ -p app.mpr --local --constant MyModule.ApiKey=sk-test-123
```
A name the project does not declare is **refused**, before anything boots — the
runtime silently ignores a value for a constant that does not exist, so a typo
would otherwise be reported as applied and do nothing.
The value is visible in shell history and in `ps`. That is fine for a throwaway
test value and wrong for a real secret.
This is worth knowing when a test asserts on something a constant feeds. Before
this was wired up, `--local` ran with each constant's *default* while `--attach`
ran with the configuration's, so the same suite could pass one way and fail the
other with nothing in the output to explain it.
For a secret that has to **persist** across runs, use the machine store:
```bash
mxcli constant set MyModule.ApiKey 'sk-live-...' -p app.mpr
mxcli constant list -p app.mpr # values from the store are masked
mxcli constant unset MyModule.ApiKey -p app.mpr
```
It writes `<project>/.mxcli/constants.json` (mode 0600), adds `.mxcli/` to the
project's `.gitignore` if missing, and then **asks git whether the path is
really ignored** — refusing to write the value if it is not. It beats the
configuration and loses to `--constant`.
By default the new value takes effect at the next boot. Add `--apply` to push it
into a `mxcli run --local` that is already up, without restarting it:
```bash
mxcli constant set MyModule.ApiKey 'sk-live-...' -p app.mpr --apply
```
That is two admin calls, not one: `update_configuration` is *staged* — the
running app keeps its old values and the call still answers success — and only
the following `reload_model` applies them. mxcli does both. It cannot confirm
the result, because the admin API has no way to read a constant back, so it says
so and points you at the app itself.
This is mxcli's own store, not Mendix's. Mendix's private configuration values
are encrypted per user account by Studio Pro from 10.9, so nothing headless can
read or write them. See `docs/11-proposals/PROPOSAL_constant_values.md`.
---
## Test File Formats
### `.test.mdl` — Pure MDL Tests
Test blocks separated by `/`, each with a javadoc comment containing test annotations:
```sql
/**
* @test String concatenation
* @expect $result = 'John Doe'
*/
$result = call microflow MyModule.ConcatNames(
FirstName = 'John', LastName = 'Doe'
);
/
/**
* @test Arithmetic operation
* @expect $result = 50
*/
$result = call microflow MyModule.Multiply(A = 10, B = 5);
/
```
**A file may open with a header comment**, in either spelling — a `/** … */`
block or `--` lines — and it does not become part of the first test. The test's
own doc comment is the last one above its statements. A `/** … */` header may
carry `@setup`, which then applies to every test in the file.
**One `@test` per block.** The `/` is what ends a test, so leaving it out puts
two tests in one block; that is refused by name rather than resolved, because
either way of resolving it runs one of the two and silently drops the other.
Both rules are #927: a file-level header used to swallow the first test whole —
it disappeared from the results with no error, and every later test reported
under the number of the one above it.
### `.test.md` — Markdown Specification
Tests embedded in documentation as `mdl-test` fenced code blocks:
~~~markdown
# MyModule Specification
## string Operations
The ConcatNames microflow joins first and last name.
```mdl-test
/** @expect $result = 'John Doe' */
$result = call microflow MyModule.ConcatNames(
FirstName = 'John', LastName = 'Doe'
);
```
~~~
The markdown format turns your tests into living documentation.
---
## Running Tests
```bash
# run tests from a file
mxcli test tests/microflows.test.mdl -p app.mpr
# run all tests in a directory
mxcli test tests/ -p app.mpr
# list tests without executing
mxcli test tests/ -p app.mpr --list
# Output JUnit xml for CI
mxcli test tests/ -p app.mpr --junit results.xml
# Skip build (reuse existing deployment)
mxcli test tests/ -p app.mpr --skip-build
# Verbose output (show all runtime logs)
mxcli test tests/ -p app.mpr --verbose
```
---
## How It Works
There are two mechanisms. `--local` uses the **test endpoint**; Docker uses the
older **after-startup microflow** pattern.
### `--local`: the test endpoint
1. Parses test files and extracts test blocks with annotations
2. Records the project's current after-startup microflow, and whether an `MxTest`
module already exists
3. Generates **one `MxTest.Test_<id>` microflow per test**, plus a Java action
that registers an HTTP endpoint, and points after-startup at a microflow that
registers it and then **chains your own after-startup microflow** —
**no test runs during startup**
4. Builds and boots the app once
5. Invokes each test by name over HTTP; each returns its own verdict in the
response
6. Restores the original after-startup setting and removes everything generated
7. Outputs results (console, JUnit XML)
Two consequences worth knowing when reading a failing run:
- **A test that throws fails only itself.** It is reported as `ERROR` with the
root-cause message, and the next test still runs. Under the after-startup
mechanism an uncaught error ends the whole flow — and because that flow *is*
the startup action, it also fails the boot.
- **Results are returned, not scraped**, so a test cannot be lost to log
buffering or a runtime that stopped echoing to the console.
Each test is a separate microflow with its own variable scope, so `$result` in
one test never collides with `$result` in another.
#### Your app's after-startup microflow still runs
The generated startup flow registers the endpoint and then calls the project's
own after-startup microflow, so tests see the app in the state it actually boots
into — a loaded cache, seeded reference data, whatever your app does. The run
says which happened:
```
After-startup set to MxTest.RegisterEndpoint (registers the endpoint; runs no tests, then runs your MyModule.ASU_Startup)
```
Pass `--skip-app-startup` when you want an empty, deterministic baseline
instead — the app seeds demo data and your tests assert on counts, say:
```
After-startup set to MxTest.RegisterEndpoint (… --skip-app-startup, so MyModule.ASU_Startup will NOT run)
```
This is why a suite behaves the same under `--local` and `--attach`. Before it
chained, `--local` ran with the app's startup logic suppressed, and a suite that
depended on startup state passed under `--attach` and failed under `--local` for
reasons unrelated to the code.
One thing rollback does **not** cover: whatever the startup microflow writes
happens at boot, outside any test's transaction, so `@cleanup rollback` does not
undo it. Under `--local` that lands in the scratch `<project>_test` database;
under `--attach` your app wrote it at its own boot regardless.
#### `--watch`: keep the runtime warm
```bash
mxcli test tests/ -p app.mpr --local --watch
```
The first run pays the cold boot; after that the runtime and the build server
stay up, and the suite re-runs on every change — to a test file **or** to the
project's model. Measured on an 11.13.0 app:
| | |
|---|---|
| First run (cold boot) | ~30s |
| Edit a test → verdict on screen | **~2s** |
| Edit a microflow → verdict on screen | **~2s** |
| The tests themselves | 20–70ms |
Editing a microflow and seeing straight away whether it still passes is the loop
this exists for. Ctrl-C stops watching and restores the project — the shutdown
prints `project restored` when it has.
Adding, editing and deleting tests all work mid-session: the suite is re-parsed
on every change, and a deleted test's microflow is dropped rather than left
behind reporting a stale pass.
`--watch` requires `--local`. The Docker and `--legacy-runner` paths can only
re-run tests by restarting, which is the thing being avoided.
#### `--attach`: no boot at all
If you already have the app running, tests can skip the boot entirely. The dev
loop has to opt into hosting the endpoint, because the handler is registered by
the after-startup microflow and so cannot be added to an app that is already up:
```bash
# terminal 1 — the app you are working in
mxcli run --local --test-endpoint -p app.mpr
# terminal 2 — runs in ~2s, no boot, repeatable
mxcli test tests/ -p app.mpr --attach
mxcli test tests/ -p app.mpr --attach --watch # ...and re-run on every change
```
The hosting app chains your project's own after-startup microflow rather than
displacing it, so it still boots normally. The endpoint and the handshake file
(`.mxcli/test-endpoint.json`, mode 0600) are removed when the app stops.
Three things to know before reaching for it:
- **Tests run against the running app's database**, not a scratch one, so they
can leave data behind in the app you are looking at. `--local` uses a separate
`<project>_test` database; `--attach` does not.
- **An attach only owns its own test microflows.** The endpoint and the
after-startup setting belong to the app hosting them, and cleanup never
touches them.
- **A change needing a runtime restart is refused** — a new entity or
association. That runtime belongs to the other process. Restart it, or drop
`--attach`.
| | Boot | Database | Owns the runtime |
|---|---|---|---|
| `--local` | ~30s each run | `<project>_test` | yes |
| `--local --watch` | ~30s once, then ~2s | `<project>_test` | yes |
| `--attach` | none | the running app's | no |
#### Security of the endpoint
It executes microflows under a system context, so it is gated four ways:
| Guard | Behaviour |
|---|---|
| No `MXCLI_TEST_TOKEN` in the runtime's environment | The handler is **not registered at all** (404) |
| Missing or wrong `X-MxTest-Token` header | 401, compared in constant time |
| Non-loopback caller | 403 |
| `mf` outside `MxTest.Test_*` | 403 — it is not a general microflow-invocation API |
The token is generated per run and reaches the runtime through its **environment**,
never written into the project. Combined with fail-closed registration, that means
a project which kept the `MxTest` module through a failed cleanup exposes nothing
when deployed anywhere else.
### Docker: the after-startup microflow
1. Parses test files and extracts test blocks with annotations
2. Records the project's current after-startup microflow, and whether an `MxTest`
module already exists
3. Generates a single `MxTest.TestRunner` microflow containing every test, and
points after-startup at it
4. Builds the project and restarts the container
5. Captures structured `MXTEST:` log lines for pass/fail
6. Restores the original after-startup setting and removes the generated runner —
the whole `MxTest` module when the runner created it, otherwise just the
`TestRunner` microflow
7. Outputs results (console, JUnit XML)
### Both mechanisms
The project's **Security Level is not modified**. The after-startup microflow runs
in an administrative context and is not subject to it, and forcing it off breaks
projects whose published REST/OData services use custom authentication. If a
cleanup step fails the run reports an error, **names every generated document
still in the project and prints the `DROP` that removes it** — the project is
modified, so it must not read as a clean pass.
Cleanup removes **every** generated `MxTest.Test_*` microflow the project holds,
not only the ones this run created. The names are positional (`Test_test_1`,
`_2`, … from the test's index in its file) and every test file reuses them, so
keying cleanup on the current suite left a flow behind whenever a later run had
fewer tests than an earlier one — and a leftover that does not build fails
**every subsequent run of every test file**, with a message about the project
rather than about any test (mendixlabs/mxcli#1104).
## Check a test file before you run it
`mxcli check suite.test.mdl` works, and is much faster than a run. A test block
is a **microflow body**, and `check` renders it as the microflow it becomes, on
the file's own lines — so a diagnostic points at the statement you wrote.
That includes the semantic rules, which is where most of the value is: a test
whose body would not compile is reported here instead of failing the injection
with nothing but "the project cannot be deployed". An `@expect` or `@verify` that
cannot be evaluated is reported here too, as `MDL-TEST01`.
One rule to know about, because its symptom is confusing and its shape is common
in tests: `retrieve $x … limit 1` binds a **single object**, not a one-element
list, so `head($x)` is `CE0097` at build time and `MDL-RETRIEVE01` at check time.
Drop the `limit` to get a list, or use the variable as the object it is.
---
## Writing Good Tests
### Test a Single Behavior
Each test block should test one thing:
```sql
/**
* @test Discount applied for orders over 100
* @expect $result = 90.0
*/
$result = call microflow Sales.CalculateDiscount(OrderTotal = 100.0);
/
```
### Test Multiple Scenarios
Use separate blocks for different input values:
```sql
/**
* @test Negative value returns 'negative'
* @expect $result = 'negative'
*/
$result = call microflow MyModule.Classify(value = -5);
/
/**
* @test Zero returns 'zero'
* @expect $result = 'zero'
*/
$result = call microflow MyModule.Classify(value = 0);
/
/**
* @test Positive value returns 'positive'
* @expect $result = 'positive'
*/
$result = call microflow MyModule.Classify(value = 42);
/
```
### Test Entity Operations
Tests can create, modify, and verify entities:
```sql
/**
* @test Create and update product
* @expect $updated = true
*/
$product = call microflow Sales.CreateProduct(
Name = 'Widget', Code = 'W-001'
);
commit $product;
$updated = call microflow Sales.UpdateProduct(
Product = $product, NewName = 'Super Widget'
);
/
```
### Test Error Handling
Use `@throws` to verify that a microflow raises an error:
```sql
/**
* @test Invalid input throws validation error
* @throws 'Validation failed'
*/
call microflow Sales.ValidateOrder(Total = -1);
/
```
The message is matched as a **substring** of what Mendix raised, so name the part
you can predict — a real message often carries an activity name or an object id.
A failing `@throws` reports both sides:
```
FAIL Invalid input throws validation error (6ms, 1 assertion)
expected an error containing 'Validation failed',
actual: Could not find object of type 'Sales.Order'
```
Write `@throws` on its own when the message is not the thing under test:
```sql
/**
* @test Invalid input is rejected
* @throws
*/
call microflow Sales.ValidateOrder(Total = -1);
/
```
Anything else after the tag is an error, not a message: `@throws not found` is
refused rather than read as an unquoted one.
`@throws` and `@expect` cannot be combined. A `@throws` test compiles to a
different shape — the verdict starts as a failure and only the error handler
clears it — and the `@expect` checks are not emitted into it at all, so the
assertion was counted and never evaluated. There is nothing to assert on either
way: the body was expected not to produce a result. Assert on the error with
`@throws`, or on the result with `@expect`.
---
## Test File Organization
Recommended structure:
```
tests/
├── microflows.test.mdl # business logic tests
├── entities.test.mdl # entity CRUD tests
├── validation.test.mdl # validation tests
└── specs/
└── sales-module.test.md # Markdown specification
```
---
## Interpreting Failures
| Failure | Cause | Fix |
|---------|-------|-----|
| `Exception during execution` | Microflow threw a runtime error | Check BSON structure, entity references, attribute types |
| `Expected $result = 'X' but got 'Y'` | Wrong return value | Fix microflow logic |
| `Test was not executed` | Runtime crashed before reaching it | Check earlier test failures or runtime logs |
| `after startup microflow should return a boolean` | Generated runner has wrong return type | Report as bug in mxcli |
---
## CI Integration
Use `--junit` to produce JUnit XML for CI systems:
```bash
mxcli test tests/ -p app.mpr --junit test-results.xml
```
The JUnit XML works with GitHub Actions, Jenkins, Azure DevOps, GitLab CI, etc.
```yaml
# GitHub actions example
- name: run microflow tests
run: mxcli test tests/ -p app.mpr --junit test-results.xml
- name: publish test results
uses: dorny/test-reporter@v1
with:
name: microflow Tests
path: test-results.xml
reporter: java-junit
```
## Related Skills
- [test-app](../test-app/SKILL.md) — Playwright UI tests (pages, widgets, browser interactions)
- [write-microflows](../write-microflows/SKILL.md) — Microflow syntax reference
- [docker-workflow](../docker-workflow/SKILL.md) — Docker build and runtime workflow
- [verify-with-oql](../verify-with-oql/SKILL.md) — OQL queries for data verification
## Warm test loop (`mxcli test --local [--watch]`, `--attach`, `run --local --test-endpoint`)
local test runs go through a **token-guarded HTTP endpoint** registered by a generated Java custom request handler, instead of compiling the suite into the project's after-startup microflow. Boot registers the endpoint and then **chains the project's own after-startup microflow**, so tests see the app as it really boots (`--skip-app-startup` opts out) — without that, a suite depending on startup state passed under `--attach` and failed under `--local`. One microflow per test, resolved by name at request time from `Core.getMicroflowNames()` and invoked with `Core.microflowCall(...).execute(...)` — so a throwing test fails only itself (not the boot), results are returned rather than scraped from the runtime log, and each test has its own variable scope. Owning the `IContext` is also what finally makes **`@cleanup rollback`** (the annotation's documented default, previously parsed and ignored) real: the handler wraps the call in `startTransaction()`/`rollbackTransaction()`, so a test's writes do not survive it — verified against Postgres, with `@cleanup none` as the in-run control. A rollback that fails is reported per test and summarised, never silent; an unknown strategy is a parse error. The handler **survives `reload_model`** (after-startup does not re-run, the JVM is unchanged), which is what makes `--watch` possible: ~30s first run, then ~2s from an edit — to a test *or* to the microflow under test — to a verdict. `--attach` skips even that boot by running against an app already up under `run --local --test-endpoint`, driving that process's serve + admin APIs over loopback; it uses **that app's database**, only ever adds/removes its own test microflows, and refuses a change needing a restart. Security: the handler is **not registered at all** without `MXCLI_TEST_TOKEN` in the runtime env (so a project that kept the `MxTest` module through a failed cleanup is inert in production), the token is constant-time compared, non-loopback callers are refused, `/list` is clamped to the test namespace, and only `MxTest.Test_*` may be invoked. The token reaches the runtime via its environment and is never written into the project. A `.test.mdl` file is **checkable**: each block is a microflow body, so `mxcli check` (and the LSP, hence VS Code) renders the blocks as the microflows they become, on the file's own lines — before #1103 the top-level grammar was applied instead, and since `RETRIEVE` is a non-reserved keyword the leftover `FROM …` started an OQL query, so the reader was told their retrieve needed a SELECT; 9 of this repo's 10 test files reported errors that way, one of them 392. `make check-mdl` now sweeps them, with `.fail.test.mdl` for a file whose annotations are deliberately unusable. Two things a failed run must not do, both reported as #1104: **a rejected build is reported with MxBuild's own errors** — `BuildResult.ErrorSummary()` was in hand and discarded by `fmt.Errorf("build failed: %s", build.Message)` on the `--attach` and rebuild paths, and that sentence is identical for every failing build, so it could not tell "your test does not compile" from "an unrelated document is broken"; and **cleanup removes every generated `MxTest.Test_*` the project holds**, not just this suite's. The names are positional and every file reuses them, so keying cleanup on the suite left a flow behind whenever a later run had fewer tests than an earlier one — and one leftover that does not build fails every later run of every test file. What cleanup could not remove is named, with the `DROP` that removes it. Docker keeps the after-startup runner (`--legacy-runner` selects it locally). Packages: `cmd/mxcli/testrunner/` (`endpoint.go`, `client.go`, `watch.go`, `host.go`, `handshake.go`, `check_source.go`, `cleanup_leftovers.go`). See `docs/15-testing/SPIKE_test_endpoint_request_handler.md`