WASP · diff
git:20261006.6aed70e to git:20261007.44a17f3
296 added, 389 removed. Audit C to C.
> **MANDATORY EXECUTION POLICY** — READ BEFORE PROCEEDING
>
> You MUST run the **actual model binary or package** described in this document.
> If the model fails to import, compile, or execute, you MUST:
> 1. Check `diagnostics/triplets.yaml` for a matching error pattern
> 2. Try the fix suggested in the triplet's `remedy` section
> 3. If still failing, report the error to the user with full details
>
> You MUST NOT substitute a simplified Python formula, regression equation,
- > or hand-coded approximation in place of the real model.
- >
- > **WASP-specific (2026-10-06):** this KI has TWO run paths. The REAL engine is US EPA
- > WASP 8.5.0 (`waspccli.exe` under WINE), run with `tools/run_wasp_engine.py`. The older
- > `tools/run_wasp.py` is an analytic Python SURROGATE ("WASP-inspired"), NOT EPA WASP;
- > its numbers are not WASP results. When a task asks for WASP, use the real engine. If the
- > real engine is missing, report that — never run the surrogate in its place. Use the
- > surrogate only when the task explicitly asks for it, and label its results "surrogate".
+ > or hand-coded approximation in place of the real model. Doing so produces
+ > scientifically invalid results and defeats the purpose of the KI.
>
+ > **WASP default route (2026-10-07):** the model is US EPA WASP 8.5.0 (`waspccli.exe`) run under
+ > WINE. Build cases with `tools/build_wasp_lake_case.py` (or `tools/wasp_wif_api.py`), run them with
+ > `tools/run_wasp_engine.py`, read them with `tools/parse_wasp_engine_output.py`. The older
+ > `tools/run_wasp.py`, `convert_forcing_to_wasp.py`, `convert_parameters_to_wasp.py` and
+ > `parse_output_wasp.py` are an analytic Python **SURROGATE**, not EPA WASP. Never run the surrogate
+ > in place of the engine; if the engine is missing, report that. Use the surrogate only when a task
+ > explicitly asks for it and label its numbers "surrogate".
>
> Before starting, run: `python preflight_check.py` (in this KI directory)
> to verify that the model binary/package and required data are available.
>
- > **DEBUGGING PROTOCOL** — When something goes wrong, follow this order:
+ > **DEBUGGING PROTOCOL** — When something goes wrong (model crashes, wrong output,
+ > unexpected values), follow this order. Do NOT skip steps or write debug scripts:
> 1. **Check triplets** — `diagnostics/triplets.yaml` may already cover this error
- > 2. **Read official docs** — The model's own documentation for expected formats/units
- > 3. **Find working examples** — Check `outputs/` or the model's shipped test data
- > 4. **Fix the tool** — With knowledge of what "correct" looks like
+ > 2. **Read official docs** — Check the model's own documentation (PDF manual, README,
+ > official examples) for expected input formats, variable names, and units
+ > 3. **Find working examples** — Look in `outputs/` for previous successful runs of
+ > this model, or check if the model ships with test/example data
+ > 4. **Fix the tool** — Now that you know what "correct" looks like, make targeted fixes
>
- > Do NOT write custom debug scripts. The answers are in the docs and examples.
+ > Resist the urge to write diagnostic/debug Python scripts. The answers are
+ > almost always in the official docs and working examples, not in reverse-engineering the binary.
<!-- KI-MAP:BEGIN (projected by generate_skill_map.py — edit the KI, not this table) -->
## KI map — what to read, and when
| when you need | read | why |
|---|---|---|
| FIRST, always | `preflight_check.py` | run it (`python preflight_check.py`): proves env/binary/data are usable and emits a machine-readable `PREFLIGHT_REPORT=` line. Do not debug a run that never had a healthy environment. |
- | to run the pipeline stages | `tools/` (5 tools) | the executable pipeline. Read each tool's argparse (`--help`) before composing a command; SKILL.md's stage table says which tool serves which stage. |
+ | to run the pipeline stages | `tools/` (10 tools) | the executable pipeline. Read each tool's argparse (`--help`) before composing a command; SKILL.md's stage table says which tool serves which stage. |
| before running a stage | `docs/s*_*.md` (6 stage docs) | per-stage procedure, verification and traps — the how-to that SKILL.md's overview compresses. |
- | on ANY error, before debugging | `diagnostics/triplets.yaml` (26 entries) | symptom → diagnosis → remedy for this model's known failure modes. Check here FIRST; the answer usually exists. Never renumber or rewrite entries. |
+ | on ANY error, before debugging | `diagnostics/triplets.yaml` (43 entries) | symptom → diagnosis → remedy for this model's known failure modes. Check here FIRST; the answer usually exists. Never renumber or rewrite entries. |
| to know what an output IS | `dag.yaml` | the model's identity: every output's medium, units, `validation_rank` (1 = the headline variable) and observability. Scoring and obs-binding read THIS — when asked 'what does this model predict', the dag is the answer, not a guess. |
| when building inputs / parsing outputs | `docs/format_spec.yaml` | exact I/O shapes + `known_issues`, projected from dag + triplets. Regenerate with `ki_tools_common/generate_format_spec.py` after changing either — never hand-edit. |
| to judge a run's skill | `docs/validation_convention.yaml` | how this model's field judges it validated: per-`dag_variable` metrics, directions and CITED pass-bands. A run is graded against these, not against intuition. |
| for claims and thresholds | `docs/gathered_papers.json` (16 papers) + `docs/papers_index.md` | the literature this KI is judged by; each entry's `text_path` is fetched full text in the central paper cache. `role: benchmark` marks the model's own skill paper. |
| for a machine-readable summary | `knowledge_infrastructure.yaml` | the manifest (package, pipeline, validation tier, counts) — projected by `ki_tools_common/generate_ki_manifest.py`; regenerate after structural changes, never hand-edit. |
+ | what past runs learned | `.kdt_evolution.jsonl` | append-only memory of previous runs and fixes on this KI. |
- *Projected 2026-08-17 from the KI's actual contents — 9 components present. Refresh: `python3 ki_tools_common/generate_skill_map.py --ki_dir <this KI>`.*
+ *Projected 2026-10-07 from the KI's actual contents — 10 components present. Refresh: `python3 ki_tools_common/generate_skill_map.py --ki_dir <this KI>`.*
<!-- KI-MAP:END -->
<!-- KI-TOOL-INDEX:BEGIN (projected by generate_skill_map.py — the discoverability contract: every public tool, exact path; PURPOSE stays human-authored elsewhere) -->
### Executable tool index (projected — complete by construction)
Every public tool in this KI, by exact path. What each is FOR lives in the
human-written Tool Inventory above; `--help` on any of these prints its arguments.
| tool (exact path) | invocation |
|---|---|
+ | `tools/build_wasp_lake_case.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/build_wasp_lake_case.py --help` |
+ | `tools/build_wasp_weather_from_source.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/build_wasp_weather_from_source.py --help` |
| `tools/convert_forcing_to_wasp.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/convert_forcing_to_wasp.py --help` |
| `tools/convert_parameters_to_wasp.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/convert_parameters_to_wasp.py --help` |
| `tools/parse_output_wasp.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/parse_output_wasp.py --help` |
+ | `tools/parse_wasp_engine_output.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/parse_wasp_engine_output.py --help` |
+ | `tools/prepare_wqp_lake_obs.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/prepare_wqp_lake_obs.py --help` |
| `tools/run_wasp.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/run_wasp.py --help` |
| `tools/run_wasp_engine.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/run_wasp_engine.py --help` |
+ | `tools/wasp_wif_api.py` | `KISSPATH_PYTHON_ENV/bin/python {KI}/tools/wasp_wif_api.py --help` |
- *5 public tools; `_`-prefixed helpers and packaging files excluded.*
+ *10 public tools; `_`-prefixed helpers and packaging files excluded.*
<!-- KI-TOOL-INDEX:END -->
- # WASP Knowledge Infrastructure
-
- **Package**: hydrocraft-wasp v1.0.0
- **Model**: WASP (Water Analysis Simulation Program) -- real EPA WASP 8.5.0 engine (`tools/run_wasp_engine.py`) + analytic SURROGATE (`tools/run_wasp.py`, not EPA WASP)
- **Domain**: Lake and reservoir water quality modeling
- **Language**: real engine = EPA Windows binaries under WINE; surrogate = Python (analytic reimplementation of WASP core physics)
- **License**: Public domain (EPA)
- **Created**: 2026-03-30
- **Validation**: SURROGATE only — Lake Erie Central Basin, WQP observations (Temp R=0.885, DO Cal R=0.816); the real EPA engine has only its official repeat-run test case (`test_cases/steady_state/`)
+ # WASP -- Knowledge Infrastructure Skill Document
- | Metric | Value |
- |--------|-------|
- | Tools | 5 |
- | Pipeline stages | 6 |
- | Diagnostic triplets | 26 |
- | Validation lake | Lake Erie Central Basin |
+ > **Version**: EPA WASP 8.5.0 (real engine) + legacy analytic surrogate
+ > **Domain**: surface-water quality (rivers, lakes, reservoirs, estuaries)
+ > **Last updated**: 2026-10-07
+ > **Validation status**: official EPA example reproduced exactly; one real case (Lake Erie Central
+ > Basin, validation - nothing calibrated): water temperature very good (NSE 0.99), DO (all data)
+ > below the bar (NSE −0.01) - the deck has no algae/nutrients (not configured yet)
---
- ## 0. Two run paths: REAL EPA WASP engine vs SURROGATE
-
- | | REAL engine | SURROGATE |
- |---|---|---|
- | tool | `tools/run_wasp_engine.py` | `tools/run_wasp.py` (+ `convert_*_to_wasp.py`, `parse_output_wasp.py`) |
- | what runs | US EPA WASP 8.5.0 `C:\WASP8\wasp\bin\waspccli.exe` under WINE 9 (official installer `wasp-version-8.5.0-install-64-bit-10-25-2025.exe`, sha256 `2bb410b59ead76d5ba3681f05f61c4b1193a1be8b5393ab7a33f234957b293d4`; banner "Wasp suite version 8.5.0 … Model master e86fb43") | analytic Python formulas "inspired by" WASP (sections 1, 7, 8 below) |
- | input | a WASP input file `.wif` (built in the WASP GUI or taken from an EPA example) | JSON parameters / WQP CSV |
- | output | `<model>.OUT` (setup echo + run statistics), `<model>.BMD2` (results, binary), CSV via EPA `BMD2_Extract.exe` | JSON / CSV of the analytic model |
- | results are | WASP results | NOT WASP results — always label them "surrogate" |
+ ## 1. Model Identity
- Everything below — "Data Preparation" and sections 1–10 (pipeline stages, converters,
- parser, parameters, Streeter-Phelps formulas, Lake Erie validation numbers) — describes the
- SURROGATE, except the subsection "Real engine" in section 2. The converters do not build
- `.wif` files and `parse_output_wasp.py` does not read `.BMD2`.
+ | Property | Value |
+ |----------|-------|
+ | Full name | Water Quality Analysis Simulation Program (WASP), US EPA |
+ | Version | 8.5.0 — banner "Wasp suite version 8.5.0, Gui develop af12325 dirty, Model master e86fb43" |
+ | Language | closed-source Windows binaries (Fortran/C++); no public source repository |
+ | License | US EPA public release (US Government work) |
+ | Installer | `wasp-version-8.5.0-install-64-bit-10-25-2025.exe` (epa.gov), sha256 `2bb410b59ead76d5ba3681f05f61c4b1193a1be8b5393ab7a33f234957b293d4` |
+ | Engine on this server | `C:\WASP8\wasp\bin\waspccli.exe` in WINE prefix `KISSPATH_HOME/engine_builds_20261006/wasp/wineprefix` (WINE 9.0); launcher in the models DB: `KISSPATH_HOME/engine_builds_20261006/wasp/run_wasp.sh` |
+ | Engine elsewhere (a copy of this KI) | every tool finds the prefix by `--wineprefix` → `$WASP_WINEPREFIX` → `$WASP_ENGINE_ROOT/wineprefix` → the server path above; wine by `--wine` → `$WASP_WINE` → PATH. Other paths: `$WASP_WQP_DIR` (WQP obs), `$KI_TOOLS_COMMON`, `$WASP_CMFD_DIR`, `$KI_PYTHON` (preflight interpreter), `$WASP_LAUNCHER` |
+ | Other EPA programs used | `wasptool.exe` (.wif data API), `BMD2_Extract.exe` (results to CSV) |
+ | Citation | US EPA ORD WASP8 documentation (Wool, Ambrose et al.) (https://www.epa.gov/hydrowq/wasp-model-documentation) |
+ | Primary domain | water quality: DO, CBOD, nutrients, algae, temperature, solids, toxicants |
+ | Spatial mode | network of well-mixed segments (0-D boxes linked by flows); the KI builds lake boxes |
- Real-engine call (python_env; engine found by `--wineprefix` → `$WASP_WINEPREFIX` →
- `KISSPATH_HOME/engine_builds_20261006/wasp/wineprefix`, wine by `--wine` → `$WASP_WINE` → PATH):
- ```bash
- KISSPATH_PYTHON_ENV/bin/python {KI}/tools/run_wasp_engine.py \
- --wif /path/to/model.wif --run-dir /scratch/wasp_run1 --extract-all
- # list the variables of a result file:
- KISSPATH_PYTHON_ENV/bin/python {KI}/tools/run_wasp_engine.py --list-variables /scratch/wasp_run1/model.BMD2
- ```
- The tool copies the `.wif` into a fresh run dir (`--copy-siblings` also copies the other files
- of its folder; sub-folders and absolute paths inside the `.wif` are not staged), runs the
- engine within one time budget (`--timeout`, default 900 s, max 1080 s, engine + extraction),
- and counts the run as good ONLY if the engine
- prints "run successfully closed out", writes `.OUT` + a valid `.BMD2`, and prints no ERROR
- lines other than "Failed to locate time function" (older example files read by 8.5 print
- these; they are counted in the summary, their effect on results is not checked). The wine
- exit code is not used (waspccli returns 2 even on good runs). Exit codes: 0 ok, 1 run or
- extraction failed, 2 bad command line, 3 WINE/engine missing (nothing run). Results:
- `<run-dir>/<model>_extract.csv` (Date_Time, Segment, one column per variable) and
- `<run-dir>/wasp_engine_summary.json`. Official test case: `test_cases/steady_state/`
- (EPA "Steady State" example river, 10 segments, DO/CBOD).
+ Build provenance: `KISSPATH_HOME/engine_builds_20261006/wasp/BUILD_LOG.md` (headless Qt installer
+ under WINE; no compilation possible — EPA ships binaries only).
---
- ## Data Preparation
-
- ### Forcing data
-
- **Data Sources**: Use `from ki_tools_common.load_forcing import load_daily_forcing` for CMFD/MSWX/NASA POWER.
-
- **Data Validation Reference**: See `data_ki/CMFD/SKILL.md` for meteorological forcing documentation.
- See `data_ki/WQP/SKILL.md` for water quality observations.
- See `data_ki/HydroLAKES/SKILL.md` for lake morphometry.
-
-
- ## 1. Overview
+ ## 2. What This Model Does
- WASP (Water Analysis Simulation Program) is a generalized water quality model
- developed by US EPA for simulating contaminant fate and transport in surface
- waters. This analytic reimplementation captures WASP's core water quality
- processes for lakes and reservoirs.
+ WASP solves mass balances for water-quality "systems" (DO, CBOD, nutrients, phytoplankton, solids,
+ temperature, toxicants…) in a network of segments, with transport set by one of several flow
+ options (stream routing, kinematic wave, flow routing, ponded weir, Lake 1D-vertical, dynamic) and
+ kinetics set by the chosen model library (here: Advanced Eutrophication, `multi-algae.dll`, with
+ the heat module). Water temperature comes from a full surface heat balance driven by weather
+ time functions; DO from reaeration toward a temperature- and pressure-dependent saturation minus
+ CBOD decay, SOD and (when configured) algal respiration plus photosynthesis.
- Core physics:
+ What the KI can do with the real engine today: run any EPA `.wif`; edit any value of a `.wif`
+ through EPA's own API; build a well-mixed lake-box case with real weather; score DO and water
+ temperature against WQP lake observations. Not yet: building segment networks for rivers,
+ layered lake decks, nutrient/algae systems, calibration loops on the engine.
- - **Streeter-Phelps BOD-DO dynamics**: Classic oxygen sag model coupling
- biochemical oxygen demand (BOD) decay with atmospheric reaeration.
- BOD decays first-order: `BOD(t) = BOD0 * exp(-kd * t)`. The oxygen
- deficit evolves as:
- ```
- D(t) = [kd*BOD0/(ka-kd)] * [exp(-kd*t) - exp(-ka*t)] + D0*exp(-ka*t)
- ```
- where `kd` is the BOD deoxygenation rate (1/d), `ka` is the reaeration
- rate (1/d), and `D0` is the initial oxygen deficit (mg/L).
+ ---
- - **Benson-Krause DO saturation**: Temperature-dependent dissolved oxygen
- saturation concentration (mg/L) following Benson & Krause (1984):
- ```
- ln(DO_sat) = -139.34411 + 1.575701e5/TK - 6.642308e7/TK^2
- + 1.2438e10/TK^3 - 8.621949e11/TK^4
- ```
- where TK is absolute temperature in Kelvin. This is the standard WASP
- formulation. At 20 C, DO_sat ~ 9.09 mg/L; at 0 C, ~14.6 mg/L.
+ ## 3. Input Requirements
- - **Logistic thermocline model**: 1-D vertical temperature profile using
- a logistic (sigmoid) function (CE-QUAL-W2 style):
- ```
- T(z) = T_bottom + (T_surface - T_bottom) / (1 + exp((z - z_thermo) / w))
- ```
- where `z` is depth (m), `z_thermo` is thermocline depth (m), and `w` is
- thermocline width (m). This captures the three-layer lake structure:
- epilimnion (warm mixed surface), metalimnion (thermocline), and
- hypolimnion (cold bottom).
+ **Exact shapes live in `docs/format_spec.yaml`** (projected from dag + triplets). The full
+ preparation plan, with every parameter, its API keyword and its source, is
+ `docs/input_preparation.md`.
- - **Carlson Trophic State Index (TSI)**: Lake trophic classification
- (Carlson 1977):
- ```
- TSI(Chl-a) = 9.81 * ln(Chl-a) + 30.6 [ug/L]
- TSI(Secchi) = 60 - 14.41 * ln(SD) [m]
- TSI(TP) = 14.42 * ln(TP) + 4.15 [ug/L]
- ```
- TSI < 30: oligotrophic, 30-50: mesotrophic, 50-70: eutrophic, >70: hypereutrophic.
+ WASP 8.5 has no text input: one binary `.wif` holds the whole case. The KI never writes a `.wif`
+ from scratch — it copies EPA's `SteadyState.wif` (`test_cases/steady_state/inputs/`, sha256
+ `41a3178f…`) and changes listed values through `wasptool.exe`.
- - **Sinusoidal seasonal temperature model**: Surface water temperature
- varies seasonally following:
- ```
- T(doy) = T_mean + amplitude * sin(2*pi*(doy - phase) / 365)
- ```
- where `T_mean` is annual average temperature (deg C), `amplitude` is
- the seasonal swing (deg C), and `phase` is the day of peak temperature
- offset (typically ~130 for Northern Hemisphere lakes).
+ ### 3.1 Meteorological Forcing
+ Daily weather time functions (days relative to the run start), from
+ `tools/build_wasp_weather_from_source.py` (`load_daily_forcing` → NASA POWER / CMFD / MSWX):
- - **Seasonal DO model**: Dissolved oxygen coupled to temperature via
- steady-state Streeter-Phelps:
- ```
- DO(doy) = clip(DO_sat(T(doy)) - kd*BOD0/(ka - kd) + DO_offset, 0, 20)
- ```
- This captures the inverse relationship between temperature and DO
- saturation, modulated by BOD loading.
+ | WASP time function (ISC) | Unit | Source |
+ |---|---|---|
+ | Solar Radiation - 1 (4) | W/m² daily mean | `srad_wm2` |
+ | Air Temperature Function 1 (17) | °C | `temp_mean_c` |
+ | Dew Point Function 1 (29) | °C | Magnus inverse of `shum_kgkg`, `pres_pa` |
+ | Wind Speed Function 1 (21) | m/s at 10 m | `wind_ms` |
+ | Cloud Cover Function 1 (25) | **0–1 fraction** | Kasten–Czeplak inverse of Rs / FAO-56 Rso |
- - **1-D DO profile**: Depth-dependent dissolved oxygen with epilimnion near
- saturation and hypolimnion depleted by sediment oxygen demand:
- ```
- DO(z) = DO_sat(T(z)) * depletion_factor(z, thermocline_depth, SOD_rate)
- ```
+ ### 3.2 Static Inputs
+ Segment depth and volume, elevation (constant 57 + segment parameter 24; DO saturation pressure),
+ latitude/longitude (constants 2/3), SOD (segment parameter 32, g O₂/m²/d), CBOD decay (constant
+ 47), initial concentrations per system. Data source of the Lake Erie case: depth 12 m from the
+ GLNPO August temperature profiles, elevation 174 m.
- Key characteristics:
- - Designed for lake and reservoir water quality analysis
- - Processes: temperature, dissolved oxygen, BOD, nutrients (Chl-a, TP)
- - Calibration against WQP (Water Quality Portal) lake observations
- - Supports multiple lakes (Erie, DeGray, Jordan, Mead) via WQP data
- - Vertical profiles via discrete profile data and NLA surveys
- - 3 + 4 calibration parameters (temperature model + DO model)
- - Output: seasonal T/DO time series, vertical T/DO profiles, TSI values
+ ### 3.3 Configuration Files
+ Only the `.wif`. Every value written must also have its **"used" flag = 1** or the engine
+ ignores it (dt_wasp_027). `build_wasp_lake_case.py` writes the full API command list next to the
+ case so every edit is auditable.
---
- ## 2. Installation
-
- ### Real engine (EPA WASP 8.5.0 under WINE)
-
- Installed on this server in the WINE prefix `KISSPATH_HOME/engine_builds_20261006/wasp/wineprefix`
- (`C:\WASP8`, headless install of the official EPA installer; no display needed). Needs `wine`
- (WINE 9.0, `/usr/bin/wine`). Run it only through `tools/run_wasp_engine.py` (section 0).
-
- ### Analytic reimplementation (Python) — SURROGATE only
+ ## 4. Build Instructions
+ Nothing to compile. The engine is installed (do not change the build):
```bash
- # No compilation needed -- pure Python
- pip install numpy pandas scipy matplotlib
+ export WINEPREFIX=KISSPATH_HOME/engine_builds_20261006/wasp/wineprefix WINEARCH=win64 WINEDEBUG=-all
+ unset DISPLAY
+ wineboot -i
+ wine wasp-version-8.5.0-install-64-bit-10-25-2025.exe --root 'C:\WASP8' --accept-licenses \
+ --accept-obligations --default-answer --confirm-command install
```
-
- ### Dependencies
-
- | Component | Required | Purpose |
- |--------------|----------|-------------------------------------------------|
- | numpy | Yes | Numerical computation |
- | pandas | Yes | Time series handling, WQP CSV parsing |
- | scipy | Yes | Calibration (Nelder-Mead optimization) |
- | matplotlib | Optional | Validation plots |
+ (epa.gov downloads work through proxy 127.0.0.1:7877.) Python side: `requirements.txt`
+ (numpy, pandas, scipy, matplotlib, requests, PyYAML) plus `ki_tools_common`. Set `KI_PYTHON` to
+ the interpreter that has them.
---
- ## 3. Pipeline
+ ## 5. Execution
- | # | Stage | Tool | Description |
- |---|------------------------|-------------------------------|----------------------------------------------------------|
- | 1 | Forcing preparation | `convert_forcing_to_wasp.py` | WQP/NLA CSV to cleaned T/DO/Chl-a time series |
- | 2 | Parameter setup | `convert_parameters_to_wasp.py`| Lake morphometry + BOD-DO kinetics + thermal params |
- | 3 | Model execution | `run_wasp.py` | Run seasonal T/DO model + profiles + TSI |
- | 4 | Output parsing | `parse_output_wasp.py` | Extract T/DO profiles, compute validation metrics |
- | 5 | Calibration | `run_wasp.py --mode calibrate`| Nelder-Mead against observed T and DO |
- | 6 | TSI analysis | `parse_output_wasp.py` | Carlson TSI computation and NLA percentile ranking |
+ ```bash
+ # official regression test (EPA Steady State example) — exit 0 = PASS
+ python test_cases/steady_state/run_reference.py
- **Parallelism**: Stages 1--2 can run in parallel. Stage 3 depends on 1--2.
- Stage 4 depends on 3.
+ # any .wif
+ python tools/run_wasp_engine.py --wif model.wif --run-dir run1 --extract-all
+ python tools/run_wasp_engine.py --list-variables run1/model.BMD2
- ### Stage skill documents
+ # the Lake Erie real case, tool by tool (weather -> case -> engine -> obs -> scores)
+ python tools/build_wasp_weather_from_source.py --source nasa_power --lat 41.95 --lon -81.55 \
+ --start 2004-01-01 --end 2014-12-31 --elev 174 --out weather_nasa_power.csv
+ python tools/build_wasp_lake_case.py --weather weather_nasa_power.csv --start 2004-01-01 \
+ --end 2014-12-31 --depth 12 --lat 41.95 --lon -81.55 --elev 174 --sod 0 --do0 13.0 --temp0 2.0 \
+ --out-dir case --name erie_cb_surface
+ python tools/run_wasp_engine.py --wif case/erie_cb_surface.wif --run-dir run \
+ --extract "Dissolved Oxygen" --extract "Water Temperature" --extract "Total CBOD" --segments 1
+ # then the two prepare_wqp_lake_obs.py calls and parse_wasp_engine_output.py of docs/s4_output_parsing.md
+ ```
+ On this server the same chain is wrapped by `KISSPATH_KI_ROOT/WASP/run_and_score.py`
+ (outside the KI; it calls these tools).
+ `run_wasp_engine.py` counts a run as good only with the close-out line, fresh `.OUT`/`.BMD2`, a
+ valid BMD2 header and no unexpected ERROR line; the wine exit code is ignored (it is 2 on good
+ runs). Exit codes: 0 ok, 1 failed, 2 bad command line, 3 WINE/engine missing.
- - [Stage 1: Forcing preparation](docs/s1_forcing_preparation.md)
- - [Stage 2: Parameter setup](docs/s2_parameter_setup.md)
- - [Stage 3: Model execution](docs/s3_model_execution.md)
- - [Stage 4: Output parsing](docs/s4_output_parsing.md)
- - [Stage 5: Calibration](docs/s5_calibration.md)
- - [Stage 6: TSI analysis](docs/s6_tsi_analysis.md)
+ **Expected runtime**: official example 3.4 s; Lake Erie 10-box case 28 s for 11 years
+ (about 2.6 s per simulated year); case build 8 s; whole runner 42 s (weather reused).
---
- ## 4. Output Description
-
- Source: `dag.yaml`. The dag is the authority for output identity; if this
- section ever disagrees with `dag.yaml`, the dag wins and this section should be
- fixed.
+ ## 6. Output Description
- **Headline output** (`validation_rank: 1`, the variable this model is judged by):
+ Restated from `dag.yaml` (the dag wins if they disagree).
- > `do_timeseries` -- Seasonal dissolved-oxygen time series (steady-state Streeter-Phelps). (mg/L)
+ **Headline output** (`validation_rank: 1`):
- Dag outputs restated from the KI:
+ > `do_timeseries` — Dissolved-oxygen concentration time series in lake/river surface water per
+ > segment (WASP 8.5), daily mean of the BMD2 variable 'Dissolved Oxygen'. (mg/L)
- | Output variable (dag `var`) | Rank | Unit | Description |
- |-----------------------------|------|------|-------------|
- | `do_timeseries` | 1 | mg/L | Seasonal dissolved-oxygen time series (steady-state Streeter-Phelps). |
- | `temperature_timeseries` | See `dag.yaml` | See `dag.yaml` | See `dag.yaml` |
- | `temperature_profile` | See `dag.yaml` | See `dag.yaml` | See `dag.yaml` |
- | `do_profile` | See `dag.yaml` | See `dag.yaml` | See `dag.yaml` |
- | `do_saturation` | See `dag.yaml` | See `dag.yaml` | See `dag.yaml` |
- | `TSI` | See `dag.yaml` | See `dag.yaml` | See `dag.yaml` |
+ | Output variable (dag `var`) | rank | File | Unit | Medium | Produced by the real-engine route? |
+ |---|---|---|---|---|---|
+ | `do_timeseries` | 1 | `<model>.BMD2` "Dissolved Oxygen" | mg/L | lake/river surface water | yes |
+ | `do_profile` | 2 | — | mg/L | lake water column | **no** — needs a layered deck |
+ | `do_saturation` | 3 | computed by `parse_wasp_engine_output.py` | mg/L | surface water | yes (diagnostic) |
+ | `temperature_timeseries` | 4 | `<model>.BMD2` "Water Temperature" | °C | lake/river surface water | yes |
+ | `temperature_profile` | 5 | — | °C | lake water column | **no** — needs a layered deck |
+ | `TSI` | 6 | — | – | lake water | **no** — needs phytoplankton/TP systems |
- The rank-1 output is the seasonal dissolved-oxygen time series,
- `do_timeseries`, in mg/L.
+ The template deck's BMD2 also holds Total CBOD, DO diurnal average/min/max, Volume, Described
+ Sediment Oxygen Demand and Flow Out of Segment.
---
- ## 4a. Unit Table
+ ## 7. Tool Inventory
- This unit table summarizes the model-facing units used by the pipeline. Exact
- I/O shapes live in `docs/format_spec.yaml`; `dag.yaml` is authoritative for
- output variables.
+ | Tool | Route | Purpose | Inputs | Outputs |
+ |---|---|---|---|---|
+ | `tools/run_wasp_engine.py` | REAL | run waspccli under WINE, judge success, extract BMD2 | `.wif` | run dir, extract CSV, summary JSON |
+ | `tools/wasp_wif_api.py` | REAL | get/put any `.wif` value through EPA's `wasptool.exe` | `.wif`, command list | edited `.wif`, JSON |
+ | `tools/build_wasp_weather_from_source.py` | REAL | weather time functions from the shared loader | lat/lon/dates/source | weather CSV |
+ | `tools/build_wasp_lake_case.py` | REAL | EPA template → lake-box case (hydraulics, site, kinetics, weather, used flags) | weather CSV, site values | `.wif`, commands, case JSON |
+ | `tools/prepare_wqp_lake_obs.py` | REAL (obs) | WQP lake obs: station typing, bbox, depth filter, whole-cruise DO sensor QC; one row per station-day with `value` (all data) and `value_qc` | WQP folder | station-day CSV + `<out>.cruise_qc.json` |
+ | `tools/parse_wasp_engine_output.py` | REAL | daily series, metrics (headline = all data `do_all`; QC subset `do_sensor_qc` reported next to it), DO-saturation diagnostic, figure | extract CSV, obs CSVs | CSVs, scores.json, PNG |
+ | `tools/run_wasp.py` | SURROGATE | analytic seasonal T/DO, profiles | JSON | JSON |
+ | `tools/convert_forcing_to_wasp.py` | SURROGATE | WQP/NLA → surrogate forcing JSON | CSVs | JSON |
+ | `tools/convert_parameters_to_wasp.py` | SURROGATE | lake presets → surrogate parameters | preset | JSON |
+ | `tools/parse_output_wasp.py` | SURROGATE | surrogate metrics, TSI | JSON | JSON/PNG |
- | Variable | Model unit | Source or context | Notes |
- |----------|------------|-------------------|-------|
- | `do_timeseries` | mg/L | `dag.yaml` rank-1 output | Seasonal dissolved-oxygen time series. |
- | Temperature | deg C | WQP / forcing / seasonal model | Convert Fahrenheit or Kelvin before use. |
- | Dissolved oxygen | mg/L | WQP observations and DO outputs | Values > 20 mg/L are suspect. |
- | Depth | m | WQP profile depth and lake morphometry | Convert ft or cm before use. |
- | Chlorophyll-a | ug/L | TSI input | Used for Carlson TSI. |
- | Total phosphorus | ug/L | TSI input | Used for Carlson TSI. |
- | Secchi depth | m | TSI input | Used for Carlson TSI. |
- | BOD decay rate `kd` | 1/d | DO seasonal model parameter | Convert hourly rates to daily rates. |
- | Reaeration rate `ka` | 1/d | DO seasonal model parameter | Convert hourly rates to daily rates. |
- | Lake area | km2 | Lake morphometry | Convert ha or m2 before use. |
- | Lake volume | m3 | Lake morphometry | Convert acre-ft before use. |
+ Stage docs: `docs/s1_forcing_preparation.md`, `s2_parameter_setup.md` (API keyword table),
+ `s3_model_execution.md`, `s4_output_parsing.md` (real engine); `s5_calibration.md`,
+ `s6_tsi_analysis.md` (surrogate only).
+ ### Shared Utilities (ki_tools_common)
+ `load_forcing.load_daily_forcing` (weather), `metrics.all_metrics` (keys `NSE`, `KGE`, `PBIAS`,
+ `RMSE`, `r`). `validation.validate_water_balance` does not apply: the lake box has no hydrology.
+
---
- ## 4b. Unit Trap Table
+ ## 8. Unit Conversion Table
- These unit conversions cause **silent failures** if wrong. The model performs
- no internal unit conversion; all inputs must be in the expected units.
+ | Quantity | Loader / obs unit | WASP unit | Conversion |
+ |---|---|---|---|
+ | shortwave | W/m² daily mean | W/m² | none (MJ/m²/d ÷ 0.0864 if another source) |
+ | air temperature | °C | °C | none |
+ | humidity | kg/kg + Pa | dew point °C | e = q·p/(0.622+0.378q); Td = 243.04·ln(e/611.2)/(17.625−ln(e/611.2)) |
+ | wind | m/s at 10 m | m/s | none (do not use `wind2_ms`) |
+ | cloud | — | fraction 0–1 | from Rs/Rso; tenths/percent are WRONG (dt_wasp_029) |
+ | SOD | g O₂/m²/d | g/m²/d (param 32) | none |
+ | loads | kg/d | kg/d | — |
+ | WQP temperature | °C or °F per record | °C | °F → (x−32)·5/9 (prepare_wqp_lake_obs.py) |
+ | WQP DO | mg/L or % | mg/L | % saturation records dropped, never mixed |
+ | GLNPO depth | text "PRFL-15" | m | parsed from ResultCommentText |
- | Variable | Model expects | Common source unit | Conversion | Trap ID |
- |-----------------------|----------------|---------------------|--------------------------------|---------|
- | Temperature | **deg C** | deg F (some WQP) | (F - 32) x 5/9 | dt_001 |
- | Temperature | **deg C** | K | - 273.15 | dt_002 |
- | Dissolved oxygen | **mg/L** | ug/L (some labs) | / 1000 | dt_003 |
- | Dissolved oxygen | **mg/L** | % saturation | x DO_sat(T) / 100 | dt_004 |
- | Depth | **m** | ft (some WQP) | x 0.3048 | dt_005 |
- | Depth | **m** | cm | / 100 | dt_006 |
- | Chlorophyll-a | **ug/L** | mg/L | x 1000 | dt_007 |
- | Total phosphorus | **ug/L** | mg/L | x 1000 | dt_008 |
- | Secchi depth | **m** | ft | x 0.3048 | dt_009 |
- | BOD decay rate kd | **1/d** | 1/h | x 24 | dt_010 |
- | Reaeration rate ka | **1/d** | 1/h | x 24 | dt_011 |
- | Lake area | **km2** | ha | / 100 | dt_012 |
- | Lake area | **km2** | m2 | / 1e6 | dt_013 |
- | Lake volume | **m3** | acre-ft | x 1233.48 | dt_014 |
+ ## 8b. CMFD/MSWX Data Conventions
+ CMFD is China-only (Lake Erie uses NASA POWER). MSWX is global; the shared loader handles both
+ through the same `--source` switch. NASA POWER is always fetched live (`REALTIME=1`).
- **Rule**: If DO values are > 20 mg/L, units are almost certainly ug/L not mg/L.
- If temperature values are > 50, units are likely Fahrenheit or Kelvin.
- If depth values are > 100 in a lake context, units may be feet or cm.
+ ## 8c. Sign Conventions and Output Units
+ BMD2 concentrations are positive mg/L; temperature °C; volume m³; flow m³/s out of the segment;
+ "Described Sediment Oxygen Demand" g/m²/d (positive = uptake). Extract timestamps are
+ `MM/DD/YYYY HH:MM` (local model time, no time zone).
---
- ## 5. Tools Reference
+ ## 9. Diagnostic Triplets (Top 5)
- | Tool | Stage | Script | Purpose |
- |----------------------------|------------|----------------------------------|-------------------------------------------------------|
- | `convert_forcing_to_wasp` | s1_forcing | `tools/convert_forcing_to_wasp.py` | WQP/NLA CSV to cleaned observation time series |
- | `convert_parameters_to_wasp`| s2_params | `tools/convert_parameters_to_wasp.py`| Lake morphometry + kinetic + thermal parameters |
- | `run_wasp` | s3_execute | `tools/run_wasp.py` | SURROGATE: execute analytic seasonal T/DO model + profiles + TSI (not EPA WASP) |
- | `run_wasp_engine` | real engine | `tools/run_wasp_engine.py` | REAL EPA WASP 8.5.0: run a `.wif` under WINE, check close-out, extract `.BMD2` to CSV |
- | `parse_output_wasp` | s4_output | `tools/parse_output_wasp.py` | Parse T/DO profiles, compute R/RMSE/bias, TSI |
+ Full corpus: `diagnostics/triplets.yaml` (43 entries; real-engine ones are dt_wasp_027–043).
- All tools follow the **validate -> process -> validate** pattern:
- 1. Parse CLI arguments with `argparse`
- 2. Validate all inputs (file existence, unit plausibility, range checks)
- 3. Process (convert, run, parse)
- 4. Validate outputs (physical plausibility, diagnostic warnings)
- 5. Return JSON: `{"status": "success/error", "output": {...}, "log": [...]}`
+ | id | symptom | remedy |
+ |---|---|---|
+ | dt_wasp_027 | value set through the API but the run ignores it | set the "used" flag (`PPARAMISUSED`/`PCONSTISUSED`/`PTFISUSED`) |
+ | dt_wasp_029 | water at 45–70 °C, or the engine never finishes | cloud cover as a 0–1 fraction |
+ | dt_wasp_035 | lake model scored against river samples | `prepare_wqp_lake_obs.py --site-types "Great Lake"` |
+ | dt_wasp_041 | whole GLNPO cruises with a dead (or suspect) DO sensor | cruise QC in `prepare_wqp_lake_obs.py`: only proven faults excluded; headline stays all data (`do_all`) |
+ | dt_wasp_042 | simulated DO stays at ~1.03 × saturation | deck has no algae/nutrients (not configured yet); report as a limit, do not tune |
+ | dt_wasp_040 | surrogate numbers quoted as WASP | use the real-engine route; label surrogate output |
---
- ## 6. Critical Domain Knowledge
-
- These non-obvious facts cause silent failures. Each links to a diagnostic triplet.
-
- 1. **dt_001 / dt_002**: WQP temperature data may be in Fahrenheit or Kelvin
- depending on the monitoring organization. Always check
- `ResultMeasure/MeasureUnitCode`. Values > 50 in a temperate lake are
- almost certainly Fahrenheit. Values > 200 are Kelvin.
-
- 2. **dt_003 / dt_004**: WQP dissolved oxygen is typically mg/L but some
- entries report percent saturation. At 20 C, 100% saturation = 9.09 mg/L.
- Using percent values directly gives DO of 80-110 "mg/L" which is absurd.
-
- 3. **dt_005 / dt_006**: WQP depth fields
- (`ActivityDepthHeightMeasure/MeasureValue`) vary in units. The most
- common unit is meters, but some stations report in feet. Check
- `ActivityDepthHeightMeasure/MeasureUnitCode`.
-
- 4. **Benson-Krause temperature sensitivity**: DO saturation drops from
- 14.6 mg/L at 0 C to 7.6 mg/L at 30 C. A 1 C error in temperature
- produces ~0.2 mg/L error in DO saturation. This is the dominant source
- of DO model error in thermally stratified lakes.
-
- 5. **Thermocline depth varies seasonally**: The logistic thermocline model
- uses a fixed thermocline depth, valid only during summer stratification
- (June-August in Northern Hemisphere). Applying summer profiles to spring
- or fall data produces large errors because the thermocline is developing
- or breaking down.
-
- 6. **Hypolimnetic oxygen depletion**: In productive lakes, the hypolimnion
- becomes hypoxic during summer stratification due to sediment oxygen
- demand (SOD) and decomposition. The DO profile model captures this via
- a depletion rate parameter, but the rate varies by lake productivity
- (oligotrophic: 0.01-0.03, eutrophic: 0.05-0.15 per meter below
- thermocline).
-
- 7. **Carlson TSI independence assumption**: The three TSI indices (Chl-a,
- Secchi, TP) are designed to agree for algae-dominated systems. Deviations
- indicate non-algal turbidity (TSI_Secchi >> TSI_Chl-a) or nutrient
- limitation (TSI_TP >> TSI_Chl-a).
+ ## 10. Coupling Interfaces
- 8. **WQP data quality flags**: WQP data includes a
- `ResultStatusIdentifier` field. Records marked "Rejected" or
- "Preliminary" should be excluded. Also filter
- `ResultValueTypeName != "Estimated"` for calibration data.
+ - Input side: any tool that writes the weather CSV schema of s1 can drive the heat module; flows
+ and boundary concentrations are API series (`PNOQFUNC`, `PBOUNDFUNC`) — a watershed model's
+ discharge and loads can be written the same way (not built yet).
+ - Output side: `<model>_extract.csv` / `sim_daily_seg<N>.csv` (daily means per variable).
+ - WASP itself also reads hydrodynamic linkage files (EFDC/HYD, flow option 3) — not used by the KI.
---
- ## 7. WASP Parameters
-
- ### Temperature Seasonal Model (3 parameters)
+ ## 11. Validated Results
- | Parameter | Symbol | Unit | Typical Range | Description |
- |-------------|-----------|---------|-----------------|--------------------------------------|
- | Mean temp | T_mean | deg C | [2, 30] | Annual average surface temperature |
- | Amplitude | amplitude | deg C | [3, 18] | Seasonal temperature swing |
- | Phase | phase | day | [90, 170] | Day-of-year of temperature peak |
+ ### 11.1 Official example (engine repeat test)
+ EPA Steady State example (`test_cases/steady_state/`), 2026-10-07 through `run_wasp_engine.py`:
+ closed out in 3.4 s, 10 segments, 9 variables, 355 output times, DO end seg 1 = 7.234781 mg/L,
+ seg 9 = 5.7412233, CBOD seg 1 = 105.90371, temperature seg 9 = 21.372213 °C — identical to the
+ 2026-10-06 build log run. **PASS (exact).** This is a repeat-run check, not a physical validation
+ (EPA ships no reference output).
- ### DO Seasonal Model (4 parameters, coupled to temperature)
+ ### 11.2 Real case: Lake Erie Central Basin surface layer (validation, nothing calibrated)
- | Parameter | Symbol | Unit | Typical Range | Description |
- |-------------|-----------|---------|-----------------|--------------------------------------|
- | BOD decay | kd | 1/d | [0.01, 0.5] | Deoxygenation rate coefficient |
- | Reaeration | ka | 1/d | [0.1, 2.0] | Atmospheric reaeration rate |
- | Initial BOD | BOD0 | mg/L | [0.5, 20] | Background BOD loading |
- | DO offset | DO_offset | mg/L | [-3, 3] | Systematic DO correction |
+ | Property | Value |
+ |----------|-------|
+ | Engine | EPA WASP 8.5.0 `waspccli.exe` under WINE through `tools/run_wasp_engine.py` (closed out, 0 unexpected ERROR lines) |
+ | Location | box at 41.95 N, 81.55 W; obs = 11 EPA GLNPO "Great Lake" stations (ER30, ER31, ER32, ER36, ER37, ER38, ER42, ER43, ER73, ER78M, ER95B) in bbox −82.5…−80.4 E, 41.35…42.6 N |
+ | Model setup | EPA Steady State deck edited through EPA's API: 10 identical isolated boxes, 12 m deep, Flow Routing; segment 1 scored; systems CBOD, DO, solids, water temperature only (no algae, no nutrients, SOD 0) |
+ | Period | 2004 spin-up; scored 2005–2014 |
+ | Forcing | NASA POWER daily (live), `build_wasp_weather_from_source.py` |
+ | Obs | WQP `wqp_lakes` Lake_Erie_Central, samples ≤ 5 m, one row per station-day (176 DO, 197 temperature rows; April and August cruises only) |
+ | Calibration | **none** — every value is an engine default, a literature value or read from the data (depth 12 m = August mixed-layer depth from GLNPO profiles). These are validation numbers. |
+ | Run | `KISSPATH_KI_ROOT/WASP/detached/real_case/c5fa2a53d2794a7c82a8b5ba157f79f0/` (re-scored 2026-10-07 with the corrected obs QC; a rebuild of the case through the current tools gave a byte-identical extract) |
- ### Thermal Profile (4 parameters)
+ ### Performance Metrics — judged against the field's bar, not intuition
- | Parameter | Symbol | Unit | Typical Range | Description |
- |----------------|-------------|-------|---------------|------------------------------------|
- | Surface temp | T_surface | deg C | [15, 30] | Epilimnion temperature (summer) |
- | Bottom temp | T_bottom | deg C | [4, 15] | Hypolimnion temperature |
- | Thermocline | thermo_depth| m | [3, 30] | Depth of maximum dT/dz |
- | Thermo width | thermo_width| m | [0.5, 10] | Thermocline transition thickness |
+ > Bar for `do_timeseries` and `temperature_timeseries` (NSE, per `arnold2012`, `efdc2021`,
+ > `deepavarsa2023`, `piccolroaz2013` in `docs/validation_convention.yaml`): satisfactory ≥ 0.5,
+ > good: no cited threshold, very good ≥ 0.75.
+ > Headline `do_timeseries`, all data (176 station-days): **NSE −0.01 → below the bar (not validated)**.
+ > Water temperature (197 station-days): **NSE 0.99 → very good**.
- ### Lake Morphometry
+ | Variable | Rows | NSE | KGE | PBIAS (%) | r | RMSE | Note |
+ |---|---|---|---|---|---|---|---|
+ | `do_timeseries`, all data (headline) | 176 | −0.011 | 0.605 | +20.0 | 0.671 | 2.91 mg/L | per-station median NSE −0.014 (11 stations) |
+ | `do_timeseries`, cruise-QC subset | 176 | −0.011 | 0.605 | +20.0 | 0.671 | 2.91 mg/L | same rows: the only excluded cruises (dead sensors) have no in-range values |
+ | `temperature_timeseries` | 197 | 0.992 | 0.953 | +2.8 | 0.997 | 0.96 °C | per-station median NSE 0.992 |
- | Parameter | Symbol | Unit | Description |
- |-------------|------------|--------|-----------------------------------------|
- | Max depth | z_max | m | Maximum lake depth |
- | Mean depth | z_mean | m | Volume / surface area |
- | Surface area| area | km2 | Lake surface area |
- | Volume | volume | m3 | Total lake volume |
+ The same run scored against the per-day network mean (`--network-mean`, support changes from day
+ to day) gives DO NSE −0.207 (n 30) and temperature 0.994 (n 33).
- ---
+ Cruise QC on this data (from the cruise's own data, never from the model; dt_wasp_041):
+ - EXCLUDED (sensor proven wrong): 2009-08 and 2010-04 (`sensor_dead`: every DO value negative;
+ the range filter already removes them, so no scored row changes).
+ - DIAGNOSTIC only, data kept: 2012-04 and 2013-04 (whole column and surface at 37–39 % of
+ saturation in a mixed April column), 2012-08 (19 %, stratified column) — `sat_offset_unconfirmed`;
+ 2005-04 and 2006-08 — `winkler_disagree_unmatched_depth` (CTD ≤ 5 m vs Winkler of unknown depth,
+ ratio 0.72 and 1.35). A sensor fault is likely for some of these, but low DO alone, or a Winkler
+ sample from an unknown depth, does not prove it, so they stay in the score.
+ - Sensitivity only (not a result): leaving the 5 diagnostic cruises out gives DO NSE 0.80,
+ r 0.94, PBIAS +7.9 % (n 124). It shows how much these cruises drive the DO score.
- ## 8. Streeter-Phelps BOD-DO Physics
+ Caveats, in plain words: the deck has **no algae and no nutrients** (not configured yet), so the
+ simulated DO stays at about 1.03 × saturation and cannot show algal supersaturation or depletion
+ (dt_wasp_042) — this explains much of the +20 % DO bias. Only April and August are observed, so
+ the seasonal contrast carries most of the score. The surrogate's older "Lake Erie" numbers (R =
+ 0.885 temperature, 0.816 DO) were scored mostly against tributary stream samples (dt_wasp_035),
+ are not WASP results and are not comparable.
- The Streeter-Phelps model describes the DO sag curve downstream of a
- pollution source (or temporally in a lake with BOD loading):
+ ### Data Replacement Tracking
- ```
- dBOD/dt = -kd * BOD (first-order BOD decay)
- dDO/dt = ka * (DO_sat - DO) - kd * BOD (reaeration - deoxygenation)
- ```
+ | Component | Source | Status | Notes |
+ |---|---|---|---|
+ | Forcing | NASA POWER via shared loader | Validated (ranges) | dew point and cloud derived |
+ | Soil | — | n/a | no soil in a lake box |
+ | Land cover | — | n/a | |
+ | DEM / elevation | 174 m (lake level) | Set | used flag checked in .OUT echo |
+ | Initial conditions | climatological (DO 13, T 2 °C on Jan 1) | Pending | one spin-up year |
+ | Morphometry | depth 12 m from GLNPO August profiles | Set | single layer |
- Analytical solution for oxygen deficit D = DO_sat - DO:
+ ---
- ```
- D(t) = [kd * BOD0 / (ka - kd)] * [exp(-kd*t) - exp(-ka*t)] + D0 * exp(-ka*t)
- ```
+ ## 12. Parameter Selection by Region
- Critical behavior:
- - If ka > kd: deficit peaks at t_c = ln(ka/kd) / (ka - kd), then recovers
- - If ka = kd: degenerate case, use L'Hopital limit
- - If ka < kd: reaeration dominates, rapid recovery from initial deficit
- - DO is clipped to [0, DO_sat] to prevent supersaturation or negative values
+ > Not calibration — starting points.
- Typical parameter values for lakes:
- - kd: 0.05-0.3 /d (lower for lakes than rivers due to settling)
- - ka: 0.1-1.0 /d (lower for deep lakes, higher for shallow/windy)
- - BOD0: 1-10 mg/L (varies with pollution loading)
+ | Situation | Key parameters | Rationale |
+ |---|---|---|
+ | Large temperate lake, surface layer | depth = summer mixed-layer depth (profiles), SOD 0, global Ka 0 (computed), CBOD 1 mg/L, decay 0.1/d | epilimnion has no sediment contact; wind reaeration dominates |
+ | Bottom layer / shallow lake | SOD 0.3–1.0 g O₂/m²/d (θ 1.04) | typical measured SOD range for Lake Erie sediments |
+ | High-altitude water body | elevation constant 57 + param 24 with used flags | DO × p/p0 (0.72 at 2700 m) |
+ | River reach | start from the EPA Steady State deck itself (kinematic wave, Ka 5/d) | the template is a river |
---
- ## 9. Validated Results (Lake Erie Central Basin) — SURROGATE model, not EPA WASP
-
- | Metric | Calibration | Validation |
- |-------------------------|-------------|------------|
- | Temperature R | 0.878 | 0.885 |
- | Temperature RMSE (C) | 4.31 | 4.10 |
- | Temperature n | 28,988 | 19,326 |
- | DO R | 0.816 | 0.413 |
- | DO RMSE (mg/L) | 2.14 | 3.77 |
- | DO n | 19,795 | 13,198 |
- | Thermal profile R | 0.996 | -- |
- | Thermal profile RMSE (C)| 0.62 | -- |
- | TSI (Chl-a) | 40.7 | -- |
- | NLA percentile | 22.3% | -- |
-
- Temperature model shows strong generalization (R improves from cal to val).
- DO model shows reduced generalization, typical for simple seasonal models
- that cannot capture interannual variability in loading and stratification.
- Thermal profile model achieves near-perfect fit (R=0.996) for summer conditions.
+ ## 13. Known Limitations
- ### Performance Metrics -- Convention Bars
+ - Lake-box case only: no stratification, no algae, no nutrients (not configured yet), so
+ simulated DO stays near saturation (~1.03 ×, dt_wasp_042); `do_profile`,
+ `temperature_profile` and `TSI` are not produced by the real-engine route yet.
+ - The shared `preflight_forcing.py` cannot identify a point NASA POWER CSV (dt_wasp_043); the
+ weather tool's own validation is the check.
+ - No calibration loop on the engine; `s5_calibration.md` is surrogate-only.
+ - Template segments cannot be removed; unused ones are isolated (tiny flows).
+ - 31 missing time functions in the 2017 template (deposition, benthic fluxes) — expected.
+ - The print interval edit has no effect on BMD2 output frequency (dt_wasp_039).
+ - `BMD2_Extract.exe` has no published manual; the control layout in `run_wasp_engine.py` was
+ found by testing.
- Source: `docs/validation_convention.yaml`. These bars are the KI's cited
- field convention for judging model skill. A null band in the convention is
- reported here as "no cited threshold".
+ ## 14. Spinup Requirements
- | Dag variable | Metric | Direction | Satisfactory band | Good band | Very good band |
- |--------------|--------|-----------|-------------------|-----------|----------------|
- | `temperature_timeseries` | nse | maximize | 0.5 (`arnold2012`, `efdc2021`, `piccolroaz2013`) | no cited threshold (`arnold2012`, `efdc2021`, `piccolroaz2013`) | 0.75 (`arnold2012`, `efdc2021`, `piccolroaz2013`) |
- | `do_timeseries` | nse | maximize | 0.5 (`arnold2012`, `efdc2021`, `deepavarsa2023`) | no cited threshold (`arnold2012`, `efdc2021`, `deepavarsa2023`) | 0.75 (`arnold2012`, `efdc2021`, `deepavarsa2023`) |
- | `do_timeseries` | r2 | maximize | 0.5 (`deepavarsa2023`, `lushui2021`) | 0.6 (`deepavarsa2023`, `lushui2021`) | 0.75 (`deepavarsa2023`, `lushui2021`) |
- | `temperature_profile` | pbias | zero_centered | 15 (`lushui2021`) | no cited threshold (`lushui2021`) | no cited threshold (`lushui2021`) |
+ One year (2004) before scoring. The 12 m box forgets its initial temperature within weeks and
+ its initial DO within days (wind reaeration); CBOD (1 mg/L) decays away in a few months because
+ the box has no loads.
- For the headline dag variable `do_timeseries`, the convention bars are:
+ ## 15. References
- - `nse`, maximize: satisfactory 0.5 (`arnold2012`, `efdc2021`, `deepavarsa2023`), good no cited threshold (`arnold2012`, `efdc2021`, `deepavarsa2023`), very good 0.75 (`arnold2012`, `efdc2021`, `deepavarsa2023`).
- - `r2`, maximize: satisfactory 0.5 (`deepavarsa2023`, `lushui2021`), good 0.6 (`deepavarsa2023`, `lushui2021`), very good 0.75 (`deepavarsa2023`, `lushui2021`).
+ - US EPA WASP model documentation: heat-model.pdf, stream-transport-user-guide.pdf,
+ wasp8_sod_module_v1.pdf, light-module.pdf (https://www.epa.gov/hydrowq/wasp-model-documentation)
+ - EPA WASP examples (https://www.epa.gov/hydrowq/wasp-model-examples) — Steady State example
+ - Literature behind the validation bands: `docs/gathered_papers.json`, `docs/papers_index.md`
+ - Alduchov & Eskridge (1996) Magnus formula; Allen et al. (1998) FAO-56 (Ra, Rso);
+ Kasten & Czeplak (1980) cloud–radiation relation; APHA (1992) DO saturation
---
- ## 10. Supported Lakes
-
- | Lake | WQP Directory | Key Variables | Profile Data |
- |-------------------|--------------------|----------------------|--------------|
- | Lake Erie Central | Lake_Erie_Central | Temp, DO, Chl-a | Yes |
- | DeGray Lake | DeGray_Lake | Temp, DO | Yes |
- | Jordan Lake | Jordan_Lake | Temp, DO, Chl-a | Yes |
- | Lake Mead | Lake_Mead | Temp, DO | Yes |
+ ## Appendix A. SURROGATE (legacy analytic stand-in — NOT EPA WASP)
- NLA 2017 survey data provides additional profile validation for deep lakes
- across the continental US.
+ `tools/run_wasp.py` and its three helpers implement a sinusoidal seasonal temperature model, a
+ steady-state Streeter–Phelps seasonal DO, a logistic thermocline profile and Carlson TSI. They
+ were the whole KI until 2026-10-06. Use them only on explicit request; label results
+ "surrogate". Their how-to is in `docs/s5_calibration.md`, `docs/s6_tsi_analysis.md` and the
+ archived stage docs in `docs/surrogate/` (its I/O contract, the KI's dag.yaml until 2026-10-07, is
+ `docs/surrogate/surrogate_wasp_analytic_dag.yaml`; its old scores are kept apart in
+ `knowledge_infrastructure.yaml` → `validation.surrogate_metrics_not_wasp`); their known traps are
+ dt_wasp_001 – dt_wasp_026. Example (surrogate, for smoke tests only):
+ ```bash
+ python tools/convert_parameters_to_wasp.py --lake-preset erie --output params.json
+ python tools/run_wasp.py --mode profile --params params.json --z-max 20 --z-step 1 --output profile.json
+ ```