AGENTS.md@web/src/features/evals · git:20260822.a7fa476 · 2026-08-22 · sha256 2e36d8ab7f49ed6d

AGENTS.md@web/src/features/evals git:20260822.a7fa476A

Immutable. This exact content never changes and is served at /api/v1/blob/2e36d8ab7f49ed6d.

# Evaluations

- LLM as a judge needs mapping of data to prompt variables - done by the user
- the code evaluators write a hardcoded mapping to the database - handled by the server. The actual data is mapped by the user in code
- Observation filters
  - During the evaluator setup users can set all kind of observation filters to select a sample
  - Filters used in rules are limited to what the `InMemoryFilterService` can process
- JobConfigurationID (legacy) and Rule ID (new) are the same (see eval migration)