git:20261018.07f0d74 to git:20261020.0d73798

### Configuration System

`Paths.user_projects_dir()` uses `extended_length_path()` on native Windows.

All document paths, including staging and retention walks, inherit the same

extended drive/UNC namespace even when the root itself is short. Persisted

`stored_relpath` values and Docker mount paths retain their existing spelling.

`project_document_path()` still resolves symlinks and checks confinement using

the same namespace for both the user projects root and the document path.

Do not prefix only paths already exceeding MAX_PATH: appended filenames and

derived companions can cross the limit later.

Operator prompt overlays: `lead_prompt_overlay` on AppConfig and

`subagents.agents.<name>.prompt_overlay` accept literal `prepend`/`append` strings.

The per-assembly snapshot owns these settings; no run-context override exists.

DeerMem owns its separate `memory.backend_config.prompt_prepend`/`prompt_append`

fields so the memory package remains host-agnostic.

Custom Agent `AgentConfig.display_name` is an optional, whitespace-trimmed Unicode

label of at most 100 Unicode code points. C0/C1 controls and bidirectional

formatting controls (U+202A–U+202E, U+2066–U+2069) are rejected before trimming.

Also reject soft hyphen, Arabic letter mark, U+200B, U+200E–U+200F,

U+2028–U+2029, U+2060–U+2065 and U+FEFF. Labels consisting only of

marks, separators or other invisible characters are invalid; ZWNJ/ZWJ remain

supported inside ordinary text and emoji.

It is stored in the existing config document by

both agent stores; it never participates in paths, routing, or authorization.

Gateway create/update/response models share its validation. It remains outside

`MANAGED_AGENT_CONFIG_FIELDS` so `update_agent` preserves it. `setup_agent`

explicitly carries forward the existing owner's display name when re-bootstrapping;

the Gateway explicitly overrides it when supplied, including null to clear.

Both stores use `parse_agent_config` to ignore only an invalid stored

`display_name` on read, logging the agent identifier without the invalid value

and without rewriting storage. Other config errors still

raise, and API create/update validation remains strict.

**Main Configuration** (`config.yaml`):

Setup: Copy root `config.example.yaml` to `config.yaml`. Startup callers use

`deerflow.env.load_selected_env_file()` for optional `DEER_FLOW_ENV_FILE` (cwd-relative,

strict readable file, process env wins). Reject a disabling `PYTHON_DOTENV_DISABLED`

when selection is explicit; otherwise retain the original `load_dotenv()`.

Keep this helper free of config imports so auth/debug cannot preload defaults.

**Config Versioning**: `config.example.yaml` has a `config_version` field. On startup, `AppConfig.from_file()` compares user version vs example version and emits a warning if outdated. Missing `config_version` = version 0. Run `make config-upgrade` to auto-merge missing fields. When changing the config schema, bump `config_version` in `config.example.yaml`.

The v46 upgrade treats an existing `tools[]` entry in the `knowledge` group as

pre-gate enablement and sets `knowledge_base.enabled: true` only when that flag

was absent. Explicit `true` or `false` values remain authoritative. Moving

legacy provider settings out of `knowledge_base` remains specific to the

RAGFlow `knowledge_search` tool; LightRAG and other knowledge providers keep

their tool-local settings unchanged.

Top-level `recursion_limit` and `max_recursion_limit` are hot-reloaded per Gateway run. The former supplies the default when a request omits or provides an invalid value; the latter caps both configured and client-provided budgets.

`tool_output` character/count limits and every `tool_overrides` value share

non-negative integer validation. Reject booleans before Pydantic coercion;

preserve numeric strings from environment substitution and explicit zero.

Zero per-tool overrides disable externalization only; a positive global

fallback budget still truncates oversized output. Regression tests exercise

YAML loading and both middleware tool-call paths in `test_tool_output_config_limits.py`.

**Config Caching**: `get_app_config()` caches the parsed config, but automatically reloads it when the resolved config path or file content signature changes. The signature includes file metadata and a content digest, so Gateway and LangGraph reads stay aligned with `config.yaml` edits even on object-store or network mounts where mtime can remain stale. The loader reads the file once through `file_signature.read_config_with_signature` and parses those same bytes, so the recorded signature always describes the parsed content: a write that races the load can only cause one extra reload, never a cache that holds one revision under another revision's signature (which the comparison could never detect). The cached `extensions` snapshot follows the process extensions singleton: `get_app_config()` also reloads when `get_extensions_config()` returns a new instance (see **Extensions Config Caching** below), and the loader takes that instance before reading `config.yaml`, so it can only be older than the YAML it merges with, never newer.

**Config Hot-Reload Boundary**: Gateway dependencies route through `get_app_config()` on every request, so per-run fields like `models[*].max_tokens`, `summarization.*`, `title.*`, `memory.*`, `subagents.*`, `verification.*`, `tools[*]`, and the agent system prompt pick up `config.yaml` edits on the next message. `AppConfig` is intentionally **not** cached on `app.state` — `lifespan()` keeps a local `startup_config` variable for one-shot bootstrap work and passes it to `langgraph_runtime(app, startup_config)`.

Infrastructure fields are **restart-required**. The authoritative list lives in `packages/harness/deerflow/config/reload_boundary.py::STARTUP_ONLY_FIELDS` and is mirrored by the standardised `"startup-only:"` prefix on the corresponding `Field(description=...)` in `AppConfig` or an explicitly registered nested config model, so IDE hover on those fields surfaces the reason inline (no need to context-switch into this table). Currently registered: `plugins`, `database`, `checkpointer`, `run_events`, `agent_storage`, `stream_bridge`, `sandbox`, `skills.container_path`, `log_level`, `logging`, `channels`, `channels.leader_lease` (validated by `channel_leader_lease_config.py` from the non-schema `channels` mapping; the multi-process gate refuses `enabled: false` while a non-GitHub channel is enabled), `channel_connections`, `scheduler`, `mcp_tasks`, `subagent_runtime`, `subagent_batches`, `run_ownership`, `dedupe_storage`, `auth.local.throttle_storage` (the login throttle store is installed once by `langgraph_runtime`; `max_login_attempts` / `lockout_seconds` stay live-read). Adding a new restart-required field requires updating the registry; drift is pinned by `tests/test_reload_boundary.py`. `scheduler.recursion_limit` is the exception inside that section: it is read from `get_app_config()` at each scheduled dispatch, so a YAML edit applies to the next run without restarting the poller.

**Persistence backend resolution**: the unified `database` section selects the

Gateway's LangGraph checkpointer, LangGraph Store, and DeerFlow SQL repositories.

The deprecated `checkpointer` section remains backward compatible and, when

present, overrides `database` for the LangGraph checkpointer and Store only;

application repositories continue to use `database`.

Configuration priority:

1. Explicit `config_path` argument

2. `DEER_FLOW_CONFIG_PATH` environment variable (a missing file is an error, not a fallthrough)

3. `config.yaml` under `DEER_FLOW_PROJECT_ROOT`, or the current directory when it is unset

4. Legacy `backend/config.yaml`, then repository-root `config.yaml` (project root is the **recommended location**)

`scripts/config-upgrade.sh` calls `AppConfig.resolve_config_path` rather than copying this order.

The legacy locations are anchored to the installed harness source, not to the caller's checkout.

`scripts/doctor.py` calls `AppConfig.resolve_config_path` rather than copying this order.

Config values starting with `$` are resolved as environment variables (e.g., `$OPENAI_API_KEY`).

This happens once, at load: consumers such as sandbox providers and ACP `env` use the values verbatim,

because a resolved secret may itself start with `$` (`tests/test_sandbox_environment_passthrough.py`).

`ModelConfig` also declares `use_responses_api` and `output_version` so OpenAI `/v1/responses` can be enabled explicitly while still using `langchain_openai:ChatOpenAI`.

`ModelConfig.request_admission` is optional and is not a provider parameter.

Its positive RPM, finite wait deadline, queue bound and optional quota-group name

configure process-local model pacing. `requests_per_minute` and `max_queue_size` are strict

integers (bools and floats are rejected) behind a `BeforeValidator` that converts a

decimal literal delivered as a string, because `$VAR` substitution always yields

`str`; any other string still fails validation. Models sharing an explicit group must use

identical policies. Restart after changing, disabling or regrouping an active

policy; conflicting policies fail construction rather than silently resetting

an active budget. This nested model option is enforced by its limiter registry,

not by the top-level infrastructure reload-boundary registry.

**Extensions Configuration** (`extensions_config.json`):

MCP servers and skills are configured together in `extensions_config.json` in project root:

Both the runtime loader and raw read-modify-write reader accept UTF-8 with or

without a leading BOM. Writes remain UTF-8 without a BOM.

Docker development mounts the project directory at `/app/project` and points

`DEER_FLOW_CONFIG_PATH` / `DEER_FLOW_EXTENSIONS_CONFIG_PATH` into that directory.

Keep mutable config files behind a directory bind mount: single-file bind mounts

can become stale or inaccessible when a host editor replaces a file on save.

Configuration priority:

1. Explicit `config_path` argument

2. `DEER_FLOW_EXTENSIONS_CONFIG_PATH` environment variable

3. `extensions_config.json` in current directory (backend/)

4. `extensions_config.json` in parent directory (project root - **recommended location**)

Extensions are optional only in the fallback *search* mode (priority 3-4 above): `ExtensionsConfig.resolve_config_path()` returns `None` when neither an explicit `config_path` nor `DEER_FLOW_EXTENSIONS_CONFIG_PATH` is given and the search locations find nothing. An explicit `config_path` argument or a set `DEER_FLOW_EXTENSIONS_CONFIG_PATH` (priority 1-2) is an operator assertion that one particular file must be used, so a missing file in either of those modes raises `FileNotFoundError` instead — including when the file existed earlier and has since been deleted. The MCP tools cache's staleness check (`deerflow.mcp.cache._resolve_config_path`) is a narrow, deliberate exception to that rule: it catches that `FileNotFoundError` locally and treats it as "unconfigured" so a previously-valid config disappearing mid-run degrades the cache to serving its last-known-good tools instead of raising out of a per-request hot path (see the MCP System section below).

**Extensions Config Caching**: `get_extensions_config()` caches the parsed file but revalidates it on every call against the resolved path and the `(mtime, size, sha256)` signature from `file_signature.get_config_signature`, so an edit made by another Gateway worker, or by another instance sharing the file (the Helm home volume, the compose bind mount), is visible to this process on its next read without a restart or an explicit reload. Writers (MCP router, skill toggle, `DeerFlowClient`) still call `reload_extensions_config()` after their write, which records the written revision so the next read does not reload it again. Once a configuration has been loaded, a revision that cannot be loaded — the file vanished, or it is truncated or invalid, for example midway through the non-atomic `EBUSY` overwrite fallback — keeps the last-known-good configuration and is logged once at warning level with the exception type only (validation messages can embed resolved `$VAR` secrets); the first load still raises, so a broken file at startup stays loud. `reload_extensions_config(config_path=...)` makes the cache follow that file (later edits included) until a reset or an argument-less reload. `set_extensions_config()` pins an injected instance until `reload_extensions_config()` or `reset_extensions_config()`; tests that inject one must reset it afterwards, because the `extensions` snapshot of `get_app_config()` follows the pinned instance too. The local-bash absolute path allowlist (`sandbox/tools.py::_get_mcp_allowed_paths`) is derived from this singleton, which is why the revalidation is a security property rather than an optimization. Pinned by `tests/test_extensions_config_freshness.py` and the extensions cases in `tests/test_app_config_reload.py`.

Extensions loads parse the bytes from `read_config_with_signature`, recording

that read's digest rather than an earlier probe's. Freshness reloads read the

probed path explicitly; a missing or invalid revision keeps the last-known-good

cache, including AppConfig's middleware snapshot, until a readable revision returns.

**Shared Reset Markers** (`shared_reset_marker.py`): caches derived from the extensions config can go stale without a byte of it changing (a remote MCP server's `tools/list`, a skill installed or edited on the shared volume by another process). `SharedResetMarker(suffix)` owns one hidden JSON file beside the resolved config, `.<config name>.<suffix>.json`; `publish()` replaces it atomically with a random generation under `extensions_config_write_lock` + `extensions_config_file_lock` (`publish_locked()` when the caller already holds both), chaining `previous_generation` and an optional `user_id`. `SharedResetMarkerTracker.poll()` re-reads the marker at most once per second (monotonic clock, never blocking behind a concurrent poller), adopts the current state silently on its first poll, and reports a `SharedResetChange`: `user_ids={user}` only when exactly one publication carrying that user happened since the last poll, otherwise `None` (retire everything), including deleted/unreadable markers and config-path switches. `note_own_publication()` lets the writer skip the redundant self-invalidation. `resolve_shared_config_path()` maps a missing explicit path to `None` (no shared directory: callers answer `scope=process`). Consumers: `mcp/cache.py` (`mcp-cache-reset`) and `agents/lead_agent/prompt.py` (`skills-cache-reset`). Tests: `tests/test_shared_reset_marker.py`.

**Credentials key** (`credentials_key.py`): `DEER_FLOW_CREDENTIALS_KEY` is env-only

(no `config.yaml` key, never derived from `AUTH_JWT_SECRET`): comma-separated

Fernet keys, first encrypts, all decrypt (`MultiFernet`). `get_credentials_cipher()`

is the process-wide entry point for consumers that store secrets at rest; it

returns a `CredentialsCipher` (`encrypt_text`/`decrypt_text`/`rotate_text`,

`fernet:v2:` prefix, legacy `fernet:v1:` readable). `decrypt_text` raises

`CredentialsDecryptError` (an `InvalidToken`); callers treat that as missing data

and never crash. Unset, it generates `{base_dir}/.credentials_key` -- file I/O,

so async callers use `asyncio.to_thread`. Error text never carries key material.

A new consumer must also join `app.gateway.deps.credentials_key_consumers`, which

drives both Gateway loading and the multi-instance refusal.

**Secret files** (`secret_file.py`): `read_or_create_secret_file` is the one way

to create a shared secret file (`.credentials_key`, `.jwt_secret`, the managed

model key): write a `0600` temp file, then hard-link it exclusively and read

back, so the name is never empty or partial and concurrent creators converge.

No hard links (SMB) or an abandoned empty file (older releases): single-winner

`<name>.replacing` claim plus rename. Never overwrites a non-empty invalid file.

Tests: `tests/test_secret_file.py`, `tests/test_credentials_key.py`.

### Config Schema

**`config.yaml`** key sections:

- `models[]` - LLM configs with `use` class path, `supports_thinking`, `supports_vision`, provider-specific fields. An optional `reasoning:` block (issue #5073) declares thinking availability (`unsupported|optional|required`), the accepted effort `values` with `aliases`/`default`/`path`, the payload `dialect`, and the reasoning `history` requirement; when present, `supports_thinking` / `supports_reasoning_effort` are derived from it and contradictory combinations fail validation in `ModelConfig`

A declared custom `reasoning.effort.path` is the only effort serialization path: `ModelConfig` rejects leftover `reasoning_effort` keys in the profile or thinking templates, even if their values are otherwise accepted.

A boolean or level-string `reasoning` (`true` / `false`, or `low|medium|high` for gpt-oss style models) remains the legacy native ChatOllama setting and is passed to the provider; only a mapping opts into the contract.

- `logging.enhance` - Log output only (`enabled`, `format`): whether log records carry a `trace_id` field, and in which format. Trace ids are issued unconditionally — the Gateway `X-Trace-Id` header and Langfuse `deerflow_trace_id` metadata are always present whatever this says (see the Request Trace Context section in `packages/harness/deerflow/AGENTS.md`); restart-required

- vLLM reasoning models should use `deerflow.models.vllm_provider:VllmChatModel`; for Qwen-style parsers prefer `when_thinking_enabled.extra_body.chat_template_kwargs.enable_thinking`, and DeerFlow will also normalize the older `thinking` alias

- `tools[]` - Tool configs with `use` variable path and `group`

- `tool_groups[]` - Logical groupings for tools

- `sandbox.use` - Sandbox provider class path

- `sandbox.ownership` - Cross-instance sandbox lease storage. Renewal interval

and TTL multiplier must each be finite, and their derived lease TTL must also

remain finite. Redis-backed lease TTLs must be at least one millisecond and

use at most half of Redis's signed 64-bit millisecond range, leaving

deterministic headroom for conversion from a relative TTL to an absolute Unix

timestamp during validation. The Redis store rounds fractional milliseconds

up so its integer TTL never shortens the validated lease.

- `skills.path` / `skills.container_path` - Host and container paths to skills directory. AIO and E2B snapshot the container path at provider startup. Their local/remote backends and the Kubernetes provisioner require one canonical absolute non-root path outside reserved platform mounts; custom roots participate in deterministic sandbox identity, and E2B records the root in remote metadata.

- `skills.deferred_discovery` - When `true`, replaces the full-metadata `<available_skills>` prompt block with a compact `<skill_index>` (names only) and registers the `describe_skill` tool so the agent fetches metadata on demand. Defaults to `false` (legacy full-metadata injection)

- `title` - Auto-title generation (enabled, max_words, max_chars, model_name; null model_name uses fast local fallback, explicit model_name uses the prompt_template LLM path)

- `summarization` - Context summarization (enabled, trigger conditions, keep policy)

- `subagents.enabled` - Master switch for subagent delegation

- `subagent_runtime` - Startup-only shared process admission (`max_running`, bounded async wait queue, queue/reject policy, and queue timeout) for ordinary and durable-batch native subagents

- `subagent_batches` - Startup-only explicit durable batch scheduler limits (disabled by default), including separate total, live, and running dimensions plus leases/retries/result bounds

- `memory` - Memory system (enabled, storage_path, debounce_seconds, shutdown_flush_timeout_seconds, model_name, max_facts, fact_confidence_threshold, injection_enabled, max_injection_tokens, staleness_review_enabled, staleness_age_days, staleness_min_candidates, staleness_max_removals_per_cycle, staleness_protected_categories, staleness_max_lifetime_multiplier, staleness_max_extension_days)

- `knowledge_base` - Hot-reloadable, provider-agnostic knowledge capability and custom-agent scope-selector flags. It gates the read-only Agent tools and selector; provider connection, allowlist, and retrieval defaults belong to the matching entry in `tools[]` (for example, the RAGFlow `knowledge_search` tool).

**`extensions_config.json`**:

- `mcpServers` - Map of server name → config (enabled, type, command, args, env, url, headers, oauth, description, `routing`, `tools`, `tool_call_timeout`, `session_init_timeout`). `routing.mode="prefer"` emits `<mcp_routing_hints>` prompt guidance; if `tool_search` defers the hinted tool, `McpRoutingMiddleware` can also auto-promote matching deferred schemas before the model call. It does not hard-disable other tools. `session_init_timeout` (default `DEFAULT_MCP_SESSION_INIT_TIMEOUT` = 60s, `null` to disable) bounds server bring-up: tool discovery and persistent stdio session initialization, so a hung server cannot block agent construction indefinitely; durable HTTP/SSE task calls use it for their ephemeral session initialization too. `tool_call_timeout` bounds individual stdio calls and durable-task calls on every transport; other HTTP/SSE tools use transport-level timeouts.

- `skills` - Map of skill name → state (enabled)

- `middlewares` - `AgentMiddleware` entries for lead and subagent runtime extension: class-path strings or `{class, kwargs}` objects. `kwargs` values must be JSON types; YAML dates and timestamps are coerced to ISO strings so they match JSON. `config.yaml -> extensions` can override these fields after validation; overrides are replace-per-field, not list concatenation.

Gateway API endpoints and `DeerFlowClient` methods can modify MCP servers and skill state at runtime; their `extensions_config.json` writes use the shared atomic replacement helper, while `middlewares` remains an operator-controlled config-file extension point.

Values beginning with `$` are resolved from the environment when the file is loaded, and an unset variable becomes `""`. Runtime writers (MCP router, skill toggle, `DeerFlowClient`) therefore read the raw file with `read_raw_extensions_config`, merge into it (`set_raw_skill_enabled` for skill state), check the candidate with `validate_raw_extensions_config`, and write that raw dict. They never serialize an `ExtensionsConfig` model back to disk: its resolved values would persist secrets in plaintext and erase the references. When the file does not exist yet, the Gateway skill toggle seeds only the cached skill states. `tests/test_extensions_config_raw_writes.py` and the placeholder tests in `tests/test_client.py` pin this.

`AgentConfig.knowledge_scope` uses the versioned `KnowledgeScope` contract as a

Gateway new-turn default. The API preserves omitted updates and clears explicit

null; file and SQL stores persist it in the existing config document. Both

`update_agent` and HTTP PUT pass the effective groups to

`preserve_non_managed_fields`: a changed list excluding knowledge clears the

inherited default, but explicit API scope fields still win. Keep it outside

`MANAGED_AGENT_CONFIG_FIELDS` so unrelated self-updates preserve it;

`setup_agent` also preserves the binding when re-bootstrapping. Admission uses

explicit message scope before this default and snapshots it onto the current

human message. Resolve the executing agent with runtime context over configurable,

including legacy callers using `context.agent_name`; bootstrap skips defaults.

Recovery (including a legacy null scope) never reapplies defaults.

The operator allowlist is still checked by retrieval; this is not authorization.

Unscoped new runs persist pre-default request digests even for unbound agents.

Digest-free legacy retries compare pre-default canonical input; explicit scopes

and recovery are excluded. Retries preserve the original run across binding edits.

The file-backed singleton entrypoints additionally merge administrator-managed shared

models from the encrypted runtime-home catalog. YAML entries win name conflicts;

managed changes create new effective snapshots and do not alter an active runtime

or an explicitly injected AppConfig. See `../models/AGENTS.md` for storage and reload

boundaries. `managed_model_providers.py` derives provider defaults from validated

endpoints; `ManagedModel.runtime_config()` combines them with profile fields.

`AppConfig.from_file()` remains YAML-only.

### Web model provider presets

Managed model provider presets live only in `managed_model_providers.py`. `managed_models.py` reads legacy custom encrypted records compatibly and converts presets into ModelConfig without changing YAML, previous snapshots or injected configurations. See the managed-model section in `../models/AGENTS.md` and the Web registration section of `backend/docs/CONFIGURATION.md`.