vllm-reasoning-parsers git:20260915.6563fb4A

Immutable. This exact content is served forever at /api/v1/blob/49bf80b5c8697e3a.

---
name: vllm-reasoning-parsers
description: |-
  vLLM reasoning-parser operator + developer reference. `--reasoning-parser` CLI wiring, `ReasoningParser` contract (non-streaming `extract_reasoning` + per-delta `extract_reasoning_streaming`), `is_reasoning_end` xgrammar gating, `--structured-outputs-config.enable_in_reasoning` bypass, 29 built-in parsers with per-model quirks, 15 production pitfalls, authoring custom parsers via `@ReasoningParserManager.register_module` or plugin.
when_to_use: |-
  Trigger on `--reasoning-parser`, `reasoning_content`, `reasoning_parser_plugin`, `<think>`/`</think>` handling, `extract_reasoning`, `extract_reasoning_streaming`, `is_reasoning_end`, `ReasoningParser`, `ReasoningParserManager`, harmony channels (`<|channel|>analysis`/`final`), Qwen3 prompt-side `<think>`, DeepSeek-R1 missing start tag, Kimi K2 tool-section implicit end, Hunyuan state machine, Granite phrase markers, `enable_thinking=False`, `chat_template_kwargs={enable_thinking,thinking}`, reasoning-vs-content split, reasoning leaked into content, content empty while reasoning filled, structured output breaking with thinking disabled, tool calls not parsed when reasoning is on, adding a new reasoning model. Symptoms — "why is `reasoning_content` null on DeepSeek-R1", "Qwen3 JSON gibberish with `enable_thinking=False`". Applies without "parser" in prompt. Also implicit — "audit reasoning config", "deploy-memo reasoning", "thinking split wrong". NOT prompt-side Jinja (→ `vllm-chat-templates`) or tool-call JSON extraction (→ `vllm-tool-parsers`).
---

# vLLM reasoning parsers

Target: operators wiring up `--reasoning-parser NAME` on a chat-completion endpoint, or developers authoring a parser for a new thinking model. Source of truth: `vllm/reasoning/` on `main`.

## What a reasoning parser actually does

When a reasoning-trained model emits a single token stream like

```
<think>user asked X, let me check Y...</think>The answer is 42.
```

vLLM splits this into two fields on the chat-completion response: `reasoning` (the CoT) and `content` (the final answer). `--reasoning-parser NAME` selects the class that does the split. Without it, the whole stream lands in `content`.

> **Field-name note.** The response field is `reasoning` (see `ChatMessage.reasoning` / `DeltaMessage.reasoning` in `vllm/entrypoints/openai/chat_completion/protocol.py`). `reasoning_content` was **removed from the output in v0.16.0** ([#33402](https://github.com/vllm-project/vllm/pull/33402), merge `c5113f60`), and v0.28.0 documents the result as a breaking client change ([#50624](https://github.com/vllm-project/vllm/pull/50624)). Many third-party docs and clients still call it `reasoning_content`. **The asymmetry is the trap: input still accepts the old name and normalizes it, output never emits it.** So a request looks fine and a client reading `reasoning_content` off the response sees `null` every time, with no error, even when the parser ran correctly. Serving **v0.16.0 or newer, read `reasoning`**.

The parser is also the gating authority for **xgrammar / structured output**: by default, grammar enforcement is held off until `is_reasoning_end(input_ids)` flips true, so the model thinks freely before being constrained to JSON. Flip that default with `--structured-outputs-config.enable_in_reasoning=true` — then the grammar applies from token 0 regardless of reasoning state (useful for structured CoT).

## The contract (`ReasoningParser` ABC)

`vllm/reasoning/abs_reasoning_parsers.py`. Every parser implements:

| Method | Called by | Purpose |
|---|---|---|
| `is_reasoning_end(input_ids)` | xgrammar, non-streaming serving, tool-call gate | Has `</think>`-equivalent been emitted yet? |
| `is_reasoning_end_streaming(input_ids, delta_ids)` | xgrammar per decode step | Same, but cheap — checks only the delta |
| `extract_content_ids(input_ids) -> list[int]` | structured output | Token IDs of post-reasoning content |
| `extract_reasoning(model_output, request) -> (reasoning, content)` | non-streaming chat completion | Full-string split; either field may be `None` |
| `extract_reasoning_streaming(previous_text, current_text, delta_text, previous_token_ids, current_token_ids, delta_token_ids) -> DeltaMessage \| None` | streaming chat completion | Per-delta split; returns `DeltaMessage(reasoning=..., content=...)` or `None` to swallow |

Optional:

- `count_reasoning_tokens(token_ids)` — for usage accounting. Default returns `0`. `BaseThinkingReasoningParser` implements it via depth counter so nested `<think>…<think>…</think>…</think>` are handled.
- `adjust_request(request)` — mutate request (rare).
- `prepare_structured_tag(original, tool_server)` — emit a `structural_tag` JSON (GPT-OSS uses this for harmony).
- `reasoning_start_str` / `reasoning_end_str` — expose delimiter strings so `ReasoningConfig.initialize_token_ids` can derive the multi-token ID sequences automatically.

**Streaming vs. non-streaming are two independent code paths.** A parser that's correct on `extract_reasoning` can be buggy on `extract_reasoning_streaming` and vice versa. Every parser must be tested on both.

## The CLI path (what `--reasoning-parser qwen3` actually triggers)

`vllm/engine/arg_utils.py:985` (v0.27.1; grep for `--reasoning-parser` — line moves between releases) declares `reasoning_parser: str = StructuredOutputsConfig.reasoning_parser`. `api_server.py` validates it against `ReasoningParserManager.list_registered()` at startup (invalid name = fast-fail with the list of registered names).

On request, `OpenAIServingChat` instantiates a **fresh parser per request** via `self.reasoning_parser_cls(tokenizer, chat_template_kwargs=chat_template_kwargs)` (`vllm/entrypoints/openai/chat_completion/serving.py:240`). "Fresh per request" is load-bearing for stateful parsers — see Hunyuan in the matrix.

Registry: `vllm/reasoning/__init__.py` has a `_REASONING_PARSERS_TO_REGISTER` dict of `name → (filename, ClassName)` that feeds `ReasoningParserManager.register_lazy_module`. Lazy import means a broken parser file won't crash vLLM startup until somebody selects it.

Plugin path: `--reasoning-parser-plugin /path/to/my_parser.py` calls `ReasoningParserManager.import_reasoning_parser(path)`, which `importlib`-loads the file. The file registers itself via `@ReasoningParserManager.register_module(["my-name"])` at import time. Then `--reasoning-parser my-name` selects it.

## The 15 things that go wrong

See `references/pitfalls.md` for each with repros and fixes. Quick index:

1. **`reasoning_content` is `null` on DeepSeek-R1** — chat template injected `<think>` into the prompt, so the model never emitted a start token. Parser must tolerate missing start (base `BaseThinkingReasoningParser` does, via the `.partition(start_token)` pattern).

2. **`content` is empty, CoT in `reasoning_content` with `enable_thinking=False`** — parser didn't branch on `chat_template_kwargs`. Qwen3 / DeepSeek-V3 / Kimi K2 route to `IdentityReasoningParser` (or internal flag) when thinking is off. DeepSeek-V3 has two thinking-default variants: `deepseek_v3` (default **off**) and `holo2` = `DeepSeekV3ReasoningWithThinkingParser` (default **on**). **`glm45` used to be in that second group and no longer is** — as of v0.25.1 it moved onto the new adapter path alongside `glm47`, so don't assume `glm45` and `holo2` still behave alike.

3. **Gibberish JSON when `guided_json` + `enable_thinking=False`** — the reasoning parser's `is_reasoning_end` on the *prompt* must return True so xgrammar enforces from token 0. Serving layer caches this result as `prompt_is_reasoning_end_arr[i]`. If the parser only checks for `</think>` in input_ids and the thinking-disabled chat template emits `<think>\n\n</think>\n\n` in the prompt, this works; if it does something else, xgrammar silently stays gated.

4. **Truncated output = wrong field** — when the model hits `max_tokens` mid-reasoning, is the whole output "reasoning" or "content"? Bound the reasoning segment instead of the response: **`thinking_token_budget`** in the request body caps reasoning tokens, so the run ends through the parser's normal path rather than by truncation ([#46727](https://github.com/vllm-project/vllm/pull/46727), on Model Runner V2 from **v0.28.0** — it is a port, and MRV2 became the default for all models in v0.29.0, so on an older build confirm which runner is live before relying on it). Quantized checkpoints are where it earns its place; long reasoning traces on them are the PR's stated motivation. Qwen3 with `enable_thinking=True` (default) → `(model_output, None)` = all reasoning. Qwen3 with `enable_thinking=False` → `(None, model_output)` = all content. DeepSeek-R1 base → `(model_output, None)`. Know the parser's policy before shipping.

5. **Nested `<think>` tags** — base parser's `partition` stops at the first `</think>`; if a CoT contains a literal `</think>` substring (sometimes seen in distillation artifacts), content gets split wrong. `count_reasoning_tokens` uses a depth counter so counts are right, but the split is not.

6. **Tool calls break with reasoning enabled** — tool parser only sees the `content` half of `extract_reasoning`'s return. If the reasoning parser returns `(everything, None)`, tool parser sees nothing. Kimi K2 handles this by treating `<|tool_calls_section_begin|>` as an implicit reasoning-end marker.

7. **Stateful parser reused across requests** — Hunyuan A13B's `extract_reasoning_streaming` is a token-ID state machine with `self.current_state` / `self.token_buffer`. Second concurrent request on the same instance = interleaved garbage. vLLM already instantiates per-request, but custom plugins must not hoist state to class-level.

8. **Multi-token delimiter** — `<think>` may encode as a single token (DeepSeek-R1 vocab) or as multiple (`<think`, `>` in some tokenizers). Single-token path: `vocab.get("<think>")`. Multi-token: `tokenizer.encode("<think>")` and do sequence match — the current in-tree worked example is `kimi_k3_reasoning_parser.py`, whose `<|open|>think<|sep|>` marker is a 3-token sequence, so its token-id helpers search for the *subsequence* rather than one id. (The old example, GPT-OSS `reasoning_end_token_ids_prefix`, was deleted at v0.27.0 — see pitfall 10.) `BaseThinkingReasoningParser` raises at init if `vocab.get` returns None — don't inherit from it unless the delimiter is a single vocab entry.

9. **Single-token delta spam** — if delta is exactly one token that *is* `<think>` or `</think>`, return `None` from streaming so the client doesn't see an empty delta. Almost every parser has this skip.

10. **Harmony / GPT-OSS is different** — no `<think>` tag; the channel markers are multi-token. **At v0.27.0 (PR #45560) `GptOssReasoningParser` was gutted**: `is_reasoning_end` / `is_reasoning_end_streaming` now `return True` unconditionally, and `extract_reasoning`, `extract_reasoning_streaming` and `extract_content_ids` all raise `NotImplementedError` pointing at `HarmonyParser`. The old backward-scan machinery (`reasoning_end_token_ids_prefix`, `reasoning_max_num_between_tokens = 20`, `eom_token_id`) is **deleted** — don't look for it, and don't cite it as the multi-token-delimiter example. Boundary detection plus all parsing now live in `HarmonyParser` (`vllm/parser/harmony.py`). Don't copy DeepSeek-R1 for a harmony model.

11. **Mistral moved onto the unified engine at v0.27.0** (PR #48947). `--reasoning-parser mistral` is unchanged, but the registry class is now `MistralParserReasoningAdapter` and `vllm/reasoning/mistral_reasoning_parser.py` is a three-line re-export — the logic is in `vllm/parser/mistral.py`, shared with `--tool-call-parser mistral`. **The hard `ValueError: The tokenizer must be an instance of MistralTokenizer.` is gone** (it existed at v0.25.1; `git grep` finds it nowhere at v0.27.0): `is_mistral_tokenizer()` now *selects a path*, falling back to the generic `adjust_request` when the tokenizer isn't Mistral or can't do grammar. Still prefer `--tokenizer-mode mistral` — grammar-enforced `tool_choice` needs the mistral-common path.

12. **Granite is regex-on-text** — "Here is my thought process:" / "Here is my response:" are phrases, not special tokens. Streaming parser has to buffer partial matches across deltas (`Here is my thou…`) which makes it the most complex parser in the tree. Read `granite_reasoning_parser.py:140+` before modifying.

13. **Implicit reasoning-end** — Kimi K2 ends at `</think>` **or** `<|tool_calls_section_begin|>`. MiniMax M2 never emits `<think>`, only `</think>`. Custom `is_reasoning_end` must encode these facts or xgrammar gates at the wrong moment.

14. **`--enable-reasoning` is gone** — older docs / Stack Overflow answers still reference it. Since roughly v0.8 the only flag is `--reasoning-parser NAME`; the enable/disable is implicit in whether one is passed.

15. **`reasoning_content` is always null — but parser worked fine.** The response field is `reasoning`, not `reasoning_content`, on every release from **v0.16.0** onward (#33402). Client-side name mismatch that looks exactly like a parser failure. Before debugging parsers, `jq '.choices[0].message | keys'` to see what fields actually exist — if `reasoning` is there, it's just a client rename.

## The per-model matrix

`references/parser-matrix.md` — one row per registered name (**29** at v0.27.0: `deepseek_r1`, `deepseek_v3`, `deepseek_v4`, `poolside_v1`, `cohere_command3`, `cohere_command4`, `ernie45`, `gemma4`, `glm45`, `glm47`, `openai_gptoss`, `granite`, `holo2`, `hunyuan_a13b`, `hy_v3`, `inkling`, `kimi_k2`, `kimi_k3`, `mimo`, `minimax_m2`, `minimax_m2_append_think`, `minimax_m3`, `mistral`, `nemotron_v3`, `olmo3`, `qwen3`, `seed_oss`, `step3`, `step3p5`) with: delimiter style, start-token-in-prompt-or-output, thinking-disable mechanism, truncation policy, structured-output gating peculiarities.

Routing (which family each name belongs to — `<think>` two-token, delegating wrapper, stateful, harmony, phrase-regex, tokenizer-gated) lives in the matrix `Family` column. The non-obvious cases worth knowing before reading it: `openai_gptoss` is harmony (`extract_reasoning` raises `NotImplementedError`), `mistral` requires `MistralTokenizer`, `hunyuan_a13b` is a token-ID state machine, `granite` is phrase-regex on text, and `nemotron_v3` swaps reasoning↔content on `enable_thinking=False`.

## Writing a custom parser

`references/writing-custom-parser.md` for the step-by-step. **This
`ReasoningParser` subclass shape remains the supported path for out-of-tree
parsers** — it is what `--reasoning-parser-plugin` loads, and 17 of the 29
in-tree names still use it.

Be aware there is now a **second, in-tree-only shape**: `vllm/parser/<model>.py`
defines one parser class per model, and
`make_adapters(XParser)` in `vllm/parser/engine/registered_adapters.py` derives
*both* `XParserReasoningAdapter` and `XParserToolAdapter` from it — the unified
reasoning+tool design of RFC
[#32713](https://github.com/vllm-project/vllm/issues/32713). **12 of the 29
names are on it at v0.27.0**: `deepseek_v4`, `gemma4`, `glm45`, `glm47`,
`inkling`, `kimi_k2`, `mimo`, `minimax_m2`, `mistral`, `nemotron_v3`, `qwen3`,
`seed_oss`. If you open one of those as a worked example you will find a
three-line re-export and the logic elsewhere.

**Do not infer membership from the filename.** Only some are named
`*_engine_reasoning_parser.py`; `kimi_k2_reasoning_parser.py`,
`minimax_m2_reasoning_parser.py`, `glm47_moe_reasoning_parser.py` and
`mistral_reasoning_parser.py` are ordinary names that are nonetheless shims. The
test is the import:
`grep -l "registered_adapters import" vllm/reasoning/*.py`.

Copy a **legacy-path** parser when writing your own. Shape:

```python
from vllm.reasoning import ReasoningParser, ReasoningParserManager
from vllm.entrypoints.openai.engine.protocol import DeltaMessage

@ReasoningParserManager.register_module(["my_model"])
class MyReasoningParser(ReasoningParser):
    def __init__(self, tokenizer, *args, **kwargs):
        super().__init__(tokenizer, *args, **kwargs)
        # resolve delimiter token IDs via self.vocab or tokenizer.encode(...)

    def is_reasoning_end(self, input_ids): ...
    def extract_content_ids(self, input_ids): ...
    def extract_reasoning(self, model_output, request): ...
    def extract_reasoning_streaming(self, prev_text, cur_text, delta_text,
                                    prev_ids, cur_ids, delta_ids): ...
```

Register decorator → lazy entry in `ReasoningParserManager.lazy_parsers`. Pass the file to `vllm serve ... --reasoning-parser-plugin /path/to/my_parser.py --reasoning-parser my_model`.

Shortcuts when delimiter is a single vocab token: inherit `BaseThinkingReasoningParser` (`vllm/reasoning/basic_parsers.py`), override `start_token` / `end_token` properties only. That covers 90% of the surface — see `deepseek_r1_reasoning_parser.py` (32 lines) for the minimal concrete subclass.

## Testing a parser

Unit tests live in `tests/reasoning/` (DeepSeek R1, Qwen3, Granite, Hunyuan, GPT-OSS, Kimi K2 all have coverage). Run a single file with:

```bash
.venv/bin/python -m pytest tests/reasoning/test_qwen3_reasoning_parser.py -v
```

Every new parser needs tests for:
- Non-streaming: start-in-prompt, start-in-output, truncated (no end), nested, empty.
- Streaming: delta that is only the start token, only the end token, end token spanning two deltas, content-before-end-token in same delta, content-after-end-token in same delta.
- `is_reasoning_end` on both pre-reasoning, mid-reasoning, and post-reasoning token ID sequences.
- Thinking-disabled path if applicable.

## Companion skills

- `vllm-chat-templates` — how the chat template injects `<think>` into the prompt (or not); why `enable_thinking=False` can change what the parser sees.
- `vllm-tool-parsers` — tool-call extraction runs on the post-reasoning `content` only. A wrong reasoning split silently breaks tool calling.
- `vllm-configuration` — `chat_template_kwargs` flows through `ChatCompletionRequest.chat_template_kwargs` into the parser `__init__`.
- `vllm-performance-tuning` — structured output tuning; reasoning parser choice affects xgrammar gating latency.
- `vllm-input-modalities` — reasoning parsers don't apply to embedding / rerank / ASR endpoints.

Upstream issue/PR anchors for each pitfall → `references/sources.md`.