Zero-or-T thinking: a frozen backbone that answers with no generated token or a full reasoning pass — 80% token cut on MiniWoB, and a latency claim caught and corrected in public
HDM's self-routing block picks between zero generated tokens and a full reasoning pass; the headline JevBench latency was internally impossible and the authors corrected it in public.

Researchers at Huawei Noah's Ark Lab, Liverpool, and UCL propose a "zero-or-T thinking" paradigm for LLM inference: one frozen backbone either answers in a single forward pass with zero generated tokens or commits to full autoregressive reasoning, switched by a learned block of a few hundred kilobytes. On JevBench their hybrid decision model (HDM) beats the no-token Jev baseline in accuracy and latency; on the agentic MiniWoB benchmark it cuts reasoning-token cost by 80.1% — at a real accuracy cost.
What shipped
HDM wraps an off-the-shelf reasoning model (Qwen3.5-4B or Qwen3.8-27B) in a "fast mode" that generates no tokens: one forward pass encodes the request and the answer is read from the candidate distribution at a readout position. The "slow mode" is the backbone's own reasoning, inherited unchanged. A lightweight self-routing block — the only trained weights — reads the readout hidden state plus eight statistics of the fast answer and estimates the value of thinking: how much utility the slow mode is expected to add over the fast one. Routing happens once, before either mode runs, and the slow mode fires only where that value exceeds a per-request price λ the caller sets. A question costs either no generated token or the full T a reasoning pass takes — nothing in between, and nothing is parsed from generated text (paper, code).
The novelty is the switch itself. Prior adaptive-reasoning work shortens the chain of thought or routes between separate models; every branch still writes an answer token by token. Here both modes live in one frozen backbone, and the router's decision adds no forward pass and no token because it reads representations the fast pass has already computed. The paper's framing: decide whether a question needs thinking at all, not how much (paper).

For episodic agentic tasks, where there is no per-decision label, they introduce Calibrated Decision Policy Optimisation (CDPO), a PPO-based recipe that trains the router from episode-level returns, discounting each episode's reward by the escalations it spent and sharing that one signal across all its decisions (paper).
The block is trained once on 1,154 MMLU-Pro questions and used unchanged elsewhere. Per the shipped model cards it is 658,692 parameters (2.7 MB) for the 4B backbone and 1,314,052 parameters (5.3 MB) for the 27B (4B card, 27B card). Code is Apache 2.0, and the two routing-block checkpoints are on Hugging Face (code).
Why it matters
The reported numbers, all vendor-reported from a preprint, are the payoff:
- JevBench: HDM-4B scores 0.857 against Jev's 0.851 (+0.7%) at a median 0.179 s against 0.652 s (abstract). Jev is itself a no-token "System One" model, so HDM beats it at its own game while keeping the ability to reason when needed.
- MiniWoB: on 27 held-out click tasks, routed HDM-27B uses 664 thinking tokens per episode versus 3,345 for slow-only — an 80.1% cut — with reward dropping from 0.519 to 0.469 (paper, CASRAI summary). Elapsed time drops from 96.1 s to 31.3 s per episode.
- MMLU-Pro (27B, λ=0.5): the paper reports giving up 0.029 of always-thinking's 0.902 accuracy for 38% fewer tokens (486 vs 778) (paper).
- AGIEval: 1.5 accuracy points traded for 37% fewer thinking tokens (paper).
- Routing quality: AUROC "where thinking helps" is 0.804 for the 4B block and 0.879 for the 27B (code).
For an engineer building agents, the switch is a price, not a tuned per-benchmark knob. λ is set per request; moving it moves the operating point along the cost-accuracy curve without retraining — a query that clears a high price goes to the slow mode, and lowering the price sends more. The block's objective is explicitly cost-aware: utility is multiplied by e^(−ρ·T) per generated token, so a correct slow answer wins only if it improves the chance of correctness by at least e^λ (paper).
That "the fast mode is confident" is not a proxy for "the slow mode would agree" is a useful design lesson. Confidence and the value of thinking come apart in both directions: thinking sometimes spoils a fast-correct answer and adds nothing where both modes are right. The router predicts the joint outcome of the two modes precisely because a confidence threshold alone misroutes (paper).
Caveats
This is an unreviewed preprint, and all accuracy, token, and latency numbers above are the authors' own — none have been independently reproduced. Several specific caveats:
- The headline latency claim was internally impossible and is being corrected. The paper reported routed HDM at 0.179 s median on JevBench, faster than its own fast-only mode at 0.226 s — impossible, since routing always runs the fast pass first. An external auditor (checkpaper) flagged it; the authors confirmed a timing-order artifact and supplied corrected warm like-for-like numbers of 0.176 s (routed) vs 0.154 s (fast-only), promising a revision (GitHub issue #4). The corrected routed number still beats Jev's 0.652 s, but the −72.5% headline uses the uncorrected value.
- MiniWoB is a trade, not a free lunch. The 80.1% token cut costs 5 reward points (0.519→0.469), and the comparison endpoints were single runs (CASRAI).
- Block size is disputed. The paper calls it "a few hundred kilobytes"; the shipped checkpoints are 2.7 MB and 5.3 MB.
- CDPO is a surrogate with weak credit assignment, since a shared episode return gives only a coarse signal per decision (alphaXiv).
- Asymmetric failure mode. Skipping reasoning on a hard query yields a confident wrong answer with nothing in the logs to flag it; the reverse mistake only burns tokens. Community commenters suggest shadow-mode logging before production gating (HF discussion).
- No venue or acceptance status is stated; the paper is a preprint.
The honest read: the paradigm is novel and the shipped artifacts are runnable today, but treat the specific numbers as directional until a revision and independent runs land.
Sources
- Paper abstract: https://arxiv.org/abs/2611.16961
- Paper (HTML): https://arxiv.org/html/2611.16961
- Code (Apache 2.0): https://github.com/zhengxuyu/HDM-code
- 4B block model card: https://huggingface.co/fredreick/HDM-Qwen3.5-4B
- 27B block model card: https://huggingface.co/fredreick/HDM-Qwen3.8-27B
- JevBench timing correction (issue #4): https://github.com/zhengxuyu/HDM-code/issues/4
- checkpaper audit: https://github.com/checkpaper/audits/tree/main/2026/2611.16961-hdm
- CASRAI summary with results table: https://casrai.org/news/value-of-thinking-adaptive-reasoning-hybrid-decision-models-2026
- alphaXiv summary: https://www.alphaxiv.org/abs/2611.16961
- Hugging Face discussion: https://huggingface.co/papers/2611.16961
- Reddit r/LocalLLaMA thread: https://www.reddit.com/r/LocalLLaMA/comments/1xxm2v8/paper_the_value_of_thinking_learning_when_to