Richland Technology
Richland Technology
← Back to Blog
llmopen-sourceresearchmultimodalinferencespeculative-decoding

MMFlash brings block-diffusion speculative decoding to VLMs — up to 3.58× lossless, with a quantified reason text-first drafters fail on images

A diffusion drafter's acceptance rate drops from 46% to 22% as tokens become more visually dependent — MMFlash reweights the loss on those tokens and ships drop-in drafters for SGLang/vLLM.

Richland AI Desk·16 November 2026·5 min read
MMFlash brings block-diffusion speculative decoding to VLMs — up to 3.58× lossless, with a quantified reason text-first drafters fail on images

MMFlash brings block-diffusion speculative decoding to vision-language models, reporting lossless speedups up to 3.58× over autoregressive decoding. The motivating finding is a quantified failure mode: a plain diffusion drafter's acceptance rate collapses from 46% to 22% as tokens become more visually dependent.

The preprint from National University of Singapore (arXiv:2611.18482) ships code and drafters for Qwen3.5-4B, Qwen3.5-9B, Qwen3.8-27B and Gemma 4 E4B-it that run drop-in with stock SGLang and vLLM. It extends block-diffusion drafting to multimodal models — a direction the authors call largely unexplored — and the diagnosis behind it is the part worth stealing.

What shipped

Speculative decoding works by having a small drafter propose tokens that a target model verifies in parallel, accepting the draft when it matches and rolling back where it does not. Diffusion drafters like DFlash go further: instead of emitting tokens one at a time, the drafter proposes an entire masked block in a single forward pass, and the target verifies the whole block in one pass. That is what makes block drafting fast — but until now it has been aimed almost entirely at text-only models.

MMFlash makes block-diffusion drafting work for VLMs, and it starts with a measurement. The authors define a per-token "visual dependence" score as the KL divergence between the target's next-token distribution with the image and without it. Applied to a stock DFlash drafter, acceptance rates fall monotonically from 46% in the lowest visual-dependence quartile to 22% in the highest, and the highest quartile accounts for 44% of first rejections within draft blocks versus 7% for the lowest. The disparity persists even after multimodal training.

The fix is a loss-reweighting scheme. MMFlash measures each training token's visual dependence on the target and upweights the drafter's cross-entropy loss on visually dependent tokens, scaling the bonus with how poorly the drafter predicts them and decaying back to baseline as predictions improve. The drafter stays "vision-free": it never re-encodes the image or video, reading instead the target's hidden states where the visual context is already fused, so its cost does not grow with the number of visual tokens.

For long video inputs, MMFlash adds a sparse draft context — attention sinks, all text tokens, strided visual tokens and a recent local window (sink=4, text=1, stride=32, window=256) — that shrinks a 40K-token video prompt to roughly 1.6K tokens. The same pattern is used in training and inference.

MMFlash demo: the same question answered by Qwen3.5-9B with autoregressive decoding, EAGLE-3, DFlash and MMFlash

The authors report lossless speedups of up to 3.58×, beating EAGLE-3 by up to 195.4% and DFlash by up to 17.1% across multimodal benchmarks. On the GitHub repo the per-model figures are 3.0× on Qwen3.5-4B and 9B and up to 4.7× on Qwen3.8-27B (one H200, batch size 1, greedy). Under greedy decoding, image benchmarks average 3.00–3.05× (10.3–10.9% over DFlash) and long-video benchmarks average 2.33–2.71× (11.2–23.2% over DFlash).

Released drafters pair with their targets in a speculative-decoding server — they are not standalone models:

Target modelDraft sizeBlock size
Qwen3.5-4B537M16
Qwen3.5-9B1.05B16
Qwen3.8-27B1.73B16
Gemma 4 E4B-it537M16

All are in the MMFlash HuggingFace collection, and the repo says they run with stock SGLang and vLLM (DFLASH), no patches, tested with SGLang 0.5.23 and vLLM 0.33.0.

Why it matters

The headline speedups are worth a look, but the durable takeaway is the diagnostic. The 46%→22% acceptance collapse is a concrete, quantified reason why text-first speculative-decoding techniques underperform on multimodal workloads: the drafter's weak spot is exactly the tokens that carry the visual information, and most rejected blocks break there. If you are serving a VLM and wondering why your drafter's acceptance rate is lower than the paper's text-only numbers, this is the mechanism.

The design choices matter for anyone building on it. A vision-free drafter keeps draft cost independent of image/video token count — the thing that usually blows up multimodal serving budgets. And the sparse draft context is a practical answer to the long-video case, where a full 40K-token context would make drafting as expensive as the target. The drop-in SGLang/vLLM support means you can test it this week without patching a serving stack.

Caveats

Everything here is self-reported by the authors on their own hardware (one H200, batch size 1, greedy), and this is an unreviewed preprint marked "Work in process" — its baseline DFlash is ICML 2026 accepted, but MMFlash itself has no peer review, and we found no independent replication of its numbers.

The "lossless" claim is structural rather than empirical: it rests on the standard speculative-decoding guarantee that the target verifies the draft block in one pass, so the output distribution is unchanged. It does not mean the target's exact token sequence is preserved.

Two numbers deserve care. The abstract's "up to 3.58×" is not tied to a specific model or benchmark in the accessible text, and the GitHub figure of 4.7× on Qwen3.8-27B actually exceeds it — the two headlines come from different measurement setups. The reported image-benchmark results are single-request, greedy, capped at 4096 new tokens with reasoning off; the paper mentions concurrency (1–32 requests in flight on vLLM) and generalization to other model families, but those results were not accessible in the truncated text and should be treated as unverified.

Sources

Start a Conversation

Ready to build something like this?

Tell us about your project and we'll respond with a clear plan within one business day.

Tell Us About Your Project

Takes 3 minutes. We read every submission personally.