Victoria: a Qwen3.8-Flash-Next cut to 288 experts, retrained in NVFP4, hits 70% on Terminal-Bench 2.1

Victoria is a Qwen3.8-Flash-Next derivative with 44% of its MoE experts pruned (512→288 per layer) and retrained natively in 4-bit NVFP4, built on an 8×B300 system Dell loaned for a few weeks. On Terminal-Bench 2.1 it scores 70.0% (avg@3) and decodes at ~280 tok/s on one B300 through its MTP draft head — right at the level of Gemini 3.1 Pro on the same harness. Every Victoria figure is self-reported.

What shipped

Victoria (rmonsurate/Victoria) is a compression of Qwen3.8-Flash-Next, Farpoint Technologies' 48-layer MoE with 512 experts per layer, ~6B active parameters per token, a 51B n-gram embedding, and a 4B multi-token-prediction (MTP) draft head — roughly 180B parameters stored. The release combines two things that are usually done separately:

• Prunes with REAP. 224 of 512 experts per layer are dropped (43.75%, reported as 44%) using router-weighted expert activation pruning, the Cerebras technique from arXiv:2510.13999. REAP scores each expert as the average, over calibration tokens, of router gate weight × L2 norm of the expert's output activation. It then removes the lowest-scoring experts per layer to reach a target sparsity.
• Retrains in the shipping format. Instead of pruning and then post-training-quantizing, the pruned model was retrained at 4-bit NVFP4 — NVIDIA's Blackwell 4-bit float format — so the weights are trained for the format they ship in.

The headline numbers, all from the model card:

• Terminal-Bench 2.1: 70.04% avg@3 (67/89, 61/89, 59/89) vs 62.54% for the previous NVFP4 build. Protocol: Harbor 0.22.0, Terminus 2 agent, 8-hour per-task timeout, 262,144 context, all 89 tasks scored, no-verdict counts as fail. Sign test vs the old build: better on 18 tasks, worse on 6, p = 0.023.
• HumanEval: 97.0% (159/164), one sample per problem, thinking on.
• Decode speed on one B300 (vLLM, single stream, thinking off): 134.7 tok/s without the draft head, 279.6 tok/s with the trained draft head (67.6% acceptance, 3.03 tokens per main-model forward pass).
• Weights: 48.0 GiB including the draft head; the 95.4 GiB n-gram lookup table is separate.
• Output tokens: 35% fewer than the previous build (69.4M vs 107.3M across three runs).

There is also a llama.cpp build: a Q4_K_M GGUF at 49.17 GiB scoring 75.28% (67/89) on Terminal-Bench 2.1 — a single run the card flags as noisy — and 93.2% avg@5 on HumanEval. The two builds are different checkpoints (NVFP4 is the latest, the GGUF the first release), so the 70.0% and 75.3% figures are not directly comparable.

The second release, Maple, is a Canada-first fine-tune of Victoria: it defaults to Canadian tax, benefit, and regulation answers instead of US ones. On 600 held-out questions graded by two AI judges, "answered and passed" rose from 6.6% (Victoria) to 21.8% (Maple) with web search, and from 1.3% to 7.5% without. The Reddit announcement adds that citation of an official Canadian source went 6.0%→62.9% and "no answer" responses fell 47.2%→23.7%; those figures are not on the model card.

Why it matters

REAP is a one-shot pruning method with a mechanism. The saliency score is router gate weight × output activation norm, averaged over calibration tokens — how often the router picks an expert, weighted by how much that expert changes the output. The REAP paper's core finding is that pruning beats expert-merging for generative tasks because merging destroys the router's fine-grained per-token mixing control ("functional subspace collapse"), an irreducible error pruning avoids. Victoria is a live demonstration at 512→288 experts: in the GGUF build the pruned model keeps 84.8% of the unpruned base's Terminal-Bench score (75.28% vs 88.76%).

Train-in-format NVFP4 is the quantization approach to watch. NVFP4 is NVIDIA's 4-bit float for Blackwell: 1 sign bit, 2 exponent bits, 1 mantissa bit, with a shared E4M3 FP8 scale per 16-value micro-block and an FP32 scalar per tensor — roughly 3.5× less memory than FP16, per NVIDIA. NVIDIA has published a train-in-format recipe validated on a 12B model; Victoria applies the same idea to a pruned 180B MoE, retraining in the format it ships in rather than quantizing after the fact. The card plots held-out KL divergence from the full model over training steps.

The draft head is the source of both the speed and the friction. The 280 tok/s figure comes from speculative decoding with the 4B MTP head. The GGUF carries the head as 32 extra tensors, and mainline llama.cpp refuses to load it with "expected 1256, got 1224". Running it today means building the maintainer's fork (rmonsurate/llama.cpp, stock b11276 plus one patch).

On an M3 Max 128 GB the fork goes from 26.8–27.8 tok/s to 34.3–38.0 tok/s with the head on; CUDA speed is unmeasured. vLLM likewise needs two backported prefix-caching fixes (vllm#53798, vllm#54076). This is what shipping a custom speculative-decoding head costs the ecosystem.

tbench.ai reports Gemini 3.1 Pro (Terminus 2) at 70.7% on Terminal-Bench 2.1 with the same agent harness Victoria used. Victoria's 70.0% avg@3 sits right there — from a 48 GiB single-GPU model, per the vendor.

Caveats

• Every number above is self-reported by the team on a loaned machine. There is no independent replication, and the card itself flags the GGUF Terminal-Bench figure as a single noisy run (standard error of roughly 5 points).
• The NVFP4 and GGUF builds are different checkpoints; their scores are not comparable to each other.
• Maple's grading is AI-judge-only — the card says "no human grading has been done yet" — and the 6.0%→62.9% / 47.2%→23.7% / 2.9%→1.0% figures exist only in the Reddit post, not the model card.
• The llama.cpp integration is unfinished: mainline cannot load the GGUF, the fork's CUDA performance is unmeasured, and vLLM requires backported patches.
• REAP's results depend on calibration data; Victoria's calibration set is not disclosed.
• Both models ship under the Qwen Community License 1.0; MaaS and AI Work Assistant commercial use requires a separate license from Qwen.

Sources

• Reddit announcement (rmonsurate)
• Victoria model card
• Maple model card
• REAP paper (arXiv:2510.13999)
• Cerebras REAP blog post
• NVIDIA: Introducing NVFP4
• NVIDIA: NVFP4 trains with 16-bit precision at 4-bit speed
• Qwen3.8-Flash-Next model card
• Terminal-Bench 2.1 release notes
• Terminal-Bench paper (arXiv:2601.11868)
• rmonsurate/llama.cpp fork