A 176.9B MoE in 29.6 GB of RAM: ISTA DASLab ships GSQ-RCO GGUFs for Qwen3.8-Flash-Next

ISTA DASLab — the group behind GPTQ — released GGUF quantizations of Qwen3.8-Flash-Next using two new methods, GSQ and RCO, plus an expert-pruned "Coder" build. DASLab reports the Coder build fits the 176.9B-parameter model in a 29.6 GB resident working set on a single 32 GB accelerator.

What shipped

Qwen3.8-Flash-Next is Qwen's sparse mixture-of-experts preview of the architecture that will underpin Qwen4: 512 routed experts per layer across 48 layers, 10 routed plus 1 shared expert active per token, 262k-token native context, and a vision encoder (Qwen model card). DASLab counts the model at 176.9B parameters, 354 GB at BF16 (main model card). The release has two parts:

• Four quantized GGUFs at 2.40–3.50 bpw (66.4–83.6 GB), plus the BF16 vision projector (main card).
• A "Coder" build with 256 of 512 routed experts removed per layer, retained weights still at 3.5 bpw, 58.4 GB total of which 29.6 GB must stay resident (Coder model card).

All benchmark numbers below are DASLab's own evaluations; the Coder build's were measured at xhigh reasoning effort (Coder card):

Build	bpw	Size	AIME25	GPQA-Diamond	LiveCodeBench v6	Zero-shot avg
BF16	16.00	354 GB	100.00	91.92	87.43	76.94
IQ3_S	3.50	83.6 GB	100.00	92.93	86.86	n/a
IQ3_XXS	3.00	75.8 GB	100.00	91.41	86.29	77.23
Q2_0	2.40	66.4 GB	96.67	89.39	81.14	78.00

The Coder build scores 75.60 on SWE-bench Verified against 82.80 for BF16 (91.3% retained) and 86.28 on LiveCodeBench v6 against 87.43 (98.7% retained) (Coder card). Both repositories ship the complete per-tensor RCO allocation (main repo, Coder repo).

Why it matters

GSQ (Gumbel-Softmax Quantization) is a post-training scalar quantization method that jointly learns per-coordinate grid assignments and per-group scales through a Gumbel-Softmax relaxation of the discrete grid, while staying compatible with existing scalar inference kernels (GSQ paper, code). That compatibility closes most of the gap between scalar and vector quantization at 2–3 bits without new kernels.

RCO (Riemannian Constrained Optimization) is the second method. Under a softmax relaxation, an exact size budget defines a smooth Riemannian manifold in logit space. RCO wraps tangent projection, binary-search retraction, and momentum transport around a standard Adam step, so the expected budget is enforced exactly at every iterate with no constraint-related hyperparameters (RCO paper, code). It does two jobs in this release: assigning a quantization type to every tensor under one exact size budget, and selecting which experts to keep in the Coder build — one exact budget per layer, because GGUF stores a single expert count for the whole model (Coder card).

The resident-memory trick is separate from both. Flash-Next's 51.2B-parameter n-gram embedding is a lookup table, held at a fixed IQ4_NL 4.5 bpw in every build. Because it is a lookup rather than a matmul weight, it can stay memory-mapped on disk and does not need to occupy accelerator memory (main card). In the Coder build that leaves only the 29.6 GB transformer shard resident (Coder card). The "1.89 bpw" headline is the joint effect of pruning and quantization averaged over the original parameter count — no individual weight is stored at 1.89 bits (Coder card).

The practical consequence, per DASLab: the 176.9B model runs on a single 32 GB accelerator. Per AlphaSignal's coverage, the GGUFs run unmodified in llama.cpp, Ollama, LM Studio, and vLLM (DASLab's own card lists llama.cpp, Ollama, and LM Studio). Early community speed tests in the release thread report roughly 21 tok/s for IQ3_XXS and 26 tok/s for the Coder build on an RTX 3090.

Caveats

• "Matches BF16" is loose. IQ3_S is 0.57 points below BF16 on LiveCodeBench v6 (86.86 vs 87.43), and the card itself notes that recoveries slightly above 100% reflect benchmark variance, not a model that beats its source (main card).
• All accuracy numbers are vendor-reported. These are DASLab's own evaluations; we are not aware of independent replication.
• The Coder build is experimental. The authors say capabilities not represented in the calibration mixture degrade, and that "degradation outside this set is an accepted cost" (Coder card). Early community testing in the release thread reports poor output in niche languages such as Delphi and C++, consistent with that blind spot.
• 29.6 GB resident assumes the n-gram shard stays on disk. Users with limited RAM report heavy paging when it does not.
• The 176.9B count is DASLab's. Qwen describes the model as 125B parameters with 6B activated plus a 51B n-gram embedding and a 4B multi-token-prediction module; the exact composition of the 176.9B total is not stated by either party.

Sources

• ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Hugging Face)
• ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF (Hugging Face)
• Release announcement (r/LocalLLaMA)
• Qwen/Qwen3.8-Flash-Next (Hugging Face)
• GSQ paper (arXiv:2604.18556) · GSQ code
• RCO paper (arXiv:2605.00649) · RCO code
• AlphaSignal: DASLab squeezes a 176.9B coding model down to 32GB hardware