Four $120 ex-mining boards run Qwen3.8-Flash-Next at 70 tok/s with 100k context — a $500 llama.cpp Vulkan/RPC cluster

Four AMD BC-250 ex-mining boards — roughly $500 of hardware — run Qwen3.8-Flash-Next at ~70 tok/s with 100k context, or two instances of Qwen3.6-35B-A3B at ~145 tok/s with 256k context, per the third update in the author's r/LocalLLaMA cluster series. The cluster runs llama.cpp's Vulkan backend with RPC multi-node inference, and the techniques behind those numbers transfer to any multi-node llama.cpp setup.

What shipped

The hardware. The AMD BC-250 is a compact motherboard built around AMD's "Cyan Skillfish" APU — a cut-down PlayStation 5 APU originally sold for cryptocurrency mining. Per the community documentation, each board has 6 Zen 2 cores at ~3.5 GHz, 24 RDNA2 compute units (1536 shaders), 16 GB of shared GDDR6, and a 220 W TDP, with Linux-only GPU drivers. The author runs four boards in the ASRock 4U12G case they shipped in, connected by 2.5 GbE adapters to a 2.5 Gb switch. Boards cost under $100 each when bought; the author notes similar boards now list around $150 on AliExpress, and Tom's Hardware reported the board resurfacing at $120.

The models. "Next Flash" is Qwen3.8-Flash-Next, the 125B-parameter MoE (6B active per token, 512 experts, plus a 51B n-gram embedding and a 4B MTP module) that Qwen describes as an early preview of the Qwen4 architecture. The GGUF totals ~177B parameters (125B main + 51B n-gram, excluding the separate MTP file); unsloth's UD-IQ3_XXS quant is 82 GB. The second model, Qwen3.6-35B-A3B, is a 35B-total / 3B-active MoE with a hybrid architecture mixing Gated DeltaNet linear attention with standard gated attention, at 262k native context. Four 16 GB boards give 64 GB of aggregate memory — the 82 GB quant only fits because Flash-Next's n-gram embedding layer can be offloaded to host memory or SSD, per unsloth's docs.

The techniques. All performance numbers below are the author's own, self-reported:

• MTP speculative decoding chained across boards. Qwen3.8-Flash-Next ships a multi-token-prediction draft head; llama.cpp documents --spec-type draft-mtp as a speculative-decoding implementation. The author runs the draft head as a chain of passes: pass A verifies the known token, then passes of two drafts each follow one board behind (six drafts, four passes), and a round ends at the first rejected draft. Adding the third pass took the Q2_0 build from 61 to 68 tok/s; six drafts took an IQ3 temperature-0 edit request from 84.0 to 90.1–90.5 tok/s.
• Recorded and replayed Vulkan graphs. Workers record each stage graph's Vulkan commands once and replay them, saving 3.7–4.5 ms of host time per graph (+2%); remote graphs' nodes are regrouped so a layer's independent products share a barrier group (+5.7%).
• Exact kernels for this architecture — routed-expert top-k, two-token expert passes, recurrent-state fusion — plus kernels for the parts that grow with context: +7.3% at 62K, then 66.1 → 68.9–69.6 tok/s.
• Context handling. Context checkpoints are copied on the boards instead of through the coordinator (665 → 18 ms each, prompt speed +60%), and prompt batches are pipelined through the boards. Usable context went from 65,536 to 100,096 tokens.

The same work carried over to the 35B: the Flash-Next kernels, graph replay and transport gave +39% to +66% over baseline; a four-draft MTP chain plus attention that decodes a K/V tile once into shared memory took a short-prompt request from 105.8 to 116.0 tok/s and a 44K-context request from 73.0 to 95.2; usable context went from ~100K to 258K of 262,144. A repeated request's prompt time at 99.6K dropped from 932–953 ms to 142–163 ms, and a pair now starts in 32–35 s instead of 105 s.

Exactness. The author reports weights, quantization and sampler settings are untouched; at temperature 0, speculative rounds are verified bit for bit against plain decoding of the same build by hashing every logits row (five depths up to 99.6K on the 35B). Against stock llama.cpp, logits differ at float-rounding level (~0.03) where kernels were rewritten.

Power and heat. Under load the cluster draws 600–700 W running Flash-Next and 800–900 W running two 35B instances. The author reports thermal throttling as the boards are better utilized and plans AIO coolers — an honest failure mode consistent with the community's experience (the wiki notes the overclocked 40-CU config hits 96 °C and requires upgraded cooling; one cluster write-up measured ~85 °C single-node).

Why it matters

The headline number is the cost. Four boards at ~$120 each is a 64 GB aggregate-VRAM inference cluster for about the price of one mid-range GPU, and it runs a 125B-parameter MoE (plus 51B of n-gram embeddings) at usable speed with 100k context. That changes the floor for what a hobbyist or small team can self-host.

The techniques matter more than the specific rig. MTP-as-draft-head is a documented llama.cpp feature, and running the draft chain one board behind the verifier is a useful pattern for any multi-node speculative-decoding setup. Recording and replaying Vulkan command graphs, and copying context checkpoints on-node instead of through the coordinator, are both transferable to llama.cpp RPC clusters generally.

This is not an isolated claim. The BC-250 community has independently documented the same ballpark: a two-node cluster running Qwen3-30B-A3B-Q4_K_M hit 77.77 tok/s generation, and a llama.cpp fork with custom Vulkan kernels for the BC-250's gfx1013 GPU reports +48.6% generation on a 9B model. The author's numbers sit inside a verifiable, fast-moving ecosystem that has turned ex-mining e-waste into distributed local inference.

Caveats

• Every performance number in this post is one practitioner's self-report. Nothing here has been independently replicated, and the exotic techniques — command-graph replay, on-board checkpoints, the MTP chain across boards — are the author's own and unverified.
• The Reddit thread itself could not be fetched (Reddit returns 403 to non-browser clients), so the "third update" framing and the earlier updates in the series could not be independently confirmed.
• MTP support in llama.cpp is in flux: unsloth's docs point to a custom llama.cpp PR for MTP, while upstream docs now list draft-mtp; which build the author ran is not stated.
• Ex-mining hardware is a lottery: one three-node write-up had 1 of 4 boards DOA. llama.cpp RPC is unauthenticated and should not be exposed to untrusted networks.
• Board prices are rising as the community grows — ~$120 at the time of the coverage, ~$150 on AliExpress per the author — so the "$500 cluster" economics may not hold for long.

Sources

• r/LocalLLaMA: Running Next Flash IQ3_XXS at ~70 tok/s with 100k context ... on $500 of ex mining BC-250 boards
• AMD BC250 documentation (community)
• Qwen3.8-Flash-Next (official repo)
• unsloth: Qwen3.8-Flash-Next GGUF and MTP docs
• Qwen3.6-35B-A3B (OpenRouter model page)
• llama.cpp README (Vulkan and RPC backends)
• llama.cpp server README (--rpc flag)
• llama.cpp speculative decoding docs (draft-mtp)
• 4claps/bc250-llama-cluster (two-node results, thermal notes)
• TechMakesArt/llama.cpp-bc250 (custom Vulkan kernels)
• Three-node BC-250 cluster write-up (ciroluciotecce.it)
• BC-250 AI inference wiki
• Tom's Hardware: BC-250 board resurfaces for $120