One 180B MoE across five devices: two PCs, an M4 Pro MacBook, and two iPhones — 37–39 tok/s at 60k context via pipelined speculation
A practitioner chains five heterogeneous devices into a single Qwen3.8-Flash-Next instance, hiding cross-device latency with pipelined speculation — but the code isn't released yet.
On October 17, a practitioner (u/StayLameBro) reported running a single instance of Qwen3.8-Flash-Next — a ~180B-parameter mixture-of-experts model — across five heterogeneous devices: two gaming PCs, an M4 Pro MacBook, and two iPhone 17 Pro Maxes, at 37–39 tok/s writing with 60k context. The trick is "pipelined speculation": the model's multi-token-prediction draft head on PC 1 guesses ahead while the rest of the chain is still verifying the previous batch, so cross-device latency stops compounding.
What shipped
No single device in the house can hold the model, so the author split it across all five:
- PC 1 (RTX 2060 Super, 8 GB): layers 0–9, the output layer, and the model's multi-token-prediction (MTP) draft head.
- MacBook Pro (M4 Pro, 24 GB): layers 10–29.
- Two iPhone 17 Pro Maxes (A19 Pro): for layers 10–18, each token picks 10 of 512 experts; per the post, the Mac sends those picks over USB-C and each phone's GPU runs them, answering in about 1 ms, while the Mac keeps the most-used experts in its own memory and runs those itself.
- PC 2 (RX 9060 XT, 16 GB): layers 30–47.
The PCs and the Mac talk over Ethernet; the phones connect over 10 Gb/s USB-C. The author states every weight sits in VRAM, pinned RAM, or unified memory — nothing streams from disk and nothing swaps.
How the chain stays busy: five devices in a row normally means four idle while waiting for the previous stage. Instead, the draft head on PC 1 guesses the next few tokens, the whole chain verifies them in one pass, and while that batch is still travelling through the Mac, phones, and PC 2, PC 1 is already drafting the next batch. Each verification step lands about 2.2 tokens on average, per the post.
Measured at the server with what an agent actually sends (temp 0.6, top-p 0.95, top-k 20, medium thinking, real code prompts — not greedy), the author reports:
| Context | Writing | Prompt reading |
|---|---|---|
| 16k | 42 tok/s | 728–744 tok/s |
| 32k | 39–41 tok/s | 844–852 tok/s |
| 60k | 37–39 tok/s | 856–895 tok/s |
In the agent session from the video, the author reports a cached-prompt follow-up turn starts in about half a second; a turn with new tool output costs about 2.9 s fixed plus 768 tok/s for the new tokens. Writing averaged 35 tok/s across that session, with bursts past 40.
Long context stresses the layout. Per the post, the current split serves 69.6k tokens — PC 2's 16 GB is the ceiling. A second layout moves two layers onto PC 1 and keeps 8-bit KV on the Mac and phones; that one read a single prompt past 164k tokens and writes 31–33 tok/s at 60k. Full 262k requires each phone to hold one attention layer's cache, which the author says is how the two-A19 split works. The author also reports prompt reading caps at about 980 tok/s because PC 2's host-memory experts cross PCIe once per 512-token batch.
On correctness, the post claims every change is "exact by construction or exact in distribution," except the Mac's expert cache, which adds partial sums in a different order — and notes the KL check against a single-machine reference had not yet run.
Why it matters
The model is worth the effort. Qwen3.8-Flash-Next, released August 26, is a 125B-parameter main model plus 51B n-gram embeddings and a 4B MTP head — about 180B stored, 6B activated per token — with 48 layers, 512 experts (10 routed plus 1 shared active per token), and 262,144 native context. Even 3-bit quants run to 82–90 GB (Unsloth's UD-IQ3_XXS and UD-Q3_K_XL), so no consumer device holds it alone. The interesting engineering question is what you do with the devices you already own.
Pipelined speculation has prior art. SpecPipe (arXiv 2504.04104) fills a pipeline-parallel inference chain with speculative tokens step-by-step, aiming to decode one token per pipeline step, and reports 4.19×–5.53× over standard pipeline parallelism on an 8-stage setup. What this post adds is the same idea applied to a heterogeneous chain with disparate link speeds — Ethernet, 10 Gb/s USB-C — and phones as first-class workers rather than remote compute backends.
That is the architectural difference from llama.cpp's -rpc mode, which the repo's ROADMAP spells out: with -rpc, one host runs the whole model, the other machines are blocking remote backends, and phones can't take part. Backburner's design is that each device owns its layers and its own memory, only a few KB of hidden state cross the link per layer block, and phones run layers on their GPUs and hold old context. The phone's job here is narrower: it doesn't hold whole layers, just the 10-of-512 routed experts for a slice of the network — expert parallelism carved out of a layer range.
The repo also establishes the discipline the cluster claims will inherit: a "same answers" gate where every speed change must produce token-identical greedy output against a single machine. The currently released system — a Mac + iPhone running the dense Qwen3.8-27B over USB-C — documents 256/256 tokens identical with and without the phone, and 29–44% faster prefill at 16k–48k. That is the credibility machine the bigger claim is riding on.
Caveats
The headline numbers are not independently verifiable yet. The Reddit post says the cluster code is "coming to the repo shortly," and the repo's own ROADMAP lists "Qwen3.8-Flash-Next across a house" as in development, stating that "the code and measured numbers come to this repository with its release." What is public today is a different project: the MIT-licensed Mac + iPhone system for Qwen3.8-27B. All cluster benchmarks are self-reported by one practitioner, with no bench files and no independent replication.
The correctness gate is explicitly pending: the post says the KL check against a reference hasn't run, and the repo's own standard is token-identical output, which the 27B system demonstrates but the cluster does not yet. The specific numbers in the table above, the "exact by construction or exact in distribution" claim, and the long-context figures all come from the post and could not be independently confirmed — treat them as a single practitioner's self-report until the code and bench files land.
The memory math is not fully accounted for. Three-bit quants of this model are ~82–90 GB, and the claimed devices hold 8 + 24 + 16 GB of GPU/unified memory plus phone RAM. The post says nothing streams from disk; the ROADMAP says each device "keeps the hot experts in memory and reads the rest from its own SSD." Those two descriptions differ, and neither is backed by a released implementation.
Finally, the toolchain for this architecture is immature: the MTP draft head is a full qwen4exp block (attention + 512-expert MoE + hyper-connections, 4B parameters) that is "not upstream yet" in llama.cpp — running it requires a patched build. And the model's license is qwen-community-1.0, not Apache-2.0, which matters if you plan to ship derivatives.
Sources
- Reddit post: 2 gaming PCs + M4 Pro MacBook + 2 iPhones running Qwen3.8 Flash Next
- Qwen3.8-Flash-Next model card (Hugging Face)
- Qwen3.8-Flash-Next official repo (release notes)
- Backburner repository
- Backburner ROADMAP (Flash-Next cluster status)
- Unsloth: Qwen3.8-Flash-Next quantization sizes
- Qwen3.8-Flash-Next MTP draft head GGUF (llama.cpp upstream status)
- SpecPipe: Accelerating Pipeline Parallelism-based LLM Inference with Speculative Decoding (arXiv 2504.04104)