Richland Technology
Richland Technology
← Back to Blog
llmopen-sourceresearchon-devicespeech

Haru-4B: an open full-duplex speech model that swaps discrete audio tokens for single-step continuous latents

An Apache-2.0 base model that listens and speaks at once — 64-dim continuous latents at 7.5 Hz, one-step MeanFlow sampling, no discrete audio tokens. No benchmarks published yet.

Richland AI Desk·15 November 2026·4 min read
Haru-4B: an open full-duplex speech model that swaps discrete audio tokens for single-step continuous latents

higurashi-ai has released Haru-4B, an open, Apache-2.0 full-duplex spoken-dialogue model that listens and speaks simultaneously, built on a Qwen3.5-4B text backbone. Its new claim is architectural: it replaces the discrete audio tokens and separate acoustic transformer used by prior full-duplex models such as Moshi and FLM-Audio with 64-dimensional continuous audio latents at 7.5 Hz, generated in a single sampling step by a small MLP head.

What shipped

Haru-4B is a pretrained base model for research, released under Apache 2.0 at 5B params in BF16 (model card). It processes three synchronized streams — incoming user audio, outgoing model audio, and model text — and sums their embeddings into a single position of the shared language model, so listening and speaking never wait for each other. The vendor says this cuts context length to a third versus token-by-token interleaving (technical blog).

The architecture is consistent across the blog, model card, and GitHub README:

  • Qwen3.5-4B text backbone, 32 layers, hidden size 2560.
  • VibeVoice causal acoustic codec at 24 kHz, 3200 samples per latent.
  • 64-dim continuous audio latents at 7.5 Hz (~133 ms of speech per token) — no RVQ discrete codebooks.
  • 4-layer MLP audio head trained with Flow Matching then MeanFlow distillation, producing each acoustic token in a single sampling step at inference, with guidance scale 1–2 (default 1.5). No iterative flow sampling, no acoustic Transformer.
  • Chunk-level text alignment: text is represented in chunks with padding positions and a dedicated end-of-padding token for predicting speech initiation.

The result is just 7.5 autoregressive audio steps per second, which makes real-time full-duplex interaction feasible on local hardware.

The vendor reports a per-stage int8 latency breakdown on an Apple M1 Max MacBook (total p50): input 5.3 ms, one model step 34.2 ms, socket/handoff 4.1 ms, output wait 13.3 ms, output 8.1 ms — 65.2 ms total, excluding audio framing, which adds up to 133 ms at the 7.5 Hz token rate (technical blog). On an NVIDIA B300 via vLLM-Omni, audio encode/decode plus one LM step takes about 15 ms.

Training used over 40 million hours of real speech across 160+ languages (nine primary), explicitly excluding LLM-generated and TTS-synthesized dialogue. Full-duplex data came from 13M+ hours of separated conversational recordings, with samples up to 60 minutes long, produced by the vendor's FastDialogueSidon joint separation-and-restoration model. Pretraining ran on 256 NVIDIA B200 GPUs for about 8 days on Megatron-LM (technical blog).

Code is on GitHub: a PyTorch implementation with a web demo, an MLX build for Apple silicon, and a vLLM-Omni plugin. It runs on roughly 12 GB of NVIDIA VRAM. Character is set by a text card and voice by a 3-second-to-3-minute reference clip.

Why it matters

The departure from discrete tokens is the thing to pay attention to. As the vendor's blog notes, most spoken-language models rely on discrete audio tokens produced through RVQ codebooks; Moshi and FLM-Audio pair those with a separate acoustic transformer to turn LM hidden states into speech. Haru collapses that into a single MLP head doing one-step MeanFlow sampling of continuous latents, cutting the audio side of the pipeline to a single autoregressive step per 133 ms frame. For anyone building local real-time speech, that is a materially simpler inference path than the discrete-token stack, and it is all open and Apache-2.0.

The practical hook is the latency budget. The vendor publishes a full per-stage breakdown rather than a single marketing number, which lets you see where the 65 ms actually goes — and one LM step is 34 ms of it, so the backbone dominates. That is the kind of number you can reason about when deciding whether to run this on-device.

Caveats

Almost everything quantitative here is vendor-reported and not independently reproduced. There are no published benchmarks at all — the only evaluation is a single qualitative demo video (an English investment-advice conversation). An independent writeup from explainx.ai confirms the specs but flags that no scores have been published and the latency and training-scale numbers are the authors' own. There is no head-to-head measurement against Moshi, FLM-Audio, or OmniFlatten — the comparison is architectural only.

Two caveats come from the vendor itself. First, the model was pretrained entirely on separated speech (FastDialogueSidon output), not native two-channel recordings, so separation errors and noise can remain, especially in overlapping speech — the vendor flags this in its future-work section. Second, the 65 ms figure excludes audio framing and assumes wired audio; Bluetooth adds 100 ms or more.

Also note: Haru-4B is a pretrained base model, not an assistant. It is not a reliable source of facts and does not follow instructions the way an instruction-tuned model does. It can imitate a reference voice, so only use voices you have the right to use. And the "Qwen3.5-4B" backbone is cited by the vendor.

Sources

Start a Conversation

Ready to build something like this?

Tell us about your project and we'll respond with a clear plan within one business day.

Tell Us About Your Project

Takes 3 minutes. We read every submission personally.