Mirage: a 21.6 MB on-device VLM that hands a JSON record to text-only LLMs — Qwen3-0.6B gains VQA without fine-tuning
A 51.3M-parameter model emits a structured JSON record that any text-only LLM can read as ordinary text — no fine-tuning, no image embeddings.
Cactus Compute released Mirage, a 51.3M-parameter vision-language model that fits in a 21.6 MB file and runs fully on-device. Its most interesting result is not the model itself: feeding Mirage's structured JSON record to a text-only LLM grants it VQA abilities with no fine-tuning and no image embeddings. On 256 VQAv2 questions from 128 images, Qwen3-0.6B went from 9.3 with the question alone to 31.3 with Mirage's record and trace, measured as VQA consensus credit out of 100 (release post, experiment).
What shipped
Mirage is a single-pass vision model for phones, wearables, and robots. It takes one image and returns a structured JSON record covering scene description, detected objects with boxes and attributes, segmentation masks, OCR text, spatial relations, and a short reasoning trace — plus a "decision" mode that scores a caller-supplied set of choices. Descriptions are supported in English, German, French, Spanish, Italian, Dutch, and Polish (release post).
The headline numbers are vendor-reported. Cactus Compute measures 74 ms end-to-end on a Samsung Galaxy S26 NPU and 107 ms on its CPU. The 21.6 MB file comes from 2-bit "Cactus Quants": most quantised matrices use 2 bits, token-embedding and mHC routing matrices stay at 4 bits, and HConvGLU bands and other floating-point tensors stay at 16-bit (release post, LinkedIn).
The architecture splits into a 7.8M-parameter image encoder, a 40.8M language decoder, and 2.6M in the OCR and spatial heads. The encoder uses HConvGLU: fixed Walsh-Hadamard transforms surrounding three learned 65-channel banded convolutions, so each convolution stores 512×65 weights — about 8× fewer than a full 512×512 matrix — while the transforms still let all channels interact. Detection and masks share the image features; a CTC text reader handles detected text crops; and the 8-layer decoder reuses the "Simple Attention" and Hadamard-MLP family Cactus already ships in its Needle and Whistle models (release post). Masks come from 32 shared prototype masks on a 128×128 grid, and object/text proposals from two CenterNet-style centre heatmaps capped at 64 objects and 32 text regions (experiment).
The model is Apache 2.0. Weights and config are on Hugging Face at Cactus-Compute/mirage, runtime binaries at Cactus-Compute/cactus, and source at cactus-compute/cactus on GitHub (HF model, GitHub).
Why it matters
The composition result is the part worth building around. Cactus's argument is that a small model wastes capacity generating prose and open-ended reasoning; instead it emits a compact, structured record, and a separate text-only LLM reads that record as ordinary text. The two models never share weights or an adapter — the JSON is the interchange format (release post).
That is a different interface pattern from a monolithic VLM. Perception is decoupled from reasoning: a 22 MB on-device model does the vision, and any text-only LLM — including one that has never seen pixels — answers questions about the image. In the vendor's experiment, Qwen3-0.6B scored 9.3 on the question alone and 31.3 with Mirage's record and full trace, with no fine-tuning. A different image's record scored 22.2, and dropping the trace scored 33.1 — the structured observations helped, but the reasoning trace did not (experiment).
The trade-offs are explicit in the vendor's own data, which is what makes the claim credible. The record is lossy: Qwen never sees the pixels, so anything Mirage misses is unavailable to the reader. In one skateboard photo, Mirage missed the skateboard and described cars, people, and shoes; Qwen guessed "a skateboard" from the question alone, then answered "a car" after reading the record (Reddit thread). Larger text-only readers gave mixed results, and format compliance is a real failure mode: the experiment required a JSON object with a string-valued answer, and Nemotron broke that format on 214 of 256 responses, while Qwen3-1.7B broke it on 34 (Reddit thread).
For engineers, the practical takeaway is a cheap way to add vision to an existing text-only stack: run a tiny on-device perception model, hand its structured output to the LLM you already run, and skip the VLM fine-tune. The cost is that the answer is bounded by what the perception model chose to record.
Caveats
Everything above is vendor-reported. Cactus Compute's own blog, experiment page, and LinkedIn post are the only sources for the latency figures, the parameter counts, the 21.6 MB file size, and the VQA composition numbers; I found no third-party benchmark or independent replication. The experiment is documented on Cactus's site with runnable code, but it is not on arXiv, so there is no peer-reviewed version to point to.
The VQA scores mix answer correctness with format compliance, since responses that broke the required JSON format received zero credit. And the on-device latency is for Mirage alone — a reader LLM's weights are additional to the 22 MB.