A 1.9B decision model learned to play Doom in 43 minutes of LoRA on an M5 Pro

A developer fine-tuned Liquid AI's d1-2B, a 1.9B-parameter "decision model," to play ViZDoom's Defend the Center from raw game frames on an Apple M5 Pro Mac — a 43-minute LoRA fit behind a 7-minute readout fit, using his new MIT-licensed mlx-decision tool. On 510 held-out frames, crosshair-aim precision went from 0.79 to 0.92, and in play the model averaged 23.2 kills over 20 episodes.

What shipped

mlx-decision is a tool for "run and fine-tune decision models locally on Apple Silicon with MLX," MIT-licensed and pip-installable (pip install mlx-decision; requires Apple Silicon, macOS 14+, Python 3.12+). Version 1.3.0, released 2026-11-08, added "images in training data" and a lora kind of fit: low-rank adapters on the MLP projections of every layer, with --rank/--learning-rate defaulting to 8 and 1e-4 and --epochs defaulting to 2.

The "decision model" framing is the whole point. From the README:

A decision model reads a state (a message, a document, a screenshot, any JSON) and a set of typed questions: pick one of several options, rate on a scale, or answer yes or no. It returns a probability for every allowed answer in one pass over the input, without generating text, so an answer is always one you allowed and comes in a fraction of a second.

So instead of decoding tokens, the model outputs a probability per allowed answer — classification, not generation. d1-2B is Liquid AI's 1.92B-parameter decision model built on the LFM3-VL-2B vision-language base, with 128K context and the LFM Open License v1.0 per its model card. The Studio below shows the workflow: an input, typed questions, and probability scores from several models side by side.

The Doom recipe, as reported by the author in the r/LocalLLaMA post:

• Three typed questions per frame: is a monster right at the crosshair so a shot now would hit (yes/no); where is the nearest monster — left, centre, right, none; is a monster dangerously close (yes/no).
• Training data from game-engine ground truth: ~3,000 frames from ViZDoom's Defend the Center arena, labelled automatically from the engine's own monster positions — no human labeling.
• Two-stage fit: a readout fit first (7 minutes), then LoRA on the model's text layers with the vision tower frozen — 43 minutes, 2 epochs, on an M5 Pro Mac. The repo's own benchmarks run on "an Apple M5 Pro (20-core GPU, 64 GB)" per its TRAINING.md.
• A Python loop turns answers into moves — turn toward the monster, shoot when a shot would hit.

The numbers are all from a 510-frame held-out set: "a shot would hit" precision 0.79 → 0.92, nearest-monster classification 92.7% → 96.8%, dangerously-close 93.9% → 98.3%. In play, the model averaged 23.2 kills over 20 episodes (best clip: 26 bullets, 26 kills) and 19.7 on Freedoom, whose monster sprites never appeared in training.

Why it matters

The demo is a template for a cheaper kind of agent. Defend the Center is a turn-only arena (TURN_LEFT, TURN_RIGHT, ATTACK) — reflex-level play, not navigation, and that's what makes it tractable. The recipe is supervised end to end: a readout fit plus LoRA on frames the game engine labelled for free, no RL, no reward engineering, under an hour on one laptop. The transferable part is the shape — any stream that reduces to a few typed questions with ground-truth labels is a candidate for the same treatment, whether a game, a test harness, or a UI log. The piece worth stealing is the two-stage fit: readout first, then LoRA on the text layers with the vision tower frozen.

The second half of the story is upstream of Doom. mlx-decision sits in the "System 1" typed-decision-model wave that Perplexity, Cloudflare, vLLM, and Liquid AI are all shipping — each is a row in the tool's supported-models table — and it is listed on the official MLX community projects discussion. Running those models on MLX has a real cost advantage: a decision model returns every allowed answer in one forward pass instead of one token at a time. That is why the full loop — frame in, three answers out — runs at about 0.1 s per decision on one laptop, and why Liquid AI benchmarks its d1 models on an Apple M5 Pro in its "edge inference" section.

Caveats

• Every Doom number is self-reported. The repo contains no Doom script, benchmark code, or checkpoints, and as of writing no third party has published a reproduction of the fit or its metrics. Treat 23.2 kills and the 0.79 → 0.92 precision gain as the author's report, not a replicated result.
• It is not full-Doom play. The arena has no movement and no navigation; an independent study of typed-decision-model actors in a different Doom map (arXiv:2610.09683) reports no variant reaching the exit. Generalization is real but bounded: kills drop from 23.2 on familiar sprites to 19.7 on unseen Freedoom sprites.
• The license is split. The tool is MIT, but d1-2B ships under the LFM Open License v1.0, and the mlx-decision README flags it as "commercial use limited by its licence."
• Question framing is load-bearing. The model's answers depend on which questions are asked together — reword or reorder the three questions and the policy could change. The 0.1 s/frame figure is the author's measurement of the full loop, not the model's raw latency.

Sources

• mlx-decision repository (README, v1.3.0)
• mlx-decision CHANGELOG — v1.3.0, 2026-11-08
• mlx-decision on PyPI
• d1-2B model card (Liquid AI)
• Liquid AI — Open d1: the d1 decision model family
• Liquid AI — LFM3 launch
• mlx-decision TRAINING.md (M5 Pro benchmarks)
• ViZDoom Defend the Center scenario config
• ViZDoom (Farama Foundation)
• Apple — M5 Pro and M5 Max MacBook Pro announcement
• MLX documentation (Apple)
• MLX Community Projects discussion (lists mlx-decision)
• arXiv:2610.09683 — "System Switch: When Should a Fast Decision Model Stop and Think?"
• r/LocalLLaMA — the original post