AWS's Strands Decider 2B drops the LM head for a pointer head: 115 ms agent decisions, Apache 2.0

On October 1, AWS's Strands Labs released Strands Decider 2B, an Apache-2.0 model that removes text generation from the agent decision loop: instead of predicting tokens, it scores a bounded set of candidate answers in a single forward pass and returns calibrated probabilities, with a median latency of 115 ms on an RTX 3090. The release landed the same day Cloudflare shipped its own decision-model pair, Clef and Clef-flash, two weeks after TypeSafe AI's Jev launched the category — turning "System One models" from one vendor's idea into an infrastructure race.

What shipped

Strands Decider 2B is a 1.9B-parameter model built on Alibaba's Qwen3.5-2B-Base. The Strands team removed the language-modeling head — the component that predicts the next token — and replaced it with a pointer head of just over a million parameters. The head scores each candidate option by a dot product between the model's hidden state at a designated <answer> position and the hidden-state representation of each option's final token, then applies a masked softmax over the options. No generation loop, no sampling, no beam search: the model reads the input once and the answer falls out of a single forward pass. The torso is fine-tuned with a rank-16 LoRA adapter.

Three question types share the same masked softmax: noul (yes/no, returns a probability between 0 and 1), choice (select one of N), and score (place the input on an ordered rubric). Because the head has no per-option parameters, nothing caps how many options a question can carry (up to the schema's 255 limit), and label sets are defined at inference time rather than baked into the weights — you can change the options you offer without retraining.

This is the second major iteration: an earlier "slot head" that mapped the final hidden state to fixed output slots performed significantly worse, and the repo's research logs preregister every run's predictions and failure conditions before training.

The release is fully open. Weights are on Hugging Face at StrandsAgents/strands-decider-2B-hobson-v19 under Apache 2.0; the code, training recipe, data inventory, and evaluation harness are in the strands-labs/strands-decider repo; and pip install strands-decider gives you a CLI plus an HTTP server. The bundled server binds to 127.0.0.1 with no authentication — fine for local experiments, not for shared infrastructure.

On latency, the repo reports a median of 115 ms (p95 299 ms) for one JevBench question on an RTX 3090 under WSL2, and a warm median of 234 ms on an M3 Pro (153 ms for small tasks, per the announcement; p95 2,628 ms). Latency grows roughly linearly with task size. Training is cheap enough to reproduce: about 11 hours on a single RTX 3090, or about 1 hour 10 minutes on eight H100s with the fast flag.

On accuracy, the self-reported numbers: 167 of 231 tasks (0.723) on JevBench v1's public set at the pre-registered 3072-token window, with a Brier score of 0.342 and expected calibration error of 0.052; 168/231 at the 4096-token serving default. The tier split is 1.000 on easy tasks, 0.875 on standard, 0.505 on hard. On the JevBench v1.4.2 board of September 25, that placed v19 50th of 89 systems overall, 3rd of 33 in the 2B class, and 1st of 30 excluding three models just over 2B parameters.

The reference integration shows the intended pattern: a before_tool_call intervention in a Strands agent fires two yes/no questions before any external tool call — are the argument values grounded in what the user actually said, and is it premature to call this tool now — and maps the answers to typed actions (Proceed, Deny, Confirm, Guide). The point, as the announcement puts it, is that a decision this cheap can sit in a path where an LLM call never could.

Why it matters

The category hardened fast. TypeSafe AI launched Jev on September 15 as a closed hosted API with an undisclosed architecture, coining the "System One models" label. On October 1, Cloudflare shipped Clef (27B, from a frozen Qwen3.8-27B) and Clef-flash (9B) on Workers AI with Apache-2.0 weights, a vision encoder, and a 64K context window, claiming the lead on the Jev Decision Index and a median latency of 209 ms versus Jev's 524 ms.

What distinguishes Strands Decider is not the scoreboard but the openness. Jev's architecture is undisclosed and its API is closed; Cloudflare's Clef needs 41–85 GB of VRAM to run locally and its training data is not public, per The Register. Strands publishes weights, the full training recipe, the data inventory with source licenses, and the evaluation harness — and the model retrains on a single consumer GPU. For teams that cannot route traffic to a hosted API, or need to run air-gapped, that is the difference between evaluating a category and adopting it.

The engineering pattern is the durable part. As the Strands announcement argues, a large share of decisions inside an agent loop — tool selection, argument-grounding checks, triage, guardrails, evaluation, policy classification — does not need language generation at all. Routing each through a frontier API is slow and expensive by construction. A decision model that returns calibrated probabilities in one forward pass can sit on the hot path, with the confidence score used as a gating signal: act above a threshold, escalate below it. And because the model reads the context once, asking several questions about the same input is nearly free — each question appends only its own tokens.

Caveats

Every headline number above is self-reported by Strands, and no independent evaluation of Strands Decider 2B beyond JevBench was found. Cloudflare's own 10-task Decision Index shortlist does not include it, so the two vendors' claims are not directly comparable.

The leaderboard placement is within retrain noise. Six retrains of the v17 recipe scored between 159 and 167 of 231 (mean 163, SD 3.2); the repo notes that one retrain moves JevBench public by about three tasks and the difference between two runs by about 4.5. "3rd of 33" and "1st of 30" describe a checkpoint whose margin over the nearest 2B-class competitor's board entry is three tasks — and Mapika's newer decider-2b v11 scores 175/231 on Strands' own harness, eight tasks ahead of v19.

There is also a class-wide reliability question that predates this release. A September arXiv paper found that on 1,200 workflow decisions, renaming options from 0/1 to no/yes changed 70.4 more answers per hundred (95% CI [67.6, 73.1]) and dropped AUC from .94 to .23 — the models follow the option name, not the rubric bound to it. The study covered Jev and two Jev-like open-weight models, not Strands Decider specifically, but the mechanism is architectural, and it is the kind of thing to test before trusting a decision model's probabilities in production.

Strands' own limitations section is unusually candid: the hard tier sits at 0.505, long multi-step documents are the weak spot (long_policy 0.368, temporal_numeric 0.267), questions are read less carefully than documents, score and noul transfer poorly to rubrics unlike the training mix, and calibration is one temperature per primitive fitted on held-out short classification. The latency figures are also narrower than the headline suggests: the 115 ms median is v19 on an RTX 3090 under WSL2, the latency graph was measured on v18, and the M3 Pro's p95 is 2,628 ms.

Sources

• Introducing Strands Decider 2B — Strands Agents (AWS)
• Strands Decider 2B model card — Hugging Face
• strands-decider repository — GitHub
• Evaluation results: latency, retrain noise, Brier/ECE — repo
• External benchmark: JevBench v1, board position and tiers — repo
• Clef and Clef-flash announcement — Cloudflare
• Introducing System One Models and Jev — TypeSafe AI
• Cloudflare tries to outplay Jev with open-weight Clef models — The Register
• Type-Safe Is Not Error-Free — arXiv:2609.26758
• AWS Releases Decision Model for AI Agents That Routes Without Generating Any Text — TechTimes