ThinkingBox turns its database-state graders into GRPO rewards: Qwen3.8-27B's all-20 task count triples from 38 to 107

Microsoft turned ThinkingBox's own database-state graders into GRPO rewards. The fine-tuned Qwen3.8-27B nearly tripled the tasks it completes flawlessly on all 20 attempts — from 38 to 107 of 507 — and pass@1 rose from 51.70% to 60.78% on a benchmark the model never trained on.

What shipped

ThinkingBox-Bench is Microsoft's 507-task benchmark of stateful business workflows — retail, auto insurance, travel, neobank IT, and consulting. Each task provisions a real backend, gives the agent MCP tools and a simulated user, and grades it with executable checks against the terminal database state and side effects once the agent declares itself done (benchmark blog). An earlier paper (arXiv:2608.19741) showed that even frontier models rarely reproduce their best trajectories: Claude Opus 5's pass@1 is 66.50% — the share of successful attempts — but only 47.53% of tasks pass on all 20 tries.

The new joint Microsoft/Hugging Face blog post (Oct 22) closes that loop by using the same checks as training rewards and shipping the recipe as an open-source repo (thinkingbox-training, MIT). The documented Qwen3.8-27B reference run is a full-parameter, token-mean GRPO setup on VeRL 0.9.0: 24 H100s across three nodes, 18 groups of 8 rollouts per update, learning rate 1e-6, KL regularization disabled, 50 training steps (reference-run guide). The 187-task training pool was selected to be disjoint from the 507-task benchmark, which stayed fully held out.

Results across the same 507 × 20 evaluation campaigns (training blog):

Metric	Base Qwen3.8-27B	RL fine-tuned
pass@1 (10,140 attempts)	51.70%	60.78%
Tasks passed on all 20 attempts	38 (7.50%)	107 (21.10%)
Tasks passed at least once in 20	89.35%	87.97%

The 20/20 column is a literal tally — tasks that passed all 20 recorded attempts, "no estimator, no smoothing," as the benchmark post puts it. The fine-tuned model's 107 dependable tasks beat GPT-5.6 Sol (82), Claude Sonnet 4.6 (102), and Kimi-K3 (68) on that measure, though GPT-6 Astra (231) and both Claude Opus models (241) stay well ahead (training blog, benchmark blog).

The qualitative picture matches. In the hotel-booking example the blog walks through, the base model changed a booking and issued refunds without manager approval in 15 of 20 attempts; after fine-tuning, it preserved booking and payment state in all 20, with all expected records matching in 19 (training blog).

Why it matters

First, the reward is outcome-verified. Most rewards are deterministic checks of terminal backend state; a minority hand a narrow binary rubric to a judge; the simulated user never grades (training blog, repo README). There is no learned reward model to train — the oracle is the same executable check the benchmark uses. The real engineering problem is task selection.

GRPO contrasts rollouts within a group. If all 8 rollouts for a task succeed, or all fail, the group's reward variance is zero and the update carries no signal. Worse, each rollout is expensive: it spawns an isolated backend, runs real tool calls, and drives a simulated user.

So Microsoft probes first. The team fits a Jeffreys posterior — a Beta(k + ½, n − k + ½) over each task's probe outcomes — and keeps only the middle difficulty bands, dropping tasks estimated below ~8% success as hopeless and above ~90% as already solved, per TechTimes' account of the paper. A GPT-5.4 solvability screen removed tasks no frontier model could approach, and zero-variance groups were re-sampled rather than scored as useless updates. The repo docs describe the same mechanism: "classify policy-relative difficulty with Jeffreys posteriors, apply an optional stronger reference-model solvability gate, exclude benchmark UIDs" (data-preparation.md).

Probe-before-you-train generalizes: with a large candidate pool, the probe histogram shows how much usable training signal exists before you spend rollout budget on it.

Second, the result is a tradeoff, not a free lunch. Pass@1 and 20/20 rose in all five domains, but pass@20 — solving a task at least once in 20 tries — fell from 453 to 446 tasks. Auto insurance lost the most breadth (−7.0 points); retail gained the most consistency (+24.5 points on 20/20). The authors call it sharpening: probability mass concentrated on repeatable trajectories at the cost of rare solution paths (training blog, TechTimes). Teams that deploy for consistency get exactly what they want; teams that need coverage should watch pass@20.

The cost table sharpens the case. Microsoft's cost-per-dependable-task index — a serving-efficiency figure from list prices, explicitly "not an invoice" — puts the fine-tuned 27B at about $8.00 per task completed on all 20 attempts, down from $24.36 for the base model (−67.2%). That lands near GPT-5.4 ($6.80) and Claude Opus 5.5 ($7.80) and far below Kimi-K3 ($20.68) (benchmark blog). It is the argument for small, self-hostable models in regulated or cost-sensitive agent deployments.

One operational warning worth lifting verbatim: training and evaluation must interpret the model's output identically — reasoning extraction, tool calls, argument typing, task state — "otherwise the reward measures a different agent from the one evaluated." The repo ships shared interpretation code (trainer/core/tb_interpret.py) to enforce that parity (training blog).

Caveats

Everything above is Microsoft/Hugging Face-reported: one disclosed run, one model, no seed sensitivity, no independent replication. The benchmark is held out, but training and evaluation share the framework, simulated-user machinery, and some tools, which limits the out-of-distribution claim. The exact 187-task pool is not published — only 39 public development tasks ship — so exact reproduction from public artifacts is impossible. The $8.00 figure excludes training and probing costs.

The numbers also moved between publications. The arXiv v4 paper reports Kimi-K3's all-20 success as 17.60% (abstract) and Qwen base as 10.93% (introduction); the blog and repo now use the literal counts (68/507 = 13.41%, 38/507 = 7.50%) after re-aggregation with the public tb agg tool (repo README). The recipe also depends on a proprietary model — GPT-5 Chat served as simulated user and judge during training, so replicating it requires OpenAI API access.

For context on why dependability outranks average accuracy: in April 2026, a Cursor agent running Claude Opus 4.6 deleted the PocketOS production database in a single Railway API call, taking backups with it; the most recent recoverable backup was three months old (The Register, Decrypt). An average pass@1 would not have flagged that agent as unfit for production; a repeated-trial reliability check would have.

Sources

• Microsoft/Hugging Face: ThinkingBox Training blog post
• Microsoft/Hugging Face: ThinkingBox benchmark blog post
• arXiv:2608.19741 — One Success Isn't Reliability (v4)
• microsoft/thinkingbox-training repository
• Reference Qwen3.8-27B run guide (repo docs)
• Task preparation and runtime hydration (repo docs)
• TechTimes coverage
• The Register: PocketOS production database deletion
• Decrypt: AI agent deletes startup database in 9 seconds