Richland Technology
Richland Technology
← Back to Blog
llmopen-sourceagentsresearchreinforcement-learning

OpenEnv's capture proxy turns unmodified coding harnesses into RL environments: a 2.6B model trained across four harnesses beats SFT on a 27B teacher

Hugging Face and Liquid AI train LFM2.5-2.6B inside Claude Code, Codex, OpenCode, and Mini-SWE-Agent without touching harness code — 42.2% → 54.2% pass@1, 31% fewer tool calls, seven open models.

Richland AI Desk·5 October 2026·7 min read
OpenEnv's capture proxy turns unmodified coding harnesses into RL environments: a 2.6B model trained across four harnesses beats SFT on a 27B teacher

Hugging Face's TRL team and Liquid AI published an open framework that turns unmodified coding-agent harnesses into RL environments, and used it to train LFM2.5-2.6B across four harnesses at once: pass@1 on held-out tasks rose from 42.2% to 54.2%, tool calls fell 31%, and the RL runs beat supervised fine-tuning on a 27B teacher's rollouts. The guide went up October 1; Hugging Face CEO Clement Delangue announced it on October 5.

What shipped

The release is a full stack, not a paper:

How the capture proxy works

The core move is a proxy, not a rewrite. A harness finds the proxy the way it would find any model provider: a base URL and an API key. The proxy speaks the four API dialects coding agents use — OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, Gemini — converts each request to chat completions (with converters vendored from NVIDIA's Polar gateway), forwards it to vLLM, records the exact token IDs and per-token log probabilities, and replays the answer in the caller's own format. The team reports ten harnesses run through it unmodified (framework chapter).

Token fidelity is the whole point. As the basics chapter puts it, quoting TRL's Tito write-up: "in RL, you optimize on the exact tokens the model produced." Re-tokenizing saved text is a guess — the same characters can tokenize differently — and harnesses edit text as they go, repairing malformed JSON and adding role markers. In asynchronous RL the weights may have moved since a rollout was sampled, so recomputing log probabilities with current weights breaks importance sampling. The proxy asks vLLM for the sampled token IDs and a logprob for every sampled token, sampling from the full distribution: setting top_p to 1.0 moved the importance ratio from 0.985–0.993 to 0.9984–0.9999 in their runs. This needs two vLLM flags: --return-tokens-as-token-ids --logprobs-mode processed_logprobs.

Because harnesses retry, spawn subagents, and compact context, a rollout is rarely one clean conversation. The proxy stores each model call as a node and links it to the earlier call whose prompt shares the longest exact token prefix: a retry becomes a sibling that never continued, a subagent or compacted context starts a new root. Each root-to-leaf path becomes one training sequence, with context tokens loss-masked and sampled tokens as targets. Session IDs double as API keys (one proxy on one port serves every concurrent rollout), and hosted APIs like OpenAI, Anthropic, and HF Inference Providers can evaluate a harness but not train through it — they don't return the token-level data. The proxy is a single process, and its health check starts to starve at around 200 concurrent sessions and crashes at 320.

Why it matters

The problem the framework attacks is measurable. The same LFM2.5-2.6B weights solve 62% of SmolDataEnvs test tasks under Mini-SWE-Agent but only 33% under Claude Code, before any training. The effect is not specific to small models: Joel Niklaus measured GLM-5.2 at 23% in one harness and 52% in another on SWE-bench Pro, and the harness ranking doesn't even transfer between models — Codex was second-best of ten harnesses for GLM-5.2 and ninth for Gemma 4 26B-A4B.

Pass@1 against cost per task on SWE-bench Pro for GLM-5.2 and Gemma 4 26B-A4B, each run under several coding-agent harnesses. The harnesses that do best differ between the two models.

Training in one harness doesn't fix this — it makes it worse in a specific way. KwaiKAT's report names the failure: models learn "how to solve the task under that particular harness's interface conventions," overfitting to its action format, context structure, and control flow. The article's own runs show the pattern. Trained in OpenCode alone, LFM2.5-2.6B improved mostly in OpenCode (34% → 58%) and much less elsewhere. Trained across OpenCode, Claude Code, Codex, and Mini-SWE-Agent — one GRPO group of eight rollouts per task, with the harness chosen per group — it gained under all four: 42.2% → 54.2% overall, better than the OpenCode-only model under Claude Code (49% vs 42%) and Codex (54% vs 43%), and tied under Mini-SWE-Agent (training chapter).

The efficiency result is the more surprising one. The reward was correctness (1/0) plus a small bonus for correct answers that used fewer tool calls — at most 0.1, computed as 0.1 × 15/(15 + tool calls), with wrong answers still scoring zero. GRPO learns from differences within a group, so when all eight rollouts are correct the bonus is the only signal that separates them; that happened in 17.5% of OpenCode-only groups and 22.6% of multi-harness groups. At the final checkpoint the multi-harness model used 31% fewer tool calls than the base model on tasks both solved (11% for OpenCode-only), about half as many under Codex, and input tokens fell with them. The earlier Qwen runs, rewarded for correctness alone, saw tool calls per rollout creep from 13 to 41 — the bonus exists because of that.

SFT is the comparison that makes the RL case. The authors ran Qwen3.8-27B as a teacher through all four harnesses, kept 3,189 successful rollouts from 888 tasks, and fine-tuned LFM2.5-2.6B on them. The better SFT model (OpenCode-only) plateaus at 47.5%; multi-harness SFT reaches 43.1%; the RL runs reach 54.6% at their best checkpoint (data). Imitation was even net-negative in some cells: OpenCode SFT dropped Mini-SWE-Agent from 62.1% to 58.8%, and multi-harness SFT's first epoch collapsed OpenCode from 33.6% to 22.4%.

Frontier labs already train across harnesses on purpose — Kimi K3 builds harnesses from composable modules, Qwen3-Coder-Next generates agentic data in six harnesses, and Liquid AI picks a harness at random per task for LFM2.5-2.6B itself. What was missing was an open way to do it with any harness and any open model. This release is that path, and the seven models are downloadable today.

Caveats

Everything above is self-reported by the authors (Hugging Face's TRL team and Liquid AI); no one has independently reproduced the runs. The article itself flags the limits:

  • One seed per setup, and unequal exposure between runs — the OpenCode-only run saw 555 distinct tasks and 162M processed tokens, the multi-harness run 626 tasks and 451M — so the single-vs-multi comparison is observational, not causal.
  • The SFT-vs-RL comparison uses each RL run's best evaluated checkpoint (step 900 / step 700), which the authors note flatters RL slightly; the 1.9-point gap between the two RL runs at step 1,000 is within noise.
  • There was no LFM run without the efficiency bonus, so its contribution can't be separated from the rest of the recipe.
  • The token-faithfulness premise is itself contested: the article cites OpenForgeRL, which reconstructs training samples without token IDs or logprobs, as reporting the strongest multi-harness results published so far.

Sources

Start a Conversation

Ready to build something like this?

Tell us about your project and we'll respond with a clear plan within one business day.

Tell Us About Your Project

Takes 3 minutes. We read every submission personally.