Train the harness, not just the policy: HarnessAdapt-RL lifts a 9B agent 9.8–12.6 points on SWE-bench

HarnessAdapt-RL co-trains a 9B coding agent with its own tools and hooks, reporting pass@1 gains of +9.8, +12.6, and +6.4 points over a GRPO baseline on SWE-bench Verified, Multilingual, and Pro. The ablations: removing the evolved tools and hooks costs up to 7.7 points on Multilingual, while the training-time "mechanism-level skills" can be stripped off without loss.

What shipped

Lingxiao Tang, Wei Tao, Yiwen Guo, and Lingfeng Bao describe HarnessAdapt-RL in "Training the Whole Agent, Not Just the Model," posted to arXiv on 20 October 2026; the work was done during an internship at Lightspeed. The standard recipe for agent RL freezes the harness — the tools, runtime hooks, and scaffolding around the policy — and trains only the policy. The authors' motivation: during training, "trajectories reveal flawed repair strategies and interaction problems that vary across tasks and change as the policy learns."

Two frozen teacher models stay in the loop while a Qwen3.5-9B policy is trained with GRPO on 1,058 tasks from R2E-Gym/R2E-Gym-Lite:

• Skill teacher. For each task, it compares two trajectories from the current policy, identifies a repair strategy, and explains when and why it applies — omitting repository paths, code symbols, and literal code (§3.1). These are the "mechanism-level skills."
• Harness teacher. It looks across tasks for recurring interaction and feedback problems and proposes changes to tools and hooks. A proposal is kept only if it "improves the observed pass rate or the observed trainable rate without lowering either" — the trainable rate being the share of rollouts that produce a GRPO gradient (all-success or all-failure groups contribute none).

The policy trains with the adapted harness, and its new trajectories drive the next round. The released package includes the two evolved harnesses this process produced, H1 and H2.

Headline numbers (mean pass@1 over three evaluation runs; deltas in percentage points over a mini-swe-agent + GRPO baseline, per the results table):

Benchmark	GRPO baseline	HarnessAdapt-RL	Δ
SWE-bench Verified	47.8	57.6	+9.8
SWE-bench Multilingual	24.7	37.3	+12.6
SWE-bench Pro	33.3	39.7	+6.4
Average	35.3	44.9	+9.6

The abstract also claims the method outperforms GRPO with OpenHands' human-designed harness, a second baseline; we could not extract its exact scores from the accessible text.

The ablations support the paper's thesis that training "produces a model-harness pair: the trained policy and the evolved tools and hooks, with skills removed at deployment":

• Skills removed at evaluation. Averaged across the three benchmarks, mechanism-level skills survive removal (38.3 pass@1). Near-answer hints — skills that hand over the concrete fix — are actively harmful once stripped: 32.7, below the 35.3 GRPO average. They inflate training success and shorten trajectories, but the model trained on them is worse once they are gone.
• Evolved tools and hooks removed at deployment. Swapping the evolved harness back to the base configuration costs −4.8 on Verified, −7.7 on Multilingual, and −4.7 on Pro. The co-evolved harness is not incidental scaffolding; it is part of what was trained.
• Re-evolving beats freezing. Continuing to adapt the harness as the policy changes outperforms freezing it after the first epoch by 2.6 points pooled, with a 95% bootstrap interval of +0.9 to +4.3 — though the intervals for Verified and Pro individually include zero.

The authors released what they call an anonymous replication package: training and harness-evolution code, teacher prompt templates, training task IDs and sampling schedule, the H1/H2 harnesses, and the trained deployment checkpoint on Hugging Face (10B params, BF16, no model card). Training runs ~46 steps on 8 GPUs (TRAINING_SETUP.md).

Why it matters

The central argument — that a trained agent is a pair, policy plus harness, not a policy alone — challenges how most teams scale agents. "Freeze the harness, scale the model" is the industry default, and if the co-evolved tools and hooks are worth up to 7.7 points, teams pouring compute into the policy alone may be optimizing the wrong axis.

Two details matter for anyone building agent RL pipelines:

• The trainable-rate criterion protects the gradient signal. Most harness changes are judged by whether they improve task success. Here, a change that improves pass rate but starves GRPO of contrastive rollouts is rejected. The paper's example is concrete: a semantic-review proposal raised pass rate 2.7% but cut the trainable rate 7.0% and was rejected, while an automatic related-testing proposal that improved both (+6.3% / +7.4%) was accepted. If you run GRPO on agents, every harness decision trades off against your optimizer's signal-to-noise ratio.
• Mechanism guidance beats answer guidance. The near-answer-hint ablation warns anyone curating agent training data: trajectories that demonstrate the fix make training look better and deployment worse. What transfers is the "when and why" — the mechanism — not the answer.

It is also cheap to try: one 9B model, ~46 training steps, 8 visible GPUs, and the full recipe is in the repo.

Caveats

Everything above is author-reported. A few limits worth keeping in mind:

• Single training run per configuration. The three evaluation repeats use one trained checkpoint, so they capture evaluation noise but not training-seed variance — the authors concede this themselves.
• Uneven rollouts. The GRPO baseline received twice as many policy-training rollouts as the co-trained configurations — an asymmetry in the baseline's favor, but the paper does not test its effect.
• The acceptance criterion is a heuristic, not a significance test. Candidate and current harnesses are compared over finite rollouts, and proposals bundle several tool/hook changes whose individual contributions are not isolated, so the acceptance rule does not attribute credit per change.
• The "frozen teachers" are proprietary API models. Per the setup docs, the skill teacher runs OpenCode serving glm-5.3-flash and the harness teacher runs Codex with gpt-5.6-sol; the teacher CLI versions were not preserved. Despite the open code, a faithful reproduction requires paid API access, and the anonymous weights can't be reproduced bit-for-bit.
• Not everything is documented. No GPU-hours or wall-clock compute is disclosed, the upstream R2E-Gym-Lite dataset revision is not recorded, and the exact OpenHands-GRPO scores could not be extracted from the public text.

Sources

• arXiv abstract — Training the Whole Agent, Not Just the Model (2610.28069)
• arXiv HTML full text
• alphaXiv summary of the paper (results-section figures)
• Hugging Face papers page
• GitHub — HarnessAdapt-RL replication package
• TRAINING_SETUP.md (base model, teachers, GPU count)
• Trained checkpoint on Hugging Face
• AI Weekly alert (initial report)