QwenGyre: elastic GPU scheduling and trajectory trees for RL on hour-long agent rollouts — 2.4T MoE up 6 points on NL2RepoBench in 48 steps, ~44% faster than async

Alibaba's Qwen team released QwenGyre, an online-reinforcement-learning framework for training agents on extreme-long-horizon ("xLong") tasks. The team used it to train the flagship Qwen 3.8 2.4T on ~700K-token rollouts: NL2RepoBench pass rate rose from 52.5% to 58.5% in 48 steps, and end-to-end training time fell ~44% versus an async baseline. The paper (arXiv 2609.33848) is a systems contribution — the headline ideas are GPU scheduling and trajectory-data construction, not a new RL objective.

What shipped

QwenGyre targets a regime the paper calls xLong-horizon: agent executions that span hours, hundreds of model–environment interactions, and close to a million tokens per rollout. At that scale, the paper argues, both standard online-RL deployments break down. A colocate design alternates the whole GPU pool between rollout and training, so training blocks on the slowest trailing executions while GPUs idle. An async design statically partitions GPUs, so neither stage can borrow the other's spare capacity. Trajectories also stop being linear: black-box harnesses compact saturated context, delegate to sub-agents, and retry failed paths, so one execution becomes a branching graph whose naive expansion explodes training cost.

QwenGyre answers with two components.

Elastic Scheduler. GPUs are split into cells, each able to serve rollouts or host a full training replica. A core cell owns the authoritative weights and optimizer state; satellite cells switch roles. The scheduler tracks a "rollout waterlevel" — queued plus running executions — and flips cells to training once the remaining cells can still serve all live work.

In-flight model calls are rerouted through a proxy so harness executions never pause, and KV-cache blocks migrate over RDMA during rerouting to avoid prefix recomputation. Satellites pull fresh weights through Mooncake's transfer engine and join an ongoing batch; "streaming training" lets cells claim micro-steps as data becomes ready. Reported role-switch latencies are 8.52 s (rollout→training) and 3.46 s (training→rollout) (MarkTechPost).

Trajectory Processor. A proxy records every model call with exact token IDs and behavior log-probabilities (token-in, token-out). Each execution's calls are organized into a prefix-shared trajectory tree that preserves each output's original conditioning context.

Timed-out executions are still scored on their preserved workspace — partial progress counts — while evaluator failures are excluded rather than treated as zero reward. Per execution, at most five trajectories are kept, ranked main agent > main summary > sub-agent > sub-agent summary. Shared targets are counted once, only policy-generated tokens are trained, and token losses are averaged within each execution so more-branched executions aren't overweighted. Qwen reports the cap-of-five selection matched uncapped sampling quality while cutting forward-backward time by 25.2% (MarkTechPost).

Training uses an adapted GSPO — Group Sequence Policy Optimization, Qwen's own group-relative RL objective (GSPO blog) — with 16 rollouts per group and no learned value function.

Scale and results. The flagship run trained Qwen 3.8 2.4T (a 2.4T-parameter MoE; A95B per Qwen) on NL2RepoBench with ~700K tokens per rollout, on 192 nodes (four cells of 48). Megatron does training, SGLang serves rollouts, and the harness is an unmodified Claude Code 2.1.220 accessed through the proxy (MarkTechPost). Qwen reports pass rate rising from 52.5% to 58.5% in 48 steps, which took 75.42 hours versus 134.55 for async and 91.47 for colocate. Across all reported configurations — Qwen 3.6 122B on NL2RepoBench, DeepSWE, and TerminalBench, plus the 2.4T run — end-to-end speedups range from 1.38–1.78× over async and 1.21–1.85× over colocate at matched scores; the largest single figure is 1.85× over colocate on TerminalBench (MarkTechPost). AICoder's summary of the release claims sustained GPU-cluster efficiency above 88% (AICoder).

Why it matters

The paper's implicit claim is that the binding constraint on xLong agent RL is systems, not the RL objective. Its design separates three lifetimes that online RL usually conflates: harness executions, GPU roles, and training samples. Harness state persists across GPU role changes; recorded model calls are materialized into training samples only when rewards arrive.

For anyone building agent-training infrastructure, the transferable pieces are concrete. First, treat GPUs as fungible between rollout and training, and make role switches cheap enough — KV-cache migration over RDMA, proxy-rerouted in-flight calls — that you never restart an hour-long harness run. Second, record model calls with token-level provenance through a proxy, which lets you reconstruct training data from a black-box harness you cannot modify. Third, score partial progress and deduplicate shared prefixes, because branching executions otherwise multiply training cost by the number of paths.

The NL2RepoBench result deserves context. The benchmark (NL2RepoBench) gives an agent an empty workspace and a natural-language requirements document, then scores the resulting installable Python library by its upstream pytest pass rate — 104 tasks with ~18,800-token specs. Its own paper reported a best average pass rate of 40.2% (Claude-Sonnet-4.5 via Claude Code). QwenGyre's starting point of 52.5% is already above that, so the 58.5% is not a head-to-head comparison — different model, harness, and training regime — but it shows a 2.4T MoE moving a long-horizon benchmark by six points in 48 RL steps.

Caveats

Every number above is vendor-reported: the figures come from the paper and the @Alibaba_Qwen announcement, relayed through third-party summaries, and no independent reproduction has been published. The paper compares against colocate and async baselines under equal GPU budgets and matched staleness. There is no code and no license disclosed — the announcement says core scheduling and trajectory-deduplication components are "being prepared for open-source integration," but nothing is public yet. The GPU model is not disclosed.

The design also has structural limits worth noting before you plan around it — these follow from the design as described, not from measured ablations. Each switchable cell must hold a full training replica, so clusters below a minimum size cannot benefit. Streaming training distributes work by each data-parallel group's progress, which limits global minibatch randomization. The whole point is reclaiming idle capacity from long, variable rollouts — short tasks leave little to reclaim.

Sources

• QwenGyre paper, arXiv 2609.33848 (abstract)
• QwenGyre paper, full HTML
• MarkTechPost: Alibaba Qwen Introduces QwenGyre
• AICoder: Alibaba Unveils QwenGyre (quotes the official announcement)
• r/LocalLLaMA announcement thread
• NL2RepoBench paper (arXiv 2512.12730)
• Qwen GSPO blog post
• Hugging Face papers page for QwenGyre