Qwen Releases QwenGyre for Long-Horizon Agent RL Training

Alibaba’s Qwen team has detailed QwenGyre — an elastic reinforcement learning (RL) framework for training AI agents on what the authors call “extreme-long” (xLong) tasks, where a single execution spans hours, hundreds of model–environment interactions, and close to one million tokens. The paper, submitted to arXiv on September 27, 2026 and announced by the Qwen team on October 12, reports up to 1.85× faster end-to-end training than a colocated baseline at matched scores, and a six-point accuracy gain on a repository-generation benchmark.

Advanced

What QwenGyre Does

QwenGyre is not a model — it is training infrastructure. The Qwen team (Alibaba Token Hub, with the University of Science and Technology of China and Tsinghua University) used it to run RL on its Qwen 3.6 122B model and on its flagship Qwen 3.8 2.4T model, with roughly 700,000 tokens per rollout.

The motivation is that the two standard ways of arranging RL hardware both fall apart at this horizon. In a colocated setup, the same GPUs alternate between generating rollouts and training, so training waits for the slowest execution. In an asynchronous setup, GPUs are split into fixed rollout and training pools, so neither can borrow the other’s spare capacity. The paper also notes that modern agent harnesses compact context, spawn sub-agents, and retry failed steps, so one execution becomes a branching tree rather than a single trajectory — and expanding every branch into a sample duplicates shared prefixes and inflates cost.

The researchers state the problem directly: “Applying online reinforcement learning (RL) to such executions poses two fundamental challenges: (1) severe execution variance and prolonged rollout delays cause massive GPU idling; and (2) complex non-linear branching generates massive trajectory redundancy, crippling training efficiency.”

Inside the Elastic Scheduler

QwenGyre’s first component moves GPUs between rollout and training without stopping live agent runs. GPUs are grouped into “cells” that share one training parallel layout; a designated core cell holds the authoritative weights and the optimizer state, while satellite cells can pull a weight snapshot and join an in-progress training batch. Megatron handles training and SGLang serves rollouts.

The scheduler tracks a “rollout waterlevel” — queued plus running executions — and hands a cell over to training only when the remaining cells can still serve all outstanding work. Model requests are rerouted through a proxy so the harness keeps running, and KV-cache blocks migrate over RDMA between engines to avoid recomputing prefixes. The paper reports average role-switch times of 8.52 seconds (rollout to training) and 3.46 seconds (training to rollout), negligible against executions lasting hours.

For the 2.4T run the paper used four cells of 48 nodes each, with a total inference concurrency limit of 384; the 122B experiments used 32 nodes in eight cells of four (or 24 nodes in six cells for the DeepSWE and TerminalBench workloads).

Turning Branching Executions into Training Data

The second component is the trajectory processor. A proxy records exact token IDs and behaviour log-probabilities, and the processor stores model calls as prefix-shared trajectory trees, so a shared prefix is counted once rather than once per branch. Executions that hit the time limit are still scored on the workspace they preserved — an evaluator failure is excluded rather than treated as a zero reward — and at most 5 trajectories are kept per execution, ranked by agent role. Token losses are averaged within each execution, then across executions, so a task with more branches does not gain extra weight.

The paper reports that the cap of 5 matched uncapped training scores while cutting forward–backward time by 25.2%. Training uses an adapted Group Sequence Policy Optimization (GSPO) with 16 independent rollouts per group and batches of 32 groups for NL2RepoBench (48 for DeepSWE and TerminalBench).

Benchmark Results

On the flagship Qwen 3.8 2.4T model, QwenGyre completed 48 training steps in 75.42 hours, against 134.55 hours for the Async baseline and 91.47 hours for Colocate — end-to-end speedups of 1.78× and 1.21× respectively, at similar training scores. Evaluation pass rate on NL2RepoBench rose from 52.48% to 58.54% over those 48 steps.

Across the three evaluated workloads the framework ranged from 1.38× to 1.78× faster than Async and 1.21× to 1.85× faster than Colocate. The researchers report the largest margin over Colocate on TerminalBench (1.809×–1.849×) and the smallest on the 2.4T model, where many queries ran close to the time limit: among queries with complete duration records, 61.1% took at least four hours.

What This Means

QwenGyre is a sign of where agentic RL is being bottlenecked. As coding agents move from single-bug fixes to repository-scale work, the limiting factor is no longer only model capability but the scheduling of hour-long rollouts across expensive GPU fleets — which is why an infrastructure paper, not a model release, is the deliverable here.

The approach also sits in a busy field. The Qwen team’s write-up compares it with ByteDance’s veRL, Tsinghua’s AReaL, and Microsoft’s Agent Lightning, noting that the speedups are measured under each project’s own setup and are not directly comparable. Unlike those projects, the QwenGyre paper does not link a public code release, so its results cannot yet be reproduced or adopted directly.

For institutions running agent evaluation or training on shared clusters, the transferable idea is the elastic cell model: keeping harness state alive across GPU role changes rather than tearing down and restarting an execution. Any long-horizon agent workload can hit that scheduling wall, independent of which base model is being trained.

Related Coverage

• Xiaomi Streams MiMo-V2.6 Reinforcement Learning Runs Live — another look at the RL training loop for a Chinese lab’s models, this time exposed as a public dashboard.
• JetBrains Ships Mellum2.1, a 12B Open Model for Coding Agents — a case where scaling up reinforcement learning after pre-training drove a coding-agent model update.
• Alibaba at Apsara 2026: Qwen 4 in Training, 10T Models Next — the wider Qwen roadmap QwenGyre’s 2.4T flagship belongs to.

This post was drafted with AI assistance and reviewed by RITS staff.

Sources

• QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents (arXiv:2609.33848) — primary paper, submitted September 27, 2026
• Full HTML text of the QwenGyre paper — used for the figures, hyperparameters, node layout, and switch timings
• AI News: Alibaba details QwenGyre to accelerate extreme-horizon agent training (October 12, 2026)
• MarkTechPost: Alibaba Qwen Introduces QwenGyre — Elastic Reinforcement Learning RL for XLong-Horizon Agents (October 12, 2026)
• Hugging Face paper page for arXiv:2609.33848
• Qwen team announcement on X (October 12, 2026)