prime-rl 0.10.0: faster training, our new RL loss IPO, and one unified eval stack

Today we're releasing prime-rl v0.10.0 and verifiers v0.4.0. We added our new RL algorithm IPO as the default, tailored for async RL stability, as well as performance improvements and new model support.

Training throughput

The trainer's defaults favor memory, so large models fit on few GPUs. This release also documents the other end: a max-throughput recipe that keeps everything on GPU and compiles with fullgraph.

Qwen3-235B-A22B SFT against torchtitan. Both runs used the same nodes back to back: 4 × 8 B300, 16k sequence length, FSDP + EP 8, full activation checkpointing, fp32 AdamW, balanced routing.

That's 1.15× faster with the same setup. Reproduce it with uv run sft @ examples/advanced/qwen3-235b-a22b/sft-max-perf.toml.

Most of the gain comes from MoE work:

• One MoE runtime and checkpoint layout now serves every model.
• Gate/up and QKV projections are fused at runtime.
• MoE blocks compile with fullgraph.
• A torchtitan-style chunked loss.
• A SonicMoE BF16 backend, plus a per-layer choice of MoE compute.

GLM-5.3. Sparse attention now runs the forward pass with FlashMLA (1.9–3.6× faster than TileLang on H200) and the backward pass with cuDNN (3–4.7× faster). Combined with a 12-node, 16k recipe without context parallelism or offloading, full GLM-5.3 SFT (78 layers, H200, fake data) improves as follows:

On real weights and data, a step drops from 31.2 s to 24.2 s. In RL, an 8-layer proxy drops from about 16 s to about 12.5 s per step.

DeepSeek V4 Flash gets these custom kernels:

• A flash-style sparse attention kernel: 46.7% → 53.6% MFU on real data.
• The Lightning Indexer scored with a fused FP8 kernel, with the CP gather overlapped with compute: 40.2% → 47.8% MFU.
• A fused mHC chain.
• One fused RoPE kernel that matches vLLM bit for bit and makes the SFT step 13–14% faster on B300.

IPO: INTELLECT Policy Optimization

In async RL, the trainer and the inference engine never agree exactly on a token's probability. The policy has moved on since the rollout was sampled, and the kernels differ. IcePop, the usual fix, masks a token when the ratio $\pi_{\text{train}} / \pi_{\text{infer}}$ leaves a fixed band. IPO instead masks a token when the probability itself moved too much:

A ratio misjudges both ends of the distribution. A rare token going from 0.01 to 0.06 is a 6× ratio and gets masked, even though almost nothing changed. A likely token falling from 0.90 to 0.20 is a 0.22× ratio and is kept, even though the policy changed a lot. The IPO mask is double-sided, ignores the sign of the advantage, and has one knob. The probability-difference criterion comes from DPPO (Qi et al., 2026), which uses it for the PPO trust region between policy updates. IPO applies it to the trainer–inference mismatch, and unlike DPPO its mask does not depend on the sign of the advantage. IPO replaces our previous DPPO-based default loss, which is removed in this release.

We ran the ablation on agentic SWE RL with GLM-4.5-Air, without router replay and up to 64 steps off-policy. This is a deliberately extreme setting: in most runs IcePop is stable. Here, though, IcePop collapsed around step 1.2k, when the trainer–inference KL blew up by more than 20×. IPO at eps 0.2 and 0.3 trained straight through. At eps = 0.3, IPO masks only about 0.25% of tokens, compared with about 2% at the old 0.1.

IPO with eps = 0.3 and no KL term is now the default. IcePop, PPO and CISPO remain available as opt-in losses. GRPO also gains two options: a length-weighted baseline and exact prompt-mean loss aggregation, which keeps prompts with long trajectories from dominating the gradient. The full setup and configs to reproduce it are in docs/ipo.md.

Other RL changes in this release:

• Top-p and top-k sampling are now replayed exactly in the trainer, using the sampling mask captured from vLLM.
• The trainer batch stays a constant size: the orchestrator backfills samples that were filtered out.
• RL freezes the MoE router by default, which keeps the trainer's expert choices from drifting away from the ones inference sampled.

New models

Every architecture in prime-rl now runs on our own optimized implementation. New in this release:

• DeepSeek V4 Flash, with the kernels above and ring context parallelism.
• Qwen3.8 Flash Next, with:
  • n-gram (engram) embeddings sharded by vocabulary rows across data-parallel ranks
  • indexed sparse attention
  • Gated DeltaNet
  • hyper-connection residuals
  • sigmoid-gated MoE
• Nemotron-H (Nano and Super), with Mamba-2 state resets at packed-document boundaries and Ulysses context parallelism.
• Multimodal RL from inline rollouts. Images stay inline through rollouts, packing and transport, instead of being shipped as pixel tensors. vLLM returns the expanded prompt IDs and image placeholder ranges, so every image token lines up exactly in training. Qwen-VL and Kimi K2.5 are supported.

One eval stack

The eval CLI moved from verifiers into prime-rl. uv run eval evaluates any model on any environment, either standalone or as the online evals of a training run:

• It runs through verifiers env servers with adaptive concurrency.
• It shows up in the local dashboard.
• It takes multiple [[source]] blocks.
• --resume continues an interrupted run from its trace stream, and only the missing rollouts run again.

Every train and eval source has a select block (include/exclude, shuffle, skip, limit). It is backed by verifiers' shared task selection, so prime-rl and vf-eval pick the same tasks for the same config. Timed-out rollouts are now their own outcome: scored in evals, and kept out of training batches.

uv run eval gsm8k
uv run eval @ sources.toml --env.retries.max-retries 3

verifiers 0.4.0

prime-rl 0.10.0 ships with verifiers 0.4.0, which completes the move to v1: the legacy v0 stack is removed and the CLIs are now vf-*. See the verifiers release notes for details.

Thanks

Thank you to everyone who contributed to these releases, and welcome to the new prime-rl contributors: @parkerpettit, @yafshar, @kcoopermiller, @biswapanda, @Alex-Shan, @cabrinha, @nxphi47, @akirillo, @Apocrathia, @artzhuravel and @skpulipaka26.

Try it now: prime-rl 0.10.0 and verifiers 0.4.0.