Introducing Model Factory. Read the announcement

Blog / Research

RL Residency: A Retrospective

NOV 19TH, 2026
RL Residency: A Retrospective

RL Residency: A Retrospective

The Prime Intellect RL Residency is a program that supports open research through compute grants and personal mentorship, while creating valuable evals, training environments, and other research for Prime Intellect and the wider community.

In 2026, this effort has been highly successful, thanks to our many great residents. In this blog post, we want to highlight their work.

There's plenty of overlap, but we've grouped the work into Evals, Training Environments, and Other Research.

Evals

Maze Bench

Maze Bench

Jonathan Pappas / @patience_cave

Maze Bench is a highly complex 3D puzzle game that even the best LLMs today get at most 16% accuracy on without coding tools, and at most 23% with coding tools. And the levels they solve are still the easy ones!

All Maze Bench levels are connected. Many levels share concepts, and those concepts are introduced slowly, level by level. Therefore, the environment is a good testbed for continual learning. Every level tests 3D navigation, often with partial observability, and challenging logical reasoning.

Read the X announcement, blog post, results, leaderboard, and code. You can also play the game yourself, or build your own levels.

Library Design Bench

Library Design Bench

Gabe Orlanski / @GOrlanski

Library Design Bench is what its name suggests: agents are asked to implement a library that is then used by another agent to solve a programming task. The eval asks: does using the library improve results compared to solving the same task without it? How does it impact program length?

The eval goes to great lengths to make the measure of library quality objective, and the leaderboard closely reflects our own experience with the models.

Read the X announcement, paper, blog post, and code.

Agents’ Last Exam Linux CLI

Agents’ Last Exam Linux CLI

sankalp / @dejavucoder

An odyssey of porting Agents’ Last Exam Linux CLI to verifiers, and fixing the many, many issues along the way.

Read the X announcement and blog post.

SWE-Bench Multimodal v2 is not multimodal

SWE-Bench Multimodal v2

sankalp / @dejavucoder

Another popular benchmark is fundamentally broken: this time, a multimodal benchmark that is solvable without images just as well as with them.

Read the X announcement and blog post.

Training Environments

These are like evals, but built for training.

PMPP Hard

PMPP Hard

Sinatras / @myainotez

Training models to become superhuman kernel developers.

Thinking Machines used this environment to eval Inkling v2 Small, highlighting it early in their announcement. It has also been used successfully to train models.

Read the X announcement, blog post, and source code.

OpenTTD-Bench

OpenTTD-Bench

Sophia Xu / @thesophiaxu

OpenTTD is an open-source version of Transport Tycoon Deluxe, a multiplayer economy simulation game.

OpenTTD-Bench builds a training environment on top of it. It tests models' long-term planning abilities in a real-time, multimodal, long-running simulation. It is multi-agent by design, and future work will improve both the complexity of the simulation and the communication between competing agents.

Read the X announcement, blog post, and code.

PretrainGym

PretrainGym

Justin Yang Chae / @HuNtEr_J1324

modded-nanogpt is a speedrun that tests how quickly a given architecture and optimizer can reduce validation loss to a given value, on a fixed training and validation dataset, with fixed hardware.

PretrainGym asks how low a loss can be achieved on fixed compute just by curating public datasets, and turns it into a training environment. It goes to great lengths to prevent reward hacking and to measure an accurate reward.

Read the X announcement and blog post.

Biocircuit

Biocircuit

Amit Prakash / @amit05prakash and Lalithadithya / @infinitasium

This auto-research environment trains models to get better at detecting meaningful attributes in feature graphs of models.

Read the X announcement and blog post.

Eleusis

Eleusis

@np_hard

Eleusis is a card game in which the player has to find hidden rules behind given actions. The Eleusis environment makes multiple agents play the game to teach them collaboration.

Read the X announcement, blog post, and code.

AnomalyXL

AnomalyXL

Nicolas Zumarraga / @nzuma0

AnomalyXL trains models to be good at anomaly detection in time series. It creates infinite synthetic data cheaply, at finely controllable difficulty. Training on this data transfers to real time-series data like ECG readings. The paper demonstrated the utility of RLMs for time-series anomaly detection, and, conversely, the utility of the environment for RLM training.

Read the X announcement, paper, and source code.

Mitosis Bench

Mitosis Bench

@dylantirandaz

Mitosis Bench brings AI for medicine into the open: it trains an agent to detect and count cancer cells that are in the process of dividing.

Read the X announcement and blog post.

Periapical Lesions Bench

Periapical Lesions Bench

@dylantirandaz

Periapical Lesions Bench tests (and trains) an agent’s ability to reliably locate periapical lesions in X-rays. Periapical lesions are pockets of damaged tissue around the tip of a tooth’s root, close to the nerves and blood vessels. Finding them reliably is important work, and current models are bad at it: the best one achieves a mere 16% pass@1.

Read the X announcement and blog post.

PACT

PACT

Aryan Madhav Verma / @aryanmadhaverma

PACT brings robot control by AI agents into the open: it trains an agent on simple robotics tasks, like pick-and-place with a robot arm.

The models get direct access to the API of the simulated sensors and actuators. They run in any coding harness and write code to abstract away the robot sensing and control, which achieves higher scores than hand-written high-level tools.

Read the X announcement and blog post.

AerialSim

AerialSim

Nahid Alam / @nahidalam

Drones are another type of robot that can be controlled by AI agents. AerialSim trains agents to navigate drones in different scenarios: teaching them to find a landing spot and navigate there, avoid obstacles, and abort if the task cannot be fulfilled safely.

Read the X announcement and blog post.

Other Research

KernelGuard

KernelGuard

Sinatras / @myainotez

KernelGuard saved the GPU MODE competition from slop. It also protects PMPP Hard (above) against reward hacking. A crucial piece of infrastructure for AI kernel development.

Read the X announcement and blog post.

Harnesses on WorldBench

Harnesses on WorldBench

Harish Krishnakumar / @harishkrik

WorldBench is a highly challenging, diverse multimodal benchmark. Harish ported the benchmark to verifiers, hardened it, and investigated the effects of harnesses on visual capabilities, compared to model scaling. The work was inspired by Alex Zhang’s Language model harnesses are compositional generalizers post. Harish showed that model scaling increases pass@1 (reliability given a task can be solved at all) but not pass@8 (number of tasks that can be solved at all) in WorldBench, but giving models a harness makes both pass@1 and pass@8 go up as the models get larger and reason for longer.

Read the X announcement and blog post.

ECHO work

ECHO on BabyAI

Billy Hoy / @BillyHoy1_

BabyAI from AgentBoard is a partially observable puzzle game that resembles real-world, embodied navigation.

Billy showed that ECHO—RL on tool calls, SFT on tool outputs—in a setting without reasoning (only tool calls) is superior to pure RL for learning and generalizing in BabyAI.

Read the X announcement and blog post.

Proxy rewards

Proxy rewards

Sinatras / @myainotez

Catching reward hacks is often expensive and slow. This post shows that we can create automated pipelines to find cheap proxy rewards that catch the same issues, at least for small policies.

Read the X announcement and blog post.

HINT4

HINT4

@secemp9

An NVFP4 quantization method that runs natively on both older and newer cards.

Native FP4 math is only available on Blackwell. HINT4 keeps NVFP4-style 4-bit weights and block scaling, but does the math on 8-bit integer tensor cores, the one tensor-core instruction that every NVIDIA GPU from Ampere to Blackwell supports natively. In its first progress report, HINT4 already beat standard NVFP4 on Kimi-Linear 48B (6.54 vs. 6.63 WikiText-2 perplexity, with BF16 at 6.41), and its decode matrix multiplications ran about twice as fast as FP16 on an RTX 3090.

Read the X announcement and blog post.

MegaGem

MegaGem

Alex Wa / @_djdumpling

MegaGem is a card game that involves auctions, missions, loans, and investments. Alex managed to train a model on the environment using specialized and creative methods.

Read the X announcement and blog post.

Join the RL Residency

Thank you to all of our residents for an incredible year of work. If you want to build evals, training environments, or other open research with compute grants and mentorship from Prime Intellect, apply to the RL Residency. We accept a limited number of residents.