Training a GPT-OSS 2048 Strategy With Verifiable Rewards

In this December 2025 tutorial, I demonstrate reinforcement learning with verifiable rewards using a local 2048 game and an open-weight GPT-OSS model. NVIDIA partnered with me on the video, and I use an RTX 5090 with an Unsloth notebook. This is one particular experiment, not a demonstration that every home computer can train every model.

A verifiable reward is an automated signal that scores an outcome against a defined check. The system can try candidate behaviors and use those scores for training without a person grading every attempt. People still design the environment, objectives and checks, and a reward only measures what its implementation actually tests.

The example is 2048: combine numbered tiles while avoiding a full board. I use GPT-OSS and a reward function to train it to produce game strategies. I characterize the initial model as not knowing the game, but the video does not establish what was present in its pretraining data.

My computer uses an RTX 5090. I suggest that other NVIDIA hardware may work, possibly more slowly, but compatibility also depends on memory, software and the model configuration. The video links the notebook and setup instructions, so readers should check the requirements for their own hardware rather than assume every GPU is supported.

I choose Windows Subsystem for Linux and Ubuntu after trying several setups. The preparation includes updating NVIDIA drivers, installing CUDA, checking GPU visibility with `nvidia-smi`, creating a Python environment, and installing PyTorch, Unsloth and Jupyter. Exact commands and compatibility can change with software versions.

The notebook is downloaded from the linked resources and opened in a local Jupyter session. I skip a cell intended for cloud setup, run the imports and load the base model. The notebook's cells provide an inspectable sequence; if a step fails, I can investigate that step and the terminal logs.

The notebook uses LoRA to fine-tune a comparatively small set of additional parameters rather than changing all model weights. It presents this as a way to reduce memory use. The amount saved and the resulting quality depend on the implementation and training configuration.

Next I run a 2048 variant that the notebook says was created with GPT-5. I test board movements and verify that the environment updates the board. This separate game implementation supplies the environment in which generated strategies will be evaluated.

The evaluation function runs a proposed strategy, checks the game state and imposes a time limit so a strategy cannot run forever. A simple baseline that repeatedly chooses one move times out. Additional checks are intended to detect ways a strategy could exploit the evaluation instead of playing as intended.

The model generates a short Python strategy, which the notebook extracts and runs in the game environment. Automated reward functions check whether the function works, whether it violates specified rules, and whether it succeeds in reaching the target. The notebook then uses those signals to train the model with GRPO.

One initial model output is incomplete because generation stops at the configured token limit, so the resulting code fails. I leave the failure in the process and explain that the feedback can inform training. A failed sample demonstrates a scoring opportunity, not evidence that every later strategy will be valid.

I start `trainer.train()` with a limit of 1,000 steps and monitor GPU activity and training metrics. The first reported reward is negative; I also point out the KL metric as an indication of divergence from the reference model. Training takes time, and the number of steps needed can vary.

In the run I show, a high-reward result appears after 84 iterations and one displayed board reaches the 2048 tile. I describe that as a successful strategy and say the adapted model can be saved or pushed to Hugging Face. I skip the separate post-training inference test, so the video does not establish a repeatable win rate or a perfect strategy.

I estimate the whole process took about six hours including compute, with setup under an hour, excluding the time my team and I spent learning the workflow. This is a single result on the hardware shown. Independent evaluation across games and starting states would be needed to measure reliability.

I see potential for local training and customization as smaller models and personal devices improve. A verifiable reward can make some tasks easier to score automatically, but a well-defined reward is not by itself proof that a system is safe, private or suitable for a consequential task.

I mention other possible applications, including games, analysis and controlling a small vehicle, wherever useful rewards can be defined. These are possibilities rather than demonstrations in this video. I thank NVIDIA and Unsloth for the collaboration and link the notebook and instructions for viewers who want to explore the experiment.