OpenAI’s gpt-oss Models: Open Weights, Benchmarks and Safety

OpenAI released gpt-oss-120b and gpt-oss-20b, two open-weight models under Apache 2.0, in August 2025. I speculate that a previously seen mystery model might be related, but OpenAI had not confirmed that connection. This video is sponsored by Together AI.

Open weights let developers run and adapt models themselves, subject to the license and infrastructure costs. OpenAI reported that the larger model could run on a single 80 GB GPU and the smaller model on a device with 16 GB of memory; actual performance depends on software, hardware and workload.

The announcement compares the 120b model with o4-mini and the 20b model with o3-mini on selected reasoning benchmarks. I find the possibility of local deployment exciting, but these reported comparisons are not a guarantee of equal performance on every task.

The models support tool use and configurable reasoning effort. Berman discusses low, medium and high settings as a tradeoff between speed and additional work. A health benchmark is an evaluation of model responses, not a license to rely on the model for diagnosis.

Both models use a mixture-of-experts Transformer. OpenAI reports 117 billion total parameters with 5.1 billion active per token for the 120b model, and 21 billion total with 3.6 billion active for the 20b model. Active parameters are only one part of deployment cost and performance.

The technical description mentions attention optimizations, rotary positional encoding and a context length up to 128,000 tokens. OpenAI says its pretraining data focused on text including STEM, code and general knowledge; local availability does not make model output automatically accurate.

Together AI sponsors the video and offers hosted access. Berman then describes supervised fine-tuning and reinforcement learning in post-training, intended to improve reasoning and tool use. The precise training methods should not be assumed to be identical across all model providers.

The displayed Codeforces estimates give gpt-oss-120b with tools a 2622 rating and gpt-oss-20b with tools 2516, compared with 2706 for o3 with tools. These are benchmark estimates under specified conditions, not direct proof that the models outperform most humans at every programming job.

On the displayed Humanity’s Last Exam and HealthBench comparisons, the larger model is below o3 in the examples Berman cites, though the results are close on some tests. He also reviews AIME, GPQA, MMLU and TAU-bench; test configuration and tool access matter when interpreting any score.

OpenAI said the models’ chain of thought was not directly supervised for alignment and discussed monitoring it for possible misbehavior. It cautioned developers against displaying raw reasoning directly because it may contain harmful or inaccurate content; such traces are not a complete or infallible safety monitor.

The company described filtering some harmful pretraining data and evaluating worst-case fine-tuning in biology and cybersecurity. It reported that the modified models did not reach its framework’s high-capability threshold in those tests. That finding does not mean harmful fine-tuning or unsafe outputs are impossible.

OpenAI also announced a red-teaming challenge with awards to find safety issues. I plan a fuller hands-on test and point viewers to the sponsor’s hosted option. The figures and availability described here are from the 2025 launch.