OpenAI o3 and o4-mini: Tool Use, Benchmarks and the First Codex CLI

OpenAI has released o3 and o4-mini, and what excites me most is their ability to use tools as part of reasoning. Rather than taking one action and stopping, an agent can inspect a result, decide what to do next and use another tool. I think this could be an important direction for improving performance, though calling it a new scaling law is my interpretation, not an established result.

OpenAI describes o3 as its more capable model and o4-mini as a smaller, faster option. The models can reason over images, including by using tools to inspect them. Although I say in the video that they also accept and output audio, OpenAI’s API model documentation lists text and image input and text output; audio is not native support for these model APIs. Product-level tools can provide additional capabilities.

In a clip from OpenAI’s launch livestream, a presenter gives o3 a screenshot of a physics poster from a 2015 internship. The task is to estimate the proton isovector scalar charge and compare it with recent work. The final value is not actually printed on the poster: the model has to interpret the plot and infer what additional steps are needed.

The presenter says the task requires extrapolating the plotted values to a physical quark mass and applying a normalization factor. The model zooms into the image and searches the web for later results. The presenter says reconnecting with the project and reviewing the literature would have taken substantial time by hand.

The livestream presenter judges the model’s estimated values reasonable and says the final paper’s value was around 1.2. The answer also notes that the older estimate has more uncertainty than more recent results. This is a single demonstration assessed by its presenter, not an independent validation of all scientific research tasks.

The demonstration illustrates the combination of image inspection, calculation and web browsing. I see tool use and agent scaffolding as important alongside pretraining and reinforcement-learning-based post-training. My further suggestion that tool use is itself a separate scaling law is a hypothesis, not a quantified law shown by this demo.

OpenAI’s launch charts compare competition math results. For AIME 2024, the chart shows o3 at 91.6% and o4-mini at 93.4% without tools; it also presents AIME 2025 and Codeforces results. These are benchmark-specific measurements, not a ranking of all uses, and tool availability matters when comparing results.

On the displayed Humanity’s Last Exam chart, o3 rises from about 20.3% without tools to 24.9% with Python and browsing, while o4-mini moves from about 14.3% to 17.7% with tools. Deep research scores 26.6% in that chart. I speculate that additional agent scaffolding and tool use help explain the difference; the chart alone does not establish the cause.

OpenAI also presents visual-reasoning, coding and instruction-following benchmarks. Its displayed SWE-bench Verified results are 69.1% for o3 and 68.1% for o4-mini, compared with 48.9% for o1. The SWE-Lancer chart expresses performance in terms of the dollar value attached to benchmark tasks; that is not evidence the models independently earned that money in the market.

I offer a theory about how the models were made: perhaps OpenAI took intermediate GPT-5 checkpoints and applied reinforcement learning to produce different reasoning models. That is my speculation based on public comments, not information confirmed by OpenAI. Strong benchmark results alone do not establish the models’ training lineage.

The cost charts compare estimated inference costs and performance at different reasoning levels. In the displayed AIME and GPQA examples, o4-mini improves on o3-mini at comparable estimated costs, and o3 medium and high compare favorably with o1 in the shown settings. Actual cost and performance vary with configuration and workload.

OpenAI also unveiled Codex CLI, an open-source coding agent that runs in a local terminal while using cloud-hosted models. It can work with local files and commands, bringing model reasoning into a coding environment. The original release described an experiment; its capabilities and approval mechanisms have evolved since this 2025 video.

This raises platform risk for developers building on a model provider: the provider can decide to build a product in an adjacent market itself. Codex CLI is a coding-agent example, even if it was initially a lightweight reference project. I think developers should consider diversification and open-source alternatives rather than relying on a single provider.

The launch also announced a $1 million initiative for projects using Codex CLI and OpenAI models, with grants in $25,000 API-credit increments. At launch, Plus, Pro and Team users were offered the models in the selector, Enterprise and Edu were scheduled to follow, and free users could try o4-mini through the Think option. These are historical launch details, not current availability.

I plan to test the models against the alternatives available at the time, including Gemini 2.5 Pro, particularly on coding. I am also interested in what developers think about the platform risks of building on top of OpenAI.