GPT-5 at Launch: Routing, Coding, Benchmarks and Reliability

I spent the week before launch testing GPT-5 and came away impressed. OpenAI describes it as a unified ChatGPT system that can respond quickly or spend more time thinking on harder requests. The router chooses an approach based on task complexity, tool needs and the user’s expressed intent. These product descriptions and access details are from the August 2025 launch.

In the thinking interface I can ask for a long story and select “get a quick answer” to interrupt the longer thinking process. A user can also explicitly ask the system to think harder. OpenAI says the router learns from signals such as model switches, response preferences and measured correctness.

At launch OpenAI made GPT-5 available across ChatGPT tiers, with different usage limits; Plus subscribers received more usage and Pro subscribers had access to GPT-5 Pro for extended reasoning. OpenAI also announced GPT-5, mini and nano variants for developers. These launch terms should not be assumed to describe current access.

OpenAI emphasizes practical work as well as benchmarks. In my own coding tests, the model performed very well, especially on frontend generation and larger-repository debugging. The GPT-5 API model has a 400,000-token context window. I was particularly struck by its handling of layout, typography and whitespace.

OpenAI’s sample apps include a simple jumping-ball game, a pixel-painting tool, a typing game and a drum simulator. The pixel tool responds a little slowly, and the runner feels basic, but the examples show how quickly a small interactive frontend can be produced. In my subjective testing, creative writing was strong, though jokes and humor remained weak.

The video is sponsored by Box, which presented its own enterprise metadata-extraction evaluation comparing GPT-5 with GPT-4.1. Its displayed chart reports 90% overall accuracy for GPT-5, five percentage points higher on that evaluation; results are specific to Box’s test and should not be generalized to all documents or tasks.

I often use AI to translate complex medical notes or test results into language I can understand. OpenAI reports improved performance on its health-related HealthBench evaluations. This can be a useful aid to comprehension, but it does not replace a medical professional; for serious health issues, consult a clinician.

OpenAI’s launch chart reports GPT-5 Pro at 100% on AIME 2025 with Python and 96.7% without tools. I wish the original charts had included more competing models, so I ask GPT-5 to search for other results and produce a comparison chart. It shows its summarized reasoning and uses web searches as it works.

The generated comparison shows figures for Grok 4, Gemini 2.5 Pro and Claude Opus 4.1 as well as GPT-5 configurations. Such a chart is useful for exploring published results, but different tools, settings and sources can make direct ranking misleading. OpenAI explicitly cautions against comparing tool-assisted AIME scores directly with scores without tool access.

The launch presentation also reports 32.1% for GPT-5 Pro on FrontierMath tiers 1–3 and discusses the Harvard–MIT Mathematics Tournament, GPQA Diamond and Humanity’s Last Exam. Berman’s generated comparison cites other companies’ results, including a higher Grok 4 Heavy score on one tool-enabled Humanity’s Last Exam configuration. These numbers describe particular evaluations, not overall intelligence.

For coding, OpenAI’s displayed SWE-bench Verified result is 74.9% for GPT-5 with thinking, compared with 30.8% for GPT-4o, and its Aider Polyglot result is 88%. OpenAI notes its SWE-bench evaluation used 477 verified tasks. Other charts cover tool calling, visual problems, video understanding and economically important tasks.

Speed stood out in my own testing: the model often produced high-quality responses quickly. OpenAI also reports that, with web search enabled on anonymized prompts representative of ChatGPT traffic, GPT-5 responses were about 45% less likely to contain a factual error than GPT-4o responses; GPT-5 thinking was about 80% less likely than o3. These are relative results for that evaluation, not universal error rates or a guarantee of accuracy.

OpenAI also says the model is more honest about missing tools and tasks it cannot complete. In one example, a prompt asks where curves cross on a chart that was not supplied. The displayed o3 answer invents an intersection, whereas GPT-5 says it cannot see the chart and asks for the image. Recognizing missing information is preferable to confidently fabricating an answer.

OpenAI describes a safety-training approach called safe completions, intended to offer helpful information where possible while remaining within safety boundaries, rather than relying only on binary refusal. I am skeptical that changing how the model responds can prevent every determined malicious user from getting around it. That is my opinion, not an evaluation result.

I also tested sycophancy: when I presented a bad business idea, the model urged me to test it before investing all my money rather than simply agreeing. OpenAI says it improved instruction following and custom instructions. At launch it also previewed four optional personalities: Cynic, Robot, Listener and Nerd.

OpenAI described GPT-5 Pro as an extended-reasoning variant using additional parallel test-time compute for difficult tasks. I had tested GPT-5 and GPT-5 Thinking but was unsure whether I had used Pro. I am excited that the base GPT-5 experience was offered to free users at launch, even though higher usage and Pro access were tier-dependent.