Did Claude Get Worse? Tracing Regressions Across the AI Coding Stack

Users keep reporting that Claude is worse: refusals, weaker coding, and sessions that lose the plot. I have often dismissed regression claims, but recently I have experienced problems myself. That does not tell us which layer changed, so I want to trace the entire system.

I asked Claude Code to help debug a Dropbox app that would not open. It said the problem was outside its software-engineering remit and suggested checking settings I could not reach. In my experience another coding agent solved the issue quickly; one anecdote does not establish a general regression, but it illustrates the kind of refusal I mean.

There are several distinct failures: an API may block a request, the assistant may refuse or abandon it, it may produce a worse solution, or it may lose the user's intent over a long conversation. Those symptoms can originate in different places, so calling all of them a weaker model conceals useful distinctions.

The MarginLab tracker shown in the video fell from roughly 57% to 55% on its weighted measure over the period I examined. It tests a changing combination of models and Claude Code; the chart is a signal worth investigating, not a controlled experiment isolating model weights or a definitive explanation of the decline.

A request goes through several layers: your prompt and prior conversation, the agent harness and its tools, the API, the serving infrastructure, and the model. The model generates output based on the supplied context. During ordinary inference that context changes the activations and predictions, not the stored model weights.

The harness supplies system instructions, tool definitions and other context in addition to the user's message. A tool call returns a result to the conversation, after which the agent normally makes another model request. Bad instructions or awkward tools can therefore affect both the answer and the cost of getting there.

I also encountered a refusal while working on a cryptographic puzzle. Filtering, routing, hardware and model behavior are all plausible places to investigate, although an observed refusal alone does not establish exactly which component caused it. Tool-heavy work creates many requests, so there are many opportunities for configuration differences to matter.

Our expectations matter too. As models improved, I began assigning harder tasks and raising my standard for acceptable work. A failure that once seemed unsurprising can now feel like a regression even if the task has changed. That is a reason to compare like with like, not to dismiss every complaint.

Custom skills, MCP servers and plugins also add instructions and context. Useful additions can help, but unnecessary material may distract the agent or consume the space available for relevant information. Users and product developers both influence the effective prompt.

One Claude Code example attempted to edit package.json but was told to read it first. It searched, retried and failed again before invoking the expected read tool. I see this as a harness interaction that wastes calls and fills the history with failed attempts, even when the underlying model understood the task.

Matt Mower's feature-implementation benchmark compared the same models in different harnesses. The result I cite showed Opus scoring around 15 percentage points lower in Claude Code than in Cursor. The result depends on the benchmark and configuration, but it demonstrates why the model name alone is not a complete description of an agent.

In another test, Claude mentioned repeated system reminders that treated my personal site as malware and said it would ignore them. Even an instruction that the agent correctly rejects can occupy attention and context. I worry that accumulating confusing reminders and tool errors makes successful work harder.

The Terminal-Bench table in the video lists Opus 4.6 with different harnesses: ForgeCode at 81.8%, Capy at 75.3%, and Claude Code at 58.0%. These are particular submissions at particular dates, not a universal ranking. I also cite a separate API hallucination benchmark that fell and later recovered, suggesting that problems are not necessarily confined to the harness.

Tokenization is another layer. It converts text into pieces the model processes. Different tokenizers can represent the same code with different numbers of tokens; my demonstration shows that whitespace and indentation can be grouped very differently between older and newer tokenizers.

In my example, the GPT-5 tokenizer uses about 225 tokens for a code sample, while an older GPT-3 tokenizer uses over 400. I have seen older models repeat whitespace at length, which wasted tokens and money. Token count is only one property of a tokenizer, however, and does not by itself prove a particular model will perform worse.

Anthropic said Opus 4.7 uses an updated tokenizer and that the same input may take roughly 1 to 1.35 times as many tokens depending on content. One independent measurement cited in the video found about 1.47 times for technical documents and 1.45 times for a CLAUDE.md file. These are examples, not a universal multiplier.

More tokens for the same material can affect usage limits, cost and how much fits in the context window. I suspect this could affect quality as well, but the analogy between a longer token sequence and a longer document does not establish a causal quality loss; the model and tokenizer have changed together.

I have little direct information about how different chips affect these requests, so explanations involving hardware are speculative. What we do have is Anthropic's September postmortem describing three infrastructure bugs that intermittently degraded Claude responses in the preceding months.

The postmortem describes Sonnet 4 requests mistakenly routed to a configuration intended for an upcoming long-context version. The issue initially affected around 1% of requests, then a load-balancing change increased the affected share to 16% at its peak. Anthropic also documented output corruption and a sampling-related issue, and said the bugs had been fixed.

I interpret that history as a warning about long-context routing and speculate about a connection to later one-million-token defaults and hardware choices. That is my hypothesis, not something established by the postmortem: a specific 2025 misrouting bug does not prove that all later million-token models are worse or that requests were deliberately sent to particular chips.

Anthropic later made the million-token window generally available for Opus 4.6 and Sonnet 4.6 without a long-context pricing premium. Long contexts demand resources, and I wonder whether serving economics influenced the decision. Pricing and availability alone cannot reveal the provider's routing or the effect on answer quality.

In the version of Claude Code I demonstrate, the model selector shows a million-token option and does not offer the old choice in the same menu. I show an environment setting for disabling the million-token context. Users should check the documentation for their own version before relying on this workaround.

AMD AI director Stella Laurenzo's team published a detailed GitHub issue analyzing 17,871 thinking blocks, 234,760 tool calls and 6,852 Claude Code session files. The authors report a correlation between changes in visible thinking and deterioration in their complex engineering workflows. This is a substantial observational report, not an isolated controlled experiment.

I worry that hiding thinking from users might also deprive later model requests of useful reasoning history. That internal-mechanism claim is my speculation: visibility in a client is not enough to determine exactly what the provider preserves or passes between turns. The report's timing correlation does not establish that redaction caused the reported regression.

The report describes visible thinking becoming increasingly redacted in March and uses signature length as a proxy to estimate thinking depth. It reports a substantial decline in that proxy. The estimate depends on its assumptions, so it should not be treated as a direct measurement of every internal reasoning token.

The team's behavioral indicators included more frustration, apparent loops and early stopping. Its observed read-to-edit ratio fell from about 6.6 to 2: the reported share of read calls dropped from 47% to 30%, while edits rose from 7% to 15%. I recognize the pattern of an agent editing before researching, though these observational measures alone do not identify the cause.

The authors also counted many instances in which the assistant asked permission to continue or declared a stopping point. The complaint is not that asking is always wrong; it is that unnecessary pauses can prevent an agent from completing work the user already authorized.

Between February and March, the report shows roughly 80 times as many API requests and 170 times as many input tokens, while the number of user prompts stayed close. The authors explicitly note that increased concurrent agent sessions contributed to the growth; the figures cannot be attributed solely to degraded behavior.

Claude Code's Boris Cherny responded by discussing adaptive thinking and the medium effort default, and suggested setting effort to high for users who prefer more thinking at additional time and token cost. That is one configuration to test against the same workload, not proof that all the reported problems have been resolved.

My conclusion is that regressions can emerge from many interacting layers, and that Anthropic should offer more reliable behavior and transparency. I suggest trying alternatives, including Codex, based on my own experience and the reports I saw. Those anecdotes and selected charts do not establish that other providers never regress or rank every product across all tasks.

Users depend on these tools for daily work and need a way to distinguish changing expectations from a real change in performance. My request is for clearer explanations, reproducible evaluations and infrastructure that can be trusted when something goes wrong.