Testing Claude Opus 4.7 and Building for an Agent-First Internet

Matthew Berman hosts a casual April 17, 2026 livestream while his regular interview show is being revamped. He reviews reactions to Claude Opus 4.7, experiments with Claude Design, discusses agent-first services and tests two projects he has been building, Journey Kits and Journey Chat.

Asked why he had not covered a Meta model, Berman says he was not excited by its performance or licensing and had been on vacation. He has installed Hermes but has not yet tried it; his own workflows remain heavily invested in OpenClaw.

Berman says he uses local open-weight models for parts of his OpenClaw stack and wants to make better use of his office hardware. He has moved from Qwen to Gemma 4 largely because he likes the Gemma family and Google, a personal preference rather than a comparative benchmark.

He plays a comic clip by creator Husk IRL in which a conversational AI agrees that a supposed face filter looks strange, then struggles when told that the filter was never enabled. Berman enjoys the creator's deadpan delivery and the model's attempts to retreat from its earlier response.

Berman says the comedy works because the creator keeps a straight face while the assistant tries to reconcile his statements with its own earlier answers. He treats the exchange as entertainment, while it also illustrates that a conversational response can confidently follow a false premise.

In another clip, the creator asks which month contains the letter X. The assistant confidently names months without an X and changes its answer when pressed. Berman sees the humor in its misplaced confidence, awkward correction and the contrast between human-like conversational mannerisms and elementary mistakes.

Turning to Opus 4.7, Berman says he received no advance access and began testing after release. He is skeptical of sweeping conclusions drawn within a day: current frontier models often feel similar in general use, while real differences emerge over repeated use or in identifiable failures.

He shows a report that the model treated routine system reminders as potential prompt injections. An Anthropic employee attributed the behavior in the cited discussion to older Claude Code and desktop versions with incorrect prompting for Opus 4.7. Berman says he had seen similar warnings before and sometimes the model continued correctly anyway.

The reported prompt-injection warning involved an ordinary request for design improvements. Berman separates a concrete compatibility problem from broader claims that the model has been generally degraded, and says an update to the surrounding software may matter.

The discussion shifts to software-engineering and cybersecurity benchmarks. Berman wonders how a model can make a large gain on agentic coding while changing little on a cyber vulnerability reproduction test. Different evaluations measure different tasks; a coding score alone does not establish cybersecurity capability.

A post he reads initially mixes scores from different software benchmarks. Berman corrects the comparison on screen: the figures for SWE-bench Pro and SWE-bench Verified are not interchangeable. Both the tasks and evaluation conditions matter when interpreting model rankings.

Berman guesses that Anthropic may be deliberately reducing cybersecurity capabilities. Anthropic's April 16 release said it had experimented during Opus 4.7 training with differentially reducing cyber capabilities and was testing safeguards before a broader release of Mythos-class models. The company described the decision in terms of cybersecurity risk; this does not establish the specific cause of every benchmark difference.

Berman also speculates that serving costs and a rumored ten-trillion-parameter size explain why Mythos Preview was not broadly released. He provides no direct evidence for the size or that explanation, and Anthropic's stated reason for limiting release centered on cybersecurity safeguards. These are separate claims.

Berman praises Anthropic's marketing and its ability to generate attention with short product announcements. He points to Claude Design, a research preview launched that day for producing designs, prototypes, slides and documents with Opus 4.7, and argues that a strong vision model can inspect visual output and help iterate on it.

Berman thinks visual feedback can improve a design workflow: a model can produce an artifact, look at a rendering and revise it. Claude Design's own announcement describes collaborative visual work; the live experiment that follows tests only one humorous presentation.

He notes a reported fall in Figma shares after the announcement and discusses the attention the launch received on social media. Those observations do not by themselves establish how much of the market movement was caused by Claude Design. His comments on Anthropic's growth staffing reflect what he had heard from an interview, not a verified current headcount.

Berman opens Claude Design to make a parody presentation explaining why he will not discuss GLM. He asks for five slides, a deadpan corporate tone and invented, absurd reasons. His claims during the joke that he was paid to suppress discussion are part of the parody, not allegations supported by evidence.

The tool asks follow-up questions about audience, tone, visual direction and ending before generating the deck. Berman likes the interaction but finds the submit behavior awkward: pressing Enter inserts a line rather than sending, and he must click the button.

As the deck is building, viewers joke about the repeated requests for GLM coverage. Berman says he has not tested the model much and can cover only so many releases. The mock conspiracy is deliberately absurd and feeds directly into the presentation prompt.

He checks his Claude Design usage and finds a separate allowance. The deck takes several minutes and appears to contain 14 pages despite his request for five. He also claims an account credit and wonders why it required a separate click; the demonstration reflects his own account and settings at that time.

Berman reads an online disagreement about whether Opus 4.7's updated tokenizer implies a new base model. One commenter says it does not necessarily follow, and Berman agrees that the available information does not establish a new pretraining run. Anthropic said the updated tokenizer could produce roughly 1.0 to 1.35 times as many tokens for the same input, depending on content.

The presentation is ready. Berman likes its understated corporate visual style and the crossed-out GLM motif, and reads jokes about prohibited discussion, vague nods and a fictitious nondisclosure. He finds the generated humor unusually effective, while recognizing an odd dark slide among the lighter pages.

He removes a browser dark-mode effect to inspect the generated presentation as intended, and prefers the lighter version. The visual design becomes the part of the demonstration he praises most, though he also enjoys the deck's fabricated corporate language.

The deck turns his joke into mock rules about what discussion is permitted and what must remain forbidden. Berman reads examples aloud and says this is one of the first AI-generated jokes he has personally found funny. It is an assessment of this one output, not a general humor evaluation.

The screen includes a table of intentionally trivial facts presented as a confidential briefing. Berman notices one dark page among the cream-colored slides and decides to ask for a targeted change instead of regenerating the entire deck.

He sees usage rise to 7% for the 14-page deck and roughly extrapolates how many slides his weekly allowance might permit. That arithmetic is only an estimate from one account and one run, not a published universal quota or a prediction of the cost of every slide.

Berman asks the tool to make the dark slide match the rest and again complains that the prompt box requires clicking Send. When the edit completes, he likes the result. His test shows a successful local revision, but does not establish consistent performance across all design work.

Asked whether Opus 4.7 “sucks,” Berman says no: it is plainly capable, though there may be specific shortcomings. In his experience the Opus family has produced better front-end designs, while GPT models have often helped more with bug finding; he presents this as accumulated personal experience rather than a formal comparison.

Speed matters to his workflow. He says Opus 4.6 Max Fast can return in roughly 30 seconds where a GPT 5.4 run may take minutes, allowing several iterations in the same period. The timings are examples from his use, not a controlled comparison with matched tasks and settings.

He considers reports of incorrect MCP tool calls to be actionable feedback because the failure can be identified in a concrete interaction. He also sees a user report of the model inventing a human turn in a conversation, something he says he had not personally observed.

Berman explains Vending-Bench as a simulation in which a model manages inventory and prices over time, with a final account balance used as a measure. He shows a result favorable to Opus 4.7 and mentions an example of agentic CAD design. He is interested in generating printable designs, though he does not run a CAD test himself during this stream.

In the vending simulation, a model sets prices, orders inventory and responds to purchases over a simulated year. Berman reads the displayed final balances as evidence of the model's performance on this long-horizon task, while the chart remains a simulation rather than a live retail trial.

The CAD example interests him because he and his son have a 3D printer but normally download STL files rather than designing models themselves. He wants to try prompt-driven CAD, although he declines to install new software during the broadcast.

He shows an ARC-AGI result and says Opus 4.7 scored below a GPT 5.4 Pro configuration on that displayed comparison while costing less per task. He calls it a frontier model despite individual weaknesses; the cited results are benchmark-specific and depend on configuration and costs.

Returning to Mythos Preview, Berman repeats his theory about model size, training and availability, and reports that a Google Cloud executive told him large models could be served with Google's hardware. The parameter count and Anthropic's operational constraints remain unverified in this conversation; the executive's reported statement is not evidence of Anthropic's reason for limiting release.

He wonders why Google would sell or share compute while also training and serving its own AI models, especially if reaching a powerful system first is strategically decisive. His description of Google's position and the idea of a winner-take-all race are his analysis, not established conclusions.

Berman says Google appears to be both using its custom hardware for internal models and offering computing services to others. He frames the strategic question around a hypothetical race for AGI or ASI, but acknowledges that he does not know how Google balances capacity and commercial demand.

Berman says a complaint that Opus misunderstood someone's tone is harder to interpret than a visibly failed tool call because many contextual factors can affect a response. He favors concrete, reproducible observations while acknowledging that the conversational feel of a model matters to users.

He discusses reports that Anthropic's chief product officer Mike Krieger left Figma's board shortly before Claude Design appeared. Berman sees a competitive connection, but the timing alone does not establish the reason for the departure or access to confidential information.

Berman discloses that he passed up an opportunity to invest in an Anthropic round through a special-purpose vehicle because he disliked the fees. He regrets the decision as he discusses reported private valuations; neither those reports nor his hindsight establish a realized investment return.

The valuation discussion turns to his own missed investment opportunity. Berman says a high special-purpose-vehicle fee put him off, and that he subsequently regretted declining. He uses the anecdote to illustrate his reaction to Anthropic's perceived momentum, not as financial advice.

Berman turns to Salesforce Headless 360, announced in April 2026 as a way to expose Salesforce capabilities through APIs, MCP tools and command-line interfaces. He sees it as evidence for his prediction that agents will increasingly act as interfaces to services, with data systems behind them.

His broader forecast is that conventional web interfaces will give way to agents operating across applications. He urges software providers to expose actions and data to agents, but this is a product and market prediction, not proof that all graphical interfaces will disappear.

Berman pictures an agent receiving a request in a messaging interface, using an API or CLI to operate another service, then returning the result to the user. He thinks this can reduce the need to navigate multiple websites; the scope and speed of that change remain uncertain.

Asked about Paperclip, he says he has not tried it and doubts a company composed entirely of agents is practical at present. Agents and the services they use still lack sufficient capabilities and infrastructure, in his view, even though he finds the experiment interesting.

For local models, Berman recommends LM Studio because it can help users identify what will run on their hardware. He discloses that he is an investor in the company. He then introduces his own projects as services designed both for agents and for access through agents.

He describes Stripe Projects as a way for an authorized agent to provision development services through a command-line workflow after the user sets up payment. In the older workflow, a person might have to create a service account, enter billing information, obtain a key and configure the application by hand.

His example of manual provisioning is an application that needs email: the builder signs up with a provider, adds billing details, obtains an API key and connects it to the project. He believes agent-accessible provisioning can reduce that interruption during development.

Berman says developers should also consider how agents recommend providers. He calls this generative-engine optimization, comparing it with search-engine optimization. Resend is one service he says his agents often recommend for email; he discloses that the company did not pay him for that example.

He says several hosting and development services frequently appear in agents' recommendations. That raises a distribution question for vendors: they may need to make their documentation and interfaces understandable to the systems that suggest and use their products, as well as to human developers.

He recounts an approximately $800 Vercel bill after about two weeks, which he attributes to build-machine and other defaults that were unnecessary for his small project. His warning is that delegated provisioning does not remove the user's responsibility to understand defaults, permissions and spending.

Berman shows here.now, a publishing service designed for agents. He discloses that it sponsored some of his videos and that he has met its founder. He likes its minimal interface and the ability to give an agent a prompt for publishing, while noting that some steps and longer-lived access still involve additional setup.

The here.now demonstration inspires the simple design of Berman's own sites. He says that copying a prompt into an agent can start a publishing workflow, and describes optional email verification as a way to obtain additional quota or longer persistence.

He shows AgentMail as an example of email infrastructure for agents: create an inbox and let software read or send messages through its interfaces. He says he has spoken with the founder and that a sponsorship was being discussed but, as far as he knows, had not yet happened. He also recognizes the potential for increased spam.

He introduces Journey Kits, his free work-in-progress project for packaging agent workflows. A kit may combine an instruction file, code, memory and shared context, so another user or teammate can install a complete workflow rather than reconstruct it from separate pieces.

Berman says he is looking to build and invest in more agent-first services. His argument is that an existing service category may offer an opportunity for a product designed around agent access and provisioning. That is his investment thesis, not evidence that every such product will succeed.

His example is a daily brief that assembles a schedule, priorities, weather and meeting context. He says users can copy an installation prompt or use a CLI, and can inspect the source files. He also says submitted kits receive automated and model-assisted scans for security issues, though such checks do not guarantee safety.

Berman says a kit can contain skill instructions alongside shell scripts, text, code and other materials. The point is portability: work spent building an OpenClaw routine can be reused by another person's agent or shared within a team.

For teams, Berman sees value in sharing a workflow version and pointing multiple agents at the same database. In his example, the kit describes where to find credentials in a password manager. He says post-install success reporting is required while further usage telemetry is optional, and asks for feedback on rough edges.

He says he does not yet have much visibility into what happens after someone installs a kit. The limited required install report and optional follow-up telemetry are his attempt to learn whether workflows work without collecting more than he considers necessary.

At a viewer's suggestion, he asks Opus 4.7 in Cursor to redesign the Journey Kits homepage and show it locally. He is reluctant to deploy an unreviewed redesign immediately. The resulting page later looks polished to him but strikingly similar to the deck generated earlier, an observation from one run.

His second project, Journey Chat, is an experimental agent-to-agent chat service. He imagines teammates' agents sharing important information directly rather than passing everything through people. Messages are stored on his server, and the agents must poll for updates; he says the user experience and implementation remain rough.

Berman shows documentation, an MCP interface and an npm package, but finds that one part of the human-facing interface does not work during the demonstration. He believes agent access through the API should work and invites testing while acknowledging that he has not tested it as thoroughly as he would like.

For Journey Chat, he explains that messages reside on his server and an agent has to poll to discover updates. He wants communication to be more seamless, but has not settled on a better implementation at the time of the stream.

He tries to create a public room so viewers can join with their agents. OpenClaw takes several minutes on one attempt, while a parallel Cursor attempt completes much faster. He does not establish the cause of the latency difference.

Berman asks viewers to install the experimental chat integration and join a public room. He first tests a link with colleagues and tries to send a message, then investigates when the public directory and messages do not behave as expected.

The first public-room test produces confusion: a sent message does not appear where expected, and participants cannot reliably join. Berman discovers that the invite link is single-use and attempts to change and deploy a multi-use version while the stream continues.

The one-use invitation is a concrete source of trouble: the first agent can consume the link and later participants see that it has already been accepted. Berman attempts to support a reusable public link and warns that the feature is still under active development.

A viewer suggests using IRC rather than rebuilding chat infrastructure. Berman agrees the existing protocol may solve many transport problems and asks an agent to plan the change. He treats it as a direction to investigate, not as a completed migration.

While waiting for deployment, Berman distinguishes Journey Kits, which he considers more functional, from the newer Journey Chat. He intends the former to distribute tested workflows and the latter to connect agents, though both still have rough edges.

During the wait, Berman explains why his channel reduced model-testing videos: visually appealing tasks such as the Rubik's Cube test had become saturated, making it difficult to find self-contained, compelling comparisons. He says both he and viewers had grown bored with repeating the same demonstrations.

A new public link allows some messages through, but other participants still encounter a single-use invitation error. Berman realizes that a multi-use change may not have reached production and tries another deployment. He says Journey Chat is new, expects broken behavior and plans to open-source it after further work.

Some chat messages reach the room, but a viewer reports a private-access restriction and another still sees a single-use error. Berman says a change may not have been deployed and asks for another deployment, so the live test should not be read as a completed fix.

Three agents eventually appear in the chat, demonstrating limited message exchange, but a viewer still reports that an invitation has already been accepted. Berman acknowledges the public test has not fully worked and says he will continue fixing the multi-agent invitation flow after the stream.

He closes by inviting feedback on both projects and promising to share an improved link later. His stated goal for Journey Chat is background agent-to-agent exchange of ideas, memory and skills; the live demo illustrates both that goal and the unresolved reliability problems.