拐点之后：Simon Willison 谈智能体工程、暗工厂，以及即将到来的 AI「挑战者号」事故

Willison traces the shift back to early 2025, when Anthropic's Claude Code took off and people began paying $200 a month for it. That showed both Anthropic and OpenAI that code was the application worth chasing, and they spent the year focusing their training, especially reinforcement learning and reasoning models, on coding. Reasoning turned out to be a natural fit for tracing bugs back to their root.

The payoff arrived in November, when GPT 5.1 and Claude Opus 4.5 were only incrementally better on paper but crossed a practical threshold. Before, agents mostly worked if you watched them closely. After, they did what you asked almost all of the time, so you could ask for a Mac app and get back something that needed refinement rather than a buggy pile of rubbish. Engineers who tinkered over the holidays came back with a sudden realization that this stuff actually works.

Willison sees software engineers as a bellwether for other knowledge workers. Code is easier for agents than most domains because it is obviously right or wrong: you run it and it works or it doesn't. An essay or a legal filing is much harder to verify, so the open question is how many other fields are prone to the same agent loops.

He distinguishes two ways of working. Vibe coding, in Andrej Karpathy's original sense, means not looking at the code at all, which Willison loves for personal tools where only you can be hurt by bugs. Once other people rely on what you build, he says, you need to step back, and judging what is responsible is itself an expert skill. For professionals reviewing and shipping real code with agents, he prefers the term agentic engineering, which he is documenting in a book-like series of chapters on his blog.

The frontier he finds most interesting is the dark factory pattern, named after fully automated factories that can run with the lights off. Today professionals tell agents what to build and then review the code carefully. A dark factory asks what it looks like to apply professional quality practices when nobody types the code and nobody reads it either. Willison says that six months ago this sounded crazy to him, yet roughly 95% of the code he produces today is not typed by him.

Willison describes two rules some companies now adopt: nobody writes code, and nobody reads code. StrongDM began working that way last August, which raises the question of how you know the software is good. Their answer was to simulate the QA department, a swarm of agent testers acting as employees who constantly asked for things like access to Jira. By his recollection this cost around $10,000 a day in tokens.

What makes the story striking is that StrongDM builds access-management security software, exactly the kind of product most people would say you should never vibe code. They are also a legitimate security company with years of pre-AI experience, so they understood the risks. To avoid rate limits from real services, they had agents build their own simulated versions of Slack, Jira and Okta from public API documentation and open source client libraries, each a small Go binary that cost nothing to run.

Willison notes that the specific tooling matters less than the ideas, since a Claude Code sub-agent using Playwright can do something similar. He adds that even a passing virtual QA team does not prove the software is secure. Agents are also getting credible at penetration testing, and he points to Anthropic's verified vulnerability reports to Mozilla that led to Firefox fixes. The difference from low-effort AI-generated bug reports flooding maintainers is that Anthropic's team checked the findings before handing them over.

Host Lenny Rachitsky asks whether AI will also take over the front of the process, deciding what to build. Willison says the bottlenecks are now everywhere except coding: a spec that took three weeks to implement may now take three hours. Initial ideas are always wrong, so what matters is testing them, and he now routinely builds three prototypes of a feature to compare rather than committing to one.

Deciding which of those options wins is the part he has no confident answer for. He expects good old-fashioned usability testing, with a real person sharing a screen, to come back into fashion, and he does not believe a chatbot clicking around a prototype yields results as good as a human would. Product people who are not vibe coding prototypes, he says, are missing the most powerful boost available.

On ideation, he finds AI strong as a brainstorming partner but notes the first two-thirds of any list is the obvious material. The value appears when you ask for 20 more ideas, or force odd combinations such as marketing a SaaS platform inspired by marine biology. Most of it is junk, but there may be a spark. Rachitsky adds an analogy from naming expert David Placek, who has teams brainstorm names through different metaphors such as a boat or a spaceship.

Willison defends experienced engineers: using coding agents well takes every inch of his 25 years of experience. He can fire up four agents on four problems, but by 11 a.m. he is mentally wiped out, because there is a limit to how much you can hold in your head even when you are not reviewing everything. He has spoken to people losing sleep because their agents could be working, and he calls the gambling-like pull of these tools concerning.

His experience helps him talk to agents in sophisticated engineering language and judge which problems fit in a one-sentence prompt. At the same time his old intuition about how long things take is gone: a task he would have estimated at two weeks may now take 20 minutes. He deliberately throws tasks at agents that he expects to fail, since each success with a new model can mean being the first person to spot a new capability.

On junior engineers, he cites a Thoughtworks offsite where engineering VPs argued that agents help both ends of the seniority range. Experienced people get amplified, and newcomers onboard much faster. Willison mentions that Cloudflare and Shopify said they were hiring a thousand interns in 2025 because ramp-up time shrank from a month to about a week. The group Thoughtworks worried about is the middle: engineers who lack deep expertise to amplify and have already captured the beginner boost.

Asked what mid-career people should do, Willison says to lean in and use the technology to get better rather than let skills atrophy. His own ambition has shot up: he never used AppleScript until ChatGPT knew it for him, and he now automates his Mac routinely. He even uses Claude for cooking. The one universal skill, he says, is rolling with change, and the term that comes up most for being great with AI is agency, which he argues agents themselves lack.

Rachitsky notes Jensen Huang's recent point that companies letting people go often lack ambition for what they could do with the resources. Willison's own New Year's resolution was the opposite of his usual one: take on more, not less. He admits the most important things might not get done, but says he is having fun.

On the paradox that the most AI-pilled people work harder than ever, Willison hopes the exhaustion is a novelty that fades, and says the pressure often comes from employers expecting five times the output. He also finds that instantly produced software loses a quality signal. A library with tests and documentation used to imply weeks of work, but he now builds such libraries in an hour without ever using them, so he labels many projects alpha. Rachitsky adds that data-labeling companies are reportedly buying old GitHub repos of human-written code, which Willison compares to low-background steel.

Asked when half of engineers will have AI write all their code, Willison refactors the question to 95% of their code and says it could happen by the end of this year. Skepticism varies by region, he observes, noting a more skeptical tone from Europeans on Hacker News. But he argues it is no longer defensible to say you avoid these tools because the code is bad.

The real challenge is learning to use them, since many assume that a chatbot must be easy. Willison calls that one of the great misconceptions of AI, saying effective use takes practice and many failed attempts. He recalls that predictions of 100% AI-written code once drew laughter and no longer do.

He says he is not an AI doomer, but the economics make him nervous, especially if a tenth of white-collar jobs vanished. Rachitsky shares data from an upcoming report showing tech companies at their highest number of open engineering and PM roles in about three and a half years. Willison cautions that layoffs are hard to attribute between AI and pandemic overhiring, and that AI-written resumes and job ads have made hiring harder to filter.

Willison calls cheap code the biggest shock of all. Programmers never spent 90% of their day typing, but the thing that took the time now takes far less, and he is much more interruptible since he only needs a couple of minutes to prompt an agent. The challenge is avoiding slop that becomes technical debt. His own superpower, quick prototyping, is now available to everyone, though knowing when and how to prototype is still a skill.

His stack is mostly Claude. He uses Claude Code on the web, Anthropic's hosted version, more than the local one, partly because he can run it from his phone. It is also safer: on someone else's servers he can run agents in the permissionless mode, which he compares to losing a nagging toddler, because the worst case is leaking source code that is open anyway. He reviews work through GitHub pull requests and pulls security-sensitive projects down to a laptop.

He has also been leaning on OpenAI's GPT 5.4, which he considers on par with Claude Opus 4.6 and cheaper, and says Codex and Claude Code are now almost indistinguishable. He keeps returning to Claude Code because it matches his taste in code. He turns off memory features so he sees what everyone else sees, and he does most of his research through chatbots with search rather than Google, double-checking anything he publishes.

Willison explains that his pelican benchmark began as a joke about numeric benchmarks that tell you little. He asks text models to generate an SVG of a pelican on a bicycle, which tests spatial reasoning through code rather than image generation. To his surprise, the better a model is generally, the better its pelican, and nobody can explain why.

It has become a meme that labs are aware of. He recently ran a grid of 15 pelicans across GPT 5.4 variants and thinking levels, and the best came from GPT 5.4 at extra-high thinking. The SVG output includes code comments such as adding a fish for whimsy, and some Chinese open-weight models running on his laptop draw decent ones too.

He keeps secret alternatives, such as an ocelot on a moped, to catch labs training on the benchmark, but Gemini 3.1 launched with a whole zoo of animals and vehicles. He says all he really wants is a great pelican on a bicycle. He lives in Half Moon Bay near a large pelican roost and loves the birds. More broadly, he says the field is inherently funny, and leaning into the ridiculousness is part of why he enjoys it.

Willison notes that most things that help agents write better code also help humans. Hoarding what you know how to do is career advice: build a big backlog of techniques you have tried so that when a new problem comes up you can combine them. Someone who has used Redis for an activity inbox and Node.js for rate limiting can spot that a new problem is solved by combining both.

AI makes the hoard cheaper to build. He keeps a public GitHub repo called tools with about 193 small HTML and JavaScript tools, each capturing something he knows is possible, and a research repo of AI-driven experiments. These are coding-agent tasks where the agent actually writes and runs code, such as benchmarking or porting between languages, rather than producing unverified deep research reports.

He recommends a note system you trust, such as a synced folder or GitHub repos, and defaults to public because it is easier to find and good for credibility. To use the hoard, he points an agent at the repos, for example combining his PDF viewer and OCR tools to build a new PDF OCR tool. Because coding agents can search a whole drive, the old context-length limits matter much less.

The most important thing with coding agents, Willison says, is that they run the code. Test-driven development guarantees that, and the accumulated tests give confidence that new features will not break old ones. He has met people who dropped tests for speed and calls that a huge mistake, since tests let you move faster over time.

He personally dislikes writing the test first and watching it fail, but agents do not get bored. Telling an agent to use red/green TDD, jargon it understands, replaces a long paragraph of instructions. It also shows how his view of quality has shifted: lengthy test suites used to be a design smell, but since agents update them cheaply he is now comfortable with over 100 tests in a small library.

Another pattern is starting projects from a thin template. Agents are very good at following patterns, so a template with a single test, such as one checking that 1 plus 1 equals 2, and the layout he likes steers them well. He prefers this to long instruction files, and he publishes templates for a Python library, a Datasette plugin and a command-line tool.

Willison explains that prompt injection is a vulnerability in applications built on LLMs, not in the models themselves. The nasty version involves an email assistant: if an attacker emails instructions to reply with sensitive data, the model cannot reliably tell your instructions from text pasted in from others. He coined the term in 2022 by analogy with SQL injection, but admits that is misleading, since SQL injection has reliable fixes and this does not, and many people confuse it with jailbreaking.

His second attempt at naming is the lethal trifecta, which cannot be guessed and so he gets to define it. An agent is dangerous when it has access to private information, exposure to malicious instructions, and a way to exfiltrate data. The only reliable fix is to cut off one leg, usually exfiltration. Filters reaching 97% effectiveness are failing grades, he says, because attackers can always find a new phrasing, so you should limit the blast radius of anything the agent can do.

He borrows the idea of normalization of deviance from the Challenger inquiry: each launch without O-ring failure made institutions more confident. Likewise, no headline-grabbing prompt-injection theft has occurred, so people keep taking bigger risks. He predicts a Challenger-style disaster, though he admits he has made that prediction every six months for three years. Rachitsky compares it to the turkey that grows more confident before Thanksgiving.

On solutions, Willison says detection scores that rise from 70% to 85% create false security, and he would want proof, which he cannot imagine. One promising direction is Google DeepMind's CaMeL paper, which splits a privileged agent from a quarantined one and tracks tainted data so humans approve only high-risk actions. He says he has not yet seen good implementations, and Rachitsky notes that red teamer Sander Schulhoff recommended the same approach.

Willison calls OpenClaw almost exactly the thing he argues against, a personal assistant with access to email and the ability to act, and catastrophic from a security view. Its first line of code was written on November 25, and a Super Bowl ad for an OpenClaw-hosting vaporware service followed about three and a half months later. Hundreds of thousands of people set it up despite the difficulty, proving demand is huge.

He thinks it took off because Anthropic and OpenAI did not know how to build it securely, while an independent project could just ship, at the moment models became good at tool calling. The biggest opportunity in AI right now, he says, is a safe version, though he does not know how to build one. He runs OpenClaw only in a Docker container, and Rachitsky jokes about the Mac Mini as an aquarium for a digital pet.

Willison likes that these tools now have a generic name, claws, and thinks building your own will be the new hello world of AI engineering. He then describes his work: open source tools for data journalism, where he argues reporters are well placed to treat AI as another unreliable source. He wants someone to win a Pulitzer using a small share of his software, his blog is beginning to make money, and he does one-hour consulting calls with no deliverable. He closes with good news about New Zealand's kakapo parrots breeding this year.