Laurie Voss on the Product Engineer After AI Coding Agents

I'm Laurie Voss, head of developer relations at Arize AI and formerly a co-founder of npm. Three years ago, Swyx argued that shipping AI products needed engineers as well as researchers. I think the change now reaches beyond a distinct AI engineer role: every kind of engineering job could be affected.

Coding agents have made producing code extremely cheap relative to the human time it previously required, though tokens still cost money. Everything after code generation did not become free at the same time. My thesis is that the bottleneck shifts first to code review, then to other stages as each constraint is addressed.

Eventually, I expect a constraint that agents cannot easily remove. If we identify the remaining work, we also need to work out how to train the next generation of engineers for it. Voss cautions that his talk compresses a large body of research into figures and slide citations.

Voss cites an analysis in which developers using autonomous coding agents produced 741% more code while releases rose about 30%. He views the difference as a sign that review is constraining delivery. The figures are from the analysis he cites; they do not by themselves establish a universal causal effect for every team.

He also cites an older Cisco and SmartBear review study, saying review accuracy falls as reviewers exceed roughly 450 lines an hour. His illustration is that a very large generated diff can require substantial human attention. The exact safe review rate depends on the change and conditions; the cited number is not a universal threshold.

The cost of producing plausible code has fallen, Voss argues, while knowing whether to trust it remains difficult. Automated review is one response. He cites adoption examples involving GitHub Copilot and company review policies, but says the tools are not yet reliable enough to assume generated changes are safe without scrutiny.

Voss describes an evaluation of four AI reviewers in which each found real issues but their findings did not overlap. He argues that running a single reviewer can miss defects. He also recounts an engineer's experience abandoning a no-review approach after serious problems, alongside company incident examples; those accounts should be understood in their specific contexts.

A coding benchmark can ask whether tests pass, while a maintainer must also decide whether to merge the change. Voss cites a reported difference between those kinds of scores, with maintainer acceptance lower. He attributes the gap to factors such as style, side effects, external systems and project history that may not be encoded in the test suite.

In his interpretation, maintainers contribute a sense of what good means for a particular project. That is information or judgment he believes agents do not yet have. It becomes the central idea he carries into the rest of the talk.

Voss invokes Eliyahu Goldratt's theory of constraints: improving a stage that is not the limiting constraint may not improve the whole system, and resolving one bottleneck exposes another. If review is accelerated, he expects operations and then security to become important limits.

His longer-term thought experiment assumes progressively better agents also handle deployment, scaling and security. He does not expect those problems to be solved within a year and suggests a five-to-ten-year horizon. That timeline and the assumption that these tasks can be automated are his predictions, not demonstrated outcomes.

The argument also assumes demand for software is effectively enormous. Voss points to poorly served needs at small businesses, governments and banks: people know how to build better tools, but time and cost limit what gets built. He asks what becomes scarce if both the desire for software and the ability to produce it expand dramatically.

Voss separates writing, reviewing, shipping and scaling from deciding what to build, defining good and making the result useful. He predicts agents will take on more of the former tasks while the latter remain difficult. This ordering is his forecast, not a guarantee that any particular task is immune to automation.

A customer may say they need to track orders, but that description means different things for a bakery, an auto-parts store or a manufacturer. The work is to discover the particular business's needs and what a solution must do. Voss argues that this knowledge resides with the people doing the work, so developers must talk with them.

Even closely related products can have different definitions of good: a calendar and a scheduling tool solve overlapping problems but have different requirements. Voss argues that a highly configurable product can still fail a user's specific need. He bets that describing the problem and its successful solution becomes the enduring constraint.

He connects this to Fred Brooks's distinction between accidental complexity—the difficulty of tools, syntax and implementation—and essential complexity in the problem itself. Voss interprets the gap between faster code production and delivery as consistent with that distinction, although the figures alone do not prove the explanation.

Voss also cites Shawn Grove's argument that code is a lossy projection of a specification and that whoever writes the specification takes on a programmer's role. For Voss, putting the desired outcome into words and tests is increasingly valuable.

If something can be verified cheaply, model builders can train against that verification. Voss describes work to define whether a pull request is mergeable and cites a reviewer that converts feedback into rules. As maintainers express their previously implicit knowledge, he expects the bottleneck to move again—including in operations and security.

Voss cites Andrew Ng on blurring product and engineering roles: an engineer who understands users, decides what to build and implements it can move quickly. He compares this with the older systems analyst role, which translated business problems into requirements.

His proposed product engineer finds out what people need, defines what good looks like and ships the solution while agents perform much of the typing. He points to Gergely Orosz's earlier description of product-minded engineers and argues that this orientation may become essential rather than merely desirable.

Voss cites growth in forward-deployed engineering postings and a reported compensation figure as signs that the market values similar skills. Job titles and samples vary, and those examples do not establish the trajectory or pay of the whole profession. He acknowledges that his chosen label is only one of many.

He recognizes a possible bias in his own thesis: people discussing automation often assume that the work they personally enjoy is the part AI cannot do. Voss asks listeners to treat his prediction with skepticism, including his belief that finding the real problem is the hard part.

The junior-developer pathway is his next concern. Historically, teams often trained juniors by giving them well-specified implementation tasks and reviewing the results. If agents perform more of those tasks, he asks how organizations will help new people acquire the judgment expected of experienced product engineers.

Voss displays an age-based chart of US software developer employment using ADP payroll data, indexed to October 2022. He says the youngest group has fallen while older groups have risen. The chart describes the displayed sample and period; it does not on its own prove AI caused the difference or describe every junior job market.

He argues that organizations need to train product-oriented engineers deliberately rather than relying on an accidental pipeline. He cites Google's associate product manager program and similar programs as examples, while estimating that many more trainees would be needed. Those demand figures are his estimates.

Universities may teach data structures and boot camps may teach frameworks, he says, but teams also need people who can sit with a customer, understand a business and write a useful specification. He urges engineering leaders to create opportunities for that learning instead of abandoning a generation of new developers.

Voss acknowledges that many people entered programming because they love the craft: clean abstractions, solving a difficult bug and making a machine do what they intend. They may not want a job centered on interviewing customers, and he has no easy answer for that loss.

He suggests that some of the creative satisfaction also comes from understanding a problem and turning it into a system. He expects programming as a craft to continue, comparing it to woodworking after furniture factories, but predicts that the paid role will increasingly emphasize different skills. That is his forecast, not an established outcome.

Voss predicts a difficult transition: junior pathways may weaken before new training exists, and people who love coding may dislike the emerging role. Looking further ahead, he expects a larger software profession shaped largely by product engineering and paid comparably to today's. The timing and result remain uncertain.

For teams acting now, he suggests identifying high-impact changes for human attention, delegating work that can be checked cheaply and expressing experienced people's judgment in specifications, rubrics and evaluations. His suggestion to narrow human review is a proposed strategy, not proof that automated reviewers are reliable for every lower-impact change.

He recommends measuring delivery, whether review comments are acted on and the gap between passing tests and a maintainer's willingness to merge, rather than treating code volume as the goal. Production behavior is another source of feedback. He closes by defining product engineering as discovering needs, writing down what good means and creating something people value.