Dan Shipper: AI makes generating ideas cheap; people still choose what matters

Every’s conference asked what great human work looks like after automation. Introducing its closing talk, Natalia Quintero says Every co-founder Dan Shipper analyzed a customer survey, built a simple customer-management tool and a game, and contacted customers with the help of a Claude subscription.

Quintero says those activities previously might have required several specialists, yet the output from a generic question about the survey would likely look much like anyone else’s. She introduces Shipper’s argument that the perspective brought to a model makes work distinctive.

Shipper contrasts two ways of thinking about the future. An aggregate view collects many observations and extends a trend; an intimate view examines one situation closely, including its messy details, and reasons outward from it. He says the first is powerful, but neither view tells the whole story alone.

As an example of the aggregate, he points to the relationship between model capability and the data and computing used to train models. He also cites OpenAI’s GDPval benchmark, in which experts compare outputs from models and humans on real work tasks.

Shipper presents benchmark figures from 2024 through 2026 as evidence of rapid improvement. He says looking only at those results can make the future of human work look bleak. The measures compare model output with human work performed without AI.

In his own week, Shipper used agents to ask many questions of a quarterly customer survey, generate a game for an all-hands meeting and build a customer relationship management tool for outreach. He says that, in earlier terms, the work would have involved a data scientist, a game developer, a marketer, an engineer and a business development representative.

He describes seeing a model perform familiar work as an “agency rupture”: people often identify with what they do, and automation can unsettle that sense of self. For him, the way to examine what an aggregate chart misses is to look closely at how a real person uses the tools.

Shipper says this style of work, with a person directing agents between other responsibilities, is already familiar to many attendees. A few years earlier, some people saw language models as unlikely to improve much, while others predicted that jobs would disappear almost immediately.

Using GPT-3 in 2022, Shipper had predicted that improved AI might let an individual have an impact comparable to a much larger organization. He now sees his own work as an example and asks what that means for the value of a team.

Every has about 30 people, he says, despite the work its agents can perform. To explain why, he returns to the same customer survey: every member of the team could inspect the data and analyze it using AI, but they did not all choose the same question or result.

Shipper noticed an opportunity to sell Every to teams and built his own tool for customer outreach. Yash, a go-to-market engineer focused on subscriber growth, found that some first-time subscribers left after one month because they received too much email.

Douglas, who led marketing and organized the conference, found a different pattern: some readers felt lonely using AI because people around them did not share their enthusiasm. Jack, a senior editor, made a clay-animation game show to communicate the survey results.

Each person had the same data and access to Claude, but emphasized something different. A default request to the model produced observations Shipper considers real, such as the value of the writing and bundle, yet no one on the team simply submitted that generic response.

The team did its analysis while preparing for the conference. Shipper says Yash not only identified a growth issue but also wrote a proposed code change, while helping build the event’s live-streaming experience. The tools let individuals explore and act on more ideas in the time available.

Could a model have produced the ideas the team found? Shipper says it could produce many candidates, but selecting the useful one is harder. Every’s head of evaluations, Mike Taylor, tested whether Claude could predict what Shipper would say in Slack.

In the experiment, Shipper says the model could include his answer among 400 candidate answers about 93% of the time. When asked to rank the candidates, however, his answer appeared in the top tenth about 60% of the time. He uses the result to distinguish cheap generation from the harder problem of choosing.

A team member has responsibility for a particular part of the business and a perspective on what matters there. Those constraints narrow where the person looks and help them decide which output deserves attention, ownership and action. Different people can therefore explore different parts of the same set of possibilities.

Shipper expects models to improve at ranking too. But he argues that comparing a future model with a human frozen at today’s abilities is misleading: in the future, the person will have more experience and will also have access to newer models.

He compares a future team member using a future model with a default request to that same model. Since both sides have access to the tool, he says the difference lies in the person’s responsibility and perspective: what they choose to investigate, how they direct the model and which result they act on.

As a model becomes widely available, Shipper expects its default capabilities to raise the baseline for everyone. An unusual insight may later become part of that baseline, but a person continuing to use improved models can also build on it. He sees value in the way people direct those shared capabilities.

This leads him to what he has called the “allocation economy.” In a 2024 essay, he proposed that people would increasingly be judged by how well they allocate intelligence and other resources to a task and ensure the work is done well, rather than only by what they know themselves.

Shipper suggests testing models on how well they perform an individual’s work from that individual’s perspective, alongside tests of agreed problems such as mathematics or programming. He argues that once a capability becomes widely available, a distinctive point of view can matter more to the output a person chooses to pursue.

His conclusion is that aggregate capability charts can compare models with people at a fixed point in time, while close observation lets people see how their own work may change as they use the tools. He closes by asking the conference audience to share what they learned from working with AI and with one another.