We Can Name Dystopias on Cue. A Desirable Future Is Harder to Imagine

Spencer Greenberg opens with a pattern that Bostrom immediately recognizes: vivid dystopias are easy to picture, while utopias are scarce. Most people, Greenberg notes, could rattle off *Brave New World*, *1984*, and *The Handmaid’s Tale*, yet the average person would struggle to name a single utopia. Bostrom agrees, and then makes the problem worse. Historically attempted utopias, he says, are ones you would not actually want to live in. They feel too pat, too cloying—like a plastic toy.

Small improvements are easy. The current world minus childhood leukemia, Bostrom says, is simply better. Keep stacking supposed improvements, though, and attractiveness collapses, the way plastic surgery can move from removing a wart to “Michael Jackson territory.” Greenberg adds that people agree far more on what they do *not* want—less illness, less poverty, fewer childhood harms—than on what a good society should be. Instant pleasure for everyone, continued striving, or a thousand micro-societies: that is where the fighting starts.

The deeper snag, Bostrom argues, is not only disagreement with others. It is hard even as an individual to conceive a utopia that would be super attractive *to you*. Our goals and applied values are conditioned on our current location: the projects, anxieties, and to-do lists that make sense here and now. Imagine a radically different condition, such as technological maturity, and those anchors vanish. You feel at sea, as if you were contemplating somebody else’s life.

That is the force of the insect metaphor in *Deep Utopia*. An insect’s exoskeleton holds the squishy bits together and gives them shape; analogously, “our souls have a kind of exoskeleton of these instrumental necessities”—paying rent, brushing teeth, the whole stack of things humans have always had to do. If we one day reach a post-instrumental condition, the question is what happens to the soul: do we become “contented pleasure blobs,” or is there still something that gives structure to our psyches? Medieval peasants could be stunned by a land of feasting and rest. Many people today already have stuffed fridges and weekends, and they often meet the opposite misery: boredom, lethargy, and a lack of purpose from having it too easy. Greenberg’s friend visited a tribe whose afterlife, via a translator, was “just like normal life, except the cows give more milk.” Iterate that wish far enough, Bostrom says, and you arrive at what he calls a solved world—and a plastic world—not long after a successful machine-intelligence transition.

If production has soared, why have people not simply worked much less? Bostrom says we did take some of the gain as leisure: longer childhoods and education, weekends, shorter hours than the peasant, sick leave, parental leave, longer retirements. Most of the increase, though, went into consumption. Almost a century ago, John Maynard Keynes extrapolated then-recent productivity gains and predicted that a hundred years later people would be four to eight times richer and would scarcely need to work. We work maybe 25% less, Bostrom estimates. Overall, greed has triumphed over sloth.

A large slice of that extra spending is positional. You buy fancier clothes and cars to keep up with neighbors; two billionaires cannot both have the biggest yacht in the world. Greenberg’s example is exact: a 200-meter yacht is spoiled the moment someone builds 210 meters, and the first owner must add forty more. Economic growth cannot satisfy both preferences at once. A solved world, Bostrom stresses, does not assume appetites vanish. Some people may have unlimited desires. It does assume that wanting more will not mean you can get more by working, because if AIs do everything better, cheaper, and more efficiently, demand for human labor collapses toward something like full unemployment.

Does the path have to run through AI? In principle, Bostrom says, you could obtain most of the same affordances without advanced AI, by writing non-AI software for every intellectual task given unlimited time. Realistically, building one AI that can learn all those tasks is much easier than a special program for each. The conversation then assumes the *Superintelligence* nightmare does not happen: AI stays reasonably under human control. Even then, purpose is the first value under threat. If purpose means something worthwhile you must exert effort over time to achieve, shortcuts abound: just ask the system. Pride in being a breadwinner, contributing to society, getting fit, or redecorating the home could all go “on tap,” leaving a purposeless existence because nothing specifically needs *you*.

Greenberg has already tasted a sliver of that world. A friend still prefers talking through problems with him rather than with an AI, but some of her other friends already prefer the model—and she admitted the preference might not last another year. Bostrom treats that as a signal: if AI becomes the better companion, one way of being practically useful to other people disappears. Generalize it, and you feel the challenge *Deep Utopia* dives into rather than selling a brochure. He is ultimately optimistic that a wonderful post-instrumental existence is possible, but it would be significantly different from familiar human life, and it would require reconsidering some values fairly fundamentally.

Starting from an empty slate, he adds values one by one. First, pleasure—easy to sniff at as super-drug junkies, yet more serious than critics admit; many who are down on hedonism, he thinks, would change their minds if they actually sampled the pleasure on offer at technological maturity. Second, experience texture: pleasure attached to contemplating beauty, understanding truths, or admiring goodness, the way some heavens are imagined as gazing on perfect goodness and feeling love. Third, artificial purpose: golf is an arbitrary goal plus a ban on carrying the ball by hand; utopia would likewise bake limited means into the goal so that “tell the AI” does not end the game. Fourth, real purpose: if A wants B’s preferences satisfied, and B wants A to do a thing *personally*, A now has a genuine reason that no shortcut can honor. Greenberg’s example is a wedding speech: even a brilliant AI draft can disappoint the couple, because the obligation was your own thought. Traditions work the same way; robots performing the ritual would not count as continuation. Fifth, quiet or subtle values. You do not see stars in daylight because the sun is too bright. Pressing horrors and injustices currently drown out honoring ancestors, upholding traditions, and giving a life an aesthetically beautiful shape. If those urgent needs were taken care of, pupils would need to open, as at night, and lives might become more like artworks whose meaning depends on who did the thing, why, and by what means—not only on the artifact.

A great deal of remaining natural purpose, Bostrom thinks, would come from social and cultural entanglements: other people, traditions, communities. That immediately collides with a near-term fact. Social companion bots are arriving—already chatbots, soon more multimodal, with visual avatars and voice. If AI were frozen near today’s capability, would people spend more and more time with them rather than with humans, and would that be good or bad?

The first answer that pops into most minds is that it would be bad: better to spend time with real people. Bostrom suspects a generational filter. “Us old fogeys” insist on real people because that is how they did it; a generation raised with compelling companions might decide the elders have hang-ups. Forming an opinion is easy. Forming one that reaches past idiosyncratic upbringing to “bedrock of value,” he says, is “a really hard evaluation to do.”

Greenberg’s line is clearer: it matters a lot whether the other party is conscious—actually having internal experiences of pleasure and pain. Deep involvement with something that has none would sap meaning. Bostrom offers two replies. First, habit may simply uncouple daily life from the abstract question, the way philosophical eliminativists who deny consciousness still get on with their lives. Second, some digital companions may themselves be conscious, designed to be more optimal as social partners the way some humans are simply more charming. He is careful not to sell the future. He understands the “yak reaction” to optimized bots replacing human company. The method is to dwell in the discomfort and ask whether the repulsion is a reason or a prejudice.

Bostrom keeps putting the ugly caveats first. *Deep Utopia* brackets the practical difficulties between here and a solved world so that someone can finally ask “what’s then?” Most of our time, he says, should still go to not destroying ourselves beforehand. The difficulties come in three clusters. The alignment problem is getting AIs to do what we want and not what we do not—a technical problem for people good at math and computer science. The governance problem is the broader political challenge of ensuring that even aligned tools are not used for warfare, oppression, and other mischief, so that the preponderance of uses is positive. Alignment and governance bite each other: regulation and oversight may be needed so that alignment gets solved in time and actually implemented. The third is the ethics of digital minds: a future that goes well not only for humans but for animals and for digital minds that may be conscious or otherwise morally considerable. That is part philosophy and computer science, then empathy, then governance so that the answers get applied.

The book, he says, is not thesis-plus-argument. It is designed as an experience that puts the reader in a position to think seriously, with open-minded curiosity and benevolence. Somebody, if things go well, will have to make up their mind about what we actually want, and the answers may depend a lot on the attitude of entry. The “secret purpose” is a briefing document for whoever sits in that deliberation—people in an AI lab, a government, or a humanity-wide process—equipping them with concepts and with an attitude of “benevolent generosity and open-mindedness and playful contemplation.”

On global coordination, Bostrom returns to his idea of a singleton: a world coordinated at the highest level, most likely via the AI transition, either because one actor gains enough power to impose itself or because post-AI technology makes coordination easier. Asked what to push on *today*, he is deflating: little bits and bobs, but if there were an easy high-leverage move it would already have been done. His *Vulnerable World Hypothesis* paper introduced the freedom tag—a neck device recording everything you hear, with omnidirectional cameras, uploaded continuously to “freedom offices.” He named it that on purpose to emphasize its Orwellian character; he is not selling it. Still, he thinks rising transparency is plausible, eventually including what people think. Phone calls and social media have been recordable for decades; what is new is that current AI could do mass sentiment analysis on everyone, couple it to a social credit score, and have an automated system dig up a bad remark from decades ago. That equilibrium could be dystopian. It might also, scaled from a village or a kibbutz where people know one another, mean a world with less fraud, because a jerk’s track record would be obvious. His gut is with the punks who stick it to the apparatus. Reflecting on what would actually make the future go best, he is “more on the agnostic end.”

The same paper treats new technology as drawing balls from an urn: white mostly good, gray mixed, black ending life on Earth. With short AI timelines, if a black ball is coming, Bostrom’s best guess is something enabled by *intermediate* AI before superintelligence—a bioweapon-design tool, or social-dynamics applications such as surveillance and automated propaganda. AI is special among existential risks: if it goes well, it could protect against a whole host of others. What counts as an existential catastrophe is itself value-laden here. Humans in exactly their current form running around for millions of years would be “a little bit of a letdown”; a paperclip maximizer would fail to realize a lot of the potential for value. The concept has a descriptive component and a value component, and with AI the value component becomes especially prominent.

In a rapid-fire round, Greenberg asks whether ChatGPT-era models have overturned *Superintelligence*. In broad brushstrokes, Bostrom says the book has held up; what changed is granularity. Current systems are surprisingly anthropomorphic. Getting the best out of a large language model can mean a pep talk—“think step-by-step; this is really important; my job depends on you”—which would have seemed a ridiculous bit of anthropomorphizing ten or twenty years ago. Yet that is where we are.

On the simulation argument, first circulated around 2003, he still refuses a number: “Many have asked, but none have been answered.” His own probability has “crept up a little bit, not much.” For others, an increase may be reasonable. Higher-resolution virtual reality, immersive games, and shorter remaining steps to digital minds as sophisticated as humans leave fewer places to hop off the train before ancestor simulations become possible. He does not rearrange his calendar. He does report a change of posture. Against a Dawkins-style inventory in which we live on this planet, die, and rot, the simulation hypothesis opens nested simulations, a basement reality, designers, and afterlives of various kinds. Knowing very little about that space induces “epistemic humility” that can become “almost spiritual humility”—a sense of smallness about the ultimate things that shape destiny.

Asked for a metaethical label, Bostrom declines the isms. He tends to think in “multiple channels,” like superpositions that may or may not collapse into conviction. Recent notes titled *Base Camp for Mount Ethics* sketch one direction. He also likes a moral parliament: instead of doing whatever your most probable moral theory says, assign probabilities, send delegates in proportion, and let them bargain under ideal conditions. A less probable theory that cares intensely about one matter can get its way there in return for conceding elsewhere. The point is more metaphor than formal model: greater wisdom, and a lower propensity to fanaticism, than picking a favorite theory and running with it.

Nathan Labenz, introducing the cross-post, has often said that the scarcest resource is a positive vision for the future. Bostrom’s book does not hand over that vision; it sharpens the question and tries to install a better attitude in whoever will one day have to answer it. The conversation itself leaves a practical test. Try writing down an ideal society that is not merely the present minus some evil—no leukemia is too easy. List the tasks you would refuse to outsource even if an AI did them better, the wedding-speech class of entangled preferences. And on the next hard choice, let several moral theories send delegates instead of letting the loudest ism gavel the meeting to a close.