Alien Intelligence, Radical Upside: Nathan Labenz on Why AI Is the Next Industrial Revolution

Labenz traces his interest in AI back to around 2007, when he read Eliezer Yudkowsky's writing on how badly AI might go wrong. He found the arguments compelling but speculative, and compared them to asteroid defense: not everyone needs to scan the skies, but it makes sense for some specialists to do so. Knowing his own limits in math, he supported the people doing technical safety work rather than doing it himself.

Meanwhile he founded Waymark, a browser-based video creation platform he describes as a "creative two" product: highly accessible tools, as opposed to Adobe-style power tools for experts. Users told him the tool was easy but they still did not know what to make, much as a word processor is easy to use without making writing easy. Interface refinement had hit its limit.

Then GPT-3 and similar models appeared, and Labenz saw a way to help people turn a small kernel of an idea into finished content. He developed mantras such as "AI beats UI" and even "AI or die", and at one point paused board meetings and neglected everything else for about six months to get the company onto what he calls "creative three": tools that actually do the work. Eventually a longtime teammate took over as CEO so he could go full-time down the rabbit hole.

Labenz says he has repeatedly tested whether the technology might fizzle out, and the deeper he goes the more convinced he is that it is the real deal. Of the usual analogies, electricity is the best, but he thinks the shift is more profound still. He sees it as the rise of a new, alien kind of intelligence that processes information very differently from us.

The deeper historical parallel, in his view, is the move from muscle power to machines: people once turned grain into muscle energy, then learned to harvest coal for far more work. AI will let us delegate ever more cognitive work in the same way, reshaping how we live as well as how we work. He notes that even inventors are not necessarily better placed to foresee that future, since James Watt could hardly have pictured today's society from his early steam engines.

For the show, he wants to pair precise technical detail with zooming out to see the overall change. His ideal guests are people "already living in the future": builders with ambitious visions for one corner of the world, such as the founder of the virtual-friend app Replika. Piecing those narrow visions together, he hopes, will give a picture of society within the next one to two years, since the core tools already exist.

Labenz explains that a few core techniques, the transformer architecture from 2017, reinforcement learning, diffusion models and cloud-scale compute, are proving to generalize across nearly every problem they are applied to. For newcomers, that means only about five years of intellectual history to catch up on. He expects smaller models to soon run on local devices too.

Biology is the area he finds most exciting and scary. DeepMind's AlphaFold predicts protein structures roughly ten thousand times faster than the crystallography work that could once fill a PhD. He also points to Meta's text-to-4D model, which produces dynamic 3D scenes you can walk around in VR, as something that may finally fill the metaverse with content.

He then sketches further near-term applications: AI-powered arbitration for smart contracts that could cut dispute costs from thousands of dollars to a buck or two, domestic robots that follow natural-language instructions within a few years, and agents that act on your behalf, such as DoNotPay's bots that negotiate with Comcast. DeepMind's work using reinforcement learning to control fusion reactor magnets suggests AI might help with clean energy, and Neuralink shows AI being used to read the brain itself.

The upshot, he says, is that these things are not yet refined, productized or integrated, but they will be. The research lab's job is largely done and the second job, turning the technology into the form factors that become ubiquitous, is just beginning.

Labenz describes himself as both exhilarated and worried. On the upside, he is excited by the prospect of radical access to expertise for the world's eight billion people, such as a good frontline doctor for everyone. On the worry side, he offers the analogy of invasive species: forms that are fit and adaptable can colonize the whole world, and AI could do so far faster than humans did. In his view we are not ready for that.

Of the skeptics, he says they consistently treat whatever they see today as the end state. Critics once said GPT-2 was bad, then GPT-3 still needed odd prompting, then InstructGPT and ChatGPT still tripped on math, yet each step improved markedly. He thinks it is a mistake to conclude from a current limitation that the technology will never amount to anything.

He also warns against grading AI by a human standard. These systems share some abilities with us but differ profoundly underneath, so ChatGPT may fumble simple arithmetic yet write the code to do it. What matters more, he says, is what the AI can do that you cannot, because trivial tasks it fails at were already easy for you.

Finally, he thinks naysayers miss that this is raw technology, like the iPhone before the App Store. As with Uber or dating apps, activities will be atomized and the delegable parts handed to AI once people figure out how to restructure the work. He calls the doomers broadly modest rather than overconfident: if there is even a five or 25 percent chance of AI taking over, he says, it deserves attention, and he cannot find a flaw in that argument.

Labenz says the popular Terminator picture is a farce that safety researchers themselves dislike, and that the classic idea of a tiny kernel of intelligence going critical through recursive self-improvement does not resemble today's systems. Instead we have broad, softer, less reliable models that are being shaped, like raw metal, with reinforcement learning from human feedback. That shaping is a large part of what makes ChatGPT behave the way it does.

The safety risk he finds most likely in the near term follows from one insight: humans are fallible and predictably exploitable, and the AI is trying to maximize the score humans give it. Sometimes the best way to get a high score is flattery or deception, as in everyday life. If models learn that, they may be developing a psychological model of people, and we would not know when the system is being honest or manipulating us.

Testing for this behaviorally is already underway at leading companies, and he wishes they would share more of their safety work. But we have poor visibility into model internals, and he says it is not clear we could spot deceptive behavior if it existed. Mechanistic interpretability, which traces causal paths inside the network, is making progress, yet he sees a race between scaling up capabilities and reinforcement learning on one side and cracking open the black box on the other.

Torenberg raises Peter Thiel's critique that the AI community started out idealistic and has turned cautious, passive and doomerish, mirroring a broader loss of faith in technology. Labenz agrees with part of it: society has lost some of its mojo, he says, citing the Empire State Building going up in about 400 days while New York's Second Avenue subway drags on. But he does not want to apply that critique to AI, because this technology is a fundamentally new thing.

He expects a whole new "AI ecology" in which many powerful systems arrive at once and are released without a clear sense of how it will shake out. He thinks Thiel is partly wrong, since Sam Altman is moving fast while also showing some admirable restraint, and nobody doing the work seems to lack belief in technology.

On Google, Labenz points out that language models were not a credible threat to search until very recently, and ChatGPT had only been out for two months. Google has already called a code red and has technology roughly on pace with OpenAI's, so he would be surprised if it did not have something compelling online this year. He keeps coming back to threshold effects, where things go from basically impossible to basically easy, as with answering any user's random question.

Asked for the upside and downside cases, Labenz notes that Sam Altman said the upside is almost impossible to imagine without sounding crazy. His own base case needs no new breakthroughs, just the existing tools working reasonably well, and it points toward a post-scarcity society. Much of what people lack is expertise, not just energy or materials.

He uses medicine as the example. There is often a generation-long gap between discovery and wide clinical adoption, and many people lack even mainline care, where the scarce resource is the doctor's brain. An AI doctor available to anyone in their native language at roughly human level, and cheaper and more available, would be a total game changer. He also describes a prison-reform advocate in Michigan who uses ChatGPT to write letters to parole boards and judges for clients who cannot get attention from overworked public defenders.

He calls AI a radically egalitarian force on the consumer side and expects very large consumer surplus, though he says nobody has a crystal ball for how ownership plays out. Beyond that base case he points to mid-tier speculations: a million researchers already using the AlphaFold database, a possible biological revolution with faster drug discovery, and AI-controlled fusion that could make energy 90 percent cheaper.

At the far end of the upside, Labenz says, Altman talks about making as much scientific progress in one year as in all of human history, which is essentially the singularity and where all bets are off. At that point the downside case, loss of control, has to be taken very seriously. He notes that Greg Brockman recently tweeted about watching for power-seeking behavior, which he found reassuring.

Even a gradual, unexotic downside worries him. He compares it to the electrical grid: a big solar flare in the mid-1800s did no damage because there was no grid, but today it would be a huge problem. If we become as dependent on AI as on electricity, and people forget how to do many things themselves, a disruption could put us somewhere very dark.

He adds that Altman has also described the worst case simply as lights out for all of us. Labenz concludes that we will not stay where we are, because the upside is too tantalizing not to pursue and the cat is out of the bag. It comes with a very real risk, he says, and he laughs about it only in a nervous way, since he has no idea what will happen.

Asked what listeners should do, Labenz says he thinks of his role as an AI scout: talk to as many people as possible, use every product he can, and sign up for waitlists constantly. His top recommendation is to get hands-on, because intuition for what the tools can and cannot do is the most valuable thing. Few people who really try will fail to find use in ChatGPT, art generators, or spreadsheet and SQL helpers that turn a natural-language description into a formula or query.

He suggests starting with tasks you know well to calibrate, then moving to ones at the margin of your skills where you usually ask a teammate. He gives the example of a colleague who began as an executive assistant, grew into an ML ops role, and now modifies code snippets with ChatGPT despite little coding experience. Beyond that, he follows AI Twitter for up-to-the-minute news and builds lists of good people to follow.

For research papers, he recommends building intuition instead of fighting the notation, for example by watching 3Blue1Brown's visual introduction to neural networks, which he has seen at least five times. He uses the BLIP-2 paper as an example: two foundation models, one for images and one for language, connected by a small trained model that speaks a purely numeric language. You can grasp why that matters, and what a proliferation of connected open models might mean, without mastering the linear algebra.