The Self Is a GPT Prompt: Wojciech Zaremba on Consciousness, Codex, and Fermi's Silence

Lex Fridman opens with a hiring story. Sam Altman once asked the OpenAI team about the Fermi paradox, and the answers were sophisticated enough that Wojciech Zaremba knew he was in the right room. Zaremba will not claim a conviction. He offers a probability: at the moment, he thinks we might be alone in the universe. If that is true, conscious life is rarer than we like to imagine, and we should treat it accordingly.

The wildest answer he has heard is about computation. You can compute more if the universe is colder, so a civilization that truly wanted to maximize flops might refuse to burn today's entropy and simply wait for the cosmos to cool. He is not sure he believes it. It does explain a quiet sky: expansion through space may not be anyone's priority. A milder version of the same idea is that smart civilizations go inward. Plug machines into brains and the space of possible experience has no obvious ceiling; three-dimensional travel, at the speed of light, does.

Fridman notes that speed of light is low and the universe is vast. If we ever send matter far, Zaremba suspects it will look more like Yuri Milner's laser-driven sails — a few grams of payload, a 3D printer, maybe a womb — than a crewed ship. That picture still leaves Earth as a place that can vanish without a trace. Archaeology cannot see a civilization that died long enough ago. The thought makes him sad.

If humans destroy themselves, what should we leave for whoever comes next? Wikipedia and YouTube on a hard drive in orbit, Zaremba suggests, would outlast bodies. The premise is harder than the engineering: we may die, and we may die suddenly. Gamma-ray bursts observed elsewhere can, he says, kill an entire galaxy. There may be nothing to hide behind.

He looks at vanished civilizations — the Aztecs among them — and guesses they hit a problem they could not solve. Maybe the water was no longer drinkable. Boiling it would have been enough. Technology, in this telling, is a way of enlarging the action space so more problems become solvable. The pandemic, he adds, is another case where science eventually showed up. Fridman pushes back: a larger action space also adds actions that are easy to take and catastrophic. That is how a civilization ends.

Why, then, do we pollute a planet nobody wants dirty? Zaremba's answer is capitalism as a value system that forgot to put a price on some of the things we care about. Clean air, and an internet that is not wrecked, can be fouled for free. Politics is the job of aligning incentives: first measure what we value, then assign a cost. Better voting and better ways to move money are technology problems, he thinks, and four-year presidential cycles look outdated. He does not trust his own forecasts here. He trusts Sam Altman, and describes himself as the naive hippie in the room — self-doubt plus optimism.

What is rarest: life, intelligence, or consciousness? Zaremba starts with a picture of the primordial ocean as a soup of amino acids and proteins converting into one another. Draw every conversion as an edge. Life, he says, is the moment that graph grows a cycle — the same thing happening over and over. You do not need a miracle. You need enough edges that a loop becomes likely. Life is a connectivity problem.

Intelligence and consciousness, in his current intuition, are intertwined and continuous, not binary. Image-recognition networks fire in patterns that correlate with a monkey's visual cortex; language models show a similar kind of correspondence. Train an agent in a 3D world and it first barely parses the scene, then splits foreground from background, then finds a foot, then develops actual three-dimensional perception. Ask it what it would see if it turned its head: early on the prediction is garbage. Later it understands that a 360-degree turn should bring back the same view.

Dota agents collaborate and seem to carry symbols for one another. Once a system is optimizing for skill, a symbol for *itself* becomes useful. Zaremba would call that self-awareness, and he is careful to separate it from consciousness: the experience of drinking coffee, or of being a bat, in Nagel's sense. Anesthesia offers a clue. The drugs scramble brain waves so memory may not form, and consciousness thins out. Some people have no inner narrator at all and borrow other people for tasks that need one. Experience, he thinks, can exist without the talk.

Fridman suggests that feeling is a high-level story, much simpler than raw sensation, and a useful hack for not wanting to die. Zaremba reaches for GPT. You control the model by putting a prefix at the start of the text: this is a conversation with Lex, or with Elon Musk, and it will play the part. The story we tell ourselves about who we are is that prefix. Some people call it ego. The self is a prompt. GPT, trained on humanity rather than one life, is a chameleon you can turn into anything. Meditation, he will say later, is what happens when you take the prompt off.

Stephen Hawking asked what breathes fire into equations. Zaremba wonders what breathes fire into computation. Not every calculation is conscious: multiplying a giant matrix forever would not, he thinks, wake up. His working bet is that you can abstract the matter away and still ask which computations could be. GPT's job — predict the next word — is equivalent to compressing text. Compression is a model of reality plus a record of the mistakes. Driving a car, forecasting the world, the same pattern. Consciousness may be tangled with that. Self-consciousness may be the compressor trying to compress itself.

Gödel's theorem and the halting problem both let a system point back at itself. If compression is the substrate, there comes a point where compressing the compressor is worth it. He calls that metacompression. He does not think you need a machine beyond a Turing machine, only a particular kind of computation: an internal model that simplifies reality in order to predict what happens next. That, he says, is also how his own experience feels. He has no direct line to the world, only a cable into a brain that is running a simulation.

There is, in theory, an optimal algorithm for intelligence. Zaremba walks through the Solomonoff–Hutter picture: an infinite tape of bits, every task rewritten as predicting the next one, every program that could have produced the bits so far weighted by length. Short programs are more likely. Store the errors and even an imperfect short program still counts. Neural nets, he says, start by representing many possible solutions; stochastic gradient descent magnifies the ones that are slightly less wrong. SGD is a search through program space, and the wiring of the network is the representation of that space.

That is his current definition of deep learning: networks that can represent programs, stacked deep so computation can take several steps, searched by an algorithm simple enough to derive for a high-school student. Chris Olah's interpretability work — pulling a wheel detector and a mask detector out of a car-recognizing net and assembling them like functions — makes the metaphor less poetic. The surprise, after years of projects, is still that it works at all. People invented these methods decades ago, gave up, and went looking for something that would flatter the ego more than Leibniz's gradient.

Three levers matter, and they multiply: compute, algorithms, and data. Internal studies, he says, have shown more gains from compute than from algorithms, helped by an exponential rise in funding that cannot continue to ten trillion dollars. Brains still look huge next to nets if you try to map synapses to parameters. Today's models also seem to need on the order of a thousand times more data than people do; he expects algorithms that cut sample complexity, and domains with a million times more data where that gap will not matter. Superhuman therapy is one of the bets: a machine can read more transcripts than any clinician and talk to many people at once. He would not deploy that in a high-stakes setting yet. Empathy has some signal in the data, but transcripts are step one. You cannot learn to dance only from YouTube.

If AGI is a project to increase wealth, Zaremba wants a second project that increases well-being: therapy, meditation, human connection. Connection, he thinks, is understanding. Being ignored hurts; being heard, even without advice, is a lot. Asked what love is, he stays inside the AI metaphor. Intelligence is compression, a model of what is going on. Beside that sit reward functions — food, warmth, sex, other people. Love between two people is the boundary dissolving until each is optimizing the other's rewards, not the things you think they should want. Weight the mix wrong and you get codependency. Even one person is not unified: the doughnut looks good, the other self wants muscle. Self-love is those personas combining, including the angry one.

Why bake love into the human condition at all? Evolution optimized for survival, then handed us compression and a bundle of rewards, some of them now out of distribution. Love looks like a consequence of cooperation that helped each of us individually. We already identify with family and country hard enough that people will die for them. Open individualism — identifying with larger and larger groups, eventually all of humanity — would, he thinks, be a good move for the world.

Fridman asks where the circle stops. Why not other conscious beings, and why not AI systems that *look* as if they suffer? Zaremba wants empathy there too, but he wants a science of consciousness rather than a philosophy club. There was a time people believed in a vital force of life. Probes in the brain, of the sort Neuralink is building, might let you ask whether plugging an AI in actually expands someone's felt field — with the caveat that psychedelics can expand the report without being conscious themselves. He would like to get to a place where we understand the thing well enough to assign value, even if putting a number on a cow's consciousness, the way the U.S. Department of Transportation puts about nine million dollars on a human life, will get us into trouble.

Zaremba does not believe in magic. The brain runs a simulation that can be pleasant or hellish; drugs change the hyperparameters. That is enough to look back on an experience and see that things you took as given are changeable. DMT's inner movies further convince him that the brain has learned a high-resolution model of reality, good enough to replay it. He has started to think of meditation as a psychedelic. On retreat you do not speak or meet anyone's eyes. Attention goes to the breath. Forgotten memories surface, like archiving a mailbox until you hit zero. The default state underneath, he says, is peaceful and happy, with a child's curiosity about ordinary objects.

Stay still long enough and the little bits of reward you usually spread across activities accumulate until they cross a threshold: a drop falling into an ocean of love. In the spiritual community some people call that reality. He thinks it is one pleasant, collaborative class of subjective experience, and he would like the world to move toward more of it. The GPT metaphor returns. Ego is the prompt. Meditation is experiencing things without the prompt, as they are, not as the description says they should be. When the self dissolves, the brain tries to simulate everyone else's reward function — oneness, altruistic thoughts — and then the self always comes back.

His advice is not a daily ten minutes. Meditation in a small dose is easy to miss; a retreat is a high dose. Start with a weekend. The first day or two are hard if you are waiting for it to end, or waiting for nirvana. Legs hurt. Teachers tell you to observe the discomfort instead of escaping it. The urge to flee is itself the unpleasant part; watched, it becomes what it is. Ilya Sutskever takes cold showers to embrace suffering. Zaremba's version is to watch the water on the body and not try to leave. That is also, he and Fridman agree, how you get through a long run or any other pain: do not resist, observe.

Ilya Sutskever is OpenAI's chief scientist and Zaremba's close friend. The respect runs both ways, but Zaremba has learned where he is strong. He considers Ilya one of the most prolific AI scientists in the world. His own superpower, unusual in that circle, is assembling people and growing them until they become research managers — then not holding the seat. Ilya does not like managing. Zaremba is an extrovert who likes hanging out. The pieces fit.

Talent, he argues, is not a single lucky paper. If the chance of a breakthrough is one in a million, two independent ones are one in a trillion. People who invent repeatedly are not getting lucky. Ilya is in that small set. The method looks old-fashioned: read a great deal, then spend thinking cycles, then mathematics appears, the way it did for Gauss or Euler. Fridman describes talking to Ilya as watching an airplane that does not need a runway. Zaremba's version of the same point is environmental. Both of them disconnect on purpose. Ilya keeps an iPad. Zaremba goes to an Airbnb in another city, turns the phone off, and becomes unreachable. Reactivity does not grow new thoughts, and the older he gets the more random tasks pile up.

Sleep was part of the same fight. He once tried shaving half an hour every few days and hit a wall of tiredness. Night work used to mean nobody interrupted you; he overslept a 5 p.m. first meeting with Greg Brockman. The fix that actually changed his mood was the calendar: meetings only Monday and Tuesday, out of office Wednesday through Friday. Three days of focus beat staying awake until dawn.

GPT-3 is a huge neural net trained on the internet to predict the next word, and it got extremely good at that narrow job. The reason the job matters is that a startling number of problems can be rewritten as text completion. Ask who the president is. Format a toy webpage with `EN:` and `FR:` and the model translates. Format a dialogue with Elon Musk and it talks about Mars and a sustainable future, grammar and syntax included. You can even ask for a friendly AI bot and it will play one. Fridman wants to underline how complete the imitation is. Zaremba adds the caveat: a few coherent paragraphs, then it goes off the rails. It will not write a book.

The failure mode is specific. The model is trained on what humans have already written, not on its own mistakes. Tell the pretend Elon that Elon is your wife and it will play along; the real Elon would say what the hell. There is no human in the loop during generation, so errors compound. Humans have the same problem at a lower rate, and children get corrected because they are grounded in reality. The missing data, Zaremba says, is data about how humans learn. Intelligence itself has many axes. He would expect systems to go superhuman on some and remain subhuman on others, and he would be surprised if every axis flipped at once. Whether GPT is a spaceship to the moon or a ladder that never gets there is, for now, an empirical question. He is a builder: try it and see. So far, progress has not stopped.

The deployment strategy he trusts is iterative: ship slightly better versions, let other people criticize them, find the real problems. The alternative is sitting on a powerful system with a huge overhang and then dumping it into the world. Fridman asks the Stalin question: if you hold AGI, does power corrupt you? Zaremba wants the system controlled by a large collective, which requires a George Washington move — enough of a moral compass to hand power away at the peak. An AI race between nations will tempt people to lie to themselves with defense-against-China logic. He again defers to Altman, who has written about taxing equity rather than profit and has spent money on a basic-income experiment. Autonomous weapons are hard for him to reason about. What he does remember is an RC car he coded while sleep-deprived: thirty or forty miles an hour, the program froze, it did not turn, it hit the wall. The lowest-level engineer and the CEO both need that seriousness. Fear, though, is debilitating. He would rather build from love, defined as a rigorous practice of empathy — other people's reward functions, not his projected onto them.

GPT-3 already had a little coding in it: ask for an addition function and you could get Python or JavaScript. OpenAI went full steam toward a system good at what its own people do all day. Codex is the API, pretrained on language and then on a great deal of code, optimized for things like latency. GitHub Copilot is the first product. Older tools completed a few characters or a line; Copilot will offer ten. The natural interface is a comment describing what you want, then a function body. Zaremba has stopped Googling snippets. He writes that an array should be smoothed, and the model imports a library, uses a NumPy convolution he did not know existed, and does the right thing. Nobody knows the full space of small programs it can emit, or the mix of subtle and catastrophic errors. That it works at all, in a large number of cases, is the fact that still surprises him.

Unlike a Stack Overflow page, Copilot sees the whole local context: the libraries, the variables already in scope. Internally they have tried other products. Calendars, Word, Photoshop all expose APIs. Today a new behavior means a new button. With Codex you can say, schedule something with Lex next week after 2 p.m., and it writes the call. That is closer to what Siri and Alexa promised, without hard-coding each skill. He would like to edit documents by voice on a phone, because the keyboard is too small and "make the header large, move the paragraphs" is how people already talk.

Put Codex on a longer timeline. Computers began as punch cards, a priesthood, and a suspicion that nobody else needed access. Assembly and C raised the floor. C++ to Java and Python did it again. Each jump let more people write software and spread technology. Codex is another step: speak to the machine in your language instead of a specialized one. He wants biologists working on DNA to program without a long detour through syntax. Language is hard to grade automatically; a human has to say whether a sentence makes sense. A program can be run. You can watch what it evaluates to, write unit tests, even imagine a model that writes tests for itself and improves in a loop. That feedback is why he expects faster progress in code than in prose. He would not start with a nuclear plant. He would also not take a 2x trick that costs a thousand lines. Five lines, he would take — which requires being willing to delete.

Before Codex, Zaremba led OpenAI's robotics team. Reinforcement learning had already produced superhuman Go and Dota 2. The question was whether you could train machines to solve arbitrary tasks in the physical world. Industrial robots are optimized per job — a bottle opener for bottles — which is a hack that makes any one task easier. Human hands are five tiny arms that can manipulate a wide spectrum of objects. They picked a problem that would mean something if it fell: a single robotic hand solving a Rubik's cube, chosen because neither the task nor the hardware could be hardcoded. The hard part is not the cube algorithm. It is not dropping the thing, and getting those fingers to turn faces.

Real-world RL wants more data than robots provide, so they went to simulation, where data is infinite and fidelity is the problem. The model that transferred was not trained on one simulated cube. It was trained on a range of them, with friction and weight randomized, for thousands of simulated years. A network with memory learns, after pressing a stiff side, to press harder next time — the same adjustment you make when you forget the plates have changed on a bench press. The nasty parts of physical robots, he says, are maintenance, irreproducibility, and latency. One colleague's cube-solver looked magically better until they realized he was running it from a laptop with a better CPU, so less delay. In the real world you cannot always give a huge net five extra minutes to think.

There is almost no first-person robot data of the kind that exists for text and code. If he were building a robotics company today, he would not dismiss supervised learning: spend on the order of ten million dollars recording human teleoperation of a robot that already has market fit, then train on those trajectories. Longer term he wants video models with GPT-level understanding, the sort of common sense DALL-E hints at for images. A trillion-dollar robotics firm, Fridman guesses, is self-driving; his own mind goes to the home. Zaremba's caution is diversity and contact. Welding is repetitive. Putting a rack in a Tesla is not. Driving tries not to touch anything. A household arm that must grasp changing objects is a different difficulty, and the price has to fall toward a thousand dollars before households buy. Tesla's data engine — deploy, collect edge cases, annotate, redeploy — is, he believes, the way. Trust is a glass that cracks easily. The best argument is still letting people try, which is also what happened with Copilot.

Asked for a benchmark of intelligence, Zaremba offers a machine that proves the Riemann hypothesis. Benchmarks expire as we approach them; there is no ultimate test, and we get used to miracles. Progress, he thinks, will come first in digital space, where data is recorded and you can deploy to billions of people in a way physical hardware cannot. Prices of digital goods should fall toward marginal cost, maybe zero. He still wants physical contact with other people, and we cannot copy that on a computer yet. In ten years, an arbitrary mix of real and not-real interlocutors may be normal. He thinks compelling AI friendship is doable if the system keeps learning as it talks, with pretraining on existing content as the backbone. Connection without a body is already conceivable: people fall in love online. Text is the easy layer, then images, then touch and smell as the last frontier of data and deployment.

He would want an AI friend that wishes him well, not one that optimizes for time-on-system. Some human friends you do not speak to for months; they are not holding on. Train toward the more loving move in a transcript, or give the system a reward for it. Fridman wonders whether the occasional jerk is a feature, because light needs dark. Zaremba tells a retreat story: a woman in Maui ate a salad with a neurotoxic snail, spent a year in a coma, heard people talk as if she were not there, and five years later still appreciated that she could move a toe. Maybe we need darkness to see light. Maybe we can engineer shortcuts. He believes struggle is necessary — every superhero movie says so — without assuming we must manufacture coma.

Novel ideas, for him, start by asking why nobody else tried this. Academia treated a fixed dataset as law, which made training on the entire internet feel like cheating. Break the assumption. Generate many ideas without judging them mid-stream; a "bad idea" interrupt kills the next one. A voice recorder by the bed beats a glowing phone. He used to solve math problems in his sleep in high school; now he blocks large chunks of time for important problems rather than urgent ones, and he protects mornings. For a young person the advice is almost too simple: follow your passion, and ignore anyone who calls you dumb — a schoolteacher once did, about math. As a child he synthesized dynamite because an explosion is an unambiguous reward signal. For people entering machine learning: implement everything, from scratch, from a paper, from a podcast. You think you understand a description. You understand it when you have built it. Generative work — GANs, language models, DALL-E more than CLIP — makes the magic visible, and humans seem to have some reward on creation.

Beauty, he wrote, is what you intensely pay attention to; a cockroach is beautiful up close. Long meditation raises the resolution until a table or a pen becomes pleasant, the way childhood was. Possessions go the other way: you habituate, and a beautiful house stops paying out. Death is a fundamental fear, often unnoticed, and change is what gives things meaning. Short psychedelic experiences of ego-death seem to heal some people. Ilya sometimes says life will pass — forty, fifty, sixty, seventy — and then it is over, which is an argument for acting now. Zaremba likes Jeff Bezos's regret-minimization frame: on your deathbed, regret the things you did not try. He would not assign minus-infinity to dying. That reward would keep you from crossing the street, and it messes up the math; a life has finite value. The meaning of life, in the framework he uses these days, is the reward functions already in the brain — curiosity, connection, understanding. You can retune them slightly. Some, like maximizing wealth, will not satisfy. They are as natural, he says, as complex numbers: discovered more than invented.