A Blank Wall Is Not More Conscious Than a Person

After the ads, Fridman opens with the most absurd question he can find: are we living in a simulation, and what difference would it make? Aaronson answers that a perfect simulation is, by definition, indistinguishable from the real thing, so we would never get direct evidence of some larger universe running it. An imperfect one would be another matter. If we could hack reality the way a memory error lets a C program out of its playpen, the conversation would stop being metaphysical. At that point, he says, it would be hard to tell the difference from what people ordinarily call a world with miracles.

The neighboring question is better posed: is the world computable? That is the physical version of the Church–Turing thesis — given the right input data, could a Turing machine simulate any physical system to any desired precision? Aaronson thinks the indications are pretty strong that our world does satisfy it; if it does not, we have not yet discovered why not. Peter Shor has joked that general relativity and quantum mechanics remain ununified because that part of the universe was left unspecified, and an experiment near a black-hole singularity would throw an overflow error. Aaronson likes the joke. Occam’s razor, he says, still points the other way: the universe has held up for fourteen billion years, and not knowing a law is not a sign that probing the phenomenon will crash the cosmos.

Fridman asks about Eric Weinstein and Stephen Wolfram and their purported theories of everything. Aaronson will not grant either of them that title. They have ideas they hope could someday lead to a fundamental theory of the interactions of physics, which is a worthy pursuit and the actual program of quantum gravity and string theory. Even after such a theory arrived, every question of emergent behavior would still be sitting there. The word *everything*, he notes, is doing limited work: it is only everything for a specific definition of everything.

Fridman steers to consciousness, and Aaronson splits the problem. The infamous hard problem is why it ever feels like anything to experience anything in a material universe. He would settle for what he named the pretty hard problem: which physical systems are conscious, and to what degree. Is a fetus conscious, and in which trimester? Is a dog more conscious than a frog? If consciousness is a multidimensional vector, say in which respects each system has it. That is the account you would need before deciding when a machine, an alien, or a coma patient counts as a moral subject.

Giulio Tononi’s integrated information theory is a rare direct attempt at that question. You abstract a brain or a chip as components linked by a pattern of influence, then compute a number called phi. The formula has changed from paper to paper, but every version involves what computer scientists call graph expansion: a high-phi system cannot be partitioned into two weakly connected pieces. Aaronson’s first reaction was that the hair stood up on his neck. The authors list axioms, offer some verbal discussion, and then phi appears. They act as if there is a derivation. There is nothing remotely resembling one. They pull the figure out of a hat.

The lethal objection is not missing details. If larger phi means more consciousness, it is easy to build a system whose phi dwarfs a human brain while it does nothing we associate with intelligence — a giant, repetitive error-correcting code, or a uniform grid of XOR gates so regular it might as well be a blank wall. Aaronson wrote a blog post making that point. Tononi’s reply was, in Aaronson’s career, the most audacious bullet-biting he had ever seen. He disputed none of it. The error-correcting chip would simply be super-conscious, because science is about challenging intuition, not confirming it.

Aaronson’s answer is about the order of operations. Science does learn non-intuitive things, but it first tests a theory on cases where we already know the answer. A new theory of temperature has to say that boiling water is hotter than ice before anyone trusts it that the sun is hotter than anything you have felt. IIT gets the blank-wall question wrong. A definition is only a good definition if it matches what we thought we were talking about prior to having one, and nobody meant XOR arrays when they said consciousness. Whatever the theory is measuring at that point, he will use a different word.

Asked whether consciousness is computation, Aaronson calls it a vertiginous question. He feels that his own consciousness is not reducible to computation, then immediately asks why anyone should believe him — and why he should believe Fridman if the claim ran the other way. Neuroscience, machine learning, and fundamental physics have all traveled an enormous distance along the road to understanding consciousness. From another point of view, the distance still to go looks no shorter than it was at the beginning. The hard part is specifying what would even count as progress, because someone can always say you have described the mechanism without saying what breathes fire into it.

Fridman offers a definition: an emulation of consciousness is consciousness if the system contains a black box of mystery relative to human inspectors. Aaronson rejects the move, because it makes consciousness relative to the observer. If the word means anything, it is something experienced by the entity that is conscious. He does not need Fridman to tell him he is conscious, and Fridman does not need him. In an essay called *The Ghost in the Quantum Turing Machine*, written seven years earlier, he explored a related idea that keeps the property inside the system: chaos plus quantum uncertainty might make some aspects of a brain unpredictable even with arbitrarily advanced future technology. That would distinguish a brain from any currently existing computer, though not from every possible one.

Fridman asks the personal version. If someone wrote down a prediction of Aaronson’s entire life, sealed it, and later showed it was right, would that destroy his sense of being conscious? Aaronson’s answer is precise. It would not destroy his sense of consciousness. It really would destroy his sense of having free will, much more than any philosophical conversation could. Whether such a prediction is even consistent with physics depends on how deep you have to go. If neurons are transistors, nanorobots could scan them. If you need the molecular or atomic level, you hit the no-cloning theorem. The harder case is a replica no one else can tell apart from you. You alone say your consciousness did not leap over. Why should anyone else believe you?

Aaronson read Roger Penrose as a teenager, and five or six years before this conversation he talked the theory over with him at a conference in Minnesota. He admires the personality and the audacity of the questions. The first clarification, though, is that Penrose is not merely saying quantum mechanics is relevant to consciousness. That would be tame. Even if the brain were a quantum computer, a quantum computer can be simulated by an ordinary computer given exponentially more time, and that is still a computer. Penrose wants the brain to exploit as-yet-unknown laws of quantum gravity that are uncomputable — not just uncomputable, but beyond an oracle for the halting problem, beyond the usual hierarchy of computability.

If you keep count, the speculations stack at least five deep. First there has to be a quantum-gravity theory of that uncomputable kind; most people in the field, Aaronson says, take AdS/CFT as evidence that nature is computable. Second, effects that show up at black-hole singularities and the Big Bang have to reach a warm brain running ordinary chemistry. Sean Carroll has made the case in detail that quantum field theory screens those effects off, which is why the Standard Model is adequate for terrestrial phenomena. Third, quantum mechanics itself has to be wrong. Penrose needs objective reduction: large enough states spontaneously collapse. Models such as GRW at least make testable predictions, which means the coming decades may simply refute them. Dirk Bouwmeester and others have been trying; they have not yet reached Penrose’s proposal.

Fourth, collapse cannot be purely random. Consciousness has to influence the direction, so that dispositions make the state more likely to go one way rather than another. All of that already would be the biggest development in physics in a century. The whole tower is then supposed to rest on an argument from Gödel’s incompleteness theorem. The overwhelming majority of computer scientists and mathematicians who have thought about it, Aaronson says, do not think the theorem can do what Penrose needs.

The argument is old. Any adequate, consistent formal system cannot prove its own consistency; that can only be done from a stronger system, which then cannot prove *its* consistency, and so on. Gödel himself apparently thought this had something to do with mind. J. R. Lucas thought so in the 1960s. Penrose is updating the line. When Alan Turing wrote his 1950 paper on the Turing test, he already treated the idea as old, well known, and wrong, and he wanted to dispense with it.

Penrose wants to say that although a given formal system cannot prove its own consistency, we humans looking at it from the outside can just see that it is consistent. The rejoinder from the beginning has been: can we, really? A computer need not be trapped inside a single system. It can adopt the hypothesis that its system is consistent, see what follows from the stronger vantage point, and keep adding axioms. Gödel’s theorem has nothing to say against an AI that does that. All it says is that there is no absolute guarantee the new axioms are right. That is the point Penrose pounces on. Turing’s reply, seventy years earlier, still holds: we do not have an absolute guarantee when we add a new axiom either, we never have, and we plausibly never will.

Almost a decade earlier, journalists had written breathless articles about Eugene Goostman, a chatbot simulating a thirteen-year-old Ukrainian boy, as the first program to pass the Turing test. Aaronson was not a judge. A reporter called; he said he was a quantum-computing person, not an AI person; they told him he could try it on the web. He asked whether Mount Everest was bigger than a shoebox. The bot parried, because it did not know what he was talking about. Those questions are obvious only because humans convert sentences into representations of objects and then do common-sense reasoning. Goostman had none of that layer. Relative to ELIZA, the 1960s Lisp therapist that threw your words back at you in a few hundred lines and still got people to pour their hearts out, Aaronson saw no significant advance.

The last few months had made the discussion interesting. GPT-3, from OpenAI, does not pass the Turing test, and nobody serious claimed it did. It is equally clear, Aaronson says, that it is a major advance over ELIZA and over anything the world had seen. Ask it for a poem on topic X in the style of poet Y and it will have a passable go, in many cases better than he would have done. Ask it for a student essay and you will get something he is pretty sure would earn at least a B-minus in most high-school or even college classes. Scott Alexander’s line was that they had ground up the entire internet into a slurry. Aaronson had wondered for a while why nobody had just trained a chatbot by deep learning over the whole web. Now someone had, at a cost of some millions of dollars, and the results were very impressive. Anyone who denies that, he says, is not engaging with what is there.

The system is a deep net doing one job: predict the next word, not decide what ought to be true. If GPT-3 raises a deep philosophical question, that is it — are we doing anything other than predictive processing? The limits are unknown, because GPT-3 was basically GPT-2 with a larger network, more training time, and a bigger corpus, and it was very noticeably better. Aaronson guesses there have to be diminishing returns, that it cannot be that simple, and he wishes he had more to base the guess on. The ironic failures are the things computers already had us beat at a century ago: balancing parentheses, basic arithmetic, word problems about locations in space. It does not know when to stop predicting and call a different subroutine. His speculation is that human brains couple predictive processing to visual reasoning and exact symbolic work. GPT-3 shows how far a single mechanism can go, and where it still has to stop.

Asked for the most beautiful idea in theoretical computer science, Aaronson goes to universality without much hesitation. AND, OR, and NOT look like a start; your first guess is that more complicated things will need more complicated building blocks. That was his guess as an adolescent learning Apple BASIC and GW-BASIC, a revelation on the order of finding out where babies come from. He assumed that to write a Nintendo game he would need some more complicated form of programming. Universality says no. Throw in a few rules and you already have enough to express everything. NAND gates alone can express any Boolean function on any number of bits. You can build a universe out of NAND gates.

The simple instructions of BASIC are already enough, ignoring details like how much memory you can address, to express what any programming language can express. The proof is to write a C or Java interpreter in BASIC. The idea goes back to Turing in the 1930s, when he pared a computer down to read, write, move left, move right, halt, and change internal state, then showed the model could simulate the rest. A second meaning of universality is the universal Turing machine: one machine that simulates any other if you describe that other machine on its tape. Aaronson had imagined an infinite hierarchy of more and more powerful languages, then kicked himself. Gödel had the same conjecture in the 1930s, read Turing’s paper, and admitted he was completely wrong.

The flattening cuts both ways. You cannot invent a language that BASIC cannot express. It is also an opportunity. Once you know the rules, the sky is the limit: you have the same weapons as the world’s greatest programmer, and it is now a question of how you wield them. What remains are the harder questions — how hard a program is to write, and what time and memory it needs — which complexity theory still struggles with.

Complexity theory studies the inherent resources needed to solve computational problems. Adding two numbers is linear: double the length, double the time, even with pencil and paper. Schoolbook multiplication is quadratic, because each digit of the first number meets each digit of the second. People later found nearly linear multiplication algorithms. That is complexity as a working subject, not a slogan.

Factoring is a different story. Give someone a ten-digit number and a few obvious tricks help — even, ends in zero — but if the prime factors are all large, it is not clear how to start, short of searching an enormous space. Most of the encryption that protects the internet is based on the belief, not a theorem, that factoring is inherently hard for our computers. The fastest publicly known algorithms still take time that grows exponentially with the cube root of the size of the number. The cube root is where the cleverness lives. The exponential is still there.

A thousand-bit key can probably be broken with a few hundred million dollars of computer power, which is to say with the resources of the NSA or another intelligence agency. The Snowden documents look a lot like they are doing that, or something like it. If that is the best they can do, Aaronson finds it reassuring: then they have not found an efficient algorithm, and two-thousand-bit numbers would be beyond them. All the worries about quantum computing came in here. Shor’s algorithm is a shortcut around that wall.

P is the class of problems a conventional computer can solve with a deterministic algorithm whose steps grow like the input size raised to some fixed power. Sorting, basic arithmetic, whatever is going on in an email reader or Angry Birds: all of it sits in P. NP, which does not stand for “not polynomial,” is the class of problems where a yes-answer comes with a short witness you can check in polynomial time. Factoring’s decision version is the standard example. Finding three nontrivial divisors of a thousand-digit number might take millions of years on known algorithms. If someone hands you the three numbers, you divide and you are done. Every problem in P is in NP, because you can ignore the witness and solve it yourself.

The central mystery is whether every NP problem is in P. If you can easily check an answer, can you also easily find it? Asked to bet all his money, Aaronson puts it on P not equal to NP. If we were physicists, he likes to say, we would have declared that a law of nature and given ourselves a Nobel Prize, then another if we later turned out to be wrong. We call it a conjecture because we are mathematicians, or descended from them. Pressed for odds on his blog, he gave two or three percent that P equals NP. The calibration is the point. If there were fifty mysteries of that grade and he guessed on every one, he would not expect to be right fifty times. That is what better-than-ninety-eight-percent is supposed to mean.

Donald Knuth conjectures that P equals NP but that the algorithm is so inefficient it does not matter — n to the ten thousand, or n-squared with a leading constant of a googol. Aaronson is not sure whether Knuth has a reason or just likes how it sounds. Logically it is possible. Equality of that kind would ravage complexity theory and force a rebuild from the ground up, while changing almost nothing in practice. If the algorithm were actually usable, you could break the encryption people use, break Bitcoin, and mint as much as you wanted. Next you would have a theoretically optimal way to train any neural network. The question “is there a small net that generates all of Wikipedia?” would no longer be “can you find it?” It would be “does it exist?”

P versus NP is one of the seven Clay Millennium problems. Aaronson’s reason for calling it the biggest is that a fast algorithm for it would solve the other six. You would ask the machine whether there is a short, machine-checkable proof of the Riemann hypothesis, and if such a proof existed the machine would find it without brute force. A non-constructive proof that an algorithm exists is theoretically possible. Leonid Levin’s joke construction enumerates every algorithm and dovetails them, checking claimed solutions along the way, so that if P equals NP the right algorithm is eventually hit. It is utterly impractical. Aaronson does not think a non-constructive existence proof is likely.

In 2002 Aaronson built the Complexity Zoo, a catalogue that by this conversation held about five hundred classes. He thinks of them as theoretical computer science’s chemical elements: the basic objects, of which only finitely many are particularly interesting. PSPACE is everything you can do with a polynomial amount of memory, reusing it over an exponential amount of time. Generalized chess and go on boards of arbitrary size live there, as do counting versions of constraint-satisfaction problems, which sit in a class called #P between NP and PSPACE. EXP, exponential time, was already proved in the 1960s to be strictly larger than P. The techniques are cousins of the proof that the halting problem is unsolvable, and they break down when the comparison is across resources — time versus space, or P versus NP.

BPP is bounded-error probabilistic polynomial time, algorithms allowed a small chance of being wrong. BQP is what a quantum computer can do efficiently. The chain is P contained in BPP contained in BQP contained in PSPACE. The last inclusion has a clean proof from Feynman’s picture: a quantum computation is a branching tree of paths, each carrying a complex amplitude, and the probability of an outcome is the squared absolute value of the sum of amplitudes that lead there. With exponential time you could write the whole tree down, which already shows quantum computers cannot solve the halting problem or do anything uncomputable in Turing’s sense. To place BQP inside PSPACE you do not even store the state. You reuse the same memory to sum the exponentially many amplitudes one term at a time.

Aaronson’s personal favorite is BQP/qpoly, quantum polynomial advice, close enough to his heart that his Gmail address is a mangled version of the name and amazingly no one had taken it. Imagine a special initial quantum state, perhaps exponentially hard to prepare, perhaps left over from the Big Bang, that can help you compute but does not know your input in advance — only its length. The class he thinks is more fundamental than people in the field realize is SZK, statistical zero knowledge. One clean definition: you have two algorithms that sample distributions over n-bit strings, and you ask whether the distributions are close or far. Anything reducible to that is in SZK.

The original discovery came from the other direction. Zero-knowledge proofs, for which Shafi Goldwasser and Silvio Micali won the Turing Award, let you prove a statement true without revealing why; they now sit at the core of some cryptocurrencies. SZK is the problems that have such a protocol with no cryptographic assumptions. To convince someone two graphs are not isomorphic, an all-knowing wizard can ask you to pick one at random, permute it, and send it over, then tell you which graph you started from. Right a hundred times, and if the graphs were isomorphic the wizard would have been guessing with probability two to the minus a hundred. You are statistically convinced, and you have learned nothing about why. The class keeps showing up in Aaronson’s work and other people’s. He thinks it is one of the most basic objects in the zoo, recognized late.

The conversation was recorded in the middle of the pandemic. Aaronson says all of their lives have changed on a scale you would have to go back to World War II to match. What stunned him was the failure of institutions — the CDC, other bodies people thought were trustworthy, a lot of the media. It was staggering, breathtaking, something he would not have predicted. He had written on his blog about the 2011 film *Contagion*, which got an airborne virus originating in China, a global shutdown, a wait for a vaccine, and everyone staying home. The one thing the filmmakers could not imagine was that everyone from the government would be anything other than hyper-competent and dedicated to the public good, with conspiracy theorists as random people on the internet for those professionals to oppose.

In trying to envision the worst, the movie had a failure of imagination. It did not imagine that the conspiracy theorists, the incompetence, and the nut cases would have captured the institutions and be the ones actually running things. Aaronson has been strongly in favor of rushing vaccines and of human challenge trials that were not done: volunteers get the shot, then get exposed, to compress a timeline that used to take years. Each month a vaccine is closer is, in his accounting, trillions of dollars of output and hundreds of thousands of lives. Testing infrastructure and contact tracing could have moved independently of a vaccine. Neither happened. He calls it one of the biggest failures in the two-hundred-forty-year history of the United States, and he wants that said without decoration.

Fridman misses the World War II mentality in two directions: breaking rules to get the job done, and togetherness. If he were president, making the virus the enemy would seem elementary; instead division increased. Aaronson finds that heartbreaking. It has been said, he notes, that this is the first time in the country’s history with a president who does not even pretend to want to unite it. Lincoln fought a civil war and still said he wanted to. With the election two months away, Aaronson is willing to go on the record: he is terrified about a contested result, about violence, about whether there will be a clear path of succession. If none of that happens, he may look foolish. He is saying it anyway.

Fridman turns to cancel culture in the machine-learning community: aggressive attacks on people trying to have a nuanced conversation, a thought police that makes everyone chant that racism and sexism are bad — things everybody already believes — and then forbids anything more precise. To say Aaronson is opposed to shouting people down rather than engaging them would be a massive understatement. He has tried to put money where his mouth is, defending unpopular people and ideas on his blog and defending norms of open discourse, then getting called a racist and a sexist on social media. He would be happy, time permitting, to say ten thousand times that he hates racism, sexism, and homophobia. What he will not do is cede to one political faction the right to define those terms, then be told he must agree with a pile of other extremely contentious positions or else he is a misogynist.

People like him, he asks, do they not also get a say in what racism is and in the most effective way to combat it? The cancellation mentality is, by its own professed goal, spectacularly ineffective. Fridman brings up Steven Pinker, who keeps trying to be thoughtful about difficult topics and keeps getting slaughtered. Aaronson is amazed Pinker withstood a cancellation attempt a couple of months earlier, psychologically as well as professionally. Having gotten to know him a bit, he finds Pinker incredibly unperturbed. He admires it and envies it. His own impulse when attacked is to engage every anonymous person on Twitter and Reddit, sit down for an hour, and show he is not that bad. Sometimes it even works. Then there are the twenty thousand other ones.

A way out? He wishes he knew. In a certain way these problems may be harder than P versus NP. Part of it, he thinks, is that there is a lot of silent support for the open-discourse, reasonable-enlightenment side, and that support has to become less silent. Many people agree the cancellations are ridiculous and are afraid to say so, which is the standard witch-hunt dynamic, understood and exploited by the people running it. More people need to say they are against racism too, and that what is being done is ridiculous. The hard part is mental energy: even if you feel safe, it takes time to phrase things exactly and to answer everything. The more people speak up, from all persuasions, the easier it gets to move forward. Asked finally about love, Aaronson says he loves his kids, his wife, and his parents, and that if you want deep insight into the nature of love you are probably asking the wrong theoretical computer scientist.