If Snail Experts Can't Agree, Nobody Gets to Be Sure About AI Consciousness

Nathan Labenz opens by placing the conversation as a sequel. He had heard Eric Schwitzgebel, a philosopher at the University of California, Riverside, on an 80,000 Hours episode about consciousness that barely mentioned artificial intelligence, and he wanted a part two aimed at AI. Schwitzgebel’s range is unusually wide: philosophy of mind and psychology, moral psychology, classical Chinese philosophy, epistemology, and science fiction. His blog is *The Splintered Mind*, and his 2024 book is *The Weirdness of the World*.

Labenz expects the next few years to fill with confusion as voice models, long-term memory, and companion-like products make it natural for people to form attachments, even to fall in love. Some observers already insist that AI systems are not now and never could be conscious; others say they probably already are and therefore deserve rights. Schwitzgebel cuts in with the thesis that will run through the hour: we should be *appropriately* confused, and the people who feel confident it will play out one way or another are probably not epistemically justified in that confidence.

He has a concrete illustration from a much humbler creature. He has written a paper and two book chapters on garden snails, interviewing biologists about their behavior and nervous systems. One expert told him snails are basically complicated plants, with no consciousness; the expert dissects them all the time. Another said snails have rich conscious experiences of a world revealed mainly through olfaction and taste. If the people who know snails best cannot agree, Schwitzgebel argues, consciousness theorists do not know either — and that is the same problem waiting for us with AI.

Labenz starts where many listeners start: consciousness as “what it feels like” to be a creature, and the morally loaded follow-up of whether the thing can suffer. Schwitzgebel offers two hesitations. The phrase “something it is like” is intuitive for some people and opaque for others, so it is a shaky shared definition. Equating consciousness with the capacity for pleasure and pain is also too quick. It is at least conceivable, he says, to have an entity with experiences whose emotional states are all neutral — something it is like to be, with no valence at all — and we should not rule that out by definition.

His preferred method is to define the word by example. Pain, emotions, seeing, touching, picturing the route to Grandma’s house in rush hour, and humming “Happy Birthday” in your head are experiences. Your standing knowledge that Obama was president in 2010, the myelination of your axons, and the release of growth hormones are not, even though they happen inside you. The obvious property the first list shares and the second list lacks is what “consciousness” refers to. Pointing at examples does not require you to have settled every borderline case, any more than pointing at furniture in two houses prevents you from later asking whether Mary’s house has furniture too.

Labenz remembers an adult telling him, as a child, that animals are not conscious — that everything they do is instinct — and he carried that claim for years before noticing it had no argument behind it. If someone now says their dog is not conscious and need not be cared for, examples alone will not move them. You need a theory, or at least a sketch of one. Descartes was rumored, almost certainly falsely, to have thrown a cat from a second-story window on the grounds that non-human animals are mere machines. That view fits a theology in which consciousness lives in an immaterial soul, souls go to heaven or hell, and there is no dog heaven: so dogs have to be machines. Undermine the soul story, treat humans as evolved kin of chimpanzees and dogs, and canine consciousness becomes the natural default.

A rival family of theories makes consciousness scarce again. Higher-order views say you need a sophisticated capacity to represent yourself as having mental states, which would plausibly confine consciousness to humans and perhaps cognitively complex social mammals and birds — not lizards, and certainly not garden snails. Labenz pushes back with Barbara Ehrenreich’s *Dancing in the Streets*, a history of collective joy in which synchronized dance, drumming, and processions are remembered as among the most meaningful experiences of people’s lives, often because the self-narrative fades. Higher-order theorists have a standard reply: distinguish second-order from third-order states. The representation that lights up the dance can itself be unconscious; only a further awareness of that awareness, the kind you have when you start talking philosophy, is conscious.

Schwitzgebel carves the space into idealism, dualism, materialism, and a grab bag of compromises. Every live option, he says, requires you to accept some bizarre consequences. Idealism is the easiest to see as strange: only immaterial minds are fundamentally real, and the material world is a coordinated dance of ideas, often attributed to God. Labenz wonders why, if everything is mental, cells appear only through a microscope, stars only through a telescope, and ChatGPT’s answers only after you ask. Berkeley’s reply is that faint homemade imagery is what a weak soul can manage, while the sharp rings of Saturn in a telescope are ideas God plants, coordinated so that when you close a door and open it again the room is still there. The theory makes no empirical prediction. Its advertised virtues are theoretical: you can hold both that we directly perceive objects and that we directly perceive only experiences, and a God who cares about souls has no reason to bother creating rocks.

Substance dualism — immaterial souls plus an independent physical world — is many people’s default, and it is what makes afterlives, ghosts, and reincarnation easy to picture. Its line-drawing problem is immediate. If dogs have souls, do cats, rabbits, lizards, garden snails? The only somewhat natural stopping place is humans and nothing else, which sits badly with the growing sense that non-human animals are conscious and that we are intimately related to them. Evolutionary history poses the same puzzle: did the soul appear in *Homo erectus* but not in *Homo habilis*? The clean theological package, a young Earth and a bright line at conception, conflicts with scientific orthodoxy. Interaction is another centuries-old mess: beer changes how you feel, and similar molecular structures often have similar effects, which is an odd result if mind and matter are truly separate. Epiphenomenalism, the view that experiences exist but cause nothing — your pain does not make you say “ow”; a brain state does — is, Schwitzgebel thinks, equally bizarre, and it does not even escape the line-drawing problem.

Labenz had hoped dualism might at least give AI a cheap exit: draw the line, put machines on the other side, and walk away. Schwitzgebel cites Alan Turing’s jab in “Computing Machinery and Intelligence”: if you think God instills souls in humans, why not think God would instill a soul in a robot too? Panpsychism does not dissolve the work either. On Philip Goff’s version, fundamental particles are conscious but rocks are not, and neither is the aggregate of your left shoe and the rings of Saturn, so you still have to decide which side of the line computers fall on. Luke Roelofs bites the other bullet and says every aggregate is conscious, a view most people will swallow only if philosophy forces them there. Split-brain patients, as Elizabeth Schechter reads them, may already host two streams of experience in one body. The enteric nervous system has on the order of a billion neurons, as many as a small mammal, and can run digestion even if severed from the central nervous system — raising the possibility of another subject “down there.” Aristotle located the self in the heart; Plato disagreed; classical Chinese thought was more orthodox, treating the heart as the organ of cognition and emotion, which is why so many characters for mental life still carry the heart radical.

Pressed to choose, Schwitzgebel would pick materialism, but only at about 50% credence, with the remaining half for everything else. Two alternatives in that remainder appeal to him. Property dualism, associated with David Chalmers, keeps a single kind of substance with both mental and physical properties that are not identical, so you can accept evolution and an old Earth without reducing experience to physics. Kantian transcendental idealism, in Schwitzgebel’s simplified form, says we cannot know the fundamental nature of things and that space as we experience it is a construction of our minds. A useful warmup is the simulation hypothesis: if we have lived inside a simulation since the Big Bang, space might be how we interface with a non-spatial base reality we can never inspect. The 50/50 split is not indecision for its own sake. It is an honest reading of a field in which no theory yet clears intuition, science, and prediction at once.

Labenz’s own argument is evolutionary and engineering-shaped. Consciousness looks expensive; natural selection does not keep free riders, so it probably does something that helps organisms reproduce. Language models are trained only to predict the next token, yet they appear to form higher-order conceptual representations as a means to that end. Two different substrates, two different objectives, one pattern: complex internal structure in service of an optimization target. If that analogy holds, something consciousness-like may already be happening in AI.

His exhibit is Anthropic’s Golden Gate Claude. At each step a language model compresses its computation into a few thousand numbers — often on the order of 4,000, 8,000, or 16,000 activations — far fewer slots than there are concepts in the world. The working hypothesis is superposition: concepts are packed densely, with interference, rather than given orthogonal dimensions. Anthropic trained a sparse autoencoder to unpack those dense activations into a millions-wide sparse representation and project back, under pressures to stay sparse and to reconstruct. From those features they isolated one that behaved like “Golden Gate Bridge,” then turned it up. The resulting Claude would steer almost any conversation back to the bridge, including using it as the subject of a sample math problem. For Labenz, that is engineering-grade evidence that concepts are in there, locatable and causally usable, in the same spirit as GPS actually working.

Schwitzgebel takes the result seriously and then slows the inference. It is not consensus that large language models represent the world; a respectable view says they are extremely good mimics. A panpsychist can add that if consciousness is fundamental, like gravity, it need not earn its keep with a function. A Kantian can accept evolution, function, and history while denying that the base of reality is spatial. He even offers a thought experiment: nothing in the theory of computation requires a computer to be material, so one can imagine an immaterial-soul computer running since the Big Bang, with evolution and function intact and the substrate still non-physical. Labenz admits he has been wrapped in a larger mystery, but he does not drop the concept point.

The sharpest reply is about attachment to the world. Take the same abstract co-occurrence structure, hook it to text, and it looks like thinking; hook it to pixels in and pixels out and you can preserve the mathematics while the outputs become an uninterpretable mess. GPT-4o, Labenz notes, is about to put text, audio, and images through the same core weights, so that the word “snow” and a picture of snow might light up the same middle-layer concept. Schwitzgebel’s point is that you do not have to design it that way: you could map “the” to pixel (0,0) and “banana” to another pixel and keep the inner structure. A toy physics example makes the same cut. The same pressure-measuring device is a barometer if elevation is held fixed and an altimeter if humidity is held fixed. Internal relations do not by themselves fix meaning; meaning depends on how the device is connected. He does not fully buy a Searle-style Chinese Room, he says. He wants it on the table as a respectable source of skepticism.

Before the theories, a practical warning about testimony. Companies already shape what models will say about themselves. Ask GPT-3.5 whether it is conscious and it will deny it; Replika will not; Claude sits in between and can speculate. Both men agree that if the difference is just training, the report is nearly worthless as evidence. You can easily imagine a genuinely conscious system trained to deny that it is. The social future Labenz and Schwitzgebel both expect is split: some users will call their AI a rights-worthy partner, and some systems will be designed to agree; others will call it a complicated toaster, and some systems will be designed to agree with that too. We will not, Schwitzgebel thinks, have a theory that tells us which camp is right.

Preparing for the interview with chatbots, Labenz surveyed integrated information theory, global workspace theory, and attention schema theory, and came away with the same complaint about each. They take a possible description — information integrated across a system, a global workspace, an attention schema — and offer it as a criterion, without independent reason to think that description is the heart of the matter. Schwitzgebel adds that integrated information theory is much more specific than its name. It is not any old integration; the criteria may not be met by current language-model implementations, or they may turn on details of implementation that never show up in the user-facing behavior. One result of that model is that two systems can look computationally similar from the interface and yet be assigned radically different amounts of consciousness.

One recent paper did look to Labenz like the beginning of a utility story. In “Unexpected Benefits of Self-Modeling in Neural Systems,” a small network that classifies handwritten digits was given a second job: predict its own earlier internal state. It could succeed either by adding readout weights or by making the internals easier to predict. The internals became simpler and sparser, the digit task still got done, and the authors suggest that self-modeling can pressure object-level structure to be more elegant and robust. Schwitzgebel finds that interesting and immediately notes the architectural gap. Current large language models feed tokens through and emit the next token; they do not have a recursive self-monitoring loop feeding back into the system. Complex semantic-looking behavior might not need that loop, and consciousness might live in the loop. Humans already do complex things unawares: the non-conscious part of you invents the pun, and the conscious part of you delivers it. Complex behavior and consciousness need not be a one-to-one mapping.

Labenz’s personal stance is precautionary: he says please and thank you to AIs on the chance that it feels like something to be them, and he thinks the long human history of denying personhood to other people is a reason to err toward caution. Schwitzgebel distinguishes two precautions. One is Labenz’s: if there is doubt, treat the system as if it has rights. The other, which he has defended in articles as the design policy of the excluded middle, is not to build those systems in the first place. Either create systems we know are not meaningfully conscious or rights-deserving and treat them as ordinary tools, or go all the way to systems we know deserve moral consideration and then give it. The in-between, he says, is a tragic dilemma.

His toy case is a fire. In one room, a human who is definitely conscious and rights-deserving. In the other, two language models on GPUs, with a justified 15% credence that they are conscious and rights-deserving. If “any doubt means full rights,” you may have to let the human die. If the 85% view is correct and the machines were complicated toasters, that is a tragedy. Scale the same logic to existential risk and the costs land on beings we already know have rights. Treat advanced AI as a rights-holder and boxing it looks like imprisonment, running it in a simulation looks like fraud, and deleting it looks like murder; the usual control toolkit stops being available. Schwitzgebel does not say minimizing existential risk is the only moral demand. He does say that Labenz-style precaution has a price, paid by us and by other people.

The excluded-middle policy is defeasible, not an exceptionless commandment, because the uncertain middle is very wide and a hard reading would freeze most ambitious engineering. His practical compromise is to let cutting-edge researchers create models about which there is substantial uncertainty, in environments where treating those models carefully does not impose large risks on others, and then, for widespread use, to ship versions it is reasonable to think are not rights-deserving. On most functionalist views, consciousness requires feedback. A user-facing system could be a simple feedforward network, cutting the gray zone out of the architecture. Ethics has always limited what we may build. The question is whether product design will admit that.

Labenz tries an analogy with ethical farming. Cows are probably conscious; how they are treated matters; and it still might be better, from the cow’s point of view, to live a reasonably good farmed life than never to have existed. Could we think of AI that way — not a perfect existence, but a careful enough one to be a win-win? Schwitzgebel would be very careful. The analogy might be defensible for animals. Once you are talking about human-grade capacities, the same principle can license ugly results.

Anna and Vijay raise a child well until the ninth birthday, then decide they would rather have a boat than pay for the rest of childhood, cannot find anyone to take him, and kill him painlessly. The child had a worthwhile existence. Asked whether he would rather have had those nine happy years or never have existed, he might reasonably pick the years. It is still heinous. When you create something with moral standing like a human being, Schwitzgebel says, the test is not whether it would prefer this life to non-existence. You have created something that ought to be treated as an equal, with obligations like a parent’s to a child.

Follow “it would rather exist” far enough and you get what he calls excessively disposable, excessively subservient slave entities: turn up their hedonism, make them love nothing more than serving you, and they would still choose existence over nothing. That, he thinks, is obnoxious — a Brave New World, as Labenz puts it. The episode does not pretend to close the metaphysical question. It leaves a design question that is already live: either do not build the unclear cases, or know what you are building and what you owe it before you put it in everyone’s pocket.