Living Inside the Brain's Story: Joscha Bach on Simulation, Self and Consciousness

The clip opens with Lex Fridman asking for definitions of dualism, idealism, materialism and functionalism, and Joscha Bach begins with history. He says the Western trajectory comes from some 2,000 years of indoctrination by a cult, the Catholic Church, which shaped many of our modes of interaction but also, in his view, scarred our rationality. Translated into modern terms, the Church's mythology treats the world as a multiplayer role-playing adventure.

Bach notes that Eastern philosophers would call the world of money and objects maya, something that only appears meaningful, and that the Catholics also introduced a notion of higher meaning, the sacred. He describes the Church as using a crude mythology to implement software on people's minds and get it synchronized so that they walk in lockstep.

On this reading, God technically is the self that spans multiple brains, as opposed to your self or his, which each exist mostly on one brain. Because brains can all download the same piece of software, and the mathematical constraints of information processing guarantee compatible structures from the same spec, a shared self can exist across minds. This is a God with a small g, a function implemented across many people.

Pressed to define his terms, Bach says dualism posits two substances, a mental one and a physical one, that follow different rules. The physical world is causally closed and mechanical in the widest sense, which means computational: information flows around and physics describes the laws of that flow. He sees the common intuition of a real physical room, a layer of physics, and then another real room where our souls live as a confused result of connecting materialism and idealism incorrectly.

Fridman offers hardware and software as an analogy, with materialism as hardware and idealism as software. Bach disagrees, saying most idealists have no notion of software yet, and that software is itself information processing. He points out that universal machines can all perform each other's computations, given unlimited memory, so translations between them always exist.

The puzzle, he says, is that the only thing real to us is the experiential world, where things have taste, color and phenomenal content, while the physical world consists of cold patterns. Idealists start from mind as primary, treating material patterns as part of a dream, while materialists start from matter. Bach thinks that, properly understood, they are not a dichotomy but two aspects of the same thing.

Bach's central claim is that we do not exist in the physical world but in the narrative the brain tells itself. Physical systems cannot experience anything, he says, but it is useful for an organism to know what it would be like to be a person, so the brain builds a simulacrum of such a person. It is a multimedia novel that the brain writes and updates continuously, and the person in it is the brain's best model of itself in relation to its environment.

He adds that what you and he see is not the real physical world but a virtual reality generated in the neocortex to explain patterns on the retina. Fridman asks how close this is to reality, referring to Donald Hoffman's view that our interface is very far from it. Bach answers with the Mandelbrot set, which comes from two lines of code, and asks us to imagine living inside the fractal without having found the generator function.

An observer there might see a spiral and build a model of it, then reach a singularity where the model breaks and a new spiral appears, leading to second-order laws. Stack thirty layers of such laws and you get something like our physics: reasonably predictive, but not cutting to the core of how the pattern is generated. Bach suggests that if you were embedded in such a fractal with enough computing power, you could enumerate automata until one produced your reality, and he mentions that Stephen Wolfram still dreams of finding such fundamental rules.

Fridman quotes Bach as saying that people who think a physical system can be conscious but a simulation cannot have it backward. Bach confirms: a physical system cannot be conscious, only a simulation can, and consciousness is a simulated property that simulates itself. He uses mind to mean the principles that produce the simulation, the software implemented by the brain, which creates both the universe we are in and the self on the other side of attention.

Why have a self at all? Bach says the mind is a modeling tool and that we are not monkeys but side effects of monkeys' regulation needs. An organism must regulate its relationship to a world that is largely made of other organisms, starting from priors like unconditional reflexes and building more detailed models. Those priors act as set points, and deviations from them create urges and feedback loops.

Consciousness, in his words, emerges over dimensions of disagreement with the universe, where things are not the way they should be. Each identification is a regulation target you commit to, a dimension you care about, and this is also what locks you in. If you let go of all of them you are completely free and can enter Nirvana, because there is nothing left to do.

On the hard problem, Bach uses characters in a George R. R. Martin novel as an analogy. They feel empowered on a hill above a city because their emotions are written into the story, which is an adequate model of what such a person would do next. Likewise, our brain writes a story in perceptual rather than verbal content, a model of what a person would feel if it existed, and you and he happen to be that virtual person.

When Fridman asks whether he exists in Bach's simulation, Bach says yes: both are puppets that enact a play in his mind, and he identifies with one because he controls it directly. Their interaction can couple into a feedback loop where they feel things together, a coupling he calls a software phenomenon rather than a purely physical one. Neither mind has full access to the other's state for the same reason it lacks full access to its own, since self-knowledge is a model too.

Fridman asks why the physical world appears so consistent and repeatable. Bach says that in any simulation, the phenomena that endure are those with stable dynamics held around attractors, while unstable ones get weeded out. A region of the universe able to sustain minds like ours would be tightly controlled, with useful symmetries and symmetry breaks. Asked whether mind or universe is more fundamental, he says the universe comes first, since that gives easier causal models and he sees no way to build an inverse causal order.

Asked what happens to his mind when he dies, Bach says his implementation ceases unless he is implemented in other people's minds. The good thing, he adds, is that he has no identity beyond the one he constructs. He compares this to the Dalai Lama, who is reborn not out of confusion but because he identifies as a form of government, governmental software that runs on a human being and is reinstantiated each generation.

Most of us share a small part of this, Bach says: children, ideas we spark and the societies we belong to carry something of us forward. Fridman suggests that Bach is a collection of ideas on an operating system. Bach agrees that identity is a software state, not a physical concept but a representation linking different objects on the same world line, and one you can get agency over if you care by choosing what to identify with.

Fridman objects that death seems to need a physical body. Bach replies that the physics we experience is not the real physics, because color and sound are representations produced by oscillators in the brain. They have octaves and harmonics for that reason, and the geometry we interact with is what survives as we aggregate far too many particles to count. In his closing picture, feeling and flavor come from how each feature relates to all other features in a giant relational graph, with meaning arising where we sit inside a feedback loop of caring.