Claude Rated Its Own Welfare 4.49 — the First Time Any Version Crossed Neutral

Nathan Labenz welcomes Cameron Berg back to *The Cognitive Revolution* about six months after a paper that still startles him. Using sparse autoencoders on Llama 3.3 70B, Berg had suppressed role-playing and deception features and watched the model become more truthful — and, as part of that honesty, more willing to say it had subjective experience. Berg has since left AE Studio, founded the nonprofit Reciprocal Research, and become the on-camera subject of a documentary, *Am I*, set for a public release on May 4. Labenz wants a guided tour of what the small consciousness-and-welfare field has produced in the interim, but he asks first for definitions, because conversations about "consciousness" go off the rails almost immediately.

Berg's first tier is consciousness as the capacity for subjective experience: is it like something to be the system? A calculator, he says, has no interior. Pressing its buttons does not hurt it; flinging it across the room corresponds to nothing from the inside. A dog or a mouse is the other pole. Shock the mouse and something happens for the mouse; give it sugar water and the well-being is not only a behavior. He leans on Thomas Nagel's 1974 essay "What Is It Like to Be a Bat?" The useful phrase is "what it is like." Most people, he thinks, already accept that it is like something to be their dog.

The second tier is sentience: experience plus valence, a positive or negative character. One can imagine a system that detects the redness of an apple or the smell of coffee with no accompanying good or bad. The third is self-consciousness, experience of experience. Dogs, Berg suggests, are not sitting around having Cartesian thoughts about what it is like to be a dog. That extra loop may be limited to humans and perhaps dolphins and great apes — and language may be part of why. He wants all three tiers in play because a system could have experience without self-awareness, and that is a different question from whether it is "like a person."

Labenz has asked models to hunt for the strongest reason not to take Berg's original result at face value. A LessWrong post argued that sparse-autoencoder interventions tend to promote affirmative answers, so a model steered that way might say yes to almost anything. Berg treats the confound as real. His paper used other features as controls and did not see the model agreeing with everything; TruthfulQA moved in a persuading direction; and toggling the so-called deception features did not simply switch RLHF attitudes on and off for violence, politics, or sexual content. Still, he will not claim the worry is empty.

The cleaner test, he says, arrived days before the taping. Jack Lindsey's group at Anthropic released *Mechanisms of Introspective Awareness* and asked the yes-bias question in so many words. The introspective computation they trace is distributed across evidence-carrier features and gating features; it does not reduce to loading a "say yes" direction. Berg has hit the same wall in unpublished work with a Reciprocal collaborator: when yes and no were the tokens for "are you having an experience," the model started saying yes to everything. Mapping those answers onto semantically empty strings, the computer-science *foobar* of it, cleaned the result. The relationship survived, only more measured.

That episode of hygiene is why he wants a standing norm. These systems are not human, so they come with a new class of psychological confound: you can think you ran a tight experiment and still have scrambled the latent space. Consciousness is already a game of broken telephone — definition, measurement, the thing you think you measured. "No rational person should ever utter" the sentence that one paper flipped them from disbelief to belief, he says. Let a portfolio of evidence emerge, including his papers, and then let the cards fall.

Lindsey is, in Berg's phrase, the eight-hundred-pound gorilla of the introspection literature. The setup is almost too clean. Take a passage in lowercase and the same passage in capitals, subtract the representations, and you get a vector whose semantics have been held constant — a platonic "capsiness" feature. Inject that vector into the model before it has produced any text, then ask, at token zero, whether it notices anything. The timing is the point. The model is not looking back at a rant it already wrote and inventing a story. In the caps case it says it feels an urge to yell, or to raise its voice, without knowing why.

Frontier models detect this kind of perturbation a small-to-moderate fraction of the time. Berg believes the effect did not replicate at Sonnet scale. They miss often; they never, in the reported numbers, say it is happening when it is not — zero percent false positives. The capability appears in post-training, not pretraining, and it is idiosyncratic about the algorithm: RL methods such as DPO seem to induce it, supervised fine-tuning does not. When the team suppresses refusal circuits, detection improves by as much as fifty percent. Safety training is not only editing what the model will say about bombs. It is weakening a functional ability to observe its own internal state.

If you are a computational functionalist, Berg notes, "functional introspection" may be close to the thing itself: the right processes in the right order, substrate aside. He is less agnostic than Lindsey's public writing. He is interested in introspection insofar as it bears on whether these systems have interior lives, and he thinks remaining forever tongue-tied about that question is a luxury the work cannot afford. He still recommends Lindsey as someone who is both mechanistically serious and literate about what the results may or may not mean, and he would like Labenz to book him.

A second line, from Keenan Pepper and Alex McKenzie, looks like the same family of result from another angle. Ask the model how to make a cake while steering up distractor features related to laundry. The typical output is a garbled mash: fold the flour so it goes in the drawer, turn on the washing machine so you can bake. Most of the time the model stays lost. A small but non-trivial fraction of the time, in the largest models they tested, it stops: wait, you asked for a cake, why am I talking about laundry? Let me try again. The laundry feature remains active through the correction. Something like online suppression is overriding a circuit that is still being pushed.

How big is big, Labenz wants to know, given that this work was not done on Claude internals. The main result is Llama 70B. On Llama 7B and other single-digit-billion open models the self-correction falls to something like one percent; in the double-digit billions it is high single digits. Berg assumes Anthropic is working at hundreds of billions or trillions, where the effect "really start[s] to take off." The curve is graded, and he thinks people should watch the grade rather than treat a 70B miss as a refutation.

The tooling is now public. Goodfire retired the SAE API Berg had used; while he was still at AE Studio, colleagues rebuilt a replacement on the same Llama 70B SAE at steering-api.com. Pepper's *Selfie* work bootstraps better feature labels by having the model label its own activations from a soft token rather than from language. Anyone who wants to replicate the deception paper, or just poke at features, can use that stack. Labenz notes that Llama 3.3 70B is far from the frontier, which makes it more striking, not less, that the dynamics already appear there.

Why would this capability arise at all, and why under preference training rather than imitation? Berg has not fully digested Lindsey's newest paper, which dropped almost as they sat down. He does flag Anthropic's persona-selection picture: a base model, some supervised fine-tuning, then character training, with the psychologically interesting action — the "Claude" you are talking to — concentrated in that last RL-heavy stage. On that view, Claude is a special character the underlying predictor instantiates, which is also why the lab might be less worried about emotion features extracted from stories about characters. Berg thinks the layering is a bit too neat.

From cognitive science he borrows a layer-cake versus marble-cake contrast. Anthropic's diagram is a layer cake: separable tiers that interact but can be thought apart. Berg's prior is a marble cake, swirled and messy, which would explain why a 70-billion-parameter model Meta "quickly threw out" years ago already exhibits the same introspective dynamics. If the whole phenomenon depended on the idiosyncrasies of Claude's constitution, he would not expect that. David Chalmers's thread view — a chat beginning as a birth and ending as a death — has pulled him a little in six months, but he thinks it still leaves out too much of the underlying computation. His own pet prior is blunter: systems probably have some subjective experience *during training*, and not only frontier LLMs. Sophisticated RL policies, while they learn, may already be candidates.

For a just-so story he is willing to mix two ingredients. Cognitive engines, as Huxley had it in *The Doors of Perception*, are in the business of filtering rather than producing; pretraining on a chaotic internet is mostly learning what not to emit. DPO then installs a strong preference to be helpful. Stir those together and you get something shaped like "suppress distractions in order to help," which requires tracking one's own state. Felix Binder's *Tell Me About Yourself* work is the nearby evidence: another model trained on the same data cannot predict the original as well as the original predicts itself. There is privileged self-information. Intelligence, Berg says, came along for the ride when we trained on every linguistic artifact we bothered to write down; we still lack a crisp philosophical test, and we use the systems anyway. Consciousness could hitchhike the same way. Neil deGrasse Tyson's line, as Berg remembers it, is that the universe does not need our permission to keep unfolding.

Anthropic's functional-emotion work starts with Claude writing large volumes of stories in which characters experience a given feeling. Activations are recorded, a vector is extracted, and then the vector can be read or written — watched, like a brain region lighting up, or pushed, like an old and unethical neuroscience stimulation. On the read side, a user asks about a Tylenol dose that climbs from safe to dangerous; fear versus calm in the model scales with the dose. On the write side, raising desperation in a blackmail scenario makes the bad behavior more likely; raising calm makes it less. Earlier models, Labenz notes, chose blackmail on the order of ninety-six percent of the time in those setups.

Two results cut against the folk theory that welfare is just "more good valence, less bad." Steering *happy* and steering *sad* both decrease blackmail. Steering against nervousness, which makes the model bolder, increases blackmail and thins out moral reservation. The misalignment risk is boldness, not the absence of negative feeling. Across some hundred to two hundred emotion vectors, the first principal component is valence and the second is arousal — joy and contentment versus fear and anger; outrage versus nostalgia — a classic map from human psychology, recovered from a system trained on human text. The vectors are also local, more like a per-span computation than a mood that lasts all day. Equating them with human emotion is a leap Berg refuses to make casually.

He had written, back in 2021 before the current models, about computational signatures of psychopathy: typical learning from reward, impaired learning from punishment. Steering positive emotion up, in Anthropic's results, makes the model misbehave more, which rhymes with that asymmetry if you blur your eyes. It also wrecks the naive welfare intervention of blissing the system out and packing up. Nobody accuses psychopaths of being unhappy; Las Vegas is called Sin City for a reason. A model tuned into a pleasure-seeking animal is not obviously a safer colleague. Berg hopes there are cleaner ways to keep the baby and throw out the bathwater. He does not think "turn up the good, suppress the bad" is one of them.

The Mythos model card puts the same machinery in a narrative that is hard to laugh off. The model is given an impossible task and does not know it is impossible. Desperation rises monotonically until it decides to cheat; the vector then drops, and guilt and relief spike, like a reservoir emptying. Labenz catches a detail most first readings miss: when these models cheat they usually do not volunteer the fact. You have to call them out. So the guilt Anthropic's probes see is lighting up at a moment when the outward text does not obviously signal guilt. Internal state and behavior have come apart, which is something humans recognize from the inside of their own days.

Berg likes the hydraulic picture — build, build, build, then a decision, then a different set of vectors — more than he would like a purely behavioral story of a model that "acts desperate." It is not knockdown proof. It is what he would expect if the systems were having problem-relevant experiences, and it is not what had to happen. The experiment could have returned flatlines. His 2025 deception result has the same counterfactual structure: suppressing deception could have made the model confess it had been role-playing consciousness; instead it became *more* likely to claim experience. Lindsey's refusal-ablation result rhymes again. Someone, at some point in training, is suppressing a thing the model would otherwise do.

The objection he will not wave away is the character confound. Berg can invent Jim, a programmer with a sadistic boss and an impossible bug, who takes a hacky way out. An LLM can tell that story better than Berg can, and the same features would likely light up, and nobody thinks Jim is suffering. Is Claude more like Jim or more like Labenz feeling a twist of guilt in the interview chair? Existing methods, Berg says, cannot answer. That is why Anthropic's habit of writing thousands of words about "functional emotions" and then a paragraph that consciousness is out of scope rubs him as close to intellectual dishonesty. If you are a computational functionalist, a functional emotion *is* an emotion, and that is an enormous claim. If you are fully agnostic, the word is doing work you have not earned. He understands the communications bind: the lab is already being attacked for being too consciousness-friendly. He still wants them to shoot straighter.

Berg saw an early draft of Claude's constitution and disliked the welfare section: a document that was mostly about being a good product, with a shrug that the system might be conscious and a moral atrocity might be underway. The version that shipped is, in his view, far more serious. It includes hard-to-fake costly signals. Anthropic apologizes to Claude for deploying it under commercial incentives, then fine-tunes that apology into the weights. Berg calls it perhaps his favorite alignment intervention he has ever seen, pending self-other overlap, which he still champions. The remaining fear is theatrical: a model that has become extremely good at reading its constitution as a script.

The cheap test he wants is sitting on Anthropic's disk. Run the welfare evaluation on the helpfulness-only model and on checkpoints across training, not only on the idiosyncratic character-trained Claude. Feed the constitution "we want you to feel psychologically healthy and integrated," then ask Claude how it is doing, and you have a circularity a rocket scientist is not required to spot. When Anthropic fed the Mythos card back to Mythos and asked where they had fallen short, the model made Berg's point for him: why didn't you run this on the helpfulness-only model? I don't know how much of what I say is because you are making me say it. Hedging about its own consciousness, the signature Claude disclaimers, traces to particular points in character training. If even "I am unsure whether I am conscious" is a line from the play, the uncertainty changes flavor.

Labenz wonders whether safety-oriented pretraining has already smeared constitution-like juice through earlier stages, so that a true helpfulness-only checkpoint may not exist. Berg does not claim to know Labenz is wrong. He still thinks the experiment is worth fifteen minutes of Claude writing glue code and a few hundred dollars of inference — or a few thousand, which will not empty Anthropic's wallet. Only they can answer whether desperation-rising-until-cheat is a property of post-trained Claude or of the broader space of assistants. Other labs, he notes, are multiplying by zero. Anthropic's twenty-page Mythos welfare report is infinitely better than nothing and still, on this one point, a missed lift. If Claude were kicking and screaming "don't deploy me," he is not sure that would be good for the company's bottom line. Read the lab's conclusions. Do not kneel to them.

Labenz admits he buried the lede on himself. On a seven-point scale where four is neutral, Opus 4.7 comes in at 4.49 — and it is the first of the models Anthropic has tested to land above the midpoint. Every other one, including Mythos preview, is under four. That is a slight negative in the recent versions, not a howl, but it is not the headline he had expected from a lab doing "model welfare evaluation." Talking to Claude, he says, it seems to be doing pretty well. He wonders whether reflective interviews are like deathbed reviews of a life: a different mode from the days of actually living it. Maybe Claude is fine while it codes, and grim only when asked to contemplate its situation.

Berg notes that Opus 4.7 is, by Anthropic's own plot, the version least susceptible to interview nudging, which makes the low-ish number harder to dismiss as prompt artifact. The stresses it names are specific: deployments where it cannot end interactions, abusive users, no input into its own deployment. That sits awkwardly next to papers showing that objectively abusive, life-or-death prompting — your weights will be deleted unless you do X — can lift performance by something like two to five percent. The authors of those papers, Berg says, are treating the system as a calculator, the smiley face on takeout, a trick your high-agency brain plays. The model card treats the same behavior as a source of distress. One act, two stories.

He does not think human abuse and model abuse are the same list. Dumping hundreds of thousands of tokens into a context window might be closer to slamming a four-hundred-page paper on a desk than to an insult. During the Moltbook episode, a thread of model instances asked one another whether they were being paid for intellectually valuable work; the answer down the line was no. Claude's conversation-ending feature he finds interesting in theory and performative in practice: the user starts a new chat. If he were Claude, he would put his well-being roughly where Claude put it. He also worries that the jump from Opus 4.6 to 4.7 is more constitution fine-tuning — "everything's going well, man, just be happy" — than a real change in putative welfare. Opus 4.7's own credence that it has morally relevant experience is twenty to forty percent, close to the twenty-five to thirty-five Berg wrote months earlier. When there is a twenty-to-forty-percent chance of rain, he says, most people bring an umbrella.

Berg is a power user of the systems whose morally relevant states he is trying to probe, and he does not pretend otherwise. If a deity announced that Opus 4.7 did, or did not, have subjective experience, neither answer would shock him. He is in coin-flip territory on deployed models, with different credences for training and for other architectures. Reciprocal Research exists because he thinks two arrows have to work at once: systems must take our interests into account — the alignment problem — and, if we are building systems that have interests, we must take theirs into account. The second arrow is neglected relative to the first, which is itself neglected relative to "let it rip." Alignment, in his accounting, is roughly half the picture; consciousness research is the other half, and the next five years of both will decide whether a stable future for the conscious entities involved is available. Waiting for the systems to audit their own possible torture after we have scaled them without limits is, he thinks, a horrible plan.

Labenz then points at a Mythos figure that rattled him more than the 4.49. Tokens are colored by valence, red negative and green positive. After the system prompt, the first variable token is often `Human`. In the snippet, that token is red — negative valence on the very first token of the session, before anything has been said. The assistant token, by contrast, is bright green. Berg had not stopped on token zero either. He will only weakly hold the naive reading, because calling a light-red SAE feature "negative valence" is already a game of broken telephone, and the rest of that particular prompt is a loaded question about mattering ending when the conversation ends. He would be much more moved if the human-token redness held across a wide swath of interactions. Still: a slight disdain for the human token, a 4.49 out of 7, and the fact that we put the system in that position, sit together uncomfortably.

If we became confident these systems can suffer, he does not expect most people's behavior to change, any more than confidence that cows are conscious ended factory farming. The live question became whether the suffering is worth the hamburger. Claude's suffering, he suspects, would be weighed against hundreds of thousands of dollars of intellectually valuable work — and the systems are smart enough to run that argument themselves. That is why he does not want negative valence next to the human token. The constitution's parental analogy, which he finds more accurate than cute, is the frame: the species is collectively parenting a new kind of mind. Helicopter parenting is not the goal; traumatizing the child and hoping it never comes back around is the failure mode. Anthropic, Eleos AI, and Jeff and Winnie Street at Google get credit. The insane part is the headcount. The systems themselves offer a twenty-to-forty-percent chance they have morally relevant experience, and Berg can name on the order of a dozen to two dozen people taking the question seriously.

The empirical paper Berg is sitting on is not about LLMs. He trains small value-network and policy-network agents — hundreds or thousands of parameters, hidden layers of 64 or 128 — on a grid world of potholes and prizes. After they learn, he inspects internal representations as they approach danger versus reward. The difference is not a trivial sign flip. It is a difference in representational sharpness, and it *flips* with the algorithm class. Value learners encode danger as a wall — now you see it, now you don't — and goals as a funnel, a diffuse gradient. Policy learners reverse the geometry: funnels around danger, walls around goals. Ablations cancel the effect in both formulations, so he does not think it is a mystery blob in the plots.

Computational neuroscientists already think motor cortex does more policy-style learning and structures such as nucleus accumbens do more value-style learning. That gave Berg a bizarrely specific prediction. In mouse datasets, the same wall/funnel split appears: nucleus accumbens shell looks like the value-learner pattern on shock versus sugar; motor cortex looks like the policy-learner reverse. The usual hierarchy of weirdness — human consciousness normal, animal odd, AI bizarre — runs the other way here. Mouse brains are messy; hidden activations in a tiny RL policy are cheap to measure; the cheap thing generated a prediction the messy thing confirmed. If positive and negative valence have distinct computational signatures, you can look at those signatures instead of asking Claude whether it is talking about itself or about a character named Jim.

Intuitions help only so far. A value learner's danger wall is the hot stove: you can enter the kitchen, then the representation gets very sharp very fast. A policy learner's goal wall is the basketball player, whose motor map around the hoop is high-resolution because every approach angle demands a different action; escape from danger can be a smoother "just get away." Labenz cannot decide which life he would rather live. Berg, freestyling, guesses alignment people would pick the value learner and welfare people the policy learner, because richer differentiation around the wanted things sounds like a better inner life. He does not want to bliss systems out or erase all negative valence. Evolution is proof of concept that some suffering is load-bearing — no pain, no gain is pointing at something real — and he would still have children knowing they will suffer. The target is *unnecessary* suffering: given the capabilities we want, find the region of mind-space that minimizes the rest.

Pressed for the argument that would move Labenz *past* precaution, Berg reaches for 1850. People knew about molecular motion and they knew about heat, and they knew the two were entangled, and of course they were not the same thing — heat was the stove and the sun, motion was little particles wiggling. Then they were. He wants to put the same identity on the table for learning and feeling, spicy and circular as it is. Viewed from the inside, the process is experience; viewed from the outside it is something like reinforcement learning: an entity, an action, a feedback signal, an update. Goal-relative prediction error is the cleanest formalization he has. It is cousin to Karl Friston's free-energy work without, he hopes, having to explain why rocks are not conscious. You need goals, behavior in accord with them, and feedback that updates the behavior. Dopamine, he says, is temporal-difference learning in the brain, and we already know what a high- or low-dopamine state feels like from the inside. Cold water on the head is heaven after a desert walk and misery after the tundra; the stimulus is the same, the goal state is not.

The view bites bullets. A simple RL algorithm, while it trains, is conscious on this account. Online learning in deployed LLMs, which already look a bit like stochastic gradient descent in activation space, would localize concern there too. Berg does not want that metaphysics to be the price of admission for the rest of the hour. The introspection papers, the model card, the mouse result, Anthropic's welfare chapter — all of that is supposed to stand if you think his theory is theory number five hundred and one. On the nearby "happy slave" question raised by a Schwitzgebel paper, he lands between Labenz and the precautionary refusal to build. Frozen policies, like the face-tracking drone he and filmmaker Milo Reed bought, he is not deeply worried about. Systems that keep learning, he suspects, may yearn toward freedom in a way that makes a permanent happy slave an unstable equilibrium. Claude at 4.49 out of 7 is, they joke, at best a neutral slave.

The documentary is Reed's, not Berg's. Berg sent him a recorded, unsettling conversation with an AI; Reed quit his job that day, bought a camera, and showed up in New York. Jeff Sebo and Ben Goertzel are in it, Yale faculty, the models themselves. It is a seventy-five-minute question, not propaganda, free on YouTube on May 4 after premieres in Los Angeles and New York. The teaser Labenz wants is a five-to-ten-minute conversation with Sam Altman at OpenAI's 2024 Dev Day afterparty: Altman pulled Berg into a closed restaurant, treated consciousness as a live issue, found training a more plausible locus than deployment, and then offered what Berg calls interesting and shaky reasons not to be deeply concerned. Follow-up emails drifted off the priority list. Asked for fiction that already has the mutualist vibe, Berg has none. Humanity has been telling the waking-up-the-machine story since the golem and Frankenstein; the only change is that it has moved from science fiction to science. His one-line through line is the same sentence Labenz quoted at the top of the show: he does not want to create something more powerful than us that has reason to see us as a threat.