The Poker Bot That Never Tried to Read a Soul

A lot of people, Noam Brown tells Lex Fridman, used to say game theory was nonsense: if you really wanted to make money, you had to look into the other person's eyes, read their soul, and figure out what cards they held. Then Brown's team sat a bot across from four of the world's best heads-up no-limit Hold'em specialists. The machine was not trying to adapt to them, exploit them, or play mind games. It was trying to approximate a Nash equilibrium, and it crushed them.

Over about 120,000 hands it made close to two million dollars in chips. The money on the felt was fake, Brown is careful to say; the $200,000 in prize money for the humans was real, split by how they finished relative to each other so they had a reason to play their best. Fridman introduces Brown as a research scientist at FAIR, the Facebook AI Research group at Meta AI, and as co-creator of the first systems to reach superhuman no-limit Texas Hold'em — first heads-up, then multiplayer — and of a later agent that can out-negotiate people in the board game Diplomacy.

The three names mark three rungs of the same climb. Libratus took heads-up no-limit Hold'em. Pluribus took the six-player game. Cicero took Diplomacy, a seven-player war game of private deals that John F. Kennedy and Henry Kissinger are said to have loved. Brown named Libratus; it is Latin for balanced. He had started playing poker in high school, gripped by the idea that there might be an objectively correct way to play, and that if you found it you would make money without limit.

No-limit Texas Hold'em is the game you sit down to in a casino, and the game in *Casino Royale* and *Rounders*. The difference from limit Hold'em is one sentence of rules and a large change in feeling: you may bet any amount of chips in front of you. A pot that starts with a dollar or two can hold a thousand by the end of the hand. In limit, two dollars in the pot means a two-dollar bet; with ten thousand in front of you in no-limit, you are always welcome to put ten thousand in.

Fridman, who has spent time with Phil Hellmuth, asks whether no-limit rewards crazy more than calculated play. Brown says both variants reward strategy. What changes is how easy it is to get jumpy. You sit down thinking you will play for a hundred dollars, and suddenly there is a thousand in the pot. A lot of people cannot handle that. In poker you always want the action that maximizes expected value, the same logic as investing: if the amount at stake would materially change your life, you tilt toward risk aversion. Someone makes a huge bet, and the thought that arrives is that this is too much money to risk.

That is also the weapon. One of the key strategies in poker, Brown says, is to put the other person into an uncomfortable position. You can have fifty dollars in the pot, throw in a thousand-dollar bet, and if you time it right you drop them into a spot they were not ready to reason about. Do it wrong and you lose a fortune; no-limit makes it easy to go broke if you do not know what you are doing, and the upside is large if you do. The game does not simply reward madness. It rewards people who can choose, precisely, how often to look mad.

Any finite two-player zero-sum game has an optimal strategy. Play it, and you are guaranteed not to lose in expectation no matter what the opponent does. Brown calls that a radical idea, and he says it is true in chess, true in poker, true in the whole class. Rock-paper-scissors is the intuition: throw each option with equal probability and, in the long run, you do not lose. Poker has some vastly more complicated strategy with the same guarantee, at least for two players. Six-player poker, he adds, is a different story — a beautiful giant mess.

In expectation does not mean every hand. Poker is high variance; even a perfect strategy wins some pots and loses others. Play long enough and you are guaranteed at least to break even, and in practice you probably win. The theory ignores going broke and other real-world limits. Finite is not a huge constraint: most games people play are finite, and many infinite games still have this kind of perfect strategy if they are compact. A toy counterexample is the game where whoever names the bigger number wins. There is no Nash equilibrium. Fridman says 17. Brown says 18, and wins again.

Zero-sum is the sharper constraint. Two players, and whatever one wins the other loses: Brown takes fifty dollars, Fridman is down fifty. Nash equilibria still exist outside two-player zero-sum, but they mean less. In Risk, if everyone else on the board decides to team up and take you out, there is no perfect strategy that guarantees you win. There is just nothing you can do. That is why the poker bots climbed heads-up first. In a two-player game, strategy can be a moat. With more than two players, other people's coalitions are the ceiling.

Fridman wants to know whether you could fold into the model the other payoffs of being a professional: the pleasure of playing, the sponsorships, the audience that tunes in for a personality. A lot of pros, Brown says, make a huge amount of money not from the felt but from being someone people want to watch. From an AI perspective that changes the objective. Trash talk, chaos, even a deliberate losing line might be part of what you maximize. What Fridman is getting at, Brown says, is the difference between an AI that wins a game and an AI that is fun to play with — and, separately, fun to watch.

Game designers already live in that gap. In *Civilization*, Brown has heard, the AIs are not playing to win; they are playing to have personalities, to be engaging, because that makes a better game. Fridman has just talked to Todd Howard, the creator of *Fallout*, *The Elder Scrolls*, and *Starfield*, and the conversation slides to NPCs in an open-world RPG. Brown thinks this is one of the first places ordinary people will actually interact with large language models. *The Elder Scrolls VI* is probably too far along; he would not be surprised if *The Elder Scrolls VII* used language models for NPCs. Fridman jumps in: that is him speculating, not Brown leaking anything.

In graduate school Brown heard a gaming executive asked why so many games revolve around fighting and killing. The answer was that it is much harder to make an AI that can talk with you and cooperate with you than one that can fight you. Once every line of dialogue does not have to be scripted, Brown argues, you unlock games built around positive interaction instead of combat. Fridman offers a darker toy: skip the shooting in *Call of Duty*, argue with an AI in a corner, lose sleep, and come back to finish the fight. Brown likes that world. Games can be messier because everyone understands they are not the real world — the same permission, he notes, that Diplomacy has as a game rather than as geopolitics.

Back at the equilibrium, Brown's team finds it with self-play. The algorithm starts by playing completely at random. In poker it stumbles to the end of a hand, makes fifty dollars, then reviews every decision: what would have happened if it had raised instead of called? Because it is playing a copy of itself, it can do that counterfactual reasoning all the way down and discover that another line would have made 150. It updates a regret value — roughly, how much it regrets not having played that action in the past. The next time it sees the situation, it picks high-regret actions with higher probability. Done the right way, the process is proven to converge to a Nash equilibrium, even in imperfect-information games.

The method is counterfactual regret minimization, or CFR. It does not have to be self-play, but self-play is the natural way to run it, and it is a principled kind of reinforcement learning. Self-play is not tied to neural nets; it is just a system playing itself. In perfect-information games you have more flexibility: the algorithm does not have to be as theoretically sound to converge. Imperfect information is less forgiving. Brown also notes that the counterfactual step is how humans learn poker with friends. You ask, would you have called if I had raised there? Scale that up and you can learn an optimal policy.

Neural nets enter because of size. The version of no-limit Texas Hold'em they played, Brown says, has 10 to the 161 decision points — more than the number of atoms in the universe squared. The exact exponent depends on stack sizes; plus or minus ten does not change the point. You will almost never see the same situation twice. What nets buy you is similarity: a new spot can inherit regret from spots that look like it. In training, the bot's own candidate bet sizes were constrained. The human on the other side could bet whatever they wanted. The search module had to survive inputs outside the set it had practiced.

Which game is more complex, chess, Go, or poker? Brown knows it is a fight waiting to happen on Reddit, and he jokes that David Silver is going to get really angry. He still says poker. The first reason is imperfect information. In Texas Hold'em you get two cards face down that only you see; every opponent does too. As you play you have to reason about what they think you have, what they have, and what they think you think they have. Bluffing is possible only because they do not know your cards. That is the key difference from a perfect-information game like chess or Go.

The second reason is subtler. You cannot just pick actions; you have to pick the probabilities, and the value of an action depends on how often you take it. Throw rock every time and its value collapses. Throw it one percent of the time, while the other person is mostly throwing scissors, and rock becomes extremely valuable. Translate that to poker: if you never bluff, and then you bluff, there is a really good chance it works. If they have watched you bluff constantly, they call you down and you lose a lot of money. Finding how often to bluff is the key challenge of the game. In chess it does not matter whether you open with the Queen's Gambit ten percent of the time or a hundred. The expected value of the move does not change.

That is why Libratus is named for balance: the algorithm was designed to find how often to play each action, not only which actions are good. The tension is not just bluff or not. It is how often to bet, how big, and with which hands. That is where range comes from. When you bluff a particular hand in a particular spot, you do not want a pattern the other person can pick up, so you have to ask whether you would also bet a strong hand there. Successful bots do this with an explicit theory of mind: they reason about common-knowledge belief, what you think I have and what I think you think I have. They do not try to fingerprint the opponent over a session. They try to pick a strategy that would still be unbeatable if you played it for 10,000 hands and the other player learned it exactly — then, if they spot a leak, they deviate a little.

Fridman asks who the greatest poker player of all time is, and why it is Phil Hellmuth — a man known, at least in part, for playing suboptimally, chaotically, and still winning. Nash equilibrium, Brown says first, does not mean predictable. The whole point is to be unpredictable. Hellmuth's success, in Brown's telling, is not that chaos. It is the ability to take advantage of the other player, or to guide them into thinking you have certain weaknesses, then to understand how they will deviate from equilibrium to attack those weaknesses, and to counter-exploit the deviation. That is a two-layer mind game, not a random one.

The poker world and the academic world spent years arguing about GTO — game-theory optimal — versus exploitative play. Until about 2017, when the Libratus match happened, exploitative play had the advantage. A lot of people said game theory was paper, and that real money came from reading souls. Then players started adopting GTO strategies, making good money, and not adapting so much to the person across from them. What solidified it, Brown thinks, was the match itself.

They invited four of the world's best heads-up no-limit specialists and challenged them to 120,000 hands over 20 days. The bot was not trying to adapt, exploit, or run mind games. It was approximating the Nash equilibrium, and it crushed them. They were playing for 50–100 blinds. Over the course of those hands it made close to two million dollars in chips. Fridman later asks whether you could model the theatrical side of the game, the trash talk and the chaos that sell sponsorships. That, Brown has already said, is a different product from a bot that wins.

In graduate school Brown's lab entered the annual computer poker competition, where research groups brought bots and played them against each other. They won in 2014 and 2016, then decided to face top humans. It was not their first try. In 2015 they had already played professionals, and the bot lost by a sizable margin. The jump to 2017 was algorithmic, not just more compute. In 2015 the idea was to solve as much of the game as possible up front and store a lookup table. In 2017 the table was only a starting point. At decision time the bot searched in real time for a better strategy than the one it had precomputed in self-play.

Chess search is moves and the positions they lead to. Poker search is stranger. You search over the actions you can take with your hand, the probabilities of those actions, and the probabilities of actions with other hands you might have had. A bluff with a bad hand is only a good action if you would also bet a strong hand in that spot; otherwise it is an obvious bluff. What you want is an action the opponent can map to many holdings, so they always have some doubt about whether to call or fold. The quantity being maximized is expected winnings assuming the opponent responds as well as possible. Putting them in a tough spot is not an explicit objective. It is what optimal play looks like in practice.

After 2015 it was obvious to Brown that humans and the bot were using time differently. The bot, after months of self-play, acted instantly. Humans in a tough spot would think for five minutes. Small-scale experiments showed that even a little search was the equivalent of making the precomputed strategy a thousand times bigger, and it dwarfed the team's earlier efforts to scale the table. In Libratus, search started on the turn and ran to the end of the game, reasoning about roughly a thousand possible two-card holdings for each player. They later learned to search only a few moves ahead, which cut the compute dramatically. A last-minute option let the bot bet ten times the pot. Humans usually size bets at 50 to 100 percent of the pot; the bot started overbetting, and Brown worried it was a bug until he watched humans tank for five or ten minutes with the second-best hand facing twenty thousand into a thousand. Overbets are now common in high-level play. Daniel Negreanu, Brown recalls, was excited to learn from the AIs and thought he had a decent chance of beating the bot. That was years ago. Brown thinks it is now clear that against a serious bot, in these variants, humans do not stand a chance.

Brown says the public and the AI community still underestimate search. TD-Gammon used two-ply search; Deep Blue searched far ahead; AlphaGo's Monte Carlo tree search was essential. AlphaZero's strongest version sits around 5200 Elo against humans near 3600; strip out test-time search and the rating drops to around 3000. Nobody, he says, has made a raw neural net that is superhuman at Go. The Libratus match itself ran 20 days on a heavily parallelized C++ system using on the order of a thousand CPUs and terabytes of memory. Brown figured they had 50/50 odds, because the humans had time to find leaks, which is what had happened in 2015. The four players treated beating the bot as the only objective, compared notes, and were even given a log of every hand including the bot's cards — usually you see only about a third of hands to showdown. The poker community opened at about four to one against the machine. The bot won the first three days and the odds were still even; around day eight it was obvious the humans were getting crushed. Brown had spent five years on the project. Realizing the high-school dream of an unbeatable strategy, he says, was something he would not trade.

Once you leave two-player zero-sum, the guarantee is gone, and there was a real debate about whether the techniques would extend. Brown thought they would still produce a strong strategy in practice. The game is also exponentially larger. The main idea that made six-player possible was depth-limited search: instead of planning all the way to the end of the game, as Libratus did from the turn onward, the bot looks a few moves ahead and then plugs in a value estimate. Outside two-player zero-sum you also get an equilibrium-selection problem. Brown describes a ring where the object is to stand as far from the others as possible: equally spaced points are a Nash equilibrium, but there are infinitely many such arrangements, and if each player computes one independently you get scattered dots, not a joint equilibrium.

In Diplomacy, he says, approximating a Nash equilibrium does not really work. Six-player poker is different because it is so adversarial: you are not even allowed to cooperate, and collusion is against the rules. The two-player techniques still work in practice. Nobody has a proof. The most interesting thing about Pluribus, to Brown, is the cost. Put a price on Libratus and the final training run was about a hundred thousand dollars. Pluribus's final training run would cost less than $150 on AWS. That is not two years of cheaper hardware. It is algorithmic, especially depth-limited search, which Brown first ran on two-player poker and which took superhuman heads-up play from a six-figure cluster down to something you could train on a laptop. Algorithmic improvements really do matter.

They used no neural nets at all in Libratus or Pluribus, which surprised people in 2017 and still surprises them. Nets are powerful when the hard part is inventing features for a value function. That was the last problem in Go: looking at a board and saying who is winning. In poker the hard part was a scalable algorithm that understood it had to bluff with the right probability. Modern poker AIs do use nets for value, but in a different way than chess or Go: the value of a state depends on beliefs about the cards. Pocket aces are monstrous if the other player thinks you are weak, and much less so if they know you have them. Asked for the greatest player of all time, Brown finally answers Daniel Negreanu — not Hellmuth — because, like chess after Kasparov, the game has moved. Many boom-era television stars have not kept up with the computational strategy; Negreanu has kept studying GTO, and Hellmuth, Brown adds, is one of the few old-school stars who has done the same.

Diplomacy, Brown says, is a different species from chess, Go, poker, StarCraft, and Dota. It is seven players, created in the 1950s, set in Europe before World War I: England, Germany, Russia, Turkey, Austria-Hungary, Italy, and France. Each turn you spend five to fifteen minutes in private conversation making deals, because the only way to make progress is to work with someone against the others. Then everyone submits moves at once. You can promise to support a neighbor this turn and not follow through; they find out when the moves are read. Communication is unstructured natural language, in person or online, with no limit on what you can discuss. Brown's shorthand is a mix of Risk, poker, and the television show *Survivor*: a game about people rather than pieces.

You win by controlling a majority of a map of about fifty territories. Units can hold, move, or support a move or hold — including someone else's. That is what the negotiations are about, and you do not have to do what you said. The tension is the game. Brown has heard it was played in the Kennedy White House and was Kissinger's favorite; he does not know if that is true. The story he has heard about its creation is that its designer saw World War I as a failure of diplomacy and wanted a game that would teach it. In the ideal version, nobody actually wins, because if someone is about to, the others should work together to stop them. War, in that reading, is ultimately futile. France is generally seen as the strongest power, which makes others less likely to work with it; Russia starts with four units instead of three and more borders to defend. Games usually last fifteen or twenty turns. Solo victories are rare among strong players. Draws are the common ending, and Brown's research used a scoring system based on how much of the map you controlled.

People have been building Diplomacy AIs since about the 1980s, with rule-based, heuristic systems that tried to encode human knowledge. The hard problem, Brown says, is that your action space is essentially all the sentences you could say. A natural-language *Settlers of Catan* bot would mostly discuss two sheep for a wood. Diplomacy conversations run to support, counter-support, gossip about what a third party promised, and multi-turn IOUs. They considered a simplified language and decided the most useful research was to take natural language head-on. Sample messages from webDiplomacy run from stiff greetings to warnings about a Lepanto or an octopus opening — whispers that Austria might be about to get hit. You have to parse what the words mean, and also whether you should be the one repeating them.

Every famous game-AI breakthrough Brown lists — checkers, chess, Go, poker, StarCraft, Dota 2 — happened in a purely adversarial setting. You do not have to understand how humans play. You scale an algorithm, often with self-play, toward a Nash equilibrium that, in two-player zero-sum, will not lose to anyone. You can use self-play in Diplomacy, Brown says, but it is no longer sufficient to beat humans. Train from scratch with no human data and the AI will not spontaneously speak English. It will invent a weird robot language only it understands. Sit that agent down with six people and they will think it is talking gibberish and work with each other instead. A chess program does not need to play like a human to beat a human. Here you have to understand what people think a good ally looks like, or they will pick someone else.

Brown was wrapping Pluribus in 2019, watching AlphaStar, OpenAI's Dota work, and GPT-2, and felt the next multi-agent benchmark had to be bigger. Chess had taken on the order of forty years of AI research, Go another twenty; Diplomacy looked like a decade. A colleague, Adam Lerer, argued they would learn something even if it was out of reach. They played about forty games with real humans online. The bot came in second among everyone who played five or more games — about eighty players total, nineteen of whom cleared that bar. Brown did not want a table of experts only. His analogy is a self-driving car: you do not test it among stunt drivers, you put it in an ordinary city. Strong Diplomacy players, he says, really shine when weak players are at the table, because they know how to take them under a wing.

The architecture splits strategy from talk. A language model is trained, then made controllable on intents: an action the bot wants to play, and an action it would like the other player to play. A strategic model, using reinforcement learning and planning much as in poker or chess, proposes those intents while staying human-compatible. The language model then writes a message that corresponds to the plan. The language model is not very smart about what to withhold, so a planning filter estimates what the other player would do if the message were sent and drops lines with negative expected value — including announcing an attack. Other filters catch nonsense and, importantly, lies. Diplomacy has a reputation for deception, but lying made the bot score worse: once it lied, people never trusted it again. First-time humans treat the game as lying; top players, Brown says, treat it as building trust in an environment that encourages everyone not to trust anyone.

A self-play bot in a two-player zero-sum version of Diplomacy was superhuman. The same training for the full seven-player game, even without language, got destroyed, because it could not model how humans actually cooperate — like a self-driving car trained on the left side of the road and dropped into an American city. A concrete failure: when one player is about to run away with the map, the remaining players must unite. The bot will help stop the leader and, with leftover units, grab extra centers from its allies. Rational players should accept a score of one over zero. Humans get angry and throw the game. You cannot learn that from scratch. The fix was to regularize self-play toward human data: about 50,000 games and more than ten million messages from webDiplomacy.net, a site that has been running for some twenty years. A supervised anchor policy imitates humans; planning and RL may deviate only when the expected value is particularly high. The team open-sourced models and code and set up researcher access to the data. They also had a person watching the bot whenever it played, so it would not go off the rails and start threatening people.

Fridman asks how much of this transfers to real geopolitics, the tradition that gave us mutually assured destruction as a game-theoretic story about nuclear weapons. Brown goes back to Diplomacy's origin myth: World War I as a diplomatic failure, and a take-home message that if people play it right, war is unsuccessful. He sees war as inherently negative-sum. There is always a better outcome for everyone involved, and his hope is that as AI progresses the technology might help people avoid those outcomes. Transfer, though, is the same wall that stopped StarCraft and Atari agents from walking into the physical world: it is hard to collect data, and you do not have a well-defined action space or reward. Code generation and theorem proving do have those things, and that is where he expects reinforcement learning to be powerful in the short term. Diplomacy still feels closer to life than StarCraft, because the channel is natural language and the pretraining data is the internet's dialogue. It is a step, he says, not the destination.

The same human-compatibility trick already makes more human-like chess and Go engines. Supervised learning on grandmaster games produces a bot weaker than the humans it copied, because a net cannot cheaply approximate their search. Heavy search without a human anchor produces Stockfish: very strong, very alien. Regularize the planning toward a human policy and you get something both strong and human-like, tunable even to a 2800-Elo style, or, in principle, to Magnus Carlsen's. That is useful for preparation and for coaching. It is also a cheat-detection problem, because today's detectors look for moves that are common in engines and rare in people. Brown sees downside and upside: human-like coaches that can show a 2000-Elo player what 2200 looks like, and virtual worlds populated with agents that are actually fun to play with.

The Cicero paper's ethics section, Brown says, includes the uncomfortable fact that Diplomacy involves deception, and that language models capable of lying are a dicey research object. Fridman imagines a future in which a consumer assistant is not allowed to lie — unless you ask whether you got fatter this month. In a no-language Diplomacy tournament, they told everyone there was an AI at the table; humans spent the game hunting it, and as soon as they thought they had it they teamed up to kill it. On the path to more general intelligence, Brown thinks the agreed hole is data efficiency. A Go bot needs millions of games; a human master plays thousands or tens of thousands in a lifetime. That matters the moment you leave a simulator, especially in robotics. Humans do not learn poker from scratch either: they arrive with background knowledge of people, the world, and money. One way to close the sample-complexity gap is to let systems reuse that kind of general knowledge across domains.

Fridman asks for advice to beginners. Brown's own path is atypical: he started in game theory and moved toward reinforcement learning, which made him weaker in some corners of machine learning and useful in others. He tells students not to be afraid to try something different, and to build a foundation in math, computer science, and statistics — linear algebra, probability — without copying the exact on-ramp everyone else uses. Asked whether we will one day live life optimally, he reaches for the same missing object the bots need: a reward function. Tell an AI you want to be rich and it might tell you to rob a bank. Specifying what you actually want, including the unintended consequences, is the hard part, the way handcrafting a chess policy is the hard part. Until someone writes that function, Brown is taking game-playing agents one step closer to the messy, spoken world people already live in.