Libratus Beat the World's Best Poker Players Without a Neural Network

Tuomas Sandholm, a Carnegie Mellon professor who had just taken a best-paper award at NeurIPS, tells Lex Fridman that heads-up no-limit Texas Hold'em has become the AI community's main benchmark for application-independent imperfect-information algorithms. Unlike the multiway games that fill television and Friday-night kitchens, heads-up is two players, me against you, closer to chess or Go — except each person holds two private cards the other will never see. Five community cards arrive across four betting rounds with no cap on bet size. The private hole cards are the imperfect information; everything after that is public.

In 2017 his system Libratus beat four of the top ten specialists in that specific game. He brought them to Pittsburgh's Rivers Casino for twenty days and aimed for 120,000 hands, enough for statistical significance. Even these fast online pros could not have sat that many hands alone, so the four rotated from morning to evening. He raised $200,000 as an appearance pool and paid it out by how each player did against the AI, so they had reason to try whether they were ahead, behind, or sitting on the money bubble.

A couple of years earlier the team had asked whether they could play for real stakes. The Pennsylvania Gaming Board said no, so the match was an exhibition, closer to a boxed fighter than a cash game. They still kept a running chip count: if the money had been real, Libratus would have won close to two million dollars. The humans played through a familiar online-style interface, with the betting history left on screen so that memory would not become a hidden AI advantage.

Eighteen months earlier Sandholm had run a similar brains-versus-AI event with a weaker bot that did not beat the humans. He knew Libratus was much stronger and still gave himself only about a 50/50 shot, because you cannot really know how you will do against the top humans until you try. International betting sites listed the AI as a four-to-one or five-to-one underdog. After three consecutive winning days they still had the match at even money.

He splits the romantic poker myth in two. People overtrust humans, he argues, because they have watched how good the specialists are, and it is simply hard to believe a program could beat them. Until you have seen AI outperform a human, it is hard to believe it can — a lag he treats as a general lesson about the field, not a quirk of cards.

As for tells, the Hollywood version is real at a Friday-night home game, where a good reader can treat opponents like an open book. Among the very best, players hide those tells so well that hunting them is barely worth the effort. At that level the strategy space, on the order of ten to a hundred-plus, dwarfs body language. The amount of possible actions, not the eyes, is where the fight is.

The game tree is on the order of 10^161, too large to solve directly even with the fastest equilibrium-finding algorithms, so you abstract first. Abstraction in games is much trickier than in MDPs or other single-agent settings because of what Sandholm calls abstraction pathologies: a fine-grained abstraction can yield a strategy that plays *worse* in the real game than a coarser one. You have to be careful. There are two kinds. Information abstraction covers what chance does — the cards. Action abstraction covers what the players do — the bets.

For information, Libratus was fully automated. The algorithms do what the group calls potential-aware abstraction: they do not just score the current hand, they score how it might materialize into good or bad hands over later streets, in a bottom-up process that uses integer programming and clustering. Action abstraction leaned more on how humans and earlier AIs had bet. At the start of the tree they used a provably convergent automated method that finds the optimal combination of bet sizes, but it does not scale, so they applied it only to the first couple of betting actions.

Fridman asks the romantic question: if betting is everything, how well could you play without looking at your cards? Variance, Sandholm says, is why they needed so many hands in the first place. If both players go all-in, the stack is a huge pile of luck; even thousands of hands can be decided by swings. That is why you need not thousands but more than 100,000 for statistical significance. He has heard that the Norwegian pro Annette Obrestad once won a tournament by never looking; he calls that extremely rare. Hands still matter.

Libratus used no learning methods and still played at a superhuman level. Since the match, Sandholm's group has published work that does use deep learning to estimate evaluation functions, in the spirit of AlphaGo. In imperfect-information games that is not enough. If you hold a bad hand and the opponent thinks you hold a good one, you are better off, and the reverse is also true. The value of a state is not just a function of the cards. It depends on belief distributions along the path of play.

In Go, chess, or shogi there is at least a well-defined state and an evaluation of that state. Here you have to do something more. One of their papers, on depth-limited search for imperfect-information games, lets the opponent choose among different continuation strategies at the leaves of the search. That refuses to assume a single way the opponent will play, and it forces the lookahead not to be too optimistic. Libratus itself never needed this trick: it only searched at the end of a hand, where the payout is real. The new work lets them start looking ahead from the first street, which is otherwise too large.

DeepStack took a different path. It randomly generated situations, treated each as the start of a new game, looked ahead to the end, and used deep learning to fit the values of those states — states that were not just physical cards, but cards plus belief distributions. Sandholm is careful not to rank the two approaches from this conversation. He is pointing at a conceptual fork: either you terminate in real payoffs, or you learn a belief-aware value, or you let the opponent pick among continuations so the search stays sound.

Beliefs are explicitly modeled, but they are outputs, not inputs. Opening beliefs follow from a fair deal — every remaining pair equally likely — with card removal: if you hold the ace of spades, the opponent does not. From there, Nash equilibrium, which John Nash introduced in 1950, defines rational play as a pair of contingency plans such that neither player wants to deviate. As a side effect, Bayes' rule supplies a probability distribution over the real states inside every information set.

Rock-paper-scissors is the toy version. Draw it as if player one moves first, then hide that move inside player two's information set. Equilibrium says player one mixes one-third rock, one-third paper, one-third scissors; player two's beliefs on that set are therefore one-third, one-third, one-third. The construction is not player-specific. Sandholm's group needed no history of how these four humans, or any earlier bot, had played. "It's all about rationality." The method comes from the design of the game, not the design of the player.

They have a separate line of work that hybridizes opponent modeling with game theory, so you can exploit weak players more. He left that module off for Libratus. The four specialists had few holes, and they are the world's experts at counter-exploitation; once you start exploiting, you typically open yourself up. In two-player zero-sum games the game-theoretic strategy is unbeatable even if it does not *maximally* beat a fish: an opponent who plays poorly may throw off your beliefs, but they lose more by playing poorly than they gain by confusing you. A hybrid can start from equilibrium and, as data arrives in parts of the tree, drift toward exploitation while staying close enough not to get counter-exploited.

Before the three-player problem, Sandholm sketches the landscape. Purely repeated games, the same matrix over and over, are a coarse model and actually rare in the world; a sourcing relationship with a shifting catalog is not a repeated game. Stochastic games sit in between: little matrix games whose actions determine a distribution over the next game. Extensive-form games — tree form, with information sets for hidden information — are the most general. Poker is that last class, which is why the AI community has been benchmarking on it.

Two-player zero-sum games are conceptually and computationally easier, even when they are huge. Any equilibrium strategy is a best response to any other equilibrium strategy, so you can play a different equilibrium from your opponent and still get the right values of the game. That compatibility collapses with two-player general-sum, and it collapses with three-player zero-sum. All finite games still have a Nash equilibrium; Nash proved that in 1950. The problem is selection. If you pick from equilibrium A and I pick from B, in a non-zero-sum game we can both be worse off than if we had coordinated. With three players, collusion appears: two can gang up on the third and do radically better.

Noam Brown, Sandholm's student, said in a Reddit AMA that there are not many ways to make poker beyond current methods, but collaboration among players would make it extremely difficult. Sandholm agrees. He and another student, Gabriela Farina, had just presented a NeurIPS paper introducing a new game representation for coalitional games, because some standard representations cannot even express the structure. Bridge builds coordination into the rules: partners share payoffs but cannot whisper their cards, so they agree strategies ahead of time and use signals the other team can also understand. Auctions, negotiations, diplomacy, and poker do not write collusion into the rules. It can still be very helpful for the colluders.

Asked how any of this leaves the casino, Sandholm points at two companies. Strategic Machine takes the technology into business, sports, gaming, finance, and electricity markets. Strategy Robot takes it into military, security, cybersecurity, and intelligence. A third company, Optimized Markets, is about combinatorial markets and optimization, not this game-theoretic stack. Modeling humans, he says, is not always the goal. In a zero-sum game he does not particularly care whether the opponent is following his model of rational behavior, because if they are not, that is even better for him.

Fridman offers a messy human example from autonomous vehicles: the nonverbal dance in which a jaywalker steps into the road, trusting the car not to kill them. Sandholm has not thought hard about jaywalking. He does have a fleet-level idea. Imagine a Waymo fleet and an Uber fleet sharing a city. The traffic code defines what is legal and still leaves a huge strategy space. Today, merging cars slow down, look at each other, and lose time. It would be better if those situations were already negotiated so the cars could merge at speed.

There are far too many situations to bargain by hand, so the idea is automated negotiation over a combinatorial bundle: in these cases I let you go first, and in exchange you let me go first in those. Whether that game should be modeled as imperfect information is, he says, a real question. Perfect-information games are much easier, so if you can get away with them you should. If the real situation is imperfect information, you will have to pay that price.

What did the match teach him about building AI? He likes performance-oriented research, and his group tries to run the full stack from idea to theory to experimental systems to fielding and commercialization. In a lot of AI, he argues, you have to build the big systems and evaluate them at scale before you know what works. Computational game theory has been a sharp lesson. Many techniques look good in the small and fail in the large. Methods with better theoretical convergence rates, including first-order methods, still lose in practice to CFR-based algorithms. The theory is not tight enough to rank the algorithms. Projections drawn from small games can be very misleading.

The first heads-up no-limit brains-versus-AI contest, he says, probably made him the most hated person in the world of poker. He did not mean to. A lot of people felt superhuman bots would scare humans off the game and take their money. The comments were "super aggressive" — everything just short of death threats. He does not think he killed the game, any more than chess engines killed chess. He was not a poker player before; the AIs taught him nuances, and they have changed how the game is played. Top humans now incorporate those "very machine ways" of betting. If anything, the work made poker richer for people, not something to walk away from.

Fridman notes that putting ideas on the line against top humans is rarer in academia than a NeurIPS best paper. Sandholm does not call it bravery. It takes a lot of work to organize something that big. In general he thinks the proof is in the pudding of real-world scale. In this particular case it was also a years-long competition among research groups to be first past the superhuman line. "A little friendly competition can do wonders for progress."

Mechanism design is the art of writing the rules so that the outcome you want is what rational players produce. Sandholm's version is automated: search for those rules rather than hand-tune them. He still believes in the direction. It is not a panacea. The field is full of impossibility results proving that no mechanism, manual or automated, accomplishes objective X on class C. Those are not statements about human ingenuity. They are proofs that a whole class cannot hit a target. Automated design, though, is not designing for a class. It is designing for a specific setting. Even inside an "impossible" class, not every instance is impossible, and you can carve islands of possibility. His group has done that for the Myerson–Satterthwaite theorem of 1983, the famous impossibility of efficient trade under imperfect information: in many concrete designs, efficient trade is still on the table.

Asked about politics, traffic lights, and other human systems, he is more cautious. Mechanism design itself has had fairly limited success. Most real-world situations are not sound from a mechanism-design perspective, even when knowledgeable theorists were in the room. People typically take insights from the theory and import them, rather than applying the mechanisms directly. The famous example is the FCC spectrum auctions, where Sandholm had a small role and "excellent economists" were working. The practical rules are still such that bidding truthfully is not the best strategy.

Mechanism designers usually try to make life easy for participants by making honesty dominant. Even in those very high-stakes auctions, with tens of billions of dollars of spectrum on the block, truth-telling is not best — and, he adds, nobody knows even a single optimal bidding strategy. The challenge is many items and many players; the mechanisms are such that even with one or two items, truthful bidding would not be optimal. Theory and the institution that actually runs have a much wider gap than the textbooks suggest.

Fridman puts Libratus next to Deep Blue and AlphaGo and asks what the next such event is. Sandholm does not really know. In game solving, heads-up no-limit Hold'em was the last widely agreed-upon benchmark. Groups are working on StarCraft, Dota 2, Diplomacy, Hanabi — recreational games, none of them acknowledged as the next chess, Go, or Hold'em. He hopes a shared benchmark returns, because different groups attacking the same problem drove application-independent techniques forward very quickly over ten years. He himself is not going to work as hard on recreational benchmarks. Strategic Machine and Strategy Robot are the point now: push the technology into practice.

The field, he argues, is in a very different place from machine learning. ML is fairly mature, broadly applied, and proven in the real world; you could argue it became superhuman in the 1990s, at least for some supervised tasks. Game solving has almost no applications yet. It has only just become superhuman. The breakthrough he would like in five or ten years is not a new algorithm on a leaderboard. It is showing that most military planning, or most business strategy, is actually done with computational game theory.

Are those strategies explainable, the way a neural net's weights are not? A little of both. A Nash equilibrium has provable properties. You are not, as with deep learning, crossing your fingers before training and crossing them again after you have the weights. You know the solution-quality guarantees. That does not mean a human can understand the strategy. Deep learning and computational game theory are "in the same boat" for interpretability. Only the game-theoretic side comes with guarantees. Seeing automated systems propose candidate strategies for business or military operations is, he admits, more a belief than a substantiated fact — an optimistic one.

On existential risk he is much more optimistic about AI's upside than Max Tegmark or Stuart Russell. His own work is the exhibit. The nationwide kidney exchange his group runs has hundreds of people walking around alive who would not be, and it created jobs at exchange operations, transplant centers, and hospitals — extra surgeons, nurses, anesthesiologists, beds. Combinatorial sourcing auctions at a previous startup, CombineNet, ran 800 large-scale events from 2001 to 2010. On $60 billion of spend they improved supply-chain efficiency by 12.6 percent, more than six billion dollars of real efficiency: less empty trucking, less waste, a smaller carbon footprint, not a transfer from one pocket to another.

Value misalignment he calls a fairly theoretical worry he has not seen, because he does a lot of real applications. The closest case was a late-1980s argument while building transportation optimizers. Someone had heard that high asset utilization was a good objective. He did not even put it in the solver; he just showed that the optimum would be to load the trucks full and drive in circles. Nothing would ever get delivered. You would have 100 percent utilization. He has known the phenomenon for over thirty years. He has never seen it actually become a disaster. Give AI the wrong objective and it will optimize that to the hilt, hurting more than a half-baked human — but the gap between the worst case you can theoretically imagine and what usually happens is large.

The threats he actually ranks first are climate change and nuclear war. He commissioned studies in two startups looking for a market or optimization angle on climate; pollution-credit markets failed for lack of political will, not for want of a better design, and Chicago's market was shut down. Mutually assured destruction is a small-matrix argument that nobody wants to initiate — a very coarse analysis that worked when there were two or three superpowers. Smaller nukes lower the threshold. Smaller countries and non-state actors may get them. He thinks the risk is higher now than maybe ever before.

What excites him at NeurIPS is getting scalable game-solving into the field, where the applications tell you which technology gaps still need filling, plus more scalable integer-programming methods. His group had a paper that summer on the first automated algorithm-configuration result with theoretical generalization guarantees: see this many training examples, tune this way, and you can expect good performance on a real distribution you have not seen. Algorithm configuration had been serious for at least seventeen years without that. Another open direction is extensive-form games that go forever without repeating — business or military strategy "from here to eternity," where even scoring whether a move was good is part of the research problem.