Earlier this year, the Libratus poker playing AI defeated a team of professional card players soundly in “Brains vs. AI”, a Texas Hold’em demonstration at Rivers Casino near Pittsburgh. The competition was organized by a team of artificial intelligence researchers from Pittsburgh’s Carnegie Mellon University.
Josh Wardini recently published a graphical analysis of the AI poker player’s skills, along with comparisons of its predecessors. That got me to thinking what lessons poker players could learn from Libratus. The answer lies in part from the ongoing development of poker playing computer intelligences.
Two years earlier, a similar contest was held at Rivers Casino between a previous AI, Claudico, and four poker players. In that poker event, the human card players soundly defeated Claudico. Doug Poker, Jason Les, Dong Kim, and Bjorn Li beat Claudico for $732,713 in house money.
In the 2-year interim, scientists Noam Brown and Thomas Sandholm of Carnegie Mellon and Michael Bowling of the University of Alberta worked to improve their artificial intelligence’s poker skills. The results speak for themselves, as Libratus defeated Jason Les, Dong Kim, Jimmy Chou, and Daniel McCauley for a combined $1.75 million.
The poker sessions took place over several days, with a series of heads-up poker sessions between the team of pros and the gaming AI. What resulted surprised and even confounded the poker professionals, who came away impressed with the AI’s newfound skills.
What Can Poker Players Learn from Libratus?
So, the question is: what can human poker players learn from Libratus? What improvements were made in the past 2 years to allow an artificial intelligence to figure out pro players’ weaknesses and exploit them? What weaknesses which Claudico had were improved by Liberatus?
What did the Carnegie Mellon research team incorporate into its latest poker playing AI that gave biological poker players so much trouble?
These are all questions to explore. The differences between the 2015 poker-playing AI and the 2017 version seems like a good place to start.
Differences in Libratus and Other AIs
At the time he defeated Claudico, Doug Polk said that the AI “used a very complex strategy, consisting of many small and large bets.”
Despite the complex strategy, Claudico was not as adaptive as its replacement. Claudico had flaws which allowed skilled players to figure out its tendencies, then take advantage. It did not mix up its poker strategies enough to keep opponents off-balance.
Libratus Can Bluff
Frank Pfennig said that Libratus is a vast improvement on Claudico in that aspect. Pfenning said Libratus can use deception to win hands, where its predecessors could not. Pfennig said, “Developing an AI that can [bluff] successfully is a tremendous step forward scientifically.”
Libratus does not use a built-in strategy. Instead, it uses an algorithm to computer new strategy, randomizing its actions to keep its opponents guessing. The algorithm is not specific to poker, but instead can be applied to many imperfect-information problems. In effect, Libratus uses Game Theory to keeps its opponents off-balance.
Libratus Was Self-Correcting: It Evolves Strategy
Perhaps more importantly, Libratus reevaluated its strategy every night. It would go through the game sessions and analyze what worked and didn’t work. Using a calculational tool about a million times faster than a human brain, Libratus figured out a player’s strategy and adjusted.
Dong Kim discussed with PokerNews how Libratus improved overnight. Libratus analyzes its own play after every day’s play. In cold, rational fashion, it finds it mistakes and corrects them. Every day is a new strategy.
Kim said, “It was really difficult for us to play. We would bring a strategy and it would be good for the day of, and then the next day it would bring something new to the table. And we were not ready for that, so it was overall really, really tough.”
Poker AI Uses a “Mixed Strategy”
Dong Kim said that humans can incorporate some of the small tricks the card-playing robot uses, but many of its tools are beyond human capacity.
Mr. Kim said, “The AI played really well and I think it’s world-class. There are a lot of good things that I would like to incorporate into my game, but the execution’s really tough.”
“It has these really, really great strategies but it’s also a mixed strategy so the moment you do one thing a little bit too much, then you’re already too exploitable, so it’s going to be pretty hard to mimic or emulate in general so we’ll see. I’ll take the simple ones.”
AI Overbets More Than Humans
Dong Kim said the artificial intelligence makes plays that do not appear rational to human beings. While that sounds like it would be a disadvantage, it played havoc with the human players’ attempts to decipher its tactics.
He added, “It overbets more than anyone. If it was a human player, we would have thought it was a complete fish, to be honest, because it just does it so much. Actually, when it overbets, it actually bluffs a lot. Yeah, it just bluffs a lot.”
Libratus Style: Lessons Learned from Poker AI
So what are the applicable lessons learned from Libratus? A surprising number of tips come to mind. Some of the poker strategy tips Libratus teaches us are common sense, traditional strategies. Not all of them, though. A surprising number of Libratus’s advantages seem counter-intuitive.
- Analyze your own play every night.
- Find weaknesses and correct them.
- Vary your strategy, to keep player’s off-balance.
- Don’t let emotions about past hands get in the way of future bets.
- Raise the bet, because it sometimes disconcerts an opponent.
- Bluffing more than you think you should, but to good effect.
Some of the ideas, such as overbetting, sound like the kind of strategies a fish would use. It’s not such a fine line, because Libratus uses bluffs and aggressive play to wrongfoot an opponent and keep them on the defensive. In short, the AI is not bluffing simply to look cool or be a showman; it has a purpose.
Whether you can analyze the game well enough to use lessons learned from Libratus is another matter. Having an artificial brain that can do the equivalent of 20,000 man-hours of calculations is a bit different than our limited human capacities. But it is a good reminder to put your opponent on the defensive and not let them dictate play all the time to you. Vary betting and don’t be a calling station.
The main lesson to learn from Libratus is to look at one’s previous gaming sessions with a clear, rational eye. Don’t let emotions get in the way. Don’t beat yourself up over bad hands and bad beats. Take a lesson from Libratus and use that to determine tomorrow’s strategy.
Win or lose, look at every hand as more information. Use that information to evolve new strategies. Be adaptive like Libratus, not rigid like Claudico.
Are Computer Poker Players Better?
Now that Libratus can beat poker professionals in a game of cards, is the human race obsolete when it comes to poker? Not yet, though even better AIs will be developed to cover some of Libratus’s shortcomings.
For instance, it must be noted that Libratus beat players in heads-up poker, but has not played a table full of players. The next thing is to put the poker playing AI at a table with 8 other poker players and see how it does. Can it evolve poker strategies to account for 8 different strategies?