Libratus' Win Highlights Online Poker's Battle against Bots

The big story in the poker world remains how the AI program Libratus defeated four elite poker players by $1,766,250. As we covered, the victory by the computer presents a realistic possibility that no-limit Texas hold’em is a solved game.

Libratus’ victory also creates more fear around the notion that online poker will eventually become filled with bots who dominate all stakes and games.

As you may know, bots have been successfully used in low-stakes poker for years. But are we now entering an age when not even mid and high-stakes action will be safe? Find out as we detail the battle to keep AIs from ruling the online poker world.

Unconventional, but Optimal Play

The advantage to poker bots is that they can play an endless number of hands without tiring. But in no-limit variants, the limitation was always thought to be that artificial intelligence can’t master the complex nuances of no-limit betting. That notion was dispelled, though, with Libratus’ dismantling of the poker pros at Rivers Casino Pittsburgh.

The pro team of Jimmy Chou, Dong Kim, Jason Les, and Daniel McAulay played 1,500 hands per day before resting for the night. As the month-long challenge wore on, the players went through lots of coffee in an effort to stay focused. Les said that the slow play of Libratus made the challenge drag on too.

“Waiting should not affect me whatsoever,” he told Bloomberg. “But sometimes you’re just like ’OK, is this going to be over yet?”

In contrast to humans, Libratus plays slowest at the beginning of hands with small pots, given that it must run through more possibilities due to having more chips in its hand. And sometimes Libratus makes big wagers out of nowhere, which is unconventional and throws human opponents off. But the machine played well enough to stage the first no-limit hold’em win for a computer against top-flight pros.

“No-limit hold’em is the game you see in tournaments, and it has the reputation of being more of an art than a science,” explained Adam Kucharski, author of The Perfect Bet: How Science and Math Are Taking the Luck Out of Gambling. “There was the idea that this game would be safer for much longer from these machines.”

Libratus Isn’t the Only Bot Beating Poker Pros

What’s telling about the future online poker may face is that Libratus isn’t the only AI beating poker pros. In early January 2017, researchers from the University of Alberta did a paper based on their program, DeepStack, defeating 11 professional players.

Tuomas Sandholm, one of the lead developers of Libratus, pointed out that DeepStack wasn’t playing top-quality pros. Michael Bowling, who spearheaded research on DeepStack, agrees with Sandholm’s assertion.

But Bowling said that his AI played against pros when they were more rested, rather than for a month straight like Libratus. DeepStack also defeated the pros by a margin three times as large as Libratus.

What both men do agree on is that poker may never be the same now that the AI has handily defeated the humans at no-limit hold’em. They also point out that conquering poker is only one small part is testing programs that will be used for medicine, cybersecurity and more.

“This is the main benchmark the community has settled on, but these algorithms are not for poker,” said Sandholm. "They’re general purpose.”

Despite DeepStack and Libratus being positioned for greater purposes than poker, their advancements show what’s possible for future internet poker bots.

The Challenges Have a Different Format that Regular Online Poker

While what the bots have done is impressive, it’s important to remember that these games don’t happen exactly like real online poker. DeepStack and Libratus only play heads-up matches, and the number of chips between the AIs and their opponents is reset following each hand.

This takes some of the psychology out of the game since players can’t use big stacks to intimidate opponents. Eric Hollreiser, head of communications for PokerStars, said that the difference between the challenge and real poker is what currently protects the online game.

“While on a functional hand-by-hand basis it mimics poker player, it is far, far removed from the reality of what happens at tables,” he said.

Based on Hollreiser’s opinion, it may be a while before bots are ready to dominate the highest level of regular internet poker. But as Chris Grove from Online Poker Report explained, the possibility is inching closer to reality. “If you’re an online poker operator, this is probably your number one fraud concern, and probably by a pretty wide margin.”

Methods to Combat Poker Bots

PokerStars has 70 employees whose sole job is to root out poker bots and other forms of fraud. Any time that fraud detection suspects a bot, they call players and ask them what strategy they’d use on certain hands.

Another method of finding bots involves asking players to play for an hour using a rotating cam to show surroundings, and that the person is indeed controlling the action.

It’s highly important that PokerStars and other sites do what they can to monitor bots because they can win profits 24/7, even if they’re not elite AIs like DeepStack and Libratus.

How Long Until the AIs Catch Up?

It’s not as if the four poker pros who played Libratus weren’t giving it their all. They met every morning to share leaks in the computer’s game.

“There were specific exploits we identified in the first few days,” Les said. “We attacked them and attacked them, and now they’re gone.”

But Libratus also made adjustments, using part of its computing power to play the hands, and another part to improve. At night, the AI switched entirely to study mode, using 600 nodes (approx. 3,000 Macbooks working simultaneously) to better its poker game.

Afterward, Les said that he was trying to figure out how he could use some of Libratus’ unconventional techniques in his own game. But in the end, he gave up on this notion. “We just simply do not have the mental capacity to do it,” said Les.

Using the Nash Equilibrium strategy in poker, AIs have proven to be superior to their human opponents heads up. But as for now, they can’t compete against skilled players on multi-player tables. That’s because multi-player games are just as much about identifying other players’ flaws as they are using perfect strategy.

How long until computer programs figure this aspect out?

It took over 20 years for an AI to beat top pros in a heads-up situation. But given the rapid improvement of these machines lately, it may not take this long for bots to conquer online multi-player games.

Even when AIs do become superior in multi-player poker, we can at least count on the fraud detection departments at major sites to help stop these programs from ruling the game.