Deep Stack Poker

DeepStack bridges the gap between AI techniques for games of perfect information—like checkers, chess and Go—with ones for imperfect information games–like poker–to reason while it plays using “intuition” honed through deep learning to reassess its strategy with each decision. In poker, a 'deep stack tournament' is a tournament where players start off with a large amount of chips relative to the size of the blinds. For instance, in a traditional tournament, players might start off with 5,000 chips and blinds may begin at 25/50, with antes being added later in the tournament.

What is Deep Stack Poker?

Deep stack poker means having a big stack relative to the big blind, typically around 150-200 bb or more. In tournaments that you start with a deep stack, celled deep stack tournaments, deep stack poker strategy also applies.

What changes in strategy in Deep Stack Poker?

When you are playing deep stack poker, there is room for more betting rounds, notably on the turn and river. Rarely, will you commit most of your chips on the flop, so you have to be prepared to play all the way through.

Also, as the bet sizes gradually increase, if the hand is bet all the way through, you will have to face huge bets and make big decisions by the turn or river.

Playing with a Deep Stack is tricky

When you play with an average stack, you will often be facing all-in bets on the flop or turn. Deciding how to respond is a relatively simple exercise of putting your opponent on a range and comparing the pot odds to your hand’s equity against that range.

However, when facing a big bet on the flop, with a deep stack allowing for two or three more bets, decisions are much more difficult. You now have to consider implied odds, protecting your hand.

So, it is best for beginner players to avoid playing with a deep stack until they gain some experience to allow them to navigate through difficult decisions.

As the decisions you get to face when you are playing deep stack poker are more complex, having a good position becomes even more critical. When you have to decide for 100bb or more, you want to have the maximum amount of information, and position can give you just that. This is the opposite of short stack poker where decisions are more trivial, and position is not a big concern.

What hands should you play?

Remember the saying, “small hand small pot, big hand big pot”? Well, with a deep stack, the pot can potentially become very big!

As simple as it may seem, this aspect changes the dynamics and value of starting hands drastically.
Hands that play well short stack, like AJ or KQ, go down in value in deep stack poker. These hands can make a lot of top pair, top kicker type of hands, and win many small pots. However, in deep stack poker, there is the risk that the betting gets out of hands and ends up too big for only a top pair. So, you can win a few small pots but potentially lose some big ones!

When you play these hands, be sure to try to control the betting and not let the pot get out of hand.
On the contrary, drawing hands, like suited aces, suited connectors, and one-gappers, and small pairs, become more valuable. These hands can make strong hands that are somehow concealed and win big pots. You want to make hands like a set, a flush, or a straight, that you can value bet with all the way through!

To sum up, big broadway hands can win many small pots, and speculative hands can win a few huge pots. In a way, when the stacks are deep, hand value normalizes.

This means that you should play them both, and keep your opponents guessing. With top-pair type of hands, you should exercise pot control.

In deep stack poker, there is also more room for creative plays, like floating or three barrelling bluffs.

An example

You have an effective stack of 200bb and open-raise under the gun with A♥Q♠. You get called by a player in middle position and by the button. The flop comes A♣J♦7♠. There are about 10bb in the pot. You decide to make a c-bet of 7.5bb and the first player calls, while the button folds.

The turn is a blank, the 4♣, and with 25bb in the pot, you decide to bet 20bb. Your opponent raises you to 60bb. What should you do?
Well, you got yourself into a mess! Playing AQo type of hands out of position in deep stack poker is tricky and requires careful play and pot control.

Deep Stack Poker Austin

In this scenario, your flop bet is ok, but your turn c-bet is questionable, as you start building a big pot. If you call in this spot, the pot will be 145bb, and you will possibly face a bet on the river for the remaining of your chips, not a good situation for just a pair!
Unless you have a read on your opponent, fold and cut your losses to a minimum.

An example

You are on the button with T♥8♥ and a deep stack. A player raises from early position and gets called by a player in middle position and the cutoff. You decide to play by calling.

The flop comes A♥J♦7♦. Your opponent makes a standard c-bet. The first player folds, the cutoff calls, and with an inside straight draw and a backdoor flush draw, you also call.

The turn is the 9♣, and you make your straight! The first player bets again, and the cutoff folds. You raise, and your opponent calls.

The river is a blank, the 4♣, and your opponent checks. There are about 150 bb in the pot, and you have 150bb more. What should you do?

You have the nuts, so there is no question whether you should bet or not. As your opponent has shown strength, this is an excellent spot to go all-in. You have position, and he may well put you on a missed diamond flush draw.

You go all-in and your opponent calls, showing AJ for top two pair.

This example demonstrates how with position and a deceptive hand, you picked-up a huge pot.

Keep in mind that you should not stick to optimal betting patterns, but mix in some bluffs or thin value bets to keep your opponents guessing.

In a nutshell

In deep stack poker, there is room for a lot of betting. Deep stack strategy requires adjusting your starting hands, valuing position, and employing pot control.

This was the last tutorial of the Advanced Poker Strategy Course. Congratulations on completing the course! You can test the skills that you learned by Passing the Quiz

Deep Stack Poker Club Hilliard Ohio

If you feel ready, you can try out your skills for play or real money to one of our Recommended Online Poker Sites 🙂

Expert-Level Artificial Intelligence in Heads-Up No-Limit Poker

Links

Twitch YouTube Twitter
Downloads & Videos Media Contact

DeepStack bridges the gap between AI techniques for games of perfect information—like checkers, chess and Go—with ones for imperfect information games–like poker–to reason while it plays using “intuition” honed through deep learning to reassess its strategy with each decision.

With a study completed in December 2016 and published in Science in March 2017, DeepStack became the first AI capable of beating professional poker players at heads-up no-limit Texas hold'em poker.

DeepStack computes a strategy based on the current state of the game for only the remainder of the hand, not maintaining one for the full game, which leads to lower overall exploitability.

DeepStack avoids reasoning about the full remaining game by substituting computation beyond a certain depth with a fast-approximate estimate. Automatically trained with deep learning, DeepStack's “intuition” gives a gut feeling of the value of holding any cards in any situation.

DeepStack considers a reduced number of actions, allowing it to play at conventional human speeds. The system re-solves games in under five seconds using a simple gaming laptop with an Nvidia GPU.

The first computer program to outplay human professionals at heads-up no-limit Hold'em poker

In a study completed December 2016 and involving 44,000 hands of poker, DeepStack defeated 11 professional poker players with only one outside the margin of statistical significance. Over all games played, DeepStack won 49 big blinds/100 (always folding would only lose 75 bb/100), over four standard deviations from zero, making it the first computer program to beat professional poker players in heads-up no-limit Texas hold'em poker.

Games are serious business

Don’t let the name fool you, “games” of imperfect information provide a general mathematical model that describes how decision-makers interact. AI research has a long history of using parlour games to study these models, but attention has been focused primarily on perfect information games, like checkers, chess or go. Poker is the quintessential game of imperfect information, where you and your opponent hold information that each other doesn't have (your cards).

Until now, competitive AI approaches in imperfect information games have typically reasoned about the entire game, producing a complete strategy prior to play. However, to make this approach feasible in heads-up no-limit Texas hold’em—a game with vastly more unique situations than there are atoms in the universe—a simplified abstraction of the game is often needed.

Deep Stack Poker

A fundamentally different approach

DeepStack is the first theoretically sound application of heuristic search methods—which have been famously successful in games like checkers, chess, and Go—to imperfect information games.

At the heart of DeepStack is continual re-solving, a sound local strategy computation that only considers situations as they arise during play. This lets DeepStack avoid computing a complete strategy in advance, skirting the need for explicit abstraction.

Deep Stack Poker Tournament

During re-solving, DeepStack doesn’t need to reason about the entire remainder of the game because it substitutes computation beyond a certain depth with a fast approximate estimate, DeepStack’s 'intuition' – a gut feeling of the value of holding any possible private cards in any possible poker situation.

Finally, DeepStack’s intuition, much like human intuition, needs to be trained. We train it with deep learning using examples generated from random poker situations.

DeepStack is theoretically sound, produces strategies substantially more difficult to exploit than abstraction-based techniques and defeats professional poker players at heads-up no-limit poker with statistical significance.

Members (Front-back)

Michael Bowling, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, Viliam Lisý, Martin Schmid, Matej Moravčík, Neil Burch

low-variance Evaluation

The performance of DeepStack and its opponents was evaluated using AIVAT, a provably unbiased low-variance technique based on carefully constructed control variates. Thanks to this technique, which gives an unbiased performance estimate with 85% reduction in standard deviation, we can show statistical significance in matches with as few as 3,000 games.

Abstraction-based Approaches

Despite using ideas from abstraction, DeepStack is fundamentally different from abstraction-based approaches, which compute and store a strategy prior to play. While DeepStack restricts the number of actions in its lookahead trees, it has no need for explicit abstraction as each re-solve starts from the actual public state, meaning DeepStack always perfectly understands the current situation.

Professional Matches

We evaluated DeepStack by playing it against a pool of professional poker players recruited by the International Federation of Poker. 44,852 games were played by 33 players from 17 countries. Eleven players completed the requested 3,000 games with DeepStack beating all but one by a statistically-significant margin. Over all games played, DeepStack outperformed players by over four standard deviations from zero.

Deep Stack Poker Room

Heuristic Search

At a conceptual level, DeepStack’s continual re-solving, “intuitive” local search and sparse lookahead trees describe heuristic search, which is responsible for many AI successes in perfect information games. Until DeepStack, no theoretically sound application of heuristic search was known in imperfect information games.

Deep Stack Poker Meaning