Small Samples: The Root of All Poker Evil
In live poker, we have to find our edge quickly. If we are making 5bb/100, a very solid online winrate, we are going to have to be playing 5/10 or higher just to make minimum wage in a live game. The reason we even bother to play the live game is because opponents are making such massive mistakes that our winrate can be far closer to 30bb/100 or higher, even at a 9-handed table.
This reality creates an urgency that separates the grinders from the crushers. Where the grinder executes their strategy and slowly but surely accumulates chips, the crusher is always seeking max value, losing the minimum, and firing just the right number of barrels. While most grinder types understand this difference, making the leap is easier said than done.
In fact, there is one main culprit that separates the crushers from the grinders, and it often determines who is who in the first place. It rears its ugly head in strategy decisions, mental game issues, game selection, and just about every other aspect of live poker.
I’m talking about small sample size. Just like the speed of the game necessitates a huge winrate, it also severely limits the amount of information we can procure, clouds our decision making, obscures our true winrate, and gives us conscious and unconscious bias. So what can we do about it?
Let’s skip to the final stage of grief and accept small sample sizes into our live-poker life. It’s not going anywhere. We can mitigate its effects by putting in more volume, moving up in stakes and playing against a smaller pool of players, or playing a less exploitative strategy. However, those are not always possible or optimal.
So what does it mean to welcome small sample sizes into our lives? It demands that we live with a heightened level of doubt or uncertainty in our decisions. It’s important to understand that doubt and uncertainty are not negatives but realities in our decisions. The only negatives in poker are making the wrong decisions. Having a tool like uncertainty to help us make improved decisions is, in fact, a positive.
Quantifying Uncertainty in Strategy
I wrote extensively about quantifying a read in another article. That article explains what we can make of information gleaned from a small sample size in terms of poker strategy. In this article, I want to talk about quantifying doubt in other areas of poker.
I think the most difficult and perhaps most important place to understand the role of uncertainty is in our results. This manifests itself in many different ways. For one, imagine we decide we aren’t 3-betting enough for value. We decide to 3-bet TT OTB against a CO open. We get flatted by QQ and barrel off on a 95335 board. Did we make a mistake? Maybe, but this hand hardly contributes to the answer either way. We normally refer to this as being results-oriented, but often when we say we are being results-oriented, we mean we are drawing on too small of a sample to make an accurate evaluation. Results matter, but we don’t have enough to make our decision.
Maybe we shake off the TT hand and chalk it up to run bad. But then we have two losing sessions in a row after a long winning streak right as we made this conscious change. The two events could be totally independent, but how likely are we to give up? In live poker, you will rarely ever be able to evaluate your strategy based on the results of your play. Still, those extra 3-bets were going to be marginal either way. A lot of the time, going from good play to great play means turning a +1bb call into a +2bb 3-bet. This might increase variance and lead to the greater likelihood of losing a big pot, but it’s still the right play.
We are always faced with uncertainty due to small sample size, not to mention the difficulty of tracking live results in a meaningful way, we will have to resort to other methods of evaluating our play. I’m talking about range analysis and other math work away from the table: the detail-oriented active study that you've probably found excuses to skip over at one point or another. For a better idea of what I’m getting at, check out our podcast breakdowns, complete software aided analyses of each podcast episode.
Uncertainty with Longterm Results
Beyond individual strategic decisions, our whole winrate is always going to fall into the category of “very uncertain”. I’ve seen estimates of a significant sample of live play ranging from 2000-10,000 hours. Even on the low-end, playing 40 hrs a week, we won’t reach a meaningful sample until a year. That’s five years if you believe the people advocating 10,000 hours. Now that sample isn’t going to be very consistent either. Do you play at the same place in the same games against the same people 2000 hours a year? If you can do that and make a good living, power to you. For me, living in New York City, my array of online play, home games, and trips to varying casinos across the country adds a lot of noise to my results. The only reason I'm sure I'm a winning player is that I can identify spots where I believe my opponents are making mathematical mistakes that I can demonstrate on pencil and paper - but more likely a computer.
The truth is, you might not be a winner in your game. How many poker players have we seen assume they are a winner based on past results and consequently dump it all to the people who put in the work? Just winning isn’t enough. You need to know why you are winning. And if you are losing, you need to know why you won’t be losing in the long run if you maintain your strategy.
Even if you have been a winner in your game, there is no reason you should count on your winrate being the same moving forward. The games could improve, get much worse, or end altogether. The point is, be prepared and be realistic. This isn’t to say you aren’t a great poker player. I’m sure you would be raking in the millions if you have the bankroll for the big game. Just be careful.
A big part of undoing the damage caused by small samples is identifying personal biases. Do you have any poker dogma? Maybe you never flat 3-bets OOP with medium pocket pairs. Is that because it’s a losing play, or because you haven’t hit a set in you past 10 attempts and the one time you called down on a good runout your opponent had AA? Whenever you hold a belief like this, make sure you test out the math and make sure you haven’t been clouded by recent results.
In my opinion, identifying personal bias based on small sample sizes and replacing that bias with mathematically sound strategy is basically the process we call leak-finding. When Zach and I do live leakfinders, we constantly run into this phenomenon. A lot of times, it takes someone else to point out these personal biases. Talking through hands with friends or on forums is an excellent way to expose these. For example, I realized from talking through some preflop spots at a recent trip to Parx with Zach that I had a preference towards flatting 3-bets in position rather than 4-betting since I have had good results playing in position. That’s not to say there isn’t mathematical reasoning for this. However, I let the fact that the math supports me in most cases cloud my judgement in a few individual cases where a 4-bet would have been best. But man, I do love flatting 3-bets in position...
Personal Consequences of Small Samples
I actually wrote this article due the aforementioned Parx trip. I had always heard excellent things about the quality of 2/5 games at Parx. About 4 months ago, I was in New York City for the weekend and was planning a stop at Parx on my way back to Cleveland. I would have normally opted to head to Maryland live on this trip, but Parx was more on the way and I wanted to see what the hype was about.
I left New York at 2 PM and expected to arrive around 4. It was a Friday, so traffic was a factor, but I was leaving early enough to beat it. Several wrong turns, a bathroom stop turned hour-long detour, and some general traffic run-bad kept me on the road until about 7. No big deal, I was ready to grind.
I sat down at my first table and promptly got smacked for about 1500. My table wasn’t great, and none of the other 2/5 looked particularly appealing. There were actually only 4 2/5 tables going on a Friday night. I didn’t get what the fuss was about. Being in a thoroughly poor mood, I decided to cut my losses and drive home. I hadn’t booked a hotel, and just wanted to get home.
Cut to two weeks ago. Zach and I were both living in New York City (I still am, he has moved on…), and decided to take our first extended poker trip since moving here after a couple MGM trips fell through. For a few we reasons, we decide to head to Parx. My expectations were wildly exceeded by the action. My tables were all fantastic. I even played with some of the same people as my first session who turned out to be much worse than I’d previously thought.
My small sample from my first trip to Parx clouded my judgement in a few ways. I felt that the action wasn’t great (wrong), the my specific opponents were decent (wrong), and that making a longer drive down to DC would be well worth the extra effort (maybe right, but not as close). I definitely regret not going back sooner. I actually think my sample from this recent session was probably too positive. Another potential sample size bias. Still, I think my new sample is much more representative and I’m accepting a certain amount of doubt into my future decisions regarding playing at Parx. For example, I am not taking good action on a Saturday as a guarantee for good action on a Wednesday.
I could easily go on and on about personal confrontations with small sample biases. I’m sure for every bias I have identified, there are others hidden. However, I do believe that my awareness of these possibilities and my acceptance of the necessary amount of doubt gives me a better ability to make better decisions at the table and in life. As you continue to play and study poker, always be on the look out for small sample sizes creating personal biases that affect your decision making. As you replace those biases with mathematically sound logic, your edge will significantly increase and the transition from grinder to crusher will commence.