Estimating Minor League Stabilization Rates

How many plate appearances do you need to see before you can trust a stat?

“Is this player for real?”

A common issue for baseball fans, and sports fans in general, is figuring out if the stats you see in small sample sizes are indicative of the true talent level of a player. If a previously unknown hitter hits three home runs in his first three games, it can be tempting to drop everything and add this player to your fantasy team as quickly as possible, even if the sample is small.

Figuring out how best to analyze small bits of data is difficult. Thankfully, the topic of “stabilization rates,” which try to estimate how long it takes for a player’s skill to outweigh randomness, have been studied for more than a decade. It’s hard to give a full literature review on the subject, but there have been multiple different ways to solve this issue. Russell Carleton of Baseball Prospectus has been the leading voice on the topic, and a good primer on sample size and stabilization rates can be found here. (Though it hasn’t been updated in about six years.)

In this article, I’ll extend stabilization rates to the minor leagues, so that we can get a better idea of how long it takes for a minor leaguer’s stats to normalize, and when it is justifiable to get excited or discouraged by a prospect’s stat line. In order to do this, I used the padding methodology¹ for estimating stabilization rates, which I first heard of through Kostya Medvedovsky’s blog post, and has been used by other prominent sabermetricians like Tom Tango. The padding method works by taking a player’s current rate stat, such as strikeout rate, and adding in some measure of league average performance. This is predictive in nature, meaning that this methodology can be used to estimate future performance, not just evaluate what has already happened. The formula is shown below:

Here, S is the raw number of times an event happened (say, how many times a hitter struck out), PA is the number of plate appearances, and X is the number of “padding” plate appearances. When X is low, that means that a stat “stabilizes” quickly; when X is high, a stat takes many more plate appearances to stabilize.

How is X estimated? Let’s use strikeout rate in an example. First, for each player I calculated their strikeout percentage after each and every game they played in the sample period. Then, I calculated the “naïve” stabilized rate, which means that I used X = 1. After that, I found the difference between the naïve stabilized strikeout rate and the player's year-end strikeout rate. Finally, I used the default optimizer in R to select the correct value of X that minimizes the root mean squared error between the stabilized rate and their rest-of-year performance.