Friday, July 28, 2006
Spread in Talent
How much has the spread in talent changed over the years, among the league’s regular players?
Here’s a step-by-step process as to what I did, and what it shows:
1 - Start with the Lahman database, and exclude all pitchers. That leaves me with 70,760 player seasons.
2 - Sort them by PA, descending order, and select a number of players equal to the number of regulars for each season. (That’s 8 players per season, with 9 for the AL-DH seasons.) This leaves me with 20,494 player-seasons.
3 - For each season, calculate the standard deviation of the OBP, if all players in the pool were equals. This is your random variation.
4 - For each player season, calculate his z-score, as the number of standard deviations he is from the mean. The career leader is Bonds, 2004, with a z-score of 13.5.
5 - For each season, calculate the standard deviation of all the players’ z-scores.
If a league has a z-score of 1.00, that means that the distribution of the observed OBP exactly equals what one expected if all players were equals. This has never happened.
6 - Finally, you multiply the z-score by the random standard deviation for the OBP to get the true distribution of OBP.
The tightest distribution in OBP talent was 1978, where one standard deviation = .025 OBP. That is, we expect that in 1978, 95% of all the regular players had a true talent rate within .050 points of the mean. The widest distribution, ignoring the early years, was 1923, with 1 SD = .042.
The 1978 may have been an anomoly, or perhaps it was at the peak of speed demons. That is, if the true OBP spread was that tight, perhaps the true speed spread was very wide, so that, overall, the speed+OBP component was normal.
Then came 2005. The true OBP spread is .027, the tightest it’s been since 1978. This is a huge dropoff of the .033 of the previous season. It is in fact the largest dropoff in OBP talent in modern baseball history.
This is not 1978, and therefore, we can’t say that the shift has gone to speed. I doubt that the shift has gone to power. When I remove Barry Bonds, the dropoff disappears. The talent level is still the tightest its been since 1978, but now the tightening has been more gradual, from .035 in 2000 to .027 in 2005.
The problem now, with the talent level of the players being so tight, is that it’s hard to figure out who is better than whom. In the past we just needed 200 PA from a hitter, to regress his OBP 50% towards the league mean. We now need 300 PA to be as certain.
Next time, I’ll look at other components, like speed and power, to see what kind of shift there’s been in spread in talent. And then pitchers.
Another way to look at this is the spread in talent for a fixed number of players. Now remember, what I did earlier was based on the number of teams, so in effect what I was giving you was the spread in talent per team.
Now, I’m going to take a fixed number of players, 200, and tell you how much the spread in talent has changed in that group.
From the early 1900s through until WWII, the spread in true OBP among the top 200 players in PA, was in the .040-.045 range. From that point onwards, until 1958, it’s been pretty flat at .035-.040. Since then, until around 1980 it’s been in the .030-.035 range.
Afte that point, the spread in talent among the 200 regular players has been increasing a bit every year, with a sustained peak from 1993-2002.
It is important to note that I did not control for park factors, and introducing Coors into a study certainly requires one to do so. Coincidentally, since the PED years happens to coincide with this time period as well, it certainly bears further investigation.
That is, while we expected a tightening in talent levels, as the years go by, since we are drawing from more players, and more people want to be ballplayers, and we are always looking at 200 players, we instead find a reversal in the last 10-15 years. (A reversal that seems to have unreversed itself suddenly in the last couple of years). Whether that has to do with PED, Coors, all the new parks, or a different selection process for players, more work needs to be done…