from Amazon
Friday, July 28, 2006
Spread in Talent
How much has the spread in talent changed over the years, among the league’s regular players?
Here’s a step-by-step process as to what I did, and what it shows:
1 - Start with the Lahman database, and exclude all pitchers. That leaves me with 70,760 player seasons.
2 - Sort them by PA, descending order, and select a number of players equal to the number of regulars for each season. (That’s 8 players per season, with 9 for the AL-DH seasons.) This leaves me with 20,494 player-seasons.
3 - For each season, calculate the standard deviation of the OBP, if all players in the pool were equals. This is your random variation.
4 - For each player season, calculate his z-score, as the number of standard deviations he is from the mean. The career leader is Bonds, 2004, with a z-score of 13.5.
5 - For each season, calculate the standard deviation of all the players’ z-scores.
If a league has a z-score of 1.00, that means that the distribution of the observed OBP exactly equals what one expected if all players were equals. This has never happened.
6 - Finally, you multiply the z-score by the random standard deviation for the OBP to get the true distribution of OBP.
The tightest distribution in OBP talent was 1978, where one standard deviation = .025 OBP. That is, we expect that in 1978, 95% of all the regular players had a true talent rate within .050 points of the mean. The widest distribution, ignoring the early years, was 1923, with 1 SD = .042.
The 1978 may have been an anomoly, or perhaps it was at the peak of speed demons. That is, if the true OBP spread was that tight, perhaps the true speed spread was very wide, so that, overall, the speed+OBP component was normal.
Then came 2005. The true OBP spread is .027, the tightest it’s been since 1978. This is a huge dropoff of the .033 of the previous season. It is in fact the largest dropoff in OBP talent in modern baseball history.
This is not 1978, and therefore, we can’t say that the shift has gone to speed. I doubt that the shift has gone to power. When I remove Barry Bonds, the dropoff disappears. The talent level is still the tightest its been since 1978, but now the tightening has been more gradual, from .035 in 2000 to .027 in 2005.
The problem now, with the talent level of the players being so tight, is that it’s hard to figure out who is better than whom. In the past we just needed 200 PA from a hitter, to regress his OBP 50% towards the league mean. We now need 300 PA to be as certain.
Next time, I’ll look at other components, like speed and power, to see what kind of shift there’s been in spread in talent. And then pitchers.