from Amazon
Tuesday, November 29, 2011
The existence of outliers doesn’t preclude luck as the cause
Given enough trials, and given a large enough coins, and we can always find outliers… and the same applies to pitchers.
Using this fact, it follows that in our first year, if we have 100 pitchers, we expect half to outperform their FIP. This means that there are 50 players that outperformed their FIP in year one. Of those 50 players that outperformed their FIP in year one, we would expect 25 (.5*50) of them to outperform their FIP in year 2 by pure chance. Of those 25 players that outperformed their FIP in year two, we would expect 12.5 of them to outperform their FIP in year 3 by pure chance. Similarly, we can continue down this path halving the number from the year before. In year four, we would expect about 6 pitchers to have continued to outperform their FIP, and by year 5 we would expect just over 3 pitchers to have consistently outperformed their FIP by pure luck.
Because we started with 100 pitchers, we expect that about three of the pitchers would outperform their FIP in 5 consecutive years, by randomness alone. Many people point to those three pitchers and say, “Clearly, FIP is not accounting for something those three pitchers do.” We can now completely discount that argument for the “simulation”, because we have assumed FIP to be perfect. Thought experiments are nice because they easily allow you to comprehend and visualize a phenomenon, but there is not a lot to glean, if the experiment is completely incongruent with reality.
To put it simply, to “win” something 5 times in a row, just by pure luck, and you have a 50/50 chance of winning each time, then you will win five and lose zero a total of 0.5^5 = 3.1% of the time. Start with 100 coins or 100 pitchers, and you will flip heads, or beat your FIP, by flipping five times in a row, a total of 3 times.
And in reality?
giving us a not so unexpected final total of 3.6%
...
and finally 4% of the original starting pitcher group
This is not to say that FIP is perfect. But, just relying on the fact that you’ve been able to identify 3 or 4 extreme cases, when that’s exactly what you would have expected to find if it was all luck, doesn’t prove your point.
You need to find MORE than the expected extreme points, and NOT just “some” extreme points. Some extreme points means nothing, unless you know how many you expected to find by luck.