from Amazon
Friday, January 28, 2011
Rating the 2010 projections once again
As promised in the “Sierra evaluation” thread, I am going to evaluate the following projection systems:
Cairo
Chone
Marcel
Steamer
Zips
MGL
If Brian or Tango can send me the Oliver 2010 pre-season batter and pitcher projections, I can include them too. I am going to start with pitchers only.
I simply computed the average error for ERA and FIP weighted by the number of 2010 IP for each pitcher. Some of the above projection systems had an FIP component. Others I had to compute it using the standard formula, (hr * 13 + (bb+hp-ibb) * 3 - so * 2) / ip + constant (around 3.something).
Some projection systems did not have HP or IBB, so I assumed that BB always included IBB and I subtracted out a few of the BB and I added in some HP, using league average proportions.
Most importantly, I scaled everyone’s FIP and ERA to their own total league FIP and ERA. So, for example, if Chone’s NL ERA weighted by number of actual IP was 4.00 and Cairo’s was 4.50. I subtracted .5 from all of the Cairo pitchers, as compared to Chone. That is not necessarily the fair or correct way to do it, but that way no one gets penalized for having a high or low total league ERA or FIP.
There were 451 pitchers who were projected by all 6 systems. For some reason, Cairo did not have a lot of pitchers that the other systems had. I am using Cairo v. 1. If someone can send me a later (but still pre-season) version, I will use that one instead.
Here are the results in order:
ERA
System Average error
Steamer .827
Chone .837
Marcel .856
Zips .857
MGL .860
Cairo .873
FIP
System Average error
Steamer .571
Chone .579
Marcel .589
Zips .603
MGL .612
Cairo .619
If we remove Cairo, we get another 50 pitchers (531) and the results for the remaining 5 systems look like this:
ERA
System Average error
Steamer .901
Chone .905
Marcel .925
Zips .929
MGL .944
FIP
System Average error
Steamer .629
Chone .631
Marcel .642
Zips .661
MGL .676
If we remove all pitchers with less than 100 IP, we get this (N=135):
ERA
System Average error
Steamer .668
Marcel .672
MGL .681
Zips .686
Chone .688
Cairo .702
FIP
System Average error
Steamer .438
Marcel .452
Chone .455
Zips .467
MGL .493
Cairo .494
Perhaps tomorrow I will do the batters. This is more work than I anticipated, especially since there is not one common set of ID’s for these projections.
Recent comments
Older comments
Page 1 of 344 pages 1 2 3 > Last »Complete Archive – By Category
Complete Archive – By Date