from Amazon
Friday, February 11, 2011
Testing the 2007-2010 Forecasting Systems - part 3 - head-to-head sample
Ok, I’m trying something on the head-to-head. For this post, I’ll show you what I’m doing, and then you guys can let me know what you think. In all cases, this is SystemX v Marcel.
First up, I’m looking at n=2490 for 2007-2010. I have 4 results of head-to-head matchup.
1. Crap: each system was at least .040 (40 wOBA points) away from the actual wOBA. So, regardless if one system was closer by 45 and the other by 75, I count them both as “Crap”. This corresponds to roughly 20 runs in a full season, so this means the difference between average and replacement.
2. Tie: the gap between the two, relative to the actual, was less than 10 wOBA points. So, if the actual was .350, and one said .345 and the other said .353, then I’m calling it too close to call, hence a draw. 10 wOBA points is roughly 5 runs in a full season.
3. Loss: SystemX was further from the actual than Marcel was (as long as it was neither a Crap or Tie)
4. Win: SystemX was closer (and not Crap nor Tie)
The result on n=2490
811: Crap
970: Tie
340: Lose
369: Win
Counting a win as “1”, the crap and tie as “0.5”, and the lose as “0”, the SystemX win% is .506.
Now, weighting by PA, so each PA is one “game” (PA=711,647)
119521: Crap
379893: Tie
104224: Lose
108009: Win
SystemX win% is .503
The Crap rate was 17%, the Tie rate was 53%, the outright Lose rate was 14.6%, and the outright Win rate was 15.2%.
***
Now, I then broke it up between whether each system forecasted a player or not.
1. SystemX and Marcel both forecasted the players. SystemX win% = .507 (PA=658,046)
2. Marcel forecasted, but SystemX did not (I give these players a .315 wOBA across the board). SystemX win% = .478 (PA=11,690)
3. SystemX forecasted, but does not appear in Marcel file (I give these players a .335 wOBA across the board). SystemX win% = .435 !! (PA=36,774)
4. Neither forecasted. (I give Marcel .335 and SystemX .315) SystemX win% = .474 (PA=5,137)
***
Comments? Surprised?
Recent comments
Older comments
Page 1 of 344 pages 1 2 3 > Last »Complete Archive – By Category
Complete Archive – By Date