THE BOOK--Playing The Percentages In Baseball

Tuesday, May 19, 2009

Forecasters Challenge 2009 - update

Just a little update for those following this.  I think this has some good potential for future research as well.  You’ll see what I mean, either here or in future posts.  First off, here are the 22 participants, who I must thank very much for coming through.  There were a few no-shows, so I had alternates ready to step in and they did:

FAN_ID FAN_TX
101 ANONYMOUS1
102 Ask Rotoman
105 Brad Null
106 CAIRO
108 Chone
109 Christopher Gueringer
110 Cory Schwartz
111 Fantasy Scope
112 FantasyPros911
113 FeinSports.com
115 John Eric Hanson
116 KFFL
119 PECOTA
120 Razzball
121 RotoExperts
122 RotoWorld
203 MGL
204 Baseball Primer (ZiPS)
207 CBS (proxy)
214 Hardball Times
217 Marcel
218 Steamer

Those in bold are simply those who readers of this blog would be most familiar with.  Those people whose “FAN_ID” starts with a “2” means that the submissions had to be “massaged”.  For example, ZiPS used the playing time forecasts of the Community (as did Marcel).  MGL used the R/H and RBI/H of Marcel, in addition to the community playing time forecast (at least substantially).  The CBS ones had to be mapped to the proper ID, the only one I accepted in that state.

Anyway, so I ran 1000 automated drafts, based on the draft list of the 22 participants.  For example, Marcel picked Matt Holliday 457 times.  There’s a clear reason for this: Marcel doesn’t know about parks, and so, clearly, all the other participants dropped him down because of his move from Colorado to Oakland.  Marcel didn’t make the adjustment.  Marcel had him #5 overall on his depth chart, and so ended up picking him 457 times out of 1000. 

On the flip-side, Marcel never selected CC Sabathia, even though he had him ranked #8 overall!  That’s because seven other participants had him ranked higher, and 6 of those 7 guys ended up picking Sabathia almost evenly.

You even had occasions where some players were select 1000 times out of 1000.  For Marcel, that was Mike Aviles (ranked 38th!), Armando Galarraga (#109), and Jesse Litch (#112).  It wasn’t just Marcel though.  Almost everyone had one of these “1000 percent” picks.  PECOTA/Clay for example had Edwin Encarnacion (#38).  How about “Ask Rotoman” with Zach Greinke 1000 times (ranked #34 overall).

Some forecasters simply must have made mistakes in their ordering.  CAIRO had five 1000-percenters ranked in his top 50 picks (Verlander, Zambrano, Bradley, Buehrle, Blanton).  The only way for this to happen is for CAIRO to have ranked these players so far above the other 21 forecasters, that it was never close as to CAIRO ended up with all of these guys.

Anyway, I’m just fascinated by the results of the drafts, having expected that every player would have made up the roster of each of the 22 participants at some point, and often enough.  With 1000 drafts among 22 participants, I would have expected each player to have been drafted around 20-100 times for each participant.  Instead, the range was 0-1000.  Furthermore, with 550 players in each draft, I’d have expected that some 600-700 unique players would have been drafted in the 1000 drafts.  As it turns out, it was 623 unique players.  So, it doesn’t matter if someone advertises that they rank 2000 players.  They may as well rank 2 million.  All you need is about 10% more than the number of players drafted in your pool.

Anyway, website forthcoming, as soon as I can code it.  Should be fun, as long as I can figure out how to show the results of 1000 drafts in some sane way.