THE BOOK cover
The Unwritten Book is Finally Written!
An in-depth analysis of: The sacrifice bunt, batter/pitcher matchups, the intentional base on balls, optimizing a batting lineup, hot and cold streaks, clutch performance, platooning strategies, and much more.
Read Excerpts & Customer Reviews

Buy The Book from Amazon


SABR101 required reading if you enter this site. Check out the Sabermetric Wiki. And interesting baseball books.
MOST RECENT ARTICLES
MAIL : You ask | We say

Advanced


THE BOOK--Playing The Percentages In Baseball

<< Back to main

Wednesday, October 06, 2010

Forecasters Challenge 2010 - Results!

By Tangotiger, 10:17 AM

Marcel did extremely well.  And in one of the four competitions, it actually won.  But, Marcel did not win the official competition.  Who did?  The same guy who won last year: Eric Hanson (who posts here as JEH).  Chone had an extremely strong showing as well.  Full results here:

http://tangotiger.net/forecast/results2010.html


#1    Rudy Gamble      (see all posts) 2010/10/06 (Wed) @ 11:31

Tango -
Thanks for all your work on this challenge!  Nice to see our Razzball entry finished in the top half this year.

Three quick comments:
1) On the ‘Official Competition’, Razzball is in 8th place for Value but is 5th in points_ct (ahead of 3rd place Marcel and 5th place CHONE). 

So would you attribute CHONE and Marcel’s lower point totals vs. Razzball but higher values (assuming because of more wins and/or 2nd-6th place finishes) to being better or to luck?

2) How come ZiPS isn’t part of the contest?

3) What was the playing time source for CHONE and MARCEL?  Will you be doing the analysis like last year for determining the best source of playing time estimates.  I stuck with BaseballProspectus again this year but will switch to Fantistics or FanGraphs community if they prove better again.

Thanks!
Rudy


#2    Tangotiger      (see all posts) 2010/10/06 (Wed) @ 11:51

I used these as Playing Time:
http://www.tangotiger.net/survey/

(See last link for all players.)

I know Chone used either mine or Fangraphs.  There was huge agreement between Fangraphs and mine, so either would have worked out the same I guess. 

I really can’t imagine NOT using the crowd for playing time.  I mean, those guys are so in tune with their baseball teams, that they know all the ins-and-outs and shifts and depth charts.  Why in the world would I spent 2 or 4 hours trying to figure out anything, and trying to do that every day, when with just the few minutes it takes me to create the survey, I get everyone else to do my job?  This is like the best possible crowd-sourcing project possible.

I *love* the crowd.

***

ZiPS, MGL, PECOTA, Oliver all had some issues with their ballots, or expected ballots. 

ZiPS/MGL only provided forecasts, not lists, and those forecasts were missing key things (saves or rbis) that I couldn’t try to convert that to a list.  I mean I COULD, but I didn’t want to spend more time on this like I did last year.

PECOTA had some serious ID mapping issues.  And those couldn’t get resolved in time.

Oliver never had a ballot submitted, and I think Brian said he had his time limits.

***

As for the details of wins and points, I wouldn’t try to read much into it.  After all, the different competition setups resulted in different orders, and, I think it’s too nuanced to try to figure out why the wins and points didn’t match up, and whether it was luck or not.


#3    Tangotiger      (see all posts) 2010/10/06 (Wed) @ 11:55

I think all you readers should DEMAND that whatever system you are considering using or buying should enter this project.

And, as you can see, I’m open to trying out different setups.  The only constraint is that everyone submit an ordered list by MLBAM ID.  After that, I have no contraints.  I’ll code whatever it is that people want me to code.

I’m all for learning more, rather than looking to crown anyone.  Though, seeing Hanson win this thing twice is definitely interesting.  I think getting the right depth charts is rule #1.


#4    Rudy Gamble      (see all posts) 2010/10/06 (Wed) @ 12:54

Yes, I like the crowdsourcing too.  Only challenge with it is timing - our site visitors are generally barking for rankings in late February for March drafting where I imagine community forecasts don’t finish up until near opening day. 

And it would be great if all the other projection systems participated.  It doesn’t take much work on their end.


#5          (see all posts) 2010/10/06 (Wed) @ 14:24

PECOTA had some serious ID mapping issues.  And those couldn’t get resolved in time.

Would it be possible to resolve those now and see how PECOTA ranked with the others?

If it’s feasible and simply a matter of somebody spending sometime unraveling the ID mapping, I’d be willing to try to tackle that.


#6    Tangotiger      (see all posts) 2010/10/06 (Wed) @ 14:33

Right, last March I asked if someone wanted to act as a trustee to one of the systems, I’d be happy to hand that off. 

For example, Dan (ZiPS) and MGL gave me their forecasts.  The trustee would come in, fill in the blanks (Saves for Dan, RBIs, Saves, Wins for MGL), calculate the points, map to MLBAM and give me an ordered list.  And, I did that for several systems last year (2009).  I took great care in doing this.  For example, to estimate RBIs, I would use a player’s historical RBI-HR per Hit, and then apply that to MGL’s forecasted number of hits, and add in HR to get an RBI estimate.  (Actually, since MGL didn’t forecast playing time, I would first need to get the community numbers.) I did some sort of trustee actions for five systems last year.  This year, I decided that I would only do one (Fangraphs Community).

Anyway, no one stepped up for the four systems I mentioned.  If someone wants to do it, I have the submissions from MGL, ZiPS, and PECOTA on my flash drive.

Rules are here:
http://www.tangotiger.net/forecast/rules.html


#7    Wells      (see all posts) 2010/10/06 (Wed) @ 14:43

Damn! There’s always next year.

Does Hanson share his projections publicly?


#8          (see all posts) 2010/10/06 (Wed) @ 14:45

I’m not interested in doing the work for ZiPS or MGL, but I’d be willing to work on PECOTA if you would send it to me.


#9    Tangotiger      (see all posts) 2010/10/06 (Wed) @ 14:54

Ok, send me an email at tom~tangotiger~net

Hanson: he said it would be posted at Fangraphs.  So, Fangraphs is going to host Hanson (tentatively called Slapshot), Chone, and Marcel, as well as ZiPS, Community, and Bill James.


#10          (see all posts) 2010/10/06 (Wed) @ 20:28

Tango,
Thanks for doing this.  Also, it’s great to hear that Hanson/Slapshot will be at Fangraphs. 

It would be interesting to participate in a fantasy league in which every owner uses the same forecasts or even the same dollar values.  Then differences in draft strategy, plus trades, injuries, and in-season add/drop moves would determine success (plus luck).


#11    J. Cross      (see all posts) 2010/10/06 (Wed) @ 22:42

Yes, thanks for doing this Tango.  Peter and Dash, although now in college, are set on making improvements to Steamer and coming back strong in 2011.  In Steamer’s defense, the list was optimized for the official competition (making sure not to draft guys sooner than we’d need to in order to pick them before the typical fan) which may not have worked as well in other match-ups.

I’ll be doing the same analysis as last year to try to see where the strengths and weaknesses of each system lie.  I should have chone, marcel, zips, pecota, steamer, oliver, cairo, baseballhq, cbssportsline, Bill James, MGL and fantistics included.

JEH, if you’re up for sending them I’d love to get Slapshot in the mix as well.  That goes Razzball and all other projectors as well.  Projections lined up with MLBAMs are much preferred, of course.  Hopefully, including MLBAMs with projections becomes the industry standard.


#12    J. Cross      (see all posts) 2010/10/06 (Wed) @ 22:49

Note: I have ZiPS lined up with MLBAMs if that’s helpful to anyone.

Mike Fast, if you get pecota lined up with MLBAM’s and are willing to send them to me I’d be much obliged.


#13    JEH      (see all posts) 2010/10/06 (Wed) @ 23:18

@11/J.Cross

I don’t have a set of projections that I can guarantee predate Opening Day or you would be welcome to them. 

I do plan on sharing my 2011 projections on Fangraphs, so they’ll be available for a postmortem next fall.


#14    Rudy Gamble      (see all posts) 2010/10/07 (Thu) @ 09:43

J. Cross -
I could send over the projections for Razzball.  Do you want the actual projections for each hitting/pitching category or just the projected points used in Tango’s contest?

Also, what’s your e-mail address?


#15          (see all posts) 2010/10/07 (Thu) @ 10:22

Rudy, that’s excellent, the actual projections.  I’m going to see who had the most success projecting playing time and each rate stat. 

jfcross4~gmail~com


Page 1 of 1 pages


Name (required)
E-Mail (optional; WILL be published)
Website (optional)

<< Back to main


Latest...

COMMENTS

May 25 06:43
Largest demonstration in Canadian history?

May 25 06:39
Lack of hustle during a game

May 25 05:00
Help needed with sticky issue…

May 25 02:38
NFLPA lawsuit against collusion

May 25 01:43
Neal Huntington’s best moves

May 24 23:50
Rooting for laundry

May 24 17:04
Firefox, IE, or Chrome?

May 24 12:07
How to beat the shift

May 24 11:11
Incredible story

May 24 09:41
Racial bias in card collecting: not the collectors, but the players on the cards