THE BOOK cover
The Unwritten Book is Finally Written!
An in-depth analysis of: The sacrifice bunt, batter/pitcher matchups, the intentional base on balls, optimizing a batting lineup, hot and cold streaks, clutch performance, platooning strategies, and much more.
Read Excerpts & Customer Reviews

Buy The Book from Amazon


SABR101 required reading if you enter this site. Check out the Sabermetric Wiki. And interesting baseball books.
MOST RECENT ARTICLES
MAIL : You ask | We say

Advanced


THE BOOK--Playing The Percentages In Baseball

<< Back to main

Wednesday, December 16, 2009

Forecasters Challenge 2010

By Tangotiger, 03:36 PM

I am once more issuing a challenge to all those who have a forecasting system to join the Forecasters Challenge 2010.  This time, rather than pitting everyone against each other in a huge league, I am going to put each one in a league of 21 amateurs, a Pro v Joes competition.  Where will I find these 21 amateurs?  I’ll tell you about that soon.  But it won’t be 21 amateurs.  It will be 100 leagues of 21 amateurs, so more like 2100 amateurs.  And I will pit the Pro in each of these 100 leagues, simply swapping out one Pro for another Pro in every draft.  This way, the Pros face the same amateur competition.  You can get a feel for this based on how I ran it in 2009:
http://tangotiger.net/forecast/prosjoes.html

Otherwise, the rules will remain pretty much the same:
http://tangotiger.net/forecast/intro.html

And if you wanted to see the results from the 2009 competition:
http://tangotiger.net/forecast/

So, if you are someone with a forecasting system, send me an email at tom~tangotiger~net, replacing the ~ as appropriate.  I’m going to limit it to 25 systems, and priority will go to those who participated in 2009.  So, first come first serve for the wait list.


#1          (see all posts) 2009/12/16 (Wed) @ 16:16

I’ll be interested to see how the Fans Scouting Report you have been helping out with stacks up against the Pros.


#2    Tangotiger      (see all posts) 2009/12/16 (Wed) @ 16:35

Steve, not sure what you are talking about?  Are you referring to the Fan Forecasts at Fangraphs?


#3    Xeifrank      (see all posts) 2009/12/16 (Wed) @ 17:07

Is there a difference between “forecasts” and “projections”?  One taking into consideration things like playing time, where a player bats in the lineup or how the manager uses them.  Opportunity stats like AB, SB, SV, RBI, Runs are all heavily affected by this.  If you are using stats like SB, Saves, RBIs, Runs etc..., I would think that using those stats introduces the skill/luck of predicting usage as a key to success in this contest.

I suppose everyone is in the same boat, unless people are allowed to send their lists in later than others.

ie - whether Chone Figgins bats 1st or 2nd has a big impact on the number of stolen bases he gets.

This seems like a fantasy baseball team draft contest, no?

vr, Xei


#4    StevenEll      (see all posts) 2009/12/16 (Wed) @ 17:48

Have you considered combining projection systems and seeing how that stacks up?  I just wonder if a combination of them will fare better than any of them individually.


#5    Xeifrank      (see all posts) 2009/12/16 (Wed) @ 17:57

#4. Do you mean combining all of them, or some of them, which ones?  If you combined all of them, I would think you’d be average.  The more you combine the more average it should be.

I think this more or list boils down to list making skill(s).
vr, Xei


#6    Tangotiger      (see all posts) 2009/12/16 (Wed) @ 18:06

Steven: actually if you read one of those links, I did exactly that.  And the consensus demolished the individual.  They do NOT come out as average, but far above average.


#7    Xeifrank      (see all posts) 2009/12/16 (Wed) @ 18:13

#6. That’s interesting that the consensus did so well, does one of your links go into detail behind why this might have been the case, or can you give a quick synopsis?

I’d like to participate, but my system does not “forecast” SB, Saves, Runs and RBIs, so I’d have to quickly make something up for those categories.

vr, Xei


#8    Wells      (see all posts) 2009/12/16 (Wed) @ 18:13

I’d love to take part if there’s a slot somewhere.


#9    Tangotiger      (see all posts) 2009/12/16 (Wed) @ 18:13

"I think this more or list boils down to list making skill(s). “

I don’t even know what that means.  How is that different than let’s say Baseball America’s top 100 prospects compared to BPro’s compared to Sporting News’ compared to Sickels’?

List-making skills is a good thing to have, but you make it seem like it’s something to sneer at.


#10    Tangotiger      (see all posts) 2009/12/16 (Wed) @ 18:16

Wells: send email as noted above…


#11          (see all posts) 2009/12/16 (Wed) @ 18:31

I’d like to see how Fangraphs’ Fan Projections do in this challenge too. As someone who participated last year, should I send you an email now to reserve a spot and send you my projections later when finished?


#12    Tangotiger      (see all posts) 2009/12/16 (Wed) @ 18:58

Yes, send me an email. You should have received an email from me this morning, as I contacted everyone who was in this last year.

As for Fangraphs… well, that’s my strategy for the “Joes”.  But, I have to talk to David about it first.  This could be huge.


#13    Toffer Peak      (see all posts) 2009/12/16 (Wed) @ 19:00

"That’s interesting that the consensus did so well, does one of your links go into detail behind why this might have been the case, or can you give a quick synopsis?”

This is why: http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds


#14    Tangotiger      (see all posts) 2009/12/16 (Wed) @ 20:19

Xei:

http://www.tangotiger.net/forecastFinal2.html


#15    Xeifrank      (see all posts) 2009/12/16 (Wed) @ 20:32

List making as in ranking strategies.  With the way the rules are setup, obviously you don’t just want to rank players from 1...N with all positions mixed together.

ie - The player with the 20th most points in my forecast (using your point method), should likely not be ranked 20th in my list.  Thus the “list making” comment (not a sneer).  Different ideas and opinions should be tolerated a little more and no question should be considered a dumb one (within reason).

This is interesting and I don’t fully understand how serious of an excercise this is suppose to be.  For example, if ZIPS wins does that mean their forecasting system is best?  Or maybe that they did the best job of ranking players?  Or both??

It may be that I am misunderstanding how this all works, but to me it seems like you make your forecast then feed them into your equation and get a value/points for each player, then send in a list to you ranked 1 to N based on that value/points.  Then you do a bunch of fantasy league drafts based off of everyone’s list with draft orders changing in each league etc… If that’s the case then this is a “list making” excercise.  Good forecasts are a must, but good list making techniques put you over the top.
vr, Xei


#16          (see all posts) 2009/12/16 (Wed) @ 21:13

Continuing on the idea of list-making skills...suppose I happen to have insider knowledge that Willie Bloomquist is going to produce like an MVP (maybe he’s been eating his Wheaties while wearing PF Flyers or something).  That doesn’t mean that I should put him high on my list...nobody else will have him there so I can get more value by taking him later.  It’s a non-trivial exercise to make up your list even if you had a crystal ball telling you their final numbers.  However, I’m not sure if that’s what we are interested in measuring…


#17    Tangotiger      (see all posts) 2009/12/16 (Wed) @ 21:17

Yes, this is exactly what we are interested, because that is the objective: who will do best in a draft.  That’s the question.

So, it is a “list-making” exercise.  Not just pure “forecasting”.

If you read my three links, I explain at the bottom of one of them why it’s important to understand the purpose of whatever “competition” you are doing, so that you can construct the correct exercise for it.


#18    Tangotiger      (see all posts) 2009/12/16 (Wed) @ 21:19

I will note that MGL did it purely as a forecasting exercise (basically, he fed me his forecasts, I applied the playing time, and replacement level for each position, and voila).

So, I don’t think the “list-making” skill is so important, like “I have Willie really high, but I figure no one has him that high, so let me drop him a little”.  Didn’t happen with MGL or ZiPS or Marcel, and they did fine.  I presume almost everybody did that.


Page 1 of 1 pages


Name (required)
E-Mail (optional; WILL be published)
Website (optional)

<< Back to main


Latest...

COMMENTS

Feb 12 00:40
Clutch analogy

Feb 12 00:38
Reader Mail of the Day: Why do we need X years of fielding data?  And what about outliers?

Feb 11 22:08
Who is Jeremy Lin?

Feb 11 20:11
Fighting leads to goals?

Feb 11 19:55
Why do players get crappy caps?

Feb 11 19:12
Hero of the month: Brittney Baxter

Feb 11 17:59
MGL: Today on Clubhouse Confidential

Feb 11 10:29
Dwight Evans

Feb 11 02:12
Performance through the ages

Feb 10 23:01
For Your Soul