THE BOOK cover
The Unwritten Book is Finally Written!
An in-depth analysis of: The sacrifice bunt, batter/pitcher matchups, the intentional base on balls, optimizing a batting lineup, hot and cold streaks, clutch performance, platooning strategies, and much more.
Read Excerpts & Customer Reviews

Buy The Book from Amazon


SABR101 required reading if you enter this site. Check out the Sabermetric Wiki. And interesting baseball books.
MOST RECENT ARTICLES
MAIL : You ask | We say

Advanced


THE BOOK--Playing The Percentages In Baseball

<< Back to main

Sunday, March 07, 2010

A new way of showing forecast results

By Tangotiger, 04:56 PM

Well, I gotta say: great job.  Even though Chone won 52% of the head-to-head matchups with Marcel, Jared is showing Chone with a 93-69 (.576) record because:

How likely is it that Chone was really no better than Marcel this year and simply got lucky?  Just as likely as it is that an 93-69 team was really an average team that got lucky.  Not at all likely.

This is a really cool way to show it.  And even though Steamer did better than PECOTA against Marcel, Jared is showing his own system as performing worse because:

The reason for it’s inferior record is that Steamer’s projections were considerably more similar to Marcel’s and therefore, given that it finished below Marcel, we can be more confident in rejecting the hypothesis that it’s just as good as Marcel.

I love the thinking behind it.


#1    JB H      (see all posts) 2010/03/07 (Sun) @ 20:15

Kind of surprising to me just how bad the previous year’s stats did


#2    J. Cross      (see all posts) 2010/03/07 (Sun) @ 20:56

Thanks, Tango.

Pitchers soon.  Teaser: ZiPS seems to know it’s pitching.


#3    Colin Wyers      (see all posts) 2010/03/07 (Sun) @ 21:17

What is being done in the case where PECOTA (or CHONE, or ZiPS) forecasts a player when Marcels does not?


#4    J. Cross      (see all posts) 2010/03/07 (Sun) @ 21:58

For hitters such players were just dropped from the sample but for pitchers I’m going to look at a wider group.  Marcel gives a default (league average) prediction for everyone who doesn’t have prior major league stats.


#5    Colin Wyers      (see all posts) 2010/03/07 (Sun) @ 22:14

I think you will see an improvement in PECOTA’s results if you do the same (predict the league average) for any player not forecasted by Marcels; at least, that’s how it appears when I ran a similar test (using a slightly different sample of players).

I would presume that CHONE and ZiPS show similar improvement under the same conditions, but I haven’t finished ID mapping them yet and so I don’t have those results yet.


#6          (see all posts) 2010/03/07 (Sun) @ 22:44

That makes sense.

yeah, the ID mapping is a pain in the neck.  Colin, I’d be happy to send you the 512 pitchers I have lined up (with pecota, marcel, zips, oliver, cairo, steamer, chone, fantistics and sporting news) but it might not have all the players/stats you’re looking for.  I’m going to try to get everyone to send me their 2010 projections *with* MLBAMIDs to make this really quick and easy for next year.


#7    Rally      (see all posts) 2010/03/07 (Sun) @ 23:35

You should keep rookies in the test and assume a league average projection for Marcel.  In some cases Marcel will still win (Wieters).

Actually, Marcel may not do too bad in a lot of cases.  Almost all minor leaguers should be projected as below average players.  If they weren’t, then they wouldn’t be minor leaguers, would they?  But the ones who are allowed to play in the majors will be a lot closer to average, so the way this test selectively samples helps Marcel out.

The real value of a projection system that uses minor league data is to tell you beforehand which players deserve a chance to play in the majors.


#8    Colin Wyers      (see all posts) 2010/03/08 (Mon) @ 00:00

Rally, so long as you’re around, do you have a copy of the ‘09 CHONE projections with IDs included? The copy of the ‘09 hitter projections I have lacks IDs.


#9    Tangotiger      (see all posts) 2010/03/08 (Mon) @ 10:34

What is being done in the case where PECOTA (or CHONE, or ZiPS) forecasts a player when Marcels does not?

Marcel actually forecasts MORE players than any system out there.  As per the FAQ:

FAQ: “But, what about a player who’s never played MLB? Where’s his forecast?” That’s simple. His forecast is the league mean over 200 PA, 60 IP (starter) or 25 IP (reliever). If you want to know what the league mean is, just take the average of anyone forecast with a reliability of 0.00. So, Marcel’s official forecast for anyone coming over from Japan is that.


#10    Rally      (see all posts) 2010/03/08 (Mon) @ 10:59

I really wish Marcel was accurate for me, I’d be worth 6-8 million bucks to a team.  Not a criticism at all, just my preference for being on a ballfield instead of a cubicle.

Colin, I’ll see what I have.  I used my own ID system last year since I was matching things by player name.  Went to MLBAM this year.


#11    Rally      (see all posts) 2010/03/08 (Mon) @ 11:11

I do have them.  Not with the projection spreadsheet but I do have a map.  Just match up by name, all my player names should be unique (ex. Drew Anderson and Drew Anderson1).

Just send me an email and I’ll shoot them back to you, or post your email here.  I know you’ve sent me stuff in the past, but I think that was before I got this computer.


#12    Colin Wyers      (see all posts) 2010/03/08 (Mon) @ 12:30

You (or anyone else, for that matter) can reach me at:

pontifexexmachina at hotmail dot com


#13    Dan Szymborski      (see all posts) 2010/03/08 (Mon) @ 21:23

Pitchers soon.  Teaser: ZiPS seems to know it’s pitching.

I heartily approve of this conclusion!


#14    Dan Szymborski      (see all posts) 2010/03/08 (Mon) @ 21:26

Do any of you guys have a good spreadsheet with MLBAM IDs?  I could try to map them - I usually just go with the DMB IDs since I store projections within DMB.


#15    Tangotiger      (see all posts) 2010/03/08 (Mon) @ 23:03

Dan, see link in Section 3:

http://www.tangotiger.net/forecast/rules.html


Page 1 of 1 pages


Name (required)
E-Mail (optional; WILL be published)
Website (optional)

<< Back to main


Latest...

COMMENTS

Feb 12 05:18
Reader Mail of the Day: Why do we need X years of fielding data?  And what about outliers?

Feb 12 04:55
Who is Jeremy Lin?

Feb 12 03:15
New PECOTA

Feb 12 02:42
Whitney Houston

Feb 12 02:23
Psst… wanna intern in Canada?

Feb 12 00:40
Clutch analogy

Feb 11 20:11
Fighting leads to goals?

Feb 11 19:55
Why do players get crappy caps?

Feb 11 19:12
Hero of the month: Brittney Baxter

Feb 11 17:59
MGL: Today on Clubhouse Confidential