THE BOOK--Playing The Percentages In Baseball

To me, it appears like the rate stats (AVG/OBP/SLG) are inflated as much as the playing time projections.

Zach, well you can’t tell whether rate stats are inflated or not unless you know the baseline (league averages) of the system.  And the way they do their playing time projections, it is next to impossible to figure the baseline.’

Then again, the rate stats of all the veteran players they project should equal their rates from last year (if they are scaling everything to last year) or the last few years (if they are scaling everything to the last few years) after adjusting for aging.

The whole thing brings up an interesting point about testing the various projections systems.  Tango usually starts by normalizing everyone’s projections to their means (I assume weighted by playing time).  Is he using the same group of players for all systems in order to do that?  If yes, the different playing time projections could distort one system’s mean as compared to another.

BTW, Ben’s explanation for having an “optimistic” (as compared to the eventual mean) playing time projection for some players, particularly the elite ones (who expect to get max playing time barring an unusual injury), is a good one, and something I never thought of before:

Take 10 elite hitters with a similar health record- say nine of them will get 600 ABs and the other (we don’t know which one ahead of time) will get hurt in the spring and get zero ABs. We could regress them all and give all 10 550 ABs, meaning we’re selling nine guys a little short and missing big on the 10th. Or, we can project all 10 for 600 ABs, sell nobody short and missing big on the same guy who was injured.

However, this explanation makes little sense and sounds like an excuse (and a poor one at that) for some odd playing time projections:

First off, we are admittedly optimistic with playing time projections, especially in the Handbook edition of the projections. The thought is to give the reader an idea of what we think a player can do if given a full season of plate appearances. We could project Jason Heyward for 200 ABs with a couple homers, but that doesn’t tell you much about what we think of his ability. We want to give him a full season of ABs to show you what we think he can do if he gets a chance.

Either your playing time numbers for marginal players are projections or they are not.  If they are not, then you should just give projected rate stats for them.  And what he said is not even close to being true.  I have the Bill James Handbook in front of me.  There are hundreds of hitters who are projected at 100 some odd or 200 some odd PA’s.  So I have no idea what he is talking about in the above quote.

Ben’s quote sounds like it’s taken almost straight from an early version (the first?) of the Handbook from 15-20 years ago. 

My understanding/interpretation of what Bill said about his playing time projections then was that basically they are the playing time that Bill (or an equivalently smart GM) would give the young players if he were running the team.  I think the example he gave in that very first Handbook was of Jeff Bagwell, whom he “predicted” to take the batting championship ahead of Tony Gwynn in 1991.  If the Astros chose to make Bagwell their first baseman in the upcoming season, Bill James believed that he could hit well enough to play full time.  He wasn’t necessarily predicting how the Astros would allocate their playing time in 1991, rather, if Bagwell hit to his expected ability, how much playing time could he likely hold down on an average major league team.

Now, I may be remembering some of this wrong or conflating some 20-year-old memories, but I think that’s the basic gist.

I also know from experience that it’s a poor way to draft a fantasy team.  You end up with lots of guys who spend time in AAA because their team’s GM disagrees with Bill James about their ability level.

Re MGL/#2:

Either your playing time numbers for marginal players are projections or they are not.  If they are not, then you should just give projected rate stats for them.  And what he said is not even close to being true.  I have the Bill James Handbook in front of me.  There are hundreds of hitters who are projected at 100 some odd or 200 some odd PA’s.  So I have no idea what he is talking about in the above quote.

Let me clarify a bit on how some guys get 125 ABs and others get the full 600.  We take it case by case, really.  Two examples:

Eric Bruntlett’s ABs since 2005:  109, 119, 138, 212, 105.  Everyone “knows” Eric Bruntlett is not going to get 600 ABs unless (God forbid) the entire team experiences a Marshall football team style tragedy on Opening Day.  Even if he did get the ABs, his production isn’t going to be worth noting for, say, fantasy baseball.  Therefore, Bruntlett’s Projected ABs for 2010:  109.  Similar guys might slip below our 100 AB threshold for even listing them in the book.

However, in the Heyward case, there’s no telling how many ABs he’ll get, especially when we’re doing the projections in October.  He could win the job out of Spring Training.  The Braves could call him up midseason Wieters-style.  Or they could decide not to rush him and he needs to spend 400 ABs in AAA.  When you’re drafting for your fantasy team in late March sometime [RE: Mike Fast/#3], you’ll have a much better idea than we did in October or even at the beginning of March when we release the Projections Update. 

We could average the probabilities and decide he’ll get 250 MLB ABs.  But as noted, that doesn’t tell us much.  If he DID get 250 ABs, he could hit .240 or .350 based on a sample size fluke alone.  The BJHB philosophy has been to err on the high side for playing time so you get a better picture of what the system thinks of him. 

Again, it’s not a steadfast rule, but a case by case interpretation. 

Lastly, it’s possible that for many hitters the BJHB projection rate stats are still a touch higher than other systems.  I personally don’t know.  I do think that you can debate a few different approaches for projections (perhaps with only marginal improvements over Marcel at best), and I’m not saying that any one system is better than others.  I do think that using weighted means for playing time and/or performance level projections leaves out some useful information, though.

BTW, looking forward to testing the projections in the 2010 Forecasters Competition!  (Sorry BIS missed last year!) Of course one year doesn’t necessarily prove anything, but it will still be fun.

Ben that makes sense, but how is the reader supposed to know which is which?  And why in the world is this not explained in the Handbook?  As I said, either the playing time are projections or they are not. If some are and some are not (clearly) then don’t you think you need to tell your readers which are which, otherwise you might as well just use rate stats?

And I still don’t understand why for a player like Heyward why you would have him with 542 AB.  Where do you even get that number from? Why not just put in what you think he might get at this time and just leave the other stats alone, prorated to however many AB’s you “project?”

“If he happens to get 542 AB, here is what his counting stats will look like.” Huh?  Why 542?  Why not 400?  And again how would you distinguish players like that and other prospects or marginal players in your Book who are listed with 100 or 200 some odd AB’s.  You brought up two players who are clearly distinguished (one is a prospect with no MLB time and the other is a veteran back up).  But there are dozens of players who don’t fall explicitly into either category.  How is the reader or even you supposed to distinguish them in terms of which “method” you are using to “project” (clearly it is NOT really a projection) their AB or PA?

I am also looking at other players who never played in MLB, like Logan Morrison, who are projected at 100 some odd AB. Is that because they are not really prospects and you expect them to have back-up roles at best?  Again, if you are making all these judgments on players as to what “category” they belong in, and then you are using these categories to determine whether you want to actually project a “real” number of AB or PA for them or whether you simply want to use some arbitrary number (e.g. 542) to represent, “if they happen to play full-time,” do you think that is consistent or fair to the reader?

I understand what you are saying, but I’m sorry, I am not buying the explanation as it makes little sense to me in the context of the Handbook and the questions I posed above.

I appreciate you stopping by and trying to explain your logic, but I am still not buying it.

I have nothing against the projections by the way, and I have no idea whether they are in general too optimistic or not, and as I said in my first comment, I am not really even sure how to define that (optimistic) especially if you are not really projecting playing times and you are leaving some players out entirely (i.e. it makes it hard to establish a baseline for all your projections combined).

Anyone else?

I looked at the wOBA projections on FanGraphs for 154 players with at least 500 PAs in 2009 (I think everyone with 500 PAs except Colby Rasmus, Chris Coghlin, and Elvis Andrus, for whom I didn’t have BIS IDs).  Taking the straight average of the wOBAs for the 154 players so that each is weighted the same amount and projected playing time is not considered, I get the following averages for CHONE, Marcel, and James:

.350 CHONE
.349 Marcel
.353 James

Comparing James to the 134 players who had Fan Projections:

.357 James
.359 Fans

For each of the 154 players, I also looked at which of the 3 non-fan systems had their highest projected wOBA and totaled up the number of highs each system had (some had ties, so these won’t add up to 154):

43 CHONE
39 Marcel
80 James

Finally, the sample standard deviation for each system:

.026 CHONE
.025 Marcel
.029 James

This obviously doesn’t include rookies or players who didn’t play much last year, so if that is where James is most optimistic, this wouldn’t pick that up.

.004 in wOBA means about 0.12 more runs per game.

Now, if you do the same for pitching, I would bet the ERAs of James’ forecasts come in at .10 runs lower than Marcel.

Basically, the James forecast are optimistic on both ends.

Pitchers with 100 IP in 2009 (excluding rookies whose BIS IDs I don’t have again), n=117, average ERA:

4.33 CHONE
4.22 Marcel
4.13 James

vs Fans, n=73:

3.91 James
3.83 Fans

Number of pitchers with low ERA forecast by each system:

16 CHONE
35 Marcel
69 James

SD:

.49 CHONE
.51 Marcel
.50 James

Kincaid: great stuff!

4.22 Marcel
4.13 James

Now, that was ENTIRELY predictable, no?

There’s no question that the Bill James forecasts are optimistic for both hitting and pitching, and it has nothing to do with any logical rationalization by Ben or anyone.

If you were to apply any kind of reasonable playing time for every player, you will find that the Marcel runs scored v runs allowed will pretty much match to each other (differential of 0), while the Bill James forecasts will end up with a differential of something like 1000 runs.

It doesn’t really matter if you treat each forecasting system as its own universe, since you can re-calibrate as you need it.  But, you can’t then try to compare individual players from each system.

Kincaid, can you show the following:

system,ERAin2009,forecastin2010

My guess is that James applies little to no regression so that the forecasts match their actual totals.

You mean for each of the 117 pitchers?  There’s a table with system, ERA in 2009, and forecast in 2010 for each pitcher (no playerIDs) linked in my name.

The average of the pitchers’ 2009 ERAs was 4.24 (same for each system in the aggregate since it’s the same 117 pitchers in each sample).  James has these pitchers improving their ERAs in 2010, Marcel has them pretty much the same, and CHONE has them getting worse.  ML average last year was 4.32.  I’m a little surprised Marcel’s average projected ERA for the group is further from the 2009 average than the actual 2009 ERAs, so maybe there’s something up with this sample.

The average absolute difference between the forecast ERA for 2010 and the actual ERA in 2009 is actually highest for James:

.53 CHONE
.43 Marcel
.56 James

It doesn’t look like James has them closer to their 2009 ERAs, he just has them improving on the whole even though they are already better than average as a group.

For some reason, whenever I try to link Google Docs in the website field, it works in preview but then cuts off the end of the URL when I post.  Here’s the table:

http://spreadsheets.google.com/ccc?key=0AqkgiU_VMWjUdENFek90OHBvT0ctRzE3WVJIYjIxQ3c&hl=en

MGL, regarding Logan Morrison:  our read was that it is extremely unlikely that he sees significant MLB time in 2010.  While he held his own at AA in 2009, he didn’t dominate the league at the level Heyward did.  Also considering the two organizations and depth at different positions, it seems that Heyward has a decent shot at the job from Opening Day on, while Morrison will almost surely spend most of the season in the minors and would seem like a perfect candidate for August/Sept call-up (hence the 145 projected ABs) and a shot at the job for 2011. 

Rather than the average of the scenarios, we’re probably closer (at least in theory, that is) to the most likely playing time scenario.  In Morrison’s case, that’s around 150 ABs.

And Bill actually does discuss his philosophy on playing time projections on page 464 of this year’s Handbook.  I’m not sure if it’s specific enough for your taste, but it does address this specific topic.  I’m not necessarily defending it; just explaining the thoughts behind the numbers.

I’ll stop short of defending the “optimistic” projections.  But consider this- if you gave everyone a “mean” projection, a “median” projection, and a “mode” projection, which would look the most optimistic?  More importantly, which is most appropriate for your specific interests, whether it’s fantasy baseball, analysis, or something else? 

I’m not sure about the answers to those questions, but most projections are an average or weighted average, and I’m not sure that’s always the best.  The way the Forecaster’s Challenge is set up seems like a pretty good way of fairly testing different philosophies for fantasy purposes, at least.

Ben, I agree that there is a legitimate question as to whether a mean, weighted mean, median, mode, etc., is best for projections, and it certainly depends on what you are going to do with those projections.  As I said, I like your explanation for having an “optimistic” playing time for elite players that does not incorporate their chance of injury. Anyway, no big deal…

Regarding post #3, my recollection is that Jeff Bagwell’s line that garnered so much attention back in the day was a minor league equivalency for the prior season (before MLEs were widely publicized), not a projection.

I agree that the Forecaster’s Challenge would be the best test for the data.  I totally understand Tangotiger’s point that hitting runs and pitching runs given up should even out but the most important thing for fantasy purposes is in ranking/ordering players. 

For more sophisticated fantasy analyses - like when I do Point Shares which estimates the fantasy value of a player in each of the 5 hitting/pitching categories - this inflation could be more damaging.  Would have to run it using the Bill James projections to identify the differences.

At Razzball, my co-blogger Grey (and the guy responsible for 90+% of our blog content) has a running joke regarding the Bill James estimates for rookies.  It started with the 2009 Chris Davis projections of 107 R/40 HR/118 RBI /.302 AVG / 8 SBs.  That looked ridiculous in the spring of 2009 (especially the AVG given he was a K machine) and looks even worse in retrospect.

Here are some 2011 BJ projections that I’d take the ‘under’ on even from a rate statistic perspective:

PA / R / HR / RBI / SB / AVG

Domonic Brown - 596 / 84 / 26 / 94 / .288 / 28
Desmond Jennings - 581 / 92 / 6 / 50 / .280 / 54
Jesus Montero - 442 / 54 / 21 / 67 / .285 / 0
Ryan Kalish - 641 / 82 / 20 / 94 / .271 / 45
Mike Moustakas - 580 / 74 / 26 / 98 / .293 / 2

(note: Projections for 2nd year players like Starlin Castro, Mike Stanton, and Jason Heyward look fairly reasonable.  Some rookies like Chris Carter on the A’s, Brett Wallace, and Dustin Ackley looks reasonable too)

It will be interesting to see the impact of these rookies underlying bullish rate statistics when applied to the fans’ playing time projections for the challenge…