THE BOOK--Playing The Percentages In Baseball

For me, your illustrative distribution for Pujols tells a lot that most non-stats folks won’t get (and I see this in many more places than just baseball)… I think people, in general, don’t understand that when you “project” Pujols to hit 31 HR that you really only think he has ~20% chance of being within a couple HR of that actual number… If someone knows how to convey the distribution behind the projection in a way that won’t make non-stats people’s eyes glaze over, I’d be interested in hearing about it…

#1/Barry: I think it’s quite simple. CHONE projects 39 HR in 634 PA for Pujols, which gives a standard deviation of almost 6 HR exactly. So he has a ~16% chance to hit 45+ HR, a 16% chance to hit 33 or fewer, and a 68% chance to hit 33-45 HR. And he has a ~2.5% chance to hit either 51+ or <27 HR.

I don’t think you need to be a stats person in general to appreciate probability.  If you make the simplifying assumption that each of Pujols’s PA are identicle in the chances of him hitting a HR, which while obviously false hopefully doesn’t effect the estimate to much, then doing what Zach does and saying that 39 HR in 634 PA means that the chance he hits a HR in any given one PA is about 6.151%.

Now we model it as a coin that comes up heads 6.151% of the time (HR) or tails the rest of the time (not HR).  Or, to be even more general but give people a better intuitive feel if they play cards, this is about the chance of you drawing a card from a normal deck and it being a non club Ace (so one of the spade ace, heart ace, or diamond ace).  So if you shuffle a deck of cards and draw one card and say Pujols hits a HR if you get a non club Ace and a non-HR otherwise you could experiment yourself drawing 634 PA worth of card draws (being sure to reshuffle after each simulated PA).

Instead of simulating it by shuffling a lot of decks of cards, we can use not too advanced math to calculate the answer.  Showing our steps, the variance therefore is 634 * 6.151% * (1-6.151%) = about 36.6 and the standard deviation is the square root of this which is about 6.05.

And if we then use a lookup table (wikipedia entry on standard deviation, for example) for the normal distribution we can see that we predict
Pujols will hit (where the lower limit is calculate in parens using the mean estimate of 39 and the standard deviation of 6):

>57 HR around 0.1% of the time (39 + 3 * 6)
51-57 HR around 2.1% of the time (39 + 2 * 6)
45-57 HR around 13.6% of the time (39 + 1 * 6)
39-45 HR around 34.1% of the time (39)
33-39 HR around 34.1% of the time (39 - 1 * 6)
27-33 HR around 13.6% of the time (39 - 2 * 6)
21-27 HR around 2.1% of the time (39 - 3 * 6)
<21 HR around 0.1% of the time

That, I think, is the sort of explanation that can work for a non-stat person.  If you want you can even go into the caveats and complications with a paragraph like:

There were a couple of assumptions that went into this that are clearly wrong which make especially the extreme estimates a little off.  The first one is that obviously facing Lincecum in SF for a PA has a different chance than facing the 5th starter for Colorado in Colorado does.  So some PA are less likely to be HR than our 6.151% estimate and some are more likely.  This will influence the estimates only slightly.  The bigger problem is that these numbers assume that Pujols gets exactly 634 PA.  If somehow he gets 675 PA he is much more likely than 0.1% to break 57.  On the other hand, the more realistic scenario is that if Pujols gets injured he is quite unlikely to get 634 PA and if he only gets 400 PA, say, then the chances that he gets fewer than 21 HR is much more than 1 in 1000.

I think the problem is that the projections when reported on Fangraphs or wherever else just give a stat line. It looks a lot like an actual stat line from a previous year. It doesn’t say “there’s an X% chance of this happening.” It just says “we project this.”

So that’s the first problem. What many of us presume when we get projections (even if we don’t know exact percentage of likelihood, a guy like me who understands this stuff but doesn’t do all the math knows that a 31 HR projection doesn’t mean 31 homers on the nose) is NOT what the lay person presumes. Why? Well, the nature of the “prediction.” When that loudmouth on the Boo-ya network gives a football score prediction, that’s the score he thinks will happen. There’s no “give or take a few points” with it (and hey, if he’s even kinda right, it counts as FIVE wins!). People think the same is true about baseball stat projections.

The next problem is this: So you’ve gotten the lay person to realize the stat projection has a wide range of possibilities. Their reaction is then, “So why is this useful?” If you tell a guy “No, I’m not saying Pujols will hit 31 homers. I’m saying he’s going to hit between 6 and 56” they’re going to say “I didn’t need a projection system to know that.”

@ Barry/#1:
PECOTA does this fairly well with their ‘percentile’ predictions. If you are not familiar, it provides 25%/50%/75%/90% projections which encompasses the range of possible outcomes for a given player.

All,

Thanks… I think JD/#4 said more eloquently and went a step further with what I was trying to get at in his last paragraph…

Also, my understanding is that the PECOTA percentiles have not been (publicly) shown to be meaningful… Although, I like the idea of presenting a range of outcomes…

That is correct: the PECOTA percentile range, while they look fine overall, are biased.  They show similar ranges for rookies as it does veterans under 30 as it does veterans over 30.

The actual range needs to be based on the uncertainty of our estimate.  And the uncertainty of our estimate is far higher for Montero and Wieters than it is for Chase Utley and Robinson Cano and Nick Markakis and their peers.

I had the percentiles last year.  It’s something I haven’t had time to reproduce for this season, though maybe I’ll do it next year.  A study on how well it works is incomplete.

http://www.baseballprojection.com/pujolal0047.htm

So this could be interpreted that Pujols had an 80% chance to hit 25-45 homers.  The way I did it shows the range as being much wider on the downside of 50% than the upside. That’s because of playing time.  His baseline projects 149 games played.  He can top that by 13 games at most (14 if the Cards tie for a playoff spot).  On the downside, he could get hurt and miss the whole season.

I think the system needs to be tweaked at the low end. 

The 2009 Marcel for Juan Pierre predicted 3 home runs.  He had only hit 1 home run in 2007-08, in 1,043 AB’s.

This is obviously an artifact of the regression applied to prior results.

Now, of course no one really cares how many homers guys like Pierre hit, any more than they care how many bases David Ortiz steals… so maybe I shouldn’t say the system needs to be tweaked, but it does fly in the face of the explanation that the predicted value is the midpoint of all the possibilities…

Take the most egregious case: someone hits 0 HR in 2100 PA in the last 3 years.  I am going to regress 12% toward the mean.

The mean will be about 17 HR in 700 PA.  And so, he’ll get 2 HR as his forecast.

In terms of a linear method, I have to accept that.  Now, if you are suggesting that the mean should be 0.1 rather than 2.0, that’s fine.  I’ll agree with you, especially if you tell me this hitter is a fielding wizard who plays SS and weighs 150 lbs, and who has 5 career doubles.

The problem is that you introduced something (the name Juan Pierre) that conjures something to a baseball fan that Marcel is unaware of.  You are using more information than Marcel, and so, if you use that information in an intelligent fashion, you’d have to be more accurate than Marcel.

CHONE knows a few things about Pierre that Marcel does not.  He’s listed between 180-185 pounds depending on the source.  He hits more groundballs than most players.  But even 180 pounders who hit an above average amount of ground balls hit some homers, so there’s still a good bit of regression.

The end result is I project 2 HR instead of Marcel’s 3.

The downside is almost nonexistant.  So he might hit 2 homers less than I project.  But he might shock the world and top the projection.  David Eckstein once hit 8 in a season.  Freddie Patek once hit 3 in a game.  Strange things can happen.

It’s also important to note that I cannot make Marcel smarter, because then someone would make a system dumber than Marcel.

The minimal level of competence is: past performance, regression toward the league mean, and age.  I don’t want to make anything better than that and call it Marcel.

"because then someone would make a system dumber than Marcel.”

then?  According to JCross’s results they are already up to the task.  Including yours truly on pitcher walk rates.

Good post.  I drafted up a similar post recently about the purpose/intention/interpretation of a projection system that’s set to run sometime next week.  After reading this, I threw a graph of the Pujols chart into the text to illustrate the point I continuously repeated...a projection is a distribution.

#4, I answered the question of ‘Why important’ by talking about expectations.  Projection systems allow us to look at more than just the players we watch 162 times a year and form expectations for them.  This might not be important for a fan who only watches one team, but increasingly, more and more fans follow multiple teams and those that participate in fantasy leagues want to have some realistic expectation for ALL relevant players.  Lastly, it can help temper our enthusiasm/distaste for our own club’s studs and busts.

It’s not literally important to have player projections, it’s just a preference.  As fans we can go into the season without any projections and be no worse for the wear.  Humans just like to project things.