THE BOOK--Playing The Percentages In Baseball

If, for example, the “standard” weights for the last 3 years is 5/4/3, then you would weight BABIP as 4.5/4/3.5, and you would weight K/PA as 7/4/1 (or some such).

Doesn’t regression to the mean take care of the 5/4/3 weighting, especially if you regress differently for each stat?

No, there are two components.

1. How much to weight more recent performance relative to older performance.

2. How much to weight performance at all.

So, for K/PA, you might say: make it 80% performance, 20% population mean.  For BABIP, you might say: make it 60% performance, 40% population mean.  (Numbers for illustration only.)

Tango - When you are estimating a player’s true talent do you regress his performance first and then weight the more recent performances, or do you weight first and then regress?  I believe the order makes a difference.

Peter, the order doesn’t matter.  Where it does matter is when I apply my aging curve.  And I think I apply the aging curve at the end, after regressing.

On a related note, how did the rule of thumb that each year gets weighed .80 of the next year come about?

Peter - I do the regression after the weighting.

If I regress first, I have a weighted sum of regressions, which means I have regressed more than once, and I don’t believe that is the correct way to do it (I believe mgl agreed with me on this last year).

Zach - Last year I ran some empirical tests. The current year only is an estimate of a player’s current talent, but the small sample size leads to wide variations. The multiyear weighted mean gives larger sample sizes and thus smaller variances. I measured the differences in both measures for each player, looking for the weight which would generate the smallest sum of the differences.

A weight of 0.8 over 3 years gives 1.00, 0.80, 0.64. 5/4/3, on the same scale, is 1.00, 0.80, 0.60.

Excellent topic, as it inspired me to look at the appropriate weights by component.  For strikeout%, I get a weight of 7-3-2-1, with the 1 being league average.

My test was all players with 400+ AB in 4 straight years between 1982 and 2005.  The R is .89 with average AB=533. 

David Wright has upped his expected K% from .190 coming into the year to .235

A couple of things to keep in mind:

One, when we talk about how much to regress, we usually talk in terms of how many PA (or whatever the units of opportunity are) we have in our sample.  However, once we weight performance, it changes the number of effective opportunities for regression purposes.  For example, let’s say we have a weighting scheme of 10/3/1.  And let’s say that we have 500 PA in each of 3 seasons.  It would make no sense to use 1500 PA to determine the amount of regression when we are barely even using season 1 (and its 500 PA).  I think that lots of projection systems ignore or neglect this issue.

Two, not only does the amount of noise in a component affect the weighting (in fact, I am not sure that it affects it at all), but the chances that the true talent of a certain component changes with time.  I think the latter is the most important determinant of the proper weighting system.  In fact, I would like to know why the amount of noise should affect the weighting.  What if BABIP were noisy but there were little chance that a player’s true BABIP changed over time?  Why would we have any weighting at all?  Likewise, what if K/PA were not very noisy, but for whatever reasons, players changed their true talent K/PA very easily and quite frequently (and that the change sustained itself for a while)?  To put it another way, what if what we interpret as “noise” (random error) is really a frequent or marked change in true talent?  For example, we generally weight overall pitching performance more aggressively than hitting performance - but not because it is inherently more noisy, in a random error sense - but because pitching is more subject to changes in true talent due to injuries, learning, and the like…

MGL - Your post #8 raises several questions for me.  Since we only want to weight that portion of performance variation that is due to actual changes in true talent rather than factors outside the control of the player, shouldn’t the regression of the performance be performed before the weighting? 

Second, if it is the changes in true talent that we are trying to weight over the course of time, how is that differentiated from the changes in true talent that are being weighted by the aging curve?  Are those true talent changes caused by time being adjusted twice?

Third, what would the proper amount of PAs be for regression purposes for the 10/3/1 weighting, 500 PA per season example you give.  500 + 166 + 50?

Seidman, best batting average on groundballs:

http://www.fangraphs.com/blogs/index.php/grounded-success/

Rally:

http://lanaheimangelfan.blogspot.com/2009/06/david-wright.html

For K, the weights I get are 7/3/2/1, which yields a new .235 K rate for Wright.

BB: 9/5/4/1. Not a whole lot of regression needed when you have 3 full years of these players. Wright comes in at .133 per PA, the only part of his game where he’s playing at his normal level.

HR: 11/7/5/2. For David, a .043 rate per contact (AB-K) which means 12 more homers, and a projected season total of only 16.

BABIP: 10/7/6/10.

Those weights are: for the last 3 years, and the last one if the population mean.

No surprises whatsoever here.

I think that is right for the regression - 500+166+50.

The regression should be done last, as it is a shortcut for a full Bayesian computation ("given this sample performance, what are the chances that this player is a true X, given that there are P percentage of true X players in the population, what are the chances that he is a true Y, given the percentage of true Y players in the population,” etc.).

Yes, aging is part of the reason for the weighting.  Aging is merely one of the things that causes a player’s true talent to change over time - maybe the most significant thing - I don’t know.

The “age adjustment” can be done at any time, although I think it should be done at the end, although it may depend on how and “why” you do it:

First I am sort of estimating the average true talent over the life of the sample.  The weighting actually allows me to estimate the more current true talent level, accounting for the fact that the player may have or likely has changed that true talent somewhat over time, including by basic aging.

Now, if we are projecting to the present time, maybe there is no need for a further aging adjustment, as that is “included” in the weighting.  To tell you the truth, I am not sure.  But if we are projecting over the course of the current season or beyond, we might need to do a minor age adjustment to account for the fact that the player’s true talent is continuing to change because of aging, and that that change is not the same for all ages. 

So, for example, if we project the current true talent of a 22 year old player to be a wOBA of .300, by the end of the season, it is probably going to be .310 (or whatever).  If the player is 35 years old, by the end of the season, it is probably going to be .285. So a projection for the entire season is going to be different for each player even though the current true talent at this exact point in time (say, when the season starts) comes out exactly the same for both players before any age adjustment is done.

I also think it is better to “age normalize” each year of stats before weighting and combining them.  That is what I do, at least.  IOW, I convert each year of stats for a player into “age 27 stats,” then combine and weight them.  Then I regress and then I age adjust back to their real age.  Maybe I should age adjust back to the real age and then regress (to the mean of similar players of the same age). I am not sure.  I am not even sure, off the top of my head, of what I do exactly in my projections.

These things are tricky and they probably have not been researched and thought out well enough by the various forecasters, although I can really only speak for myself.

I think that’s right for the regression.  Easy way to do it is use 1 for the first year weight and express all other weights as fractions of that.

I apply regression after combining all the years .  If you don’t, you need to be careful you aren’t over-regressing.

David Wright decided that regression just isn’t his thing. He went 4-4 last night.

The increased K’s. the decresed HR’s, the increase BABIP.....it seems like he may have altered his approach at the plate and if so I think thats even more reason to weigh recent performance alot more than his career totals.

> BABIP: 10/7/6/10

That’s a pretty big surprise if that isn’t a typo.

It’s no typo.  Not too far off from Tango’s guess, 4.5/4/3.5.  And he also guessed 60% data/40% population mean for regression.  This suggests 70/30, but it’s not an extreme difference.

That makes sense the more I think about it.  The league average weight just looked so out of place, but, um, I guess that’s the point of this thread in the first place.  Thanks.

BABIP: 10/7/6/10

Rally - In the general example of the above (the questiona applies to all components expressed in that form)

Isn’t this giving the regression as a fixed percentage of the sample, instead of a fixed amount to be added to the sample?

For example, Tango might say to add 600 PA’s of league average performance for some category. A player with 400 effective weighted PA’s will have 40% observed, 60% league mean, while a player with 1400 PA’s will have 70% observed, 30% league mean.

In Rally’s formula, league mean will always be 30% of BABIP, regardless of the player’s sample size. I trust that this is the correct amount of regression for a full time player, but it will break down with larger or smaller samples.

I thought the intent of the regression was to weight the player towards league mean in the absence of observed data. As the sample becomes larger, a larger percentage of the projection should be based on the sample, and less on the population mean.

mgl - this is my procedure
1. normalize each team/season for each player
2. sum into a single line for each season
3. weight each season
4. sum into a single line for each player
5. add regression and aging

"In Rally’s formula, league mean will always be 30% of BABIP”

I presumed Rally meant that formula for a regular.

That 30% is when a player has 530 AB in each of 3 seasons, as in the study I did last night.  It will vary depending on every player’s input.

Tango/21 - I presumed those were the weights used for everyone. I’m sure they work fine for regular players, but if those weights are used for a player with limited playing time, they would be regessed a smaller amount than what might be correct.

Another way to express that formula for babip is year1 + year2 *.7 + year3 *.6 + 530 AB of league average.  Hopefully that will be less confusing.

Rally/24 - that looks fine to me

Just curious… if you ran a marcel on a specific player, what would the difference be if you regressed first or regressed second? I just want to know the kind of range of expected outcomes we could see here.