THE BOOK--Playing The Percentages In Baseball

Hmmm ... you’d expect the double and SF to come out right in the regression.  One possible explanation is small sample size ... do the confidence intervals for those values include the known, actual value?

It could also be the “other stuff”.  If outs is one of the “other stuff"s, and SF is an out, then the regression value of the SF is actually the sum of the two values. 

Also, if you use team seasons as your regression data, they all implicitly contain the same number of outs (pretty much).  That means that ... more to come, I just got called away.

You should check out the apbrmetrics (NBA) site.  It’s pretty much all regression all the time.

Phil, I’m not a statistician, so I’m not sure how to find the confidence intervals.  Minitab didn’t spit out anything recognizable to me as a confidence interval, but I don’t really know what I’m looking for.

However, the results are similar to every long term regression on team seasonal stats I’ve ever seen.  In Curve Ball, Albert and Bennett found a very high value for the SF (I don’t have the book in front of me, but IIRC it’s around .6).

Here is the regression equation from M Schell, for all teams 1947-2003.

.52*1b + .80*2b + 1.11*3b + 1.52*HR + .32*(BB+HBP) - .111*(AB-H)

To me, this looks much better for the 2b. I’m not sure what is the difference of Schell’s technique, but he does state that “each component has been adjusted for its own ballpark effect, rather than using a single ballpark effect for runs.”

So, could the 2b problem have to do with the idea that many good HR parks are not good for 2b?

Excel gives you the significance of each value.  Do Tools/DataAnalysis/Regression

I remember I looked at this a few years ago, and posted the results here.  It was some very wide range, like 0.66 runs +/- .20 runs 95% of the time.

You can also select your dataset carefully enough (with hundreds of lines of data) to get a run value of the triple to exceed that of the HR.  Which is more than ridiculous.

The point is that the Run Expectancy and Markov model work far far better than any regression analysis could hope for.  Aggregating data at the team name level to reduce sample size and introduce bias is not something that an academic would choose to do.. and yet, they are doing it.

The run value of a double is around .30 from a single, and any model that shows otherwise has decided to become a slave to regression, and is worthy of whatever scorn you wish to heap upon it.

The standard deviation of the SF coefficient for my data is .145, and the coefficient is .715, so 2 standard deviations in either direction would be (.425, 1.005) (assuming that this is valid math).  So that would not include the empirical value in the range.

The double has a coeff of .642 and a stdev of .043, so that would be (.555, .729), with the empirical value outside the interval.

To kind of echo what Tango is saying, regression is such a lazy way to go about looking at a problem like this, when it is relatively easy to make a good model (and even if you want to disagree with the characterization, Lindsey, Palmer, Smyth, et al have already done it for you). 

The reason I focused on regression in my post was not so much to rip it, but as to illustrate the problems with RMSE, Average error, etc. as the sole deciders of the value of a run estimator.  Since the regression by definition will be the most accurate model, the logical conclusion of the “lowest RMSE always wins” line of thinking is to become a slave to the regression.

My point is that there are a myriad of factors that are important; RMSE is just one of those (as are the +1 values, the logic of the model, the ease with which it can be applied, etc.)

Tango, just to be devil’s advocate here—if the regression results in a low run value for doubles because it is “standing in” for something unknown ("factor X"), then don’t you have to stand by the regression until you know what “factor X” is? On another thread when we were talking about batter/pitcher matchups, one poster seemed to be arguing that because he “knows” that some pitchers own hitters, that a 7-for-12 lifetime batter/pitcher matchup must have meaning. But regardless of whether pitchers and hitters really do “own” eachother, the point is that the only data you have to go on is the 7-for-12, and the result of looking at all prior 7-for-12 matchups indicates that there is little predictive value. Likewise, even if you “know” that the true value of a double is .805, the data you are using does not account for “factor X”, and so you have to take the available data at face value.

To put it another way, if MLB only tracked singles, doubles, triples, HRs and runs scored, then your regression values for the hits would be higher than standard linear weights (because you wouldn’t be accounting for walks, steals etc). In that situation, you would be better to use the results of the regression rather than the theoretically correct values.

Dackle (or advocate-Dackle), I would argue first that we should present two pieces of data, one as +.80 for the double, and another for -.15 for the proxy-double.

However, when we process the data at the pbp level, we know exactly the value of the double, and we get the same value when you create a full Markov chain.

The -.15 value for the proxy-double occurs ONLY after aggregating the 6000 PBP records into a single seasonal line for the team.  And only when restricting the dataset to a certain period of time.  So, the .15 represents some sort of sampling bias.  If you want to argue that you need it at the team level, to counteract the bias, ok. 

But, for anyone to argue that we need to introduce the team sampling-bias proxy at the player level?  That is to be a slave to the regression and makes no sense.  Above all else, we need to be rooted in logic.

Regression users deserve the full scorn from those who wish to say they should get their noses out of their spreadsheets.

I guess I just have a feeling that if one or two additional stats were introduced into the regression, such as extra bases taken on hits by baserunners, then the double would be worth .80 (ie the theoretical value).

If that’s true, then wouldn’t it make sense to use .80 if you have the “extra bases taken on hits” stat available for players, and .65 (or whatever the regression value is) if the “extra bases” stat is not available (ie you’re just using generally available stats)?

Running a regression on the 1878 team-seasons from 1919 to 2007, I get an r=.97, with the following coefficients:
0.56: single
0.80: double
1.50: triple (standard error = .05)
1.40: homerun
0.37: walk

The triple can be standing in for the stolen base.  The single could be standing in for reaching on error.  The walk can be standing in for the hit batter.

Who knows?  Who cares?

I’m not going to be a slave to these numbers, even if it best-fits me. 

And what if I take some other time period, say, 1969-1992 (608 team lines)?

r=.97
0.50: single
0.76: double
1.24: triple (standard error = .10)
1.43: homerun
0.36: walk

Well, which value of the single do you use?  What about the triple?

See where I’m going here?  What regression coefficients do I apply to the 1992 Expos?  Someone is going to suggest using seasonal lines within 10 years, meaning 1982 to 2002.  How does that make it any better?  Especially since baseball changed in 1993.

The whole thing is just so much b.s.

Regression as the final step = lazy or clueless analyst.

Someone feel free to find me any sabermetric research that proves me wrong.

The reason triples have such a high value going back to 1878 is because they correlate highly to reached on errors. 

I don’t know if that’s true for modern teams, but teams in the deadball era hit a lot more triples than they do today, and scored many more runs given their h, w, hr, etc. inputs because errors were so frequent.  That’s what the regression is picking up.

I am no statistician at all, but I don’t see that Tango’s valid points means that regression should not be used. It is a powerful tool. It seems to be the case that most of the problem is not with regression per se, but with how it is applied. So, apply it not to the the team-season data, but instead to the individual inning level. IOW, try to get rid of the inappropriate stuff that causes problems, and to leave behind the best unbiased high sample size data, and THEN apply the regression.

Done well enough, that MIGHT be the best way to find the best run values for, say, a basic estimator. To separate out the TRULY hidden information from the other stuff.

As I mentioned a few posts ago, M Schell did some sort of park adjustment which seemed to pretty much get rid of the 2b problem, even though he is still using the team-season data.

I said that regression can be a starting point, not the end point.  Too often, the analyst/academic is a slave to the regression, not even acknowledging the bias in the data, only concerned about minimizing RMSE.

As Patriot so well showed, if you want to minimize RMSE, you will end up with a ridiculous value for the double.  Or in my case, the triple.

Regression is a tool, and that’s as far as it should go.

-----I said that regression can be a starting point, not the end point.”

And I was saying that, if all of the prior work, selection, and adjustments are done properly, maybe regression can be the endpoint.

You have often pointed out the difference between a stat/method, and its implementation. The regression thing seems to be mostly an implementation problem.

No?

I believe that it is much more likely that, rather than finding hidden information when you do an inning-level regression, you will find that the results are similar to what we know to be true about the values of the events.  That is not to say that it is not worth taking a look at; I’m just predicting that you will find little of value.

And the “ecological fallacy” may rear its head at some point.  Even if team data (even on the inning level) implies the existence of hidden information, it is still a leap of faith to apply it to the individual.

I don’t think that using regression in general is the problem. It’s the modeling and interpretation that causes misleading results. Part of the problem is that there is a proliferation of software that can allow any fool (like myself) to dump a bunch of variables into a black box and get a result.

Regression is inappropriate for a deconstruction of runs in baseball. The reason doubles are only appear slightly more important than singles in scoring is that teams that hit lots of doubles also hit lots of singles. Many of the same skills that lead to doubles also lead to singles. The crux of the issue is that singles, doubles, etc. are really just intermediate outcomes toward scoring. The true underlying skills are things like power, pitch selection, bat speed, coordination, and running speed. Both singles and doubles are proxies for those underlying skills.

There is a similar issue in football research, and regression is used frequently. Most football scoring models will include passing stats, running stats, *and* 1st-down-rates (or 3rd-down efficiency). First downs are intermediate results between running and passing skills and scoring and do not belong in the model.

100% of sac flies turn into runs, so a regression would be bound to provide a biased coefficient. Sac flies really just a result, not an input. They’re a subset of the dependent variable, which makes it invalid in as a regression estimator. It’s the model, not regression in general.

For a valid regression model, you’d want to start with something like hits and walks, factor in variables that advance runners to 3rd base, then add pop-fly outs. Of course, pop-fly outs rarely lead to runs so its coefficient would be near zero. But if you add an interaction variable of pop-flies*runner-advancement, you’re on your way to a logical model.

I think David Smyth makes the point more elegantly above.

Regression can be used as a ‘final step’ as long as a researcher does his homework in the preliminary steps. Say we’re trying to estimate a pitching prospect’s probability of success in the majors. We can use variables of his tools (velocity, movement...), height, health, handedness, etc, and estimate the weight of each factor based on a historical sample. Sure, more research could be done on pitching prospects using other methods, but that doesn’t make the regression result any less of a valid endpoint than any other method. Indeed, in this case it may be superior.

Pointing to the fact that regression is problematic as a run estimation method as evidence for its general inappropriateness for sabermetrics is a itself a fallacy. Run estimation does not equal sabermetrics.

I think we’re on the same page here.  My basic concern is that the regression is used as a crutch, with its limitations completely ignored, allowing the analyst to become its slave.  It could be used as a great tool… but I haven’t seen much if any.