THE BOOK--Playing The Percentages In Baseball

This is an interesting debate (for me at least). I don’t think anyone is necessarily wrong in this argument but it is a classic case of interpreting (or not) statistics correctly.

If you build a model to correlate a dependent (wins) and independent (payroll) variable if you want to see how good a fit the model is to reality then one must use the coefficient of determination (R squared). We get statements such as .... the model expalins 18% of the variance between the variables.

The question is: so what? Is that statement relevant. Well, not really because it doesn’t tell you anything; it is just talking about the reliability of a model in technical, statistical language.

By looking at R instead of R squared one can start to put the model into real language. If a team spends an extra $25M then it can expect to increase its wins by 4.3 (or whatever the right number is). This is more real / relevant for the average baseball fan and is why one should use R to interpret the data and not R squared.

However, as Guy succinctly pointed out in some of the comments the wider question is not wins but getting to the post season (they are linked, of course). Here the evidence is much stronger that adding payroll buys success.

However, we also know in basball a good dose of luck is involved, even over 162 games and that *may* be more important that salary. One would expect a std dev of ~6 games based on luck along for a .500 team (which can be a difference of 12 wins at 95% confidence), which is significant.

This is just my opinion but I tend to find that studies by economists focus too much on regression and less on other statistical / analytical techniques. Moreover they tend interpret these studies based on wider econometric criteria rather than a baseball specific context. That is not to knock all economists: some like JC and Cy Morong add significantly to our collective baseball knowledege; it tends to be other who just dab their hand in now and again.

Wow, this is ridiculuous.  Measuring my height in centimeters rather than inches doesn’t make me any taller!  Ditto for this—an r^2 of .18 means exactly the same thing as an r of .42.

What would be useful would be looking at the breakdown of variance.  For the question at hand, there are three elements of the overall variance:
1. Random luck
2. Salary-dependent talent
3. Salary-independent talent

We know 1, can measure 2 easily from the regression, and can infer that 3 is whatever is left over.

We all know that random luck is a big element in baseball; what would be interesting is how tightly team talent correlates with salary.

Quite. But the argument is (well, was originally) whether an R squared on .18 is a big effect for linking payroll and wins. TWOW authors say no, but by interpreting the statistic one can only include that it is a substantial, if not the largest effect.

*conclude*, obviously; not include

I don’t know what “big effect” is supposed to mean, but it is significant.  With sample size = 30 you need an r > .361 or an r^2 > .13.

We have that.

If one standard deviation in winning percentage is .070, then 30% of the variance is explained by pure randomness and 18% by payroll. So that’s about 50% that is due to other factors. So among non-luck-dependent factors, payroll is about 25% of the equation. Seems like a good amount to me.

That 30% is binomial randomness only ... other luck such as injuries, unpredictable player loss of effectiveness, etc., isn’t included in that 30%.  So the total “luck” is higher.

Assuming you want to count that sort of thing as luck.  Some might call it bad scouting, signing a player who gets hurt a lot or is about to have a weakness strip him of value.

Another factor is draft skill.  But how much of that is luck?  If none of the 30 GMs thought Albert Pujols was worth drafting in the first 12 rounds, doesn’t that suggest that his development couldn’t have been predicted?

Pujol’s development is the most shocking I have ever witnessed.  He hit well in the minors in 2000 (324/389/565) but that was in low A ball.  To make that jump to the majors with less than 100 AB between, improve every one of your rate stats and then prove its not fluke - I think its more shocking than even what the steroid boys did.

I suppose we could break performance down into 6 factors: 
1) Luck—performance vs. true talent
2) Talent due to payroll quantity
3a) Excess/shortfall of true talent relative to payroll for FAs—luck component
3b) Excess/shortfall of true talent relative to payroll for FAs—skill component (Moneyball pt. I)
4a) True talent of wage slaves—luck component
4b) True talent of wage slaves—skill component (Moneyball pt. II)

(Payroll size impacts one’s ability to retain good arbitration-elig players as well as signing FAs, but let’s put that wrinkle aside.)

Under this scenario, it’s easy to imagine that the total impact of luck (1 + 3a + 4a) explains something like 65% of variation.  In that case, if payroll size explains 18%, it would be about half of the factors a team (and MLB) could control.  In any case, it would be interesting to try to sort out the relative weight of the factors.