THE BOOK--Playing The Percentages In Baseball

Tango, could you give me the “talking to a 5 year old” version of your first point.  Are you basically saying that Robbie Cano’s true talent split w/RISP versus no runners on isn’t nearly as dramatic as his performance to date would indicate?

As tango pointed out, the true talent difference is statistically likely to be non-zero - but isn’t necessarily the 51-point difference that we see in the statistics. That said, it might be a 10-point difference, and it might be an 80-point difference. (Given the comment about a 0.5% chance of being zero - it is also very unlikely to be 1% as well.)

One more time when it would be nice if people would report confidence intervals on their analysis.

For the batting averages listed (the 51-point difference mentioned), the 95% confidence interval on the observed difference in proportions is (0.017, 0.085) - so between 17 and 85 points - using the Agresti/Caffo interval (difference in population proportions).

This, of course, does not take into account quality of pitcher differences between the ‘groups’, etc, but at least puts the non-zero difference in some perspective.

... and the CI for OBP is essentially the same - between 18 and 85 points.

Tango,

Regarding 1), yes, of course.  I tried to state that explicitly, but perhaps I didn’t get my point across:

For example, the probability that the 51 point difference in OBP is just a statistical fluctuation (i.e. that Cano’s true OBP in the two situations are the same) is about 0.5%.

Actually, the first part of the statement is vague, but I tried to clear it up in the parenthetical remark.

I was thinking of something else you wrote earlier in the post.  I agree that showing the confidence intervals (not just you, but EVERYONE) would simply make everything clearer.

Disclaimer:  I have not read the articles.

Ok.

These are all (most of them at least) Bayesian problems and we must know the “a priori” before we do or we analyze tests of significance.

We flip a coin from our pocket and come up with 10 head in 10 flips.  Is there any significant chance that the true rate of heads for that coin is more than .5?  No!  Why?  Because 99.999999999999% of all coins from my pocket are pretty much fair.

If Cano’s RISP splits are very large and that gap is statistically significant, but upon testing all other players we find that there is little or no RISP split “skill” (which is the equivalent of testing thousands of coins and finding out that they are all pretty much fair), guess what?  The large split we find for Cano means nothing.  We KNOW that it is pretty much a fluke, no matter how large.

There is some small chance that while we cannot find any observable skill among the population that there is indeed a small skill, in which case Cano’s split gets regressed a lot, but not quite 100%.

There is some chance that a few players in the population have a large skill, but the percentages of these players is so small that it is barely noticeable.  If that is the case, that still won’t change our answer (that most of the observed split is likely a fluke) since the chance that Cano is one of these players (with a large skill) is much smaller than the chance that he is NOT one of these players and that his sample split is a fluke.

That last point is like when a person not in a high risk category for contracting AIDS gets a positive AIDS test.  He is still like a 10-1 or 100-1 dog (not likely) to have AIDS.

Getting back to the batting order thing. Manages and commentators spend WAY too much time thinking and talking about batting orders, especially “true lead-off hitters.” Construct a good team, even if that means 8 middle of the order guys, and then put them in a reasonable batting order.  End of story.

Does any manager or GM seriously think that a lineup of 8 or 9 Manny’s or Mauer’s, even if you have to put one of them at leadoff, will not score a ton of runs?