THE BOOK--Playing The Percentages In Baseball

Don’t you have to account for the fact that the pitchers who throw in low leverage situations tend to be worse and once they get better, they throw in high leverage situations?

pm: that’s not the case for the upper-tiered pitchers. 

In any case, we don’t see the result of that anyway, that they pitch poorly in low-lev and good in high-lev, as a matter of bias as you are talking about.

You have two issues, as always, with “splits.” One, that is the average split.  In this case, if it is zero, than we can say that according to the evidence we have some certainty (the certainty dictated by the standard error of the sample) that closers, on the whole, have little or not difference in save versus non-save situations.  That is a relatively straightforward piece of information and probably not that interesting.

The second question is much more interesting and useful. And that is, “Is there any spread of talent as far as those splits go, and if so, what is that spread?”

In other words, it is a given that we will see some pitchers who have small and large splits and everything in between.  That does not tell us, however, whether those splits are real, or just a random fluctuation.

Now here is the critical part, which is what many, many people miss.  The fact that a particular player, like Valverde, has a very large split, does not tell us that that split “must mean something” (because it is so large) until and unless we determine what that spread of talent is, if anything at all.

Once we determine that spread of talent and quantify it, then, and only, then, can we determine for any individual player, how much of their observed splits is real and how much is random.  Essentially, we can determine how much to regress his observed splits toward the league average or the average for the population that that player belongs to (that average or mean, could be zero, or it could be non-zero, small or large - it doesn’t matter).

For example, if a RH hitter has a really large platoon split, it would be likely that most of it is random and that his true platoon talent is close to the mean (around 1.09 for OPS I think), somewhat dependent on the sample size (number of PA or AB versus RH and LH pitchers).  (When there is a small spread in talent the amount of regression is less sensitive to the sample size.) That is because we have determined that the spread of platoon talent for RH hitters is small.

For a LG hitter with a large observed platoon split, we would estimate that much of that is talent is some of that is luck, again, depending on the sample size.

This is what the, “Surely such a large split (clutch, day/night, home/road, pitcher/batter matchups, etc.) has some or a lot of predictive value” argument, which we hear all the time, does not necessarily hold any water, until and unless we figure out the spread of talent. If the spread of talent is small or none, as it is with many things, like clutch, day/night, home/road, RHB platoon ratios, pitcher BABIP, etc., then no matter how large the observed split, there is going to be very little true talent inherent in that split (and thus, predictive value), unless there is SOME spread of talent and the sample size is enormous.

So, until and unless we figure out the spread of talent among closers for save/non-save situations, for any individual closer with large splits, like a Valverde, the jury is out how much of that is “real” (and has predictive value) and how much is random fluctuation.

(In the interest of full disclosure, even without knowing the spread of talent, we can still estimate (and the uncertainty on that estimate is going to be high) the percentage of talent and the percentage of luck in any observed split.  The latter is generally going to be much higher, even for large splits, especially with small sample sizes, when we don’t know the spread of talent…