THE BOOK--Playing The Percentages In Baseball

I’m really glad you guys did the study on clutch hitting that you did, because I was always uncomfortable with previous methods of proving/disproving clutch ability.

I hear all the time that if clutch ability was a true skill, the same players would be clutch year over year. And people point to the low year-to-year correlations in “clutchness” as evidence that clutch hitting does not exist.

So, I did an experiment. Everyone agrees that getting on base is a true skill, and different players have different OBP ability. So, a few weeks ago I took it upon myself to look at two random months from this season (I think it was June and July) and see what the month-to-month correlation in OBP was. The correlation I got was less than .10 if I remember correctly (I dont remember the exact numbers, and I cant find the spreadsheet with the data).

I figured that however you define “clutch” plate appearances, it can’t reasonably account for more than 1/6th the total number of PAs in a season, or about a month’s worth of PAs. Thus, a low year-to-year correlation for clutchness only disproves the existence of clutch ability if the low month-to-month correlation of OBP disproves the existence of differences in OBP skill.

I thought that the methodology you guys used in The Book was way more analytically sound than previous studies.

The correlation is dependent on many things, two of the most important is: underlying population skill spread and sample size.

Let’s look at OBP.  r=.50, when you have 200 PA in each of your “before-and-after” groups.  So, if you take say April/May and compare to June/July (or better, take 200 random PA and 200 other PA), and do a correlation of OBP, you will get r=.50 (more or less).  If you don’t get it for 2009, then do it for more years.

The shortcut equation becomes r = PA / (PA+200)

So, if all you have is 100 PA in each of your groups, then your correlation of OBP will be roughly r=.33.  If you have 22 PA, you will get r=.10.

For clutch, you need around 5000 (total PA, of which 500 will be clutch PA) in order to get r=.50. 

That’s the reason you can’t really find it, or make any decisions on it.  If you need 10 years of performance data to know clutch as much as two MONTHS of performance tells you about a player’s OBP talent, you can see why it’s pretty tough to rely on the numbers to figure out clutch.

Basically, throw the numbers out the window, for all practical purposes.  If you think you know clutch, only base it on your guts.  The problem is that managers and fans will look at the numbers, the very numbers that are clouding the issue.

Btw, BPro’s BBTN (Woolner/Silver) also did an excellent job of find a correlation.  r was .33 or something.  But, they split a player’s entire career, so their PA total was huge, probably 3000 PA in each group.

So, again, this would similar to getting an r=.50 if you had 6000 PA.

Clutch skill is there, it is detectable, it exists.  But, it’s hard to find for any individual, and it’s not actionable.

Bill James can call it “fog” and JC can say whatever he wants to say.  But, the reality is what I said.

My own contribution to the cause is linked off my name.  Basic findings: At a sample size of 5000 PA, I got a split-half correlation of about .65.  So, we can probably talk about clutch careers, but for practical purposes of picking who will be most clutch next year, the current WPA - WPA/LI definition is inadequate.  Now, maybe if we looked at clutch from a different vantagepoint, or with a new statistical formula, it might stablize, but as of right now, the proper thing to say is that given our current conceptualizations, clutch hitting is not stable ("skill based") from year to year.

Pizza was marked for moderation.

***

His key table is this:

And if we use my handy-dandy shortcut, we get this as the “PA to add” to get that r.

So, if you do:
r= PA / (PA + 4747), you get r=.174 when PA=1000.

As you can see, the value is pretty much around PA_to_add of 4000-5000 (depending which dataset of players you’d use).

With only 20 players in the 5000 PA camp, that .656 is hardly reliable.

If we make our regression equation as:
r= PA / (PA+4500) across the board, we get this level of r:

Compare Pizza’s empirical data from his given sample, to my quick estimate.  That pretty much nails it no?

Considering that all such tests get 4000-7000 of league average PA to add, that’s pretty much the limit of what you’ll EVER find.  Pizza’s probably got the best correlation to date, vis-a-vis sample size used.  That is, his might be the most sensitive of the attempts so far.

Pizza may get a higher correlation because he’s measuring performance across the full LI spectrum.  A hitter is considered clutch if he performs equally well in mid- and hi-LI PAs, but worse in low-LI PAs (if I understand clutch definition).  Most clutch studies just compare hi-LI (or alternative definition of clutch situation) to all other PAs, which of course is how most fans think about clutch (not “yay, my favorite player hit .190 in blowouts!").

One question is whether this picks up some aspects of how pitchers approach different types of hitters, as well as any hitter talent that may exist.  For example, are power hitters more likely to face good pitchers, or platoon disadvantage, in high-LI PAs (compared to their low-LI PAs)?  This could be a problem for any clutch study that doesn’t control for pitcher handedness/quality, but may be more so with this approach. 

If Pizza’s study covers many years, that can paradoxically be a problem too:  over the last 3 decades clutch hitting has been in a secular decline (more use of relievers), so that will create some correlation among the split-samples based on a player’s era. 

I also wonder about LHH vs. RHH.  I would think LHHs might benefit some in the clutch because there’s often a baserunner on 1B, and because so many closer are RHP they may enjoy the platoon edge more often (then again, use of LOOGY’s may offset that).  Probably not a big issue.

I agree with Guy that you have some “non-clutch” factors which may contribute to “clutch-looking” data in these studies when you don’t control for platoon, handedness, and relievers.  So the likely clutch skill is probably less than all of these studies suggest.  Again, as Guy says…

Let’s say that there were no clutch skill.  In high leverage situations, managers will tend to bring in a same-side reliever against the better hitters only. That will appear to make the better hitters look less clutch than the worst hitters, thus a “clutch skill” will be found when none exists.  Same thing for good lefty hitters. Managers will tend to bring in LOOGY’s against them in high leverage situations, making them look even less clutch than good RH batters.

Tango, did you think about that or control for that in any way in the Fan clutch experiment? 

IOW, if you choose the best hitters and the fans choose who they think are “clutch” players but not necessarily the best hitters, did the best hitters face more good relievers in high leverage situations and did they lose the platoon advantage more often in high leverage situations?  And are there more LH hitters in your team than in the fans’ team?

No, I did not control for the quality/handedness of opposing pitchers.

If we can identify (after 7000 PA, or whatever), specific players as clutch or not clutch, how well do the conclusions observers jump to after one or two seasons line up with the eventual results?  Is there a tell that observers are keying on that can be seen if you are watching the approach rather than the outcomes?  Or is the reputation purely outcome driven?

If clutch is really a skill, how would we identify where that skill peaks in the 10 year plus window we use to identify it?

How long does it take for reputation to catch up with change?

This is good stuff!!!!!!

Tango, how did you come up with the .020 figure in the post, and what does it mean?  Is .020 the observed *belief* in the SD of clutchness?

Right, that is the belief, by the fans.  Fans behave in such a way that they will trade away 20 points for a better hitter in order to get the clutchier player at bat.  That’s the same tradeoff as the platoon advantage.

That’s the extent of how much clutch fans think there is.

Discovering clutch hitting by looking at individual hitters is naturally going to be something that takes a lot of plate appearances to judge, and by the time we actually discover that someone has been clutch, he may not even be clutch anymore. 

I have thought for a while that certain *types* of hitters are more or less likely to be clutch, though.  I did an article a while back when I wrote for The Good Phight blog (http://www.thegoodphight.com/2009/1/29/741980/there-is-clutch-or-the-cas) in which I discussed a byproduct of the shift.

Lefty sluggers tend to get shifted against so that defenses can cut down on their BABIP.  With runners on base (which are naturally higher leverage situations), defenses are not able to do this as they need to hold on runners, be able to turn double plays, etc.  Thus, my theory was that lefty sluggers would have better BABIP with runners on base than righty sluggers.  I looked at the top 20 hitters in SLG with 3000 career PA or more (as of a year ago when I did the study), and split the group into 8 lefties and 12 righties.  They had similar HR and SO differences with runners on vs. bases empty, but the LHB had much higher BABIP.  The BABIP gap with runners on was only .009 for the 12 righties and .021 for the 8 lefties.  The t-stat on this difference was 2.20, which is pretty damn significant. 

To me, this seems like a far more practical way to determine clutchness.  You can wait until Ryan Howard and Prince Fielder are 40 to look at their clutch stats, or you can say that lefty sluggers rip a lot of hard hit balls between first and second base, and that will be relatively tougher to defend with a couple guys on and expect higher BABIP as a result. 

I’m sure there are many other contextual reasons that some people will have structural clutchness rather than other clutchness.  Lefties’ ability to hit with a larger hole on the right side in general with men on first can be considered a structural clutchness too.  Hitters who hit pitchers with hard fastballs (more typical of closers who pitch in higher leverage situations) will probably be clutch too.  That seems like a way to determine clutchness scientifically without waiting until someone has retired to retroactively crown them.

Tango:  have you ever created a situational hitting metric, analogous to “clutch?” I guess it would be something like RE24 minus (RE24 divided by RE-LI)?  It might be interesting to examine how much of a repeatable skill situational hitting is.  It seems likely to me that most if not all clutch skill is really a situational hitting skill.  Perhaps that skill could be identified more effectively if looked at it isolation from the rest of clutch (though you probably still need very large samples).