THE BOOK--Playing The Percentages In Baseball

Are 750 PA spread across 35 players equivalent to 750 PA for one player?

For all intents and purposes, I would say yes, as long as there is no inherent bias in the players (beyond the one you selected on).

Can we get results on how each team is doing overall, including non-clutch situations?  Isn’t that relevant?

If you follow David’s links, you’d get to here:
http://www.fangraphs.com/clutch.aspx?type=0

***

Now, overall… what if what fans find are guys that improve, not only in clutch situations alot, but also overall (at least a little).  Or the reason they don’t go for the overall better hitter is because they think he’ll get worse, not only in clutch (alot) but overall (at least a little).

It’s possible that what the fans are seeing is that they can forecast specific players better, not just in clutch situations, but in any situation.

To me, that still qualifies as using your guts to forecast clutch.  What if Joe Crede is a career .335 wOBA hitter, but in 2008, his wOBA in clutch situations is .370… but in non-clutch situations, it’s .360.  What do you do with this guy?  Are you going to suggest that something fundamental has changed with Crede?  His new Marcel forecast would now be say .345, so that .360 can’t be taken at face value.

But, Marcel himself said he was a .335 hitter… I can’t now penalize the fan for suggesting Crede is a top clutch hitter.  The fan was simply asked “who do you want with the game on the line”.  He picked Crede over someone who was clearly better.

My basic point is that I could have also asked “who is a much better hitter than people think”, and the answer would likely have also been Joe Crede.

The Fans may be telling us something very important here.  We’re not sure that we can necessarily ascribe it to clutch, or simply overall hitting improvement, or both.  But, they seem to possibly know something the monkey doesn’t know.

I was going to ask the same question as #3.  I agree that we can’t “penalize” the fans if they are doing a better job at forecasting players and that is why their team is going better in the clutch, but we certainly want to know if they are identifying clutch skill or are in fact, just doing better at forecasting players.

So, what is the answer?  What is the overall (clutch and non-clutch) performance of your team and the fans’ team?  And what did Marcel say about each group before the season started?

Marcel had a 20 point advantage to my team, around .365 wOBA against .345 for the Clutch team:

http://www.hardballtimes.com/main/article/with-the-game-on-the-line-i-want/

(Those stats includes IBB.)

As we know, a .345 wOBA implies a .345 OBP, and the corresponding SLG (which is probably around .420).  Pretty much, the Fans selected guys who were overall league average.

As it is, the Fans players are performing a bit better than expected, while my guys are simply tanking.

As for the “answer”, unless David creates an overall seasonal page for all those players, I’ll figure that out at the end of the season.

"We already knew that the Clutch skill existed, as per The Book.”

I don’t think The Book proves clutch skill exists because it doesn’t really define the term “clutch” very well at all. In fact, that’s the one area of The Book that I thought came up way, way short.

Obviously you have to cherry-pick situations to figure out what clutch is, but I could think of a bunch that weren’t included that I think are clutch situations. Similarly, I think The Book counted some things as a clutch situation that weren’t.

I think it’s reasonable that some players handle pressure better than others, but I still haven’t seen anybody even define the terms properly, so we still can’t say what effect (or how big) this actually has in baseball.

JD no on has to define clutch is any particular way, because there is no one definition of clutch.  That is not the answer anyone is looking for.

In The Book, or with this experiment, the question that is being analyzed is, “Given my definition of clutch, whether you like it or not, is there a spread of skill in the population?”

So yes, given our definition in The Book, whether you like it or not, we have shown that it is likely that a small clutch skill exists.

Is it possible that there is a spread of skill using one definition of clutch, but there is no spread of skill using another?  I suppose so, but as long as all the definitions are similar and reasonable, I highly doubt that that would be the case.

Would it be “easier” to find a clutch skill if it were defined as close as possible to how the players themselves perceive “clutch?” Well, yes, of course, by definition it would.

Can reasonable people argue over what the criteria might be (that the players themselves perceive as clutch)?  Yes, of course.

Is there anything that is even close to a definitive answer, as to what is the best definition of clutch for the purposes of finding a clutch skill?  No there isn’t.

Does it matter all that much, “it” being the definition of clutch, as long as it is reasonable?  No, I don’t think so, as I already have said.

Should we care about what you, JD, think is the best definition of “clutch” for the purposes of these studies?  With all due respect, no, I don’t think we should, given that if we asked a thousand different smart and informed analysts and fans, we would get about 1000 answers, and you would just be one of them.

Would it help a little if someone formed a “committee” of smart analysts and fans and came up with what they collectively thought might be the best definition of “clutch” for the sole purpose of finding a spread of skill if one exists to any practical degree?  Sure, why not.  But as I said, it does not really matter, I don’t think.

Would it help if we issued a carefully thought out questionnaire to the players?  Sure, I think it would.

But, bottom line is that I don’t think that saying that, “I don’t agree with your definition of clutch, and therefore I dispute your conclusions as far as whether there is or isn’t a spread of clutch skill,” has much merit, as long as the clutch definition is not out in left field.

MGL is right.  Even if we didn’t prove that a “clutch” skill exists, we did prove that a skill exists when looking at close and late situations.  Call it a Qualtu situation, if clutch is inappropriate or a loaded term.

As you know, I thought there were a lot of problems with the methodology for this study, and I think some of those problems are coming to roost.

First of all, let’s stop saying that the fans are better so far at identifying clutch.  Yes, they have simply picked the best player performance in clutch situations so far.  But whether that is because they have projected the players better than you (Marcel) have or not, overall, is another story.

The ONLY way you can measure clutch, of course, is to look at the difference between clutch and non-clutch performance.  That is the definition of clutch.  For their to be clutch, there has to be a difference in clutch and non-clutch performance.  Calling good performance in clutch situations “clutch” without comparing it to non-clutch performance, makes no sense.  No sense at all.  There has to be a baseline for something to be “good”, which distinguishes good from bad.

Until and unless we see that (the clutch and non-clutch difference) in both teams, let’s please stop talking about one or the other side being better at recognizing “clutch” skills.

Now, yes, in fairness to the fans, you did not ask them to choose players whom they thought would have the best clutch/non-clutch ratios or differentials.  Which was one of the major problems in setting up the study, at least in terms of it being a “clutch” study.

You simply asked the fans to choose which players they wanted up at the plate in clutch situations.  The fans could have chosen their players based on overall performance, whether they believed in clutch or not, or they could have chosen them based on overall and clutch performance.  We have no idea.  In fact, you would think that plenty of the fans thought that there were better clutch players on certain teams than the ones they chose, but they also knew that the better clutch players were not nearly as good overall, so they would bot have chosen them.  I am sure I mentioned that originally. In fact, where did we ever get the idea that this study was about seeing whether the fans had any ability to identify clutch players?

If you want to compare one group’s ability to identify clutch skill as compared to the other group, since it is too late to change the selection criteria, the least you can do is to show us each team’s clutch/non-clutch differential (or ratio).

What if the fans’ players are doing great overall but a little worse in the clutch and your players are doing terribly overall, but a little better in the clutch, such that they are killing you in both situations, which may be the case for all we know? 

Are you going to proclaim that, yes, the fans have an ability to discern clutch players?  I hope not!

Personally, I think that there is a tremendous rebound effect from PED use this year which is affecting a ton of players.  Consequently, I would not be surprised of two things:  One, that the fans can do a much better job at forecasting players this year.  Two, that the list of top players in past performance, and therefore in this year’s forecasts, is heavily biased by players who used PED’s in the past and will not live up to their Marcel’s.

I am reasonably convinced that the power shortage in the AL this year is real and that it is because of players no longer taking PED’s.  And that it just happened to be that many more players in the AL were taking PED’s.

The purpose of this study is to determine if Fans’ perceptions will lead them to select players who perform better in clutch situations than other players who are better overall hitters.

Whether we call that clutch or better forecasting or quatlu or whatever isn’t that important to me.  It’s simply that the fans “see” something about Joe Crede and Marco Scutaro and Derek Jeter that Marcel doesn’t, and that the Fans “don’t see” something about ARod et al that Marcel salivates over, when looking only at clutch situations.

That’s all this study has been designed to do.

That’s fine, but I object, and I think rightfully so, to you calling that “clutch ability.” It is not.

The name of this thread is “Does Clutch exist?” All I ask is that you call it and that you are impressed with what it is.  That the fans team has done better than yours in clutch situations in no way shape or form suggests that they “know” or can identify clutch hitting more than you can.  The only way to look at that is to look at clutch differential.  I just don’t see any argument there, Tango, I really don’t.

So while the purpose of the study is to see whether fans can...., please don’t then call that “clutch ability” or “clutch performance.” It is not.  Not any more than if I told you that my team of lefties had an OPS of .800 versus left-handed pitchers and last year, they were .750 overall.  You cannot say, “Wow, they are really having a great year platoon advantage-wise!” No, you have to see what they are hitting against RHP to make any statement about their platoon performance or skill.

So yes, the fans have performed better than you in clutch situations.  Agreed?

We have no idea whether the fans players or your players have performed better in the clutch this year.  Agreed?

If you agree with #2 above, then please let’s not invoke the word “clutch” other than in reference to the situation that you happen to be tracking.

It’ll be pretty simple for me to include what each player is doing overall including his Clutch wins on the season, the (WPA/pLI) - WPA/LI version.

Clutch for the fans is at + .71 wins, and for Tango’s team it’s at - 5.3 wins.

WPA WPA/LI Clutch
5.79 4.92 0.71 Fans
10.01 14.91 -5.36 Tango

Tango’s team has been far and away more productive, but the fans are still considerably more Clutch so far, by this particular measure.

Let’s go back to this clutch differential.  If Joe Crede is a career .335 wOBA hitter, entering 2008 (or whatever he is) and he happens to hit .370 in clutch PA this year, and .365 in non-clutch PA this year, why should I look at the differential as .005, and not, say .030?

After all, his career to-date wOBA is now .340 right?

The comparison should not be necessarily sample-to-sample, but sample-to-true talent.  If Crede is a true talent .340, and he hits .370 in the clutch this year, then he’s got a 30 point differential.

In fact, this is identical to your (MGL) hot streak study in the book.  You didn’t look at the player’s rest-of-season hitting right, but the surrounding years?  You could use anything, and really the more years the better, with the right weighting.

So, I reject the idea that we, necessarily, need to look at differentials as sample-to-sample within the same year.

As it is, I’ve got 36 hitters, who probably had something like 100,000 PA of .345 wOBA hitting.  If I ask Fans who do they want as their hitter with the game on the line, and they select these 36 hitters and they hit .370 wOBA in 750 PA, it’s against those 100,000 PA of .345 that I’m going to compare against, and not the 7500 PA this year.  (Well, now, I should compare to the 107,500 PA, with more weight on this year.)

I don’t see any reason to look at this as an in-year differential, if I also have 100,000 other PA to consider.

Whether the Fans are picking out a Clutch skill, or they are picking out a skill that shows an overall improvement, or they saw a skill improvement in Clutch that leads to overall better skills, or whatever it is, then that’s what it is.  That *could* be a fair definition of clutch skill. 

As it is, I am comparing the differentials.  I am taking 36 hitters who are a true talent .345 wOBA and 36 hitters who are a true talent .365 wOBA, entering 2008.  Right now, in clutch situations, the .345 hitters are whipping the .365 hitters.  They could very well be also whipping them in non-clutch situations. 

Clutch doesn’t necessarily have to be about differentials as we’re accustomed.  It could be an improvement in a hitter to such an extent that his hitting in crucial situations also elevates his hitting in non-crucial situations.

Otherwise, I’d have to tell the Fans that yes, the .345 hitters they chose did in fact best easily my .365 hitters in clutch situations, but I’m not going to count that as much, since those weak hitters also best my great hitters in other situations.  That the only way for them to win is to beat my guys in clutch, but lose to my guys in non-clutch.

I don’t see that as necessarily the way to look at it.

David/13: well, that’s mighty interesting as well!

You are really stretching the limit, Tango.  If a player going into this season was a career .350 and hits .410 in non-clutch and .360 in clutch, I, and I think most everyone else, would have a really hard time, saying, “Wow what a clutch hitter he has been this year!”

Bottom line is when measuring clutch skill in some reasonable way, it always includes some kind of differential.  If you want to measure performance in clutch situations without comparing it to anything else, that’s fine by me. That is, however, a related yet completely different animal.

The way you have it set up now is the second thing.  If you want to set it up to compare to last year’s non-clutch or career non-clutch to this year’s clutch, although that is more than a little odd, that’s fine too, but that is not the way it is set up.  It is not set up now to compare to anything, so until it is, let’s please stop calling it “clutch skill.”

You’ve given me an example of another way to call something clutch skill (measuring some period of time clutch performance to another period of time non-clutch performance, which is a little silly, BTW - using two periods of time - but I’ll give you the benefit of the doubt), but the way the study and results are set up now, there IS NO comparison to anything.  Until there is, there ain’t no “clutch skill” being measured.

As I said, if the fans do better at forecasting, and there is no such thing as a clutch skill, then they will win this competition (and you will apparently for some strange reason say that they beat you in “identifying clutch players").  Ergo, the way this competition is set up does not necessarily (it might of course) measure clutch skill.  Which is basically the same thing as I said when it was set up a few months ago.  I think the problems then are coming to roost now, which is why we are having this discussion.

But, I’ll pretty much stay out of it. I was not a fan of it before, and I am definitely not a fan of it now, no pun ("fan") intended.

Seriously, why not just use clutch differential for this year, if you are going to talk about identifying clutch skills among players?  Forget how they do overall.  That is just noise.  However, if you want to include how well the fans picked players overall, then don’t use any differentials.  But just don’t do one thing and then call it something else.

"Personally, I think that there is a tremendous rebound effect from PED use this year which is affecting a ton of players.” This is an interesting theory, but the time lag since the start of testing seems substantial.  Would it take this long to see a performance decline?  If an athlete stops taking PEDs, but continues working out, how long does it take for the PED advantage to dissipate?  2 mos?  6 mos?  18 mos?  I have no idea.  Has this been studied in other contexts?

Guy, I have no idea of course.  I think that the Mitchell Report and the publicity which accompanied it was a watershed for players not taking PED’s, much more so than any testing procedures.  That is why I point to this year as one in which production may be down due to less or a cessation of PED use.  It could be a gradual effect too, I suppose. I also think that many, many more players have taken PED’s than a lot of people think, and I think that PED use has had much more of an effect on power/performance than many people think.  Every time I see players like Giles, Pudge, Luis Gonzalez, Sheff, and a ton of others hacking away at the ball, I think to myself, “Yeah, not so easy to hit a baseball without chemical help, is it?” I could be all wet though.

The clearest, most objective and most comprehensive statistical definition of “clutch” (higher success level of performance in high leverage situations than in low leverage situations) seems to me the Clutch stat developed and calculated over at Fangraphs.  In a thread last year (http://www.fangraphs.com/blogs/index.php/is-wpa-predictive-for-batters) David Appelman at Fangraphs pointed out that when he he ran a season-to-season correlation of players’ Clutch performance, he got a correlation of .01.  If that result holds up over a substantial sample of seasons, that seems to me clear, definitive evidence that “Clutch” is not a skill but an unrepeateable manifestation of luck over a relatively small sample of PAs that disappears over multiple seasons.  What perplexes me most is that after Fangraphs wonderfully innovative work to develop the Clutch stat gave us the ability to do the correlation that David ran last year, and after he reported the zero correlation that he found, this topic is even an issue anymore.

birtelcom, as I’ve said, I don’t think you will get different results no matter what reasonable definition you use for clutch.  So it is no surprise to me that David found a low correlation.  No one has yet found any more than a low correlation using any definition of clutch.  I doubt that there is some definition of clutch hiding in the weeds such that if we tested it it would yield a high correlation.

The topic is NOT really an issue.  It has been shown again and again, that based on mere statistics, that we cannot identify players who have any kind of significant clutch skill, at least without using 10 or 20 years of clutch stats.  However, some people, including Bill James, think that there is some magical difference between “none” and “a little.” There isn’t.  Whenever we find no evidence of something like clutch, we really are saying that “the evidence suggests that little or no ‘X’ exists.” Whether it is nothing or a tiny amount is a technicality and cannot be answered anyway, using sample data, since there is always some uncertainty or sample error when analyzing sample data.  When Bill James or anyone else tells us, “Just because we can’t find something does not mean it doesn’t exist,” I say, “No shizit!”

That being said, all these tests are answering the question of whether we can identify ‘X’ from a statistical record.  Whether ‘X’ exists and can be identified from observation is another story. In some cases, it is a completely different story.

It would have been nice if the fans were given an opportunity to try and identify pure clutch hitting, as in some differential between clutch and non-clutch situations, but, as I have been ranting about off and on for several months, they were merely told to choose the players they would most like to have at the plate when the game is on the line, which, IMO, and I think in anyone’s reasonable opinion, means the best overall hitter in the fans’ opinion, although it obviously could include some clutch skill.

If you want to see whether fans can identify clutch skill, you simply ask them to make a list of the players they think have the greatest clutch skill.  Then you track the clutch skill of those players.  Period.  Simple.  I don’t know what Tango was thinking when he set up the experiment.

"Bottom line is when measuring clutch skill in some reasonable way”

Well, I don’t think I’m stretching it, and I think it is reasonable.  I find no problem comparing a sample against our best estimate of the true talent (which, by the way, includes all of his performance to-date… so your example is a bit off, but its general point stands).

I agree on the differential point being made.  I simply disagree that we need to make it a sample-to-sample comparison.  I find it more unreasonable to throw away 100,000 PA from 36 hitters.

The best way is to compare the clutch sample performance, to the Marcel estimate.  That’s the differential.

***

birt: you cannot find it in the performance numbers.  What I’m offering here is if fans can find it with their guts. 

That is, given that we cannot establish much clutch skills by looking at previous performance numbers, can we therefore find it in other places.  And, the other places is in scouting (observational analysis).

Just as you can only somewhat find fielding skills in the non-PBP data, and you can in fact find MORE fielding skills simply by watching the players (presuming again no PBP data exists), then observational analysis is legitimate.  The scout’s guts on a college player’s fielding will certainly trump whatever performance data you have on that player’s fielding stats.  Those stats will be filled with alot of biases and selection issues, things that a scout can filter out by wathcing the guy.

Same thing here.  We can’t find clutch by looking at the numbers.  So, let’s look at it a different way: can Fans spot those guys.

***

“I don’t know what Tango was thinking when he set up the experiment.”

At the risk of repeating myself, Fans selected as clutch hitters guys who were overall .345 wOBA hitters in their career (or, more precisely, MArcel’s forecast entering 2008).  I selected guys who were overall .365 wOBA hitters.

We would expect, therefore, that in any kind of “random” (non-biased) split, that the Fans’ hitters and my hitters will show a 20 point differential to my favor (given a strong enough sample size of course).

Since I specifically asked the Fans to select their hitters believing that there is a bias at the Leverage level, that’s what I’m tracking.

So, I *am* tracking differentials: I am tracking two groups of players that are distinct in their talent level and their perceived level of clutchiness.  The study is, in my view, perfectly acceptable.

Now, I can, and should, re-estimate the Marcel overall wOBA of these two groups of 36 hitters at the end of 2008, so that we have the best estimate of their true talent levels.  If, for example, all 36 of our hitters ended up with a .400 overall wOBA in 2008, after entering 2008 with a .345 wOBA, our revised true talent estimate for the Apr-Oct 2008 time period might end up being say .360.

But again, it’s the same thing: compare the sample at the split level being studied to the overall true talent that we’d expect if there was no clutch bias.

I refuse to compare the sample-to-sample clutch/non-clutch, if that means ignoring in totality the 100,000 career PA of our hitters.  To do so goes against everything we’ve done when trying to figure out the true park factors and true handedness splits, etc.  The comparison differential is, and will remain, as sample-to-true.  And the experiment was designed exactly as that.

Our disagreement is purely that you (MGL) prefers a sample-to-sample, while I prefer a sample-to-true comparison, where the true uses all career data to-date (weighted ala Marcel).

As I’ve said, Tango, the only problem with that approach is if the fans are able to do better than Marcel at projecting overall performance, it will look like they are better at identifying clutch, even if no clutch skill exists in the universe, no?

And, as I also said, your approach, with certain results, does not pass the “WTF” test!  The example I gave above is what I am talking about. If a player going into this season was a career .350 and hits .410 in non-clutch and .360 in clutch, I, and I think most everyone else, would have a really hard time, saying, “Wow what a clutch hitter he is!” Yet you will use that data (I assume) as evidence that we have found clutch skill.  WTF!

Well, an estimated true .350 hitter entering 2008, who we now estimate being a true .360 hitter entering 2009 ended up hitting .360 in the Clutch, then I will say that we found not Clutch skill.  But, if that hitter ended up hitting .380 in the Clutch, then I will say “yes”.  Even if that guy hit .410 in non-Clutch in 2008.  After all, we establish his true skill is .360.  What else am I supposed to conclude with him hitting .380 in the Clutch?

Given a large enough sample size.

Now, if you say that you have 36 such hitters, and, in 2008, Fans managed to take these .345 true hitters and make them hit .380 in the Clutch and .400 in non-Clutch, such that their true talent is now .360, then what we have found is that fans were able to find overall breakout players by their expectation of players being Clutch.

MGL, while I see what you’re saying, you’re just confusing things by throwing in their non-clutch talent in 2008.

As tango is saying, each player has a ‘true talent’ of X. If they hit over X in their clutch plate appearances, then that’s a manifestation of being more clutch than their talent would indicate.

By bringing Y (their non-clutch performance which is really a subset of X) you’re just confusing the matter unnecessarily.

My principal “beef” is that the fans were not asked to pick the players they thought were the most clutch, yet somehow this turned into a “clutch experiment.” I find that odd if not a little disconcerting.  But we’ve gone over that before.

As tango is saying, each player has a ‘true talent’ of X. If they hit over X in their clutch plate appearances, then that’s a manifestation of being more clutch than their talent would indicate.

While that may technically be true (edit: actually it is not, as I will explain later), it is a ludicrous way to present evidence of a clutch skill.  I’ll give my example one more time:

A group of players has an overall projection of .350 (wOBA or whatever) going into season X.  They hit .370 in the clutch in season X and .380 in non-clutch situations.  Their overall true talent estimate is now .360 (the .350 going in and the .378 overall, or whatever it is in season X).

We show the world that they hit .370 in clutch and .380 in non-clutch this year, but because the .370 is greater than the .360 overall, we proclaim them to be good clutch hitters!  You don’t see the problem with that?  If people didn’t think that sabermetricians were “nuts” before…

Plus, you have to say that these players are even better in the non-clutch, since they hit .380 in the non-clutch and they are .360 hitters overall.  Well, guess what? You can’t be good in the clutch, but better in the non-clutch and be a clutch player overall!  So, it is 100% completely incorrect to compare their clutch hitting in year X to our estimate of their overall true talent and leave out their non-clutch hitting in year X.  The proof is just what I said.  You cannot say that they hit 20 points higher in the non-clutch than their true talent (.380 to .360) and 10 points higher in the clutch than their true talent (.370 to .360) and proclaim them a clutch hitter! 

The problem with Tango’s methodology, and it is a problem - it does not work and is not mathematically correct - is that you cannot compare a sample from one year with the true talent estimate from that year and previous years.  What you have to do, if you don’t want to look at one year only, is to take the clutch from this year and combine it with the clutch from all previous years and do the same for non-clutch and then compare them.

But, you CAN’T do that for this experiment because both the fans and Tango were allowed to choose their own players and obviously they could choose players who already had a large clutch/non clutch differential.

When you are choosing players and then comparing one person’s choice with another, for obvious reasons, the only data you may use is the data accumulated after the choices are made.  Thus, the true talent of these players, using past data (before the players were chosen) can NOT be used in any way shape of form to make inferences about who had the “better” choices.

Since you can only use “post hoc” data, and you want to make inferences about “clutch talent” the ONLY thing you can do is look at THIS year’s clutch and non-clutch data.

Tango could NOT be more wrong about this. I am 100% certain and would stake my reputation (which isn’t much of a stake of course) on it. I’ll leave it at that.  Anyone else can pick up the debate if they want.

My principal “beef” is that the fans were not asked to pick the players they thought were the most clutch, yet somehow this turned into a “clutch experiment.” I find that odd if not a little disconcerting.  But we’ve gone over that before.

You’re correct, they weren’t asked to pick which players were clutch, but which players they thought would perform at the highest level when the game was on the line.

And since tango discarded the best player both by metrics and by fan support, you’re left with players that the fans feel are better when the game is on the line than whoever the Marcels choose. And that is for clutch situations.

The fans are saying that they would rather have player A with a wOBA of .345 over player B with a wOBA of .365 in that situation—if it weren’t for increased clutch performance, we’d have to guess that the fans simply aren’t good predictors of performance (only it seems so far that that’s certainly not the case).

While that may technically be true (edit: actually it is not, as I will explain later), it is a ludicrous way to present evidence of a clutch skill.  I’ll give my example one more time:

A group of players has an overall projection of .350 (wOBA or whatever) going into season X.  They hit .370 in the clutch in season X and .380 in non-clutch situations.  Their overall true talent estimate is now .360 (the .350 going in and the .378 overall, or whatever it is in season X).

We show the world that they hit .370 in clutch and .380 in non-clutch this year, but because the .370 is greater than the .360 overall, we proclaim them to be good clutch hitters!  You don’t see the problem with that?  If people didn’t think that sabermetricians were “nuts” before…

Why is that absurd? They are outperforming their expected performance in the clutch. Just because they’re outperforming it by more in non-clutch situations doesn’t mean the player isn’t clutch. I’m rather surprised since this is coming from someone who said, ”JD no on has to define clutch is any particular way, because there is no one definition of clutch” in this very thread.

Let me give a counter-example.

Player A has a true talent level of .350.
His performance is far better this year, raising it to .360.
In non-clutch situations, he has a wOBA of .380, but he has a BABIP of .400 despite a LD% of 20.
In clutch situations he has a wOBA of .370, but he has a BABIP of .250 despie a LD% of 25.

Do we blame him for poor luck on balls in play in clutch, and ignore the luck for balls in play in non-clutch situations?

The problem with Tango’s methodology, and it is a problem - it does not work and is not mathematically correct - is that you cannot compare a sample from one year with the true talent estimate from that year and previous years.  What you have to do, if you don’t want to look at one year only, is to take the clutch from this year and combine it with the clutch from all previous years and do the same for non-clutch and then compare them.

But, you CAN’T do that for this experiment because both the fans and Tango were allowed to choose their own players and obviously they could choose players who already had a large clutch/non clutch differential.

When you are choosing players and then comparing one person’s choice with another, for obvious reasons, the only data you may use is the data accumulated after the choices are made.  Thus, the true talent of these players, using past data (before the players were chosen) can NOT be used in any way shape of form to make inferences about who had the “better” choices.

So how, pray tell, should this be conducted? Or is it entirely useless and hopeless and not worthy of the effort put into it?

It seems to me like you want to ignore past performance and compare clutch performance (which you don’t believe needs to be defined, apparently) to non-clutch performance over the same time period. The problem is that people won’t select players based on what they think will happen in 2008 ignoring past data, but based on previous data (and assumed true talent levels) to decide which players to vote for.

Tango wants to see if fans can accurately predict who will outperform their true talent projection in the clutch, which would be a lot closer to what I’d expect to be studied by a study looking at clutch.

MGL is right.  The comparison doesn’t hold water if you are using different timeframes and different baselines.

You need to compare how the player performed in the given situation (aka however you define clutch) to how the player performed outside of that situation, and over the same time period. 

To expand on MGL’s example (We show the world that they hit .370 in clutch and .380 in non-clutch this year, but because the .370 is greater than the .360 overall, we proclaim them to be good clutch hitters!), you cannot, in this case, use past results to determine current efficiency.  You may be able to claim that the player is hitting better in the clutch now that he has in his career, but how they have performed in the non-clutch (this year) is just as important.  To disregard that is just wrong, and looks to me to be an attempt to legitimize a “clutch skill” using bogus logic.

If 2008 split “X” is better than career average, it’s worthless in proving anything other than 2008 split “X” is better than career average.

AGNP

In post 23 I concluded:

Now, if you say that you have 36 such hitters, and, in 2008, Fans managed to take these .345 true hitters and make them hit .380 in the Clutch and .400 in non-Clutch, such that their true talent is now .360, then what we have found is that fans were able to find overall breakout players by their expectation of players being Clutch.

Does anyone have a problem with this statement?

This would be, if you stretch it to its absurdity: “Tell me which players do you think will watch the finale of American Idol” and then end up with players who ended up with a seasonal wOBA that is 20 points above Marcel, so that you can make the claim that Fans know who will breakout, if they know who watches American Idol (given large enough sample size).

Now, I specifically asked: “Who hits better relative to his teammates with the game on the line” (or words to that effect), and I end up with players who, gasp, hit better relative to their teammates with the game on the line.  That is, in a head-to-head matchup of guys they think would hit better than their teammates in the clutch, they actually hit better in the clutch!

So, at the very least, we know teh Fans know *something*.  What that thing is, is up for debate.

Now, if those guys happen to also hit better in non-clutch, then at the very least, we can say that the Fans can forecast an overall breakout better than Marcel.

If those guys happen to NOT hit better in non-clutch, (which as David is asserting here is the case so far), then we CAN say that Fans are picking clutch hitters.

To summarize, given the data and results in hand (and presuming sample size will be our friend to the end), then the conclusion will be that Fans know clutch.

To summarize, the objection is one of an academic nature of what happens if the Fans’ hitters ALSO happen to hit better in non-clutch.  Since this is not even the current result, the objection is purely theoretical.  If this were to be the result, then at the very least, we can conclude that Fans know how to pick out breakout players, by being asked to pick Clutch players.

If those guys happen to NOT hit better in non-clutch, (which as David is asserting here is the case so far), then we CAN say that Fans are picking clutch hitters.

Correct, but it’s because of the differential between current clutch and current non-clutch performance, not because of clutch performance this year compared to career overall performance, which is what my point was (and I believe MGL’s as well).

To summarize, the objection is one of an academic nature of what happens if the Fans’ hitters ALSO happen to hit better in non-clutch.  Since this is not even the current result, the objection is purely theoretical.

It’s only theoretical in the sense that the current group of players selected fit that mold.  MGL’s assertion that differential over identical time periods is the important aspect of the comparison still is valid and is hardly “theory”.

If this were to be the result, then at the very least, we can conclude that Fans know how to pick out breakout players, by being asked to pick Clutch players.

Or that the original question posed to “The Fans” was ambiguous, and that it’s difficult to draw any meaningful conclusions off of it.

AGNP

The question was not ambiguous.  On the ballot, the only text that appears there is:
- Pos
- Player
- When the game is on the line
this hitter’s performance
relative to his teammates
will be…

That is it.  Every other text on that ballot is the player’s position, his name, and the dropdown.

I can’t believe there’s a single person that voted on that ballot that didn’t understand the question.

So, if the question is:
“When the game is on the line, Derek Jeter’s performance relative to ARod will be...”

If someone says “GREAT”, then we know he thinks Jeter will outhit ARod.

Nowhere there does it talk about “compared to non-game on the line situations”.  Therefore, the ambiguity would only be introduced by someone looking more into the question than is actually there.

Furthermore, in the introduction to the ballot, I showed this:

Note: select the guy you want at bat, when the game is on the line, regardless of how good or bad he is otherwise.

- So, even if you believe that Darin Erstad or Willie Bloomquist can step it up a notch or two with the game on the line, they’d still be several notches below the other players on your team. Unless of course you really think otherwise.

- And, even if you might think that ARod loses a notch as a hitter in pressure situation, and Jeter raises a notch, you have to decide if that is enough to vault Jeter over ARod with the game on the line. If so, then you choose Jeter as your clutch hitter, the guy you want with the game on the line. If not, then you choose ARod as the guy you want with the game on the line, that a substandard ARod is still preferable to an overachieving Jeter.

So, come on guys.  Gimme a break here.  There’s no way anything in there is ambiguous.  Only if you didn’t bother to read the intro page and didn’t bother to read the actual question.

The study is capturing who is better at predicting preformance in the clutch in 2008 (fans vs Marcel).  The study is not capturing clutch performance relative to non-clutch performance.  There is no assumption about whether clutch performance should be better or worse than non-clutch performance.  (It could be that the average hitter performs worse in the clutch than in non-clutch situations.)

The winner will be the side that picked the players that end up performing best in the clutch in 2008.  There are many possible reasons for one side to win over the other including:

There is a real clutch skill that fans can identify.
Fans are no better at predicting clutch performance than Marcel.
Flukey good/ bad year from one set of players.
More or more significant injuries in one set of players.
One set of players either improved or declined (real skills) more than the other.
Uneven distribution of pitcher skill / defense faced (in clutch situations)
Uneven distribution of park factors (in clutch situations)

Given the sample size what sort of differences would we have to see in the data to think we were seeing evidence one way or the other about the ability to ID clutch players?

Good recap.

Those rules could apply for Marcel v PECOTA v Chone v MGL for forecasting players too.

I selected 36 players on each side, and we’re expecting some 2000-2200 Clutch PA for each group by the end of the year.  1 SD = .011 wOBA.

The true talent level of the Fans’ picks, as our best estimate entering 2008, was .345, and my players was .365.  If the Fans end up with a Clutch wOBA of .370 and I end up with .365, that gives us a 25 point differential from expectations, making this 2.3 SD of difference.

That difference will stand as fairly significant.

If it *also* happens that this difference applies in non-Clutch situations, then we can say that Fans focusing on expected Clutch ended up really forecasting players better overall, in clutch and non-clutch. 

If the difference applies only on Clutch situations (and the non-Clutch will simply reflect the Marcel overall forecast), then Fans know what they are talking about and have provided a significant insight.

I expect this last paragraph to prevail, that any difference we do see between our two groups of players will be concentrated at the Clutch split level.

Lots happens in 2 weeks, enough that my team has slightly overtaken the Fans.  But at this point, we should call them neck-and-neck.  We are now at almost 900 PA.

Remember, with the Fans taking decidedly worse hitters (as per Marcel, entering 2008), any performance whereby the Fans’ players are equal to mine is a huge testament (in some capacity or other) to the Fans.

After 1000 PA, the Fans are slightly ahead.  1 SD = 16 points at this point.  Remembering that the Fans started at 20 points behind, we are still at just 1 SD.

Still not much, but the Fans sure are hanging in there real strong.  Just not “statistically significant” strong.  Yet.

After 1100 PA, the two teams are running neck-and-neck, with the Fans having a slight lead of 0.26 (unleveraged) wins.

However, if you look at OBP and SLG, the Fans have a more sizeable advantage, which translates to around a 9 wOBA points lead.

Noting that 1 SD = 15 wOBA points, and remembering that the Fans started with 20 wOBA points in the hole, that puts the Fans at nearly 2 SD above Marcel’s team.

After 1200 PA, we are in a dead heat in OBP, SLG, RBI, XBH.  1 SD is still 15 points in wOBA, and the fans are only 2 points ahead of Marcel’s team (and rembering that the Fans chose players that were, overall, 20 points worse than Marcel’s team, entering 2008).  The gap is 1.5 SD, which still doesn’t tell us much, but so far is a feather in the Fans’ cap.

Can you give us an update, and what they mean, since the numbers on the fangraphs site mean little to me.  You seem to be significantly ahead in OPS, WPA, WPA/LI, etc., but again, I have no idea what that means.  I think that WPA is capturing some kind of “clutch” but I have no idea.  In the posts above, you keep talking about wOBA and OPS in high leverage situations, but I don’t see any stats like that on the fangraphs page.  Is there?

Also, is there an easy way to get from the fangraphs home page to the page with the daily updated stats?  I don’t see a link from the fangraphs home page and if type “tango” or “clutch hitting project” into their search box, I just get a link to an old update.  I have to go to this thread to get a link to the daily updates.

I now realize that the numbers they give on the website ARE in high leverage situations only, although I don’t know what the cutoff is.  Plus, I am not sure what is considered “clutch.” The WPA in those high leverage situations?  The WPA/LI?  Just the wOBA or OPS alone?  In any case, it looks like Tango’s team is beating up the fan team by a decent amount.  I am not sure what the projected overall difference in OPS is supposed to be (between the fan and Tango teams), but Tango has a 10 point edge right now.  I think that the difference is supposed to be more than that, and if it is, there is a suggestion that the fans do in fact have players who up their performance in high leverage situations.  We’ll see, I guess.

Can Fangraphs post the players non-clutch OPS/OPS2 so we can be sure it’s a matter of the fans’ hitters “taking it up a notch” when the game is on the line rather than, as MGL feared, simply doing better overall than Marcel expected? That’s the fundamental question that needs to be answered.

What’s interesting is that the two groups of hitters have basically the same OBP2 and SLG (and thus an identical OPS once IBB’s are subtracted), but the fans’ choice have a significantly higher BA and fewer strikeouts. So people generally prefer contact hitters who put the ball in play rather than draw walks and hit for power. That’s what I’d expect since fans can not stand it when a guy strikes out with men on. For example, Mike Cameron was well liked while he was with Seattle, but he was still constantly criticized for being un-clutch simply because he fanned a lot. On the flip side, a guy like Darin Erstad gets raved about for being clutch. There is definitely a strong bias.

Yes, I noticed the K thing and thought the same thing - that fans hate guys who strike out a lot and automatically assume that they are not clutch.

It was all PA where the LI >= 2.0, as noted in the original article.

The two best stats to focus on are OBP2 (which is OBP with IBB removed) and SLG.  These two are fairly identical.  The Marcel forecast, coming into 2008, was the wOBA of the Fans’ choices was 20 points lower than the wOBA of the better choices (Tango team).  20 wOBA would be equivalent to 20 OBP and 25 SLG points.

With 1300 or so PA, 1 SD in wOBA is 14 points.  So, we can see that for the Fans’ players to bump up their performance in clutch situations shows around a 1.4 SD of difference.  It doesn’t pass the “statistical significance” test, but it is a pretty big feather in their cap.

I asked David to also show the performance of these players in non-clutch situations, and he said he’d do it a little later.

The conclusion will either be:
Fans were asked to select players who were clutch and pitted against players who were true better overall hitters, and those fans’ hitters did in fact hit better in the clutch…

a) ...while both groups of hitters maintained the 20point wOBA gap in non-clutch situations, so Fans really did a great job in picking out clutch players

b) ...AND non-clutch situations, so Fans really did a great job in picking out breakout players.  That to them, a guy who they *think* is clutch will, eventually, become an overall better hitter, so that his non-clutch game will finally come up to his clutch game level.

Regardless of the conclusion drawn, the Fans come out smelling pretty good here.

***

And the Fans clear bias against K players may be justified.  After all, we know that the run value of the strikeout jumps like crazy with a runner on 3B and less than 2 outs.  It’s possible that the Fan will think that a guy with high K won’t have the aptitude to be able to change his hitting approach to tailor to the situation.  That could be their reasoning.  They could be right.

The fans also picked players with fewer walks.  Walks in high-leverage situations never get noticed and rarely immediately score runs.  But the small change in WP adds up over all those walks.  I wonder, with the similar overall lines of the fans’ team and Tango’s team, if the difference in total WPA can be attributed to the WPA from walks.

In general, I’d say fans intuitively see clutch ability as success in driving in runs (RBIS) versus failure (striking out or other outs) with walks as a push.

I’d be surprised if this had more than a tiny effect, but is there any way for David to factor in the difficulty of pitchers faced in the clutch situations? And the handedness?
Tango’s group is made up of somewhat better hitters with significantly more power - is it possible that they were more likely than the other group to face the opponent’s best setup man or a lefty/righty specialist because they were perceived to be bigger threats?

Tango’s group is made up of somewhat better hitters with significantly more power - is it possible that they were more likely than the other group to face the opponent’s best setup man or a lefty/righty specialist because they were perceived to be bigger threats?

That is a good point, but if that were the case, it should show up in their past histories and their projections as well.  Remember that projections are based almost exclusively on past history for veteran players and the strength of their opponents are already included in that past history and therefore the projections.

I don’t see wOBA in the web page.  Again, is there a way to navigate to the “clutch project” daily update from the fangraph home page?  I don’t see one and, as I said, if I try to use their search feature I don’t get there either.

OK, since your team is supposed to be substantially better overall, at least according to Marcel, it looks likes the fans were able to shrink that margin by some amount, but isn’t the bottom line still WPA, and you are over 5 wins ahead, which is quite a bit, no?  So you are killing the fans pound for pound, no?  After all, the fans were allowed to choose whomever they wanted.  They didn’t have to choose players who were expected to be a little worse overall.  If the season ended today, while we could say that the fans may have some ability to identify clutch players, at least as compared to a Marcel (which may be biased), we could also say that the evidence points to a conclusion that we would still be much better off picking the overall best players rather than the ones that the fans really want, to the tune of around 5 wins (in WPA) in around 1300 high leverage (>2.0) PA (which is around 10% of all PA, I think).

We already know that clutch exists to a small degree. I am willing to even concede that fans can do better than a player’s clutch stats can.  But when we would actually do better than a basic Marcel by letting the fans choose any players they want in high leverage situations, give me a call!  I won’t be waiting by the phone, BTW.

WPA/LI would be the better metric to look at.

For WPA, some PA “count” as 10 and others at 2, for an overall average of 3.  That could be fair, but it also might not if the total number of opps (PA) is only 1300.

All I asked is to identify players with LI greater than 2.  I don’t think we should therefore weight some PA more than others in that classification.

What the WPA points out, compared to WPA/LI, the substantial gap here, is that my team must be doing super fantastic with the LI greater than 4 or 5 or so.

MGL: regarding the Clutch page on Fangraphs.  David did that mostly as a favor to me, and really doesn’t fit with anything on his site.

If you use Firefox (3.0), you should just type in the address bar:
Clutch Fangraphs
and it will appear as the number one link

Firefox is a fantastic browser, and makes life very easy.

David has come through, and he has posted the ALL SITUATIONS sheet right here:
http://www.fangraphs.com/clutch.aspx?type=3

In terms of WPA/LI, my guys are +35 wins and the Fans are ZERO (i.e., league average), after 13,000 PA.

Now, I hope you just stopped and had a heart attack here.

Overall, my guys, excluding IBB are: .353/.480
The Fans, overall: .335/.427

That’s around a 25-30 point difference in wOBA, even more than the Marcel forecast of 20 point difference overall.

And yet, when you look at the 10% PA of the highest leverage situations, both groups are much closer.  Right now, my guys have around a 5 point lead in wOBA in clutch situations.

My guys went from .353/.480 overall, to .339/.444 in the clutch.

Fans went from .335/.427 overall to .337/.428 in the clutch.

Since the average player loses about 5 points in wOBA in the clutch (faces tough relievers, etc), for the Fans to have a tiny tick up is good.  And for my guys to just tank is not good at all (or it’s good from the Fans’ perspective of making them look smart in ignoring those hitters in favor of “lesser” hitters).

Now, are we all impressed?  Just a little?

By the way, I’ve never felt so happy at the prospect of losing!

Andy said in The Book that based on his numbers, we should expect 1 SD = 6 or 8 points or so in wOBA, so finding 30 players that have a clutch wOBA of +10 and 30 players that have a clutch wOBA of -10 is pretty consistent with this.  (Remember, my hitters had an overall wOBA of 20 points higher than the Fans, but now look to be equal in clutch situations.)

But, Andy, when trying to pinpoint individual players, had a heckavu time finding guys with even 1 or 2 wOBA points (basically, too much noise in the numbers).

If Fans can really pick out the clutch players (and right now, at 1300 PA, it’s still too early to tell), that gives the Fans a big feather in their cap.

At the same time, both sides will be able to agree that there is a limit to clutch, and that no one expects or acts as if it’s any more than around 20 wOBA points.

If all this plays out, perhaps this will be the final chapter in the clutch discussion.

OK, the fans identified players who were just a little better than expected in the clutch, around 6.5 points, if I am reading the data correctly (we expect 5 points of wOBA less, and their players are around 1.5 points more).

The fact that they “avoided” your players means nothing whatsoever.  The only thing that matters is their players’ clutch differential as compared to the average player.

I am not going to jump up and down over a 6.5 point difference in wOBA over 1300 PA (what is the SD in 1300 PA again?).  Might not even be statistically significant.

However…

The more interesting and troubling thing is why Tango’s team, which are presumably the best overall players on each team, is doing so poorly in the clutch this year?

Again, the fans should not get credit for Tango’s lousy clutch players, but I find it odd that they are doing so poorly.

Is it possible that the best players overall have high clutch/non-clutch differentials, at least in wOBA or OPS because they are IBB’d or pitchers around so much and IBBs don’t count?

Tango, can you tell us what the historical average clutch differential for players in that talent range is? 

Could it have something to do with the fact that Tango’s players are high K, high BB?  What is the average clutch differential of a high K, high BB player?

Inquiring minds want to know these things!

Actually, I will give credit to the Fans for avoiding the better hitters like ARod et al.

As for the profile of the batter, check out BP’s BBTN where Nate Silver’s clutch chapter was focusing on those kinds of players.  I seem to remember him talking about the Mark Grace’s of the world being better clutch hitters simply because of their profile.

Is this the secret to the Angels’ success?  They’re known as a low-K, low-BB, high-AVG offense.  Does this give them an advantage when it matters most?

After 1400 PA, my players are performing at 6 wOBA points ahead of the Fans in clutch situations, and 28 wOBA overall.  The Marcel forecast entering 2008 was that my players were 20 wOBA points better overall.

1 SD in 1400 PA is 14 wOBA.

Depending on whether 20 wOBA is our best estimate or 28 wOBA, or somewhere in-between, being only 6 wOBA ahead is somewhere between 1 SD and 1.6 SD.

Still too close to call it statistically significant.  But, enough for us to stand up and take notice.

By the end of the season, 1 SD will come in at 11 wOBA.  If the Fans can continue this pace of being just 5 or so wOBA points behind my guys (or even get ahead), then we’ll have broken the 2 SD “barrier”.  Even if they are at 1.8 or 1.9, that would still be enough for the Fans to own bragging rights.

"Intuitively” it seems like certain types of batters are or would be better in clutch situations.  It could be our minds and eyes playing tricks on us, but maybe not.

Last night in the Philly game, they had bases loaded and 2 outs in the 9th, down by 4 runs.  The Dodgers closer, Broxton was on the mound.  Utley was at the plate.  Broxton was trying to avoid throwing Utley anything real good to hit (for fear of of the game-tying HR), especially the fastball.  He kept throwing Utley low sliders under the hands.  Broxton of course is a very tough pitcher to hit.  His biggest weakness is his wildness, which in this case is not such a negative (actually not the guy you want to bring in at the beginning of an inning with a 4-run lead).  He throws 96-98 with a 90-91 mph slider.

Anyway, Utley was laying off the real low sliders and fouling off the marginal ones.  He was also able to foul off the high fastballs which Broxton mixed in a couple of times.  Finally Utley got a bloop hit.

Next up was Howard.  He is overall a slightly better hitter than Utley.  Obviously he strikes out a lot, but that makes no difference in this situation (2 outs).  But, intuitively, you had a feeling that this was a much better matchup for the Dodgers.  That Broxton would get him out on low sliders.  Howard is basically a swing hard at everything type of hitter, just in case you make good contact (which works for him).  Sure enough, Broxton started him off with a low slider. Howard swung like it was a fastball and missed it by a foot.  The next pitch was also a low slider and Howard swung again over the top and hit a weak game-ending grounder to second base.

Maybe there is something to the savvy, contact hitter, good-eye, low-K batter (Grace) being somehow able to come through in the clutch more than the high-K swing at anything just in case you hit it type of player.  As another example, think of Jeff Francouer.  He is not a good hitter but he is not a bad one either.  About league average (worse for a corner OF of course).  He gets his hits and HR occasionally.  But it seems like with the game on the line, and with a good pitcher on the mound, you just throw him junk out of the zone and he gets himself out 80% of the time.

Of course all of this may be just “fanspeak” on my part, but you never know.  I would like to see some research in this area.  It is one that has always intrigued me.

Since Sean shows the batting lines of players by Leverage (LI >= 1.50 is high leverage on his site… gets you about 20% of the PA), this should be a real snap for any saberists out there.