THE BOOK--Playing The Percentages In Baseball

Would it be fair to say that WPA is a quantification of what people mean when they say Youk is or isn’t “clutch”?

Though now that I think about it, perhaps what people mean when they say “clutch” can’t be decided by anyone but the person doing the speaking.

I would say that that is a fair statement, as WPA should track what people consider clutch.

Melvin: right, Clutch is like pornography… you can’t define it, but you know it when you see it.

***

One way to define Clutch is the way MGL defined a hot player: purely by the performance.  No one talks about Juan Pierre being a “clutch” player if he bumps his overall play from 20% below league average to league average in clutch situations.  Pujols is clutch because he is good.

Another way to define it is by comparing a player to himself in the same time period.  This way, you won’t have Pujols being called a clutch player if he has a .400 OBP with a .600 SLG in the clutch, if he also does that performance in a blowout.  He’s not clutch per se, just really really good, regardless of situation.

Another way to define it is by comparing a player’s performance level in high leverage situations to his true talent level.  This way, you don’t have to be restricted to looking at his non-clutch performance in the same time period.  A guy can be a true .350 OBP hitter, who hits .400 in the clutch.  That he happens to also hit .380 or .420 or .280 in the same time period in non-clutch doesn’t take away from what he did, compared to your prior expectation as to how he should have hit. Now though you get the chance that he not only turned in a clutch performance, but he turned in an overall better performance.

Your prior expectation however is constantly being revisited (after every PA actually).  So, he may have been a .350 at the start of the season, but by now, he may be treated at a .370 hitter, based on his performance-to-date.

This last one is the one I prefer. MGL I think prefers the second one.

MGL and I have long disagreements on this issue, and you can find it by searching for the word Clutch in this blog.

Fangraphs has a statistic called clutch, which measures a player against his own performance through the formula WPA / pLI - WPA/LI.

Right, that was actually introduced here and is definition #2.

You take the sum of his WPA and divided by the average of his LI for term #1.

You take each individual WPA and divided by the LI for that play… then add all that up for every PA.  That’s term #2.

The difference is his clutch score.  Ortiz was out of this world for those 2 years.  Pujols was unbelievable two years ago.  ARod s-cks -ggs.

In close, late game situations, WPA is not just measuring feelings, but providing a quantifiable value on how important those situations are. If managers had not felt those situations were different, the modern day reliever would not have been developed. With WPA, an objective number can be given to all those situations, allowing a manager to understand how important they are, instead of just relying on his feelings.

Cam, actually what you are referring to is LI (leverage index).

LI is the change in Win Expectancy (WE), expected at a given time, compared to some random situation.

WPA however has decided to take the change in WE and ASSIGN it to the player(*).  It is in that assignment that those who decry WPA speak up.  You can make a strong case that the assignment of the WPA should go mostly to some “timing/luck” bucket, and just a portion to the player, since the player doesn’t control where he is.  He just happens to be there when the lottery ticket was handed out. 

It’s a question of whether a monkey winning a million$ lottery should mean anything.  It really doesn’t, but then again, he’s the one who is actually living a different life, partying it up, while we are here writing about it.

Anyway, your basic point stands, but I just wanted to get the nuanced point out there.

(*)Hidden behind all the WPA, it can be approximated, more or less, as sum(WPA) = sum(LI * staticLWTS), for each play.  In the case of relievers, his talent level will allow him to have a higher LI than an average pitcher.  But still, do we necessarily want to count one of his PAs 5x, and another 0.7x, when talking about his talent level?  No, of course not.  But, he is out there collecting the lottery ticket.

To clarify a little, you could evaluate the situation/importance/value using either LI or WE(pitcher enters game) - WE(pitcher leaves game). If a pitcher throws a shutout in the 9th, he will accumulate a lot more WPA coming in with a 1 run lead, than a 6 run lead, obviously.

Before WE and similar concepts, there was no way to numerically measure the effectiveness of relievers, other than with saves then holds. Which then led to ideas such as tough saves and easy saves and later to your concept of LI. Now the value of relievers to their teams can be looked at by comparings their WPAs.

But my point was more about how the different game situations are not just subjective feeling states, but objective value states, which WE seems to do a good job of quantifying.

What WPA represents is the quantification of your feelings as the game unfolds.

I believe I originally penned that one.  Sam Hutcheson also had a nice way of putting it over at BBTF.  WPA tells the story of how a game unfolds, numerically.  When you pull out a player’s individual WPA, you’re quantifying his role in the story of the game.

I like to call WPA the “story stat” cause it tells the story of the game and each player’s role in the game.

Agreed that studes originally coined the thought.

I made a post at #18 here that simply summarizes this thread a bit:
http://www.baseballthinkfactory.org/files/newsstand/discussion/fan_graphs_cameron_rangers_mvp/

http://www.baseballthinkfactory.org/files/newsstand/discussion/tht_studeman_do_veteran_players_know_how_to_win/

More words here, which I will reprint:

Mike/13: thanks for the kind words. 

For those who haven’t checked, Fangraphs shows LI numbers for each play, game, and seasonal totals for all players going back a few decades (so far).  It’s really cool to see how the leverage numbers of pitchers change.  As an example, and focus on the column “pLI” near the bottom of the page: Keith Foulke

That 2002 season, where he was pulled from the bullpen (and you see it from his LI plumetting so much that season), and his clutch score being so negative.  But, his WPA/LI, which gives a better view as to how he was actually pitching, really shows him to be the same year after year.

That’s what WPA and LI deliver for you, something that the traditional lines don’t: the LI shows that he wasn’t used in as high leverage.  His WPA shows that he had much less impact than in his surrounding years, and yet his WPA/LI shows that he was really the same pitcher, and his clutch score shows that he pitched bad in some crucial situations, which probably led to his demotion.

***

Chris/16: “I think LI and WPA are flawed ways “

LI has nothing to do with your concern.  Your WPA concern is legitimate, but your LI one is not. 

***

Bob/21: agreed.

***

studes/25: agreed, it is harder to explain WPA/LI compared to WPA.

***

The issue of baserunning and fielding in WPA is one of implementation, not the framework.  The more granular you have the data, and the more programming you want to do, the more you are going to represent it to what it should.  Obviously, the hitter should not get penalized for the runner being thrown out.  That’s not a WPA problem, but either a data collection or programming problem.

***

As for the HR in the 1st or 9th: the assumption of win expectancy, WE, (as in life) is that everything in the future is unknown, and therefore, your expectancy is that average things will happen.  So, hitting a HR in the 3rd increases your win expectancy by, I dunno, +.130 (say from .500 to .630), because you have a certain expectancy of what will happen in the following 6 innings.  So, the WE represents how much you are willing to bet on the game’s outcome at that point in time, and the change in WE (i.e., WPA) simply pays off on that bet.

In the 9th inning, you are “all in”.  You may have say won most of the poker hands before that, but if someone goes all in, and he wins, you lose.

That’s all WPA is… an accounting of money.  In real-life terms, as studes put it, it simply tracks your emotions, and quantifies it.

So, I can see why WPA gets a bad rap… either because it is misunderstood or misapplied.  OBP doesn’t get a bad rap, even though it weights a walk and HR as exactly 1.  Its limitations are self-evident, and therefore, no explanation needed.  With WPA, we have to work harder at explaining what it is and what it isn’t.

Belittling WPA, as a stat, makes no sense, because it represents what it purports to represent.  Nothing more.

I’m not sure if I’ve got a handle on what WPA/LI is. Is the only difference between it and WPA that WPA factors in the importance of the inning while WPA/LI treats all innings the same?

In most cases, the average safe play is worth twice that of the average out play.  So, if the average out play is worth -.027 wins, the average safe play is worth +.054 wins.  If some other situation has the average out worth -.0405 wins, the average safe play is worth +.0810 wins.

With me so far?

What WPA/LI does is simply index it so that the “range” remains constant.  That is, the -.0405 and +.0810 are prorated down, so that the -.027 to +.054 range remains intact.

In this case, we divide the higher leverage plays by 50%.  If you got a +.0810 play when the LI is 1.5, you divide that +.0810 by 1.5 to get you +.054.

In effect, we are saying that you “earned” +.054 wins, and the other +.027 wins was “timing”, of which you had no control.

In baseball, the range is even wider, because the HR is worth alot more than a walk.  So while the average safe play is worth +.054 wins, the walk is +.03 and the HR is +.13.

Again, whatever it is you did, you divide by LI, so that you get the “deleveraged” numbers: each PA counts the same.

However, let me try to give an extreme example: bases loaded, tie game, bottom 9th two outs, aka walk is as good as a HR.

So, in this particular case, the relative value of each safe play is worth the same.  So, whether it was a walk or a HR, it gives you the same “earnings”.  This play could say give you +.54 wins, and you’d get that whether it was a walk or HR.  But, if this is a LI=10 play, then you divide everything by 10, and this play now gives you +.054 wins.  The rest is treated as “timing”.

So, WPA/LI tries to center things, such that the range is kept “normal”, while also using baseball knowledge, such that in some cases a walk is a HR, and they each earn you the same wins.

Tango, I responded to your second post at that thread.  A “Neutral WPA”, in which each game’s WPA bucket is spread among players after they’ve been credited with WPA/LI, seems like an interesting concept to me.

I replied back:

In some respect, I’d prefer to leave the timing bucket alone, because it is what it is. At the end of the season, you’d see how much of the whole thing was dependent on the timing.

On the other hand, WPA does serve a purpose, if you look at it from the pitcher. A pitcher is the one who puts himself in the position he finds himself. So, any bad timing, he’d have to answer to.

As for relievers, Mo sort of puts himself in LI=2 situations because he is Mo the Great.

You could in one sense carry two timing buckets, one for pitchers and one for nonpitchers. For the nonpitchers, they share the timing bucket, and for pitchers, they own theirs.

You can’t have a reliever with a 1.00 ERA who only pitches in mopup situations “earn” alot of money. He’s basically Steve Jobs in 1978… he’s not yet STEVE JOBS, even if he worked as hard in 1978 as he did in 1984. The money just wasn’t there. Or someone who keeps winning penny-ante pots, but is not put in the high-stakes games.

I posted the following at BTF:
http://www.baseballthinkfactory.org/files/newsstand/discussion/tht_beamer1/#3142413

==============================================
Seems we go through this every six months, and always on a Friday too.

As an added benefit to those who don’t want to RTFthread, I basically agree with the following statements already made:

It’s not even a stat, it’s a way to follow the ebb and flow of a bseball game.
...
WPA is measuring a lot of luck and team context. But that’s how the real world works.
...
Would you accept a compromise in the form of re-naming WPA as “Fan Happiness Added?”
...
It only has value in the instant the event occurs, since you can’t tell the future.
...
WPA/LI might have some actual value though.
...
WPA does exactly what it says it does- measures each play’s contribution to the team’s chances of winning. Like it or not, that’s something a lot of people are interested in.
...
It measures each play’s contribution to the team’s chances of winning at the time the play occured when you do not know what will happen in the rest of the game.
...
I like WPA when used on squiggly Fangraphs graphs that show how big the swing was when a team turned a one-out double play with a guy on third in the top of the eighth. I dig that. I just wouldn’t use it to decide who the best player was.
...
One of the DBacks blogs I read used to use those charts after the game for doing a play-by-play of each event. Adding a narrative. In that context it was good. Using it to assign value to a player, not so much.
...
Seriously. Fan Happiness Added.
...
Some people are interested in this kind of thing. Some aren’t. I don’t understand why the people who are not interested insist on disparaging WPA. You’re not arguing against anybody.
...
I think you’re asking too much from WPA. It measures one thing, it measures it in an unambiguous way, and it happens to be something that people find interesting. That’s enough for one stat.

I can sort of agree with this…

Let me put it another way, when a a stat says that the guy who hit a sac fly in the bottom of the ninth to win a 5-4 game contributed more to that victory than the guy who hit a three run homer in the first inning, that stat is useless in telling us anything.

...but that means then you don’t give Mariano Rivera’s 80 innings any more or less credit if they happened in the 9th inning of close games, than in the 7th innings of blowouts (that his team wins). So, you need to resolve how you can justify paying Mo and his ilk 10MM. 

But it still doesn’t answer why you would rate the player who hits the sac-fly in the 9th (in a 4-4 game), as more “clutch” than the player who hits the 3-run HR in the 1st (in a 0-0 game than ends up 5-4. The 3-run HR seems WAYYYYY more “clutch” to me.

...ONLY if you knew that the game would be 4-4 in the 9th inning.  If you didn’t know that, if there was a chance that the game would be 9-4 or 4-9, then your view of that 3-run HR would change.  Your view of that sac fly would not change.

people that tout WPA tout it as if it is a holy grail type of stat

Those people are in the minority.  They are not the ones to listen to any more than you would listen to people who scream insanity and say “God” at the end to mean that God-worshippers are morons.

I disagree completely with thise statement, among many others:

and has managed to hang around ever since despite all its uselessness being exposed over and over.

***

Larry/32: Sean has exactly that on his site.  I gave him my LI for exactly that purpose.

I think the thing that draws people to think so highly of WPA is that it can be directly derived from the game, so doesn’t seem as “detached” from the actual games as other (more insightful statistics). For instance, if people are following a game, and see that Albert Pujols’ home run added 23.1% to the Cardinals’ chance of winning, they see exactly where the numbers came from, and then will believe that WPA is the supreme performance analyzer.

While I agree that WPA mirrors a fan’s happiness throughout a game, it does have some purpose, as it roots out the Latrell Sprewell’s of baseball. Sprewell used to come into games in which the Knicks were down by 20, and score 15 points that had absolutely no impact on the game, but boosted his stats. Of course, the impact of scoring these 15 points shouldn’t be held against Sprewell, but these points are certainly not as valuable as a Michael Jordan buzzer-beating layup.

At its best, WPA is an excellent in-game tool, and can add insight to a box score, showing which players really impacted the team.

Also, I am confused on how win expectancy is determined. Is it based off of a computer simulation using Markov chains from Phil Birnbaum’s data, or is it an extension of run expectancy?

Extension of run expectancy. 

It’s explained completely in The Book.  You don’t have to buy it.  Check out Amazon (link at the top of any page on this site), click the “Look Inside” feature that lets you browse the whole book.  Do a search for “Markov Chains”, and it should be around there somewhere.

Now it all makes sense. I didn’t previously have a copy of The Book, but the sneak preview convinced me that I need to have a copy. Thanks.

Studes had a good compromise for WPA a long long time ago.  I made a brief comment on it, but I encourage all to take a look at it.  It’s at the bottom of this thread, and this was my quick comment on it at the time:

=======================================

http://www.hardballtimes.com/main/printarticle/long-live-baseball-analysis/

***

At the bottom of that is studes’ take on a WPA compromise.  It’s fair, but does have a chicken-egg feel (i.e., if Garrett Anderson hits 10 RBIs, that game is a blowout, so his performance wasn’t that important, but without his performance, it’s not a blowout, etc).  In any case, it’s a reasonable attempt at finding some common ground.

Here’s the quote from studes’ article:

Maybe you don’t believe there’s a difference between a first-inning home run and ninth-inning home run in the same game, but I believe there is definitely a difference between a home run in a one-run win and a home run in a ten-run win. In general, hits in close games have a bigger impact on winning than hits in blowouts.

To prove the point, I pulled all the WPA events from 2006 and calculated the average impact of each event on the win probability of the batter’s team. Not surprisingly, I found that the same type of event (such as a single) in close games has a larger win impact than that type of event in blowouts. Based on the data, I have estimated a standard multiplier for calculating the impact of a hit or out in a game with a particular victory margin. It’s still under development, but I don’t think the final version will look much different than this:

Margin Impact
1 1.38
2 1.13
3 0.97
4 0.86
5 0.76
6 0.66
7 0.63
8 0.57
9 0.51
10 0.47

You might call this a “Game Leverage Index,” as it shows that a single in a close game is worth about 40% more (in wins) than a single in a three-run game (1.38 compared to 0.97). And it’s worth nearly three times as much as a single in a game with a 10-run margin (1.38 vs. 0.47). The same multiplier can be applied to any type of event.

I was actually surprised that the differences weren’t greater, but this table makes a lot of sense to me. I think it could represent a workable compromise between those who want to value performance in games, but don’t believe that the difference between “when” an event occurred should matter once the game is over.

As I said elsewhere, I really like the idea behind it. It is similar to James’ “Victory-important RBI” or whatever he called it, where an RBI in a blowout counted less than an RBI in a close game. What he did was if you gave up 3 runs, then you needed 4 runs, so you proportion all the RBI the winning team got down to 4.

Studes’ concept at least satisfies the condition that a run in the 1st counts as much as one in the 9th, while at the same time making sure that ALL runs scored in a close games count more than all runs scored in a blowout.

I certainly can’t speak for all “WPA skeptics,” but I think Studes has identified a great compromise.