THE BOOK--Playing The Percentages In Baseball

You’re so Canadian.  It’s $8000, not 8000$.

This makes a great general point, however.  If you accumulated a lot of money, for example, you can get a higher interest rate on your savings, since it’s unlikely you’ll need to move around the large amount of cash often, and the banks want to compete for your money.  One reason the rich get richer.

Not sure if it’s a CDN thing.  Just my thing.  If I read “eight thousand dollars”, I like to see “8000$”.  But, I’m not a stickler for it.  8000$, 8000CDN$, $CDN8000, CDN$8000.... I don’t know what the standard is.

In a complex method called “chaining” (which may have originated with Patriot, or with me I can’t remember, but did originate back at the old Baseball Boards RIP), we try different kinds of gamblers because SOMEONE has to bet on that horse.  It’s going to happen.

Well, no. It’s not inevitable. It’s possible that horse could break his leg in an earlier race because you’re letting an overweight jockey ride him around and saving your right-sized jockey for the main race.

Now, yes, once you assume that the quantity of late-and-close chances is fixed (be it saves, one-run games in the ninth inning or whatever) then the lottery ticket model makes sense. Someone has to pitch those close spots. But they aren’t fixed - that’s the whole point of the article.

The question, to me, is not who’s winning the most lottery tickets but who’s producing the most potential winning tickets.

And this is what I presume you’re talking about when you say chaining:

http://www.insidethebook.com/ee/index.php/site/comments/valuing_relievers/

http://www.beyondtheboxscore.com/2009/4/29/856308/bullpen-chaining-and-reliever-war

Which doesn’t address the point I’m trying to make at all.

"Well, no. It’s not inevitable. It’s possible that horse could break his leg in an earlier race because you’re letting an overweight jockey ride him around and saving your right-sized jockey for the main race.”

This is about Gil Meche, Trey Hillman, and Joakim Soria, isn’t it?

Nothing is fixed of course.  It’s a question of expectations.

***

Yes, post 6 and post 28 of my thread is what I am talking about.

***

Colin: I’m not sure of the implication of the point you are trying to make.  Can you use an illustration to see how Rivera or Hoffman is going to look based on your point?

With respect to chaining history, I believe I introduced it with respect to reshuffling innings on a pitching staff (in the context of baseline/replacement level issues), and Tango and others then applied it other things (like leverage).  That application was definitely not me, since I’m in Colin’s camp on that--both leverage and WPA are real-time value constructs, and I don’t think we should have to restrict ourselves to that particular time perspective.

If we have to setup an official position, mine is the following: give each reliever an implied leverage based on the chaining principle, and tied to his true talent level.

So, a pitcher who gives up runs at 50% of league average would get an LI of say 1.7.  A pitcher at 70% of league average would get an LI of say 1.4.  If he’s at 100%, he gets an LI of 1.0.  If he gives up runs at 125%, he gets an LI of 0.7.

Something like that.

The way I think about it is very similar to the way you’d see hockey players.  Think of the #1 defenseman, how he plays on the power play, the penalty kill, and the 30 minutes of ice time he’s given.  It’s predicated on his (perceived) true talent level.

There’s a little fog in this one for me.

I think that WPA shows a contrast between how “good” a reliever is, and how much “value” he adds. Yes, there is bias in how a reliever is used, and yes, an incomplete picture of his “skills” is painted. But do we actually get an erroneous estimate of how much value he added to the team, by being used in that way? Or do we get a correct value?

Let me try to be clearer. When setting a lineup, you then don’t have any control over when you want a guy to hit. If you enter the ninth inning down by one, you can’t choose to have your 2-3-4 hitters come up to hit. You can, however, decide who pitches at any point in the game.

Suppose you could do the same for hitters though. Suppose that whenever there is a RISP opportunity you can bring up 2008 Justin Morneau (under the assumption that hitting with RISP is an actual skill), who had a huge RISP/empty split. If we could do that, it wouldn’t change how “good” Justin Morneau is, but it would definitely improve his value to our ballclub.

So what I’m saying, in a very convoluted way, is that if we use WPA-based evaluations of our closers/relievers we surely get a “false” depiction of their ability level as opposed to their actual value. But we get a better picture of the value they bring to the team. In other ways: Soria isn’t as good as Halladay, and he isn’t as valuable in a context-free evaluation. But they had roughly the same WPA (according to FG) last year. Can we claim, counter-intuitively from my point of view, that due to their respective patterns of usage, they brought the same net value to their respective teams? I would think that Soria is much easier to replace than Halladay, but regardless (and even taking into account luck-based opportunities and the work of his team-mates) he did, in fact, produce that WPA. A little like 2008 Justin Morneau killed it with RISP. He might not have been as good as others that year, but because of the chances he received, he added a ton of value to his team.

Right, this is the point of my headline.  Who gets the money?

A better example is in hockey, where you get to have power plays (the opposing team has someone in the penalty box).  Invariably, the team will sent out its best scorers, because they can leverage their scoring skills the most.

But if they didn’t send those guys out, they have to send out SOMEONE.  And those guys would get goals scored as well (just not as much as the better scorers, but more than they would score at even strength).

Who gets the power play money there?

The point comes down to you holding a lottery ticket that you may have been completely lucky to hold (say the last guy in the bullpen coming into the 18th inning to save a game), or you holding a lottery ticket that you had a great deal to earn (you are Mariano Rivera, in the 9th inning of a 1 run game in the World Series, Game 7).

How much do you allocate the outcome to the pitcher, and how much do you allocate the outcome to “timing”?

And if Mo blows it in Game 7 as opposed to some schlub reliever?  Do you treat the value differently?

These are just a small set of questions that you have to wade through, but you need to start off with your objectives.  What is it you want?  What is the question that is being asked?  After you do that, then you can figure out if you need a hammer or screwdriver.

This whole thread is off on a completely different discussion than what Colin talked about.

Colin talked about whether leverage is appropriately credited to relievers under the current formulations.  He didn’t raise the question of whether to use LI or WPA/LI or straight unleveraged stats.  He talked about whether the formulation of LI is flawed by not properly accounting for the true leverage of a given game state.

Given the assumption that you want to credit “luck” to someone’s account, Colin is arguing that the credit is being assigned incorrectly.

Okay, Tom, let’s look at your LI chart:

http://www.insidethebook.com/li.shtml

Bottom of the seventh, tied game, no outs, runner on second: LI of 1.8

Bottom of the ninth, two run lead, no outs, bases empty: LI of 2.0

I know WHY leverage index says this - the end of game states are more predictive of the final outcome than the seventh-inning state. I want to know why anyone should think that using your best relief pitcher to get three outs in a situation where an RA of 9 would get the job done is better than using him in a tight jam like that.

Or just sticking to the bottom of the ninth, look at the two out states. Why does a one-run lead have a higher LI than a tie guy?

For that matter, all the ninth inning states, top and bottom half, have a lower LI for a tie game than for a one-run lead for the pitching team in the same base-out state.  Math-wise that works out, but it’s nonsensical from an actual game leverage standpoint.

Take the two situations Colin presented in #12.  Give yourself a 3.00 ERA bullpen ace, a 3.50 ERA setup guy, and 4.00-4.25 ERA bullpen filler. If you simmed the rest of the game, which order of usage would give your team the most wins, and what magnitude is the difference?  Also relevant, I suppose, is who gets used—using the ace in this game probably limits his usage in future games to some degree.

Take the bottom of the ninth situation, no outs, and compare the tie game to the 1-run game.

You have Mariano Rivera, who gave up runs in 10/61 appearances last year, and Chan Ho Park, who gave up runs in 24/53 appearances last year.  Assume for the sake of the exercise that those rates represent their true talent levels.  Assume also that each pitcher will pitch only the ninth inning, and that if the game goes to extras that each team has a 50% chance of winning.

If you pitch Rivera with a 1-run lead, you’ll win 84% of the time in the ninth plus 16%/2 = 8% in extra innings = 92% wins.

If you pitch Park with a 1-run lead, you’ll win 55% of the time in the ninth plus 45%/2 = 22.5% in extra innings = 77.5% wins.

If you pitch Rivera in the tie game, you’ll lose 16% of the time in the ninth plus 84%/2 = 42% in extra innings = 58% losses = 42% wins.

If you pitch Park in the tie game, you’ll lose 45% of the time in the ninth plus 55%/2 = 27.5% in extra innings = 72.5% losses = 27.5% wins.

In either case, Rivera gives you an additional 14.5% wins.  That implies the leverage should be the same for a 1-run lead and tie game in the bottom of the ninth.

There might be some subtle changes if you allow the pitchers to be replaced if they put runners on base, but I can’t imagine that would be enough to shift the true leverage by over 50% as LI says it does.

What is “RA of 9”?  Is that run environment?

***

Before I discuss those two situations, note that LI, as I’ve described it, is about the leverage for that plate appearance.

In the case you are talking about, the one-inning reliever, we should be looking at LI on the basis of impact on the inning (not the PA), much like I described it in The Book.

Ok, so let’s take a look. I’m going to change the situations slightly, just for ease of calculation.  I’ll presume these two situations still make your point:

Top of 9th, up by 2, bases empty: 1.6 LI
Top of 7th, tied, bases empty: 1.5 LI

The win expectancy for the two situations presuming both teams are equal:
.923
.500

By the time the half-inning is over, the following are the expected outcomes. 
0 runs allowed: 71% of the time
1 run allowed: 16% of the time
2 runs allowed: 7% of the time
3 runs allowed: 3% of the time
4+ runs allowed: 3% of the time

Now, let’s look at it from the 7th inning perspective.  If no runs are allowed (happens 71% of the time), then you gain in win probability by +.094.  If you allow a run in the inning, your win prob goes down by -.140.  Allowing two runs means a change of -.286 wins.  Allowing three runs means a change of -.377 wins, and so on.

If you weight the absolute value of each by the frequency of how often they can occur:
71% x .094
16% x .140
7% x .286
3% x .377
3% x .430

And if you add that up, you get a weighted average absolute difference of 0.133 wins.  This means that, on average, by the time the half-inning is over, the change in win prob would have been 0.133 wins in some direction.

If we repeat this for the 9th inning scenario, allowing 0 or 1 run means automatic win, or +.077 wins.  If you allow 2 runs, it’s a tie game, your chance of winning goes down by -.289 wins.  Allowing 3 runs means your chance of winning is -.729 wins, and allowing 4 is -.837 wins.

Repeating as above:
71% x .077
16% x .077
7% x .289
3% x .729
3% x .837

We see that the downside is much bigger, but the upside is somewhat smaller, and occurs far more often.  The sum of the above is 0.134 wins.

This is a virtual even match to the average difference of 0.133 wins for the 7th inning situation.

So, to answer your question, presuming I understood it: the impact of the pitcher in the 7th inning tied, or 9th inning up by 2, is virtually identical.

The above was in response to Colin.

Mike/16: I go through your scenario and see what it gives us.

You’re so Canadian.  It’s $8000, not 8000$

It’s a Quebec thing.  The rest of the country puts it on the left.

Responding to Mike/16:

I’m going to rely on data from The Book for a great pitcher and an average pitcher.  For those following along, it’s tables 8 and 9.  Here are the chance of allowing the following runs, with the average pitcher first, and great pitcher second:

0: 70.2% 78.5%
1: 15.7% 13.0%
2: 7.5% 5.2%
3: 3.6% 2.1%
4+: 3.0% 1.2%

I’ll also need bottom of the 9th win%, so I’ll take them from The Book, page 41 (translated for pitching team):
if tied: .351
give up 0 runs: .500 (game tied, into extra innings), or +.149 wins
give up 1+ runs: .000 (naturally), or -.351 wins

if up by 1 run: .781
give up 0 runs: 1.000 (naturally), or +.219 wins
give up 1 run: .500 (game tied, into extra innings), or -.281 wins
give up 2+ runs: .000 (naturally), or -.781 wins

So, what happens in these two situations, with the different pitchers?

1. Game tied
a) average pitcher
70.2% x .149
+ 29.8% x .351
= .209 wins

b) great pitcher
78.5% x .149
+ 21.5% x .351
= .192 wins

So, on average, these pitchers will cause a swing in win percentage of around 19% to 21%

2. Up by 1
a) average pitcher
70.2% x .219
+ 15.7% x .281
+ 14.1% x .781
= .308 wins

b) great pitcher
78.5% x .219
+ 13.0% x .281
+ 8.5% x .781
= .275 wins

So, on average, these pitchers will cause a swing in win percentage of around 27% to 31%

Comparing, we see the average pitcher is .209 swing when tied, and .308 swing when up by 1.  308/209 = 1.47

The great pitcher is .275/.192 = 1.43

Doing it the other way, by looking at win% (which is how I do it in The Book, and how Mike/16 did it):

1. Tied game
a) average pitcher
70.2% x .500
+ 29.8% x .000
= .351 wins

b) great pitcher
78.5% x .500
+ 21.5% x .000
= .393 wins

That’s +.042 wins for great over average.

2. Up by 1
a) average pitcher
70.2% x 1.000
+ 15.7% x .500
+ 14.1% x .000
= .781 wins

b) great pitcher
78.5% x 1.000
+ 13.0% x .500
+ 8.5% x .000
= .850 wins

That’s +.069 wins for great over average.

.069/.042 = 1.64

***

Anyway you want to break it down, the LI for up by 1 will be about 50% higher than tied.  I don’t see why we would necessarily think this is nonsensical.

#16/Mike: If you don’t pitch Mo in the ninth, one could argue he deserves to pitch the 10th (or, perhaps, any future bottom half-inning if his team gains a small lead.)

Mike, the problem in your illustration is that you are presuming the pitchers either give up 0 or 1 run:

If you pitch Rivera with a 1-run lead, you’ll win 84% of the time in the ninth plus 16%/2 = 8% in extra innings = 92% wins.

If you pitch Park with a 1-run lead, you’ll win 55% of the time in the ninth plus 45%/2 = 22.5% in extra innings = 77.5% wins.

That’s why it looks like Chan-Ho would win so much.  Dude is going to give up a substantial number of 2+ runs (and therefore lose in the 9th) if he’s only getting 55% scoreless innings.

If you work out your example to include 2+ runs, you will see that you will get something like an LI of 1.5.

Try it out, and report back the results.  I need guys like you who are not sold to do the work and sell yourself, and therefore, sell everyone else…

Right.  So I forgot to account for multiple runs scoring.  Again, take the bottom of the ninth situation, no outs, and compare the tie game to the 1-run game.

You have Mariano Rivera, who gave up no runs in 51/61 appearances last year, 1 run in 6 appearances, and 2+ runs in 4 appearances.  Chan Ho Park, no runs in 29/53 appearances last year, 1 run in 14 appearances, and 2+ runs in 53 appearances.  Assume again for the sake of the exercise that those rates represent their true talent levels for pitching one inning.  Assume also that each pitcher will pitch only the ninth inning, and that if the game goes to extras that each team has a 50% chance of winning.

If you pitch Rivera with a 1-run lead, you’ll win 83.6% of the time in the ninth plus 9.8%/2 = 4.9% in extra innings = 88.5% wins.

If you pitch Park with a 1-run lead, you’ll win 54.7% of the time in the ninth plus 26.4%/2 = 13.2% in extra innings = 67.9% wins.

If you pitch Rivera in the tie game, you’ll lose 16.4% of the time in the ninth plus 83.6%/2 = 41.8% in extra innings = 58.2% losses = 41.8% wins.

If you pitch Park in the tie game, you’ll lose 45.3% of the time in the ninth plus 54.7%/2 = 27.4% in extra innings = 72.7% losses = 27.3% wins.

Rivera gives you an additional 20.6% wins with a 1-run lead and an additional 14.5% wins in a tie game.  That’s a ratio of 1.42.  Huh.  So the math does work out to have a higher LI in the 1-run game.  That still doesn’t feel right to me.  And I’m not sure why I went down that route because I knew (and had already granted) that your math was correct.  Colin wasn’t challenging the math, either.

Colin was, I think, in his article raising the point in Sky/22 (with the additional point that Mo could have pitched in the 8th or a previous inning if he didn’t pitch in the 9th).

Typo correction in #24:
That should be Chan Ho Park allowed no runs 29 times, 1 run 14 times, and 2+ runs 10 times.

That’s a ratio of 1.42.  Huh.  So the math does work out to have a higher LI in the 1-run game.  That still doesn’t feel right to me.  And I’m not sure why I went down that route because I knew (and had already granted) that your math was correct.  Colin wasn’t challenging the math, either.

I have to say I very much like that you said all this.  Most people I think would have just backed away and stayed quiet.  You obviously worked it out in real-time, and saw that it worked and let the post stand.

***

Ok, so we agree, the math works.  We agree, the impact will be felt more with the 1 run lead in the 9th than the tied game in the ninth.  That is, we can “earn” more money given the same amount of money, because we can “leverage” (borrow on margin) loaned money.  And Mariano has more starting cash than Chan Ho Park has starting cash, so while they both get to borrow 50% on margin, Mariano simply is starting with more money in his pocket.  And when the game is over, Mariano will end up with alot more money than he would have otherwise gotten without the leverage.

Right, so we agree to all that?

Given that those are factual assumptions, what is it that we are talking about exactly?  Or are we disagreeing on something that I think we have agreed to?

Ok, so we agree, the math works.

Yes.

We agree, the impact will be felt more with the 1 run lead in the 9th than the tied game in the ninth.

No.

And I will concede freely that Tom’s math in post 17 is correct, although I haven’t checked it so you can take that concession for whatever it’s worth. It’s just that in simplifying the example you removed almost ALL of the interesting features from the example I proposed. So you’re right, I guess, you’re just completely missing the point.

It’s just that in simplifying the example you removed almost ALL of the interesting features from the example I proposed.

This is what you had proposed:

Bottom of the seventh, tied game, no outs, runner on second: LI of 1.8

Bottom of the ninth, two run lead, no outs, bases empty: LI of 2.0

The thing I changed was the runner on second to bases empty (and moving bottom to top for both). 

Are you saying that the interesting feature you proposed required that I keep the runner on second?  I mean, I can do that, but it just makes the math a bit harder.

***

We agree, the impact will be felt more with the 1 run lead in the 9th than the tied game in the ninth.

No.

I don’t understand this.  Whether you used my math or Mike’s math, the 1 run lead showed a bigger impact than the tied game.

I’m not sure how you can concede the math works, but not agree to this statement.

Or perhaps I can ask this: accepting that the math works, what is it telling us?

The thing I changed was the runner on second to bases empty (and moving bottom to top for both).

Right - I don’t get why you did that. The example is a lot more interesting to me if you look at the bottom of the inning, particularly the bottom of the ninth, because a game is more likely to end in the bottom of the ninth than the top, and the end-of-game state is the most relevant feature of LI to this discussion.

And as for the runner on second - look at what happens when, at the bottom of the seventh, you move a runner over from first to second. The LI goes down. That’s a counterintuitive finding - but it makes perfect sense in the context of LI. I’m not arguing that it does not follow naturally from the principles of win expectancy. What I’m asking is if looking at the real-time change in WE is the correct way to even approach the questions I’m interested in.

I changed it from bottom to top only because the LI were closer I think.  But, that’s an easy enough change, so let me do that now…

In that scenario, the change (for the inning) in bottom of the 7th is 0.133 wins, but in the bottom of the 9th, it’s 0.165 wins.

I don’t know what the average change in wins is on a per innign basis.  Takign a reasonable guess, let’s say it’s about 0.09 wins on average.  So, dividing the above numbers by .09 and we get this:

Inning LI
bottom 7th, tied: .133/.09 = 1.48
bottom 9th, up 2: .165/.09 = 1.83

Contrast that to what I have on my site (which is on a per PA basis):
http://www.insidethebook.com/li.shtml
1.5
2.0

Roughly similar.

***

As for the runner on base, you will see an even more counterintuitive finding if you move the runner to 3B (with less than 2 outs) in many of the game state situations.  And the reason is because win expectancy builds in that it’s an almost guaranteed run.

So, that’s why you see something like that, where the LI goes down, even though it looks like a more precarious situation.

It’s as if you are looking at the leverage from the point of view of a long pass, and it’s about to get into the WR’s arms, and the DB is 5 yards away.  The “pressure” goes down because the WR’s is expected to score and it was more likely than at the time of the snap.

It’s fascinating and a bit mind-blowing.

I think I’m beginning to understand more of what you are saying (I think).

The framework that more closely matches your line of thinking is Doug Drinen’s Pressure.  In that framework, Drinen only looks at what happens if you end the inning scoreless.  So, in his case, moving the runner from 2B to 3B decreases the defense’s win percentage (true statement), but if they give up no runs, it’s alot more “pressure points” gained.  Basically, the “almost guaranteed run” represented by having a runner on 3B with 0 outs is instantly turned to “guaranteed no run”.

Am I correct then that this more closely aligns to your point?

Leverage Index doesn’t work like that.  It looks at the swing in win percentage, and presumes expected wins as already banked based on the frequency of scoring 0, 1, 2, 3..., n runs.  And then we’re looking at offset from this baseline.

Drinen however asks “what if no runs score 100% of the time: how much do I gain?”.  That’s why he gets results that may be more “sensical”, because maybe that’s how most people think.

Woolner has a similar kind of idea when he asks “what if exactly one run scores 100% of the time: how much do I lose?”.  You might get some sensical things, but you also get alot of nonsensical things there.

Leverage Index asks: “what if the number of runs allowed is exactly equal to what we expect the number of runs to be allowed given that base/out state: what’s the change?”.  And the result is exactly what I’ve published.

Adapting Sky’s suggestion of a simulation, if you have a situation where you are the home team, you have a tie game after 6 complete innings, and you have three relievers who are each going to throw one of the three remaining innings in the game, does the order in which you sequence them affect your chances of winning the game?

The correct answer is clearly no.

However, the average LI for the six possible sequences will not be the same.

I took three example relievers with an average runs allowed of 2.07 R/9, 3.66 R/9, and 5.26 R/9, with their runs allowed distributions modeled roughly on that of Mariano Rivera, David Robertson, and Chan Ho Park in 2010.  (I adjusted Park’s distribution a bit because he pitched a number of multi-inning outings.)

Rivera: no runs 51/61, one run 6/61, two runs 4/61
Robertson: no runs 50/64, one run 6/64, two runs 6/64, four runs 2/64
Park: no runs 32/53, one run 14/53, two runs 4/53, three runs 3/53

I get average LI results per pitcher for the sequences as follows.
Rivera-Robertson-Park 1.50-1.73-1.72
Rivera-Park-Robertson 1.50-1.73-1.46
Robertson-Rivera-Park 1.50-1.64-1.72
Robertson-Park-Rivera 1.50-1.64-1.38
Park-Rivera-Robertson 1.50-1.47-1.46
Park-Robertson-Rivera 1.50-1.47-1.38

We know that’s not true in reality, though.  In reality, all of them should be the same in terms of the actual leverage each pitcher gets on average.

I want to edit my last sentence in #34 to say, “In reality, each pitcher has exactly the same impact on the chances of his team winning the game in each of the six possible sequences.”

Using the word “leverage” may be too vague because we’re debating about what the word should mean.

I should specify that in #34 I am speaking of LI at the beginning of each inning, and I simulated in one-inning chunks based on the run distribution I detailed.

Mike, are you assuming the pitcher’s team doesn’t score more runs?  If so, those numbers aren’t right, but the point about pitchers creating leverage for the next pitcher is quite apparent.

If the idea is to optimize games won, then you also want to optimize the usage of your better relievers.  One way to do that is to save them for the higher leverage innings, which tend to appear later in games, because then you are more sure they are being used in situations that actually affect the game (if they get used in the seventh, and your team scores eight more runs, oops).

But the later you save them, the less likely you are to have a situation where they make a difference, as Mike’s table shows above (I think—accounting for the pitching team’s runs scored is important).

In other words, the better the setup guy for Rivera, the more leverage Rivera is going to see (in quality and quantity of innings) and the more WPA he’s going to accumulate.  If true, that extra credit surely goes to the setup guy, not Rivera.  How do we define “extra”?

Ah, Sky/37, good catch.  Yes, I did assume the pitcher’s team doesn’t score.  So that’s only one possibility in the simulation out of many.  It still makes the point, though.

So, yes, you could have an additional number of similar tables one for each number of runs the pitcher’s team scored.

Mike/34: you are absolutely correct that if you have a predetermined rotation of pitchers to use, and all those pitchers will be used, then the impact they will have on winning for the team will be identical.

Leverage Index however knows nothing about the future.  It presumes that the rest of the game will be pitched by “average”.

***

Leverage Index also tracks how much the game swings when the pitcher is there.  So, we’d need to see a different LI.  But, this is a question of description more than anything.

Try thinking of it this way, Tom.

If your current win expectancy is .7, and it’s the first inning, what’s the margin of error around that estimate. Now what if it’s the fourth inning? Sixth? Eighth?

No, it’s not a question of description, it’s the crux of the disagreement.

The problem with LI-based value measures is that they ignore that pitchers create and destroy LI for following pitchers.

Someone is certainly welcome to program a more sophisticated simulation that accounts for more things, but I would think that showing one instance where LI fails would be illustrative of the problem that is being pointed out.

Sky/37, if I assume that the home team scores a run in the bottom of the 7th and no run in the 8th, the sequences have the following average LI:

Rivera-Robertson-Park 1.50-2.09-2.38
Rivera-Park-Robertson 1.50-2.09-2.25
Robertson-Rivera-Park 1.50-2.00-2.38
Robertson-Park-Rivera 1.50-2.00-2.13
Park-Rivera-Robertson 1.50-1.94-2.25
Park-Robertson-Rivera 1.50-1.94-2.13

And if the home team scores none in the bottom of the 7th and one in the bottom of the eighth, it looks like this:

Rivera-Robertson-Park 1.50-1.73-2.38
Rivera-Park-Robertson 1.50-1.73-2.25
Robertson-Rivera-Park 1.50-1.64-2.38
Robertson-Park-Rivera 1.50-1.64-2.13
Park-Rivera-Robertson 1.50-1.47-2.25
Park-Robertson-Rivera 1.50-1.47-2.13

And one run in the 7th and one run in the 8th:

Rivera-Robertson-Park 1.50-2.09-1.85
Rivera-Park-Robertson 1.50-2.09-1.98
Robertson-Rivera-Park 1.50-2.00-1.85
Robertson-Park-Rivera 1.50-2.00-1.92
Park-Rivera-Robertson 1.50-1.94-1.98
Park-Robertson-Rivera 1.50-1.94-1.92

And if two runs in the bottom of the 7th and none in the 8th:

Rivera-Robertson-Park 1.50-1.60-1.85
Rivera-Park-Robertson 1.50-1.60-1.98
Robertson-Rivera-Park 1.50-1.58-1.85
Robertson-Park-Rivera 1.50-1.58-1.92
Park-Rivera-Robertson 1.50-1.69-1.98
Park-Robertson-Rivera 1.50-1.69-1.92

In all the cases, we can see that Rivera is creating LI for his bullpen mates, and Park is destroying LI for his bullpen mates.

We could iterate over all the possibilities for run scoring by the home team, and we would find the same thing going on.

In all the cases, we can see that Rivera is creating LI for his bullpen mates, and Park is destroying LI for his bullpen mates.

We could iterate over all the possibilities for run scoring by the home team, and we would find the same thing going on.

I just realized that this is false.  It’s true most of the time, but not when Park takes a 2-run lead in the 7th and blows it.  Then he creates LI for his teammates rather than destroying it.

I’m unsure of the implications of that without thinking through it further.

So is creating leverage good/bad/either or is that not the right way to look at it?

For example, you can create a higher leverage situation by keeping a close game close.  Or you can take a lead and put it in peril.  The first situation seems easy—you take some credit from the second guy and give it to the first.  But the second situation, well, I’m stumped.

And how is this related to having one guy do the job of both pitchers?  No matter how you go from point A to point B, the WPA will be the same.  But the rockier the route, the higher the LI.

So is creating leverage good/bad/either or is that not the right way to look at it?

Creating leverage is good if your current win expectancy is bad, but bad if your current win expectancy is good.

And Tom, I said in the article I thought my point applied to any win expectancy-based leverage framework, not just LI. That’s including Woolner’s LEV. I’m talking in terms of LI right now because frankly I think we’re both more comfortable talking in those terms, not because I think it’s more or less problematic than any other framework in regards of what I’m talking to.

So is creating leverage good/bad/either or is that not the right way to look at it?

I think the answer to that is that creating leverage is good if your change in the win expectancy is positive, but bad if your change in the win expectancy is negative.  But I’m not terribly confident that captures everything.

Btw, if sum up almost* all the possible scoring distributions for the home team, based on them scoring with the same distribution as a composite of Rivera+Robertson+Park, here’s the LI per pitcher by sequence:

Rivera-Robertson-Park 1.50-1.75-1.85
Rivera-Park-Robertson 1.50-1.75-1.70
Robertson-Rivera-Park 1.50-1.67-1.85
Robertson-Park-Rivera 1.50-1.67-1.62
Park-Rivera-Robertson 1.50-1.55-1.70
Park-Robertson-Rivera 1.50-1.55-1.62

*I actually only summed up 98% of the possibilities and divided the result by 0.98.  The other 2% is almost as much work as the first 98%.)

Carrying it out to 99.7% of possible home team scoring distributions, here’s what I get:

Rivera-Robertson-Park 1.50-1.75-1.83
Rivera-Park-Robertson 1.50-1.75-1.68
Robertson-Rivera-Park 1.50-1.67-1.83
Robertson-Park-Rivera 1.50-1.67-1.60
Park-Rivera-Robertson 1.50-1.55-1.68
Park-Robertson-Rivera 1.50-1.55-1.60

Hopefully the remaining 0.3% won’t change things too much.  It’s the situations where the home team scores 6+ runs over the final two innings.

If your current win expectancy is .7, and it’s the first inning, what’s the margin of error around that estimate. Now what if it’s the fourth inning? Sixth? Eighth?

I’m not sure “margin of error” is the correct term.  But, let’s make it even clearer: you have a .500 chance of winning the game in the first inning, and you also have a .500 chance of winning the game in the last inning.

Those presume that we know both teams are equal in talent.

Now, what is the question exactly?  That perhaps we don’t know the true talent level of both teams, and therefore, you are asking what is the margin of error of our initial estimate?

No, it’s not a question of description, it’s the crux of the disagreement.

First, let me just make clear that LI is a *byproduct* of win expectancy.  It *is* a descriptive stat.  It simply measures the swing possibility in win expectancy at a particular point in time.  That’s all that it is.

The problem with LI-based value measures is that they ignore that pitchers create and destroy LI for following pitchers.

What you are saying here is that players affect the future win expectancy of future players.  That the future players’ context is totally dependent on the performance of the previous players.  This is totally true. 

But this has nothing to do with LI (though LI *describes* this affect).  This is a win expectancy issue.

Someone is certainly welcome to program a more sophisticated simulation that accounts for more things, but I would think that showing one instance where LI fails would be illustrative of the problem that is being pointed out.

I don’t know where LI, or I presume you really mean WPA, fails.  You are taking one situation (future knowledge that you will have three pitchers each pitch one inning), and applying it to a framework (win expectancy), that presumes no future knowledge other than average.

***

I really don’t think we agree on the basis enough to be able to disagree on anything.  We’re talking about two things that are related, and trying to prove one thing by talking about the other.

***

Maybe we can ask each other some more specific questions first, so that we can both move from step a to step b together?

To elaborate further on LI: it measures the level of fire you happen to be in.  It doesn’t ask how it got to that level.  It doesn’t ask who is responsible for it.  It simply says: “This is a two-alarm fire.” It will also say after you are finished that it’s been downgraded to a one-alarm fire or a three-alarm fire, so that the next fireman is going to deal with a new fire.

And, to downgrade to a one-alarm fire, it’s not necessary that you do a great job.  For example, giving up 0 runs, or giving up 7 runs can both change the LI from a 2.0 to a 1.0.

LI is not about quality of performance.  It’s only about description of context, and it presumes that everyone after you will do an average job.

My question to you: what is the question you are trying to get answered?  What is the objective that you are trying to meet?

Or, are you trying to describe to me why you can’t do something?

I’m bit busy tonight, so I will have to be more brief than I would like and perhaps a bit imprecise in answering your questions but hopefully it will get us a step further anyway.

I don’t know where LI, or I presume you really mean WPA, fails.

Right.  This is an indictment of how WPA is being used.  When we say that we don’t question the math, what we mean is that we believe that how you are calculating LI is correct once you have decided to use WPA as the basis for LI.

What is the objective that you are trying to meet?

Proper valuation of relievers.  Measuring the impact a reliever has on the chances of his team winning the game.

That’s why I gave an example where are all the relievers have an equal impact on the chances of their team winning the game, but they have different LI. 

If LI can be used to measure the impact a reliever has/had on the chances of his team winning, then you can assign value to relievers that way.  In fact, many sabermetricians do this.  I believe Fangraphs values relievers using LI.  But if LI does not reflect the impact that a reliever has on the chances of his team winning the game, as it does not in the example I gave, then it cannot be used to measure reliever value.

You are taking one situation (future knowledge that you will have three pitchers each pitch one inning), and applying it to a framework (win expectancy), that presumes no future knowledge other than average.

I don’t have the time to simulate all the possible usages of relievers.  However, the example I gave is a very, very common one, and if WPA (and LI) are broken for valuing relievers in that example, it indicates to me that LI can’t be used to value relievers at all.

I’ll let others speak for themselves, but I think the potential problem is that WPA- and LI-related metrics aren’t accurately measuring the value of relief pitchers, even with (LI+1)/2 style adjustments.  Now, it’s quite fair to say that they weren’t designed to measure value.  But since they get used like that, it’s worth finding a better way to incorporate the “more importantness” of certain situations.  And part of the issue is the one-way nature of LI. 

In other words, this isn’t about a problem with WPA or LI, but a problem with how they are sometimes applied to value.

One thing that might clarify.  You say that I’m taking future knowledge in my example that WPA doesn’t have.  No, not really.  I’m not taking any knowledge about who will score when.  I’m using a distribution that’s close to average, much like WPA does.  Now, it’s not perfectly average, but I don’t think it’s enough different that it matters.

The knowledge that I’m adding is which points we’re going to measure the WPA at.  And it’s easiest to do that at the inning boundary because the baserunners are erased at the ends of innnings, so I don’t have to worry about more than one base-out state.  To do otherwise makes the simulation incredibly complex.  If someone wants to code that up in Python or whatever, that would be great, but I doubt the results will be much different.

What I’m showing is how the order of the relievers makes a difference for how WPA and thus LI are apportioned.  But the order makes no difference for who wins the game.  This is not one specific strange situation.  It’s a very broad category of situations.  I couldn’t do all situations because it’s beyond my capability to simulate in one afternoon.  But it’s a very representative situation.

To elaborate further on LI: it measures the level of fire you happen to be in.

Tango - I think the above quote and the title of your leverage index chart as “Crucial Situations”, may be part of the miscommunications.  It seems from some of Mike’s and Colin’s comments that they don’t think that LI are correctly evaluating the “crucial situation” or “level of fire” when LI has a higher value with a man on first and no outs in the bottom of the 7th than it does with a man on second.  The latter situation would be defined by most people as more crucial, but would not be reflected as such when a relievers total LI is used as a measure of the “importance” of the situations in which he is brought into the game.  Mike is also questioning whether LI is an appropriate measure for the proper evaluation of relief pitchers if the LI is dependent on the sequencing of a succession of relievers rather than the quality of the reliever’s actual performance.

But if LI does not reflect the impact that a reliever has on the chances of his team winning the game, as it does not in the example I gave, then it cannot be used to measure reliever value.

But your example requires that a reliever be used regardless of situation, since you must use 3 relievers, and their use has been predetermined.

The ONLY extra value Mariano Rivera has is that you can pick and choose which game to use him in.  If he were required to pitch in 60 random games, then it doesn’t matter what the LI was.

That’s why it’s called leverage.  You get to pick and choose the game he does, or does not, pitch in.

What I’m showing is how the order of the relievers makes a difference for how WPA and thus LI are apportioned.

Only because it’s been predetermined the games he will pitch in.  It’s not the order, but the random games you’ve thrust Mariano Rivera in.

This is what I’m talking about with future knowledge.  Not the outcome of the batters, but that you’ve preselected the games he will pitch in.  Because you do that, then of course it won’t matter if he pitches the first two or last two innings of the game.  There’s no leverage occurring whatsoever. 

His “usage” LI should be 1.00, regardless what LI we end up observing from him.

The latter situation would be defined by most people as more crucial, but would not be reflected as such when a relievers total LI is used as a measure of the “importance” of the situations in which he is brought into the game.

Well, you can’t hold me to “crucial” by MErriam-Webster, when I go out of my way in the article for “crucial situations” to define and talk about nothing other than “swing win percentage”.  That’s the point I keep specifically noting in the article that it’s all about the swing in winning percentage.

For those who never read it, here’s the three-part article:

http://www.insidethebook.com/articles.shtml

And, the swing in winning percentage with a runner on 3B happens to be less than with a runner on 2B in many situations precisely because just about everyone is going to score from 3B with 0 outs.  It’s practically money in the bank, and should be considered an almost lost cause. (87% chance of scoring or something)

Mike is also questioning whether LI is an appropriate measure for the proper evaluation of relief pitchers if the LI is dependent on the sequencing of a succession of relievers rather than the quality of the reliever’s actual performance.

But, the game he has been selected for is, ideally, based on the quality of his EXPECTED performance.  We really don’t care at what point in the game he is selected to pitch… just the game he happens to pitch.

I agree, the unit is the game, not the particular PA, because you need the 27 outs.  But, you can choose, or not, to bring in the pitcher.  That’s an important point that is not being talked about.  Sky I think brought it up really early on in this thread, and I didn’t pay attention to it.

But, it’s the key point.

If for example Mariano Rivera was brought in to pitch the 9th inning of every single of the 162 games, and if the LI of the 9th inning is 1.21 or something, then I agree that Mariano’s impact will not be 1.21 times whatever his performance is.

Nor if he were brought in to every game in the first inning with an average LI of those 162 games of 0.92, that we’d use that.

Nor if he were brought into a random point in some random game, and the LI happens to have been 1.32, should we use 1.32 times his performance.

The key is that we pick and choose which games he enters, and we do so in a non-random manner.

If you are looking for a simulation and that shows that LI is effectively measuring the impact of a closer, then everyone should do himself a favor and read this:

http://www.baseballthinkfactory.org/files/primate_studies/discussion/robw_ood_2003-02-04_0/

It’s one of the best studies ever.  Because after reading this, I’m not sure what the objection would be.

Tango/61, I don’t see what your point is with that study.  It would even seem to undermine what LI finds and support Colin’s point, though there’s hardly enough detail in the study to tell for sure.

The key is that we pick and choose which games he enters, and we do so in a non-random manner.

Of course that’s the point.  What Colin is alleging, and what makes sense to me, is that LI is not telling you correctly HOW to choose which games he should enter and when.

But, the game he has been selected for is, ideally, based on the quality of his EXPECTED performance.  We really don’t care at what point in the game he is selected to pitch… just the game he happens to pitch.

That’s not what the LI framework says.

This is what I’m talking about with future knowledge.  Not the outcome of the batters, but that you’ve preselected the games he will pitch in.  Because you do that, then of course it won’t matter if he pitches the first two or last two innings of the game.  There’s no leverage occurring whatsoever.

His “usage” LI should be 1.00, regardless what LI we end up observing from him.

Good.  So we agree on that.  The LI should be the same.  But it is not.

Tom, I apologize if my last couple posts come across as curt.  I was attempting to respond quickly before going to bed at far too late an hour, and my normal personal warmth and affability may have suffered as result.  So please disregard if you don’t find the responses valuable.

Right, LI measures the potential swing in win expectancy. But what win expectancy is telling us is our current ability to PREDICT the eventual winner, based upon the inputs, which most fundamentally are:

* How close the game currently is (including current run expectancy), and
* Proximity to the end of game state.

It’s the second where… I mean, I understand why LI behaves the way it does. But in what sense does proximity to end of game state tell us anything useful about a pitcher’s performance?

I didn’t take it as curt.

***

If you select the 60 games that Mariano Rivera will pitch based on putting him into the game when the LI reaches 2.0 or higher in the 7th inning or later, you will *absolutely* win more games than if you insert him into 60 random games.

I am making this as an assumption of fact.

Are you agreeing or disagreeing with this?

Tango/65, I agree with that statement.

It’s not a question, though of using LI on the one hand or completely ignoring game situations on the other.  That’s a false way to state the problem.

What I’m contesting is that LI is not the best way to determine when to put Rivera into games.  Using LI is probably better than inserting him randomly, but it is not the best way.  LI appears to be flawed in how it determines the importance of games.  It doesn’t account for the fact that previous relievers create and destroy LI for following relievers.  So Rivera can have more value for his team if he is inserted in certain types of situations where LI is lower rather than the 60 games where LI is the highest (or the games where LI crosses a threshold that you think will make it the highest LI in 60 games, you obviously don’t know ahead of time exactly which 60 games will have the highest LI).

How much of the issue is the assumption that all future innings are pitched by average pitchers?

If you put Park in for the 8th, you know you’ve got Rivera for the 9th if he holds the situation.

If you put Rivera in for the 8th, you know you’ve got Park for the 9th.

From an information perspective, you likely need both guys to be successful to win the game.  Before pitching Mo, you want to know if Park did his job.  If not, you can keep Mo for a different game.  Or, if your team scores more runs, and you don’t need Mo, you can also save him for a different game.

--- ---

Now, with Park pitching the 8th, you’re going to have fewer 9th inning “ace” situations than if Mo pitched the 8th.  But if Mo pitches the 8th, you’re also going to have more blown 9th innings situations from park.

Does that balance out?  Current thinking would say no.  If Park blows it in the 9th or your offense comes through with more runs, you’ve wasted Mo.  And he can’t pitch ALL of the 8th inning situations (there are more of them than 9th inning situations).  So in games when you end up using Park/other guy instead of Mo and it turns out you need both to succeed to win the game, Mo isn’t available.  Overall, you’ve shifted Mo out of some games where he’d be useful to some games it turns out he wasn’t needed.

LI appears to be flawed in how it determines the importance of games.  It doesn’t account for the fact that previous relievers create and destroy LI for following relievers.

Let’s also take as an assumption of fact that the math used to calculate this chart is accurate:

http://www.insidethebook.com/li.shtml

In the top of the 9th, with the home (pitchng) team up by 1, the Leverage Index is 2.9.  I would call that a situation where you can bring in Mariano Rivera.

Let’s say the score is 5-4 for the Yankees when that happened.

Now, are you suggesting that if the score in the top of the 8th inning was 5-0, that that somehow matters, as opposed to it being 5-4 in the top of the 8th, or even if it was 0-4 in the bottom of the 8th and the Yanks rallied for 5 runs?

It sounds like you are saying that it matters how you got to a 5-4 score in the top of the 9th.

I’m saying that it is irrelevant how you got to a 5-4 score.  What matters is that in the here and now, the score is 5-4, it is a hugely pivotal moment in the game, and we don’t care how we got there.

Tango #68 - If it is 5-4 in the top of the ninth, there is a good possibility that it was also 5-4 in the top of the 8th and the LI would have been 2.2.  Wouldn’t that also have been a high enough LI to have warranted using Rivera?  What if the top of the order was due up in the top of the 8th?  Would the higher run expectation of the better batters increased the need for calling in Rivera?  And if Rivera does come in in the 8th and holds at 5-4, and NY fails to score in the bottom of the inning, should the reliever who pitches the ninth get more “credit” from LI than Rivera for also pitching a scoreless inning?

Tango, most of this discussion is over my head, so I don’t have much to contribute.  But, what I think they’re saying is, when you have that runner on 2nd or 3rd that lowers the LI, you’d still bring in the perfect reliever who can strike out the side and strand that runner.  Whereas WPA and LI assume the run has already scored, in reality, it hasn’t, and employing a better reliever will lower the chance that it does score.  I might be off base on that, but I think that’s a part of their angle that is eluding you.

I suppose one step towards an alternative might be to construct a separate WPA that works from only knowledge of the score and the inning, and maybe the outs.  I’ll call it run-WPA.  The difference between the two WPAs would indicate expected wins that haven’t been banked yet, as it were.  When the runner’s on 3rd, WPA says he scored, but run-WPA says he hasn’t.  I don’t have time to think through whether this would actually achieve anything.  I was only looking at the one situation.  But it’s the idea that until that run actually scores, it can be prevented.  That might inform usage decisions, but I don’t think it helps the valuing relievers question.

But, what I think they’re saying is, when you have that runner on 2nd or 3rd that lowers the LI, you’d still bring in the perfect reliever who can strike out the side and strand that runner.  Whereas WPA and LI assume the run has already scored, in reality, it hasn’t, and employing a better reliever will lower the chance that it does score.  I might be off base on that, but I think that’s a part of their angle that is eluding you.

If that is what they are saying, then I’ve already noted that they prefer the Doug Drinen “Perfect Pitcher” Pressure system.

I’ve already noted in the article (part 2) on Crucial Situations why it’s not great to do it that way.  It’s not a terrible way, but it’s also not a great way.  It’s ok. 

The reality is that the runner on 3B and 0 outs may not have scored, but it should be treated as having scored a certain percentage of the time, even with Mariano Rivera on the mound (whatever percent Mo gives up with runners on 3B and 0 outs, which is probably still 75-80% of the time).

***

Think of NFL, you are on the 1 yard line, and it’s first and goal.  What’s the chance of holding the team scoreless or to a field goal?  Now, what if you are on the 11 yard line?  21 yards line?  31?

I don’t know where the highest leverage is between the 1st and 31st yard line, if you have on the first down, but I am almost positive it’s not on the one yard line.

I’m not saying what Newcomer/70 said.

What I’m saying is much more along the lines of Peter/69.

If you use Rivera in situations prior to the ninth that have a lower LI, you are much more likely to have higher LI situations in the 9th inning where you can use a lesser reliever.  You are more likely to win the game that way than to use the less reliever in the lower LI situation earlier in the game.  The lesser reliever makes it less likely that you will ever have the higher LI situation to bring in Rivera.

Btw, since it doesn’t seem to be clear to Tango, perhaps I should specify that when I say “use Rivera in situations prior to the ninth that have a lower LI”, I most emphatically do not mean “use Rivera randomly”.  I still mean to use him in crucial situations.  I just don’t believe that LI defines the “crucialness” of the moment properly.

Peter/69 and Mike/72:

Alright, now we are getting somewhere! 

The question is one of optimization.  And, yes, I agree totally that solely relying on the LI (calculated as the change in swing win percentage *on that PA*) is not the best way to do it.

And, I’ve offered that a better way is the change in swing win percentage *in that inning*.

But, even THAT is not enough.  Because, as is noted, if you restrict yourself to a pitcher to only pitch in one inning (or say face 5 batters), then you also have to worry about future innings.

Take for example that you play for a powerhouse scoring team.  Would you necessarily bring in Mariano in the top of the 8th up by 2, if due up in the bottom of the 8th is the Yankees top of the order, and the opposing pitcher is likely some Royals scrub?  No, because it’s very possible that by the time the game is over, the Yankees will have won that game by 5 runs, and so, using Mariano would be wasted.

Therefore, in order to figure out how to use Mariano, you would have to create a much more complex framework that would simulate the “rest of the game” and see in which places Mariano would best be used, of where he’d have the most impact.

***

Now, Rob Wood in his fantastic article, create a set of simple basic rules.  And using those rules, he did show that he can create great leverage for a Mariano Rivera, that he can get more wins than putting him at random.  Or even in the traditional role (including bringing him in with a 3-run lead).

Is it possible that we can find even better more optimal ways to bring in relievers beyond single-PA LI or rest-of-inning LI?  Yes, absolutely you could.

For example, you can probably show that brining him in in the 7th, even if the (single PA) LI is 2.5, is not good enough because there is too much game left.  And that a single-PA LI of 4.0 in the 7th inning might be equivalent to a single-PA LI of 3.0 in the 8th inning, and to a 2.5 single-PA LI in the 9th inning.

All of that is definitely possible.

***

Further complicating creating the optimal entrance point for a reliever is that he needs at least a 2-batter lead time.  This *really* makes our job tougher.

***

I think if you read The Book, and if you read what Woolner wrote in BBTN, we both dance around this issue, of when to best bring him in, and we come up with some general rules of thumb that are better than the current practice.

So, you can come up with something like:
single-PA LI of at least 3.0 in the 7th or later
single-PA LI of at least 2.5 in the 8th or later
single-PA LI of at least 2.0 in the 9th or later
single-PA LI of at least 1.5 in the 10th or later

And for every day he has not been used, drop those numbers by 0.25.

Something like that.

***

Is this something that is we can more commonly agree on?

Another way to say it is that when Mariano comes into a game with an LI of 2.0 in the 9th innning, that the average LI of all those batters he faced in that inning would be say an average of 1.8 to 2.2.

But, if he comes into the game with an LI of 2.0 in the 8th inning, the average LI in that inning would be 1.8 to 2.2, but the average LI in the NINTH inning might be between 0.5 to 3.5 (for an average of 1.3).

And so, it might not be the best use to use him in the 8th because of the chance that the game would be non-crucial in the 9th might be too high.  And therefore, in order to justify using him in the 8th, you need a higher LI baseline.

Tango/74, I agree that we seem to be talking about the same issue now.

I’m not sure I agree with your rule of thumb, though.  My inclination would be that it should go the other way from what you say, that you should bring in your ace reliever with a lower threshold for the single-PA LI the earlier in the game that it is. 

Moreover, I’m not convinced that the WPA framework is even capable of properly capturing the necessary information to make the decision.

Great, fantastic.

I will say that this is one of those things I’ve been meaning to do ever since I introduced Leverage Index some seven or nine years ago.  That while single-PA LI is a great starting point, rest-of-inning LI is better (and the reason I used that in The Book).  But, rest-of-game LI would be the best.  That basically, the main thing we care about is what happens if this pitcher pitches to 4-8 batters, and then some other pitcher comes in to finish the job.

And the question is are we going to find any relationship between single-PA LI and rest-of-game LI?

(I’m inclined to think that my rule of thumb is on the right path, but, I don’t know that I would bet on it more than say a 60/40 bet.)

This is really hard to program.  I mean, really hard.  You’ve got alot of variables to worry about, not only the rest required for this reliever, but the rest of your bullpen as well.

Until that happens, single-PA LI is the best we’ve got until someone does rest-of-inning LI and rest-of-game LI.

Colin/65 was marked for moderation and is now open.

But in what sense does proximity to end of game state tell us anything useful about a pitcher’s performance?

It only tells us about the impact his performance can have.

And, it’s only valuable if we can pick and choose the games he will pitch in.  If he would pitch in all 162 games, and pitch one inning in each (to start the inning), then LI tells us almost nothing. 

If you can pick and choose at what point in the middle of the inning he would pitch, and pitch for 3 outs (meaning getting 2 outs in the 7th and one in the 8th), then LI would become HUGELY valuable.

Tango/77, yes.  And I wish it weren’t so hard to program.

One reason I think single-PA LI or even rest-of-inning LI is inadequate is the assumption that everything that follows is average. 

I think we’re on the same page with that.

If you choose to bring in Rivera now, you’ll have below-average pitchers available later but you’re more likely to be in a more advantageous score differential when you do have to bring in those other relievers later.  If you choose not to bring in Rivera now, you’ll have Rivera available later, but you’re more likely to be in a disadvantageous score differential when you do have the opportunity to bring Rivera in later.  And the single-PA LI and rest-of-inning LI can’t capture that effect.  They expect average results from the reliever you bring in, and an average set of relievers to be available when the next reliever, if any, needs to be called upon. 

Both of those factors are important.  One, that you change the quality of available relievers, militates toward saving Rivera toward later innings, i.e., the increasing single-PA LI threshold for earlier innings that you suggested.  But the other, that based upon the quality of the reliever you choose that you change the likely future score differential when this new reliever exits the game, militates toward using Rivera earlier, i.e., the decreasing single-PA LI threshold for earlier innings that I suggested.

Mike/80: I can’t really disagree with anything you say there, other than saying it is “important”.  I don’t know that it is “important” to know, but that’s just an argument over adjectives at this point.

Other than Mariano Rivera (and a few big names, like Nathan, Soria, Hoffman), there really isn’t the kind of big difference in relievers.  Basically, you have a .290 wOBA reliever, and a .310 wOBA, and a .330 wOBA.  The spread in talent among those that you would even consider bringing in in a close game is pretty tight.

It’s not like we are trying to figure out: should we bring in our #1 or our #4?  The choice is really between your #2 and your #3, or your #2 and your #1.

But, that’s really a disagreement on adjectives.

***

Having said all that, Rob Wood’s simulation is the model that needs to be surpassed.  It stands as the best we have, and someone can start to build bits and pieces of additional parameters beyond what he did.

Thanks guys.  It took two days and plenty of posts, but at least we got there.

For those coming late, and don’t want to read everything, I think you can start at Tango/66 and read all the way down to here.  It’s a good discussion.

Otherwise if you have the time, the whole thread is a good read.

I’ll also recommend Studes’ article from years back:

http://www.hardballtimes.com/main/article/long-live-baseball-analysis/

To prove the point, I pulled all the WPA events from 2006 and calculated the average impact of each event on the win probability of the batter’s team. Not surprisingly, I found that the same type of event (such as a single) in close games has a larger win impact than that type of event in blowouts. Based on the data, I have estimated a standard multiplier for calculating the impact of a hit or out in a game with a particular victory margin. It’s still under development, but I don’t think the final version will look much different than this:

Margin Impact
1 1.38
2 1.13
3 0.97
4 0.86
5 0.76
6 0.66
7 0.63
8 0.57
9 0.51
10 0.47

You might call this a “Game Leverage Index,” as it shows that a single in a close game is worth about 40% more (in wins) than a single in a three-run game (1.38 compared to 0.97). And it’s worth nearly three times as much as a single in a game with a 10-run margin (1.38 vs. 0.47). The same multiplier can be applied to any type of event.

I was actually surprised that the differences weren’t greater, but this table makes a lot of sense to me. I think it could represent a workable compromise between those who want to value performance in games, but don’t believe that the difference between “when” an event occurred should matter once the game is over.

Basically, studes looked at the margin of victory for each game, and calculated how much impact each event had.  Basically, a hit in a one-margin victory counts as “1.38”, but a hit in a ten-margin victory counts as “0.47”.  It looks at things from the point of view as the game as the single “unit”.  It is an interesting way to look at things.

Nice discussion. I especially appreciate being told I can skip comments 1-65!  A few random thoughts:

1) Mike’s point that the sequence of relievers used, and thus their respective LIs, has no post-hoc meaning for the value of their pitching seems right to me.  So we can’t evaluate reliever usage simply on the basis of whether ERA and leverage are highly correlated (negatively) within our bullpen.  Some kind of “game leverage” a la Studes does seem like a better approach.

2) That said, I doubt very much that using closers much more often earlier would be the right move, and using LI as part of a decision rule for when to use closers may still be correct.  To assess usage, we need to assign a value to NOT using our closer in a game (and having him available/better tomorrow), and consider that in addition to game leverage.  Once you do that, holding the closer back for the 8th or 9th, and sometimes getting the bonus of not needing to use him, will often be an attractive strategy.  Yes, you will sometimes lose a lead with an inferior pitcher in the 7th, tut that inferior pitcher was going to pitch the 8th or 9th anyway, so that’s a wash by Mike’s own logic.

3) While LI has limitations for evaluating usage, one piece of it—run expectancy—is very important.  You really do want your best relievers pitching in high RE situations.  With men on base in the 8th, or even 2 outs in the 7th, bringing in the closer will often be correct (assuming he can warm up fast enough).  A usage metric needs to account for that.  And that’s why using LI as a decision tool may work pretty well in practice.  I certainly don’t see how Mike’s speculation of a negative relationship between LI and correctness of bringing in the closer could be correct.

4) One reason to think about bringing closers in earlier (contradicting myself now) is to keep the other team’s better relievers out of the game.  If you maintain your lead in 7th and 8th, the opposing managers will use inferior pitchers, increasing your chance of scoring.  You want the opponent to conserve his best relievers for another day (hopefully against another team).
I doubt this is a big factor, but maybe it is.

5) Both Colin and Mike talk about good pitchers “creating leverage” for successor pitchers.  That can be true, but more generally the relationship is the opposite:  good pitchers are leverage destroyers.  In general, for the team with the lead leverage = danger, for the team behind leverage = opportunity.  Mike’s examples show the opposite because he begins with a tie game.  But if you assume a 2-run lead after six, the results are very different.

6) One of the most counter-intuitive results of using LI is that using closers with a 1-run lead is more important than in tie games. Tango: do you think that is correct?

I don’t think it’s counter-intuitive at all.  Tie games simply means that the game will proceed around .500 inning to inning.  But, as soon as you get a lead, you are taking a march toward 100 win%.  And the fewer outs left, the faster you get there. 

What doesn’t seem obvious to me is the leverage for tie and 2-run leads.

What did seem counterintuitive was the leverage with runner on 2B or 3B.

I certainly don’t see how Mike’s speculation of a negative relationship between LI and correctness of bringing in the closer could be correct.

I wasn’t saying that.  I said that sometimes it would be correct to bring in your best reliever in a lower-LI situation than pure-LI-based theory (at the PA level) would suggest.  I’m not claiming that you would always want to bring in your best reliever in lower LI situations.

Both Colin and Mike talk about good pitchers “creating leverage” for successor pitchers.  That can be true, but more generally the relationship is the opposite:  good pitchers are leverage destroyers.  In general, for the team with the lead leverage = danger, for the team behind leverage = opportunity.  Mike’s examples show the opposite because he begins with a tie game.  But if you assume a 2-run lead after six, the results are very different.

We talked about both possibilities and about when creating leverage was a good thing.  I suggested that creating leverage was a good thing when your performance increased your team’s win expectancy and a bad thing when your performance decreased your team’s win expectancy.

I did not, however, suggest a framework for how one could apportion value between relievers on that basis.

I said that sometimes it would be correct to bring in your best reliever in a lower-LI situation than pure-LI-based theory (at the PA level) would suggest.

In a practical sense, which situations might you be referring to?

Mike/86:  I was responding to this statement you made in 77:  “I’m not sure I agree with your rule of thumb, though.  My inclination would be that it should go the other way from what you say, that you should bring in your ace reliever with a lower threshold for the single-PA LI the earlier in the game that it is.” You seem to suggest here a general rule (not always true) that the earlier in the game it is, the lower the LI we should require for closer usage.  I think it is extremely unlikely that is true.

Mike/87:  I understand your caveat, but I’m not sure it’s even true that it’s inherently good to increase LI when increasing WE—I’d have to think about it.  In any case, my point was only that it won’t usually be true (I don’t think) that a pitcher simultaneously increases WE and LI.  Many of your comments seemed to imply that creation of leverage for future pitchers, was, in general, a sign of good pitching.  For example in 77:  “If you use Rivera in situations prior to the ninth that have a lower LI, you are much more likely to have higher LI situations in the 9th inning where you can use a lesser reliever.  You are more likely to win the game that way than to use the less reliever in the lower LI situation earlier in the game.” Using Rivera early will not, in general, raise leverage later.  And even if it did, how is it a good thing to magnify the impact of the lesser pitcher?  And why would that lead to more wins?  I don’t see it....

A random thought after reading this thread with great interest: Would it be of any use to calculate a “second-order” LI (call it LI2) that measures the volatility of LI from a particular game state?  Much the same way that LI itself measures the volatility of WPA from a particular game state?

Intuitively I suppose that might sort of measure how much opportunity there is to “create more leverage” (or, alternatively, to reduce leverage).  At least it could provide more information than the raw LI figure and inning number alone.

***
I also have a related question: Am I correct to assume that the weighted average of all the possible LI values from one inning to the next remains the same?  In other words, let’s say at the start of an inning that the LI is X.  If we weight all the possible one-inning run totals for each team by their probability, would the weighted average / expected LI for the start of the next inning also be X? (Assuming avg pitchers, batters, and no HFA?) Or does LI inherently follow a non-flat trend-line on average from inning 1 to 9+?

No, you can’t use LI like that.

Win expectancy is that, but not LI.

It’s a good question.  The best way to think about it is to think of a 2-out scenario where the LI is very high.  And then, after you get an out, you will see the future LI will plummet REGARDLESS of what happens.