THE BOOK--Playing The Percentages In Baseball

I hope that someone links to this study in BBTF. 

As I said, I love to be proven wrong, since that gives me an opportunity to learn something.  However, after all the bashing I incurred at BBTF, I think it only fair for them to see this new research.  Not that I wouldn’t get bashed again.  After all, it is BBTF.

As well, I think I learned a lot from this research, even though in the end, I think I was vindicated, which is kind of a silly notion anyway.  As I have always said, and Guy put it aptly, my opinions are almost always informed.  Sometimes they are specifically supported by evidence and sometimes they are not.  They are never, however, out of my a**, as most “lay” opinions on sports are.  After all, I am an expert in the field of sports analysis.  You would think I was a lay journalist opining on sabermetrics, like Jayson Stark, Buster Olney, or Murray Chass, if you read the comments on BBTF.

I also encourage other people to do similar research.  For example, why is it that wOBA is so much higher when the game is close?  I’m sure we can speculate, but without looking at the components and perhaps even the pitch f/x data, I don’t think we can be too sure of anything in that regard.

I would also like to see how pitching with runners on base comes into play.  The starters obviously always started the 9th, but the reliever data I looked at was anytime during the inning.  It could be when they started the 9th or when they came in in the middle of the inning, often with runners on base.

As well, although I didn’t adjust for platoon issues, the relievers definitely faced more same-handed batters, especially when the pitching team was losing, suggesting that they were brought in specifically to face same-handed batters at some point in the inning.  This needs to be looked at too.

So I don’t think that the story is over, although I think we found a very significant factor that was affecting the data in the prior research…

Fantastic reasearch.  I have been keeping up with this discussion and I have enjoyed the systematic way that you and the commenters here have tried to pinpoint biases in the data. 

I think it would be instructive to list the error bars (+/- 1SD) to some of the measurements above so that in each instance the sample size issue is made clear.  It might help explain some of the jumps in the data.  (For example, the small sample sizes of the “Home team starters pitching the 9th, Score differential Occurrence wOBA against, adjusted for pitchers and batters”, up 3 and down 3 scenarios (.298 and .381))

Tremendous stuff!

1. It’s unlikely that BTF would post a link to this thread.  Most of the saber-threads they link to get very little comments these days.  It’s simply a different crowd over there than from 5-10 years ago.

MGL grandstanding about a topic though?  That will get him a link.

2.

However, in The Book, we found that the 4th time through the order, pitchers actually allowed a wOBA around equal to their overall wOBA against. Most of the “times through the order” effect is seen in the 1st and 3rd times.  We suspect that the 4th time effect is tempered by colder weather in night games (and most games ARE night games).

A few weeks ago, I posted a thread breaking up the wOBA by day and night games, and the 4th time effect came through, to the tune of 3 or 4 wOBA points I think, still much less than the 10 we expected.  However, since the 4th time through would include the 9th inning dampening effect, then I might get the 10 points impact I would have expected.

In light of MGL’s research here, I should therefore remove all 9th inning performances, so that I don’t have two competing effects going on.

3. Perhaps the effect also has to do with bunting?  MGL said he removes those flagged as SH, but maybe that’s not enough.  Perhaps ALL bunts with men on base should be treated separately.  Not to mention that if you have a runner on 3B of a tied game with less than 2 outs, the OF is going to play way in.  So, a breakdown by less than 2 outs and runners on would also be in order.

Basically, let’s look at all the component numbers, and by base-out state.  Obviously sample size is going to kill us.

4. The reason I look at 1993-2010 is because that run environment reached a new plateau.  So, we could increase sample size to all the REtro years, but then we have a different usage pattern of starters.  Probably 1983-1992 would be another era, 1969-1982 could be another, 1950-1968 could be another.  It might be interesting to see if this effect is pervasive.

5. I agree with DavidS that seeing the sample size (in PA) for each subgroup would be good.  Calculating 1 SD for wOBA is a snap: 0.5/sqrt(PA).  MGL almost always posts that, and I presume it’s only because this was posted early in the morning that it slipped by him.

"1. It’s unlikely that BTF would post a link to this thread.”

You would be wrong (no comments yet, though).

http://www.baseballthinkfactory.org/files/newsstand/discussion/mgl_starters_and_relievers_in_the_9th_inning_and_score_differential/

Mitchell,
Why the heck are you only giving us two summarizations by run differential when we should be looking at all of them? You need to show those so that you won’t get accused of massaging the data to justify your earlier comments.

And it would be very interesting to see a similar study done for lower-scoring run environments--like 2011--to see if there is any impact on these findings. Tom covered this idea in his fourth item in #3, and I cannot agree with him more strongly.

You should give thanks to the BTF folks despite their purported “torpor”: they energized something that otherwise might not have been re-examined for a long, long time--particularly since it seems y’all had pretty much decided you’d thought of everything when you did The Book.

Oh,yes, and please keep a close watch on TLR now that he has his SMA (that’s “Sorry Managerial Ass") back in the WS.

"Why the heck are you only giving us two summarizations by run differential when we should be looking at all of them? “

?  He gave the breakdown by tied, 1, 2, 3, 4+.  He ALSO gave the larger breakdowns at the end.  If you don’t like that one, then stick with all the other breakdowns. I don’t get the objection.

Tango, yes I think that the 9th inning should be removed when looking at times through the order effects.

I doubt that bunts would have a big impact on the data.  Since I removed SH, what is left are some hits and ROE’s on bunt attempts and some non-SH outs.  Maybe though.

Up 2 or more

.326 (.343) (N=12,099) .314 (.327) (N=115,034)

Difference of 16 points is 3.35 SE (1 SE=5.78 points)

Up 1 or less (or down)

.346 (.339) (N=3747) .335 (.332) (N=141,427)

Difference of 11 points is 1.33 SE (1 SE=8.3 points)

Well, on BTF, one ad hominem attack out of 3 (I make an assertion which turns out to be essentially true after getting lambasted for being wrong, and the criticism is that, “Well, if I hadn’t been so arrogant in the first place about my assertion..."), and one who made a comment but apparently didn’t read the post very well:

“Are they facing a bad team? Does the opponent basically give up* and/or replace a bunch of starters with scrubs?”

Since I adjusted for the quality of the batters, that is not an issue.  Plus, I doubt that teams give up when they are down by 2, 3, 4, or 5 runs.  Or more.  But you never know…

DavidS, because of the small samples in each category for the starters, it is almost inevitable that there will NOT be a smooth transition and pattern, which is one reason why I broke it down into only 2 categories at the end.

#5, is the Don Malcomb of Big Bad Baseball?  My, you’ve come a long way down.  Do you enjoy attacking me for little reason?  Is it professional jealousy?

As always, if you don’t have anything substantive, intelligent, or otherwise valuable to say in my house, I’d prefer you stay out…

This is great data.  My guess is that the main reason for 9th inning offensive decline when trailing by multiple runs is hitters’ taking more pitches than they “should,” since in that situation a walk is literally “as good as a hit.” Complicating this is that the incentives switch for any hitters who do come up with runners on:  they may go for the long ball more than usual (especially with 1 or 2 outs), reducing their OBP in order to raise their ISO. While these are very different strategies, in BOTH cases we are seeing suboptimal hitting strategies compared to what hitters do most of the time. 

It would be interesting to see if these hitters’ differ in terms of number of pitches seen, OBP, and/or ISO.  However, we might need to look separately at those who hit with bases empty vs. those with runners on to really see any effect.

My only concern (minor) with MGL’s approach is using seasonal wOBA to measure pitchers’ true talent.  For ninth inning relievers (many of whom are closers), a large share of their seasonal stats is derived from 9th inning performance itself.  So it will be harder to see any independent 9th inning effect.  That may be why, in MGL’s final numbers, the relievers’ performance seems somewhat less sensitive to a 9th inning lead effect.

For example, why is it that wOBA is so much higher when the game is close?

Because wOBA does not represent either what the batters are trying to do or the pitchers and defense are trying to prevent.  In the 9th inning, particularly when it is close, WE and RE no longer have a mostly linear relationship.  So, the linear weights no longer represent the relative weights of various events.  The equilibrium between the two teams would seem to involve a higher proportion of high linear weight events and a smaller proportion of low linear weight events.

5 out of 9 continuing to mock me while adding nothing to the discussion.

And one with this:

“I don’t need to see any research to show me that good starting pitchers who have pitched strongly and efficiently through the first eight innings should be kept in to try and complete the game. I suspect that almost every serious baseball fan in America knows this instinctively.”

What a pathetic excuse for a serious blog…

"Because wOBA does not represent either what the batters are trying to do or the pitchers and defense are trying to prevent.”

Larry, right, we know that.  The question is, “Exactly what changes - why and how?”

It would be interesting to re-weight the wOBA components and look at that.

I can come up with wOBA by game-state, if we’re going to focus just on the 9th inning, bottom/top, tied.  I’ll do that after 4 o’clock ET.

5 out of 9 continuing to mock me while adding nothing to the discussion.

And one with this:

“I don’t need to see any research to show me that good starting pitchers who have pitched strongly and efficiently through the first eight innings should be kept in to try and complete the game. I suspect that almost every serious baseball fan in America knows this instinctively.”

What a pathetic excuse for a serious blog…

Joey is a troll who never contributes anything useful and almost everyone either mocks him or ignores him.

Small note: if you quote someone, you can use
{quote} the stuff they wrote {/quote}

Change { to [

Thanks…

It appears that in close games, the batters’ wOBA is much higher than when the game is not so close (or when the batting team is losing) both for relievers and starters.

I don’t want to appeal to emotion/behavior, but I feel confident that in the last inning batters take a different approach dependent on the score.

In a close game, the focus is likely high, as is the plate discipline. In a not so close game, the focus is more likely on what’s going on after the game. I know millionaires are supposed to play the game to win all the way to the final strike. I’m suggesting that during regular season games, they don’t always do that. They’re human.

I would not be stunned to learn about a losing team’s kangaroo court fining a player for taking a walk in the bottom of the 9th while trailing by a large amount. *wink*

I could also see situations, based on experience, where a team down by 3 or 4 runs “needs runners”, so batters are taking more pitches than they would regularly (especially the first pitch), so they could be hitting in pitcher’s counts almost continuosly in those situations. When the game is closer, the pitcher has to be more careful and the situation is more balanced.

The data is interesting, as is trying to identify the cause.

“I don’t need to see any research to show me that good starting pitchers who have pitched strongly and efficiently through the first eight innings should be kept in to try and complete the game. I suspect that almost every serious baseball fan in America knows this instinctively.”

Until that pitcher gives up the lead in the 9th, and then everyone insinctively knows that the closer should have been brought in. I mean why pay a guy millions to be ‘the closer’ and then like y’know not use him to y’know like ‘close’ and stuff. I mean duh, you bring in the closer, because that’s like y’know why he’s the closer. Everyone knows that.

People know everything instinctively ... after the fact, of course.

People also instinctively know that you IBB the other team’s best hitter every time a base is open, and they know you always bunt the guy over, and the fast guy always has to steal, and RBIs are an indiciation of hitting prowess, and ... well, people instinctively know a lot of stufff about baseball. Some of it is accurate, some of it not.

In a close game, the focus is likely high, as is the plate discipline. In a not so close game, the focus is more likely on what’s going on after the game. I know millionaires are supposed to play the game to win all the way to the final strike. I’m suggesting that during regular season games, they don’t always do that. They’re human.

Circle, the data do not support that.  The wOBA is low when the batting team is losing by 2 or 3 runs. Surely players are not mailing it in then.  As well, wOBA is high when the batting team is up by 3 or 4 runs.  You would think that players are not exactly busting their butt in the 9th with a 4 run lead.

We can speculate and hypothesize about how hard players are playing with various leads and deficits, but the data does not seem to support any “common sense” theory along those lines. 

If we are going to speculate let’s comport with the data.

Incidentally, “GuyM” is the same “Guy” referenced in MGL’s ‘introspective’ portion of the post so memorably excerptbed by BobM in #5. So if you’re wondering why he’s defending MGL so doggedly...fanboy.

Ah, the politics of posting. 

Guy is as fair and impartial as they come, never shy to challenge, always fair to praise.

It’s called mutual respect if you want to be reasonable.

5 out of 9 continuing to mock me while adding nothing to the discussion.

In *your* comments on *this* thread on *your* site you make a lot of statements about BTF that offer nothing to the discussion. Don’t pretend to be innocent.

Since you obviously want to talk about BTF criticism… Most of the criticism you’ve had over the years at BTF has not been about your analysis. It’s been about what you did in #8. Your comments toward Don Malcolm are not different - no more warranted, no less civil - than what you claim to detest at BTF.

Don’t believe me? Go to BTF, and separate the comments that deal with your post from the comments that deal with your off-topic comments in this thread. It should be easy to do, given the volume of the former, right? (There were 30 total comments, last I refreshed, so feel free to stop there instead of having to wait until that thread ends before doing this.) Now, in how many of those dealing with the substance of your post are you getting, as you put it in #1 above, “bashed”? How many in the other pile of comments are bashing you, and what specifically are they bashing? Don’t psychoanalyze; just look at what they are actually criticizing or mocking. 

I have a hunch - and I could be entirely wrong - that you won’t do this. You won’t because you’ll decide it’s not worth your time. You’ll decide this despite it having been worth your time to check in there repeatedly today and classify the posts you see. But not now. You’d rather not, because you like being able to dismiss the whole lot. You won’t try to glean what the criticism is really about, because there are some things you don’t want to learn.

Murray Chass is criticized for making hyperbolic statements beyond what the data shows (such as “worst managing ever").  He is blasted for rejecting the consideration of even constructive criticism on the grounds that he is an expert and unworthy of criticism from others. He is criticized for ad hominem attacks against people who dare question him. You object to being treated on BTF like Murray Chass. Consider this comment as an opportunity for dialogue and education.

A few questions:

What was the average actual score for each grouping?  In otherwords what was the average score for say up 1, up 2, so on and so on?

What is the performance like when a SP doesn’t allow any runs through 8 but yet his offense only gives him 1 run?  I would think 1-0 is a different kind of game than a 4-3 game going into the 9th.  Just like a pitcher that has given up 3 runs through 8 has performed differently than a pitcher that is throwing a shutout through 8.

Another question is what are the odds of good things happening and bad things happening?  Does the average properly reflect this?  Meaning, if say a pitcher enters into the 9th with a 1 run lead how often does he put up a peformance of under wOBA.200?  Under .300.  Over .350 so on and so on.  Call it the blow up factor if you will since protecting the lead is the most important thing here.  Then do this for a reliever coming into the game for the first time in the 9th.  How do they compare?  Which one is more likely to put up a bad performance that will cost the team the lead in the 9th.

Finally how does IBB work in this discussion?  Are they raising the wOBA?  Are they raising one side more than other?

Consider this comment as an opportunity for dialogue and education.

Wow, reaching back at least a year, and referencing something I wrote!

For those who are new, that is how I ended an open letter to Murray Chass, something for which I was mocked as being condescending.  I then contacted Murray directly, and he was appreciative of my letter, and he took my letter as a true and sincere attempt at dialogue. 

I have no idea of village’s purpose of throwing that last line, whether he is being sincere in having a dialogue, or he’s just interested in getting his opinion out there and threw in a zinger.

#21, you raise some interesting points.  I certainly don’t know the answer to your questions although I suspect, as I did at the onset, that when the smoke clears, we will simply find that no matter how we dice it, the “on” starter pitches no better or worse in the 9th (or perhaps marginally better - not enough to do anything but use it as a tie breaker) than he would at any other time.

One of the interesting things we can infer from the above data is that managers actually seem to know what they are doing in terms of using the starter or reliever in the 9th. If the leverage is low, i.e. the pitching team has a 2 or more run lead, the manager tends to use an “on” starter, which saves his bullpen if nothing else.  If the game is close, he tends to use a reliever (presumably an elite one), and correctly so. 

IBB, BTW, are not generally included in wOBA. I don’t include them (or SH).  It shouldn’t make that much difference although I suppose if lots of good hitters are being IBB’d in close games, then wOBA would be reduced simply because those good hitters are not being allowed to have a PA that gets included in wOBA.

Unfortunately, I do count an IBB as a PA for purposes of determining the quality of the batters in any of the bucket, which I shouldn’t.  I don’t think it is a big deal though.  Maybe it is.  I don’t really know.

BTW, I am writing an article for BP which looks at starter and reliever usage by each team/manager over the years (probably the same 93-10 period).  I am going to use this data to estimate how much a manager costs/saves his team by allowing starters to bat in high leverage situations rather than inserting a pinch hitter and a reliever.  The same thing for just leaving starters in for the 7th, 8th, or 9th inning rather than a reliever, even if there is no pinch hitting situation involved.

I want to see how much a team could save in WE (win expectancy) by using a more optimal strategy and how this varies for all of the managers/teams.

I realize that bullpens are not an unlimited resource, at least the good relievers, so I will use conservative numbers for the relievers.  For example, if I think a below average starter should be taken out for a pinch hitter in the 5th or 6th innings (say, when he is facing the lineup for the 3rd time), I might assume a replacement reliever would take his place.  In the 7th, 8th or 9th innings, of course I will assume better relievers although not necessarily an elite one, since often times managers are allowing their starters to pitch into the late innings because their bullpen has been used a lot lately.

I think it will be a great study. I really have little idea how much optimal use of “replacing starters as soon as possible” can add to a team in terms of WE, but suspect it is a lot, maybe 1-2 wins a year.

I also want to redo my numbers above (in the original post) including only relievers who start the 9th (so we don’t have relievers who face more batters with runners on than the starters).

I might also look at wOBA distributions to placate McCoy, although I don’t think we’ll find any differences between starters and relievers.  BTW, McCoy, of course you are going to find more 1-0 and 2-1 games when starters pitch the 9th and more 4-3 and 5-4 games (for 1-run differences) when relievers start the 9th.  I am not sure how that matters though. In terms of pitching and batting and fielding approaches, a 1 run lead is a 1 run lead.

And of course the more you slice and dice things, the smaller your sample sizes and the more likely you will see anomalous results and make Type I and Type II errors.  That is why I always try and choose the largest samples possible while retaining the integrity of the issue being examined.

For example, if I only looked at elite starters, our sample sizes for the various score differentials would be tiny even for 18 years of data, and we likely would not be able to draw any conclusions with any level of certainty whatsoever.

I chose to use all starters, because I have a hard time imagining that very good starters might show predictive value to pitching exceptionally well early in a game, whereas other starters wouldn’t.  Those are the sort of informed compromises you have to make when dealing with limited empirical data…

” I am not sure how that matters though. In terms of pitching and batting and fielding approaches, a 1 run lead is a 1 run lead”

I agree with you on the view that a 1 run lead is a 1 run lead but I’m just wondering about a pitcher’s performance on that day.  Kind of like “on”, “really on”, and “really really on”.  I think it is safe to assume that a guy who hasn’t given up any runs is going to pitch better than a guy who gave up 1 run and that guy will pitch better than a guy who gave 2 runs and so on and so on. 

I understand the problem with sample size on this question so I’m not really looking for concrete stuff more like smoke or ripples in the water.

One last question with one caveat (I haven’t read the whole thing yet.  I’ll do that later tonight when I have to fill out a bunch of new insurance forms).  What is going on in tie games on the road?  How come batters are hitting like Albert Pujols in this situation?  Is this a partial inning problem or something? Or is this a case of teams not putting their “elite” relievers?

why is it that wOBA is so much higher when the game is close

In the case of “up by 2+”, how much of the difference can be explained by an increase in singles at the expense of doubles (i.e. “conservative basreunning")?  Plus perhaps a few doubles that might ordinarily have been triples.

It’s important to note the context that the run environment of 1993-2010 is fairly higher than 2011, and Pujols himself in 2011 is worse than his career.

What matters is the baseline, which looking at MGL’s numbers above is around a .340 league average.  I don’t know what it is in 2011, but probably under .330.

Anyway, the .380 is about 40 points above the league average.  I don’t know the sample size of the .380, but you’d figure one SD as 0.5/sqrt(PA).

I think the bunt thing is in play here… MGL removes sac bunts, but allows bunts-for-hits (with men on base).  I think that pumps up the wOBA enough.  Maybe.

"I think the bunt thing is in play here… MGL removes sac bunts, but allows bunts-for-hits (with men on base).  I think that pumps up the wOBA enough.  Maybe.”

Would that also be a problem for leading by 1 run as well?

What is going on in tie games on the road?  How come batters are hitting like Albert Pujols in this situation?  Is this a partial inning problem or something? Or is this a case of teams not putting their “elite” relievers?

Some of that must be influenced by defensive decisions. If you get a runner on third with less than two outs, a whole lot of balls that would be outs turn into hits as a result of the infield playing in and the outfield playing more shallow than normal.

Additionally, I wonder if the walk-off nature of those game conditions has an effect. As soon as you give up the game-winning hit, you don’t have any more chances to record an out. Not sure if that would bump things any.

"I think it is safe to assume that a guy who hasn’t given up any runs is going to pitch better than a guy who gave up 1 run and that guy will pitch better than a guy who gave 2 runs and so on and so on.”

It is not safe to assume that.  That is the whole point of the data I presented. Despite pitching like Cy Young in the first 1-8 innings, in terms of wOBA at least, starters in the 9th pitch exactly as you would expect if you didn’t know how they pitched in the first 8 innings - almost exactly the same as their full year’s numbers.

My baseline for 93-10 is wOBA of .344.  That does not include any pitcher hitting and is the full version of wOBA, including ROE but not including IBB or SH.

As far as a high wOBA in tie games, that is what we are discussing.  It is unlikely that anyone is actually hitting better. It is likely that wOBA is overstating what the hitters are actually doing because the batters, pitchers, and fielders have a different approach.

BTW, when I re-run the numbers including only relievers who start the 9th inning and not those who come into the 9th in the middle of the inning, presumably often with runners on base, the wOBA in all buckets goes down almost exactly 3 points.  Apparently pitching with runners on base or perhaps having to come into an inning without proper warmup, causes an significant (3 points) increase in wOBA, again, after adjusting for the batter and pitcher pools.  This is another feather in the “use your reliever” cap.

"That is the whole point of the data I presented.”

Your data is based on score differential but I was talking about how many runs the pitcher gave up.

“As far as a high wOBA in tie games, that is what we are discussing.  It is unlikely that anyone is actually hitting better. It is likely that wOBA is overstating . . . “

If this is true how can we trust this data or really say anything based on it?

On BBTF, Don Malcolm posted this (among other things):

As wonderful as wOBA may be, it’s really not a substitute for looking at how many actual runs are scored in the inning(s) in question.

While that is true, we generally use wOBA or even OPS instead of actual runs because there is much less noise to it. In huge samples and/or situations where we are not trying to tease out small differences, of course we are better off using actual runs scored rather than wOBA (or some other component measure). Unfortunately, in this analysis, I don’t think we have the luxury of being able to use runs scored for obvious reasons (small samples and small differences).

More importantly, though, as long as we match up most things, in this case the game situation being the most important, using wOBA should be just fine, and as I said, maybe even preferable.  I suppose that if for some reason, the base runners and outs distribution for reliever and starters in the 9th (assuming that relievers start the inning) are different, then comparing wOBA might not work, but I don’t see any reason why that would be the case (other than random differences).  Also if somehow the starter and reliever had different run allowing skills over and above wOBA, that also might make a difference.  You could posit, I guess , that the “on” starter is having great runner stranding skills that will continue into the 9th, as opposed to the reliever, who has his normal runner stranding skills, but I think this is somewhat of a stretch, but you never know.

I wouldn’t mind looking at actual runs scored as long as you once again match up starters and relievers by score differential, which is the key, as we have seen.

What is amazing is the debate on BBTF about the actual data.  Some people are saying that the data suggest that starters are better than relievers in the 9th and some people are saying the opposite - and everything in between.  Of course that depends on what they mean by “better” - in relation to their overall true talent?  As compared to what we would expect the 4th timne through the order?  Etc.

In any case, the data is right there for everyone to see.

First let’s clarify the “times through the order” effect.

1 .353 .345
2 .353 .354
3 .354 .362
4 .353 .354

Keep in mind that the expected is based on the batter pool and the pitcher’s overall wOBA against.  That “overall” wOBA against is based on a pitcher facing the lineup around 3 times on the average.  So even if each time through the lineup the hitters get better, a pitcher’s average wOBA will be about equal to the 2nd time through the order, which is what we find.

Now, we would expect that the 4th time through the order, that the actual would be maybe .366 or .367, but it is only .354, 1 point more than expected.  The primary reason for that is that most games are at night, and it is colder in the later innings.  When we break it down by day/night, we see that in day games, the actual is indeed higher than the expected.

In any case, it is unlikely that any lack of 4th time effect is due to starters pitching in the 9th, because that is a relatively rare occurrence. Most 4th time through the orders occur in the 6-8th innings I would think.  Regardless, it is what it is.  Overall pitchers pitch around to their average levels the 4th time through the order, whether it be because of cold weather or some other factor.  So it is not really fair to say, “Let’s bring in a reliever because it is the 4th time through the order and we expect this starter to get hammered.” We expect him to pitch to around his average overall level (in day games, we expect more, but that would also be the case with relievers).

And of course, we expect a reliever in the 9th to pitch to his average overall level because he almost always pitches one time through the lineup and often pitches in the 9th.

So all we care about is this:

After we control for score differential, do starters in the 9th who have been cruising (the data I posted in another thread shows that on the average a starter in the 9th has pitched exceptionally well in innings 1-8) pitch at around their seasonal level in the 9th, as expected based on the “4th time through the order” numbers above, or do they continue their exceptional pitching in the 9th?

If the answer to that question is, “They pitch as expected, to their seasonal levels, or they pitch a little better or worse and so do relievers,” then the decision, all other things (like resting your bullpen, showing confidence in your starter, platoon and other match-up issues) being equal, is an easy choice - whoever has the best overall, seasonal wOBA (more accurately, whoever has the best wOBA projection at that point in time) is the better choice.

I’ll actually leave the choice up to you guys and I’ll take no side one way or another. I am merely going to show who is likely to pitch better in the 9th, according to thedata.  Again, I am just reading the data, not interpreting it.

Here is the last and most comprehensive data I presented in the original post.  It speaks for itself:

Score differential Starters Relievers

Up 2 or more .326 (.343) .314 (.327)
Up 1 or less (or down) .346 (.339) .335 (.332)

This says that a starter will pitch around 17 points better than his overall wOBA against with a lead of 2 or more, and a reliever will pitch 13 points better.  I have updated those numbers to only look at relievers who start the 9th inning - not those who come into the middle.  The reliever is 3 points better when starting the inning.  So that makes it 17 points for the starter and 16 points for the reliever. If you want to call that a 1 point advantage for the starter, that’s fine with me.  If you want to call it a dead heat, that is also fine by me.  If you want to call it a 4 point edge for the starter, using my original data, that
‘s also fine by me.

With a 1 run lead or less, or losing, the starter pitches 7 points worse than overall, and the reliever pitches 3 points worse or the same if we use the updated, no middle inning entrance, numbers.  If you want to call that a 4 or 7 point advantage for the reliever or you want to also call that a dead heat, that’s fine by me.  The numbers are the numbers.

If we combine the two score differential (and weight them equally), thus including ALL 9th inning appearances by starters and relievers, we have the starters pitching 5 points better than their seasonal stats, and the relievers also pitching exactly 5 points better than their seasonal stats!  If we use the updated reliever numbers, reflecting only those relievers who started the 9th, the relievers pitched 8 points better than their seasonal numbers!

So clearly, at least according to these numbers, a reliever who is better overall, is the better choice, all other things being equal or ignored. 

I’ll leave it to you guys to decide how to incorporate things like bullpen rest, or even how to determine who is the better pitcher overall, like Motte and Carpenter. If you think that Carpenter is a better overall pitcher for any one random inning, than Motte, that is fine by me.

One more thing: It IS true that a starter in the 9th pitches better than his overall seasonal stats and the 4th time through the order (where we expect batters to hit only 1 point better than expected) would predict - by 5 points.  And so does a reliever for some reason. 

The reason for this, I would guess, is simply that in the 9th inning, wOBA gets depressed no matter who is pitching because of the different approaches in all score differentials, and because it is coldest in the 9th (as opposed to the 4th time through the order in the earlier innings).

MGl, nice research, I’ll just throw a few questions.

1. When I see pitcher pitching the 9th down by 3 or more, I’m not thinking “he’s on”, I’m thinking “bullpen is tired”. If question is whether pitchers who are on through 8th pitch better in 9th, shouldn’t you exclude down by 3 or more?

2. Could you separate starters whose seasonal wOBA against is at least 10 points better than average for starters and relievers whose seasonal wOBA against is at least 10 points better than the starters’ cutoff, and show results for those four groups ("elite" starters, other starters, “elite” relievers, other relievers)? (10 points is nice round number, but important point is to separate those who can be considered elite.)

3. How would results change if all walks with tying/winning/go ahead run on 3rd base, less than two outs and base open were treated like intentional walks? Would that bring 1-run and tied wOBA back to other states?

4. Not about research, but about the game that started all this:
Motte has career line 188/243/299 vs righties, and 284/366/444 vs lefties. Every year since 2008 lefties hit for OPS better than 738 of him.
Carpenter has career line 251/303/374 vs righties, 272/331/426 vs lefties, and 2007 was last year lefties hit him better than righties and lefties hit for OPS better than 665.
Philadelphia had Utley, Pence and Howard coming up. How can you put Motte in against them?
Pulling Carpenter for anyone but closer is non-starter, even TLR would probably be fired for that.
Real error was letting Carpenter bat in 8th.

These are real questions, not meant to mock or anything, but to see if there is more to be learned about the subject.

Davor/32:

shouldn’t you exclude down by 3 or more

MGL posted this:

Home team starters pitching the 9th

Score differential Occurrence wOBA against, adjusted for pitchers and batters

Up by 4 or more: 53% .326
Up by 3: 12% .298
up by 2: 11% .316
Up by 1: 9% .334
Tied: 8% .356
Down 1: 7% .319
Down2: 3% .349
Down 3: 2% .381
Down 4 or more: 1% .350

He put the frequency of each category and the success rate of each category. 

There is enough information there that YOU (or anyone else) can merge that data to come up with a wOBA for whatever run levels you want.

***

How would results change if all walks with tying/winning/go ahead run on 3rd base, less than two outs and base open were treated like intentional walks? Would that bring 1-run and tied wOBA back to other states?

I’ve been meaning to generate and post the wOBA by game-state since the day I first created wOBA.  I’ve always resisted because of the number of game states: at least 11 runs of differential x 20 half-inning states x 24 base-out states = 5280 equations.

However, since the 9th inning is so different from all the other innings, then I should at the least produce this.  Just a matter of finding time.  I’ll still need to produce 528 equations.  The quantity is not that big of a deal, but rather the presentation of it.

Well, it wouldn’t be 528… you can’t bat in the bottom of the 9th with a lead.  So, it would be:

11x24 for the top of the 9th
6x24 for bott of 9th

(Presumes no larger than a 5-run differential.)

So: 17x24 = 408 equations

MGL:
Am I right in thinking that your score differential is for the start of the inning only?  That is, if starter begins with 3-run lead and gives up 2 runs, the rest of his PAs remain in the 3-run bucket, not the 1-run bucket—right?

And do you think you need to worry about the pinch hitter penalty?  Not for comparing starters and relievers, who I assume face similar numbers of PHs, but when comparing 9th inning to expectations.  Maybe that explains a portion of the depressed 9th inning offense?

"Pulling Carpenter for anyone but closer is non-starter, even TLR would probably be fired for that.
Real error was letting Carpenter bat in 8th.
“

If this is the real error and you Motte isn’t answer against lefties and Tony wasn’t going to put someone else in in the 9th how does one avoid this error?

MGL:
Very few of your home relievers are pitching the ninth inning while trailing (13%).  But this should be something closer to 40-45%.  Have you limited the pool here to “good” relievers, or in some other way?  Or did I just miss a step?

Very few of your home relievers are pitching the ninth inning while trailing (13%).  But this should be something closer to 40-45%.  Have you limited the pool here to “good” relievers, or in some other way?  Or did I just miss a step?

Guy - It appears that MGL miscopied the starters percentages for being behind onto the relievers chart.  They are identical and the total of the relievers percentages don’t come anywhere near 100%.

McCoy,
Carpenter batted leading off 8th. Pulling Carpenter then and pulling him in the 9th are completely different things. If TLR pulls him for PH, he improves* chances of scoring in that inning (presumably, I don’t know if STL had reasonable hitter on the bench). That improvement (and following WE improvement) should be greater than WE reduction** from pitching relievers in 8th and 9th. And Howard is 7th in line, so game could end with him on deck.
At the start of the 9th, only options are Carpenter or closer, and two of the first three batters are lefties.

*Halladay was pitching, probably the best pitcher in the MLB right now. I can understand thinking that replacing Carpenter with PH means replacing one out with another out.

* *I think that most calculations say that WE should be improved because elite relievers are better than Carpenter, but STL’s elite relievers have problem with lefties, and they would be facing PH (probably LH), Rollins (SH) and Utley (LH).

Tango #33,
thanks. I looked only at percentage of innings by starter/reliever, and completely ignored this table.

As for wOBA table, with less than two outs and man on third as tying/go ahead/winning run, deep flyout or even grounder to the right are preferable for batter than walk. So, quick-and-dirty, setting them as neutral may indicate how much of wOBA increase for tied game comes from such walks.

(Sorry for the third post in row)

If I remember correctly, somebody on this site said that too many pitching changes aren’t good because it increases the chance of finding pitcher who doesn’t have it that day. Should that be taken into account when analyzing pitcher changes?

One of the interesting things we can infer from the above data is that managers actually seem to know what they are doing in terms of using the starter or reliever in the 9th. If the leverage is low, i.e. the pitching team has a 2 or more run lead, the manager tends to use an “on” starter, which saves his bullpen if nothing else.  If the game is close, he tends to use a reliever (presumably an elite one), and correctly so.

I found this statement by MGL in post #23 above to be really the crux of the matter under discussion.  Have the managers been operating below optimally in their choices of which pitchers to use in the 9th inning?  And if so, how many wins could they improve if they were to optimize their choices?  Although the other question being researched: whether a pitcher’s earlier innings have predictive value for the 9th inning is also an interesting and important question, it has little practical import if the managers have been acting optimally already and their are no or very few games to improve. 

MGL says that his research implies that managers have been making good decisions about when to use a reliever in the 9th and when to keep the starter in.  As another test I decided to look directly at the win loss outcomes of the game by run differential for all games where the starter had completed 8 innings and compare what the outcomes were when a starter or reliever began the 9th.  I used data from 2000 through 2010.  The data is in the chart below.  Run differentials less than -3 or greater than 6 existed but were irrelavant since teams that were behind by more than 3 lost every game whether a reliever or starter began the 9th, and teams that were ahead by more than 6 won every game. 

DIFF ROLE GAMES WINS WIN_RATE
-3 Relief 52 4 0.077
-3 Start 11 0 0
-2 Relief 99 6 0.061
-2 Start 34 5 0.147
-1 Relief 187 22 0.118
-1 Start 75 12 0.16
0 Relief 424 194 0.458
0 Start 156 81 0.519
1 Relief 369 310 0.84
1 Start 165 140 0.848
2 Relief 325 309 0.951
2 Start 229 210 0.917
3 Relief 328 318 0.97
3 Start 275 271 0.985
4 Relief 289 284 0.983
4 Start 294 293 0.997
5 Relief 217 216 0.995
5 Start 251 248 0.988
6 Relief 173 173 1
6 Start 203 202 0.995

Looking at the win rates for each run differential shows that there is very little difference between starters and relievers.  In 5 cases of the 10 relievers had a slight edge, in the other 5 starters had a small edge. If the starters were replaced by relievers in the score differentials where the relievers have an edge and the relievers performed at the same rate as the relievers who actually were used at that run differential then the pitching teams would have won 12 more games overall.  If relief pitchers had been substituted for starters in all the games in the 5 run differentials where the starters have an edge and had performed equally, then the pitching team would have lost 20 more games.  There were a total of 2147 games where the starter began the 9th inning so less than 1% difference.  More to the point less than 2 games a season for all 30 teams combined.  This seems to me about as close to optimally as one could possibly expect.

I took Peter’s chart, and took the minimum number of games for each run differential, so that the pools of games were the same for starters and relief.  Then I just multiplied the win rate at each run differential for each role by that number of games.  I got this:

If we break it down by: tied or losing in one bucket (i.e., we don’t see Nathan/Rivera coming into these games), and up by at least 1 into another bucket, we get this:

As you can see, when ahead, the choice of going with starter or (presumably elite or high quality) reliever is an almost perfect match.

When tied or behind, and presumably a lesser kind of reliever in the game, then the starter has an advantage. 

That observed gap (15 wins) is almost 2 standard deviations.  So, we’re probably seeing something non-random, but, like I said, the quality of the reliever coming in tied or behind is going to be worse than ahead.  Therefore, the non-randomness may simply be the bias of the quality of reliever.

All to say: teams do a fair job of letting the starter enter the 9th, or going to the bullpen.

A lot of good stuff here.  I’ll comment on a few and then come back later.

1) Carpenter may be better than Motte versus a lefty.  That’s not really the issue being discussed here.  It spawned the issue, but it is trivial.  Like I said, I’ll leave it to you guys to estimate (that is all we can do) who is the better pitcher given the lineup in the 9th. For what it is worth, my projections for Motte versus RHB and LHB is 2.87 and 3.81 (sort of an “ERA") and for Carp, it is 3.18 and 3.49.  So we definitely have some platoon issues here.  Good catch by Davor.

2) Pinch hitters in the 9th could definitely account for some of the depression (for starters and relievers).

3) Tango, relievers in tied games are at .324 for the season and when leading by 1,2, or 3, they are at .316 (and then jump to .335 with a 4 run lead or more).  When down by 1, they are at .337.  We certainly see too large a gap there.  I doubt that that is optimal.  Down by a run in the 9th, I assume, is a fairly high leverage situation.  To have .337 relievers (about an average reliever) is probably sub-optimal.  And then to have .316 (basically closers) when up by 3 runs is really sub-optimal.  But, we pretty much already knew that.

More to the point less than 2 games a season for all 30 teams combined.  This seems to me about as close to optimally as one could possibly expect.

4) We can certainly call that a wash and I would have no problem with that. If, we knew that using starters rather than relievers in the 9th cost a team 3/20 of a win (.15 wins) per season, I would consider that significant, notwithstanding any “tiring your bullpen” issues - which are legitimate of course.  I have always said that each type of managerial mistake probably costs on the order of tenths of a win per season, but when you add up the dozens of types of mistakes, you might be at 1-3 wins, which is huge of course.

More importantly, using your starter or reliever optimally in the 9th is NOT a significant issue, as I did mention, because, if nothing else, of the infrequency with which the starter actually does pitch in the 9th in high leverage situations (and the fact that it is usually a very good starter).

So we are not discussing that at all.  We are just discussing the general concept of whether managers can tell whether their starter is “on” in the early to mid innings by looking at how they pitch in the late innings and we are looking at how score differential affects wOBA.  Both of these things are interesting, I think, and getting a better understanding of them can help us to determine optimal strategies in many situations.

What I think we will find is that a manager leaves a lot of wins on the table by letting starters pitch the 6th, 7th, and 8th innings (and sometimes even the 5th), especially non-elite starters, because they have been “cruising” so far (and EVERY manage thinks this has LOTS of predictive value), especially when they let them bat (in the NL parks of course) in high or medium leverage situations.

It gets back to the point I have made many times on this blog (and others).  The correct general strategy, I think, is to get your starter out as soon as possible, depending upon the pinch hitting opps, the leverage, and the quality of the starter.  I think we are looking at 1 win or more (maybe a lot more) per year per season, which is enormous of course.  We’ll see (or at least get an estimate) when I do the research for the BP article.

5) As far as pinch hitting for Carp in the 8th, again, that is not really the issue.  The issue is what numbers do we use to do the analysis.  The idea that “Against Halladay, anyone is going to make an out,” is a ridiculous way to look at it of course.  And I’m sure Tony and most managers DO look at it that way. The tend to look at everything in black and white, yes/no, win/loss, right/wrong - binary results.  The correct way to look at it, of course, is the loss in WE by not pinch hitting and then comparing that to any loss or gain in the 8th and 9th by using the various pitching alternatives.  Sure, with a great pitcher the WE difference between Carp batting and a PH will be smaller than with a lesser pitcher (I think), but it is still something. Neither batter is going to make an out 100% of the time.

And, as we discussed, the leverage was 2.5 times higher in the bottom of the inning than the top, so that has to be factored in.  Again, the current opinion is probably that letting him bat was no big deal. If it was a much lesser starter, then it probably would have been a big deal (as one time big deal’s go - they are never more than a few hundreds of a win)…

Leverage Index:

Top of 9th, pitching team down by 1: 0.7

So, I’m not surprised that we see .337 relievers pitching down by 1 in the 9th.

Leading by 1 or 2, seeing .316 is again no surprise.

Up by 3, seeing .316 relievers is no surprise, and disappointing.

Tied, and seeing .324 is… well, I had no real expectation.  I guess we’d expect some setup type guy, so .324 seems about right.  The LI when tied and when up by 2 is the same, so, we’d rather have the ace in this situation.

Basically, what you’d like to do is make a swap between the 3-run-lead and tie-games, using your closers less in the former and more in the latter.  Otherwise, the usage seems to basically make sense.  The use of “too good” relievers with a 3-run lead is almost certainly a function of the save rule; if no saves were awarded for 3-run leads, I doubt this would happen.  Perhaps this is also the cause of teams not using their best relievers in tie games (no save awarded for success)?  But maybe it’s something else, like managers thinking you should reserve these pitchers for the job of “preserving a lead.” It would be interesting to see if under-utilization of good relievers in tie games also occurred in the years before saves became a big factor in player reputation and compensation.

Sending up your best reliever to hold a tie means at some point down the road you’ll have to send up a subpar reliever to either preserve the lead or to protect the tie again. 

Now obviously one has to be able to get to the point of preserving the lead and sending up a subpar reliever in the 9th to keep the tie might have this happen less often but it still has to be factored in.

If it is the 9th inning and the score is tied in all likelihood you have used your second best reliever already (or he is going a second inning) and perhaps your third best one as well and or a LOOGY.

Right, just slide over the usage from one set of 3-run groups (up by 1, 2, 3) to another set of 3-run groups (tied, up by 1, 2).

As for what they did in the old days, I don’t have access to my DB at the office, but we can take a look at a recent find, Satchel Paige:

http://www.baseball-reference.com/players/gl.cgi?id=paigesa01&t=p&year=1952

He entered around an equal number of times from down by 1 to up by 2.

Hoyt Wilhelm had his big WPA year in 1959.  24 of his 32 entries were with the tied game, and most of them in the 4th or earlier inning.

Dick Radatz’ big year was 1963.  Most of his entries were down by 1, tied, or up by 1.

So, it’s a great question: when did the shift toward entering with a 3-run lead start?

Sending up your best reliever to hold a tie means at some point down the road you’ll have to send up a subpar reliever to either preserve the lead or to protect the tie again.

Half the time this is true, but of course half the time it isn’t.  And even in road games, you may score 3-4 runs in the top of an inning, creating a low-leverage bottom of the inning. Teams are making a mistake by not deploying their closers unless they have a lead.  In home games, this philosophy means you will never use your closer despite being in a high-leverage game.

McCoy: you are correct. 

But, that’s the point of Leverage Index.  You are better off bringing in your ace tied, than up by 3.

If you are up by 3, you lose 4% of the time with a regular reliever instead of 2% of the time with an ace.  That gap, .02 wins in one inning (or .18 wins per 9 innings) is exactly what you’d expect with average leverage: the ace reliever having an ERA of 1.80 better than the average reliever.

Hence, up by 3 has average leverage.

A tied game does not have average leverage.  Hence, the ace reliever will have more win impact in a tie game than up by 3.

Thus

“but it still has to be factored in.”

For post 49.

Re post 50

I don’t disagree about up by 3.  I wasn’t advocating using your best to protect a 3 run lead instead of using him in a tied ballgame.  I’m discussing why a manager might not use his best in a tied ballgame.  He does so because at some point he thinks he’ll have a lead to protect or he’ll be able to use his best at some point down the road in that game.  The issue then is how often does A)a subpar reliever cost the team a lead in the 9th B)a subpar reliever cost the team a lead after the closer has been used C)how often the closer’s team scores enough to render his usage moot and D)how often subpar relievers performance in extra innings renders a closer’s appearance moot?  OF course there are more as well but I’m watching House at the moment.

Basically, what you’d like to do is make a swap between the 3-run-lead and tie-games, using your closers less in the former and more in the latter.

Guy - I don’t know how you are coming up with this idea but its exactly wrong.  You need to take a second look at my chart.  Teams are already bringing in their closer 73.8% of the time when the game is tied in the 9th, a higher percentage than any other non-negative score differential.  And the closer wins significantly less often than the starter.  I believe that the reason for that is that a tie game in the ninth is much more likely to go into extra innings.  Having already used up a relief pitcher in the 9th would decrease your options for strategies in extra innings.

Right, I agree with McCoy and Peter.  You can’t just use the leverage in a tie game to determine who to bring in. If you bring in the ace, there is a future cost and that has to be factored in.  Heck, even with a 1 run lead, and you bring in your ace, there is a future cost.

Peter, I think that 73.8% is wrong.  Managers rarely use their closer in the tied game when they are on the road (they wait for a lead), so 73.9% cannot be right overall.  Maybe for the home team, although that seems high as well.

BTW, this is one of those instance where the “averages” can be deceiving as far as determining whether managers are using relievers optimally, at least as far as LI goes.  It is like SB% and advancing extra bases.

For example, Tango says that .324 in tied games may be about right, and is not surprising.  Well, if the average reliever is .324, that probably means that closers are brought in maybe half the time (and correctly so) and that really crappy pitchers are brought in at least some of the time, incorrectly so (I realize that sometimes managers have little choice or they don’t have any good short relievers).  But I hope you see my point.  The numbers we see in high leverage situations SHOULD be really low, because there should be a ceiling (don’t bring in a crappy reliever, ever, unless you absolutely have to), and the range should be tight. I doubt we would see a tight enough range.

IOW, when we see what look like “correct” numbers overall (like league SB% of around 70%), that means that a good percentage of the time, managers (and players) are actually making the wrong decisions (like attempting to steak when the chances are significantly below the BE point)…

Peter, I think Guy means that: given that you have decided to bring in a reliever, then better to bring in the ace, than someone else.

Speaking only for myself, I’m not talking about starter v reliever.  Only ace reliever v other reliever.

Peter:
Your table indicates you looked at all relievers, not “closers.” Was that a mistake?  I doubt it, since if closers are used 74% of the time then I can’t see how MGL could find the relatively high wOBA he reports.  In fact, teams are often using their second- or third-best reliever in tie games, and THAT is probably why teams win more often when they use a starter (who is almost always an excellent pitcher). 

This idea of holding your closer for later use will almost always be a mistake. Tango has told us that tie games and 2-run leads have the same leverage, which means that the only “better” time to bring in your closer is with a 1-run lead.  To fail to use him when score is tied, just because you MAY end up ahead 1 run later (and again, that can only happen on the road), is a poor percentage move.  For every time you get to use him with a 1-run lead, I suspect there are at least threee games in which you lose before the closer ever comes in, or take a large lead and don’t really need him.

Guy - Yes, I spoke too soon.  The 73.8% is the percentage that a reliever is brought in to replace the starter in the ninth.  I have no way to know whether the reliever was a closer or not.  I am not sure how MGL calculated the wOBA of relief pitchers in the ninth.  The way I read his description of his study it sounded as if he was including all relief pitchers who started the ninth.  I only included relief pitchers who were replacing starters.  Relief pitchers who start the ninth and are replacing other relief pitchers, or who start the ninth but have pitched in previous innings may be causing the higher wOBAs.

I certainly agree that bringing in a reliever other than the closer to start the ninth instead of a starter when the closer is available and the score is tied is a costly mistake.  It would be important information to know how often this happens in the 424 games.

Right, all my reliever numbers are when any reliever is on the mound.

Someone suggested that when starters pitch the 9th in games they are losing by 4 or more runs (or other blowouts), that they probably have not been pitching well - the manager just wants to save his pen.  That turns out not to be true.

They pitched to a .211 tune in 1-8 when they pitched (started) the 9th down by 4 or more runs.

There is one qualification and it is a big one.  It is based on only 62 PA - it rarely if ever happens!

However, with a 3 run deficit in the 9th, they still pitched .241 in 1-8 (only 155 PA).

With a 2 run deficit, it is .247 (287 PA).

With a 4 or more run lead, it is .225.

3-run lead, .238.

2-run, .224.

So to pitch in the 9th, no matter what, you have had to been pitching a great game, the score not withstanding.

Starters generally don’t get “hung out to dry” in the 9th just because the pen is tired. Of course, in order to pitch the 9th, you have to have a relatively low pitch count, even if the manager wants to rest the bullpen.  And the only way to have a low pitch count in the 9th is to have pitched a decent game, at least, wOBA-wise.

BTW, the score differential is used for the buckets on a PA by PA basis. IOW, if a starter enters the 9th with a 2 run lead and then allows a run, some of his PA will go into the 2-run lead bucket and some will go into the 1-run lead bucket.

MGL: I don’t think I like that bucketing.  1-run lead with runner on 1B and 2 outs is different than runner on 3B and 0 outs, or just bases empty.

If you are going to do that, then you should use win expectancy or leverage index to create your buckets.

MGL- I have 6 starting pitchers that started the 9th down 4 runs from 2000 to 2010.  6 pitchers would have had to face more than 62 batters in 8 innings even subtracting pitcher PAs.  How can you have fewer pitchers and PAs using 7 more years?

Peter, I’ll check.  Can you post the list of pitchers and what game?  Thanks.

Tango, right, I don’t think it will matter much, but I’ll change it perhaps to score differential minus base runners, or something like that…

GAME_ID------PITCHER--H_SCORE--V_SCORE

BOS200507170--waket001---1------5
CHA200409200--buehm001---2------6
DET200007210--weavj002---0------4
DET200605310--roben001---0------4
TBA200008070--rekab001---0------4
TEX201007100--lee-c003---1------5

Peter, those are the number of batters faced in the 9th inning.  And it is 137. I don’t know where I got the 62 from.  I have been running so many different queries, I could easily have mixed them up.

Tango, I changed the bucket criteria to adjust for runners on base.  I simply subtracted all runners from the score differential, however, if the differential were 0 or greater (pitching team winning), the “runner adjusted differential” could not be less than zero. 

It doesn’t change things much.  Here are the numbers for starters only.  The numbers in teh first parenths is the pool of pitchers (in that bucket) seasonal numbers.  Second parenths is performance in innings 1-8.

Starters

Up 4+: .333 (.346) (.225) N=8061
Up 3: .306 (.342) (.238) N=2004
Up 2: .310 (.339) (.224) N=1624

Up 1: .342 (.337) (.210) N=1211
Tied: .346 (.338) (.218) N=1787

Down 1: .298 (.338) (.248) N=439
Down 2: .343 (.340) (.247) N=411
Down 3: .389 (.341) (.241) N=184
Down 4+: .361 (.342) (.211) N=125

(Pitching team losing numbers were adjusted to be H/R neutral since they all occur while pitching at home of course.)

So, starters will pitch 5-8 points worse than their seasonal numbers when allowed to pitch the 9th when tied or up 1 run (like Carp in that infamous game), despite pitching lights out in innings 1-8. Can we please put to rest the argument, at least as far as the evidence suggests (there obviously is a significant standard error)?

Or

Up 2+ .325 (.344) (.227) N=11689

Up 1 or tied: .344 (.338) (.215) N=2998

Down 1+ .335 (.338) (.244) N=1159

Relievers

Up 4+: .323 (.340) (.366) N=56149
Up 3: .304 (.340) (.324) N=20948
Up 2: .303 (.319) (.315) N=35786

Up 1: .300 (.319) (.309) N=29965
Tied: .346 (.346) (.329) N=45937

Down 1: .325 (.341) (.309) N=8956
Down 2: .351 (.341) (.311) N=15138
Down 3: .353 (.345) (.321) N=13031
Down 4+: .353 (.351) (.365) N=40602

Or

Up 2+ .313 (.335) (.342) N=113883

Up 1 or tied: .328 (.335) (.321) N=75903

Down 1+ .350 (.346) (.340) N=777137

We see the same basic pattern with relievers.  Their wOBA when ahead by 2 or more is a lot less than their seasonal wOBA (22 points). In close games or when losing, it is close to their seasonal averages.

So again, the reason that the starters look like they pitch so well in the 9th is because they tend to pitch with a lead of 2 or more runs, in which case, everyone, relievers and starters, have a much lower wOBA against than their seasonal numbers.

By sorting the way you are sorting aren’t you simply shuffling away outcomes from one bucket and moving them to another?  Well, obviously that is happening but is that a good thing?

Do teams radically alter the way they approach the game within an inning?

If a team is down by 3 and then then get a walk and then hit a homer they are down by 1.  At this point the outcomes are an indication that pitcher that is pitching does not have it.  If he then goes out and loads up the bases and then gives up a homer all four of those positive outcomes are charged to the up 1 run pitcher bucket instead of up 3 pitcher bucket.  In otherwords doesn’t this help buffer blowups for almost all of the buckets?  Does the good/bad stuff that gets moved into the bucket offset the good/bad stuff that gets moved out of the bucket?

MGL - Is your N the number of batters faced in the 9th?  The .211 wOBA in innings 1-8 for pitchers with a -4 run differential doesn’t seem remotely possible.  I looked at the 6 pitchers that I provided to you and their linear weights in the first 8 innings were only slightly better than average.  Numbers that might have translated to a .320 wOBA.  Lines like 3 HRs,3 doubles and 2 walks for 8 innings. That’s around a +7.13 linear weight. Minus about 7.20 for the 24 outs in the 8 innings gives a total of -.07 runs for 32 PAs.  I just don’t see allowing hits enough to allow 4 runs to score and having it translate to a .221 wOBA.

Just to try and clear up a few questions that were raised in this thread before it disappears.  First, my error in post #54 in assuming that the relief pitchers that replaced the starter in the ninth in tie games were closers. As MGL, Tango, and Guy quickly correctly pointed out they usually are not closers.  I found a way to quantify how often they were either the closer or another pitcher who saved nearly as many games in the same season as the closer.  Those pitchers came into the game 36% of the time.  As others have pointed out, not using the best pitcher available in the tied in the ninth situation is going to cost the pitching team some wins.  However, as MGL pointed out in post #55 using the closer in that situation burns him from being used in a future high leverage situation if the game remains tied and goes to extra innings so estimating exactly how many wins not using him in the ninth is complex.  Also unknown is whether the closer is actually available to be used in that particular game.  He may be injured or have been used in previous games so much that he is physically unavailable.  Unfortunately, the specific 25 man roster available for each game is not information that Retrosheet provides.

We can look at the win-loss outcomes for the tie games where a closer does come in the ninth and those where other relief pitchers are brought in.  If a closer is brought in the pitching teams eventually win 53.2% of the time (82 of 154).  If another relief pitcher is brought in the pitching team wins only 41.5% of the time (112 of 270).  These percentages should be tempered somewhat by the fact that in 89 of the 270 games where a closer doesn’t start the ninth, he comes into the game later and his teams win 59 of those games.  And of course as noted above, we don’t know if he was even available to come in for the 181 games where he never makes an appearance.

To summarize, it appears that managers are making good decisions on when to leave the starter in the game to finish the ninth even in tie games.  A team leaving the starter in eventually wins 51.9% (81 of 156), very close to the 53.2% of closers sarting the ninth.  There seems to be more chance of gaining wins by utilizing the closer instead of a lesser relief pitch in tie games.  This is a subset of the more general principle stated in The Book that it would be wiser to use the best pitchers available whenever high leverage opportunities arise.

The second point of clarification concerns the high wOBA that occurs in tied games in the ninth for both starters and relievers.  I looked at the component events in that situation and it appears that the out rate is the factor accounting for almost all the difference from wOBA in other run differentials.  The out rate is affected by two things, the incomplete innings of walkoffs and the sacrifice rate in tie games where MGL is not counting the sacrifice for the purposes of wOBA.  There were 580 ninth innings halves that were started in a tie.  If there were no walk-offs, that sould have resulted in 1740 outs. Usually around 2.5% of the outs are non-batter event outs or OOB outs leaving about 1698 outs that would usually be included in wOBA if it weren’t for walk offs.  But only 1550 are recorded.  So 148 fewer outs due to walk-offs.  There were also 59 Sac Bunt outs in tie games, about 55 more than expected in games that started where run differentials were greater than 1.  Together that’s about 203 fewer outs in the wOBA calculations for tie games than you have for the other run differentials.  Walkoffs also often decrease the value of the hit events that cause the walkoff if they are not HRs.  But this effect is minor compared to the outs.  The important fact to remember is that each inning that isn’t a walkoff ends on an out.  So every walkoff inning is going to have a higher wOBA rate than a similar inning without a walkoff. 

All the other components have rates similar to the other run differntials except IBBs which are also much greater in tie games.  The only effect of the increased IBBs would be the decrease in positive hit ball events by taking the bat out of the hands of the better hitters, but we don’t actually see this negative effect on the positive hit ball events.  The probable reason for this is that unlike IBBs in normal situations most of the extra IBBs in the ninth innings of tie games are IBBs to get force plays and not necessarily on the better players.

Great post by Peter, especially the part about confirming the bunt issue (which I was going to look at later today).

However this part is wrong (for analytical purposes, not for point of fact):

“So every walkoff inning is going to have a higher wOBA rate than a similar inning without a walkoff. “

This is a selection bias issue.  Once you have decided to look at things after-the-fact, you are going to get something that looks weird, and this is a prime example.

This is no different than saying that an at bat that ends on the pitch following a 3-0 count will get you an OBP of .950 or something like that.

Yes, if you know with 100% certainty that the at bat will end on that pitch, then you will have an obscenely high OBP because the only other way to end at that at bat is to actually swing and make an out on that pitch.  Since 92% of the time you are going to take on a 3-0 pitch, and since we are removing all the strikes you are taking on, all you are left with are the balls you did a take on, and the few pitches that you put in play.

This has been a recurring theme in these threads about the bias of the incomplete innings affecting the wOBA.  This is not an issue whatsoever.

The sac bunt though absolutely is.  I’m glad Peter tackled it, and I’m going to try to quantify it further in the other thread.

Peter, good stuff, I have to spend more time on it. I only skimmed it.

I don’t know that you are going to find much by looking for small differences in WP, are you.  Please give us the sample sizes and the standard error?  I almost never use WP unless I have enormous samples for obvious reasons.  Plus, when you use WP, even though that is what we are ultimately after, you have lots of other confounding variables in play that you are not controlling for. At least with wOBA we can control for most of those confounding variables.

Yeah, that .211 1-8 wOBA in games where the pitching team is down 4 or more runs doesn’t sound right. I’ll check on that.

I was thinking that since the sac bunt is around a neutral event, that not counting it in wOBA didn’t matter.  But I forgot that the only reason it is a neutral event is because the runners advance with the out.  wOBA obviously doesn’t know that the runners are advancing, yet we are not counting the out, so that is probably the main reason for the high wOBA in tie games.  And somewhat the opposite in games with large leads - there are no sac bunts at all, which would depress the wOBA, compared to the overall wOBA in a game (when I am not counting sac bunts). I wonder what it would look like if I simply counted sac bunts as an out?

MGL: see my other thread, where I look at base-out states.

Peter:  Great data. I agree that it’s hard to evaluate whether closers should be brought in more often in tie games, because we don’t know about their availability.  However, the fact that on average lower quality pitchers are employed in tie games than with a 2-run lead—despite the LI being equal—is prima facie evidence that teams could use the closer more in tie games (seems unlikely that availability of closers is any higher in one situation than the other). 

In games that are tied after 8 innings, can you tell us whether closers are used less frequently in home games than in away games (in the 9th, and/or in all remaining innings)?  It’s my perception that some teams fail to get their closers in at home because they will not bring him in w/o a lead, which of course is impossible at home.  A related question would be: is HFA any smaller in these games (tied after 8) than it should be?

Guy - Games tied going into the top half of the 9th, starting pitcher through 8: total-353 games, starters stay in 98 times, relief pitchers come in 255 times, closer comes in 131 times.  Games tied in the bottom of the 9th, starting pitcher throgh 8: total 227, starter stays in 58 times, relief pitcher comes in 169 times, closer comes in 23 times.