THE BOOK--Playing The Percentages In Baseball

I posted to Max:

Max, did you distinguish between home and away?

Can you look only at pitchers pitching in the top of the inning?

Bottom of the 9th offers its own limitations to number of runs a pitcher can allow, especially in your case, where he’s got at most a 2 run lead.  If you have a disproportionate number of starters there, then that’s a source of bias.

I’m actually curious about the good/bad day issue.  Controlling for batter quality, time of day and park does the variation in wOBA allowed by a pitcher really show a spread larger than a binomial process would predict?  OK, the answer to that is obviously yes, but how much more?  Enough that good day/bad day is a reflection of a significantly altered underlying talent level, as the figures imply?  I’m actually skeptical of this, it smells like the hot-hand fallacy to me.

The experiment I’d suggest would be look at wOBA against for the first N batters of the game, where N is small enough that almost all starting pitchers get there unless they get hurt (somewhere between 9 and 18).  I’d start with 9, in case the time through the order effect has an inning (or time) dependence.  Then for each pitcher compare the distribution of the wOBAs over a season to the expected binomial distribution and calculate a p-Value (or Chi-Squared).  Then look at the total distribution of p-Values (or Chi-Squared).  The p-Values should be uniform.  The non-uniformity tells you how much day-to-day performance differs from a single underlying talent level.  The goal would be to control for day-to-day variations that would also benefit a reliever (weather, park, umpire) as opposed to a variation in talent level for the starter.  I’m not currently set up with a database to do this, I wish I was.

Did this line make anybody else laugh?

I’m pretty sure La Russa knows this; otherwise, he would not have relievers in his bullpen.

I’m pretty sure La Russa would still play the matchups with a pitching machine on the mound.

If the percentages say it’s pretty much a coin flip, then you should definitely leave the (elite) starter in because then you’d still have the reliever in case you go into extra innings. I mean, obviously you’re looking to win the game, but if there’s equal chance of a tie game with either starter or reliever, then save the reliever for the 5th (or 6th or whatever) time through the order.

(of course, in the NL, if the pitcher is due up, pinch-hitting for him might make more sense.)

Aren’t these the situations managers are supposed to employ their knowledge, experience, and intelligence to decide whether to play the percentages?

In essence by having Carp bat, TLR was banking on the Phils no being able to score on CC29. Given how the game was going, all indications seemed to point to that conclusion.

PHL was not making good contact and StL had their best defense on the field.

While Iwas aware of the averages, and I could hear MGL ringing in my head when Carp walked to the plate, my overall feeling was congrruent with TLR. If Philly is going to win the game, they’re going to have to beat our best guy on one of his best nights.

If PHL did actually beat Carp on this night that is something I could live with. However, if the BP comes in and doesn’t match Carp’s effectiveness, %’d have regret and second guessing forever.

If there’s one guy that does not have an aversion to pitchibg changes it’s TLR. My guess is that he’s seen this Carpenter before and had a high degree of confidence in how it would play out.

TLR had more riding on this than any of us. If Carp blows the lead or the bullpen does, TLR would shoulder the blame and it could be something that cost him his job.

I think Carp’s single was mostly luck, but the CG SHO was just an extension of the great command and control leading to weak contact that we saw all night. IMO Carp and Wain are the only 2 guys that get this consideration because we’ve seen it before. Lohse, Jackson, Westbrook all get pulled and congatulated for a strong 7 IP.

Reminds me of this bit from May 25, 1981 SI article “He Does It By The Numbers” - “all American League starters in 1979 who held a lead in the eighth inning won 586 times and lost 18, a percentage of .970, or [12/1000]” (http://sportsillustrated.cnn.com/vault/article/magazine/MAG1124493/index.htm)

...I read that a couple years ago and instantly realized 9th inning closers are WAY overrated. Apparently some things in baseball never change.

What I find in intriguing is one-in-four of these “hot” pitchers achieved significantly worse results in the 9th inning compared to the rest of the game. After giving up 0 or 1 run through 8 innings, it appears one quarter of these pitchers proceeded to allow roughly 1.5 runs in the 9th inning. It’s true the relief pitchers did no better, but it still points out the results for the starters through the first 8 innings probably had more to do with random variation that any apparent “hotness” on that particular day.

Circle, your “English” explanation is useless.  Do you really think that manager’s “experience, intuition, and expertise” can figure out the right answer?  If you do, then you are dreaming.  I mean managers make ridiculous, silly mistakes all the time, believing in erroneous things like the hot hand, and batter/pitcher matchups, and other small sample nonsense.  Do you think that they magically become genius savants when it comes time to figure out when to take out their starter and when to bring in a reliever.

I certainly agree that if it is a tossup, you probably want your starter in order to save your reliever for a possible extra inning game or for tomorrow.

But, if your starting pitcher comes to bat late in a close game, and it is not an obvious bunt situation (with no outs), and especially with runners on base or leading off an inning, since the pitching aspect alone is a tossup (presumably), then it is a no-brainer as far as pinch hitting is concerned since there is always a large difference in WE and RE between a pitcher hitting and a pinch hitter hitting in a high leverage situation.

All the nonsense about, “The manager believes that the starter can shut them down, and the other team is not making good contact, and the team is ahead in the game anyway,” is just that - nonsense. All those “English” explanations will not help to facilitate the right answers in any way, shape or form.

Anyway, I have not read Max’s article yet, but I agree that there needs to be a lot more investigation and controlling of all the variables before we declare it a tossup…

Wow, as much as I love Max’s work, I am not crazy about this article to say the least.

He makes conclusory statements before presenting any evidence:

Carpenter on his best night is probably better the fourth time through the lineup than the average Motte coming out of the pen.

Then the evidence is scant at best.  He looks at the distribution of runs scored and concludes that it is a coin flip?  Maybe it is, but what about wOBA, OPS, ERC, or even RA.

And as we have discussed (and come to no conclusion whatsoever) there are tons of possible biases in both groups such that we may not be comparing apples to apples.

Finally, he says this at the beginning:

Full disclosure: If I were managing in a deciding game and that situation occurred, I would NOT substitute for my starter.

Why?  Would Max deliberately make a sub-optimal decision for some reason? If yes, I’d like to know the reason.  Fear of criticism from the fans or media?

If, as Max opines after presenting some evidence, it is around a tossup as far as the 9th inning pitching is concerned (and sure, if we use the closer, we may not have him for extras, although her certainly might be able to pitch another inning), wouldn’t it be a no-brainer once you throw in the substantial loss for not pinch hitting?

Max paraphrases one of my arguments as this:

“According to Lichtman, a.k.a. MGL, going to a better hitter was a no-brainer according to the numbers.

Well, isn’t that pretty much true (see my above sentence), given Max’s research (that it is a tossup for the pitching alone)?

I’ve already admitted that I may have been wrong about the pitching, but I don’t think I am wrong about the pinch hitting (in this game and in general).

Typically, you give up .25 or more runs by not pinch hitting.  So the starter would have to be 2.25 runs per 9 better than the closer for him to pitch the 9th. If you want to throw in the fact that the closer might not be available for extras, OK, we’ll cut that 2.25 runs per 9 down to 2 or 1.5, or 1, or .5!  Is there any data that indicates that a starter who is “on” is that much BETTER than a closer?

I think Max was more interested to make an illustration about good-day Carp and bad-day Carp.  I was happy with the way he laid it out,

I think when a guy has managed thousands of games, he has some experience of seeing aces pitch on their best nights and has some idea of whether the opponent is likely to get to him or not.

In this case Carp was not showing signs of fatigue such as walking batters, getting hit hard, and/or elevating pitches.

I concede having Carp bat was a big risk. If he makes an out, he sits and rests, and StL has the top of their lineup at the plate and the meat will bat in the 8th or 9th. I wouldn’t bank on a multi-run rally against Doc, and they were able to load the bases via a GB single, FC on bunt, bunt, and IBB. Bases loaded 1-out for 4 and 5 hitters. Not scoring there was def against the odds.

But, it’s TLR, so we’re looking at Carp for 2 IP versus possibly 4 StL relievers, Scrabble and Rhodes against LHBs and Motte and another righty against the RHB’s and possibly a 5th reliever if Utley or Howard reach base.

What I am saying is that I would go with Carp on a short keash before turning to a handful of average relievers. If I have Rivera or I’m tied or down a run Carp is out without a second thought. See 2009 LDS for an example.

My guess is Philly would have loved to see relievers enter the game for the same reason I don’t in that very specific situation.

Again, you can spew all the rhetoric you want, but the bottom line is that by not pinch hitting for a pitcher in the late innings, you give up .25 runs or more.

Please explain to me how you can gain those .25 runs in one inning, with numbers (and words, of course) rather than just words?

You’s make a great politician, Circle!

If I could do it with numbers I would (at least try) but I don’t have that skill. It would also save us some time.

The .25 runs lost by CC batting, does that factor in that the PH would be a below average hiiter coming off the bench to face the best pitcher in the NL on an “on” night, on the road, in an elimination playoff game? Or is it based on large amounts of regular season data encompassing all situations?

Likewise, over 2 IP is an “on” Carp better than .25 runs of a few average, inexperienced relievers, who are also pitching on the road in an elimination playoff game?

I’m not trying to be a politician as I realize that means you think I’m just talking BS but trying to see how non-average factors affect average data results.

In short I trust Carp for 2 more innings on a night when he’s pitching masterfully over a handful of inexperienced, some newly acquired, average or below relievers pitching on the road against the best team in a shit your pants game.

I’m normally a data-driven decision maker, but what I am questioning is how a unique environment affects the average data. Like I said if I could accurately quantify all of the aspects and calculate the projections in runs for the various situations, I would.

I like Max’s method, which should control for a lot of factors. But I would like to see the 2 samples compared by these factors, to be sure:
Home/away
RS and RA (thru 8)
Year
AL/NL

"I’m not trying to be a politician as I realize that means you think I’m just talking BS..”

Nothing personal, and I think that you provide lots of great and interesting insight and experiences on this blog, but yes, that’s exactly what I think.  Not intentionally of course…

"does that factor in that the PH would be a below average hiiter coming off the bench to face the best pitcher in the NL on an “on” night, on the road, in an elimination playoff game”

But isn’t Carpenter ALSO going to be facing that same pitcher in those same circumstances?

Are you suggesting that because of that, it’s going to affect the PH more than it’ll affect Carpenter?

Nothing personal, and I think that you provide lots of great and interesting insight and experiences on this blog, but yes, that’s exactly what I think.  Not intentionally of course…

No offense taken.

Like I said, if I could do the math and could accurately quantify some of the variables I’m considering, I would.

I would be interested in seeing the actual runs of:

[1] Carp batting and pitching the 8th and 9th.
[2] PH vs. Halladay, performance of 3-4 relievers over the 8th and 9th, based on matchups.
[3] PH vs. Halladay, 1-2 relievers in 8th, Motte in 9th.

The article in this thread just shows carp vs. Motte in the 9th. But the actual situation is reliever over the last 2 innings, and Philly staggering their RHB and LHB, so TLR possibly uses 3 relievers in one inning with Scrabble and Rhodes likely being used against Utley and Howard.

-------------------------------

Are you suggesting that because of that, it’s going to affect the PH more than it’ll affect Carpenter?

No.

Since you said “no”, then that means that particular argument you set forth is irrelevant to bring up.

I said no because I know due to wOBA that any PH has a better chance than Carp at getting on base.

My view was that how Halladay was pitching it doesn’t matter, it’s a longshot for Carp or PH to get on base.  Whether the PH craps his pants or not probably doesn’t matter since his odds of getting on base are so low anyway.

The 2-4 RP StL would use in the 8 and 9 inn are the ones I’m worried about. I’ve onky seen data presented on Motte. I haven’t seen how many runs having a 4th reliever in the game instead of ace affects the run expectancy.

But you have to do odds ratio, yes? And the effect of the great pitcher will be the same on both using odds ratio, which creates a larger arithmetic penalty for the better hitter.
For instance, assuming both are in the same OBP=1/3 league (this is close enough, I assume), if you take a .200 OBP hitter and a .400 OBP hitter and have them both against a .250 OBP against pitcher, the .200 guy drops ~.057 points and the .400 guy drops a whopping ~.092. So a bigger difference on the better hitter. How much of a difference is based on how good the respective hitters and pitcher are. So I’m not sure who you want to pinch hit with, and I don’t have wOBA handy, but just using OBP, Carp this year is .178, Halladay was .269 against this year, we get an expected .145 matchup - he’s lost .023 points of OBP. Now if you’re PHing with a .300 hitter, he loses about .048 points, a .350 hitter and a .400 hitter both lose about .053 points. So whoever you’re PHing with takes a little over double the penalty of Carp in PHing.
Furthermore, this is late enough in the game you can’t just look at .25 runs. You should look at the chances of each discrete run level, both in scoring and allowing. I’m not sure who this favors, and it’s harder to do, but it’s to the point where WE has taken over, no?

Having said all that, I’m still pretty sure the ‘right’ move was PHing, assuming a suitable number of subs, particularly relievers were available (and I have no reason to think that wasn’t the case).

The crux of Max’s argument appears to be; “Though we should expect Game Scores variation for robots as well (due to luck), it’s safe to assume a significant portion of the variation in the chart is caused by Carpenter having good and bad days (due to health issues, psychological factors, luck...whatever).” I don’t see any research to back this up, and his subsequent chart of Carpenter on good and bad days appears to be pure guesswork. My instinct is that I would be astonished if there were anything resembling this much variation in actual talent level from day-to-day; simple, random chance will produce massive variation in game results, and perception bias leads us to believe a pitcher is “on” or “off” on a given day. I’m sure there really is day-to-day variation, but I would expect all of Carpenter’s days to occupy a tight band which may not even overlap Motte’s.

The Book largely exploded the myth of hot and cold streaks over multiple days, hasn’t the same analysis been done for in-game performance?

The PH thing is pretty much a gimme, even if the odds are long against either one reaching base on Halladay. StL loses runs by having Carp bat. But, that’s only HALF of the equation.

The other half is whether that -.25 runs (if that is the actual number in this specific situation) if balanced or defeated by a benefit of having Carp pitch the last 2 innings.

Philly will send up the following batters:

8th Inn
------------
R-Polanco
R-Ruiz
L-Gload (or other PH)
S-Rollins

9th Inn
-----------
L-Utley
R-Pence
L-Utley
S-Victorino
L-Ibanez

So, StL could use any number of pitchers over these 2 innings, really anywhere from 3 to 5 relievers. We all know TLR is not going to go with 1 reliever in each inning given the arrangement of PHL’s lineup.

What I want to see illustrated, and I wish I could do it myself, is how the projected performance of these relievers fares versus a cruising Carpenter over the last 2 innings.

Granted if you have a lights out closer, it’s a much easier decision. When you don’t, it’s not. StL don’t.

So, does the question still remain whether [1] X runs lost with Carp batting > [2] Y Runs saved by having Carp pitch instead of Z relievers over the last 2 innings? Or am I the only one dense enough to not see it’s obvious?

We’re likely looking at:

Dotel/Salas & Loogy in the 8th.

Possible pitching change in between 8/9.

If not, then loogy to face Utley and Motte to face Pence and Howard (or other loogy to pitch to Howard).

Once you take Carpenter out, you turn LaRussa loose to do what he does ... and I’m not convinced that having these specific relievers in this specific game are not worse than having Carp bat and finish the game. Am i reasonably unconvinced or just being naive and/or stubborn?

Max’s study—even if it holds up to scrutiny—doesn’t necessarily demonstrate a hot hand effect.  It would simply show that the starters allowed to pitch the 9th are roughly as good as the closers.  To decide whether that implied a hot hand, we’d have to compare this to what we’d expect these same starters and closers to normally do.  It could be that we’d expect them to perform about the same in any case (it depends on how talented this pool of starters is, compared to the relievers). 

This may be one of those rare issues where multivariate regression is actually the best tool.  There’s a lot we want to control for here.

I seem to recall that MGL did some new research on this issue in the past 18 mos or so, looking at starters’ performance in the early innings and finding it had little or no predictive value for the late innings once you accounted for various factors (including temperature).  Anyone remember that thread?

In order to see if the degree to which you have good-day pitcher and bad-day pitcher is to look at the distribution of observed wOBA, and see how it compares to the distribution if the pitcher had a non-changing talent level each game.

You’d have to control for the different opposing lineup, park, temperature, and one or two other things.

That would be tricky, because you’d need to account for varying sample size and selection bias.  You would initially appear to have a lot of evidence for pitchers having “bad days” (i.e. more high-wOBA days than expected due to chance), simply because they were taken out after a few bad innings and there was no chance for mean reversion to work its magic that day.

Tango/24:

Yes, that’s the experiment I proposed in 2.  I think umpire, times through order and defensive quality would be needed as controls as well.

I’m not sure how much resolving power this would have, and it isn’t clear whether finding anything would indicate changing talent level or some other uncontrolled factor.  Injuries would almost certainly result in some indication of changing talent level.  You might also want to throw out starts that are preceeded or followed by a DL stint.

I think this is a really important study to do, though.  I seem to recall a similar thread for hitters that actually showed remarkable agreement with the hypothesis of constant career OBP talent.

Guy/25:

You would look at only the first 9 or 18 or some constant number of batters faced.  It would have to be small enough that almost all starters face that many hitters, so you capture all bad days, but big enough to find an effect.  That would also, hopefully, control for times through the order.

Guy: it’s simple enough to handle the differing number of PA.

If let’s say a pitcher faced on 16 pitchers, and his observed wOBA was .500, then we figure out how much 1 SD is for 16 pitchers:
1 sd = .5/sqrt(16) = .125

And then we figure out how far away that observed wOBA was from his rest-of-season observed wOBA (say that it was +.150 away).

So, his z-Score is going to be 1.2.

If you have a perfect game (wOBA = .000) in 27 PA, and he’s an otherwise .320 wOBA pitcher, that’s 3.3 SD.

You do a standard deviation of all the z-scores, and, if you get 1.00, then it’s all-random.  If you get higher, then you have good and bad days. (Or, you didn’t account for some source of bias, either those I listed, or some other ones.)

Does someone else want to do this study?  Fame and fortune awaits.

Tango:  doesn’t it matter that performance and sample size are correlated?  There will be 4 IP @ .420 wOPA outings, but no 4 IP @ .180 wOBA outings.

This is certainly a valid approach.  But I think it’s more on point to look at the relationship between late-inning and earlier performance within games.  What we really want to know is how much predictive power, if any, there is in the starter’s performance in the first 5, 6, 7 or 8 innings.  I don’t think that knowing there is “excess variance” over the first X batters will give us that answer.

Guy:

I actually think the answer to the existence of game specific talent level is interesting in its own right.  Not to mention determining the size of the effect, if it exists.

I think Tango’s right, accounting for the variance the way he describes and doing a test of the distribution of the resulting samples is sufficient.  I’d do a more powerful statistical test like K-S, rather than just measuring the SD of the z-scores, but the idea is the same.  If you were doing a regression an not weighting properly, then it is a problem, but not the way Tango has described.

I actually really want to do this study, but don’t have a database set up.

Here’s my back-of-the-envelope calculations: Carpenter, career-wise, retires about 70% of the batters he faces. If he faces 36 batters in a (usually not complete) game, the usual margin of error would be about 16%, so he could retire anywhere from 54% to 86% in a particular game just by random chance. One game in six he’s 62% or worse, while one game in six he’s at 78% or better. Isn’t 62 to 78% enough of a swing to give the appearance of “can’t fool anyone” to “he’s really dealing tonight”?

I think the 13 GBs through 7 IP is an indication of CC29 dealing. 5 of the last 6 outs were GB outs.

I think looking at a pitching chart of the game would reveal just how dominant he was in terms of strikes, getting ahead, keeping the ball down, etc.

I don’t want to be the guy that’s talking about the “art” aspect of it. But, going through the game did anyone really feel as though either pitcher was going to give up any more runs? I’m not saying that trumps data, but rather I would imagine the data aligns with the observations with a high correlation.

Did we really feel that either pitcher was just going to lose their command and control or become hittable over the course of one inning? That’s the part that is confusing to me, there were no indications that either guy was slipping at all ... not in pitch location, batter contact, etc.

We know that pitchers generally do worse each time through the lineup. It would be interesting to examine if their location, movement, etc changes or if it is simply batters getting a better gauge on their speed/movement/sequence/etc. There has to be a cause for the effect. So, if a “cause” can be established that leads to “effect”, then not seeing the cause in the observed game could lead one to believe that they won’t see the “effect”, right? (hopefully I worded that correctly).

Lex, I looked at Halladay and Carpenter, the typical PAs per start is more like 29.

That makes Sigma_OBP = 0.084 (I used OBPa = 0.285, which 30 starters equalled this year)

So, 5 times per season OBPa is less than 0.200 over 29 batters just on random variation alone.  That might not qualify as cruising, but with park effects, opponent quality and umpires widening that distribution, there’s going to be a lot of apparent cruising going on without any underlying talent variation.

Circle:

Carpenter has a career GB% of 51%.  Through 7 IP he had allowed 22 BIP.  13/22 = 59%.  That’s not even moderately unusual.

5/6 GB happens 10% of the time without any talent variation.  Your evidence is not evidence of anything at all.

Perhaps.

His 18 GB for the night was 3 higher than his season high.

What would be interesting is to see his other stats (i.e, runs allowed) when he’s getting a lot of ground balls (more than average).

In general it means an effective Carpenter, but there have been times when he has gotten above average GB totals and given up a handful of runs.

The GBs were just one item I mentioned. His command and control stayed sharp the whole night. That’s what I’d look at primarily.

Based on experience, command (throwing the pitch you want with movement where you want it) is the first to go, namely location and balls inching up in the zone ... often to the pitcher’s PAS. So, for a RHP, that would mean pitches ending up being a little higher and middle in than in previous innings. That’s a major red flag for me, I didn;t see any of that from Carp (as opposed to tonight’s game where he was clearly off.

I wouldn’t use GBs alone as the deciding factor of whether to leave him in or bring in a reliever.