THE BOOK--Playing The Percentages In Baseball

fantastic post

Thank you!  We aim to please!

Interesting…

What was the criteria for selecting the events to include in the 6000 PA’s that constituted the “got hammered” set? 

I was thinking if you could find the 50/50 point in wOBA (regardless of number of PA’s, i.e. not always looking at the first time through the order), where half of the pitchers were taken out and half were left in, that would be a good way to divide up the data.  Then you could look at the half that stayed in, and see if the rest of their game was better or worse than that dividing line wOBA…

Anyway, it is interesting to see that there is apparently an asymmetry here, in that superior performance is not predictive, but subpar performance is (or possibly is?)

One last thing, I’m wondering why you figure that controlling for things like park, weather and home/road would reduce the magnitude of the observed difference?  Can you give a quick rundown of why you think that would be the case?

Tango did the research, so he’d have to answer most of those questions.

As I say in the post, pitchers who have pitched well at any point that you look at in a game will tend to have pitched in “pitcher’s conditions” (cold weather, pitcher’s parks and at home).  So they will also tend to pitch better than expected (if you are not controlling for those things) in any other time period during the same game.

I am actually not sure that Tango did not control for these things. He doesn’t say he did in The Book so I assume that he didn’t, but it is important to do so.  How important, I am not sure. It could end up being only 1 or 2 points in wOBA, which would make it not that important.

Just like you have to control for the opponents, which he did of course.  Any group of pitchers that pitch well, for example, the first time through the order will have tended to have pitched against weak opponents, so if you look at the second time through the order, they will look like they pitched well again, if you don’t control for the opponent.

Any time you look at players who performed well or poorly in one “split” and you want to look at how they performed in another “split”, you have to control for every external (outside of the player’s ability) factor that can bias performance otherwise you are always going to get a positive correlation having nothing to do with the player per se.

This is what I’m talking about. You are asking and answering two different questions.

You ask if a pitcher is pitching well, will he continue to pitch well? Then you plug in the wrong data. The data you are using will answer, if a pitcher is lucky, will he contine to get lucky? We all know the answer to the second question, but nobody seems to know the answer to the first.

Everybody on this website and some managers know whether or not a pitcher gets nine up and nine down has very little to do with how well he’s pitching. Some managers might not, and that might lead them to making some bad decisions. In that case it is their misread of the previous data that causes them to make the mistake. They are using the wrong input.

As for this study, I would much rather see it done in xFIP or QERA or anything that limits the luck factor, because you have to adjust for that as well weather/opponent/etc. If you use a luck adjusted stat, your subset of pitchers will have a much higher percentage of pitchers that are actually pitching well. From the looks of it, even more importantly, you will have a much smaller percentage of pitchers who are pitching poorly.

This is excellent stuff but #5 is right too.  I might go one further (b/c over such a short time frame xFIP is mostly luck too) and say that we should ask questions like - If a pitcher has had above average velocity (for him) is he likely to continue to have above average velocity?  If a pitcher has had above average movement is he likely to continue to have above average movement?

Re: #5 “If you use a luck adjusted stat, your subset of pitchers will have a much higher percentage of pitchers that are actually pitching well”

Are you suggesting that xFIP or QERA should be used to determine which pitchers are pitching well?  You don’t think that pitchers who have retired the first 9 batters with 29% K rates will have good xFIPs?

This post is obviously not targeted at people who have already read and believe the studies in the book, since it’s basically a summary of a study in the book. 

The pitch f/x stuff would be interesting, if a little time consuming.  ‘Good’ pitch movement vs. ‘Bad’ pitch movement would be especially tough to define without manually reviewing each pitcher.  Velocity would be easier, but is there evidence that increased velocity helps?

Velocity would be easier, but is there evidence that increased velocity helps?

There’s evidence that guys who throw harder have more success (link).  Not exactly the same, I know.

Yes, of course ideally you want to look at pitch f/x data or something like that.  But at the same time, there is the question of whether if a pitcher is pitching well “to the naked eye”, will he continue to pitch well (or vice versa).  That is a legitimate question since that is what fans think and many managers.  You don’t get too many managers who look at a pitcher who has given up 3 hits, 1 walk, and zero runs in 5 innings and say, “I’m taking him out.  Despite those numbers, he’s not really pitching well.” Not too many at all…

True, the manager can’t (and maybe even the catcher can’t) see the difference between a good movement/velocity day for a pitcher and an average one.  The catcher can definitely tell whether a pitcher is hitting the glove and a manager can definitely see if he’s falling behind in the count, getting swings and misses or giving up hard line drives and deep fly balls.  So, they do have more info than ERA/wOPS provides.  Do they use it?  That would be an interest study in and of itself.

btw, in addition to controlling for weather, opponent and park, you’d have to control for home plate umpire, I think.

Parts of this post come close to my curiosity about the 100-pitch limit now used almost across the board.

Is it your opinion that managers might be helping their team with this strategy even if inadvertently by jerking a pitcher throwing well out after the sixth inning when they seem to be sailing along versus letting them pitch another inning. Or even, God forbid, a complete game.

I think the 100-pitch limit is primarily used to limit bad arms but could it also be an improvement versus pitching a guy until the manager begins to think he is getting hit or tiring?

Basically I’m wondering if you think the use of the 100-pitch limit is a step forward in terms of strategy or if the approach of the older days was better? This obviously depends on the quality of the bullpen and the manager’s decision making. But all things being equal, what’s your take based on the numbers like these?

Great stuff. The outlook for pitchers who get intially hammered might be even bleaker than your numbers suggest: the manager might have had some reason for leaving in those starting pitchers who got hammered first time through the order. That is, the data presumably necessarily exclude pitchers who got hammered immediately and were taken out thereafter; these pitchers might have performed even more terribly in later innings than did their counterparts who were left in. Of course, perhaps managers have no (statistically significant) idea what they’re doing in this regard. I get that feeling when I look at most of them. I virtually know it when I hear Charlie Manuel, Dusty Baker, etc. But perhaps worth taking into account.

Great post!

I’m confused about this part:
But, remember that the 4th time through the order, we expect that a pitcher will be around 8 points worse than his overall wOBA allowed or around .25 runs in ERA.  So our gain of 29 points above for a pitcher who has been pitching well, is only a net gain or 20 points or so, which is still a lot - around .70 in ERA.

If a pitcher typically does worse the 4th time through the order, doesn’t that make the improvement the 4th time through the order even more impressive? Maybe I’m missing something here, but it seems to me the effect should be 37 points of wOBA or so, which is an ERA difference over 1.

#7, As a group, of course they are going to have better xFIP than normal, however, you will still have plenty in there that don’t. By using a luck adjusted stat, you will definitely have a higher percentage that are pitching better than normal within your subset and likely increase the sample size as well.

#8, All that shows that most managers don’t know who is playing well and who isn’t. No need to argue that one. To use their faulty decision making in a study to determine if someone who is playing better/worse continues to play better/worse is just as bad as them making the decision in the first place.

A couple of other questions that pop up for me.

If inexperienced pitchers are more likely to pitch below there true talent level the second time through the order after getting shelled, they must pitch better than normal when they don’t get shelled. How much can we carry that from start to start? Is their a similar effect with inexperienced hitters?

Good questions and comments.

Jack, yes, that is why I said we have a selective sample.  Obviously when a manager leaves in a pitcher who was getting shelled, he has some reason to have left him in.  Unfortunately, we can only look at pitchers who were left in and not the ones who were taken out.

#14, yes there is the the issue of symmetry.  If a young pitcher pitches really badly after getting shelled, he must pitch exceptionally well after pitching well.

Same problem with Tango finding that 9 batters of great pitching does not have predictive value but 9 batters or getting shelled does.  There is no symmetry there, which doesn’t make much sense. There has to be symmetry.  IOW, if you find that one subset of performance is better (or worse) than average, the remaining subset HAS to be worse (or better) than average. That is what I call the “law of symmetry.”

shawn, yup, I definitely blew that one.  37 points, not 20, which is over a run in ERA!

I’s like to do some research over again to answer some of these questions.

Sometimes when you are looking at sample data and your samples are not all that large, it is difficult to answer questions like these with any certainty. As well, if you attempt to ask and answer too many questions, you will likely end up making Type I and II errors.

Another question about pitchers. How did the experienced guys do when they were inexperienced compared to after they were experienced? In other words, is being able to regroup after getting shelled a skill that is learned with experience or is it something that does not improve once they get to The Show?

"including the fact that short relievers are much better pitchers than starters in general”

I was under the impression that starters as a group were much better pitchers than relievers?  If short relievers are better than starters, why not make them starters to get them more IP?

To try and put this as clearly as possible:

The average starter has a higher ERA/RA/FIP/whatever than the average relief pitcher - when each is used in those roles.

The average starter, used as a relief pitcher, would perform better than the average reliever.

The average reliever, used as a starter, would perform worse than the average starter.

To further Colin’s comments, that really only applies to “short relief.” When pitchers are used in long relief, they are about as good as when they start.

Thanks for the clarification, I just misinterpretted the meaning of the statement.  Do you think a more optimal use of pitchers might involve using a lot of pitchers in short stints rather than a starter in a long stint with the bullpen to finish it up in short relief (at least as a replacement for the back end of the starting rotation)?

@colin#19/mlg#20
bit of a tangent, but…
is the difference between a short reliever and a starter completely attributed to the fact that a short reliever never sees a batter more than once (in a game)? in which case, you could say that the reliever’s statistics/total performance is better, but only due to a selection bias. (see also: LOOGY)

jsolid, it is presumably that, fatigue, and the fact that they can go all out, knowing that they are only going to pitch 1-2 innings.  For example, you see a difference in fastball speed, although I don’t know what that average difference is - have to ask one of the pitch f/x guys.

“e clarification, I just misinterpretted the meaning of the statement.  Do you think a more optimal use of pitchers might involve using a lot of pitchers in short stints rather than a starter in a long stint with the bullpen to finish it up in short relief (at least as a replacement for the back end of the starting rotation)?”

Yes, exactly.

jsolid: see The Book, plus a recent thread, I think in October.