THE BOOK--Playing The Percentages In Baseball

Here is an example or a manifestation of how everyone thinks that pitching win games and especially post-season series:

It is the bottom of the 7th in tonight’s MIL/PHI game and the announcers just said:

“The story tonight is the Philadelphia pitching.”

Let’s see.  The score is 5-2.  The Brewers have batted in 7 innings.  They are supposed to score against Myers and relievers around 4.5 runs per 9.  In 7, that is 3.5.  So they scored 1.5 runs less than what they were supposed to.

The Phillies scored 5 runs in 6 innings.  They were supposed to score around 3.7 per 9 versus Sabathia and his relievers.  That is 2.27 expected runs for 6 innings.  They scored 2.73 runs more than they were expected to.

So how is that,"Pitching is the story of the game?”

If it’s true that postseason games are lower-scoring than regular season games, then I can see how the skill of their closer could have a little more weight than in regular season games.  If you tell me the Red Sox-Rangers are going to have a 5-2 game, I’ll guess the Red Sox are 60% favorites.  If you tell me those two teams will have a 3-2 game, I’ll bump it up a bit in favor of the Sox, because I think they could hold a 3-2 lead better than the Rangers, due to the strength of Papelbon.  But I can’t even imagine how negligible this difference might be.

Last I checked, since 1969, the average playoff team scores 5 runs and allows 4 runs in the regular season.  In the playoffs, it’s 4 and 4.

So, when we see Derek Jeter have the same hitting stats in regular and post, this is mightly impressive.

The same applies in the NHL by the way.

Well sure, you have teams with some of the best pitching (as well as the best offense), then you have only the top 3 or 4 starters pitching, then you have the best of the bullpens, and closers pitching as much as possible.  Plus lots of lefty/righty jockeying. If you hit as well in the post-season as in the regular season, you are indeed doing one heck of a job.

Agree with the post.  Nate Silver’s use of his “secret sauce” has always made me cringe.

Even if the main idea behind secret sauce were sound, since when is one season’s worth of WXRL an effective way to measure closer effectiveness? And since when is FRAR a desirable way to measure defense? The idea that WXRL and postseason success have a surprisingly high correlation is like saying that having a last name beginning with M has a high correlation. It’s total applesauce.

Perhaps Baseball Prospectus would be kind enough to debate their point here, presuming this blog is not too small potatoes for them.

That would be great (#7).  I am open to hearing why their position might be valid.  If they just say, “Well, that is the way the regression came out,” I am not convinced.

As I always say, almost everything is a Bayesian problem more or less.  That is why the “why’s” are important.  They serve as a proxy for the a prior probability.

For example, let’s say that a regression came up with a very significant correlation between, as Phil says, teams that start with the letter “L” and playoff success, we would know that that was likely a Type I error because going in (the a priori p), the probability of team names correlating with post-season success is near zero.

Whenever you deal with sample data, you always have the possibility of Type I and II errors, no matter how large the sample.  To reduce the chance of you accepting them, asking “why” is often an important component of the analysis.

Not to mention that there is always the chance that there is a mistake in the methodology or computations.

So, as I said, I am open to a discussion of the merits of their conclusions, but I am more than skeptical.  There are only so many ways to win (or to have a certain chance of winning) a baseball game and it all boils down to your offense and defense (and base running).  Sure there are subtle differences between post-season and reg season, and there are always leverages to be exploited in any game, but it still all boils down to offense, defense, and base running.

I especially don’t like the idea of defense being more “important” than offense, other than the fact that they are not quite equivalent, pound for pound.  But, for example, the idea that “good pitching beats good hitting” in the post-season but not in the regular season is kind of preposterous, IMO, and certainly unlikely.  And finding a strong positive relationship in a regression equation is not going to convince me.

But I would certainly love to hear the “explanation.”

As I always say, no matter how strong your opinion is on something, it really isn’t worth a whole lot until and unless you hear both sides of the story from some kind of experts.

"Last I checked, since 1969, the average playoff team scores 5 runs and allows 4 runs in the regular season.  In the playoffs, it’s 4 and 4.”

So, what does this imply? Does this imply that good pitching stays fairly consistent regardless of the opponent, whereas good hitting varies more according to the opponent?

I always thought of this as sort of a “playoff paradox”. Hitters generate their regular season stats playing a combination of good pitchers and bad pitchers. If they only faced good pitchers, their stats would most likely be worse than in the regular season.

Likewise, pitchers generate their regular season stats playing a combination of good hitters and bad hitters. If they only faced good hitters, their stats would most likely be worse than int he regular season.

So, when you get to the playoffs, most match-ups are good hitters versus good pitchers. Yet, it’s impossible for both pitchers and hitters to post worse statistics, seeing as how baseball is a zero-sum game, and any good thing that the offense does is a bad thing for the pitcher, and vice versa. Thus, one effect has to dominate. Either the good pitching has to shut down the good hitting, or the inverse must be true. You can’t have both pitchers and hitters being worse in the playoffs than in the regular season.

Suppose you have a league of 4.5 runs per game, where the good team have pitching giving up 4 runs per game and the hitting gets 5 runs per game.

In the playoffs, only the really good pitchers pitch.  They allow say 3.5 runs per game in the regular season.  Only the good hitters bat, and they score say 5.2 runs per game.  That dynamic would show that we should expect 4.0 runs per game.

That is, there’s been a shift in the talent base of who is actually performing in the playoffs.

I can’t say that I’ve looked at this for MLB, but I have for the NHL, and the most significant factor in playoff performance is (pythagorean) winning percentage.  Strength-of-schedule, deadline deals and injuries play a role.  But goal scoring and goal prevention are the most significant factors - and I didn’t find an advantage for high-scoring vs strong-defense teams.

BP suscribers can read Nate Silver’s secret sauce article from 2006 here:
http://www.baseballprospectus.com/article.php?articleid=5541

Or, apparently, even if you don’t subscribe.

Here is the crux of the article:

With the pennant races settling down more quickly than we might like, it’s not too early to start thinking about which teams might have the pole position in the post-season derby. As Dayn Perry and I found in Baseball Between the Numbers, regular season success is no guarantee of playoff performance. Rather, there are three particular characteristics of teams that win more than their share of post-season games. These characteristics are as follows:

More remarkably, all three of these characteristics relate to run prevention, rather than run scoring. That does not mean that offense is of no importance in the playoffs. But there is a lot of noise in the postseason record, and offense did not produce enough signal to emerge through it. The reasons are too complicated to get into here, but have to do with what happens when good offenses face good pitching. Pitching does have some tendency to dominate these match-ups, whether they occur in the regular season or in the playoffs. Because “plus pitching” versus “plus hitting” duels occur more frequently in the post-season, we tend to notice the effects more then.

This of course:

But there is a lot of noise in the postseason record, and offense did not produce enough signal to emerge through it.

means that the samples were too small.  It makes NO sense to say that offense is not significantly correlated with winning in any game or series, be it reg season, post-season, or spring training.  How do you not score a lot of runs and not increase your chances of winning?  Remember that these correlations assume that everything else is held constant.  In other words, even if there were lots of teams in the post-season who had good offense but they also happened to have had bad pitching and defense to go along with it, it should not have made a difference as the regressions would have adjusted/accounted for that.

The reasons are too complicated to get into here, but have to do with what happens when good offenses face good pitching. Pitching does have some tendency to dominate these match-ups, whether they occur in the regular season or in the playoffs.

If that is the case, then yes, it might be that good pitching is more important in the post than in the reg.

It should be easy to check if it is true, and if it is, to what degree.  Tango did NOT find that to be true, as far as I can recall.  I think it is in The Book.  Maybe he can explain.

If you just look at the offense and pitching of teams in the post-season, as measured by their offense and pitching in the reg season, then of course you will conclude that somehow good pitching beats good hitting.  I would hope that is not what BP did.

Let’s say that all your post-season teams score 5 runs and allow 4 runs in the reg season in a league that scores 4.5 rpg.  IOW, all post-season teams are a half run better on offense and pitching.

You would expect that in the post-season, teams will score and allow 4.5 rpg.  But, in reality, they will score and allow 4 or 4.2 rpg.  You might conclude that pitching was more important, that the good pitching somehow depresses the good hitting more than a half run a game.  But you would be wrong.

What is happening, of course, is that teams that score 5 rpg during the reg season will score around 5 rpg in the post-season if they faced average pitching.  But teams that allow 4 rpg in the reg season will have much better pitching in the post-season regardless of the opposition.  Why?  Because they only use their 4 best starters and the better starters get more starts than the worse ones.  And the starters throw fewer innings per game.  And they only use their best relievers and the closer pitches a lot more, percentage-wise.  Basically a team that has a 4.0 rpg pitching staff in the reg season has a 3.5 or 3.6 rpg pitching staff in the post-season.

So the correct way to figure how many runs should score in the post-season is to look at the personnel and the playing time in the post-season and to see what their “true numbers” are.

So instead of 5/4 teams going into the post season, you really have 5/3.5 teams going into the post season and thus in the post season only 4 rpg are scored.

That is the reason why fewer runs are scored in the post-season, BTW (besides the colder weather).  You have better starting pitching, you have more judicious use of the bullpen (the best relievers and lots of righty/lefty matchups), and perhaps a little more small ball.

Other than that, I really don’t think there is any “good pitching beats good hitting” going on. We don’t see that in the reg season when we look at good pitchers against good batters.  Why should we see it the post-season?

I hope that BP did not run their regressions by using reg season numbers and then regressing those on post-season success.  That would be a poor methodology and would yield incorrect results.

Let’s see, Rivera throws 5% of the Yankee innings in the regular season, throws 10% in the post-season, makes up a significant portion of all relief innings in the post-season of ALL pitchers in the last 10 years, has a performance in the post-season that is many standard deviations from anyone else, and what should we expect from a regression analysis that does not isolate Rivera?  Well, duh, that post-season closer performance highly correlates with winning more than it does in the regular season?

And, if you DID include Rivera as a parameter in the regression, what should we find?  I would bet that the non-Rivera closers would have much less significance to the correlation once you do that.

I have the BBTN book, I read that chapter when it came out, but I have not re-read it.  Maybe I should.  But, this simply looks like a correlation/causation issue.

I say again, that conclusions drawn from sample data without “explanations” are next to worthless.

Pharmaceutical companies don’t just manufacture random drugs and then do double blind tests and if the tests come up positive (with statistically significant results) for their drugs they claim that they work!

They develop drugs for certain conditions and diseases based on scientific evidence and principles and lots of research and development that suggest that they might work. They also usually (not always) have an idea as to why they work. Then they do the tests.

Otherwise we would have lots of useless drugs on the market that just happened to have had positive results in these tests due to Type I errors.  (Maybe that happens sometimes, I don’t know.)

All I ask is that BP or someone explain to me (to us) why the “usual things” that make one team better and win more often in a regular season game (score more runs or allow fewer runs) are not the same in a post-season game?

I suspect that BP made a mistake in their research. Tango’s thing could be one of those mistakes.

Pharmaceutical companies don’t just manufacture random drugs and then do double blind tests and if the tests come up positive (with statistically significant results) for their drugs they claim that they work!

Not to distract, but this is not entirely the case—see Viagra as the most prominent example. There are instances where they’re testing for A and discover B, which they didn’t think of at all, and B ends up being the prescribed use.

More to the topic, it’s interesting to look just at this post-season and see how it worked out.

2008 Secret Sauce

Philly, at 12, beat Los Angeles (19), Milwuakee (12) and then Tampa (6). LA beat Chicago (3). Tampa (6) beat Boston (1) and the White Sox (5). Boston (1) beat the Angels (2).

Secret Sauce didn’t have a good time of it.

BP was not the only ones to come to this conclusion with a study.  The Hardball Times also studied this separately and with a different methodology and came to the same conclusion (don’t recall exactly nor time to search for).  Now whether their methodologies are up to snuff, that’s for others to day.

Here is why I think this makes sense, from observing baseball.  Ron Shandler’s Baseball Forecaster devised a new methodology, Pure Quality Starts or PQS, which is a saber-oriented way of determining a quality start.  http://www.baseballhq.com/free/free03.shtml?src=hqf

Looking at the data for pitchers, the best pitchers are able to dominate a large percentage of their starts (the best over 50%, the elite over 70%, roughly, from my estimation) and can have a large effect on the games in which they start in.  Generally, the best pitchers strike out a lot of batters and pitchers who get a lot of strike outs tend to have a better PQS rating, as two of the metrics are related to strikeouts. 

That also tends to be true of the best closers, that they strikeout a lot of batters.

And we know from DIPS that pitchers who can strike out more than other pitchers can control more of their games than other pitchers.

And obviously a good defense helps with all the BABIP when the pitcher isn’t controlling things.

I also observed how the D-backs with Johnson and Schilling were able to get through the playoffs, how having a superb 1-2 punch can make your team that much better in the playoffs.

And from other sports, I’ve seen so many football and basketball teams that were offensive juggernauts but fell short on the defensive end and never went far in the playoffs, assuming they even made the playoffs.  But strong defensive teams were able to get into the playoffs and do well still.

That is why their conclusions make sense to me.  Defense, via pitching and fielding, can keep the team within striking distance.  A good offense does not always do that, even good offenses can be shut down by the opposing pitcher. 

In addition, the exponential nature of Pythagorean means that the better the defense you have, the lesser offense you need as well in order to win.  But the offense has to be that much more better in order to gain the same winning percentage.

The better the group of pitchers you have, in terms of PQS, the better able they are to take (relative) control of a short series relative to a team with lesser pitchers, and strikeouts being a key contributor to PQS (in fact, it is essential for a good quality start, because a pitcher can only attain a PQS of 4 or 5 - that is, a dominating start - if he can either strikeout enough batters or strikeout at least twice the number of walks) you basically need a high strikeout pitching staff.

People always point to the hitters playing in most games and saying that they have an effect on more games than a pitchers does.  And he does.  But if you have a staff of dominant pitchers, then that team would have an equivalent (or better) to that lineup of hitters.  You would have a staff of pitchers who can dominate a large percentage of their team’s starts, and it won’t matter how good the other team’s offense is, relatively speaking.

Meanwhile, we have all seen our favorite hitters be rendered harmless in the short series of a playoff situation.  Barry Bonds never hit that well in any playoff series before 2002 and his clutchiness was put in question because of that.  And even when he was dominant in 2002, he ended up losing.

But pitchers who are the elite, the Randy Johnsons when he was at his peak, the Johan Santana’s, the Tim Lincecum’s of today, can dominate a very large percentage of their starts, 70%+, keeping their team close and giving their team a great chance to win almost every start.  The better your core starting staff, the better your chances in a short series. 

Hence why offense matters more in the regular season where the lesser starters take up 30-40% of the starts, but not as much in the playoffs, where the other teams have better pitching staffs (generally) and the best pitchers pitch about 75% of the games.

That is how I see defense - pitching and fielding - being more important than offense in the playoffs.

While all of that may be true, Tango is 100% correct.  The conclusions by BP and by THT are both obvious and nonsense.

Someone did point out that high K pitchers tend to be more “consistent” which may be marginally true and that if you are above average, which playoff pitchers tend to be, you benefit from being more consistent. Other than that, ditto everything that Tango said almost a year ago…

"I also observed how the D-backs with Johnson and Schilling were able to get through the playoffs, how having a superb 1-2 punch can make your team that much better in the playoffs.”

They had a great run.  But thinking it’s generalizable, check out the playoff run of the 2002 Diamondbacks.  Randy and Curt were every bit as great in 2002 (regular season) yet they got swept out of the first round.

I agree with some of what you guys are saying.  It’s definitely dumb to do a regression based on regular season stats to predict wins and try to conclude anything from it.  And I’ll also agree that there are not going to be any big differences between the types of teams that succeed in the regular season and the types of teams that succeed in the postseason.

But I do think we will see some small differences, and we can reach that conclusion from just reasoning it out and not doing any regressions.

obsessive has it exactly correct when he points out that the good pitchers are pitching a disproportionate number of innings in the playoffs.  It’s pretty straightforward to conclude that it’s more important to have aces in the playoffs and less important to have depth.

Further, I think that there are types of hitters that do disproportionately well against bad pitchers.  Since pitchers vary more in their ability to prevent walks/homers than in their ability to prevent singles, “take and rake” types are more sensitive to quality of pitcher. (I’ll note that while I feel confident saying this, we’ve had this discussion before and not everyone agreed with me).

Anyways, I don’t think these effects are big and BP is indeed being silly thinking that they can isolate them with the regression they did.

On the other hand, I bet MGL could test out the ideas here in his simulation and it would agree that there are some small differences in what it takes to win in regular season vs post season.

(Specifically, I’d say if he created Team A, a take-and-rake team with lots pitching depth and Tem B, a high BA low secondary team with aces and spaces in the staff, he’d find that Team A did worse in a post-season matchup relative to their regular season strength).

As Tango says in the intro, it is “no shizit” that in a 5 or 7 game post-season series, where there is a day off every 2 or 3 games, it is an advantage to be front-loaded with 2 or 3 great starters and your 5th starter matters not at all, and your 4th starter’s value is diminished and you can throw your closer until his arm falls off (plus he gets those days off too).  We don’t need BP or THT or regression to tell us that!

Any time you do a regression or other “indirect” analysis and you get a result where you have to say, “We don’t know why, but...” that should raise a red flag…

It’s less obvious that take and rake teams would do worse in the postseason, but I bet that your sim would agree that they do.  (Although the effect is small, probably less than than 1 win per 100 games).

What is a “take and rake” team?

I just mean a team that gets more walks and homers (and strikeouts) than average, I think “take and rake” is originally a BP term.

The effect is not big, but I do expect that walk/homer players would be more sensitive to pitcher quality than high-batting average players.  And there’s more good pitching in the playoffs than the regular season ...

I can only presume someone like Adam Dunn?

#20 I thought the post was written by you, MGL.  Your comment implies that it was written by TangoTiger. 

Either way, I’m confused.  You say it (meaning what I wrote) might be all true, but then you say Tango is 100% right.  Not if what I’m saying is true.

Then you say that the findings are obvious and nonsense.  I don’t see how it can be both.  At minimum, saying that offense doesn’t add much to the chances of a team in the playoffs is pretty non-obvious, though I assume that is the part where you think it is nonsense.  The obvious part I suppose is that pitching matters, but in my experience, any large scale study will come to some obvious conclusions that at least is confirmed plus some non-obvious conclusions.

#21 I knew it would be long so I didn’t want to get into this too much, but I was just using that as an example.  Another example I would point out is the combo of Koufax and Drysdale during the 60’s.  I never said it was infallible, I said that it would improve the chances of a team to win in the playoffs.

Where that edge comes from is my bringing up of PQS, where pitchers who are elite level are able to consistently (70-80% of their starts) keep their teams close and able to get a rally going to tie or take the lead.  There are not a lot of these types of pitchers in the majors.

Most good pitchers are in the 40-49% range in terms of dominating starts, the best in the 50-70% range, the elite 70%+.  That is roughly one and a half to two times the good pitchers can do.  It doesn’t guarantee a win each and every matchup there is, but that should improve a team’s chances in a short series.

Remember, the conclusion of the research is not these will guarantee success, but that their research found that these factors provided a measurable advantage to a team in the playoffs, whereas they found no measurable advantage to having a good offense or to having good homerun hitters.

#22 BP never said that there was big effects, only that there was statistically significant effects.

Clearly you haven’t read that chapter.  It would greatly help your understanding if you did.  The correlations found were small - but it was significant solely for defensive metrics they tested, related to pitching and fielding.

For all the metrics found to have an effect, they used regression to figure out which were the real drivers.  Those are the three discussed:  Closer WXRL, Pitching Strikeout Rate, FRAA.

Of course, they were regressing to a metric they named Playoff Success Points, which they assigned to each team based on the following rules:

3 points for making playoffs
3 points for winning LDS
4 points for winning LCS
4 points for winning the World Series
1 point for each postseason win
-1 point for each postseason loss

Now this is where I can see people having a problem, maybe more point should be assessed for each level reached (or less).

Their initial correlation was between the following stats and PSP:

W-L percentage
Runs scored/game
Runs allowed/game

0.22 correlation for the first and last, 0.00 for Runs scored.  They noted that one must expect low correlation “given the high degree of luck that the structure of postseason play introduces.  Still, we shouldn’t be too quick to discard small but positive statistical relationships, given all the possible variables that could be in play.”

Here are more excerpts/paraphrases that should illuminate BP’s position, since it seems that some have commented but never bothered to actually read this chapter.

What’s strange is that it isn’t hard to detect, even without the sophisticated math:

* Since 1972, there have been 27 teams that made the postseason in spite of having below-average offenses.  Of these, 7 won the World Series:  85 Royals, 87 Twins, 90 Reds, 95 Braves, 96 Yankees, 2000 Yankees, 2005 White Sox.  All but 87 Twins had excellent pitching staffs.

* Conversely, 20 teams have made the postseason with below-average run prevention.  None of them won the World Series, only 2 made it, and 16 of the 20 lost in the first playoff series.

“First, we don’t buy that the quality of an offense is of no importance in the playoffs… What has probably happened is that offense was swallowed up by the myriad other factors that come into play in the playoffs.

What does seem clear is the diminished importance of offense in the playoffs.  But they don’t believe that it is the extra days that causes this, as some have suggested.  They think a key factor is that there are no bad teams in the playoffs.

They tested this in regular season, finding “great pitching, average offense” with “great offense, average pitching” pairs, 28 since 1901.  Using the log-5 method for predicting wins, the great pitching teams were expected to go 230-238 but instead went 241-227.  Only 2-3 percentage points difference, but still an advantage. 

They posit that good pitchers have a structural advantage against good hitting teams.  And that advantage comes to the forefront when many/all the teams are good hitting in the playoffs.

That’s what I was pointing out with PQS and pitchers who can dominate 50% to 70% or more of the time they start a game.  That for the best pitchers it don’t matter as much when they face good hitting teams:  they still screw up maybe 20-30% of their starts, but compared to a good pitcher who can screw up 30-50% of their starts, that is an advantage that, while it won’t show up in each and every series, does show up over time, over multiple series.

"They posit that good pitchers have a structural advantage against good hitting teams.  And that advantage comes to the forefront when many/all the teams are good hitting in the playoffs.”

Yes, that would be an advantage, but I have NEVER seen any evidence that that is true (that good pitching has more of an advantage over good hitting than poor hitting).  IOW, if a good pitcher is 1 run per 9 better than average, I have NEVER seen any evidence that they will have more than a 1 run advantage against good hitting and/or less than a 1 run advantage against bad hitting, or that there is any difference than a basic log5 method would predict.  Never.  If there is, someone please show it to me.

Yes, I meant that there were some things that were nonsense and some things that were obvious and the basic thesis was nonsense as well.

The big problem is in the wording, “in post-season games.” What does that even mean?  Post-season games are like any other games, with teams that tend to be above average teams. It also means that there are more days off.  It also means that there are very few games left (so players have a chance to rest).  It also means a few other things.  But if someone is doing research on this kind of stuff, it makes NO sense to say we have found that X works in post-season games moreso (or less so) than in regular season games. I have no idea what that means.  And that is the problem with doing regressions and correlations with some “post-season success score.”

We want to know the why’s, what’s and how’s in order to make any sense of these correlations and regressions. Without them (the why’s, what’s and how’s) we CAN NOT just take the results of these regressions at face value.  There are sample size issues, there are inter-dependency issues, and other things like that.

For example, imagine that you didn’t know that post-season teams only used 3-4 starters - and that the 1-2 starters pitch 40 percent of the games. If I just told you that your number 1 and number 2 starters were extremely important to post-season teams and that the skill level of the #5 starter had zero correlation with post-season success, you would say, “WTF?!” If I then told you, “Oh yeah, I forgot to tell you that #5 starters don’t pitch, other than as long relief in the pen, maybe, in post-season games,” you would say, “Why didn’t you tell me that in the first place!”

That is one reason why just looking at some post-season success score (which suffers from sample size issues in and of itself) and correlating that with all of these things, in a vacuum, is problematic.

Basically, as Tango said quite eloquently last year in the intro to this thread, post-season games are just games.  Tell us what special things are involved in post-season games and it will become pretty obvious what things lead to post-season success.  And if you find some things that are not obvious, you better have an explanation, otherwise I don’t care what kind of correlation with what kind of p value you got - I’m not buying it.  For example, if one team has a choice between +10 runs on defense or +10 runs on pitching or offense, I am not buying that they are all the same in regular season games but not in post-season games. Not buying that at all…