THE BOOK--Playing The Percentages In Baseball

Is the average of an extra 1 pitch per start really the right way to look at this?  A team that is -30 runs per 150 games is not making mistakes each game that cost .2 runs, right?  More likely in any one game they are making mistakes that cost 2ish runs and in other games are making good plays that result in 1ish runs saved over average (the amounts are pure speculation), such that after 150 games, averaged out, they cost their team .2 runs/game over average.

If there is going to be an effect on starting pitchers (and I have no idea if there is or is not), isn’t it going to be experienced discreetly, rather than continuously?  Isn’t the right way to look at this to identify those games where a defense played poorly and played well and compare defense independent pitching statistics between the pool of pitchers in each group?

Vince Gennaro did an article for my Mariners Annual on the secondary effects of defense.  Because Maple Street owns the rights to the article, I won’t go through his whole process, but he actually finds that there is a legitimate effect.

Good defensive teams are able to allocate their innings more efficiently, limiting the work of the worst pitchers on the staff,

Overall, the swing between the best and worst defensive teams can be an additional couple of wins per year, based on innings allocation.  It’s not huge, but it’s there.

David, I don’t know what you mean.  I HIGHLY doubt any effect could be “another 2 wins a year” (that IS huge) but I am willing to listen.  Is there a reprint on the internet somewhere?

“Is the average of an extra 1 pitch per start really the right way to look at this?”

I believe so, yes.  The same bunching of good and bad defensive plays occurs for bad and good defensive teams.  The difference, as I computed, is 1 pitch per game.  So I don’t see any problem with my methodology.  I could be wrong though.

There isn’t a reprint on the internet, since it was written for the press product which MSP is hoping to make a profit on.  I’ll see if I can get them to let me send you a copy of the PDF for commentary purposes.

Dave,

I found the audio of the presentation from last year’s SABR convention, as well as the PowerPoint presentation.  Here is the URL:

http://convention.sabr.org/archive/sabr39/presentations/102-how-does-the-quality-of-a-teams-defense-impact-pitching-usage

I think I have a very good as to what he did.  He is not in disagreement with anything I have said here.  The difference is that he looked at teams with extreme defense, both good and bad, in any one year, which clearly includes lots of noise/luck and concluded that the difference in the number of IP per start (for the starting pitchers on those teams) was .5 IP per game or so, which is 81 innings per season. That is something like 8 or 9 pitches a game.  And that is the difference between the 1st and 4th quartile in team defense (according to Dewan’s numbers), not the difference between the average team and the best or worst defensive teams. IOW, the best defensive teams (4th quartile per year, I assume) averaged 6.0 IP per starter and the worst averaged 5.5, which is around 4 or 5 pitches per game for the best and worst defensive teams.  I came up with 1.5 per game, and I am sure that is because I was using a +-30 run team in true talent, and he was using a +- 80 run, in actual performance, team or whatever his 1st and 4th quartiles averaged over his sample.

So, sure if you are talking about 9 pitches a game, then sure, there very well could be a positive or negative effect for a good or bad defensive team.

But I easily show that if we are talking about a reasonably good or bad (not the all-time best or worst) defensive team, true-talent-wise and not a one year fluctuation, then we are only talking about 1 pitch a game. Even Gennaro would admit that the impact of 1 extra or fewer pitches a game will be negligible.

That is the ONLY point was I was making.  I was not arguing that significantly fewer or more pitches by the starter would not have an effect on a team’s RA - only that for a reasonably good or bad defensive team, that the impact in extra or fewer pitches is very small.

So Gennaro and I are talking apples and oranges and are not in disagreement in any way, shape or form that I can tell.

His 2 wins a year, BTW, that you mention, are based on the difference between the worst and best defenses he could find AND the team with the largest difference between their starter and bullpen talent, an unreasonable, extreme case.  In fact, for the team he used, the 08 D-Backs, they had a 3.37 FIP for their 4 primary relievers and a 5.01 for the rest of their pen.  So in coming up with a gain or loss of 2 wins (actually, he said a 3-4 win “swing” which is a gain or loss of 1.75 runs), he assumed the absolute worst or best defensive performance (not true3 talent) in 2009 AND he assumed that the back of the pen was 1.64 runs (per 9) worse than the top 4 (by IP) relievers.  So again, to illustrate his point, he used a ridiculous composition of situation made up of the most extreme good and bad defenses AND the most extreme difference in FIP he could find between the top 4 relievers and the rest of the bullpen.

So I stand by everything I said, which was simply that a plus or minus 30 run team, which I consider a typical good or bad defensive team adds or subtracts around ONE pitch a game to their starting pitcher.

I’ll leave it to the reader to determine how much of a runs allowed secondary impact that has on a team over and above the primary impact of a plus or minus 30 run team (.2 runs a game).

Anyway, all someone has to do is show us, on average, how much a starter inning “costs” per pitch, and then we can go from there.  It might not be linear though.  One pitch per game might cost nothing while 10 pitches might cost a lot.  I don’t know.

One assumption that the general public makes is that a bullpen inning is worse than a starter inning.  That is not true, everything else being equal, since bullpen RA is around the same as starter RA, maybe even less.  Of course, marginal bullpen IP are probably worse than starter IP since marginal pen IP probably does not include your good, short relievers, and probably shifts innings from your decent middle and long relievers to your worst ones, although a good use of leverage can mitigate this (use your bad relievers when the leverage is low in those extra IP that bad defense creates).

But, let’s assume that all marginal IP taken from a starter cost .25 runs (per 9) in IP, and I think that is generous.  My 1 extra pitch per game is around .06 IP per game or 9 IP per year.  That is 2+ runs a year, nothing to write home about.  That is assuming a plus or minus 30 runs team, so that is an extra 7% in secondary effect from defense.

I’ll grant that.  7% more for defensive value from a secondary effect on pitching, on the upper end. How’s that?

Woah, I really screwed up that last part.  9 extra IP per year from the pen for a -30 run defensive team is an extra .25 runs, not 2+ runs.  So we are talking about a .8% premium on defense caused by this secondary effect.  Can we put this discussion about whether defense is not properly accounted for to rest?

"The same bunching of good and bad defensive plays occurs for bad and good defensive teams.”

Well, presumably not the same bunching.  A good defensive team will have more good and fewer bad defensive games than will a bad defensive team.  In any event, it seems to me that the question should not be the overall defensive ability of the team, but should instead be the defense present on a particular day (luck included).

My understanding of what is being suggested (and maybe I have that wrong) is that bad defense (and presumably good defense) effects pitching beyond merely shortening (or lengthening) the innings and lowering (or raising) the run total.  If this is so, I think we would be able to see a FIP/UZR negative relationship (on the game, not season, level).  Maybe I am completely missing what the Neyer is suggesting, however.

What Neyer and other people are suggesting is simply that with a bad defensive team, the number of pitches thrown per game is increased (and vice versa for good defensive team of course) and that that causes even more runs to be allowed (over and above the extra runners put in base and fewer outs recorded) because the bullpen has to go deeper, thus resulting in an overall worse quality of pitching per game.

I don’t dispute that.  I am merely showing the number of extra pitches a bad defensive team allows per game is on the order of 1-2, and therefore the effect of having to go deeper into the bullpen is minimal.

I generally like the work I’ve seen from Gennaro, but that SABR analysis is really misleading.  It’s like a textbook example of how to take a small effect and create the illusion of a large one.  A few of the magnifiers (I may have missed some):

*Compare top defensive teams to the bottom, rather than average.

*Use unregressed top and bottom plus/minus outcomes to measure the difference between teams.  This hugely overstates the real defensive difference.  Looking at 2007 (which I had handy), plus/minus shows a difference of about 130 runs between top and bottom quartile.  That’s at least twice the true talent difference, and probably more.

*Use season outcomes to measure the difference between top 4 relievers and back of the bullpen.  The true talent gap between these two groups will of course be much smaller.  (And Gennaro makes it worse by focusing on DBacks’ extreme disparity.)

*Ignore leverage, which makes the use of back-end relievers much less damaging than it appears.  These guys pitch more on bad fielding teams, to be sure, but that’s because these teams are BEHIND more often.  The cost of bringing in weak relievers then is smaller.  Sure, you would win more with good fielding because you wouldn’t fall behind as much in the first place, but that’s already captured in the traditional fielding
metrics’ run saved number.

* Ignore the fact that a starter is much less effective by the fifth/sixth inning.  So knocking him out a bit earlier has a much lower cost than it appears, and may often even be a good move for the team (teams are often too slow to remove starters).

Stir, put in the over and let rise, and you’ve got 2 wins for your team.  But in reality, there’s not much there there.

MGL,

I think you might like the Mariners annual from Maple Street Press.  It’s got a pretty good article from Dave Allen using PITCH f/x.  It’s Mariner-centric obviously.  There’s a good long interview with the manager.  There’s some good Inside Edge charts (batted ball stuff, performance by strike zone location, etc).

I REALLY wish that all books would include sample pages like we do for our book and what Amazon’s Look Inside does.  I think you can easily double sales by doing that.  Right now, a reader is blind as to whether to buy the book.

I think the real issue is whether or not pitch counts mean anything within each inning. Or, does pitch quality deteriorate as the number of pitches increase? For instance, can it be determined that after 15 or 20 pitches without rest a pitcher loses some velocity or movement? If not in the early innings, at what point in the game might this become a factor?

Great discussion ... Mickey, you probably know that I wasn’t suggesting that an effect does exist, just wondering if it might. And I suspected that if it did, it would be more relevant within the inning as opposed to later in the game. While granting (in my head, anyway) that any effects would be small and hard to find.

One question for you ... What’s the rationale for sticking to your per-150 standard when talking about team fielding? I understand why you do it for individual players, but (almost) every team plays exactly 162 games. Why not use 162 for team defense?

"Mickey, you probably know that I wasn’t suggesting that an effect does exist, just wondering if it might.”

Sure. 

You know that I just kind of have a six sense for things that are true or not, at least in baseball.

“What’s the rationale for sticking to your per-150 standard when talking about team fielding?”

None really.  I rarely talk about team defense, so I just end up using the same denominator as with individual players.  Even with individual players, there is no compelling reason to use 150 games.  I just started doing that 15 or 20 years ago because it is a “nice round number” and most reasonably healthy players probably end up with around 150 games a year.

Guy, ditto.  I didn’t want to say anything, but I agree that it was a very misleading presentation by Gennaro.  While his methodology was reasonably sound, he should have told the audience what the practical implication was.  He ended up having everyone thinking that the secondary effect of good or bad fielding was “worth up to 2 wins a year.” He even had David fooled as you can see from #2 above.

Just to be clear, I’m not accusing Gennaro of being dishonest (and I’m sure MGL isn’t either).  We’ve all gotten excited about a new idea/insight, gone in search of supporting evidence, and found ourselves unconsciously grabbing on to the best case evidence (while not searching nearly as energetically for factors that might undercut the theory).  This just ended up being a pretty extreme version of that, though I’m sure done in good faith.  This is why having pre-release reviewers is always a good idea....

I had some similar thoughts when editing the piece, but still thought it was worth publishing.  This is an issue that hasn’t been talked about much, and I think he’s right to shed light on it, even if the conclusion may be a bit overstated.

I’m not sure applying a magnification of around 10x counts as “a bit overstated.” But sure, it’s an issue worth exploring. 

One thing this brought to my attention is the much larger variance in team defense in plus/minus, as compared to UZR.  Dewan has a gap of about 130 runs between the top and bottom quartiles in 2007, while UZR is more like 85 runs.  Is that generally the case?  Has anyone explored why plus/minus estimates so much more variance, or compared the y-t-r correlation of the metrics?

I think MGL’s new UZR are tighter in range than the old UZR.  Never proved it though.

Plus/minus should have a wider range because it gives larger credits/penalties for each play.  When a play is made by a fielder, UZR gives the fielder credit for the probability that no one makes the out.  Plus/minus removes plays made by other positions from the denominator in figuring the value of a play made by a fielder.  John Dewan gave this example which was discussed in a thread last year:

* Plus/Minus is a little more aggressive in awarding credit/penalty. An example: 100 balls in a ‘bucket’ (specified type, velocity, location), 30 fielded by the 2B, 20 by the 1B, 50 go through for singles. On a groundout to the second baseman, we give +50/(50+30) = 5/8 = +.625. UZR gives +50/100 = +.50. On a single through both fielders, Plus/Minus gives -30/80 = -.375 to the 2B, and -20/70 = -.29 to the 1B. UZR gives -30/100 = -.3 to the 2B, and -20/100 = -.2 to the 1B.

There was another thread last year on a fielding powerpoint presentation that said the same thing and showed the difference as

plus/minus = UZR/(probability no out is made + probability an out is made by a fielder at that position)

or

plus/minus = UZR/(1 - probability an out is made by a fielder at another position)

Where those represent the difference in how UZR and plus/minus assign credit and debit for a given bucket.  The magnitude of plus/minus will always be equal to or larger than UZR for a given bucket since you would divide UZR by a number between 0 and 1 to get plus/minus.

Thanks, Kincaid, very informative.  I realize now that I overstated the gap between UZR and P/M, because I forgot P/M is denominated in plays rather than outs.  There appears to be some gap, but not huge, at the team level.

But Plus/Minus’ system for allocating credit will, I think, significantly exaggerate variance at the player level.  In the example you cite above, a ball fielded by the 1B gets credit for +.71 plays, even though we expect 50% of the balls in that bucket to become outs.  The only logic I can see for this is that Dewan figures that if the 1B makes the play, it probably wasn’t a play the 2B could actually have made, and so in fact the likelihood of an out was truly just .29.  There’s probably some truth to that, but it’s ALSO true that any ball which is converted to an out is—on average—easier than the average ball in that bucket.  The fact that the 1B made the play probably means the actual probability of an out was .39, or maybe .49—we don’t know.  If we don’t account for that, we will give the fielders too much credit.

To offset this problem, P/M must then also assign too much blame for non-outs.  In this example, it debits the two players -.67 on a ball which only had a 50% chance of being caught.  It essentially assumes for both fielders that the ball was really in “his” part of the bucket—which obviously can’t be true.

The perverse effect of this is that within any given bucket, Dewan’s system assumes that balls which become outs actually had a lower out probability (than the bucket overall), while balls that become hits had a higher out probability—exactly the opposite of what must be true!  In reality, a 50% bucket must include a distribution of actual probabilities—some 40% and some 60% BIP—and the ones that become outs will disproportionately be at the high end.

MGL, I know you have thought about this issue for UZR.  Do you take steps to adjust for it?

The above discussions on how UZR and Plus/Minus apportion responsibilty to plays in zones where 2 or more fielding positions routinely field balls seem to be assuming that their two options are the only ones available.  When I designed my BZM fielding metric I tried to correct what I thought was an irrational feature of both UZR and Plus/Minus; that a ball fielded in front of an infielder should be treated in the same manner as a ball fielded by another infielder behind and infielder. 

It is not rational to consider a ball hit in the SS zone (or any shared zone in UZR or Plus/Mius) an opportunity for the SS if it is fielded by the 3B (for an out or not) before it reaches the SS.  So I treat those balls as Dewan does in Plus/Minus, by subtracting them from the total opportunities for the SS.  However, for balls fielded by the SS BEHIND the 3B it makes perfect sense to treat those as an opportunity for the 3B and so they are included in the calculation of his percentage of balls fielded for outs.  These balls are treated similarly to UZR, but not exactly the same.  The SS gets additional credit for a play made out of his zone, the 3B is debited a partial amount (the value of an out) for failure to make the play, but is not debited the value of a single because the SS ultimately prevented the single. 

MGL doesn’t do this in UZR because he has publicly stated that he doesn’t believe that players taking plays away from other fielders is a major issue in his system.  For BZM there was a large variabilty in the amount of balls that were fielded by pitchers, catchers, 3B and 1B in front of their respective fielders and this seemed the most equitable solution.

Peter, the first of those allocations makes complete sense to me.  But I don’t see why you want to debit the 3B on the second play, even partially.  When such a ball is successfully fielded by the SS, it seems highly likely that one of the following occurred:  a) the 3B saw that the SS had a better play on the ball and let him have it, which by definition was the correct call (since an out was made), b) both the SS and 3B were shifted toward the LF line, presumably for a good reason (or at least managerial decision), or c) the ball was hit hard enough that the 3B really had no meaningful chance at the ball (a slow GB to this zone won’t become an out very often).  Do you think a lot of these plays are really ones on which the 3B should have made the play?  And what difference would that have made to the team?

*

Also, do you have any thoughts about the issue I raised, that balls which become outs are in fact higher-probability outs than the “bucket” as a whole?

Guy - BZM only has one big zone for each fielder.  It is dependent on the assumption that the distribution of balls hit in that zone are close to being similar for all fielders at that position.  This obviously isn’t true for small sample sizes and probably isn’t true for a single year, but I think it may be very close to true for players with 3 full years of play at a position.  But I think the lack of accuracy of subjectively determined hit location makes such an assumption necessary.  Given that assumption, I don’t think I have any choice but to include all balls in his zone not fielded by somebody in front of him as opportunities.  Once I do that, I have no choice but to calculate his percentage of plays made as the number of outs he personally contributed, divided by the opportunities.  Inherent in the assumption is that the other third baseman, with a higher percentage of plays made, had balls hit at similar speeds in identical locations, but had the range to make the play that this 3B did not.  Its not perfect, and I may choose to do things differently once we have accurate hit ball speeds and locations from Field f/x, but with the data at my disposal I think it works best.

As for the issue that you raised, I think the circumstances are similar, but on a reduced scale because of the smaller zones.  The reason UZR uses smaller zones than BZM’s one large zone for each fielder is to specifically address the problem of some fielders having hit ball distributions that are easier or more difficult than an average fielder at that position.  But the subjective locations are still a problem, and trying to make the zones even smaller (as in Plus/Minus) creates a different problem of small sample size for each zone.

”...balls which become outs are in fact higher-probability outs than the ‘bucket’ as a whole?”

Absolutely!  That is one reason we do regression to reflect what actually happened and not just “true talent”!

I have been talking about this for years.  Un-regressed numbers don’t tell us what actually happened, at least in the context of these defensive metrics (for offense, they tell us what happened, in terms of it actually was a single or a double, but even then, they don’t tell us if it was a cheap single or a hard hit one).