THE BOOK--Playing The Percentages In Baseball

Tango - Actually I think there were several things that I did in my study that differentiated it from your study and what studes does in the batted ball reports.  First, I kept everything in terms of runs above average.  Rather than use an FIP like estimator of ERA based of the quantity of HR,K,BB I used the actual run values of those events.  Second, I adjusted the in play hit ball run values for the pitcher’s team defense to give a defense independent estimate of a pitcher’s hit ball run value.  Again I used the run values of those events rather than the usual BABIP.  Third, and the most important for me, was to differentiate between how one constructs a descriptive metric using different information and methodology than when one is trying to construct a predictive metric.  This is an important distinction that has often been lost in these discussions.

I like DIRVA+ a lot. The only criticism I have is that by subtracting the pitcher’s run value on BIP from the expected run value of his BIP is that it assumes that a teams defense will remain constant for each pitcher.  I don’t think that is an accurate assumption and it could mean that DIRVA+ will be giving too much credit for pitchers with extreme values on BIP.  So, if you assume I’m right about that (and I might be mis-reading your article), than it might not be taking enough information out of ERA.  The question is whether or not it strikes a better balance overall than FIP or tRA.  I have a feeling it does - but maybe not for extreme players.

For example, Jair Jurrjens last year had -25.8 PITCHER_HIT_BALL_RUNS, meaning he allowed that number of runs less than a league average pitcher would on balls in play.  The Braves as a team were below average defensively, so he actually gets awarded for having a super low DIRVA on BIP.  However, in all likelihood, the Braves played exceptional defense when Jair was on the mound for whatever reason (random variation probably) but DIRVA assumes that his defense actually hurt him. 

FIP says he was -17.2 runs last year.  ERA says he was -43 and DIRVA says he was -43.  I think he was probably a lot closer to his FIP than that, and his defense played really well behind him.  The opposite is true with Verlander.  The Tigers were an excellent defensive team this year; however, he was +30 runs on BIP, so in all likelihood, they played much worse when he was on the mound - yet he gets penalized for having a great defense.  ERA has him at -22 runs last year, FIP has him at -38 and DIRVA+ has him at -18. 

So I think that the team adjustment for DIRVA+ isn’t going to be strong enough as any player than has an extreme hit ball runs likely received extreme defensive support.  The way to fix that problem is to customize each pitcher’s defensive support, using BZM or something.  But at that point, how is it any different than UZR adjusted ERA? 

It’s late now and I’m tired so I might be misreading your article, so don’t flame on me if I missed something obvious.

Nick - The only thing that you may have missed is the paragraph where I acknowledge that the defensive adjustment might not be equitable.

A more problematic concern is that a particular pitcher’s hit-ball location distribution may vary from the staff average, and either have more balls hit to the better defenders or the poorer defenders. This problem is correctable, but only with accurate hit-ball location data and much programming. I opted for the less accurate but simpler method that could be used for datasets without any hit ball location data.

I think that the pitcher’s hit ball distribution is a more likely cause than the situation that you describe of a team’s fielders playing particularly well or badly behind him.  Of course there is going to be some variation in how well the fielder’s play just because of sample size, but the sample of hit balls in a year is pretty large for a full time starter.  Much larger than the DIPS events PAs and larger even than a batter’s PAs for a year.  And this is really the crux of the problem isn’t it?  To be able to separate the variation in a pitcher’s fielding support that is caused by sample size from the legitimate variation caused by the pitcher allowing more easily fieldable balls?  You still have that problem even if you go to PZR like system, and you will continue to have that problem until you have Hit F/X data and you know how hard each pitcher’s hit balls are hit. 

The next step with DIRVA+, that I hinted at in the article, is to look at how persistent a pitcher’s hit ball run saving ability is with larger sample sizes.  If preventing hit ball runs is a real skill that certain pitchers possess, then it will remain in the larger samples. If it is just an artifact due to sample size, then it should disappear with big samples.  For some pitchers it remains a significant portion of their overall run value (more than 25%) after more than 5000 BIP.

Thanks for taking the time to read and thoughtfully comment on the article.

Peter’s point is an important one that I think has been missed in many (most?) DIPS discussions.  If a pitcher has a very low BABIP in a season, the potential causes are:
1) pitcher skill
2) fielder performance
3) random distribution of BIP (separate from pitcher’s inate skill).
Many discussions focus only on #1 and #2, and conclude that fielder performance is the main or only determination.  The name DIPS is somewhat unfortunate in this regard, because many people assume that the opposite of “defense independent” must be “defense dependent.”

In fact, though, factor #3—random variation in difficulty of the BIP—is by far the largest determination of BABIP.  A low BABIP doesn’t (usually) mean a huge number of web gems, it just means some grounders are hit in the hole instead of to the SS, and some LDs are hit to the alley instead of to an OF.  It is entirely defense independent, but it also isn’t a skill.

So, if we’re trying to answer the question “how well did this pitcher actually perform,” we have to decide how much to credit him with that BIP distribution.  DIPS and similar metrics say little or none, because we know the pitcher can’t replicate the performance.  On the other hand, he really did induce those BIP—and who else should get the credit/blame for that?  I just don’t see the case for stripping this out in a performance metric.  Or if we decide we have to do that, then we should do the same for hitters.  And there’s actually quite a bit of luck in BA as well.  But I wouldn’t support that either.  These are real performances.

"These are real performances” .... by the team, with that pitcher on the mound.  In no way is it a performance by that pitcher, as if it’s on his own.

But if we remove (2) fielder performance, it *is* a real performance, and by the pitcher. Where (or how hard or whatever it is) his balls in play go may be luck, but it’s *his* luck.

Tango:
Let’s say a pitcher gives up a BIP distribution that translates to a .270 BABIP, and that’s what he has (fielders provide average fielding on those balls).  Are you arguing that the luck involved in that BIP distribution belongs to the team rather than the pitcher?  I suppose you could make that case, but it seems odd not to give credit/blame to the only defensive player (the pitcher) who touched the ball. 

And if we follow LIPS to it’s logical conclusion, there is in fact no difference between “performance” and “skill.” We should just do the best projection we can and say that was a player’s “actual performance” this year, since any departure from that must be either luck or the influence of teammates.  Does that really make any sense? 

And then we need to create LIHS for hitters, since there’s also a lot of luck (though less) in a hitter’s performance.  Opposing pitchers just happen to pitch him a little better/worse, FBs drop in, LDs hit right at a fielder, etc.  Again, we can just do a projection, and say that’s what a hitter “really” contributed to his team.

One advantage:  our projections will all be perfect!

Guy, the batted ball distribution is his.

The conversion of those plays into outs or hits belongs to both the pitcher and fielder.

More specifically, it’s pitcher + pitcherluck + fielder + fielderluck.  If his batted ball distribution would normally yield a .270 BABIP and the actual BABIP is .270, that still doesn’t mean that he’s average.  Even if you account for the fielder performance, you still have all the luck associated in there.

The level of luck is proportional to the number of balls in play.  And, with 500 BIP, the luck portion on the pitcher himself is 1 SD = .020.  Given that almost all pitchers have a true talent level of BABIP that is within .020 of the league average (meaning a true talent level is probably 1 SD = .080 or .010), then that’s a ton of luck at play here.

If you agree to credit the pitcher with a .270 BABIP, given a .270 BIP distribution, then I think we agree.  I’m not sure I follow the rest.  And if we take BIP distribution as a given, what counts as “fielder luck?”