THE BOOK--Playing The Percentages In Baseball

And by “luck” you mean luck and possible BABIP skill, correct?

If it were all luck, there would be no year to year correlation especially for pitchers who switch teams/home parks. There is however.  Which means that a “medium speed ground ball to position x/y on the field” by one pitcher is NOT the same as it is for another pitcher.  For example, I have found that pitchers will high GB percentages have easier to field GB than pitchers with high FB percentages.  IOW, if both of them had a “medium speed” ground ball, it is likely the GB pitcher’s GB speed was 48 mph and the FB pitcher’s GB speed was 51 mph, or something like that.  Of course, we can account for the GB/FB percentages of the pitcher in PZR, just like I do in UZR.  But even then, I think we will see some “skill” in the difference between actual and expected outs.

Matthew, don’t confuse BABIP skill with what Tango is talking about.  He is already considering the speed, location and other characteristics of the batted ball (and the game, like outs and baser runners).  So, it is not the type of BABIP skill that we normally talk about.  It is sort of a subset of that.

So PZR is really the gap from expected hits allowed considering all known factors (defense, park, hit location, velocity, etc.)and real hits allowed. How that gap happened varies from pitcher to pitcher - some may be luck and some other factors.

An idea I haven’t gotten to yet is after doing the fielding metrics with the pbp data, rerun for the pitchers (as Ben has done) but factor in which fielders are behind each pitcher when calculating the expected results to obtain a defense adjusted PZR. This should be closer to what we want in measuring the pitcher’s skill in hit prevention.

Another option is to use this batted ball derived expected BABIP as a mean to regress the observed BABIP towards.

I believe that Rally adjusts each pitcher’s runs allowed by the Total Zone rating of each fielder behind each pitcher.  It’s very good to do that.

You want to adjust pitchers for the regressed or projected defensive rating of the fielders behind them and not their actual rating for that year, otherwise you are just using a PZR for the pitchers.  Of course, you might want to do that anyway (assume a certain out rate, based on the batted ball characteristics and assuming average fielders), but if you do that, you lose any chance to preserve any BABIP skill that the pitcher might have.

Here is an example. Last year, I think that the Rays or the Mariners had a really high team UZR.  It would be completely wrong to use that to adjust pitcher stats (for pitchers on those teams) for last year. The reason is that if a team has a UZR of, say +100, for a year, there is almost no chance that that is what happened.  That is one reason why we regress small sample stats - because what the data say happened is not likely what actually happened when the data is an extreme value.  I explain that in the UZR primer on FG.  If a team is +100 for a year, it is more likely that their fielders played like a +30 or +40 team.  So adjusting pitchers by that +100 is really wrong.  I adjust my pitcher stats by the next year projections for the fielders.  So, to adjust a pitcher for 2009, I use the 2010 UZR fielding projections for their fielders.  It is basically the same thing as adjusting hitting and pitching stats by a long-term regressed park factor and not the park factor for that year, which as I explained in the UZR thread, is almost the same thing as just using a player’s road stats and ignoring his home stats, at least on the team level it is, which is obviously wrong.

MGL, understood.

I would adjust the pitcher’s expected batted ball results by a long term measure of each fielder, such as a three year weighted mean of each fielder’s performance, or a WOWY method.

But, as I said, not something I’ve actually sat down and done yet, just on the to-do list to play with for many months now.

I don’t know about that MGL.  Let’s say that the Mariners pitching+fielding was 100 runs better than league average.  And UZR says that the fielding was +100 runs.  So, don’t I have to show the pitchers as 0 runs?  How else can I make sure it adds up?

You are suggesting that the 100 runs that the UZR “shows” is actually 60 runs of true talent and 40 runs of luck.  Fine.  But the pitching+fielding (plus luck) was +100 runs.  If UZR (talent) is +60, that still leaves me with +40 runs.  Why would I necessarily give all +40 to the pitching, or even +20 to the pitching (and 20 to luck)?  Why not 0 to pitching?

Either way, whether you use +60 for fielding and +40 for luck, or +100 for fielding, I still end up with 0 for pitching.

Right?

Matthew,

From what I understand, Ben calculated how many hits you’d expected Cole Hamels to give up this year given his batted ball distribution (velocity, location, type, etc.) and an average defense.  Real hits - expected hits is the amount of help a pitcher received from his defense alone.

Nick -

assuming a large enough sample to give us a normal distribution of batters

Expected - observed = pitching skill + defense skill + luck

Also, with enough sample size, luck should diminish. Let’s find a way to separate pitching from defense.

Tango, I’d have to think about what you said, but I am pretty sure that I am right, if you are trying to estimate pitching talent.  As I said, the key is what happened, not what UZR “thinks” happened. We don’t really care what UZR thinks in terms of what to do with pitching.  As far as pitching is concerned, we simply want to know what the pitchers did if the defense were average - right?

If UZR tells me that a team was +100, our best estimate of what happened is that the defense played excellent defense to the tune of +30 or +40 runs.  So why would you want to use +100 to adjust the pitching when that +100 was just a mirage?  It was due to error in the data.  Remember that the “error” is some of these metrics is data error plus the difference between true talent and lucky performance.  Because there is so much data error in UZR, the first part of that equation is large. So forget the numbers (I don’t follow your example) and just focus on my logic. You want to adjust the pitching for what actually happened.  And +100 is NOT what actually happened. Much of that is because UZR thinks that lots of balls were harder than they really are (and some of it was great play by the team of course, which SHOULD go into the pitching adjustment).

So, now that I write this and follow my own logic, I am 99% certain that you don’t want to use the +100 runs to adjust the pitching (to take out the defense).

As I said, you can probably accomplish the same thing by just using a PZR where you don’t care at all about the actual result of each batted ball - you just use the league average result of each batted ball based on its bucket.  But, as I said, the drawback to that is that you lose park effects (so that you don’t want to park adjust those “PZR’d” stats, although some of the characteristics of the batted balls by influenced by park factors, like altitude and temperature) and you lose pitcher BABIP (PZR) skill.

As I said, I do the fielder projection thing for the pitcher adjustments and it sounds like Brian is on board for that as well.  There is no way I am going to use a team’s or the fielder’s actual UZR.  If we did that for the Mariners last year or TB the year before, their pitchers would look horrible I think (I think those two teams in 08 and 09 were 100+ runs in UZR - that is like .6 or .7 runs per game to adjust the pitchers!).

I’m not sure why we “want” the true talent pitching for PZR, but not true talent fielding for UZR.

I’m suggesting we split up the observed +100 pitching+fielding+luck into UZR (fielding+luck) and PZR (pitching+luck).

If UZR is +100, then PZR is 0.

UZR can be +60 fielding and +40 luck or whatever.

PZR could end up being +20 pitching and -20 luck, or -20 and +20 or 0 and 0.

Tango:  What you’re missing is that UZR is not skill + luck.  It’s skill + luck + measurement error.  When MGL says a +100 fielding team wasn’t really a +100 team, he’s not talking about skill—he’s saying the PERFORMANCE wasn’t really +100, i.e. the fielders faced an easier ball distribution than UZR estimates.  So in fact, the pitchers did a better job (whether by skill or luck) than UZR thinks they did.

I wonder if it would help to break things down between outs and expected outs.  Say you have two +100 teams:
Outs/Expected Outs (vs. avg)
A:  +75 / -25
B:  +25 /-75
Intuitively, I’m inclined to think A is actually the better fielding team, having made 50 more outs than team B (assuming equal # of BIP).  But maybe my intuition is incorrect, and the outs measure contains just as much error as the expected outs estimate....

I’m not sure why the measurement error would need to be regressed toward the mean of UZR 0.

That is, I know why sample observations are regressed toward the mean population.

I don’t know why the measurement of the sample needs to be regressed toward the mean of the population.  Why would the measurement error be biased (correlated to UZR), and simply not be random measurement error?

Indeed, why even start with UZR?  Why not start with PZR and then subtract out the “pitching skill” that UZR is capturing?  (i.e., when Mariano Rivera is pitching, EVERYONE’s UZR looks good, and it has to do with more than his batted ball distribution… just like if you did HZR, it would not capture Ichiro or Tony Gwynn or Wade Boggs well enough.)

"Why would the measurement error be biased (correlated to UZR), and simply not be random measurement error?”

It will be random with respect to skill, but highly correlated with observed UZR.  Same way pitchers with a low ERA were luckier than those with a high ERA, even though good pitchers are obviously not any luckier than anyone else.

MGL- when did you test your PZR correlations, and with what data set?  I’m hoping to repeat your study and play with it some more. 

I’ve also got a few other factors I’d like to account for but haven’t yet.  In theory these could reduce the correlations you saw.

Some old PZR and UZR data can be found here:
http://www.tangotiger.net/mgl/

UZR has gone through several iterations since then.

Right, the observes UZR is data error + defensive skill + actual defensive play over and above or below actual skill.

We want to adjust the pitching for everything but the data error.  A +100 team includes data error. How much, I don’t know.  When we regress, we regress to adjust for 1st and the 3rd thing above.  I don’t know the percentage of each.  Interestingly, the actual defensive play over and above skill will shrink to zero as the sample gets larger and the data error will shrink, but not to zero, since there are biases in it (not just random data error).  And when we say “data error” we don’t just mean actual “mistakes” in the data.  We mean in the data and the methodology.  For example, for two samples of data that have 20 balls in the same exact bucket, those balls will NOT be the same even though they are treated the same in the methodology.

Anyway, how to adjust pitchers to try and get to their skill is tricky.  We probably want to tease out the fielder skill AND the lucky or unlucky fielder performance (over and above or below their skill). That is MORE than just teasing out the fielder skill (which is the skill+fielding luck regressed) which is what I do now when I use each fielder’s projection. 

Of course if you start with the assumption that pitchers have little or no control over their BABIP. especially for small samples, this whole discussion about how to adjust pitcher’s stats for defense and luck is moot. You would just use a DIPS ERA or FIP anyway.

Now, the benefit of the way I do it, is that once I factor out the defense by using an estimate of the fielders’ true defensive ability, everything else is luck and pitcher skill and I can then do whatever I want.  IOW, whether you think that pitchers have no, a little or a lot of control over their BABIP, the least you want to do is to factor out the real talent of their defense and then go from there.  It is exactly the same thing as using a long-term regressed park factor (your estimate of the park’s true effect on all hitters combined) to park adjust a batter or a pitcher’s stats. That is the first thing you want to do with their stats.

Rats!  I already subscribe to four baseball websites but not ESPN’s subscription only content.  Looks like a good article from the brief excerpt.

You can get an ESPN sub for 4$ for a year.  Do a search of this blog for details.

Sorry to bring up such an old thread (the last time I did that, I confused some people)…

I was comparing rWAR totals from 2001-2006 with Alex K’s PZR WAR from 2001-2006.

I noticed that many of the pitchers who were docked the most in terms of defensive support according to PZR (Maddux, Glavine, Moyer, for example) all had much better career BABIP on GB than league average.  Several of the pitchers who look much better according to PZR have below average career BABIP on GB’s (Schilling, Johnson).  Mussina, who was around league average for his career with BABIP on GB was rated about the same by TZ and PZR.

All of these pitchers had much less variance in BABIP on FB, and only a few were outliers on BABIP on LD.

I know the sample size is small, but could this be a plausible reason for the differences in “defensive support” between the two systems?

Looking at dozens and dozens of long-career pitchers, there seems to meaningful differences in pitchers’ BABIP on each batted-ball type. What options do we have to not assume the same out rate for every pitcher for every ball type - especially for those rare GB pitchers who suppress BABIP anyway?

There are several reasons why pitchers have a skill component in PZR (outperforming or under-performing their expected outs).  One I mentioned above.  Ground ball pitchers tend to have easier to field GB and fly ball pitchers easier to field FB.  Although this is somewhat accounted for in UZR, if you use almost any other defensive metric (which does not account for pitcher G/F ratio) you will find that GB pitchers seem to outdo their expected outs on GB and vice versa for FB pitchers.  For pitchers with great control like Maddux, fielders are probably better positioned for all of their batted balls.  Other pitchers, even without great control, might tend to pitch to certain parts of the plate and thus it is easier for the fielders to position themselves.  There are probably more reasons, but those are two…

Thanks!

I wonder why people do not talk more about this potential issue with PZR, etc.  I mean over a career, stats that assume the same BABIP rate within each batted ball types would claim that Maddux should have allowed 300 more runs over his career than what he did. And that is just on GBs.

I guess guys like Maddux are rare enough that it isn’t worth turning over an otherwise pretty-reliable stat for.

I noticed that the y-t-y correlations on BABIP on GB and FB were all pretty low and LD rate is a little higher, but still not great.  Do we know where the .5 regression point is for each batted-ball type?  I know 3,700 or BIP is needed for BABIP overall.