THE BOOK--Playing The Percentages In Baseball

If the bias has something to do with the Mariners, wouldn’t you look at the Mariners’ hit distribution versus their opponents’ hit distribution in the same games? It seems like a stringer who favored the Mariners would be more likely to judge a ball hit by a Mariner as a line drive and a ball hit by an opponent as something else. And obviously, Ichiro the fielder is unaffected by the Mariners’ hit distribution.

A stringer doesn’t have to favor the Mariners.  He simply has to favor balls to be marked as line drives.

What you want to do is look at how each hitter’s FB/LD rates are in Seattle compared to elsewhere, and see how much more LD they are being credited in Seattle.

You can also, as you suggest, see if there’s an extra home bias, over and above the typical home bias (both for pitchers and hitters).

Regardless, the evidence certainly points to Ichiro being biased against, by at least one group of scorers (wherever Retrosheet gets their data).  There is a bias somewhere between STATS and BIS as well, as shown by his bUZR and sUZR numbers.

The only thing I do know for sure is that there is no player that Fans love more for his fielding than Ichiro.  From 2003-present, he has been #1 or #2 every year, meaning that, overall, for that time period, he is a clear #1.

I knew there was something wrong with those numbers.

Tango - It should be easy enough for MGL to test whether the classification of FBs and LDs by BIS and STATS for Ichiro is creating the differences in UZR for him.  What Harry did shows a general stringer bias for either LDs or FBs, but what you are suggesting is a particular bias for Ichiro due to his style of play.  Have you checked whether the same WOWY bias exists for other Seattle outfielders?  Is the bias more evident in home games?

Peter, I didn’t track the home park or even home team in my compilations (but I do track by year), so I’d have to go back and redo those.

But, if I just pick and choose a few guys, let’s see what it gives me.  Mike Cameron was in Sea from 2000-2003.  He was a monster in those years, where WOWY gave him at least +40 plays above average in each of the three factual categories (batter, pitcher, park).  For BIP, he averaged +16.

In his non-Sea years, his WOWY on the three factual categories was similar to his BIP WOWY.

Let’s see… Randy Winn?  In Sea in 03, 04, and most of 05.  For all of those 3 years, he was at least +32 in each of the three categories, and +12 in the BIP category.

I’m not cherry picking.  Just whoever comes into my head.  Let’s see… Jeremy Reed?  In Sea 2004-2008: all 4 categories, he was pretty much average across the board.

I wonder if there is some cognitive bias in the scoring that causes the same ball to be judged as a liner if it falls for a hit and a fly ball if it is caught. I would be surprised if that weren’t true to some extent.

If such an effect were uniform, it would just regress fielding numbers toward the mean, but if some scorers exhibited it more than others it could be an additional source of error in the resulting fielding metrics.

I have no doubt that it’s true, that take two balls with the same HITf/x, that a scorer will more likely mark one a liner if it fell in for a hit and mark it a FB if it was caught.

If Ichiro is catching a bunch of “flyballs”, it’s possible a disproportionate number of those are infact line drives, and he deserves more credit for it.

And indeed, BIS DOES give Ichiro more credit for the balls he catches than STATS does (via UZR).

Andruw Jones is another huge source of difference, and Andruw played a very shallow CF.  I have no doubt that his positioning plays a role into whether a ball is marked as a liner or a FB.

It’s for this reason that I am VERY dubious of batted ball classification.  Not to mention that since I prefer going to multi-year data anyway, where things like batted ball distributions should even out, then I don’t see much reason to look at anything other than factual evidence at this point.

I mean, I’d probably only go so far as seeing if a ball is on the ground or in the air (even though that’s a source of contention as well, since a line drive to the 3B feet could be a GB if one-hopped or LD if caught on the fly).

Tango - I checked the percent of LD + FB that were recorded as LD by Retrosheet for Ichiro 2005-2008.  2005 he was 16th ( from the lowest percent) of 31 RF that qualified for my fielding metric with 44.2% LD.  The range was 36.2% to 51.7%.  2006 he was 11 of 31 with 42.6%.  Range was 37.8 to 48.8.  2007 he didn’t qualify as he played mostly in center.  2008 he was 22 of 36 with 45.6%.  Range was 34.7 to 53.3%.  Total for the 4 years Ichiro was 21 of 41 at 44.1%.  Can’t get much more middle of the road than that.  My BZM fielding metric also has Ichiro as a so-so RF.  But if the problem is related to misclassifying air balls it would seem to lie in parks other than Seattle.  Hunter Pence (34.7% LD), Jacque Jones (38.7%), Michael Cuddyear (40.5%) and Franklin Gutierrez (40.8%) were at the low end.  Ken Griffey (47.8%), Magglio Ordonez (47.9%), Bobby Abreu ( 48.4%), and Alex Rios (48.6%) were at the high end.

Peter - Is that F+L or F+L+P? (For those who don’t have a Retrosheet database - are popups included?)

Good stuff Peter.  Ok, let me check specifically 2005-2008 (since that’s the one breakdown I do have handy), and only for RF (I guess I have that breakdown too!).

This one is not egregious.  Here are his total WOWY by category:
Pitcher: +79
Park: +57
Batter: +42
BIP: +20

Note that for “pitcher”, I control for the identity of the pitcher and handedness of the batter.  For batter, I do similarly.  For park, I control for the handedness of the batter and pitcher, as well as the park.  For BIP, I control for the handedness of the batter/pitcher.

If I have to pick out a year that is the most out of whack, I’ll choose 2001.  In that year, the league average RF converted 7.7% of all balls in play into outs.  Of the batters that Ichiro faced, those guys have a career 7.7% outs on balls in play by all RF.  Of his pitchers, theire career out rates by RF (other than Ichiro) was 7.6%.  And in the parks played by Ichiro (but when Ichiro is not there), the RF converts 7.6% into outs.

So, in 2001, we can say that Ichiro faced very normal conditions.  However, his batted ball distribution was such that we’d have expected 8.9% of the balls in play to be turned into outs.  That is, the batted ball distributions noted shows a disproportionate number of them going to RF and/or “easily” catchable.

Ichiro, in his rookie year, making his mark that turned on the nation, converted, as it happens, 8.9% of all balls in play into outs.

Now, what is more likely: that Ichiro was a true fielding star, who got to far more balls than an average RF?  Or, that he converted as many balls into outs as expected?

You have a similar situation in 2003-2004, and almost as bad in 2002, 2005.  So, the big gaps are in 2001-2005.

In 2006 and 2008, the batted ball distribution looks fine.

So Peter, if you want to focus on one year, and if we are limited to 2005-08, then I’d say if you can report back on 2005, that’d be great.  I know you showed the FB/LD breakdown, but can you show all the BIP breakdown (including GB and pops).

Why are we reporting on the retrosheet classifications?  They are independent of STATS and BIS and I don’t know of anyone who uses them for an advanced fielding metric (maybe there is).

Are we trying to see if ALL stringers mis-classify balls hit to Ichiro because of the way he plays?

Let me also say this:  We have to be REAL careful about looking at the proportion of LD/FB hit near a particular fielder, and compare that to the distributions hit to other fielders, even in the same parks and with the same pitchers on the mound, and then concluding that there must be a stringer bias and rejecting a metric like UZR, plus/minus, or Pinto’s.

The WHOLE POINT of these metrics that use batted ball data is to credit or debit fielders based on what kind of balls they receive. If we start out assuming that all fielders receive the same distribution of batted balls, then what is the point of the advanced metrics in the first place?

Tango, if we find that the batted balls that were hit near Ichiro are not hit to the typical zones, are we going to conclude stringer error and/or bias too?

Certainly with small samples we WANT to give a fielder credit or not for batted balls that are not typical - that is the whole point of these metrics.  So, at what point (at what sample size) are you going to say that if a fielder does not have an average distribution of batted balls, both location-wise, type, and even speed, given the park, pitchers, and batters, that there is something wrong with the stringers?

I find this whole discussion disturbing for those reasons.

That being said, I know the answer.  It is that at any point in time, be it a small sample or a large sample, when a fielder does not seem to have the distribution of batted balls that he “should” have, the truth lies somewhere in between.  And guess what?  That is one reason why we regress sample fielding metrics! And the WOWY data and results can help with those regressions.

For a situation like Ichiro’s, yes, the stringers are likely making mistakes and/or are biased (the measurement error I talk about until I am blue in my face), yet at the same time it is also likely that Ichiro truly did not receive a typical batted ball distribution, even for 5 years or his career.

I think you are giving too much credit to WOWY by assuming that it is the gospel and that any difference between it and a fielding metric like plus/minus or UZR is because of a data recording error with the latter.  No, no and no!  Again, the likely truth is somewhere in between!  Which means that WOWY will err when the distribution of batted balls between the subject fielder and “all the rest,” even after controlling for pitcher, batter, and park, are NOT the same. And that will be always, BTW.  Might be close, but it will never be the same. Never.  They will likely be close the larger the sample, but there is some chance that they will be a little to a lot different regardless of the sample size.

And guess how we can tell when there is a greater than “average” chance that WOWY screwed up - i.e., the batted ball distributions between the subject fielder and “all the rest” were NOT very close?  By looking at the fielding metrics which take those distributions into account!

So basically, rather than treating your WOWY as the gospel, which it isn’t, which can identify errors in the other metrics, you can turn that around and say the same thing about the other metrics - that they can identify when WOWY thinks that the distributions between the “withs” and the “withouts” are very close to one another, but they are not!  Basically, the truth lies somewhere in between.

It may be true that WOWY starts to be the better metric after a certain sample size (do you know when that is?), but UZR and the other PBP ones will ALWAYS add something to WOWY. And of course the Fans Scouting reports and other “observational” data can add something to both of them and can “resolve” to some extent a discrepancy between UZR and WOWY.

Tango - I can’t show popups and GBs very easily as I didn’t collect that information for my fielding metric.

MGL - I think it’s because the only play-by-play data that is available to the rest of us is Retrosheet and the Gameday stringers.

Now, it’s unlikely that BIS and Retro have the same park stringer biases for batted ball types (and the way BIS does it it’s very possible they don’t have a park bias at all - maybe BenJ can shed some light on this for us). But it’s the only way we can study the issue.

Colin, Tango is suggesting that there is an issue, at least partly because UZR using STATS and BIS has never liked Ichiro very much.  So he is suggesting that those companies have a stringer bias.

Is there any suggestion anywhere that the retrosheet stringers have an Ichiro bias?  That is what I am asking.  And if no, why check Ichiro’s batted balls?  What about all the other players?

As I also asked, is there some reason to think that Ichiro “encourages” stringer bias in general?  Do all outfielders who play shallow (does Ichiro even play shallow?) create stringer bias?  That might be worth investigating…

...yet at the same time it is also likely that Ichiro truly did not receive a typical batted ball distribution

Ichiro was on the field for 33,000 batted balls.  His Retro-classified batted ball distribution shows a 150 out difference over that time span.

For BIS/STATS, the gap was large, but not that large, something like 70 or 80 runs (about 100 outs).

My instincts tell me that getting 33,000 batted balls, after controlling for batter, pitcher, park, should give you the standard batted ball distribution.  How much uncertainty should we expect?  If I do sqrt(.1*.9*33000) that gives us 1 SD = 54 outs.  I’m not sure that that equation is valid, but I use that as a starting point.  Ichiro is 3 SD away, which by itself means nothing if it was cherry picked.

"UZR using STATS and BIS “

Actually, bUZR does like Ichiro.  It’s sUZR that doesn’t.  There’s a huge gap BETWEEN the two, not that the two confirm each other.

Yes, it was cherry picked, so it means something but not as much as if it were not cherry picked.

And I don’t care if it is 1 SD or 100 SD.  This is strictly a Bayesian probability issue.  What are the chances that any given stringer will be biased must first be answered.  I don’t know the answer to that.

What if you and I looked at every batted ball together and neither one of us were drunk or stoned, and we both concluded that all of Ichiro’s batted balls were recorded correctly?  And yet he was still 3 SD from a typical distribution.  What would that 3 SD mean?  Nothing.  That is why it is strictly a Bayesian problem and you need to know the chances of stringers being biased or making errors…

And my comments still stand.  If in 33,000 (or 100,000) batted balls, we find that Ichiro’s batted balls were recorded as “non-typical,” we don’t conclude that the stringers made a mistake and that there should be a perfectly standard distribution.  We conclude that the stringers likely made a mistake AND that Ichiro did indeed have a non-standard distribution. That is a very important distinction.  That means that his likely true defensive talent is somewhere between UZR and WOWY. How much between depends on 2 things:  One, the sample size and how likely it is that he would get a non-standard distribution by chance, and two, how likely it is that a stringer would make mistakes of the various magnitudes.  But still, the real answer is still somewhere in between, no matter what.

So, you tell me, Tango, if Ichiro is 3 SD from the mean in batted ball distributions, how much do we regress that to estimate his true batted ball distribution, if there were no mistakes in recording that data? It is not 100%. Is it 50%?  10%?  90%?  I have no idea and I think nether do you. So let’s dispense with the notion that it is 0%, which was the assumption in the original discussion.

To answer my last question (which can be answered), how much to regress those numbers, we would have to look at the spread of batted ball distributions for many players and compare that to what is to be expected by chance if there were no stringer bias or mistakes. Alternatively, or in conjunction with that, we can try and figure out the rate of stringer error, by experiments or by analyzing the data in the various parks, etc.

Peter put Ichiro in context in #9, but Tango - could you provide the LD/(LD+FB) with Ichiro and without in your tests. I want to see the numbers when controlling with WOWY.

I am going through Hit f/x data. As Tango has guessed a few times, there definitely is a bias having hits more likely to be called a LD, given the same launch parameters.

For example 15 degrees verticle, 100 mph (all numbers grouped to nearest 5) 80.2% of all LD+FB are LD’s. If it’s a hit, 84.4%, if an out 69.8%.

20 deg vert, 100 mph, 31.3% of all are coded as LDs, 36.4% when it’s a hit, 23.9% when it’s an out.

These are two of the largest samples, but the pattern is consistent.

I’m working on putting an article together from the numbers.

Brian, I don’t know if I can do it fast enough.  I’ll do it in the offseason when I have to rerun all my numbers with the 2009 data.

As for your HITf/x data: great stuff!

First off, look what Brian did: if the ball was hit at the same speed, but one was at 15 degrees and another at 20, then it was marked a LD 80% of the time in one case and 30% in another.  Right there, you have one potential source for bias: the (perceived) launch angle.

Then, as expected, if the ball was caught for an out or not had a huge impact in determining if a ball was an LD or a FB.

The problem is that the stringers work at one park, rather than being spread out, so that these biases are not random but systematic.

Brian - The problem with your examples is that because of the differences in the magnus force for balls hit with an initial angle of 15% or 20%, many of them WILL actually end up as fly balls and when they do they will be more likely to be outs.  If we had hang times we could readily tell the differences in LDs and FBs for balls hit at the same initial angle, but we don’t, and until we do your proposed study won’t have much meaning.

Peter - just to clarify - you are saying that if there are two batted balls, with the same horizontal and vertical angles, and the same speed off the bat, that one may carry further (have more hangtime) than the other because of a possible difference in the magnus force (which is unrecorded) - even if all the recorded parameters are equal?

What I am seeing is when plotted against vertical angle, line drives, fly balls and popups all have a bell shaped distribution, with overlap regions. Balls between 15 and 30 degrees are coded as either line drives or fly balls. Between 35 and 60, as either fly balls or popups.

For balls with a vertical angle of 20 degrees, 466 were coded as line drives, 472 as fly balls. Looking at the speed of these 20 degree vertical balls

speed num BABIP   LD   LDifhit LDifout
 60    18  .500  .944   .889   1.000
 65    27  .852  .852   .826   1.000
 70    38  .921  .947   .943   1.000
 75    59  .953  .734   .754    .333
 80    77  .584  .597   .644    .531
 85   108  .343  .583   .757    .493
 90   180  .300  .465   .725    .353
 95   212  .354  .330   .440    .270
100   166  .596  .313   .364    .239
105    96  .510  .427   .449    .404
110    27  .370  .556   .600    .529

At 20 degrees vertical, 90mph off the bat, 70.0% were caught for outs, 30.0% were hits. 35.3% of those outs were recorded as line drives, while 72.5% of the hits were.

Peter - just to clarify - you are saying that if there are two batted balls, with the same horizontal and vertical angles, and the same speed off the bat, that one may carry further (have more hangtime) than the other because of a possible difference in the magnus force (which is unrecorded) - even if all the recorded parameters are equal?

Yes, Brian, that’s exactly what I am saying.  At the Pitch f/x Summit I said in my presentation that I didn’t think that misclassification of batted ball types was a major problem.  To defend that statement and perhaps put this matter to rest, I decided to compare April 2008 Gameday hit ball classifications to Retrosheet’s.  I have no idea where the Retrosheet data comes from.  I am assuming that it is a different and independent source from Gameday since they are not identical.  I ended up with 22604 hit balls to compare.  There were 20 differences.  14 involved LDs classified by Gameday that were classified as FBs by Retrosheet. 6 of these were on hits in the field of play, 7 on outs and 1 on a home run.  3 FBs by Gameday were classified as LDs by Retrosheet, all on outs.

Only one misclassification happened in Seattle.

I don’t have access to BIS or STATS data.  Perhaps there is a bigger problem between those two datasets.

It is possible that Retro gets its data from MLBAM.  Indeed, that you found so few discrepancies tells me that they did.  I don’t think you need an identical match, because of the data translation from one system to the other.  If you ask Dave Smith if he got it from MLB, he’ll tell you.  He doesn’t volunteer the information, but he answers when asked, especially when confronted with data like this.

I’ve studied the matter similarly, comparing Retrosheet ‘08 to Gameday ‘08, and I do think that’s where Retrosheet is getting the data. There’s way too much similarity in my mind for it to be anything else. (I posted the results in a thread around here somewhere.)

I agree with Peter; the known Hitf/x parameters don’t constrain the ball’s trajectory (and catchability) enough to require the conclusion that the stringers are mis-classifying based on outcome. Both trajectory and outcome partly depend on an unknown launch variable (initial amount of backspin for the batted ball).

Presumably stringers don’t code batted ball type based on initial trajectory, but rather on overall trajectory. Theoretically, two balls launched in the same direction at the same speed at an initial launch angle of 20 degrees but with a large difference in backspin will end up varying significantly in terms of the apex and shape of their trajectories. [I don’t have any idea how much variation in backspin there can be in practice for matched launch angles.]

There is another problem. Brian’s 5 degree “binning” of launch angle isn’t sensitive enough to avoid creating a biased sample, though there isn’t enough available Hitf/x data yet to make smaller bins very meaningful in themselves. Nonetheless there is a clear pattern of BABIP declining by launch angle in this range of launch angles, independently for each trajectory type. The table below uses bins with 1 degree increments:

angle N L% L_BABIP F_BABIP L_SOB F_SOB
15 200 91% 0.76 0.56 92 100
16 203 81% 0.73 0.61 90 96
17 230 79% 0.69 0.61 91 94
18 211 59% 0.67 0.48 89 95
19 183 57% 0.66 0.52 85 93
20 193 49% 0.64 0.34 82 93
21 177 46% 0.55 0.38 84 92
22 177 36% 0.68 0.39 83 92
23 163 30% 0.63 0.39 81 89
24 172 22% 0.55 0.35 82 89
25 151 17% 0.60 0.26 80 88

[the headers are vertical (launch) angle; number of balls in play described as flies or liners; pct of these which were line drives; BABIP on line drives; BABIP on fly balls; speed off bat for line drives; speed off bat for flies]

Consider the launch angle bins from 18 to 22 degrees, which should correspond to Brian’s 5 degree bin around 20 degrees. The distribution of balls classified as line drives declines with increasing launch angle in this range, such that there are twice as many 18 degree launched line drives as 22 degree launched line drives, which is qualitatively what we expect. Meanwhile the distribution of fly balls increases through this range, such that there are 33% more fly balls launched at 22 degrees than at 18 degrees. Using a 5 degree bin centered at 20 degrees means that 22 degree flies have about as much weight in the sample as 18 degree line drives, and twice as much weight as 22 degree line drives. Even leaving aside the unknown backspin effects, Brian’s binning really just captures the effect of 18 and 19 degree line drives being harder to catch than 21 and 22 degree fly balls. That doesn’t demonstrate stringer classification bias ...

I appreciate the comments - I want to make sure I understand this stuff correctly as I move forward.

Smaller bins means more table entries, fewer entries in each bin, and we have three elements to consider. After rounding to 5 degrees, I used 20 because it had a large sample, and was split almost 50/50 between liners and flies. Then I broke that one down by SOB. Trying to show it’s a consistent effect without having to do a complete data dump!

I was trying to answer what Tango had asked - if two balls are hit the same (as best we can tell from the provided metrics), is the classification as LD or FB biased by whether the batted ball ends up as a hit or an out? So far, I think so.

In Joe’s table above, line drives have higher BABIP and lower SOB than fly balls hit at the same vertical angle. I would have thought, at the same angle, that line drives would be hit harder than fly balls! Is this not confirming that if it’s a hit it is more likely to be coded a LD? If so, isn’t that evidence of a bias? If not, please convince me why.

That was in an attempt to answer Tango’s question, as relates to Ichiro and fielding ratings based on the hit classifications.

Part of that, as well as hitting analysis, is assigning a fixed BABIP for LDs and FBs in computing an expected value. What I see is that there’s a distribution of liners and flies, on a bell shaped curve, and that they overlap.

Is it possible that a batter (Andruw Jones, Jason Giambi, Junior Griffey, Joe Crede) who hits a lot of popups and flies (high angle) and fewer liners and grounders (low angle) will have a distribution of air balls that will give him lower than average BABIP for both his LDs and FBs, while someone like Derek Jeter is a low angle hitter who will have a higher than average BABIP on his LDs and FBs - therefore, their mix of G/L/F/P can indicate sustainable extreme low or high overall BABIPs?

Instead of doing one regression, as in xBABIP, I think we should regress to find a BABIP for each of GB, LD & FB, then sum for the overall.

Brian - Using Alan Nathan’s trajectory simulator with inputs of 100MPH SOB, a vertical angle of 20 degrees and varying the backspin from 400 RPM to 3000 RPM ( the range I found on air balls that I calculated by hand from video footage) I found the distance could vary from 311 feet to 395 feet and the hang time from 3.24 seconds to 4.92 seconds.  Ball A would probably be classified as a line drive and ball B a fly ball by almost every observer.  Ball A, the 311 foot line drive, would probably end up as a base hit.  It is possible that it could also be an out if the batter wasn’t known to have power and the outfielders were playing shallow.  Ball B, tthe 395 foot fly ball, might be a home run, a base hit, or a long out depending on the horizontal angle at which it was hit and the positioning of the fielder.

I don’t know if my comment this morning is still stuck in moderation, or if it got lost ... Trying again

Brian,
As far as sample size within bins go, the problem is worse - horizontal (spray) direction definitely matters too, and should be another dimension upon which binning is based. But with one month of data and using usefully small bin sizes, you’d have few bins with more than one or two balls in them.

I’ll rephrase my point about the 20 degree bin, in case it makes it clearer.

angle #LD #FB
18 125 86
19 104 79
20 94 99
21 82 95
22 63 114
tot 468 473 (binning at 5 degrees)

You have similar totals of LD and FB in your 5 degree bin, but this disguises the fact that the balls in your bin aren’t that similar. The average launch angle of a LD in this bin is 19.7 while the average launch angle of a FB is 20.2. That 0.5 degree spread is 10% of the width of the bin. There’s clearly a large drop in BABIP (at least 70-100 points), whether FB or LD, across the width of the bin. You either have to bin differently, or try to adjust for the biased sample. 

As far as speed off the bat goes, one observation and then two theories. The curve for average SOB vs launch angle is different for flies and liners. For line drives it peaks at launch angles somewhere under 10 degrees, and you basically don’t have any fly balls at all until you get out to launch angles of 14-15 degrees, and average SOB immediately declines, but slowly. As you mention (sort of), different hitters tend to “aim” for different launch angles. 1) within the family of line drives, a 20 degree launched line drive is more of a mis-hit than a 20 degree launched fly ball. 2) stronger batters may “aim” to use their strength to hit long fly balls - because of their strength they will have higher average SOBs for any batted ball, but hit a disproportionately large proportion of fly balls, while not hitting a disproportionately large number of line drives. If that’s true, you need to adjust for the batters in some way before drawing any conclusions about differences in batted ball speed.

It is intriguing that the SOB is higher for balls classified as flies than for balls classified as line drives, given similar launch angles, but there are other plausible explanations to investigate besides stringer bias.

If you missed it, you may want to look at Harry Pavlidis’s article http://www.hardballtimes.com/main/article/an-early-look-at-hitf-x/ There are probably other good ones I’ve missed or forgotten ...