THE BOOK--Playing The Percentages In Baseball

If I were in any sort of position of power with any MLB team, one of my spring training rituals would be to put prospects in the field at their respective positions, hit a slew of balls at ‘em, and do the f/x-type tracking of their performance (from ball off bat to completion of pseudo-putout). The idea would be to quantify fielding ability independent of game situation, to a reasonable extent. It wouldn’t take very many sessions to build a sizable database, on both individual players and in the aggregate. It would provide a useful basis for comparisons.

In addition to classifying each batted ball by it’s typical out-rate, one should also consider the impact of an out not being made. 

As an example, think of a duck-snort fliner in front of a charging outfielder vs. a ball that would just barely clear the fence.  In the first case, the outfielder would have to make a shoe-string catch.  In the second, he would have to jump and pull back the would be homer.  The typical out rate, albeit very low, could be the same.

However, if the the outfielder fails to make the former (and plays it on the hop), it will be a single with each runner advancing one base.  For the latter, the result of not making the play is a homerun.

Wes: agreed. 

While I definitely mean my point to be in terms of run value, not out value, it’s easier to have the discussion if we talk about the out rate (akin to a batter’s OBP rate).

Brooks - Just the kind of evaluation system that you describe above was suggested in the Glenn Shoenhals presentation at the 2010 Pitch Fx Summit: http://baseball.sportvision.com/summit/archive/2010.  Also discussed was the possibility of programming pitching machines to simulate hitting in a controlled way to different parts of the field to more quickly pinpoint the limits of a player’s range.  Both ideas make a lot of sense for evaluating young players especially.

Tango, can you clarify the standard deviation numbers used in the article. It reads as though the standard deviation for OBP for a single plate appearance is 2.5% when in fact it is more like a 100x that amount, so it’s unclear to me what you’re saying. Similarily, the standard deviation after 625 plate appearances would give you a range around the mean of about 10 points, say .324 to .342, or about 2.5% on each side of the curve, so where does the 0.1% come from. The numbers don’t really change the point of your article, but I’d like to be sure everybody is on the same wavelength when using standard deviations and related percentages. Thanks.

Thanks for pointing out the confusion.

I’m talking about the CONTEXT.  If, for example, every batter faced Doc at Citizen’s Bank Park, and in front of the same 8 fielders, at 25 celsius, at night, then the standard deviation of the *context* would be zero.

But, the reality is that you have a spread in talent of opposition, of parks, of temperature.  Your “strength of schedule”.  That spread is going to be fairly tight.  That is, the context of Doc in good conditions for him would be an OBP of 25%.  The context of some crappy Royals pitcher in fairly not-good conditions would have an OBP of 40%.

So, that’s the context that a batter faces for every PA.

***

A fielder is not like that, and the reason is because the ball is already in flight.  Whereas the ball is still in the pitcher’s hand when we establish the context for a hitter’s environment as being 25% to 40% in OBP range (1 SD = 2.5%), the context for a fielder is that the ball is in flight, and so, he’s got a 0% to 100% chance of getting an out on any batted ball.

Tango, excellent explanation.

If we were starting from scratch, I would want the fielder’s initial position, the ball’s position (then calculate distance and angle from fielder to ball) and the time for the ball to go from the bat to that position. Field f/x or Trackman should be able to give us that.

But we are working with the publicly available data, and with the descriptions and measurements chosen by others.

Tango correctly points out that trying to be too fine with the classifications introduces bias. For example, using Gameday’s xy coordinates to create a vector (horizontal angle) for balls in the air to the outfield, I found that with any more than seven vectors, hits in the gaps were recorded as being farther from the outfielders, while outs were recorded as being closer. So given the limits and biases of the available data, left field line, left field, left center gap...etc is as detailed as we can go without getting noticeable biases in the data.

Brian, yes, great way to describe the issue of “too fine” of a classification.

I think this is why Peter goes the route of the “Big Zone” metric, and why Retro zones might be better to use than the finer STATS/BIS zones.

Interestingly, MGL used to take all the STATS/BIS zones and collapsed them into the much larger Retro zones, until some people (maybe me even) told him to keep the data as-is from STATS/BIS.

That’s something that Colin had been pointing out, and I was able to verify in the Gameday data. For however many outfield vectors I created, I calculated the babip in each. It should fairly smooth curves, with the lowest hit rates where the ball is closest to an outfielder, higher away from the outfielder. But with more than 7 zones it starts getting bumpier, not just random variation, but you can see new high and low points reflecting the classification bias. Halfway between the outfielder and the center of the gap would have a lower hit rate than right at the outfielder, because catches were being clustered away from the center of the gap, towards the outfielder.

I’m always curious about the anomolies, which can show you where the current system goes wrong on an individual level. This year, Carlos Lee had great OF ratings. Everyone agrees that he is not a good outfielder - he fails the visual “looks bad out there” test, previous years and various methods have him as being at best average, and typically bad. So most people will agree that Carlos Lee, for whatever reason, got lucky this year. There are two basic ways I see to get lucky.
1. He got a lot of plays that fell into the “anyone could make that” category, above the normal percentages by a significant amount. Coaching coud get some credit here, but OFers can go long stretches without any particularly difficult chances coming up.
2. He made an unusual number of low percentage plays.

There are also data issues I could see happening for a bad player - if balls consistently fall a long way from him, maybe the stringer thinks they were liners instead of flyners (or similar). Or the zone classes get messed up, for reasons already touched on here. RZR had him very low, out of zone plays (as reported by fangraphs) didn’t have him particularly high on a per inning basis. Aside from a good arm rating, I’m not sure, taking a quick look, at what exactly the systems think he was good at.

We know some hitters will fluke into a high average, and we can usually tell why (high BABIP on grounders and/or flyballs, high HR rate, low IF pop rate, etc). What I think needs to be stated more clearly is what exactly is happening when a player “flukes” his way to a high defensive rating by the current systems. Is there any way to tell for a particular player?

David Pinto is one guy who does a fantastic job to highlight these issues.

Ideally, what we’d see from MGL (and Dewan and everyone else) is how much each adjustment contributes to the overall rating.

For example, you might have Carlos Pena as -12 runs unadjusted, and +1 run adjusted.  But in-between, adjustments are made for: location, trajectory, base-out, pitcher tendency, park, etc, etc, etc.  It would be lovely to see how each adjustment makes the changes.

Are the adjustments on the objective parameters, or subjective ones, and by how much.  Are those that are influenced more by stringer bias the ones that have the most adjustments?

Is there a free source for fielding play-by-play data, ie, who caught or should have caught a given hit?

There’s Retrosheet and Gameday.

Retrosheet denotes who was the first fielder to touch a ball, and that’s the one I use.

Biggest problem there is if you have a CF that touches a groundball, you don’t know if it went to the SS or 2B side.  Similarly, a LF that touches a GB may have gotten to a grounder in the hole or down the line.

At least, it reduces the opportunity space down to 2 infielders.

(Of course, the quality of that data has not been tested.  While the scorers are motivated to get the outs correct, and backup procedures to get those right, they are not necessarily equally motivated to make sure they got the hits-touches correct.  Who knows exactly how accurate those are.)

You can try to make some reasonable guesses as to what happened, and that’s what Dan Fox, Sean Smith, Colin Wyers, Brian Cartwright and/or Peter Jensen might be doing.

As far as who SHOULD have made the play, Josh, you should know much better than me!

Ha, true.  Just looking for some pbp data to combine with my own dataset.

Retrosheet denotes who was the first fielder to touch a ball, and that’s the one I use.

Gameday does, too, although not in such a direct manner as retrosheet.

It’s a shame that the f/x era missed the artificial turf era by a good 10-15 years. When a number of parks had grass fields and a number of other parks had turf, you had a source of variation just begging for quantification by the quantitatively inclined. Not so much, these days, even though the tools are now here. Too bad that the eras really didn’t overlap.

"Gameday does, too, although not in such a direct manner as retrosheet.”

Not sure what you mean by that.  Retrosheet has been getting their data for recent years from the gameday files.

Parsing the Gameday files is nontrivial, at least compared to parsing Retrosheet event files, is what I think he meant, Rally.

Rally/19

Yes, I meant what Colin said in 20.