THE BOOK--Playing The Percentages In Baseball

I’m not sure if I’m just forgetting something, but in the THT 08 annual you had a few different methods, looking at stadium, pitcher, and one or two others. You had different lists for each variable. Here you only have one list, so I’m curious as to how you consolidated it?

How did you define a season for this?

Any easy way to compare side by side with UZR for those same years?  Do you have UZR data going back that far, Tango?

Nice job!

Season = BIP / 4000, excluding all bunts.

MGL: I have sUZR back to 2003, and bUZR to 2002.  (Or the other way around?)

Out of curiosity, what was Alfonso Soriano in LF and at 2nd base?

Tango:  do you have any estimate of the error ranges around these results?  I would think that Andruw’s, for example, might be relatively high despite 10+ years of data, because he missed so few games.  So the non-Andruw sample is fairly small.

He’s not just comparing Andruw to his backups, at least I think.  There’s batter outs to CF against Andruw and against other CF, and pitcher flyouts to CF with/without.  But I’m not sure how all the parameters are combined into one number.

Is further explanation on deck?

I’d like to compare those to Totalzone, time permitting.  Anyone who has the time, you know where my site is.  I’ll start with this:

Everett +65
Jeter -108

In runs, not plays made, but the spread is still much smaller than WOWY.

Rally, yes, I noticed the spread in your numbers is much smaller than mine, and somewhat smaller than UZR.

***

“So the non-Andruw sample is fairly small. “

Guy, as Rally noted, I am not limiting it to Andruw’s backups.  A refresher:
http://tangotiger.net/catchers.html

I am comparing:
1. How pitchers who have had Andruw as their CF compared to when those pitchers have not have had Andruw, at any point in their careers (split by bat hand)

3. How Andruw does in each park compared to other CF at those same parks (opponents, and mates, split by bat, pitch hand)

3. How batters hit when Andruw is in CF or not (split by bat, pitch hand).

4. How Andruw does by the different ball classification (fb, gb, ld, pop… not crazy about this one, certainly at a career level, because the identity of the pitchers and batters should cover me here).

Anyway, how I combine all that is fairly complex, but the one-line summary is that the more BIP, the more weight I place on #1 and 2, and less on 3 and 4.

The presumption is that a pitcher will not change his ball in play distribution, so over the course of career like Andruw, he’s had so many pitchers, and they’ve had so many CF, that it’s really irrelevant what their actual BIP distribution is, because they’d have the same one for Andruw and not.

Is this presumption valid?  I haven’t tested it (yet).  But, the results certainly gives me confidence that the presumption is at least realistic.

This is why I’m not crazy about #4, because you guys know I don’t trust the GB, FB, LD, Pop classifications too much.  Knowing the pitcher and batter should tell me what kind of batted ball type I should expect (given a large enough sample size).

Wow I had no idea Sosa was such a great fielder, or that Maddux was as good as his Gold Gloves.

Very interesting numbers that seem to correlate very well with scouting observations.  The name that sticks out to me as a surprise is Luis Gonzalez.  Erstad really was a great outfielder all those years.

In THT ‘08 you use the identity of the baserunner, and not the batted ball classification (IIRC). Why the switch?

Also, what did you decide to do, if anything, about the extremely wide spread in your numbers relative to those of TZ, UZR, etc.?

Sorry to dig up this thread after a month of inactivity, a friend today tried to convince me of Derek Jeter’s defensive excellence.

In the annual, I only looked at “on 1B or not”, and 70% of the time it was “not”.  So, it doesn’t really tell me much.  I should really look at more targetted runner/out/score/inning configuration, specifically: “in, DP, back, cover the line”.  I didn’t get around to that. 

I would not consider it a “switch”, but simply “temporarily drop”.

Nothing for me to do about the spread: it is what it is.  It’s extremely wider than Rally’s, but only somewhat wider than UZR.  Remember, these are plays, not runs.  Multiply by .75 for the IF and .85 for the OF to get it in runs.

Tango:
Are you going to turn this data into an article for THT or THT annual?  Seems well worth doing.

One thought on the park version of WOWY:  It seems to me that the park comparison is problematic, because the “other” fielders in the home park are facing only the fielder’s teammates as hitters.  That’s a very limited pool, which may have extreme GB/FB, LH/RH, or other configurations.  In looking at Jeter, for example, it’s likely that opposing SSs in YS look worse than they really are because the Yankees have had an above-average proportion of LHH over Jeter’s career.  And it is the park version of WOWY in which Jeter looks least bad.

Of course, there are factors not controlled for in every WOWY comparison—but this one seems potentially large and easy to remedy.  It might be better to handle park as a park factor, rather than straight WOWY.  You could compare the overall SS out% (Jeter + other NY SSs + opposing SSs) in YS vs. all other parks (same players), and calculate a “YS SS factor” which is applied to your WOWY estimate. 

Maybe sticking with a single methodology is easier for people to follow.  But saying you park adjust the WOWY results doesn’t seem like a hard thing for people to understand.  Anyway, a thought….

No, not an article.

Note that the “Park” component also controls for handedness of pitcher and batter.  So, it would be: how does Jeter do at Yankee stadium facing LHH with a LHP on the mound, and how do other SS do under the same conditions.

And, remember, it’s not just that year, but EVERY year from 1993-2008.  The pool of LHH/LHP is not just limited to say Jason Giambi for a given year.

And, Yankee stadium is only half the parks Jeter plays in.  In the other parks, the pool of players shoots up, as I’m not ensuring that it’s only Yankee players involved.

In my new WOWY (the one that includes handedness of batter, pitcher), Jeter is terrible across the board, including park.  He only looked good, I presume, because he faced a good share of RHH.

I didn’t realize you were now breaking the park analysis down by batter/pitcher handedness.  Done that way, it should work fine. 

If you have the interest/energy, I do think there would be an audience for a post or article reviewing your revised WOWY methodology and your findings. Maybe gold gloves of the decade (2000-2009)?  I imagine that many will disagree, but for players with a lot of seasons at a position (6 or more?), I think I’d have as much confidence in WOWY as in the play-by-play metrics.

I am struck by the huge disparity between WOWY and Total Zone in the rating for Jeter (both through 2008):
WOWY -327 runs
TZ -108 runs.
Both systems are controlling for pitcher/batter handedness and the identity of opposing hitters.  WOWY controls for other factors as well, but none of them really change the rating very much AFAICT.  So how can the systems reach such different conclusions? 

WOWY is much closer to what we would get by simply comparing Jeter’s outs/BIP to that of an average SS.  So the question is whether he really had far fewer opportunities than a typical SS, as the TZ rating implies.  The major difference in methodology is that TZ knows (in many cases) the batted ball type and which OF fielded the ball on GB hits.  So one quick check Sean could do would be to see if there were far fewer GBs with Jeter on the field, and/or far more GB hits fielded by the RF than is typical.  If not—which I’m pretty sure is the case—than Jeter cannot have had many fewer than average opportunities. (He could still have had more difficult opportunities, but TZ doesn’t try to account for that.)

* *

What I think is happening is that TZ artificially pushes all fielders too close to the mean, by reducing the imputed opportunities to bad fielders and increasing opportunities for good fielders.  This would explain why the TZ variance is lower than WOWY or UZR.  However, I know a lot less than others here about TZ and other defensive metrics, so please set me straight if I have this wrong (an unnecessary invitation, I’m sure).

Let’s take a simple example:  a team where the SS makes 1 fewer play per 100 BIP, and the team thus gives up 1 more hit (.310 BABIP).  All other fielders are average, as are the pitchers and opposing hitters.  If the version of TZ w/o PBP data, the SS is charged with about .17 extra hits, giving him a fielding % of .66 if the average is .70.  That tranlates into -.41 plays for the SS, who in fact is of course -1 play. The reason for the gap is that his failure to make the successful play reduces his opportunities, while he is only charged with a fraction of the hit he gives up.

In the more advanced version of TZ, the SS’s extra hit allowed would be a GB hit fielded by either the LF or CF, and the SS would be charged with approximately .5 hits on these plays.  In this version of TZ he will be rated at about -0.6.  That’s closer to correct than the simple version, but still gives about 40% of the blame to the 3Bman and the 2Bman. 

So another check on TZ is to see if poor fielders generally have fewer imputed opportunities than good fielders.  I think they will, but of course that can’t really be true.

All of this is reversed for great fielders:  their denominator will be increased precisely because they are making so many plays, and so will receive less than full credit for extra plays made.  Andruw is +180 in TZ, +329 in WOWY; Rolen is +128 in TZ, +276 in WOWY.

WOWY gets a different answer by focusing on expected OUTS at each position, given the batter/hitter/park combination.  That should give us a better estimate of players’ real opportunities, because it is independent of the player’s own fielding success. 

Does this make sense, or have I misconstrued TZ in some major way?

The spread in TZ is about half to two-thirds that of WOWY, I believe.

WOWY is a bit wider than UZR, which you’d expect.

I think that Rally either:
a. intentionally suppresses the numbers
b. double-counts a player by having him compared against himself in part of the sample
c. treats the out/run conversion differently
d. has a bug
e. is right and I’m wrong

***

Let’s look at Adrian Beltre.  The average 3B converts 8.9% of all balls in play into outs.  If you look at his particular batters, 3B (other than Beltre) converted 8.9% of their BIP into outs.  If you look at Beltre’s pitchers, their 3B (outside of Beltre) converted 8.9% of their BIP into outs.  If you look at his park, it’s 9.0%.  If you look at his batted ball distribution, it’s 9.0%.

Basically, Beltre has faced a very neutral context in his career (through 2008).

He has faced 38,114 balls in play (I remove bunts, HR that clear the fence, and pitchers-as-batters).  He has made an out 3579 times. The baseline (average) is somewhere around 8.9% to 9.0%, or roughly 3400 outs.  That is, Beltre is roughly +179 outs above the average 3B, give or take (my range is between +156 and +189 outs, or +125 to +150 runs).

Rally has his TZ at +90 runs.

***

- UZR has him at +101 runs (but starting only in 2002). 

- If I start him in 2002, WOWY has him between +87 and +126 plays, or roughly between +70 and +100 runs.

- TZ has him at +73 runs.

***

I think this is pretty common for TZ.  Rally rarely (never?) has the kind of out-of-this-world numbers that UZR or WOWY has.  I have to believe his system is built with that kind of suppression.

Beltre is +90 runs for UZR (2002-2008… I had included 2009 earlier).

***

Btw, when I say that someone has a neutral context, this means that we need NO fancy-schmancy adjustments.  The point of the adjustments is to control for things that are out-of-the-ordinary.  In the case of Beltre, he hasn’t faced that (over his career), so that all the breaks evened out (eventually).

If you want an example like that at SS, we have Edgar Renteria.  He faced neutral conditions, and his WOWY is -113 to -168 plays.  If you made zero adjustments (just looked at his BIP and league out rate per BIP), he’d be -156 plays.  So, he is around -117 runs, give or take 10 or 20.

TZ has him at -12 runs.

That would mean that even though he converted 11.6% of his BIP into outs (compared to the league average of 12.2%), TZ thinks that the context Renteria faced was severely against him.

Like I said, it’s possible that players had the propensity to not his balls to Renteria, compared to other SS, for whatever reason.  Just bad luck.  Possible.  However, when I look at his batted ball distribution, I don’t see it.

In CF, we have Junior.  He’s faced a very neutral-type of context (since 2002).  Without any adjustments, he’s made 113 fewer plays than the average player, or about -95 runs.  (He’s also been average in RF, so that in CF+RF, he’s at -95 runs relative to positional peers).

UZR since 2002 in CF, has him at -73 runs.

TZ has him at -82 runs (in CF and RF).

So, this is an example where all three systems pretty much agree.

I agree (confess) to pretty much every point Guy made in #16.  TZ will be more conservative because a groundball that a shortstop misses to left gets charged as 0.4 plays (with part of that going to other fielders).

Let me answer these:

I think that Rally either:
a. intentionally suppresses the numbers
b. double-counts a player by having him compared against himself in part of the sample
c. treats the out/run conversion differently
d. has a bug
e. is right and I’m wrong

a. Not for retrosheet era stuff.  For the JAARF numbers used to estimate pre-retrosheet fielding, I did regress because I didn’t have as much trust in the results.
b. I think you mean that I don’t subtract Jeter’s stats from the league total before comparing him to league average?  I plead guilty, but one player makes up only a tiny part of the league average, this is not a big deal, and probably 99% of the time would change things by less than a run.
c. Run/out should be standard, .75 for singles, etc.
d. Probably true in many cases but I think the system, as designed, will result in tighter spread than WOWY, even if it was bug free.
e. Who knows?

Tango:  It’s not a, b, c, d, or e.  It’s just an inevitable result of a methodology that defines a fielder’s opportunities as outs made plus some fraction of hits allowed.  The fewer outs you make, the fewer opportunities TZ imputes to you.  That has to reward bad fielders and penalize good fielders.  Maybe Rally can just tell us how many opportunities a few good and bad fielders have had under TZ, compared to average.  I think the pattern will be clear.

The question to me is whether there’s some way of blending WOWY with TZ.  TZ has additional and presumably valuable information than WOWY doesn’t:  where the hits were fielded.  That should provide a more precise estimate of opportunities.  But WOWY uses a very powerful piece of information that TZ ignores:  how many outs a fielder would normally record given certain parameters. 

Suppose for each hitter you calculated the total of (SS outs + SS errors + GB hit/LF + GB hit/CF), and determined that X% of these GB were SS outs on average.  Then a SS’s actual outs against that hitter would be compared to this average.  Would that work?

Guy: I see.  If the number of hits allowed is somehow dependent on the number of outs recorded, then you are 100% correct on the bias.

Perhaps this make more sense for any single year, but once you get to the career level, the two need to be independent, as by that point, so much of it has worked itself out in the wash.

To illustrate this more clearly, consider an Adam Dunn level SS, who only makes outs on 6% of BIP rather than 12%.  So he’s -6 plays per 100 BIP.  TZ will define his opportunities as approximately outs + .5 * GB hits (to LF or CF), or about 14 in this case.  Then his out% is 6/14 or 43%, vs. a league average of 70%.  Apply the difference of -27% to his 14 opportunities, and he’s now just -3.8 plays (rather than -6). 

The trick is we’re now saying this SS had only 14 opportunities per 100 BIP, while the average SS has over 17.

Excellent illustration.

Actually, we can see that the hits allowed are inversely proportional to the outs made.

For example, over a long period of time, we know that the number of opps will be the same for every SS.  So, if one SS has more outs, then he also allowed less hits.

Rally’s model (is this the same problem with Dan Fox’s?) presumes that the two are independent, when in fact, they should be inversely proportional.

The problem is that for any given year, we don’t know if a SS faced a normal number of opps.  So, if he had few outs, we presume that he probably had few opps as well.

Like I said, this probably makes sense on a seasonal level.  But, if this keeps happening to the same player (coughJetercough), then we know this is incorrect.

It would seem that perhaps if Rally were to “split the difference”, then maybe he can get a better estimate.

In the example above, you have two estimates for “hits allowed”.  One is to treat the two independently, and therefore have the SS get 50% of the 16 GB hits to the left side.  That’s what Guy did, who gave him 8 GB hits, plus his 6 outs, for a total of 14 BIP.

The other is to say that if a guy made only 6 outs, when 12 was expected, then of the 14 BIP, maybe more than 50% should count toward him.  You look over, and see that the 3B made 11 outs, when 9 was expected, so less than 50% should count toward him.

I’m not even sure why 50%.  I’d first start off with the 12/9 split as “expected”, and make it 60% for the SS and 40% for the 3B (I guess that’s what Rally does).

Since the actual outs are 6/11, then let’s give 35% to the 3B and 65% to the SS.  That is, flip their responsibility.

This way, you get more hits to the guy who made fewer outs.

***

Of course, the problem is what if the 3B really got more opps.  This process will shortchange him.

My answer is two-fold:
1. test it, at the seasonal and career level
2. minimize the error as much as possible (at the seasonal and career level)

The answer may simply be that you will haev to have two different equations: one for seasonal and one for career.

What the effect would be of an Adam Dunn at shortstop is those hit to center field would be partially charged to the 2B, and hits to left would be partially charged to the 3rd baseman.  He’d make them all look bad.

Kinda like an angry sergeant who doesn’t care exactly who was laughing behind his back.  He’s gonna make the whole platoon drop and give him 20 pushups.

“I’m not even sure why 50%.  I’d first start off with the 12/9 split as “expectedâ€?, and make it 60% for the SS and 40% for the 3B (I guess that’s what Rally does).”

Actually, for groundballs fielded by the LF, it’s 60% to 3B and 40% to short.  This split is based on the project scoresheet zones in the 1990s data.

Since the actual outs are 6/11, then let’s give 35% to the 3B and 65% to the SS.  That is, flip their responsibility.

Good suggestion, but I think that could turn out to be very difficult to code.  Just thinking about the ramifications (you don’t have one shortstop or 3b, wouldn’t you have to go back and set that split for every 3b/ss pair?) makes my head spin.

I definitely would not have time to do it.  If someone wants to try I will offer the code used to create TotalZone and you can modify it as you wish.  There’s nothing there that is “secret sauce”, if my previous explanations are incomplete it’s due to the complexity of the calculations and no effort to conceal intellectual property.

I don’t think “splitting the difference” will get you what you want.  But couldn’t Rally’s data be used to calculate projected outs for each PA?  Given the hitter (and pitcher hand) and the knowledge that it was a GB, there would be an expected out% for each infielder. The difference between actual and expected outs is their plays made above/below average. 

In the meantime, I think we have to assume that the correct Total Zone rating for most IFs is about 65% larger (in both directions) for the PBP version, and about 100% larger for the non-PBP version.

Guy, you are correct.  That is WOWY, but with one extra dimension.

I do pitcher identity and batter hand for one metric.

I do batter identity and pitcher hand for one metric.

I do park and pitcher/batter hand for one metric.

I do batted ball type and pitcher/batter hand for one metric.

You are suggesting combining more of them, namely including batted ball type in the first three metric. 

My problem is that we are taking factual information and adding subjective information.  I REALLY don’t like the idea (especially because of the bias in the batted ball data).  What I could do is keep both, meaning the batter metric that includes and excludes the batted ball info.  This way, I can carry 6 metric (the first three, plus the next three with the batted ball info) as opposed to the 4 metrics I currently carry.

I could merge them all into one super-duper metric, but that gets into a whole other world of black boxes.  As it stands, everything I do is replicable.

All fair points.  And my suggestion doesn’t incorporate information on which OF fielded a GB hit, which seems to me to be Total Zone’s major “value added” as compared to WOWY.  And I can’t think of a way to do that, without adopting the formulation of opportunities = outs + x%*hits, which I don’t think really works for the reasons we’ve discussed.

Unless someone can figure out a way to incorporate the data on who fielded hits into WOWY, I’m inclined to agree it’s not worth changing the system.

*

Another issue:  I’ve always wondered how much value the fielded-hit data has for OFs.  Don’t fast/good OFs get to a lot more gap hits than slow/bad OFs?  And don’t weak-armed OFs defer to strong-armed OFs on fielded XBHs?  I’m sure these account for far less than 50% of all LD/FB hits, but I would think it happens enough to cause some problems.

My problem with the batted ball classifications is not only the zone locations, but also that they are broken down as GB, FB, LD, Pops.  Personally, I would set the trust level as GB v AirBall.

Even then, you have an issue.  A “ground” ball is any ball that hits the ground in the infield.  You can see how an infielder can shoestring catch it for a liner.  Indeed, it is IMPOSSIBLE to get a groundball classification, if the ball doesn’t hit the ground.  Therefore, there’s a bias based on the outcome of the batted ball (whether the infielder short hops it, or catches it on the fly).

Obviously, HITf/x clears that up for us, since what we really care about is trajectory and time to fielder (or some fixed point).  We don’t particularly care if it actually hit the ground.

So, I’ve got a strong inclination to not really incorporate all that data, especially since in 2 years, I would dicard it in favor of HITf/x.

I totally get why Rally isn’t interested in overhauling his system (despite the big paycheck he receives for this work!).  But if anyone else is interested in trying to improve the metric, I think the key task is to make opportunities independent of outs made (or as close to that as possible).  I think you could do that by expanding the definition of opportunities for each position.  For each IF, the pool needs to be that player’s outs, the hits fielded by relevant OFs, AND the outs recorded by adjoining IFs. For 3B, opportunities = 3B outs + LF hits + SS outs.  For SS, opportunities = 3B outs + LF hits + SS outs + CF hits + 2B outs.  Then, a fielder’s outs/oppor’s is compared to an average fielder.  To adjust for opposing hitters, the average fielder rate would be adjusted to reflect the out/opportunity rate for that position against a given pool of hitters. 

This obviously does not isolate “opportunities” in the sense of balls that a player could have actually fielded.  But it isolates the opportunities from the fielding ability of the player (and the fielding ability of adjoining IFs), because a play made in the IF reduces the hits fielded by OFs and vice-versa, keeping the pool of opportunities stable.

I suppose this would be similar to WOWY but instead of using all BIP to evaluate a fielder, it would limit the analysis to the relevant BIP type (GB for IFs, LD/FB for OFs) and to BIP generally on his “side” of the field.  That should increase the accuracy a bit.  I’m less concerned than Tango about subjective classification, as I imagine at least 90% of BIP are unambiguous and large samples handles the rest. 

At the same time, my guess is that once you have 5 or more seasons any adjustments beyond pitcher and batter handedness are unnecessary.  When all is said and done, I think we’ll discover that range factor told 90% of the defensive story after all (again, IF you have several seasons of data).

Range Factor is PO+A per 9IP.  So, no, it won’t tell the story to “90%”, whatever that means.  As we discussed in another thread, the denominator has to be, at most, BIP.  And the numerator, for 1B especially, and 2B/SS probably, should not include PO.

Back to what Guy is saying.  Right, in WOWY, I count all BIP.  He is right that perhaps I could exclude the “obvious” BIP that did not involve the SS.  For example, any ball fielded by the 1B or RF did not involve the SS.  So, I could remove those.

You could still have a bias, however, in the stringer recording of data for the non-outs.

And, as noted, the FB/LD issue is another one, as a line drive over the head of the SS should count as an opp.  And if it does, what about a line drive over his head but called a FB?  There’s scorer bias.

I agree you could go down that road.  I just have very little interest in doing so.  I don’t think the payoff is there, especially as Guy concedes and agrees with me that when you look at the career level, all those batted ball distributions will cancel out anyway.

And you are left with balls fielded for an out in the numerator, and all balls in play in the denominator, and all you have to do is adjust for the identity of the batter, pitcher, and park.  Which is what I do with WOWY.

WOWY basically becomes the sanity-check standard for long careers (I dunno, say at least 20,000 balls in play).  If a metric deviates from that, it better have a good reason, like guys hitting lots more GB than expected of them, when that particular SS is on the field.

“Range Factor is PO+A per 9IP. “

Just to remake the point, the denominator of Range Factor INCLUDES strikeouts, and EXCLUDES hits.  That is an abysmal tradeoff.

Tango:  I wasn’t suggesting you make any changes whatsoever to WOWY.

If your point is that BIP is a better denominator than innings played, of course I agree.  And if you’ve got the data, might as well use is.  But I think for players with at least 5-6 seasons of data, the distinction will usually be small.  For Jeter, 74% of opposing PAs have resulted in a BIP, and lg. avg is also 74%; for Everett, it’s 73% (vs. league 74%). 

I suppose, though, that assists/9 would be an even better metric than range factor for infielders. 

As for my admittedly imprecise “90%,” I just mean that if you compared WOWY and assists/9 for players with 5+ seasons, I’d expect to find a high correlation.  And btwn WOWY and assists/BIP, an extremely high correlation.

Bumping… and I’d recommend AT LEAST reading starting at post 23. 

But really, read the whole thing.  This is a great thread, and thanks to Guy and Rally and everyone else for making it as enjoyable, even three years later.