THE BOOK--Playing The Percentages In Baseball

I find myself wondering how seriously one can take such a remark by Epstein, given that he’s talking about one of his players. Right off the top of my head, I can think of a couple of reasons why he would say something like that, even if he really thought that Ellsbury was a below-average centerfielder.

First off, as GM of the Red Sox, Epstein has to consider how the player will react to a public statement on his part. What good can come out of Epstein publicly saying that Ellsbury grades out as below-average at CF according to their system? Nothing. However, something bad could come out of it. If Ellsbury took pride in his CF play, he just might take umbrage at such a public remark.

Second, Epstein may be engaging in gamesmanship. Suppose he would entertain a good trade offer for Ellsbury. The Red Sox have a reputation of being a sabermetric friendly organization. They’ve also had recent success, which might suggest to other GM’s that they know something other organization’s don’t. If Epstein says that their grading system shows that Ellsbury is a plus CF, other GM’s just might be inclined to believe it.

This doesn’t discount the possibility that Epstein believes it but is in error. However, we don’t really know that.

#1/Chucko has it, I think. Even if Epstein does believe Ellsbury is a bad fielder, he wouldn’t want to devalue that asset in the trade market. Part of his GM role is sales guy, and you know sales guys love to make up a story when it helps a deal.

Completely agree with ChuckO/1 here. Epstein has his hands tied talking about his own player, but all the signs are that he himself doesn’t drink the Kool-Aid:

-Ellsbury’s “temporary” shift to LF being sold as an opportunity to run more on the basepaths

-The Red Sox payroll commitments strongly suggesting that Ellsbury will be supplanted in the near future by Kalish

Epstein is very calculated and diplomatic with the players in everything he does.

The only other defense I can think of for these remarks is the assertion of some that Ellsbury often shaded to left-center to protect for Jason Bay’s limited range. Don’t know how that would shake out in UZR, but I haven’t seen any real hard data on that.

It’s a PR statement.  If the Red Sox thought Ellsbury was really above average in center, Cameron would be playing left.  Actions speak louder than words.

For all three oufield positions combined, I have Ellsbury at +14 in 2006, +13 in 2007, +25! in 2008 and -1 in 2009. A projection from the last three seasons comes to +6. There may be a reason that explains 2009’s low results, but Epstein isn’t sharing.

There have been a few articles in the Boston media on this topic obviously.  In every one of them it’s been clear that while the writer is specifically honed in on Ellsbury’s 2009 performance that Epstein is only talking about Ellsbury’s true defensive abilities.  There’s always room for the idea that Ellsbury is an above average defensive player who may have had a bad year.

That careful divide isn’t as obvious here, although that may perhaps be because it’s a talk radio interview.  I think there is a little wiggle room in the verb tense of that last line though:

We do our own stuff and it showed that he is above average.

I may very well be reading too much into it, but when he says what Ellsbury currently *is* that seems to me as much a statment of Ellsbury’s innate defensive ability which probably is above average based on scouting reports and Ellsburys full MLB performance than whatever Ellsbury’s 2009 performance *was*.

Now he shouldn’t have said that these online numbers are worthless, but that may only be sloppy and not wrong.  He’s been bombarded with people suing those single year numbers to definitely say that Ellsbury is a bad defender.  Those statements are worthless.

Geez, and I’m usually more of a Epstein realist.  I hope this isn’t too much the fanboy.

By the way, that was an incredible interview - by far the best I have seen online at least considering the status of the interviewed.

All right, there are ways you can break down a player’s performance, his statistics, what he does, and come up with a value on how much he adds offensively vs. an average player or a player who is readily available. You can break that down and say he’s worth this many runs per season. A great player would be worth, say, 40 runs more than an average player. It’s about 10 run equals a win, so you say that player vs. a guy that we can call up from Triple A is going to win four more games. Significantly you could do the same thing with his base running and with his defense.

Is this VORP?

They are all part of the same family. We have our own proprietary stuff that we use, but it’s all part of the same family. It’s just a matter of how well you adjust for context and how accurate that is. You can do the same thing for defense and base running. Defense you can have the same type of swing. You could have a two or three win player defensively who is worth three wins more than an average defender. I think the important thing to do when you are looking at how valuable a player is. You say you like good players and don’t like bad players, the good players are the ones who contribute a lot of runs above what an average player or replacement could do offensively and defensively and run the bases OK. When you look at the total package some of that is subtle. You don’t come away from the ball park every day saying, “You know what we really won that game because the center fielder got a good break on that ball in the gap and caught it without leaving his feet.” But you come away saying, “Hey this guy hit a game-winning three run homer in the ninth inning.” But those two plays might be just as significant to whether you win the game or not. It’s a well rounded player who does everything well that we are looking for.

I will bet you 100 internet dollars that Epstein uses something very close to Tango’s WAR framework in evaluating players.

Not that i know anything about what he’s working with, but it is logically *possible* that Epstein’s shooting straight here. The thing is, we don’t know exactly what data they’ve got at their disposal. We all know there are shortcomings to the data inputs for these defensive metrics that may be avoided with some proprietary system Boston’s got going. You know, perhaps the balls hit to Ellsbury were somewhat more difficult than average balls hit to those buckets and made him come out worse than his reality.

And also, it’s not like he said UZR is worthless. He said that UZR’s mark for Ellsbury isn’t worth anything, and it is possible that the metric got a sharply negative realization of its volatility. In fact, based on Brian’s data for the last few years, we’d probably lean towards that conclusion anyway.

Tango,

I had a question sort of related to this about the fans scouting reports. I know you put the disclaimer at the end of your request to the fans, but how do you get around the fact that most of the surveys are filled out by sabermetric fans who are conscious of a player’s UZR/TZ/Plus/Minus numbers, even if they try to not think about them?

I think if we look at a correlation between declining/increasing UZR totals and the FSR, I would find it more likely that the average saber-fan just puts what they know statistically rather than that they are good scouts. I just find it hard to believe that fans can forget about their statistical predispositions and assign values based on their eyes alone, primarily because it’s human to do so, but more importantly because out of all baseball fans, the group mostly doing the survey is filled with people who know that their eyes lie but they number don’t.

A suggestion: ask the readers at the beginning of the survey to fill in, on a scale of 1-10 (with possible half-point markers) how strongly they believe in advanced defensive metrics and/or sabermetrics. I would love to see how saber and non-saber fans feel about the same players.

PAt,that’s a fair enough disclaimer.

However, how do you account for rookies in the pre-FAngraphs days?  Fans were ALL OVER Ryan Zimmerman, far earlier than the fielding metrics.  Longoria ranked high by the fans.  Gutierrez was very high by the fans.

Basically, you look at all the rookies that ranked high (Andrus, Escobar), and you look at their UZR over the next 3 years, and the FAns likely did as good, or better, than their rookie UZR would predict.

When I read Epstein’s statement, I immediately thought of the context nature of uing UZR appropriately, namely that it’s best to use ~3 yrs. worth of data when trying to get a good estimate of a players true defensive talent.  So in that sense, 1 year of UZR that says Ellsbury is -18 or whatever, could in fact be called “meaningless”, as Epstein goes on to imply that his true talent level in CF is above average.

FanGraphs may have UZR, but THESE are the stats that bring Theo joy.

On top of the other comments about how Theo has zero motivation to say something bad about Ellsbury, I don’t think his comment is necessarily damning for UZR. Theo just said the number attached to Ellsbury is bad. He didn’t say UZR on the whole is worthless.

I guess I will have to repeat what I have said several times on this blog:  A UZR number, especially a small sample, does NOT necessarily tell you what a player has done defensively.  Measurement error, measurement error, measurement error!

Let’s say you have a 3-year sample of UZR and the total is +5 per 150.  You might estimate that the player is a true +3 or +4 defender.

And let’s say that his yearly values (per 150) are +10, -9, +16.  Does that mean that he actually played terribly in the second year, which is possible?  No, no and no!  It means that he likely played worse than his true talent (but NOT necessarily) AND there was substantial measurement error in UZR, such that he played better than a -9 defender.

Take it to the extreme and take a 3-year UZR and break it up into 2500 individual plays (or however many plays there were) and record the UZR for each play.  Do you think that each “one play” UZR tells you how he actually played on that play?  Of course not. Play one could have been a ball at the right edge of a zone while the player was shaded over to the left.  He had zero chance of making the play, but UZR thinks that he should have had a 20% chance of making the play.  Etc.

What about if we break that 3-year UZR into 1259 segments of 2 plays each? Same thing. Lots of those pieces will have UZR’s that completely misrepresent what actually happened.

By the time we get to one year pieces, sure, there will be a much greater chance that the UZR pretty much represents what happened, but it will NEVER represent exactly what happened for any sample size, and there is a reasonable chance that it doesn’t represent even approximately what happened even for one year’s worth of data.

Remember that there are two sources of error in a sample of UZR (or any metric) that are completely different and independent entities:  One, the difference between what actually happened and what the metric “thinks” happened (like the previous example, or when a so-called easy ball is hit to a fielder and it takes a bad hop).  And two, the difference between how the fielder actually played (good, bad, medium, etc,) for that one sample and his “true talent” (how he would play over an infinitely large sample assuming that his true talent was around the same for the entire sample).

Anyway, I agree that what Epstein says about Ellsbury in public means nothing.  On the other hand, if it is your player and you are an extreme sabermetric team, especially one like the Red Sox who spend a lot of time and resources on evaluating defense (I assume), there are many ways to add to and improve upon a “blind” metric like UZR, plus minus, Total Zone, etc.  Many ways.  For example, for any sample of a metric like UZR, you can clean up lots of the measurement error that I was talking about, just by watching each and every play.  Plus you can watch video add all the stuff that UZR does not capture, like all of the extra things that Dewan does - the great and bad fielding plays - that is not captured in his plus minus (or UZR), like robbing home runs, bobbling balls that are hits, missing the cutoff man, throwing to the wrong base, etc.

Catching up now that I’m back from my sleep -

I have previously mentioned that Ellsbury’s decline was matched by an increase in Bay’s rating to almost average. I’ve had a nagging suspicion the two may be related, but haven’t been able to test it yet.

I am limited by my data source, which is Gameday. They record which fielder retrieved the ball, but I want to know which fielder had the best chance of turning the play into an out, and they are not always the same. If I watched video of every play, I could make that call myself, and know if Ellsbury was retrieving a disproportionate share of Bay’s missed attempts. I am sure that Epstein has somebody watching Red Sox video. Does he have someone watching everyone’s video, to get the same baseline for every team? Maybe. Would take about 4 hours each day to watch the condensed version of these previous day’s games. An intern or two could do it.

Brian- Where in the Gameday data is the fielder stored? It doesn’t seem to be in the inning XML files.

Wells, it’s not.

The starting nine are in the batters file (box score). Then the defensive changes are in the action file. From there link them to each PA (from atbats) by gamename and eventnum.

In my BZM fielding metric I calculate runs on plays three different ways.  My “Generic” calculation is similar to what UZR and Plus/Minus use, a generic value of around .8 runs for an outfielder making a play.  I also have a “Linear Weights” calculation which uses the average linear weight of a single, extra base hit, and out for the particular part of the playing field where the play occurs.  The third calculation uses the actual Run Value Added (the difference between the run expectations of the prior baseout state and the after play baseout state.  For outfielders this is the number that includes some of the fielders arm value. 

The interesting thing about Ellsbury’s 2009 numbers are that his Generic runs are 10 runs below average, his linear weights are 6.6 runs below average, and his RVA is only .6 runs below average.  There is a lot of luck involved in a single year’s RVA, but usually not 9.4 runs worth.  I am not surprised that the other metrics that use generic run values have him ranked so low.

As Brian mentions in post #15 Gameday’s method of showing where the ball was fielded instead of where it landed poses a problem for BZM as well as his metric and TotalZone.  I think a fast outfielder in Fenway would suffer the most from this problem.  In most parks the outfielders are able to field most hits before they contact the outfield wall.  The angle of a hit fielded before it hits the wall will vary a little from where it landed to where it was fielded, but not a lot.  In Fenway when a ball hits the outfield wall it almost always rebound toward the centerfielder because of the angle of the wall.  The faster the centerfielder is the more likely that he will be in a better position than the left fielder to field those balls.  Having a left fielder that has below average range increases the number of balls that hit the wall as well.  So again, I am not surprised that the Red Sox have a proprietary defensive metric that controls for these factors and has Ellsbury as a plus fielder even for 2009.