from Amazon
Tuesday, September 20, 2011
UZR yapfest
Fangraphs posted an article on UZR, which they subsequently pulled, due to inaccuracies in statements. That led to a post explaining why they did it. Anyway, I put in my two cents (which I’ll repost below the fold). And there’s also alot of UZR discussion that is now taking place at this new thread. Some good stuff, if somewhat disjointed.
My UZR contributions:
***
“- not assessing the starting position of a player”
UZR takes the position that the starting spot of a player is a skill. And it assigns it to the player.
You can agree or disagree. But, it’s not a bias. It’s a feature, insofar as UZR sees it.
And yes, there are random biases (which are solved by larger sample size) and systematic biases (which are exacerbated by larger sample size) to recording data.
There’s also random and systematic biases to the “eye test”.
Pick your poison.
***
Otherwise, what’s the alternative? Your own personal smushing system? Is that somehow going to give you more confidence than fWAR?
And if you don’t like UZR, just replace it with whatever you’d use in your smushing system.
***
(The measurement error for those not following is that we don’t know how many opportunities Tulo actually has in any given year, nor the “quality” of those opportunities. While we can figure out on the batting side what kind of opps he has, facing Halladay, facing Lee, etc, we have a problem figuring out what kind of chance Tulo has on any ball in play. So, we ESTIMATE each and every single ball in play and say, “ok, that one had a 90% chance of making an out, and that one had a 20% chance”. But, we don’t REALLY know that it was 90%. It could have been 80% or 95%. It’s a huge difference. Unlike batter v pitcher where if you face Halladay, you know, plus or minus 1%, the chance of getting on base against Halladay. Measurement error, if random, is of course “solved” by sample size. If systematic, it’s actually made worse by sample size.)
***
My yapfest contribution:
“Unfortunately, Bradley’s peer review was done publicly after publication, and was obviously embarrassing. ”
This is actually a Fortunately, not Unfortunately.
I live and die on the informal public peer review process. At my blog, I’ve got access to thousands of readers, all willing to put up with style, so they can learn, and *I* can learn. I love those readers.
At Fangraphs, there’s ten (hundred?) times as many readers. The quick and fierce reaction to the author missing the glaring aspect of measurement error was fantastic. It actually stopped embarrassment before it could fester into something huge.
***
There were two things wrong in that thread:
1. As Dave noted, taking an “official” stand on something that has multiple and reasonable points of view is akin to taking a political stance. You’ve got a 60/40 decision to make, and the official stance cements it at 100/0.
For example, my “official” stance on fWAR and rWAR is to split the difference and call it a day. You can make reasonable arguments on either side, and instead of saying there’s only one right answer, I can just acknowledge that everyone has a valid point. Not to mention that there’s an uncertainty level to begin with anyway with any metric.
Anyway, Dave recognized the path that this series was going to go, and put a stop to it. Again, this is a good thing.
2. The author did not step in to stop the commenter runaway train by acknowledging that there was a glaring hole with the non-mention of the measurement error.
[Editor’s note: I pulled the measurement error paragraphs out of this section, and placed it above the fold. It fits better there.]
Not to make this about me, but, what the heck, since I’m here: on my blog, this rarely happens. That’s because I have way too much free time at the office, and so, I respond almost immediately to any comment. And before there’s a chance that a legitimate comment can then be misconstrued into something else which leads to misunderstanding or just plain trolling, I give my answer, and hopefully, we can move forward.
***
What happened with the comments was well-intentioned, and well-done at the beginning, but then it was simply slipping away into summary conclusions. Not that I necessarily blame the commenters, because they don’t have the time or inclination to swim in the deep end like I do. They want some clarity, but instead they got more confusion.
