from Amazon
Friday, July 16, 2010
State of fielding
Colin is wondering exactly where we are:
I’m not really sure that we’ve gotten any further than where we were when Zone Rating and Defensive Average were proposed in the ‘80s. And if we have gotten further, I’m not sure how we would really tell.
...
Are we being objective about fielding analysis? In other words, do we know what we think we know?
...
So our metrics don’t do a very good job of agreeing. We don’t know which methods are “better,” only which ones we like more. And our data hasn’t been validated against some objective standard. To me, this opens up a simple question—how good are our defensive metrics? Are they useful? How useful? And if we go back to the beginning, where we talked about what sabermetrics is about, it doesn’t seem to me to be good or valid sabermetrics to accept these metrics without some sort of evidence, some objective facts that show they measure what we think they measure. And I think the burden of proof is on those who are making claims based upon these metrics to provide that evidence.
My response to him, and later readers:
I had already responded to Colin’s point regarding the comparison of correlation for offense and defense. I will reproduce here:
What is the relevance here? You are taking known hits, known extra base hits, and known outs, and you have one system that arranges it one way and another that arranges it another way. The correlation would have to be in the high r=.9x. With fielding systems, you are taking known outs (in some systems), estimated outs (in another), and estimated hits (for all systems), and trying to find the correlation.
***
Otherwise, I share Colin’s general skepticism of subjective data being treated as objective data. He fairly asks legitimate and nuanced questions regarding the advancement level of fielding stats.
However, I am bothered that a reader, after reading Colin’s piece, would come to a conclusion like:
“Seeing as we haven’t made much progress in 20 years with defensive metrics...”
The only fair conclusion to make is that we don’t know how much progress we have made, and not that we haven’t made much progress.
We’ve made “some”. Is that a little? A lot? You can’t say “not much”. This is part of the nuance in Colin’s piece that may be glossed over, if said reader is representative of a portion of the readership.
===
I guess this is part of the nuance. Colin said this:
“The short version: I’m not really sure that we’ve gotten any further than where we were when Zone Rating and Defensive Average were proposed in the ‘80s. And if we have gotten further, I’m not sure how we would really tell.”
So, he’s asking two questions:
“Have we gotten any further?”
“How can we tell if we have?”Rather than specifically making it questions, he’s wondering. But, he’s not concluding.
His actual conclusion was questions:
“To me, this opens up a simple question—how good are our defensive metrics? Are they useful? How useful?”And that’s where we are. We’re in the investigation stage. And in the noted thread, I said this:
“Room for improvement and discussion, as long as you start with the [recorded, not estimated] data. ”