from Amazon
Saturday, August 20, 2011
The value of a metric is not in its consistency, but in how it correlates to what we want
Dave Berri provides a good example here with NFL. He shows the correlation of various metrics to each other, year to year. And the winner was success rate.
But, in the context of what he’s talking about, what does it matter? What we care about was who’s the best quarterback. And so, you need to correlate each metric in year 1 to the SINGLE metric in year 2. What is that single metric? Well, it could be NFL’s passer rating, it could be ESPN’s QB rating, it could be WPA. It could be alot of things. But, each metric in year 1 needs to be compared to the single metric in year 2.
Furthermore, the correlation is influenced by many things, not the least of which is opportunities. Put simply, the more opportunities, the higher the correlation.
It’s also influenced by the spread in talent as measured by that metric. For example, a hockey goalie has exactly one job: stop the puck. This is why their save percentages are so tightly bunched. It’s because they all have to be good to great at stopping shots. But, what about a nonpitcher in MLB? Well, they can have talent at hitting, running, fielding, in hitting for power. There’s many different skills that a nonpitcher can have to be good. This is why you see a spread in talent in each individual metric. And something like strikeouts has a very very wide range, precisely because we don’t really care about their skill set there: it’s not a requirement of being in MLB (to a point of course). That’s why K rates stabilize so quickly.
Anyway, all to say to just be careful in how you interpret correlations. They may not be saying what you think they are saying.
Recent comments
Older comments
Page 1 of 344 pages 1 2 3 > Last »Complete Archive – By Category
Complete Archive – By Date