from Amazon
Wednesday, September 08, 2010
Secret Sauce?
I’ve never been a fan of BPro’s Secret Sauce. First, it’s hard to test it, if the formula is based on the universe of data we are interested in. There’s no expectation that future years will match the regression.
Secondly, given that there are only three components to the metric, and one of the components is relief performance, and an abnormally disproportionate share of the playoff relief performance has Mariano Rivera in it, and Mariano Rivera has the most extreme performance of any post-season pitcher ever, what the secret sauce could just as well say is: fielding, starting pitcher K rates, and Playoff-Level Mariano Rivera.
And, lo and behold, now that the Yankees aren’t powerhouses since the Secret Sauce was launched, it looks like it doesn’t work any more.
This reminds me alot of what Bill James did with his secret sauce 25 years ago, in trying to figure out which team would be favored to win in the playoffs. Just because you can fit a metric to the data, doesn’t mean that the out-of-sample data will be similarly fitted.
Logic should trump rationalization. And all these secret sauce metrics are rationalizations. Show me otherwise.