from Amazon
Wednesday, June 08, 2011
Do PECOTA Percentiles make any sense? No!
No. Or, paraphrasing the immortal words of Colin, “PECOTA Percentiles are wrong”!
Ideally, I would just appeal to my own authority, as someone who has pointed out that when Nate did his percentile forecasts that they were wrong. I knew they were wrong, based simply on logic. Indeed, Colin tested it last year when he took over, and they were very wrong. I furthered his research, and came to an even stronger conclusion how the percentiles did not make any sense. You really don’t even need to test it empirically, because it fails the logic test. If it fails logic, then it won’t pass empirical testing (other than by pure luck).
But, I know some people don’t want to just drink the Tangotiger Kool-Aid, on the off-chance that I’ve spiked it with something. And so, I will present logically why PECOTA Percentiles are wrong.
(All data as of Mar 29, 2011.)
Giving Felix an 80% chance of an ERA between 2.36 and 2.85 is not a good range. Similar for Lincecum (2.37, 3.03).
Let’s say that your top flight starter has a .270 wOBA and 2.70 ERA. One standard deviation (SD), if we treat 1000 PA as a constant and .270 as his 100% true rate will give us .255 to .285 of wOBA, which translates to 2.40 and 3.02 ERA at the one SD level. That means if you have zero uncertainty of his true talent, and you are guaranteed 1000 PA, 68% of the time, we’d get a wOBA of .255 to .285 (which translates to 2.40 and 3.02). The 80% range would get us to 2.32 and 3.11. (This does not even account for differing performance with men on base and bases empty, so not even a real ERA but a component ERA.)
So, your starting point for a top-flight starter for the 80% range must be a range of 0.80 runs. And that’s with perfect knowledge and zero uncertainty.
Add to this the three uncertainties:
1. playing time
2. his true mean
3. performance with men on base v bases empty (like Doc and Lee last year)
And this should expand the 80% range to say at least 1.00 runs for a top-flight starter. PECOTA has Felix at 0.49.
I was going to do a Marcel Percentiles for pitchers (I did one for hitters several months ago), where I’d look at the results empirically. Not sure when I was going to do it, but maybe sooner rather than later? Who knows. I think we’ll see something like the 80% range be 1.00 to 1.50 for a top-flight starter once you account for all this, and much higher for a regular pitcher.
Recent comments
Older comments
Page 1 of 344 pages 1 2 3 > Last »Complete Archive – By Category
Complete Archive – By Date