from Amazon
Sunday, February 05, 2012
Is Nate Silver alot more certain than he lets on?
I’ve been following Nate’s mean forecasts for the five primaries so far. So far, he’s made 25 predictions over those 5 primaries (and obviously, they are interdependent). His worst forecast result was Santorum in Iowa, where he gave him a mean forecast of 19.1, and he ended up at 24.6, for a difference of 5.5 points. His average error over those 25 forecasts is 2.34 points, with one standard deviation being 2.71 points.
However, his posted uncertainty level is much higher than that. Let’s take Mitt in Iowa as an example. He gave him a mean forecast of 24.5, with a range of 13 to 32 (a range of 19 points). In another article, he notes that his range is the 5th and 95th percentiles. Those levels are reached at the +/-1.645 standard deviations (or a range of 3.29 standard deviations). This means that one standard deviation for Romney is 5.8 points.
So, I calculated it for all 25 forecasts, and one standard deviation averaged 4.6 points as Nate’s uncertainty level. However, as I noted earlier, the actual observed standard deviation was 2.71 points. This means that Nate’s uncertainty level is 4.6/2.71 too wide, or 1.7 times too wide.
Now, either he made a calculation error of his historical data (making the width of his uncertainty level almost twice what it should have been), or this year, things simply worked out alot closer to the mean than expected, just by luck (after all, we only have 25 data points).
Here’s the data for those who want to take a crack at it:
Recent comments
Older comments
Page 1 of 344 pages 1 2 3 > Last »Complete Archive – By Category
Complete Archive – By Date