THE BOOK--Playing The Percentages In Baseball

Saturday, February 05, 2011

Forecasting the best rookie pitchers in the last 10 years

Brian was kind enough to run a rookie report for me (us now) using his latest forecasting system.  I asked for his assent to publish these findings, and he has granted it.

***

Quoting his selection criteria:

I queried the park adjusted projections from 2001-2011
using current Oliver formula
where player had rookie eligibility
and sample size (weighted past pa’s) >=600
and projected era <4.00
and age <27

I looked at his top 11 pitchers and how they played in their rookie MLB season.  (I discarded anyone that never made it to The Show, or had just a few innings.)

The idea that I have is that his forecast for the top 11 pitchers should match their actual ERA, as a group.  For example, I did this a few years ago:

There were 14 pitchers forecast with between 140 and 160 strikeouts. Let’s go to the tape.

Player Strikeouts
Jake Peavy 216
Carlos Zambrano 202
Javier Vazquez 192
Mark Prior 188
Brandon Webb 172
Barry Zito 171
Josh Beckett 166
Livan Hernandez 147
Mark Clement 146
Freddy Garcia 146
Rich Harden 121
Ted Lilly 96
Kerry Wood 77
Kelvim Escobar 63

Average forecast? 151. Actual? 150.

That’s the basic idea in forecasting that you should nail the group, even though you will have lots of variation within the group.  That’s why they play the game on the field.

Anyway, back to Brian’s list.  Among his top 11 are names… big names:
Hughes, Phil
Strasburg, Stephen
Beckett, Josh
Bumgarner, Madison
Prior, Mark

As a group, those 5 were forecasted at 3.08 (implied 74% of league average).  They actually performed at 3.56, with a runs allowed rate of 85% of league average.  As we can see, the forecast was a bit optimistic. 

But wait, we also have six other names… small names:
Slowey, Kevin
James, Chuck
Petit, Yusmeiro
Burnett, Sean
Stephens, John
Williams, Jerome

They were forecasted at 3.39 (implied 77% of league average).  They actually performed at 4.58, with a runs allowed rate of 104% of league average.  Forecasts were way too optimistic.

As a group, these 11:
forecasted = 3.25 (implied 76% of league average)
performed = 4.12 (96% of league average)

Hence, we can say that Brian’s forecasts are pretty optimistic.  The forecasted ERA needs to be increased by 27%. 

Hughes and Strasburg were the runaway leaders in best forecast for rookies at 2.78 and 2.79 respectively.  In my quick approach here, their forecast should have been 3.54.  Strasburg would have killed that forecast with his 2.91 ERA, but Hughes would have been decimated still at 4.46 (average for the two: 3.69).

But maybe going 11 deep is too much.  If I went 5 deep instead:
forecasted = 3.05 (implied 72% of league average)
performed = 3.84 (91% of league average)

Based on this data point, Brian’s forecasts are still optimistic.  We need to increase his forecast by 26%.

But, here’s something interesting.  What if we think there’s some adjustment period in MLB?  What if rather than looking at the forecast for their rookie year, we look at their best season?  For these 11 pitchers, that’s an average of 80% of league average.  For the top 5, it’s 76%.  These numbers are pretty close to what Brian forecasted!  (But, of course, I picked out their best seasons, and hence, that means those seasons are more good luck than bad luck, and hence, better than their actual true talent level.)

***

Here is Brian’s data, with the last three columns added by me via BR.com

Read More