from Amazon
Wednesday, November 24, 2010
Bill James speaks
Good interview.
If someone has studied the data and can demonstrate that our projections are over-optimistic, of course we’d look at it. If someone speculates that this is true, I’m not really too interested.
Intuitively, I doubt that that is true. Our projection for Jason Heyward last year was extremely accurate—a few points high on batting average, but an extremely good projection. For Buster Posey, we projected .270 with 11 homers, 54 RBI. He actually hit .305 with 18 homers, 67 RBI. We had projected Jose Tabata at .273. He hit .299. We had projected Tyler Colvin for 4 homers, 17 RBI; he had 20 homers and drove in 56. We had projected Michael Stanton for .228 with 9 homers, 22 RBI; he hit .259 with 22 homers and 59 RBI.
As part of the process of producing the Handbook, we look at every projection that we made the previous year, and compare it to what the player actually did. I study those charts every year, looking for any systematic problems. I would be surprised if anyone else actually looks at them as closely as I do after the fact, comparing what the hitters actually did to what we had projected for them, and I would be surprised if we were systematically optimistic on young hitters.
Other than playing time.
Last year, or two ago, I sent Bill something simple, on a league-wide basis: the runs scored per game on offense was quite different from the runs allowed per game on defense.
I never checked regarding biases by age, but if he uses the three best hitting rookies (Heyward, Posey, Stanton) as evidence that he is not optimistic by nailing their forecasts, then that’s evidence that he is too optimistic.
This is how a forecast works: you forecast a player’s true talent level, and presume that there is no good or bad luck involved. That is, you do NOT forecast luck. If, AFTER THE FACT, you select the three best hitting players of whatever population you want (rookies, outfielders, 35-and-over, whatever), then your forecast for them better have been LOWER than what they actually did. Because, as a group, any player that performed better than league average is more likely to have benefited from good luck than bad luck.
HOWEVER.... if you selected Heyward, Posey and Stanton because they were the three guys you had the best forecast on, that is, you selected them BEFORE THE FACT, then you would definitely want to nail their forecasts.
So, it all comes down to how you select your players to show if you are being optimistic or realistic. So, to test Bill James’ assertion, you need to rewind the clock to April 1, 2010, and select all the players that had, say, less than 100 career MLB plate appearances, and of those players, select the 10 guys that Bill James had the highest wOBA (or OPS or OPS+) for them. And THEN compare to what they actually did.
If it’s a match, then Bill James was being realistic.
This is another one for you aspiring saberists…