from Amazon
Thursday, February 10, 2011
Yet another article that “samples” is not “true”
Carson does something that I’ve been hoping to see someone do for the longest time. What he does is figure out the surplus value of a player drafted, and link that surplus to the scout who signed him. So, if you signed Tulowitzki, you look like a genius. But this is a sample. It still is not something true, something real.
After all, I presume all thirty teams had Tulo ranked somewhere between #1 and #10. How much does it help us to give the 50MM$ surplus (or whatever it was) to the Rockies’ scout who drafted him #7, and 0$ surplus to the other 29 scouts who also ranked him quite high? In that same draft, Alex Gordon was drafted #2. I don’t follow the college scene, but presumably all the teams had him ranked pretty high as well. Do we slap down only the Royals for drafting him, because they are the ones that bought the Alex Gordon lottery ticket, even though the other 29 teams ALSO wanted to buy that ticket?
If you look at it as a sample, then you can say, yeah, the scout won the lottery ticket, that’s the money he made. But, you have to look at it from a true talent perspective. Perhaps you need to regress 99% of what you see from Carson’s process. Scout Bill Buck is credited with 74MM$ in surplus. Perhaps his true value is 740,000$, and the rest was his good fortune for having exclusive dibs on players.
The problem with the process is the exclusivity of it, the binary outcome: did he, or did he not draft that player. In many respects, it’s like looking at a player’s single game, where he puts the ball in play 4 times, and we’re trying to figure out what the true talent of the player is based on the observations of these binary outcome based on whether the batter was safe or out. Because, in this case, you would also regress what the batter did in 4 contacted PA 99% toward the league mean. On the other hand, if you had say his launch parameters (how hard he hit the ball, the spray and vertical angles, the spin imparted), then those 4 PA might regress 95% toward the league mean.
Today is sample-is-not-true day!