THE BOOK--Playing The Percentages In Baseball

I added a blurb about the Gold Glove at the end.

Well, instead of the misleading title “Fielding Bible Awards”, when it’s only a poll, I would have preferred the actual results of the Dewan plus/minus system.

Gary Matthews is a “youngster”? He’s 32.

Ouch, I thoughts he’s only been in the league 3 or 4 years.  And, being the Jr of Gary Matthews, who I grew up with, I was sure he was in the Time Raines Jr age class.  I didn’t realize he is an old journeyman.

As for the results, he’s got the top 10 for 2006, and 2004-2006 in the BJ Handbook, and I think studes posted somewhere that he was going to have the complete results?

I meant “grew up watching”.

When you say they were “given the BIS data,” what do you mean?  I find that odd.  I also agree that the title was misleading and I certainly expected the leaders in the plus/minus.

We were given the MLB2006 version of Dewan’s Fielding Bible book (updated through mid-Sep), which included the top 10 for 2006 and for 2004-2006.

Well that is really odd.  They told who who was the best according to his plus/minus system and then they asked you to vote on whom you “thought” was the best?  What kind of vote is that?  What was your vote supposed to be based on?  The data from the book? Your observations?  I don’t understnad the point at all.  Were you instructed to consider the plus/minus data?  Ignore it?

The intent was to be helpful and provide data, and all were instructed to use the data or not, as they saw fit. 

In one sense, it’s a good thing, since Dewan provided something akin to this:
http://sportsillustrated.cnn.com/baseball/mlb/stats/2006/fielding/ml_6_byZONE_RATING.html
but better.  And for multiple years.  The booklet was something like 70+ pages, so it’s not like it was a cheat sheet.  It was comprehensive data.

As a scientist, I don’t like it, because it introduces a bias that could have been avoided.

But practically, Dewan could have just presented the link above instead, to the same effect.  So, in the end, Dewan distributing the booklet probably had more pluses than minuses.

The point I am trying to make is that who the heck knows how much stock each person put in the data.  Some people probably didn’t even read it.  Some people probably don’t understand a PBP defensive metric.  Some people watch a lot of games.  Some people probably don’t.

I am mystified as to the point of such a vote.  If he didn’t give you any data then he could say that this is a vote based upon the opinions and observations of some so-called experts (although, what good is being an “expert” if you don’t watch or at least read about all the players in the league - and who can have an opinion on every player that means anything at all when it is impossible to closely watch more than a couple of players anyway?).

By giving you the data I have no idea what the vote is supposed to represent.  If it were me, and I watch a ton of games, I would simply rate the players according to their plus/minus ratings.  I have no confidence in my ability to figure out someone’s true defensive worth by watching them play, unless perhaps I were focusing on one or two players and taking notes throughout the season.

I completely diagree that this vote has any significance whatsoever.  I think it is a joke.

I’m surprised by your conclusions, since it’s not supported by the data.

For example, I ran a correlation one panelist to another (45 such correlations), and the average was 0.34, which is pretty much what we’d expect any two intelligent people/systems to correlate, if they were independent.  The average correlation to the BIS video scouts and to Dewan was 0.41, which is a bit higher than we’d like to see.  The problem is we can’t tell if it’s because BIS and Dewan “know” more, or if the data presented influenced the vote.  However, the effect of this would be rather small. 

I mean, Dewan and the BIS video scouts themselves only correlated at 0.33, which is what we’d expect if they were independent.

One could create their own overall score based only on those panelists that didn’t correlate high with Dewan and the BIS video scouts.

An interesting case is Nate Birtwell, who works in the BIS operations.  He correlated at .56 with the BIS video scouts, but only .28 with Dewan.  Neyer correlated .35 with the BIS video scouts, but .63 with Dewan.

If you wanted the most “independent” group, you’d probably want Bill James, Dewan, BIS Video scouts, Richman, Murphy, and the Fans.  My guess is that the overall results would probably be similar to the 10 panelists results.

In fact, let me do that now.  The top 2 1B stick, but reversed.  Top 2 2B stick.  Top 2 3B stick, but reversed.  Top 7 SS stick.  Top 10 LF stick.  Top 2 CF stick.  Top 3 RF stick.  Top 2 C stick.

So, I think mgl’s complaint is more of a theoretical problem, and in all practical purposes, the voting itself is legitimate.

I see that Dewan has forwarded a link to this blog entry to the panelists, so perhaps one of them will post, and describe their process.

I am really trying (and failing) to get a handle on what this “poll” is attempting to represent.  “Experts” at what?  As I said in my last post, how would these “experts” know who was the best defensive player at each position without looking at the data, regurgitating (and processing I suppose) scouting and other information they may have heard or read, or some such thing?  Surely they have not watched in agonizing detail every player on every team, or even close to that.  As I also said, I watch as many minutes of baseball as anyone alive (who has somewhat of a life) and there are scores of players whom I barely even “know” let alone how good they are on defense.  So what “expertise” is this poll supposed to represent?  And why should we care (and believe) who James, Neyer, Tango, et al. thinks is the best defensive players at each position?

At least with the gold gloves, the coaches and managers presumably see these players a lot more frequently and have a lot more practical “expertise” then these so-called experts.

Again, I fail to see any significance to this poll.

The significance of the fan ratings is that there is tremendous observational (and other) wisdom among these fans who admittedly watch almost every game a player that they submit plays (I think).  We can combine this with UZR or some other PBP metric, especially for players with limited playing time, to get a pretty good idea of a player’s true defensive worth.  With these guys, the panel of experts, it is a completely different story.  I would not trust their opinions any more than I would trust the opinions of any 10 (or however many there were on the panel) fans, and even less since it is preposterous to think that a bunch of guys with real jobs and real other interests and families could possibly watch enough games to get even an iota of a handle on how good EVERY player in baseball is.

Even if this panel gets a lot of the votes “right” it is because they constantly read and hear about who the best defensive players are and because they have access to the PBP defensive data and metrics.

Tango, just give me some reasons for why you think this poll is so significant or interesting without all the correlation numbers (which mean almost nothing to be, BTW, and I suspect the standard error is so high that the differences you cite are meaningless as well, but I could be wrong about that) and maybe I can comment about that.  Right now, we are talking around each other.

I can’t vouch for all the panelists, or even any of them.  The BIS video scouts, as I understand, do in fact watch every single play (multiple times) for every game.  That *is* their job!  So, they are probably outside your argument.  If you remember in last year’s Fielding Bible, they pulled out the top 30 and bottom 30 plays for Everett and Jeter.  Presumably, they did a UZR-style analysis on a PBP, and simply logged for each play, where these guys were +.50 or better on a particular play, or -.90 or worse on a particular play (or something like that).  I’m sure I’d be impressed with their logging system.

Dewan has something UZR-like, and James probably has his own number-crunching system.  Richman is Strat’s founder, and, again, it’s his job to watch every day.

I can’t speak for the others. 

And, like I said, even if you take just these 5 or so, you still end up with around the same answer.  Given a choice between these 10, and the 50 coaches in each league, these 10 are probably better.  Certainly, no Jeter-like pick was made.

I mean, Dewan and the BIS video scouts themselves only correlated at 0.33, which is what we’d expect if they were independent.

How do you know that the 0.33 correlation is what we’d expect if they were independent?

Re-read my blog entry, where I discuss that .35 is the correlation we expect of two independent systems.

I have no doubt that any 10 sabermetric-oriented “experts” most of whom watch a lot of games, some of whom watch ALL the games, or even 10 “super-fans” will do A LOT “better” than 30 managers or coaches.  So what?

I still don’t get the significance of this particular poll or why it is called the “Fielding Bible Awards.”

After reading Neyer’s Insider column today, I finally have a feel for what these awards are supposed to mean.  Maybe I was just being incredibly thick and obstinate for a couple of days.

Anyway, they were supposed to mimic (I guess) the GG awards except they were voted on by 10 persons who analyze baseball for a living, most with a decidedly sabermetric slant.  As well, all of the voters were provided with some objective relevant data to inform them if they so desire.

O.K., I got it.  I’m still not sure I care much about the results or about the correlations that Tango came up with.  And I think I would have preferred to see the results sans the data being available and then the results of the data alongside the data-free vote, or something like that.  But I am not so sure about that either.