THE BOOK--Playing The Percentages In Baseball

When Dan and I were discussing this back-channel, my concern with the Teahens of the world was that people were using a category bias and rating “right fielder” rather than “Mark Teahen.” If I told you that Larry lives in Arkansas or Oregon or Massachusetts or Florida, you’re opinion of him is likely to be tinged by your knowledge of him belonging to a category of (insert state here).

Oddly enough, Tom, you might be inadvertently causing the very effect that Dan has pointed out.  The instructions to *not* think about position sounds a lot like the old “Don’t think about a pink elephant.” What are you thinking about?  A pink elephant, of course.  It’s called a cognitive rebound effect.  In undergrad, my research methods class actually did a study on this very thing.

I did a WOWY fielding analysis of RetroSheet data. Later I will rerun on GameDay data, when I have it available, which will add vector and sharp/soft classifications.

Bruan was bad at fielding fly balls, allowing 77 when 50 were expected, average at linedrives, allowing 165, 166 expected (lower variance between fielders on LDs).

However, he was better than average at keeping the batters who got hits from getting extra bases.

Fly balls: 77 total, -4.5 DO, +0.5 TR
Ground Balls: 85 total, -8 DO, +1 TR
Line drives: 165 total, -11.6 DO, +2 TR

All: 327 total, -24.1 DO, +3.5 DO

I don’t deny that there may be a positional bias.

However, if I do not put that in the instructions, I will definitely get the evaluations relative to position. 

So, I don’t think your point applies in this specific issue (it may apply elsewhere).  Indeed, many people add comments, and they would add things like “I tried hard not to think about position like you asked”.  In other cases, people would blatantly ignore the request and say “I evaluated so-and-so as a 2B”.

The psych research says that you’d actually be better off without the warning, or at least not making it so up-front.  In studies of racist behavior, people who are expressly told not to think about their stereotypes about group X actually behave in ways that suggest that they are thinking about their stereotypes of Group X.  Yet, they swear up and down that they are actively suppressing those thoughts.  I’ve seen this happen with my own eyes.

You might be better off putting the “don’t think about positions” into paragraph format near the beginning of the instructions and then talking about the rest of the instructions to try to let the rebound effect wear off.  You’ll never rid yourself of it completely, but that would probably be the best way to minimize it.

I will have to disagree.

Let’s take different ranges:
1. I make no mention of it at all
2. I make a passing reference that most internet readers won’t even notice
3. I talk about it a bit
4. I hit them over the head with it

What will happen?

Well, in situation 1, since I don’t mention it, a baseball fan will implicitly presume I am asking him to rate the fielders relative to position.  I am 100% convinced that the majority of the balloters will do this.  A heavy majority.  There’s no question about this.

Does this follow with the racist tests?  Probably not, which is why I don’t buy into the analogy here.

In situation 2, it will look almost like situation 1.

In situation 3, maybe half will heed the warning and half won’t.

In situation 4, 80% of the people will listen to me, and 20% won’t.

I don’t think there’s a scenario I can construct where I can overemphasize too much, simply because the fan almost never thinks of rating fielders irrespective of position.  It is such a foreign concept to them, that it is critical that I hit them over the head with it.

If this were a professional scout, who already know NOT to compare against a position, then you WOULD have a point.  In this case, if you overemphasize the non-position, he might start to think of it in position-terms.

So, you have to first be aware of what the target audience’s proclivities are before we can say the effect of how the “hitting over the head” instructions will bias the answers.

The research says that the suppression rebound effect (don’t think about his position!) is actually stronger than what a control condition (not saying anything about positional bias) would be.  The problem is that because you’ve introduced a social desirability effect into the mix (you want to please the person who is saying, don’t think about position), so you say that you are doing it.  The problem is that people are not very aware of what they are thinking and how it affects what they are doing.  So, they will actually be more likely to rate by position and less likely to realize that they are doing it.

Another method, which may seem counter-intuitive, is to talk about how “there are a lot of things that we impute to a player just based on what position he plays.  RFs are asumed to have good arms.  SS are assumed to be quick, lithe, and nimble.  Consider how often you make these same mistakes.” Then talk about how you want it to be position-free.  The research on the topic says that this approach, in which you point out the problem and bring it fully to mind rather than forbidding people to think about it, is better.  The data show that this is the way to go if you want people to suppress a categorical effect, which is all racism really is.  I agree it’s counter-intuitive, but not everything that makes sense is true, and not everything that is true makes sense.

I agree with your second paragraph.

Thanks for the linkage...seeing this post just made my day. And good point on Ryan Braun being a better example. I was going to use Kelley Johnson as well, but he last played outfield prior to having major surgery, so I didn’t think it was appropriate.

I think fans read the question as a comparison to other players at the same position. I would compare Scott Rolen to other 3rd basemen when making a choice on the survey.
Also, I have heard that how you throw the ball when you are in the infield and outfield is different (at the major league level). Not everyone is capable of throwing both ways. I can’t say from experience because my throwing sucks big time!

Peter made a good point in the comments section of the original post about throwing being different in the infield and the outfield. While this is true, I don’t think it explains the huge difference in the ratings.

I think the fans see the margin for error for an infielder being smaller than it is for an outfielder, and that’s where my problem arises. A throw from an outfielder that misses by 5 feet is no big deal, but a throw from a 3B that misses by the same margin is often an error. So they miss by the same amount, but because of the difference in position, they receive different “grades.” Obviously you try to eliminate this as much as you can, but I don’t know if you can really fix it.