Monday, January 09, 2012
What is evidence? Colin v Neyer
Rob and Colin are talking about evidence.
It sounds to me that Rob is REALLY talking about observational bias, not evidence. I tried finding a good definition for evidence. Kind of hard. At the minimum, it requires data or information or something that otherwise exists. It also requires an event to have occurred, or that relates to an entity’s property or behaviour.
So, evidence would require some sort of association of information to property.
What is the evidence that Edgar is more likely to have been a PED user than Jeter? That there were more PED users during Edgar’s time than Jeter’s? Well, that is an inference based on information. But, then we’re not talking about Edgar and Jeter specifically, but rather them as representative of a population. Rob is asking us to think of those two specifically.
We can list the 17,000 players in MLB history, and list their odds of having used PED from 0.00001 to .999999, and using nothing but Bayes and the Mitchell Report. Just because you use evidence and you apply Bayes doesn’t mean you have used evidence to learn anything about Edgar and Jeter SPECIFICALLY.
So, a third requirement in evidence would I think be that you can associate the information and event more directly to the specific entity being targetted.
Otherwise, we’re getting into an impractical philosophical discussion with no hope of having a resolution. That’s what drinking at a bar at 2 AM is for.


There’s evidence, and there’s PERMISSIBLE evidence.
Evidence is simply anything that changes your estimate.
Being mentioned in the Mitchell report is evidence, in the sense that, all things being equal, being mentioned increases the probability that player X took steroids.
That evidence may be unfair to use against player X in certain contexts, but it’s still evidence. Statistical evidence, or circumstantial evidence, or hearsay evidence ... but still evidence.
There are times when it’s unfair or rude to use statistical evidence as if it were specific evidence. I’ll bet you better than even money that, statistically, African-Americans do eat more collard greens per capita than whites; Jewish-Americans eat more Matzo balls per capita than non-Jews; and and Hispanic Americans eat more tacos per capita than non-Hispanics.
But if a black person I’ve never met comes over for dinner, and I make collard greens just on the basis of a (probably true) statistical stereotype ... that’s rude. And there are very obvious reasons it’s considered rude.
On the other hand, if I decide to send more Matzo Meal per capita to supermarkets in New York than supermarkets in New Orleans, that’s probably OK—and good business sense.
So, it’s not Evidence vs. Not Evidence. It’s Evidence that’s OK To Use In This Particular Context, versus Evidence That’s Not OK To Use In This Particular Context.