THE BOOK--Playing The Percentages In Baseball

Wouldn’t the prior probability simply be the baseline PED rate of MLB sluggers, and not yet consider the personal qualities of Braun?

Well, it depends on how many players you think are actually juicing (at least, juicing with stuff that can be detected). Very few MLB players get his by this. Some off the cuff calculations show that if say, 6 players were juicing for real then the chances of a false positive for Braun is somewhere around 10%.

You probably also have to consider how many guys use PEDs but are off cycle, deliberately or otherwise, when tested. No idea what that number is.

I fail to see the connection with Bayes. Bayes says
P(cheating|positive test) = P(positive test|cheating)P(cheating)/P(positive test)
which is a great too if we don’t know the posterior probability, but in this case we do know it. So why does it matter? Because the conditional probability is more informative than the marginal?

If you stick some not too unreasonable parameters into Bayes as follows:

50% Probability of positive test given cheating
10% MLB cheat
1% false postive rate

I get probability of Braun cheating of 85%.  Obviously the key is the 1% false positive.  I find it hard to believe that the tests are 99% accurate (plus that the doctors are 99% sure that the testostorone didnt generate naturally in the body) but thats just me.

I still think he is guilty on balance of probabilities though and if pushed I think I would be 80-90% certain.

What I found online was a 7.5% false positive rate. Not sure how accurate that is, but it’s definitely higher than 1%.

I have to believe the false positive rate depends on what you’re testing for - so different drugs have different false positive rates.

Also, if this is at the stage where Braun is appealing a suspension, then both his A and B samples tested positive. The sample is given once, split into two parts, and the A part is tested. If that’s positive, then they test the B part. Only then is a test considered positive. So this reduces the likelihood that operator error led to a false positive, since you’d need that operator error on both samples. Now some of the false positive rate in testing may come from other factors which don’t change between an A and B test, so it’s still possible to have a false positive, but the process is designed to lessen the likelihood of these errors.

And it’s also biased in the player’s favor, in that if the A sample comes back negative, they don’t bother to test the B sample, and the player is presumed clean. Thus the player gets the benefit of any false negative results.

If that were the case we’d be getting upwards of 50 false positives a year, and I think the players would be (rightfully) up in arms about it.

Your post is confusingly worded.

We have a posterior probability that he took a banned substance, which is the 1 minus the probability of a false positive or some other sort of “mistake.”

No, you do not have the posterior probability. The posterior probability is P(cheated|positive test). You are trying to derive that given that you know that:

1) cheated = yes
2) positive = yes

Then, as Perceptron alluded to in post #4, you can use P(positive|cheated) to derive the posterior probability P(cheated|positive). P(positive|cheated) is the complement of the false-positives rate, which can be interpreted as the accuracy of the test.

Oops, I made a typo. What I meant to say in point #1 was:

1) P(cheated) = known

not

1) cheated = yes

P(cheated) is the prior belief that Braun cheated, which we have as a subjective prior.

If you stick some not too unreasonable parameters into Bayes as follows:

50% Probability of positive test given cheating
10% MLB cheat
1% false postive rate

Do some more math and tell me how many positive tests, overall, you would have seen in 2011 if these assumptions were true, given that Ryan Braun is not the only player who gets tested… then compare that to how many positive tests were reported in 2011…

gogurt, right, forget I said that.

I think the false positive is much lower than 1%, given the A/B protocol.  And I suppose it would depend on what the specific B test is given that A was positive for something.  As Greg said, it would have to be much lower than 1%, otherwise there would be lots of players coming up positive.

From what I understand, the first test is relatively cheap and inaccurate, with a fairly high false positive rate (which all makes sense).  If the first test comes back positive, then they do a second test, which is more expensive, more accurate, and with a much smaller false positive (and I assume false negative) rate. Again, that makes sense.

So if the first test had a 5% FP rate, and the second one a .5% rate, both numbers being reasonable I think, although completely made up, then the overall FP rate is 1 in 4000.  Even then, that would probably lead to too many players being banned who did not take anything.

And don’t forget that there are other ways to get a “false positive” other than by a clinical “false positive”. Contamination, someone framing someone.  Mixing up test samples.  Etc.

Also, if a player ingested something with no intent and no negligence, and that triggered a positive result, isn’t that the same thing as a false positive? Even if the chances of that are very low, that can change the overall numbers quite a bit.

I don’t understand why people make the argument that (1) MLB has to win every arbitration case in order to maintain credibility and (2) MLB can claim they are clamping down hard on PED usage.

A lower percentage of “dirty” players in the leauge means higher odds a positive test comes from a “clean” player.  The cleaner the player pool is, the more likely a positive test came from a clean source.

Keeping the 50% False Negative and 1% False positive ratios, the probability of a positive test coming from a “dirty” player given the test was positive ...
1% dirty population = 33.6% probability
5% dirty population = 72.5%
10% dirty population = 84.7%
25% dirty population = 94.3%
50% dirty population = 98.0%

Updating for the assumptions in MGL/12

Assuming the 50% False Negative and 5% False positive ratios for the first test, then 10% False Negative and 0.5% False positive ratios for the second test (completely independent of the first test errors, which is some hand waving) then the probability of a positive test coming from a “dirty” player given the test was positive ...
1% dirty population = 94.8% probability
5% dirty population = 99.0%
10% dirty population = 99.5%
25% dirty population = 99.8%
50% dirty population = 99.9%

I don’t follow your math.

1) Assume false positives rate of 1% = P(positive|no cheating) = 0.01

2) Assume false negatives rate of 50% = P(negative|cheating) = 0.50

3) 10% dirty population = P(cheating) = 0.10

4) P(positive) = 0.108 by the law of total probability

P(cheating|positive) = [P(positive|cheating)*P(cheating)]/P(positive) = 0.463

-Jimmy

Next we conditionalize on a subsequent negative test. We also conditionalize on a long history of not-failed tests.

What we should believe only goes as far as we assess our procedure as rational AND our inputs (information) as complete and accurate.

If anything bothers me about the current discussion of Braun--by the way, I tak the present post just to be pointing out good evidential procedures, not to be among those discussions that bother me--is that for the most part, Braun is on trial in the media that doesn’t really have the evidence, just scattered reports about it. Assuming he’s being honest, his reputation will be tarnished anyway. Assuming he’s dishonest, if this were a real court, he could at least demand a mistrial.

You know, life ain’t always fair. Sometimes shizit happens that is not in your control and you suffer for it. In this case, in return for being able to make tens and sometimes hundreds of million dollars you have a small chance of being accused of something you didn’t do and consequently having your image tarnished. How many of you would turn that down going in?

I ask again, how many of you would turn this down:

You have the opportunity to play major league baseball and make millions perhaps hundreds of millions of dollars. In exchange, there is a very small chance, maybe 1 in 1000 or less that you will be falsely accused of taking a banned substance. 

Who is turning that down?  I expect the room to be awfully quiet.  So let’s stop feeling sorry for Braun even if he is 100% innocent.

Population of 100
10 dirty X 50% = 5 positive
90 clean X 1% = .9 positive

5/5.9 = 84.7%

MGL #17

I look at it this way: if what we’re saying here is that due to the finite rate of false testing results, some small % of MLB players are going to get falsely accused of using PED’s, I’m still going to “feel sorry for” the unfortunate one who draws that short straw.

We all drive cars, and in doing so (and deriving the manifold benefits thereof), we all incur a small but > 0 risk of being involved in a fatal accident in which we are 100% not at fault (say someone wanders over the yellow lines and hits us head on).  Should we not feel sorry for the unlucky ones who have that happen to them?

I get what you’re saying: overall, being a major league superstar is a positive thing, but still, I’m going to feel sympathetic to anyone who is falsely accused.  Nobody deserves that…

There is a huge difference. As I said, it is a quid pro quo. You know going in that you might be falsely accused (and falsely convicted). You sign an implicit acknowledgment on that major league contract that in exchange for millions of dollars, fame, adulation in some cases, and the privilege of playing a boy’s game for living, you willingly accept the chance that you might suffer that fate.

In driving, you do not have much of a choice and there is not a multi-million dollar payoff. More importantly, getting involved in a serious accident is not even close to being falsely accused of using PED’s. Not even close.  Terrible analogy. You have no reason whatsoever to care that you are accused of using PED’s. If the fans don’t like or respect you because of it, so be it. Do you feel sorry for me when I get lambasted on BBTF?  I would hope not. It does with the territory! I can choose to let it bother me and i can choose to not let it bother me.  There is no inherent damage, just like there is no inherent damage to being accused of using PED’s.

As Tango said on this blog a while ago, he would issue a statement at the outset that goes something like this:

“I may at some point get accused of using PED’s. There is some chance that it will be true and some chance that it will be false and in either case, I have little chance of proving my innocence.  If that should arise, I will make no comments whatsoever. In fact, this is the last time I will ever comment on this matter. So don’t even try and ask me anything if that contingency should arise.”

Again, getting accused of using PED’s is NOTHING compared to the benefit and privilege of playing ML baseball for a living and making millions of dollars.  How you can compare that to a fatal auto accident is beyond me…

@dmlanoue/18

You’re right-I made a typo in my very ad hoc calculations. Thanks!

MGL #17 and #20

Let’s use a different analogy: let’s suppose you get pretty much the most an ordinary person can expect from life: you get good friends, a wonderful family, a job you like, financial security. Few would ever pass that up. Now, let’s suppose that the woman you love is also a member of an identifiable minority that you are not a part of (e.g., you’re white in America, but she’s black.) In choosing to marry her, and get the life you want, there’s a risk that you and your wife will be discriminated against because of your interracial marriage. That doesn’t make the discrimination okay, no matter how foreseeable or quid pro quo your choice was.

People can control whether they’re unfair and we shouldn’t let their unfair behavior off the hook because “life is unfair.” We say “life is unfair” in the face of shizit that’s just beyond our control--Lou Gehrig’s Disease, for example; when it’s in someone control, we say “that’s bullshizit.”

(This highlights that we’re really looking at two different questions, “should we feel sorry for Braun?” and “should we tolerate unfair treatment of Braun?” My own inclination is to say “no” to both. The unfair treatment of Braun is the same human inclination that produces a lot of social ills--it’s rushing to judgment, harmful to others without proper cause, and unjust. We should react against those inclinations wherever we find them.)

#22:  I agree it’s two different questions.  However, if Braun turns out to be innocent, I will feel sorry for him.  He’s going through a really tough time right now (so I guess I should say I disagree with MGL’s #20 that being accused of PED’s should not bother anyone).  If it turns out he didn’t deserve this ordeal, he will have my profound sympathy.

#20: not trying to equate the two things by any means, but rather I was partly trying to illustrate that to me, whether or not I feel sympathy is unrelated to the severity of the harm, if they played no part in their misfortune.  I probably didn’t do a very good job of making that aspect of my point clear, I’ll admit.

#23 Yeah, maybe I would feel sorry for him. PED accusations are actually pretty huge in baseball.

PED accusations aren’t the equivalent of a fatal auto accident, but how about an accident that badly scars your face so that, for the rest of your life, almost everyone who sees you looks away? That’s not a horrible analogy, and I would feel sorry for the person who got the scar.

Fool, completely different!  One (the discrimination) is bad behavior and should not be tolerated.

The other is just the luck of the draw - no one did anything wrong. There is no inherent reason for Braun to feel badly.

MGL, my sympathy for an unjustly accused person is a) for the suffering they undergo while their innocence is in question, at the hands of those who presume guilt, and b) the residual damage to their reputation due to people not ever entirely believing in the “acquittal”.

Part of this stems from the way our justice system doesn’t ever actually declare anyone “innocent”, but rather “not guilty”.  So it’s quite understandable that to an outsider, a person accused of, then acquitted of a crime may not be 100% trustworthy thereafter.  There’s no way to get a jury verdict that declares you innocent, so a person unjustly accused of a crime can never be returned to their pre-accusation state in such a case.

Now, in this instance there’s no trial, so it’s possible that if Braun is in fact being unjustly accused, he may eventually be fully cleared of wrongdoing.  However, it seems more likely that even if he wins his appeal, his reputation will be damaged, and if that happens to someone who did nothing wrong, I will feel sympathy.

I think I may have misunderstood your point in #17. I took you to be responding to me, #16, where I said

If anything bothers me about the current discussion of Braun--by the way, I tak the present post just to be pointing out good evidential procedures, not to be among those discussions that bother me--is that for the most part, Braun is on trial in the media that doesn’t really have the evidence, just scattered reports about it. Assuming he’s being honest, his reputation will be tarnished anyway. Assuming he’s dishonest, if this were a real court, he could at least demand a mistrial.

What I’m complaining about there is the fairness of rushing to judgment with incomplete evidence. This is different from a case in which people with all the available evidence get a false positive. If in #17 you merely mean to be talking about the risk of judgment on the basis of a responsible assessment of evidence, this qualifies as a “life ain’t fair” situation. But if you mean to include among the risks Braun accepted that people make irresponsible assessments of evidence, this is a case of people being unfair to Braun and that’s the sort of risk other people are liable for. That is analogous to discrimination.

MGL/12 said,
“Also, if a player ingested something with no intent and no negligence, and that triggered a positive result, isn’t that the same thing as a false positive? “

At least in Olympic sports, it’s not at all the same thing. Usually rules against doping are written such that intent doesn’t enter into the equation; simply failing (or refusing to take) a test counts as a violation. Now in practice sometimes “intent” and other matters on appeal have led to somewhat reduced suspensions, but I’m not aware of such an explanation ever sufficing to overturn a finding of guilt.

Now what the rules are for organizations (what I write here applies to track and field, and I believe, IOC sanctioned sports) and what you or I may think the rules should be can indeed differ. But if having the substance in your system provides a training and/or performance benefit, irrespective of intent, then the player might be advantaged against those who don’t take it, and part of the point of the rules is to prevent some athletes from gaining an unfair advantage.

In criminal law, intent often matters, if only to determine which degree of crime should be charged. But for IOC rules against doping, intent was explicitly kept out of the picture.

Geoff, I was just referring to the numbers. I meant that the chances of accidental ingestion of a substance that tests positive (with no negligence) can be lumped in with a false positive for the sole purpose of calculating the chances of a player cheating with intent.

Fool, sure, if he is being crucified in the media or by the fans, unjustly, then that is wrong on their part. Do I still feel sorry for him? No. Because I have a cold heart? No. Because he makes millions and that is an occupational hazard, with no actual harm done to him (other than his feelings get hurt). It is good for someone not to care what others think of them. If he chooses to be hurt because of what others think of him, that is a character flaw in my opinion.  Maybe it’s a good thing. Maybe he will learn that his self worth has nothing to do with what people think. If he is innocent, he has nothing to feel bad about. It’s not like an innocent person going to jail for 10 years. It is an innocent person missing out on 50 days of salary and still making millions.

And again, the ONLY way you make those millions is to agree to the .1 chance that you will test positive. So take it like a man!

Here is a perfect (IMO) analogy: I used to play poker (and before that, blackjack) for a living, many years ago. Obviously I would have winning and losing sessions. Did I expect anyone to feel sorry for me when I had a losing session?  No! Absolutely not! It is part of the profession. The only way you can make money playing poker (or whatever), besides playing with an edge or course, is to have some winning and some losing sessions.

Part of playing baseball for a living and making millions of dollars and doing what every boy dreams of doing when they grow up is this:

1) Losing some games
2) Being away from home for long periods of time
3) Getting booed by the fans sometimes
4) Get excoriated by fans and the media when you are not playing well.
5) Occasionally getting a false positive on your drug test.

So do you feel sorry for players when they experience 1-4?