THE BOOK--Playing The Percentages In Baseball

#1 - One item I should have included in the article was that the average days off for the players looked at was ~32 days.

Question - What is the average age drop in talent from one year to the next in wOBA.  There is a nice drop in talent from the year before the injury to the year ofter, but it may just be normal aging patterns.

Indeed, several of the players did have longer break periods than the minimum 15 days. The average number of days increased over the years, but the sample size was small so it’s hard to say how reliable the numbers are. Also there is the problem of concussions not being properly diagnosed.

Jeff- the opposite of that could be true if it’s a younger player and he would be expected to play better as he reaches his “peak”.

It’s be 2 or 3 runs per season, which is about 5 wOBA per year.  This of course presumes everyone is over 30.

Realistically, you’ve got one-third pre-peak and two-third’s post peak, so overall, it probably cancels out (guys younger have a steeper slope up).

Doesn’t that make sense?  You want to get the guy back into the lineup as soon as he can help the team.  That might be be 40 points below his normal production.

That is: if you put him in the day after the concussion, he might be -300.  A week, -100.  A month, -40.  Two months, -20. 

As long as there’s no further risk to his health, putting him at -40 might be just fine.

BTW, I wonder what the chart would look like for other injuries ...

#4 - I know Colin ran the numbers for wrist injuries in his fall preview of PECOTA.  I may do them all eventually.  Right now, I am looking back at Tommy John Surgeries over the years.

I dunno Phil.  That’s 40 points of wOBA on the hitting side.  How about on the fielding side?

Plus, if he’s only back as replacement level, I would hope he’s back as a bench player, not full time.

Plus, if he’s only back as replacement level, I would hope he’s back as a bench player, not full time.

Unless getting back to your prior production is a matter of innings played, not time!

On April 21, 2003, Sammy Sosa got beaned in the head, shattering his helmet.

http://reds.enquirer.com/2003/04/21/sosa_zoom.jpg

From the beginning of the season through April 20th, Sosa hit .407/.577 (OBP/SLG). After the beaning, he hit .301/.527 for the rest of the year, and then .314/.459 through the rest of his career.

#8, the important thing for Sosa would be what did he hit in his career prior to the beaning versus after it, adjusted for age.

Choosing Sosa and such a small time frame (April 1-20) is an example of cherry picking.

Actually, such a small sample has nothing to do with cherry picking, other than the fact that it is easier to cherry pick (find anomalous results) from small samples.

Cherry picking can even be caused by publishing bias.  In fact, they are almost the same thing.

Say I think to myself, “I wonder how Sosa did before and after his beaning?” Then I look it up and it turns out it is the same.  I won’t “publish” that data.

Even Jeff’s results could be somewhat the result of cherry picking/data mining/publishing bias.

Had he found little or no difference, it is less likely he would be writing about it.  Also, he may look at all different kinds of injuries and be more likely to publish the data when there is a large before/after gap.

That is also one reason why it is nice to have out of sample data to test on.  In this case, we don’t have any.

Given the magnitude of the results, it (publishing bias, et al.) is not likely to be much much of a factor, but I’m just sayin’…

#9 I would publish the results no matter what the outcome.  As stated in the article, the sample size is uncomfortably small.  No difference would be bigger news then that there was a drop off.

Publishing bias is not only if you publish, but how you publish and how you study the issue.

Say that Nate Silver wants Pecota to come out looking good.  He’ll do a couple of good faith efforts, see results that confirms that Pecota is good, and then publishes those results.  But, what if his intent was to look for Pecota to NOT come out looking good?  Well, he’ll keep studying and looking for new angles, until finally, he’ll find a test that makes Pecota look not good.  And then he’d publish those results (in addition to, or in place of the original findings).

Now, we’re not talking about Jeff here in particular.  We’re just talking about generally speaking, we apply a Bayes process to researchers motivations.  There’s a certain percentage of researchers that have a certain motivation and a certain percentage that have different motivations, and so on.

Since we can’t get a prior = 100% for any researcher, what are we to do?  We have no choice but to presume that there’s a non-zero percent chance that each researcher has some nefarious reason to publish what they publish.

Otherwise, we’d have to do a case-by-case analysis of each researcher’s motivations to publish what they publish.

I’d like to think that I’m the sole exception on the entire planet, that my motivations are pure.  But I’m human, and who knows what’s subconsciously affecting my brain.  Unless I publish a minute-by-minute log of what I’m thinking and what I’m doing, we have to accept that there’s publishing bias in some form or other with everything being published.

Even this blog post!  I mean, what motivation do I have to post this?  Is it to look impartial?  Is it because I just want to take a break from my work?  Is it to make people think I’m impartial so that I can use that to my advantage for when I want to hoodwink them in the future?

So, you’ve got to Bayes my a$$, and you’ve to to Bayes everyone’s a$$ too.

Anyway, that’s MGL’s basic theory I think.

I’d like to think he’s wrong.  But, I have never thought about it in those terms before he brought up publishing bias.

That’s why you have to replicate studies.  If Nate publishes a study where Pecota comes out looking good or not good, someone else should try a study with different criteria to see what happens.

This is especially important where a study is structured kind of unusually ... if you ever find that you’re asking yourself, “why did that researcher decide to do it THAT way?”, that’s a very strong indicator that someone needs to try a replication.

Tango/6: if the guy is .900 (OPS) before the concussion, but only .860 after the concussion, it’s still worth playing him, no?

But if the guy is .700 before the concussion, and .660 after the concussion, you’ll want to sit him until he’s better.

In any case, my point is that it might be OK to play the guy before he’s completely recovered, so long as it won’t hurt him further to play.  And a great player is still a very good player with .040 less than before.

I want one of those original, “You’ve got to Bayes my a$$”, T-shirts!

40 wOBA points is not 40 OPS points.

In Jeff’s study, he showed a drop of 36 OBP and 48 SLG points, or 84 OPS points (see image at top of this thread).

So, a 750 OPS player who, after 32 days after rest, comes back at 664, then, no, he is NOT worth coming back.  What if him coming back “too soon” means that it will take him even longer to recover?

It took him 32 days of rest to come back as a replacement-level player.  I’d want him to come back alot closer to his average level.  Maybe the average rest should be 45 days or 60 days and not 32 days.

Sorry, my bad ... read wOBA as OPS.  40 points of wOBA is indeed a lot.

And, agreed that if coming back after 32 days means it takes him longer to recover, you probably want to rest him further.  Unless, of course, you’re about to take him to arbitration. 

The only way to properly assess a concussion effect is to look at a non-concussed group that also had a 30+ day break in action.  I would have to guess that this group also does pretty poorly after coming back.

Most concussions come from being hit in the head. 

After being hit in the head, some players might not be as willing to step into the pitch, might not be as willing to do after an outside pitch, or not turn on an inside pitch they way they did before?

Could this be as much mental as physical?

"The only way to properly assess a concussion effect is to look at a non-concussed group that also had a 30+ day break in action.  I would have to guess that this group also does pretty poorly after coming back.”

Sure, you want a “control group” to account for time off, and you’d also like to know the dropoff for other injuries or for the DL in general.

My guess would be that for other injuries, the effect is much less (depending on the type injury of course), and that for a healthy player that just takes 30 days off, the effect is negligible (less than 10 points), at least compared to a 40 point drop in wOBA.

Jeff, publishing bias does not have to be something sinister or even conscious.  You don’t have to defend yourself by saying that you would have published the results no matter what.  There is nothing to defend.  Plus, neither you nor we have any idea what you would have done had you found different results.  And it is not an either/or.  There is little doubt that ANY researcher, including yourself, would be LESS LIKELY to publish ANYTHING where the results are not interesting.  That is why some people in certain important fields, like medicine, would like all studies to be pre-registered so that the researcher is compelled to report any result.

I have said this before, but I tinker around with all kinds of studies all the time.  If I find something interesting, I might publish or talk about it.  If I find nothing interesting, I move on to something else.  It is not sinister.  That is just the way it is.  Classic publishing bias.  One form, at least.

Here is an interesting article about publishing bias in clinical medical trials despite efforts to pre-register:

http://www.nature.com/news/2009/090911/full/news.2009.902.html

I certainly agree that publishing bias is an issue.  However, I think there’s less of it in our field than in others.  In the medical field, if you do an experiment and find no effect, it’s hard to get it published.  In sabermetrics, if you do an experiment and find no effect, you can blog about it anyway (or send it to “By the Numbers”—we’ll take it).

The more easy and open it is to publish, the less selective sampling there is in what gets released.

Sometimes the cherry picking/selecting sampling/publishing bias/data mining comes simply as a result of this:

Someone notices or reads/hears something, like, so-and-so (player A) really played poorly after coming back from a concussion injury so they decide to look at the issue.  Of course the rest of the players are out-of-sample which is fine, but the initial player (the one you heard or read about or notice) is cherry picked and should not be included in the study, but they usually are…

I try to not have any preconceived notions to prove or disprove - instead find an interesting question and see what turns up.

Recently there have been pieces written on HRs off pitchers (Matt Cain) and HRs by batters (Jose Bautista), on whether a pitcher’s low rate or a batter’s high could be maintained season to season, plus expected babip by fly ball/hr hitters. The past couple days I put together a WOWY, pitcher, batter and ballpark, along with the pitcher’s hand, batter’s hand, hit to lf, cf or rf, and ld or fb. Got some interesting numbers, new stuff I’ve just never had the opportunity to look at before. I’ll study it some more and see what conclusions I come up with, write an article, then you guys can tell me whether it’s bs or not!

I finally got around to running the numbers on the year before and after a concussion.

Year Before Concussion
BA 0.279
OBP 0.356
SLG 0.472
Year After Concussion
BA 0.277
OBP 0.354
SLG 0.471

Great stuff Jeff!  This makes it seem like it’s very transient, so that a concussion will eventually lead to full recovery.

So, it’s just a matter of figuring out the recovery period.

***

In the chart image above, we should ignore the “15 day before” stats, and perhaps even the “60 day before”, though those stats are consistent with the “year before” stats.  Both are about a .375 wOBA.

The 15 days after is a .312 wOBA and 60 days after is .339 wOBA, making the 45 day in-between a .348 wOBA (i.e. at days 16 to 60).

***

Jeff, if you can show performance in 30 day slices post-return that would be fantastic.

So, day 1 to 30 post-return
day 31 to 60
day 61 to 90
and so on

We’re looking to see at what point is his wOBA back to normal.

It seems to me that the day 61 to 90 might be the point where things get back to normal.  Remember, that means they should only return 61 days MORE than their normal return (which Jeff says was 32 days).

So, a concussion recovery period may very well be at least 90 days.

***

This has potential to be really fantastic research, worthy of mainstream media attention.  Go for it Jeff!

#9 Yeah, sorry for offering such a small smattering of information there. I was in a hurry and didn’t have time to mention Sosa’s pre-injury numbers.

I should have anticipated MGL would assault my meager offering with all the mountain man ferocity with which I picture him. Allow me to mend my earlier glibness:

From 1989 thru 2003:
.349/.546 (OBP/SLG)

From 2003 thru 2007:
.327/.486

Is it aging or a concussion effect? It’s hard to say. There’s a lot more pre-concussed data, so it possibly unfairly weights that side, but at the same time, Sosa’s change in ability seemed incredibly sudden.

It also might be worth mentioning that Sosa didn’t even take a day off after the beaning. Maybe that plays into the apparent lack of recovery.

Those were Sosa’s age 34-38 seasons.  His drop-off also coincides with the start of PED testing, which he reportedly failed in 2003.

Sosa also started progressively getting more DL time starting in 2003.  He had back spasms, and a few DL trips for foot problems.

Wouldn’t you have to control for a group that had ANY injury and missed a similar period of time? Doing a matched pair shouldn’t be too hard—this guy missed 32 days, easy enough to find another. Not hitting live pitching for that long is going to be the biggest effect, I’d guess. Do any of these guys have a rehab assignment?

"Wouldn’t you have to control for a group that had ANY injury and missed a similar period of time?”

It depends on what you want to look at, doesn’t it?  If you want to look at the effect of a concussion (as opposed to no DL stay), then you don’t need to do that.  If you want to look at the effect of a concussion over and above any other injury given the same DL stay, then you do.

Right?

#26, I wasn’t railing on you. Just trying to make a general point.

“Is it aging or a concussion effect? It’s hard to say.”

It is not hard to say.  Or you can say that everything is hard to say.  With sample data, we know nothing (for certain).  Every inference from analyses of sample data are likelihood distributions and mean or median likelihoods.

We have a pretty good idea of the typical aging curve.  If we “factor that out” for Sosa, then what is left we assume is the effect of injury and (discontinuing) PED use (and all other unknowns).  That is the way analysis of sample data works.  Saying that something is “hard to say” is tautological if you mean that it is “hard to say with (near) absolute certainty…

The author mentions it prominently, but it bears repeating: this so-called “effect” is based on a sample size of *fifty*.  There is nothing the author or anyone else can do about the lack of data, but publishing it invites all sorts on inappropriate conclusions.  The truth is we don’t know what impact a concussion has on a player 30-60 days later.  And there is simply not enough data to determine what it is.

Do you mean *fifty* or do you mean FIFTY or do you mean fifty?

He looked at 60 days worth of playing time for 50 hitters.  That would mean say 150-200 PA per player x 50 players, or 10,000 PA.  I should say TEN THOUSAND PA.  That’s a single player’s entire career.

One SD is only 5 wOBA points.

A sample size of 50 is actually quite alot.  I would have been happy with 20.

This is in the same grain as the study above.  I wonder if players get “rusty” from not playing.  For a full time player who doesn’t play for a day, week, month, etc. do we expect his performance to be better or worse than his pre-break performance?  Does the rest help him?  Does the effect of lingering injuries hurt him?  I know there are 1000 variables with some helping and some hurting.  Its probably not an easy thing to test but might be worth looking into if somebody could figure out a way to cut through all the issues.  One idea would be to split it into two groups, one that went to the DL and the other that was not placed on the DL.

Also, does giving a struggling player a break help his performance?  This is done a lot but is unclear if it helps.  It would be hard to separate this from players who have small injuries (and don’t go on the DL) but would be an interesting study.

Does time off hurt hitters or pitchers more?  Does it hurt player defense?

It would be hard to find in season layoffs that are not related to injuries or sub-replacement level play.  Bereavement leaves would usually be pretty short.  We may be able to find some cases where a manager just didn’t like a guy and benched him for some time, but I think we aren’t going to find many cases useful as controls.  (Thinking some more...) How do players do when coming back from suspensions?

He looked at 60 days worth of playing time for 50 hitters.  That would mean say 150-200 PA per player x 50 players, or 10,000 PA.  I should say TEN THOUSAND PA.  That’s a single player’s entire career.

Fair point, but it does, of course, matter how many players those 10,000 PA are distributed over.  The sampling unit has to be the player not the plate appearances.

Jared: no, not always.  It depends how the players are drawn.  In this case, one player of 10,000 PA or 10,000 players of one PA would be the same thing.

This is 10,000 post-concussion PA.  I see no reason why it would matter, in this case, how many players are involved.

Tango, in terms of the sample data, doesn’t the medical uncertainty around concussions matter? I mean, this isn’t like Tommy John surgery, a medical procedure which has a relatively tried and true rehab process, or like other injuries where a player’s health can be visibly captured in some way (x-ray, MRI, etc.). Likewise, it would seem something like the incident quoted below simply won’t show in the data:

But Posada was almost knocked out by a foul tip in a Sept. 7 [2010] game against the Orioles, returning to the dugout feeling disoriented and dizzy.

“I remember telling [former pitching coach] Dave Eiland, ‘Something’s wrong with me, I just don’t feel right,’” Posada said. “I felt like I was about to throw up, I was dizzy, everything felt weird. The next day I was still having headaches. It was scary, I have to admit.”

http://bit.ly/fqbRQO

In this case, the player never went on the DL, but his remaining at bats for the season would seem to be worthy of scrutiny for research like this.

To put it all simply, it seems as though it would be difficult to properly evaluate post-concussion player performance if the medical community is still struggling to properly evaluate concussions.

Tango, PA’s by the same player aren’t independent trials.

It’s the same logic by which 100 lab rats in 100 cages is different that 100 lab rats split between 10 cages.  If the interactions between rats matter, the sampling unit better be the cage.

The number of players matters as well as the number of PAs, unless we know that all concussions have the same effect.  Most likely, the amount a player’s true talent drops after a concussion varies based on things like the severity of the concussion, the player’s body/healing/abilities, the player’s behaviour/environment during his rest period, the length of time he takes off, etc.  Because of this, there are two random distributions in play. 

There is the random distribution of the sample performance around the true mean of the group of hitters, which depends on the total PAs.  At 10,000 PAs, the sample performance should fall in a fairly narrow distribution around the group’s true talent as Tango says.

However, there is also the random distribution of the true talent of the group around the true talent of the population.  If the population of MLB players drop an average of .30 points off their normal true talent wOBA after returning from a concussion, then there are going to be some players whose true talent has dropped .040 points at the time they return, and some players whose true talent has dropped .020 points at the time of their return.  When you sample 50 players from this distribution of possible true talent drops, there is going to be uncertainty of the true talent of the group around the true talent of the population.

If you have 10,000 players with 1 PA each, then you are going to be fairly certain that the average drop in true talent by that group of 10,000 players is going to be very close to whatever it is for the population as a whole.  If you have 1 player with 10,000 PAs, you will be just as certain about how close his observed drop is to his true talent drop, but you will be much less certain that his true talent drop happens to be close to that of the whole population.

I have no idea how much uncertainty there is in the group’s true talent drop around the population’s true talent drop at 50 players.  That might be enough players to get a fairly narrow distribution for the group’s true talent, or it might not.

Kincaid: excellent explanation, thank you.

Basically, we have two uncertainties:

- the uncertainty around the mean
- the uncertainty of THAT uncertainty

The one player of 10,000 PA v 10,000 player of 1 PA transfers the uncertainty from one to the other.

That is a good point. 

There’s another difference as well.  If you had one player with 10,000 PA before *and* after the concussion those 10,000 PA groups would be likely to differ by a statistical significant amount even if the concussion had no effect b/c the “after” events are more highly correlated with other “after” events than they are with “before” events.

This is certainly less of a concern if all of the PA’s are clustered within a month or two before and within a month or two after.  But, still, the result of a PA in April is (I assume) somewhat more highly correlated with another PA in April than it is with a PA in July.

I asked Andy to comment:

Hi Tom,

Post #40 is right on.  For the purposes of this study, where you care about average effects instead of player-to-player variations, more players and fewer PAs increases the confidence in the result.

Estimating the population dispersion would follow exactly the same logic as the clutch hitting analysis.

To estimate the uncertainty in the mean drop-off, you’d need to first estimate the population dispersion.  The estimate of the mean from a single player equals
deltawOBA = wOBAconcussed - wOBAbaseline
And has an uncertainty of
sigmadeltawOBA = +/- sqrt( populationDispersion^2 + sigmawOBAbaseline^2 + sigmawOBAconcussed^2 )

Recall that sigmawOBA goes something like sqrt(wOBA*(1.1-wOBA)/PA), and has to be calculated for both the baseline (say, preceding and following years) and the period being studied.

The combined estimate of the mean equals
sum(deltawOBA / sigmadeltawOBA^2) / sum(1 / sigmadeltawOBA^2)
And the uncertainty of the mean equals
1 / sqrt(sum(1 / sigmadeltawOBA^2))

So, to look at the extremes, if you had 1 player with a sample size of 10,000, then the sigmawOBA terms basically go to zero, and you end up with an uncertainty of the mean equal to the population dispersion.  If, instead, you had 10,000 players with a sample size of 1, the sigmadeltawOBA would basically be driven by the large uncertainty of the sample (sigmadeltawOBA ~= sigmawOBAconcussed), and the uncertainty of the mean would equal sigmawOBAconcussed / 100.

Andy

@33—Your post is an embarrassment.  The sample size should be viewed in terms of PA, not number of players?  So if he did a study on *one* player, who had 5,000 PA before an event and 5,000 PA after an event, and there was a difference in performance between the two groups, it should be concluded that any similar event would have the same effect on any player in the future?  Good luck with that.  I mean, if a player has a divorce in the middle of his career, and his performance declines, one should probably conclude that all future divorces will portend poorer on-field performance?  The sample size is pretty huge, maybe even 10k PA.  So done deal, no?

For the record, and to save you some keystrokes, that was *one*, not *1*, won, or even juan.

Does this factor in the player who doesn’t come back at all?  Either because of the severity of the injury or because he wasn’t all that good to begin with.

... or a player that experiences a concussion but doesn’t miss any time?

The situation I have in mind in that on the same day David Wright was beaned in the head and left the game followed by a big ordeal, Ian Kinsler was beaned in the head and went to first. For all we know, Kinsler might have also had a concussion but played through it.

The concussions where players miss significant time might just be the severe cases. Minor concussions might go undiagnosed altogether.

Minor concussions might go undiagnosed altogether.

I figure this happens to catchers much more than we realize, especially those who wear the old-style mask, not the hockey goalie style. As a catcher in high school, a much, much lower level compared to the minors or MLB, I recall the blunt hits to the head. They never hurt in my experience, but I just can’t imagine they’re isn’t some serious risk there considering how little room there is between the brain and the skull.

For certain those foul tips to the mask slosh the brain around. Heck the force is often enough to “spin the mask”.

In a sport where the only reprieve is the time it takes for the ump to give the plate a courteousy cleaning while the C puts the mask back on and shakes his head so both eyes face foreward, catchers aren’t going to ask for a concussion test.

We might as well be asking what effect dehydration has on performance in many of these (unreported) cases. Even when OFs crash into the wall, we primarily check their wrist or shoulder. As long as they can guess within one how many fingers are being held up or say what country they’re in, they stay in the game.

With catchers, concussions are going to get reported like they are for offensive linemen. They probably aren’t. Only for certain positions are concussions acceptable to be reported in our current culture.

Here’s another article on the effects of concussions. Most of the information is a rehash of what is already out there, but this post differs in that the included video shows a researcher cutting into the brain of former football player Dave Duerson and describing what she sees.

http://www.guardian.co.uk/science/2011/jul/19/nfl-star-brain-injuries-destroyed