THE BOOK--Playing The Percentages In Baseball

I don’t have a link, but my recollection is that he did a review at the end of last season (or maybe the year before?) and the result was that green players were healthier than yellow and red players, but there was no difference between yellow and red players.

The system seems to work in terms of is player X likely to be healthy or not, but can’t further separate the likely not healthy players into very high risk vs high risk.

Tango, I had an idea I had for this along the lines of your community forecasting system.  Simply ask the community to project the number of PAs/IPs that they think a hitter/pitcher will accumulate in the upcoming year. 

If you want, add a disclaimer saying “Do not consider managerial decisions or the impact of performance on playing time, only health” (although I would not do that).

Why?  Because every year when we evaluate forecasting systems, we focus on how well the prognosticators predicted rate stats.  Nobody ever looks at playing time.  The intellectual challenge in any forecasting system IS the rate of production.  The playing time predictions are, IMO, just so much hooey.

There’s a good reason for this: predicting playing time is, algorithmically, very difficult.  It may be more accurate to try the holistic approach, using a large sample size (wisdom of the crowds). 

In other words, we need to start bifurcating performance projection from playing time projection.  Will Carroll is predicting playing time (in a gross fashion), the forecasters are predicting playing time (but we never check on them anyway).  I lay out the case for separating the two here:

http://www.hardballtimes.com/main/article/game-of-risk/

Skip down to “projection” section.

“Case study: Rich Harden, one of my favorite pitchers. His 10th, 50th, and 90th percentile PECOTA projections are 78 IP/6.78 ERA, 114 IP/4.07 ERA, and 140 IP/2.43 ERA. I can tell you that 78 IP from Harden this year is a pretty decent probability, considering his last two injury-plagued seasons. I can also tell you that a 6.78 ERA, while possible, is way down the list of any A’s fans concerns about Rich Harden. More likely than a 78 IP/6.78 ERA is 78 IP/2.43 ERA; a good performance in an injury-shortened season has been on everybody’s mind.

sal/2: To be clear, I will NOT be re-running the Community Forecast for 2008.  I WILL be finally compiling the data for the 2007 season. 

In that study, I did ask the people for playing time for all the players, so we’ll see how well they did.

***

philly/1: I have a vague recollection of that, but you need a baseline.  For example, is Will better able to pick out the injured players more than Marcel?  Marcel is good at forecasting say Ken Griffey and Moises Alou to be injured, based strictly on how many games they played in the last two years.

What is ballsy with Will here is that he selects players who were injured to have a green light.  THAT’s what interests me.

Ballsy. Nicest thing you’ve ever said about me, Tom!

And here’s another: http://www.baseballprospectus.com/article.php?articleid=3743

My apologies to Will.  I should not have said, “Will Carroll is predicting playing time (in a gross fashion)”

I should have said, “Will Carroll’s system can help us predict playing time (in a gross fashion).”

Clearly, Will’s system is not meant to predict playing time, only - if I understand correctly - the likelihood of injury.

Will - does your system intend to assess injuries that will require a DL stay only, or any injury, or simply the possibility of ending up on the DL?

If someone wants to seriously do a study, send me an email (click on my name), and I’ll set you up with Will.

Perhaps easier to study:  any relationship (after controlling for other factors) between Pitcher Abuse Points and actual innings pitched in the following season?

I highly doubt it.

Meanwhile, in light of the Livan Hernandez signing, Twins fans continue to pray that there is a strong but heretofore unrecognized relationship ...

Will posted links in post 4 and 5 regarding evaluations.  They were queued up for me to unspam.

I finished compiling data for the 2007 season, PA and IP projections were from PECOTA:

Green Hitters- avg DL days/hitter- 13.28, projected PA/hitter- 522, actual PA/hitter- 522, 69% with no DL trips

Yellow Hitters- avg DL- 18.54, projected PA- 504, actual PA- 526, 59% with no DL trips

Red Hitters- avg DL- 19.66, projected PA- 430, actual PA- 464, 64% with no DL trips

Green Pitchers- avg DL days/pitcher- 24.86, projected IP/pitcher- 128, actual IP/pitcher-126, 56% with no DL trips

Yellow Pitchers- avg DL- 47, projected IP- 130, actual IP- 124, 57% with no DL trips

Red Pitchers- avg DL- 46, projected IP- 112, actual IP- 101, 44% with no DL trips

Whoops, 2 typos:

Yellow hitters should be 60% without DL trip.  Green pitchers should be 58% without a DL trip.

I’ll make this simpler:

Red should be 50% injured or above.
Yellow is 33-59%.
Green is 7-33%.
Blue was supposed to be 6% and below, but it was terrible. I told Victor to include it with Green when he asked.

So for me it looks like:
Red = 36% hitters, 56% pitchers
Yellow = 40% hitters, 43% pitchers
Green = 31% hitters, 42% pitchers

The biggest “miss” is the red hitters.  I wonder if this was a small sample that’s just out of whack bc past years have been much more in line with expectations. The underlying actuarial table did adjust down for hitters by a bit, so I think there’s going to be some correction. Green pitchers are also higher than they “should” be, but as with the blues, I think there was an error in an underlying assumption that’s been corrected in this year’s system.

Thanks to Victor for crunching the numbers. I think I’m better than Marcel’s injury-predicting monkey cousin.

What are the sample sizes for each group in Victor’s study?

I’m not so sure Will.  Marcel forecasted the following, as per Victor:

Green: 522
Yellow: 504
Red: 430

We’d expect the “actual” to be a bit higher for the green and somewhat lower for the red.  In fact, we got:
522, 526, 464

So, knowing the “light” indicators would have been no help here.

For pitchers though, there is the pattern we’d hope for.  Red pitchers should be forecasted with 10% fewer innings, yellow 5%, and green 0%.

I thought Victor said the 522/504/430 were PECOTA projections, not Marcel.

Does Nate use Will’s injury colors to fine-tune PECOTA?  Or is that totally independent?

Unless I’m reading this wrong, we still don’t know what Marcel projected for those players.  Too early to brag about beating the monkey I think.

Tom, you can’t use innings, something my system pays no attention to at all, to measure the system. That’s bad measurement. My system measures injury risk, in percentages. In a one year sample, it’s pretty good. I better over three or five years, it’d be better.

Perhaps it was PECOTA.  And RAlly makes a good point if PECOTA uses Will’s light information.  I’m guessing it doesn’t.

***

Will, the point about the IP forecast v IP actual is this:
1. first off, we really don’t care if a guy is hurt or not… all we care about is how often he will pitch and how well he does it.  So, you can get the exact right % of players injured, but unless you get the right outputs (IP, ERA), it doesn’t really matter.

2. And most important, your Red pitchers may get hurt more than your Green pitchers, but, we need a baseline to compare against.  After all, Marcel could have given you the same percentages as well, based strictly on IP in the last 2 years.  This is the point.  How much more information does your insight have over and above what we can infer from a player’s seasonal stat line?  E.g., Marcel would infer a Red for Mark Prior.  So, if you give Prior a Red, that’s really irrelevant to us.

And from the looks of it, it seems that if Marcel has two pitchers forecasted the same, and Will Carroll comes along and says, “Green for you, Red for him”, then we should lop off 10% from the Red pitcher’s innings.

Injuries can show up in two ways - either less playing time or poor performance.  You might have some guys who pitch as many innings as expected, but if their ERA’s are 0.50 higher because they pitched through pain, fantasy players will appreciate the Red light.

I have no idea if the data supports this or not, but its something that should be looked into before either Will or Marcel starts bragging.

Right, that was my point #1 in post 17. 

Victor: did you also have the ERA for those pitchers in your study?

Tom—YOU don’t care, but I do. I’m not interested in IP or ERA in the least. I’m interested solely in risk and the controlled usage of risk. To use a financial analogy, you’re looking at naked calls while I’m trying to find the baseball equivalent of a straddle. (Actually, if anyone reading this is into options, what I’d love to find is the baseball equivalent of a condor.)

Ok, why do you care about injury risks, if it’s not to know if it means a decrease in either performance or usage?

That is, you can estimate injury risks better than I could, but if I can estimate IP and ERA better, what’s the purpose of the injury knowledge?

19- no, I don’t.  I could look up projected ERA/EQA vs actual ERA/EQA for pitchers and batters . . .

If you can accurately project IP, we wouldn’t need the injury info. The problem is that the volatility is so sharp that you wouldn’t have to miss on many 150 or 200 inning guys to screw up your whole model. Let’s say you’d projected Schilling to carry a 200 inning load this season. Oops. Or like me that you’d expected Frankie Rodriguez’s arm to fall off. I’d be curious how Marcel vs PECOTA would do on IP projections, since PECOTA implicitly looks at injuries. (Oh—I use PECOTA attrition in the THR, but PECOTA doesn’t use anything of mine. Yet.)

Sure, one or two guys can scr-w up the model.  If I miss 150 IP on Schilling, but there’s 150 pitchers in his Light class, that still gives me an error of only 1 IP per pitcher, on average.  If Victor is using 3 years worth of data (Victor, can you give the “n” for each group), that mitigates most of the problems.

Clearly, you are right if your sample size is small, and you very much care about the injury rates.  But at the levels we are looking at, at the group levels, we pretty much don’t care.

In the end, we are both right!  Love it when that happens.

Here are the numbers per group, remember I only ran data for 2007.  I also took out a few guys like Papelbon who Will had rated based on Papelbon being a starter.

Green hitters = 128
Yellow hitters = 41
Red hitters = 53

Green pitchers = 77
Yellow pitchers = 47
Red pitchers = 55

I think Will should stick to writing UTK.  I knew the Team Health Reports were useless, and this data supports that feeling strongly.  The categories are too broad to be of use and the columns are typically full of explanations as to why a player has received a red, green, or yellow light when other data indicates that the player should be categorized differently.  The reports also do not seem all that focused or consistent ... see the discussion of Carlos Lee vis-a-vis his weight, saying that it “should be taken into account.” Okay, so does the yellow light account for it already?  Should it become more yellow?  Does this really add any more than the casual fan looking at him and seeing a fat guy who is clearly a decent athlete despite the spare tire?  I don’t think so.  One may also look to Will’s defense in today’s Unfiltered blog, for instance, of the red light for Ellsbury as compared to the yellow light for Ortiz.  Will starts talking about how an injury to a speed-dependent player like Ellsbury would hurt his performance more, on a relative basis, than another type of player.  I thought the lights were supposed to convey probability of injury, not whether an injury would sap a player’s value.  The scatterbrained reply Will gives amounts to nothing more than “the table says that he gets a red, so there” and then seems to discuss the extent to which an injury would hamper a speed-dependent player.  The actuarial models do not lead to useful data and it’s odd to see him continue to go out of his way to try to prove the worthiness of a system with little value.  I wish we could get some analysis of actual injuries, not a useless rating system that I have not known anyone to find of value.

For some reason, some people have alot of venom ready for Will.

***

Will’s Yellow players are supposed to be injured at a 40% rate, and that’s what Victor’s study shows.

His Green players should have averaged 25%, but he missed (hitters were at 31% and pitchers at 44%). 

He missed here on the pitchers.

The Red should average 55% or 60%.  His Red pitchers were at 56% and hitters at 36%.

He missed here on the hitters.

***

In fact, it looks to me that all the hitters Light groups were random.  The forecast PA matched the actual PA for the hitters.  In effect, I would ignore the Light players from Will on hitters.  Either that, or Will should have a “confidence level” on each Light.  For example, he might make Junior a Red, and he’s got a “strong confidence” on that, and he might make Ellsbury a Yellow, with “low confidence”.  Otherwise, I think we’re justified in ignoring his Light on hitters.  The status quo must be the default until proven otherwise.

***

As for the pitchers,it seems that he should only have Yellows and Reds, since he gave out way too many Greens.

Will gave out the following:
77 Green (of which 44% were DLed)
47 Yellow (of which 43% DLed)
55 Red (of which 56% were DLed)

So, in all, 179 pitchers, of which 47% were DLed.  Clearly, if Will wants to get his Green pitchers at a below 33% level (likely an average of 25%), there’s simply NO WAY that he can make 43% of them Green.

Follow me.  The mean of all pitchers is 47% DLed.  If you are looking for a subgroup of that, that tops out at 33%, and likely has a mean of 25%, that’s going to be a very small group of those pitchers.  Maybe 20-25% of his 179 pitchers will be Green?  Certainly not 43%.  That’s absurdly high.

So, if Will can get his Green pitchers much lower, then we can listen.  Otherwise, we need to lump his Green and Yellow pitchers into the same bucket, and leave it at that.

Good suggestion on the pitchers Tango.  There are no green pitchers.  Brings me back to my definition of a durable pitcher:

“A pitcher who has not been injured yet.”

I’m looking at Will’s data, and it is really impossible to get a subset of his Green group to be “really green”.

For example, we have the following: 77 Greenlighted, of which 34 were DLed.  Now, what if Will comes and tells me that of the 77, there were 32 that he felt were REALLY Green pitchers, and the other 45 COULD have been Yellows.  In that case, his 32 Green pitchers would have had 8 DLed among them.  That leaves the other 45 with 26 DLed among them (34 minus 8), or a rate of 58%, which is a RED rate.  Basically, of his 77 Green pitchers, he has to choose those that were really Green and those that were really Red.  No Yellows, which doesn’t make sense.

On the other hand, if he simply said that he cannot have any Green pitchers at all, then his 124 Green+Yellow (now Yellows) have 54 DLed pitchers, meaning 43.5%.

Basically, if Will really believes that there are Green pitchers, they would have to be a tiny minority, and I mean even less than 10% of all pitchers.  No way will he put himself on that kind of line.

Unless someone wants to challenge me on it, my conclusions are:

1. All hitters get the same light, unless you want to pick out a tiny minority (less than 10%) as Green, and a similar minority as Red.

2. All pitchers get either a Yellow or Red light, split pretty closely to 50/50.

***

A similar public study should be done for the PECOTA percentiles.  Do the ranges work for guys with little playing time?  Lots of playing time?  Power hitters, speed hitters, K pitchers, relievers, etc. 

As we can see here with Will’s Lights, these groupings simply don’t work and are not tested.  The PECOTA groupings have to be treated with similar skepticism, meaning they have to be discarded until proven.  I am pretty sure they are wrong in any case.

I used to think we could single out some hitters as especially injury prone or durable, but my faith was shattered in 2007 by these numbers:

JD Drew 140
Tejada 133

I’m pretty much in agreement with #29.

There’s a lot of venom for Will because he writes in an arrogant manner and doesn’t add any value other than in UTK and, occasionally, in Will’s Mill.

In today’s Team Health Report, we got to learn that Johjima shouldn’t have broken down due to heat because he “plays in a dome.” Do you need me to tell you why this is asinine?  (Hint:  the Mariners haven’t played home games “in a dome” for a decade.)

Look, I understand that it’s not that easy to add value and to write well, particularly on a frequent basis.  I’m not saying that the mistakes/issues I’ve cited in this and my previous post should, on their own, cause us to say that the THRs are crap.  But what I am saying is that I’ve seen no evidence that the THRs are anything other than crap and that the errors/foolishness that is so conspicuous in randomly-chosen pieces by Will causes me to doubt the validity of his data and conclusions.  The guy is writing for a paid service and needs to bring up the quality if he expects, as he appears to, to be treated like a legitimate sports journalist.

Will responds on the dome issue:

http://baseballprospectus.com/unfiltered/?p=782

***

http://www.retrosheet.org/2007/2007sea.zip
Retrosheet event files say the following:

56 Sunny
46 Cloudy
39 Unknown
15 Dome (3 of them in Houston)
6 Overcast

6 90+ F
13 80s F
58 70s F
54 60s F
31 50s and under
Average temp: 68 F

48 Day
114 Night

***

Otherwise, I appreciate that you try to support your opinion, rather than the standard “he sux” comments most would write.

Bump.