THE BOOK--Playing The Percentages In Baseball

Doesn’t everybody basically ignore the Bill James forecasts?

PECOTA has been spitting out odd things the last two years.  Others have noted that this coincides with the time Silver has been gone from BP.  It seems Neyer has noticed this now as well. 

I happen to agree with you here that the Mo/Jeter forecasts actually aren’t crazy at all (just slightly pessimistic, which I can understand from a projection system based on comparables, and we’re talking about a 36-yr old SS and a 40-yr old pitcher). 

Actually, the Jeter forecast is something you didn’t even discuss, I assume b/c it’s not really worth discussing (PECOTA’s forecast is right in line with all the others).  Mo… well hell, people have been missing high on Mo’s ERA for years.

It might take more space than is usually allotted these things, but forecasts like this might be understood better if they were explained more like this:

“totally healthy & effective”: 44 saves, 2.00 ERA; likelihood: 10%

“mostly healthy & effective”:  35 saves, 2.00 ERA; likelihood: 15%

“mostly healthy, mostly effective”:  30 saves, 3.00 ERA; likelihood: 15%.

.
.
.
.

“significant injury time, effective:” 10 saves, 2.00 ERA; likelihood: 10%

“significant injury time, mostly ineffective”: 5 saves, 4.00 ERA

“season-ending injury, mostly ineffective:” 0 saves, 5.00 ERA

Then people could better understand how you arrived at the projection you did, and what the risks are, as perceived by the forecaster, for the upcoming season.

These sort of calculations *have to be happening* in projection systems, but this detail is submerged.  Maybe for a few key players it ought to be uncovered and described in gory detail… I think everyone would benefit from that.

Yes, I agree, it should happen.  Maybe one page for one player, you do that.  And then the rest gets the summary treatment.

Buuuuuuuuuuuut, even PECOTA, which DOES shows the percentile forecasts, have those percentile forecasts ignored in favor of discussing the mean forecasts!

When SG presents his CAIRO projections for Yankee players over at rlyw, he shows baseline, but also the 20%, 35%, 65% and 80% forecasts.  I think that presentation really does help.

As I have said before, you have to define what you mean by “forecast” as Greg says above.  It is not that the fans are necessarily wrong.  It is that they are giving you a forecast, given that “so-and-so stays healthy and plays most of the season.”

Marcel and the rest of the forecasting systems include the possibility of a player getting hurt and not playing for some of all of the season, which, especially for pitchers, can be a significant factor in the forecast.  Fans don’t think that way, and there is nothing wrong with that.  If the fans forecast Mariano at 40 saves and he doesn’t play or plays half the season, they don’t consider that a failed forecast.  They consider that, “Well, he got hurt.  Of course the forecast was ‘wrong.’ Had he not gotten hurt...”

So it is more about defining what you mean by a forecast than bad forecasting by the Fans or even James.  If you told the fans, “I want you to include the possibility that so-and-so gets hurt and can’t play for X number of games,” they might have a different forecast, although that concept (of including chance of injury in a forecast) is a hard one for them to wrap their arms around and even harder still for them to translate that into a “number.”

Do you mean the “Breakout”, “Collapse”, etc. groups?  Because those don’t seem quite enough to me. 

I think some sort of grid that crosses a playing-time prediction model with a performance prediction model would be great.  Show how likely each is, show a typical stat line for each block, and then shaded in the block deemed most likely by one’s projection system.  Then interested fans could clearly see upside from more playing time and/or better performance, and downside from poor performance or lost playing time due to either weak performance or injury.

There would no doubt be some interactions, too, as poor performance might lead to reduced playing time, while strong performance could increase playing time for a part-time hitter or a reliever, or someone not originally expected to be in the rotation…

Put grids for the whole pitching staff on one page, and you could even show the dependencies between players, as a downside risk for playing time on your ace pitcher would be reflected in an upside likelihood on playing time for a lesser pitcher on the staff.  You could even figure a way to display the likelihood of different leverage levels for pitchers based on the whole playing time model…

Hopefully I didn’t just exactly describe some fantasy baseball guide everyone but me has read - I suspect some of these ideas are already out there, but I know I haven’t seen a 2-dimensional grid crossing playing time with performance.... yet.

28 saves is almost certainly not the appropriate over/under for Rivera, though. After all, Marcel only knows about a pitcher’s save totals from the past three years—it does not know what his role will be in the upcoming season. There’s a ton of turnover among closers (see http://bookofodds.com/Daily-Life-Activities/Sports/Articles/A0144-Few-Baseball-Closers-Return-for-Seconds), so Marcel’s saves projections will obviously tend to be conservative for those who actually stay in the closer slot. We know that if Mariano is healthy, there is a 99% chance he will close for the Yankees all season. That is not the case for many of the players on your list. Therefore, with the knowledge that Rivera is the Yankees closer, his projection should be higher than Marcel says—probably on the order of 33 saves or something.

David/7: I agree with you and for exactly the reason you stated, which is why I said:

So, exactly where would Rob Neyer and Allan Barra put Mariano Rivera’s 50/50 line, enough that they’d take bets on either side of that line?  It sure as heck is not going to be 44 saves.  They’d get at least 3 times as much under action as they’d get over action.  I agree that 22 saves is pretty low. Somewhere in the high 20, low 30s is right.

Rivera’s median saves in his 13 closer seasons is 40.  Saying 33 sounds perfectly fine to me as an over/under.

We have seen in the past that some pitchers with really ugly ERAs can achieve rather high save totals if they are given enough opportunities.

Estimate how many wins this team should win, then how many team save opportunities there should be. Do a depth chart and split up those opportunities among the available relievers.

Even the worst pitchers can hold their lead 70% of the time (Sv+Hld)+(Sv+BS), the best 95%+. Do an analysis of ERA vs Hld%. Then multiply by opportunities.

(numbers made up)
Chacon converts 70% of 40 opps, gets 28 saves with a 6.00+ ERA. Rivera gets 90% of 50, for 45.

Predicting saves is kind of like predicting RBI’s.  RBI’s of course are dependent on opportunity (ROB’s, and runners in scoring position) which is somewhat dependent on position in the batting order and quality of offense, and the hitters HR totals which is dependent on park and PA. 

Saves are dependent on number of opportunities, as well as the distribution of 1, 2 and 3 run save opportunities, and usage (pitching more than 1 inning and coming in with ROB)

With the Yankees offense and projected wins, I would have to imagine Mo would be projected to have a high number of save opportunities, with the distribution of saves more toward the 3 run save, which even lousy RP’ers pitchers can be expected to bank.

He had 46 save opportunities last year, even if his save rate drops to a poor 80%, if he gets the same number of opportunities, he is good for 36.  His save rate would have to be below 70% to save less than 30 games.

I would not bet on Mariano getting 44 saves, but I suspect he will be closer to 44 saves than 22, or even 28 saves.

Only way Rivera gets 22 saves is if he blows out his arm and does not finish the season, or loses his job due to poor performance (which would likely be due to arm trouble), or if the Yankees lose 100 games.

What you guys are missing here is the 3.53 era that pecota is projecting.  I think Neyer is questioning that just as much as the save total.  Rivera has NEVER had an ERA over 3.15 since he became exclusively a RP in 1996.  3.53 is an absurd estimate IMO for his projection as Rivera has had more seasons under 2.00 than over 3.00 as a RP.

Interesting - CHONE is at 2.70 for Rivera. ZIPS was 3.1 at the start of the season. So 3.53 does seem a tad high.

Anyway ...

Part of the issue here is that the number of 40 yo HoF-quality pitchers is tiny. And quite a few of that sample, the sample that Mo is being compared to, will see a sharp year-to-year spike in ERA, which is why Mo’s historic numbers will be regressed so much. We know PECOTA is heavy on comps, right?

One of the issue with any mathematical forecasting system is that at the extremes it is more liable to fail.

If an alien landed on this planet and looked at all similar pitchers to Mo (if there are any!) and then looked at Mo’s track record perhaps they’d pick 3.4 as the 50th percentile.

I’m guessing, but systems like CHONE/ZIPS which don’t really use comps suffer less from this issue (if it is an issue - I’m not sure it is - it is an equally valid way of looking at how his 2010 might evolve).