from Amazon
Tuesday, February 23, 2010
The Angels, Pecota, and some BS from BJ
There is a post on TBA by Rich Lederer which discusses the fact that Pecota has under-projected the Angels season w/l totals for 6 or 7 years in a row. In that post, Rich quotes Bill James from his 2010 Handbook. The bolding is mine.
More after the jump...
The most efficient team in baseball is usually the Los Angeles Angels—anyway it was in 2009, and it was in 2008, and it has been in other years. The Angels do little things so well that they are consistently able to grind five or ten more wins a year out of their team than what one would think was available. We don’t really understand how they do this, to be frank, but since they do it every year, we know it’s not luck. Saying that they “do the little things well” is just a way of covering for the fact that we don’t actually know how they do it.
If it wasn’t for the Angels, we might think it was all luck. There are a couple of parts of the Angels’ success that we do understand. For one thing, they run the bases extremely well. They picked up about 96 bases last year, or about 20 runs, just by running the bases better than the average team. Twenty-two of those bases are “stolen base gain,” but 74 of them are bases gained by things like going first-to-third on a single or tagging up and advancing. That helps a lot. The Angels in 2009 had 221 “Manufactured Runs,” by far the most of any major league team. Second, they usually have a good bullpen, which means that they can put a good pitcher on the mound when the game is close. Even in 2009, when they didn’t have a really good bullpen, they also didn’t have a really bad bullpen. Those things help to make a team “efficient,” as we are using the term.
Let me start with James. Sometimes I wonder how such a smart guy can write such dumb things.
“We know it is not luck.” We obviously don’t know that it is not luck simply because it “happens every year,” and a smart and responsible sabermetrician would never say something like that. Now, it is likely that there is some bias somewhere, but the question is how much is likely “luck” and how much is likely bias. I use the word “bias” to mean that there is something about either the team or the Pecota methodology (not necessarily unique to Pecota as a forecasting model), or both, that is causing them to under-rate the team in the pre-season.
As for the team’s baserunning runs, why does James assume that that is not included in the Pecota model? I am pretty sure that it is, or at least it should be. And if a team was 2 wins runs above average one year, what would you expect blindly the next year, especially given that not all the same players will be on the team (although part of that is probably a coaching thing)? Maybe 1 win?
As far as the 221 “manufactured runs,” I don’t know the definition of that, but is that even a good thing? Does that mean that a team with lots of these kinds of runs will outperform their traditional projection, using just offense, pitching, defense, and baserunning?
Finally, as far as the bullpen is concerned, first of all, it is only the short relievers and presumably the better relievers in the pen who have higher than average leverage situations. On average, I think a bullpen pitches in around an average leverage situation. Second of all, again, I would assume that Pecota, like most serious team forecasting systems, uses leverage when it figures out the value of a pitching staff. For example, I assign a closer a 2.0 LI and the set-up man, a 1.5 LI. Really, how a manager can influence his bullpen’s affect on his team’s w/l record is to make sure that the best pitchers (before the fact - not after) pitch in the highest leverage situations and the worst relievers are saved mainly in junk time. Do we know that the Angels do this particularly well?
Just an awful quote from James, in my opinion, and for the above reasons.
Finally, I will admit that the Angels have a significant (non de minimus) chance of doing something significant that results in just about every forecasting system under-rating them pre-season. There is also a significant chance that there is nothing going on and that they have just been lucky. Again, the key question is how much to “tweak” a system’s (like Pecota) projection. Personally, I would not be comfortable with more than 2 wins or so. 6 or 8 wins would be ridiculous, in my opinion. What about all the other teams? Shall we start to tweak everyone’s projection based on the differentials (difference between projected and actual wins) of the last 5 or 6 years? Moreso for teams with the same manager? Next thing you know, you are tweaking everyone’s record. Maybe that’s is not a bad idea, but that tweak should surely be A LOT less than the average differential over those 5 or 6 years. Maybe the correct regression formula is the Pecota (or other system) projection plus some weighted average of the last few years actual w/l record? I don’t know. Some research would have to be done, but again, I suspect that past records are not going to be a particularly significant addition to a forecast system’s w/l projection. If you are trying to capture the manager’s influence on a team that does not show up in any stats, how much can that reasonably be? 1 win? 2 wins at the most?
Finally, let’s not forget that we have an extreme case of publication bias in discussing the Angels anomaly and Pecota. We have lots of projection systems, lots of years, and lots of teams. What are the chances that one team in one projection system is going to be over or under rated every year by a lot? Probably not as small as you might think.
I read about this guy who got hit by lightning. Then again, I figure the chance of that happening (that this guy would get hit by lightning) is a million to one or so, so I figured the story must be phony…
mgl,
I could be wrong, but I think the “efficiency” Bill James is talking about concerns actual team wins and expected wins based on the offense and defensive performance (broken down categorically: walks, home runs, stolen bases, hits allowed, etc.). I don’t think that quote has anything to do with PECOTA.
Are you confusing that quote (posted by Lederer a few months ago) with the table posted 2 days ago comparing Angels actual wins and PECOTA’s projection?