from Amazon
Wednesday, December 05, 2007
Baseball Prospectus and Sabermetrics
BP is one of the most notable and manistream sabermetric sites. I happen to like it. It has its weaknesses (don’t we all), but I think it is generally informative and valuable, if not occasionally groundbreaking. I also think Pecota is one of the best and most innovative forecasting systems publically available.
Joe Sheehan is one of their most illustrative writers, I think. I have talked to him on several occasions and he is a good guy. He wrote this today about the recent Guillen trade. To be fair, I am taking these quotes out of context:
Guillen’s performance has actually been consistent, and consistently good.
You hear comments about “consistency” all the time from the mainstream media, usually in reference to it being a good thing. Acutally it is often used as a proxy for consistently “good” or just plain “good.” First, I don’t think that any player has any control over his consistency or that it is a repeatable skill, so why even mention that when talking about a player’s value in the future or analyzing a personnel move involving that player? And even if it were worth mentioning, is there any extra value to an offensive player being consistent? I doubt it. Obviously it is more important for a player to be “good” whether consistent or not. Sometimes (often actually), you would think that the mainstream media and baseball insiders would somehow prefer that a player be consistently mediocre or average than inconsistently good. Obviously you would prefer the latter. Not to mention, “What defines being consistent or not?” I have no idea. I assume there is a whole spectrum of performance distributions that could be interpreted as consistent OR inconsistent.
Now, has Guillen been consistently good as Sheehan says? Well, his last 4 years’ offensive lwts (according to me) per 150 games were:
+10
+14
-14
+24
Is that good? Sure. Is that consistent? I have no idea. If you include 2006, which Joe wants to “throw out” (I’ll get to that in a minute), it does not look too consistent to me. Even if you throw out 2006, 2007 is a big bump from 04 and 05. Like I said, “consistency” is probably in the eye of the beholder.
And Joe did not say offensive consistency, although he probably meant that. What about Guillen’s defense?
04 -19
05 +10
06 -1
07 -19
Does not look too consistent to me. But what do I know.
I really wish that any serious analyst would NOT use the word “consistent” (or “inconsistent") in their evaluation of a player (other than perhaps in describing something that occurred in the past without necessarily implying anything about the future). If I ever do, you have my permission to shoot me (virtually of course).
Toss out 2006, which was a season marred by injuries
Let’s just toss out 2006! He may have been injured, but to just decide to “toss out” an injury-plagued year? First of all, what a player does is part of his performance history, injured or not! Players get injured. Part of a projection includes the chance that he gets and plays injured in the future. If we only use and project “perfect health” data, well.... And who is to say what data gets tossed and what data doesn’t? That’s easy, right. If we want a rosy projection, we’ll just toss out all the bad data when we “think” that a player was playing injured. If we want a poor projection, we’ll just keep that data. I don’t have much of a problem “tweaking” data and projections based on injury histories and the like, but if you do that, you are treading in dangerous waters. And to just “throw out” an entire year (albeit only 264 PA so it is not THAT big a deal), well, I already said enough about that.
and since his 2003 breakout Guillen has played in 148, 148, and 153 games, with EqAs of .282, .283, and .285.
Two things here. One, if I ever use the word “breakout season” in an analysis, you have permission to kick me. Technically, he did have a season in 03 which was well above his career norms up to that point, but Tango, I and others have shown on numerous occasions that breakout or banner years have no more or less predictive value than any other year. When you say “breakout year” you are implying that that year is a new level of true talent and that you can more or less throw out all prior years. Nothing could be further from the truth (O.K., some things could - just listen to George Bush, Dick Cheney, et al.). In 03, Guillen was +28 in lwts. In 02, he was -3 and in 01, he was -9. Without 01 and 02, we might be tempted to think that the +28 was his new level of true talent (of course, regression toward the mean would tell us anyway that it was not). However, a +28 after a -9 and -3 is A LOT different from a +28 after a +20 and a +16, for example. Sure, players sometimes change their true talents a lot (I guess). But ALL players have the potential to fluctuate randomly from year to year like a mad dog (no reference to Maddux) or a banshee. It is no accident (well, maybe it was) that in 04 Guillen “regressed” to +10 and in 05, +15. We could have anticipated that from his 01 and 02 numbers, prior to his “breakout” (choke, cough!) season.
Second thing: Sheehan says that his EQA’s in 04, 05, and 07 were .282, .283, and .285. (Oddly, he says, “Since his 03 brealout season” - apparently he is throwing out 06 again, although he does not specifically say that in those sentences.) Anyway, since EQA is supposed to be a proxy for lwts, his lwts rate should all be around the same in those 3 years, right? Wrong! Here are MGL’s (the most accurate account of a player’s context-neutral offensive performance that I know of - the “gold standard"):
04 +10
05 +15
07 +23
O.K., those are fairly close, but as you can see, EQA, like OPS and the other “lwts proxies,” are crap when you are trying to rigorously evaluate a player. Well, maybe our park adjustments are different but equal in “value” so I’ll give him (and EQA) a pass on that one.
He is generally considered a good right fielder with a strong arm; Clay’s system gives him -16 FRAA this year, which may or may not have something to do with Ichiro Suzuki’s play in center field, but regardless, that data point is aberrant.
First, I doubt that any bad defensive rating has anything to do with “Ichiro’s play in CF” and anyway, FRAA is a goofy metric and does not use PBP data I don’t think. A metric for the OF without PBP data is practically worthless I would think. That being said, his UZR in FRAA this year was -19! About the same as Clay’s (blind squirrell - actually a good non-PBP metric for the OF will obviously come close to a PBP one).
Good arm? Arm lwts is one of the easiest things to measure. You look at base advances and assists per opportunity and compare that to the average player at that position with park adjustments and you can even do adjustments for where the ball was actually hit in the OF. And BTW, we commonly forget, but a player’s “arm” rating is a combination of the strength and accuracy of his arm (as well as his reputation) AND how fast he gets to the ball! Anyway, while in LF, Guillen had a GREAT arm rating, and that was last in 04 when he was young. Since then, he has been in right field (where the average arm is MUCH better) and he has NEVER had a positive arm rating. So yeah, he probably has a strong arm (although probably not as strong as it used to be), but it probably is now about average for a RFer.
In any case, Guillen, according to UZR, is NOT even an average defensive outfielder any more. In Dewan’s system, I don’t see him on the 05-07 trailer or leader lists. I think that Tango’s fans rate him as average. So who “generally considers him a good RFer?”
Finally, if I ever use the word “aberration” to refer to a particular season…
Seriously, some analysts use that word as if to imply that we should throw out or give less weight to a period of time in which a player’s performance is far from his established level, both before and after. Sheehan certainly does imply this. First of all, according to UZR, it is NOT an aberration (04 was the same). More importantly, again, as Tango, I, and others have shown, “aberration shmaberration.” All levels of performance gets treated equally in terms of estimating a player’s true talent or projection. Doesn’t matter whether it is close to a player’s established level or not. A lot of people don’t get this. Apparently Joe is one of them.
The price, $12 million a season, is almost reasonable in this market; just to pull a name out of thin air, Guillen has outhit Gary Matthews Jr. in three of the last four seasons, and is almost certainly a better player than the man with four years and $40 million left on his deal.
Almost certainly a better player than Matthews?? Wow! I have Guillen projected next year as 1 to 1.5 WAR. Matthews as 2.5 WAR. He is right but has those players completely backwards. The positions alone, even if they were defensively average at their respective positions are like 1 to 1.5 wins apart! How can Sheehan get this so wrong? And if I ever say something like, “So and so has outhit so and so in 3 out of the last 4 seasons,” please, well, you get the idea.
Player A
+3
0
-21
-4
Player B
+1
-2
+9
-5
Well, you get the idea again (Player A outhit player B in 3 of the 4 seasons)…
So Guillen is a better player than I thought he was, a better player than I give him credit for. Does that make this a good signing for the Royals? I’m not sure what they gain by adding a low-OBP, high-SLG corner outfielder.
Does it matter if a player is a low-OBP, high- SLG, or high-OBP, low-SLG (assuming you are not using OPS ro evaluate them), high-forehead, low-hanging...? You get the picture again.
Just adding Guillen to the Royals makes them perhaps a win better
I have no idea whether that is true or not. He lost me at “consistent.”