THE BOOK--Playing The Percentages In Baseball

Monday, November 16, 2009

VORP v RARP

Baseball Prospectus has two key signature stats for offense: RARP developed by Clay Davenport and VORP developed by Keith Woolner.

Exactly what is the difference between the two? 

At their core, Clay’s metric is Linear Weights-based, and Keith’s is a (theoretical team) version of (the basic version of) Runs Created. On the one hand, Clay’s got the better core piece (Linear Weights) while Keith has the better theoretical construction (team runs, with and without player).  They have the additional nuance of not handling each position the same.  That’s alot of sausage making, In the end, after all that processing, are we left with two of the same meats, or are we left with an Italian sausage and a Kielbasa?  Or worse, is one just a hot dog?

The first step is to get the data, and the revised version of the BPro stats page makes that super easy.

The key columns we are interested in are RARP and VORP.  Before we get to that, a note regarding RARP.  The sum total of all RAR (that’s runs above replacement, regardless of position) is 5767.  The sum total of all RAP (runs above position), which theoretically should be 0, is actually -693.  The result is that the sum of all RARP, which should match RAR, instead comes in at 5029.  The sum total of VORP is 5363.  They are roughly in the same ballpark in terms of scale.

First up, let’s exclude pitchers.  The sum total for all other players’ RARP is 4817, and for VORP it’s 5362.  Per 700 PA, which is roughly a full season’s worth of one ballplayer, we get 18.6 for Clay and 20.7 for Keith.  While I’m closer to Keith, both are reasonable and justifiable.  You just have to do a mental adjustment in your head that you should expect an extra 2 runs for Keith’s players than Clay’s.  No big deal.

Now, let’s see the breakdown by position.  And since BPro is good enough to provide primary positions for all their players (including DH, PH, etc), we go with that.

The average 1B is +28.5 runs with Keith, while he’s only +20.9 with Clay.  Remembering that we expect a 2 run gap to begin with, there is still an extra 5.6 runs advantage for Keith’s 1B.  That is, Keith has a lower replacement level for 1B than Clay does, enough that those guy gain 5.6 runs.  On the other end are catchers, where Keith is at +13.9 runs while Clay is at +19.7 runs.  After considering the 2 runs we talked about, that’s an 8 run difference in catcher evaluation. 

For the 8 fielding positions, Clay is pretty much centered around the same number, as all positions are between +18.4 (LF) and +21.8 (RF).  Keith, as noted, makes a huge distinction with 1B and C, and he explained his reasoning a few years ago in an excellent article in the back of a BPro Annual (2004 or 2005).

Another note: the stats pages provides the data by primary position.  Whether these two guys are using the primary positions, or the actual positions played, I don’t know.  This would be another potential source of difference.

Let’s go to the DH, which is a huge source of difference.  The average DH for Clay is +8.6 runs per 700 PA, which seems, at first, absurdly low.  Keith has the average DH at +26.3 runs, which is close to his 1B totals.  Given that we are talking about identical players, how can we get a 16 run difference (after the 2 run adjustment)?  Well, Clay may be including a “fielding” component to DH.  Since DH don’t have fielding stats, the average DH would come in at “0” runs above the average DH for his “position”.  Clay realizes this, and so, decided to nip it in the bud by giving a 16 run penalty from the outset (a number very close to what I use).  The net effect is that Clay DOES allow you to add in, say, UZR to any player’s RARP to get a “total” value (while adding 0 for the DH), and make all these players comparable. Keith’s metric doesn’t make it so easy.

The PH for both are comparable.

The largest point of disagreement between the two is pitchers-as-hitters.  Clay gives the pitchers +24.7 runs per 700 PA (comparable to all the other positions), while Keith has obviously made a conscious decision to give them a different baseline, as his VORP is +0.1 per 700 PA (essentially 0).  I do it the way Keith does it.  Clay’s version will be unduly biased toward NL pitchers.  Not to mention that he has the potential of double-counting the replacement level for pitchers.

Ok, all that was fun.  But, for practical purposes, say in discussions for MOP (Most Outstanding Player), what’s the net effect?  If you’ve been following, you see that Keith likes 1B, DH and doesn’t care for C.  So, we should see some big differences along these players when we compare to Clay.  Here are the biggest outliers:

These are the guys that Keith likes more than Clay:
diff POS NAME
22.8 1b Miguel Cabrera
18.8 dh Adam Lind
17.7 1b Kendry Morales
16.9 dh David Ortiz
16.7 rf Ichiro Suzuki
16.6 dh Jason Kubel
16.6 c Joe Mauer
16.5 1b Billy Butler
15.8 dh Vladimir Guerrero
15.5 cf Jacoby Ellsbury

That’s a huge difference.  Miguel Cabrera is considered 23 runs better in VORP than RARP (or 21 runs after the 2-run adjustment).  Mauer, it should be noted, was both a DH and C.

And these are the ten guys that Clay prefers to Keith:
diff POS NAME
-11.9 c Jason Kendall
-11.2 c Russell Martin
-8.9 1b Adrian Gonzalez
-7.7 c Gerald Laird
-7.5 ss Everth Cabrera
-7.2 c Geovany Soto
-6.8 rf Andre Ethier
-6.7 c Dioner Navarro
-6.6 2b Dan Uggla
-6.6 c Nick Hundley

As you could have figured, Keith’s not crazy on catchers.

Even Albert Pujols is scathed, as he’s either +98 with VORP or +86 in RARP.  There may also be an issue as to how the IBB is treated.

So, the RARP v VORP is a combination of good/bad choices as well as a philosophical choice.  It’s not one where you can say “well, I’ll just take a bit of both”.  If you don’t care to learn the difference, I suppose you could be both a Catholic and a Protestant, or even a believer and an atheist.  Otherwise, ask Baseball Prospectus to make up its mind already, and choose a religion (or create a new one, which takes the best of both, and can be done with limited effort).