THE BOOK--Playing The Percentages In Baseball

Tuesday, December 02, 2008

RARP v VORP, take 2

Three months ago, I did a comparison of RARP and VORP.  I presented my findings to Clay, and he said he was going to make at least one change (as it relates to pitchers-as-hitters).  I don’t know if he did any other changes, yet.  (As I right these words, I haven’t looked at the data.) You can click on the above link to get all the particulars.  I made no changes to everything I’m about to analyze, other than to update the data.

Let’s get on with it:

VORP v RARP

BP has two measures that do similar things.  One is called VORP, by Keith Woolner, and another is called RARP, by Clay Davenport.

The total VORP for 2008 is 5352 runs.  And for RARP, it is 5257 runs.  This is based on around 187631 PA, which means that per 700 PA, we have this:

VORP: +20.0 runs
RARP: +19.6 runs

As you can see, both have a very similar replacement baseline.  Indeed, this is a very common baseline.  MGL uses +19.4 runs, and I use (roughly) +19.8 runs.  We can feel very confidant that overall, we’re talking about the same currency and the same scale.

The question is if either measure shows much difference.  This is the 10 guys that VORP likes more than RARP:

The concern is on the high-end guys here.  Ramirez, Berkman, Pujols, Holliday, and Youkilis were all MVP candidates, and these two measures sees them as 10 to 15 runs different.

And these are the guys that RARP likes more than VORP.

The issue is best exemplified here:

RARP sees Mauer and Berkman as being 1.7 runs apart (with Mauer being better), while VORP sees them 15.4 runs apart the other way (Berkman better).  That’s a 17.7 run difference.

I think it would be nice for someone at BP to respond to why we see such an enormous gap here, and which numbers we should trust.  Or at least give us enough information for us to decide.

Here are the positional totals:

We see an enormous gap among catchers and pitchers, of which RARP really loves, while VORP is much more high on 1B and DH. 

We’d like to see the runs over replacement to be somewhat similar across all the positions.  They don’t need to be identical, as it would imply things that are not true, but neither can they be so far different from each other that it would lead to absurd results.  While RARP paints the even-ish picture, VORP does not: 1B have over twice the value of catchers.  This explains why we have such a difference between Mauer and Berkman. 

I don’t know why BP writers are using the VORP numbers, when the more reasonable RARP numbers are never cited.  Will someone at BP explain this?

Also, as other readers have pointed out: how are IBB treated?  As regular walks?  Or something else?

***

The range of pitchers according to RARP is Zambrano (+12 runs) to Bergman (-3 runs).  VORP’s range is +17 for Zambrano to -6 for Bergman and Jimenez.  Thats’ 24 runs (after rounding) for VORP to 15 for RARP.  Which is right?

Let’s look at Linear Weights, via Pete Palmer, via b-r.com, for pitchers with at least 70 PA, making the leader and trailer for VORP and RARP as Zambrano and Jimenez.  The gap is +13 runs using RARP and +24 runs using VORP.

Palmer says Zambrano is +2 runs above average and that Jimenez is -17 runs.  That range is 19 runs.  Well, that doesn’t help us!  Maybe Zambrano is too much of an outlier.

Looper, Haren, Arroyo, Cook are the next leaders in both RARP and VORP (at least they agree on that), with RARP at +6.5 for them, and VORP at +6.0. 

At the bottom, if we look at the bottom 7 for each, we have these six common pitchers: Jimenez, Sheets, Myers, Lohse, Billingsley, Pelfrey.  That’s an average of 0 for RARP and -4 for VORP. 

So, the gap is +6.5 for RARP and +10 for VORP. 

What does Palmer say?  The top 4 averaged -5.7: Looper (-3.2), Haren (-7.2), Arroyo (-5.8), Cook (-6.6).  The bottom 6 averaged -13.6: Jimenez (-17.1), Sheets (-14.5), Myers (-12.6), Lohse (-12.7), Billingsley (-12.6), Pelfrey (-11.8).  That gives us a gap of 7.9 runs.  Once again, that doesn’t help us either as Palmer is right in the middle.

Let’s try to infer the replacement level for each.  The top pitchers, after Zambrano, were +6.5 RARP, while Palmer said they were -5.6 RAA.  That sets the replacement level as 12.1 runs.  The bottom pitchers were 0 for RARP, compared to Palmer’s -13.6 runs: that sets the replacement level at 13.6 runs.  So, Clay sets the replacement level at around 13 runs, for around 75 PA.

The top pitchers were +6.0 VORP, which sets the replacement level at 11.6 runs.  The bottom pitchers were -4 VORP, which sets the replacement level at -9.6 runs.  So, Woolner sets the replacement level at around 10.5 runs for 75 PA.

With 6000 PA for pitchers, this 2.5 runs per 75 PA gap will lead to 200 runs difference between the two measures.

All to say that I really don’t know which one is better.  I can see what Clay did, and I think I’d lean towards his numbers.