THE BOOK--Playing The Percentages In Baseball

A slight tangent relating to this BTF thread:

http://www.baseballthinkfactory.org/files/newsstand/discussion/beyond_the_box_score_pindelski1/

where post 60 says:

As a tool which tells us *at the exact moment* how much difference a play makes WPA is perfectly fine. As a tool to assess afterwards the contributions of each player, I remain unconvinced it tells us anything more than we would get from asking a 6 year old.

I agree with everything that is said here.  WPA is simply about attributing the change in theoretical winning percentage at that moment in time.  When Vlad hit that grand slam in the playoffs against the Redsox in the 8th or 9th inning, that was an enormous hit. 

That the Angels eventually lost is where the issue comes in.  To the WPA-adherence, Vlad was VLAD and deserves VLAD treatment.  From the just-tell-me-who-won-the-game perspective, the 6yr old will simply treat Vlad grand slam as delaying the inevitable (like say The Catch by Endy Chavez).  You can even reasonably conclude that whatever Vlad did was irrelevant, since after-the-fact they lost.

The point is that you, the fan, comes to the table with a particular perspective.  And, under one set of conditions, WPA for a given at bat is both fascinating and (aggregating these WPA is) completely irrelevant.

Dismissing the aggregating of WPA simply means you don’t share that perspective.  Neither side is wrong.

Post 87:

Like the man said, the problem with WPA is that probabilities aren’t additive. Also, just as win shares suffers from the absence of loss shares, one wonders why we hear no talk of WPS.

WPA is equal to win advancement (WA) minus loss advancement (LA).

On the Fangraphs site, WA is shown as WPA+ and LA is shown as WPA-.

The Mills Brothers would show Player Win Averages (PWA) as WA/(WA+LA).

Oh, and the probabilities ARE additive with respect to what is being discussed.

Since Putz is the one being discussed, let’s follow through here.

Putz has 46 IP which is roughly 5 full games.
Presuming a .600 closer is our baseline, such a closer will probably be +0.65 wins above replacement.

A .600 closer given Putz LI (1.80 let’s say) would be +0.9 wins above average.  Putz is, gulp, +4.7 wins above average.  So, Putz gets credit for +3.6 wins above closer, plus the +0.65 wins above replacement for a total of +4.25 wins above replacement.

ARod is +4.5 wins as a hitter, plus a bit more for fielding.  He gets no change for his position (3B).

ARod still gets the MVP, but Putz is definitely close.

anyway- using BPRo’s “lev” stat

Ugh.  Why would anyone use this stat?  Fangraphs displays LI perfectly fine on their site.  Why choose something clearly inferior?

BP: I’m begging you, please, stop the insanity!  Don’t confuse your readers.

anyway, the real problem with WPA is not that it assigns greater value to one grand slam over another, it’s that it assigns greater value to a two run walkoff single than a first innning grand slam. If you design a stat that gives a two run single more value than a grand slam, it’s time to start over.

I’m not sure that it does.  I don’t have my charts handy, but we can use this for now:
http://winexp.walkoffbalk.com/expectancy/search

Home team has a .540 chance of winning.  If they give up 4 runs, they are down to .207.  That’sa .333 change.

If it’s the bottom of the 9th, runners on 2b, 3b and say 1 out, down by 1, home team has a .551 chance of winning.  The single (or double or HR) gets you to 1.000, or a +.449 change.

Ok, so the single is worth more here.  But, that’s because that’s what it is.  At that moment in time, the single is worth more than the grand slam.

Another post:

By WPA 7 of the last 12 Cy Youngs would go to relievers. Does this seem reasonable? It doesn’t strike me as reasonable.

And of course, that’s because the baseline being compared against is wrong. 

***

Dave Cameron is right that the MAriners have real wins.  We are simply trying to allocate those real wins to players and not put it in the “luck” bin, just like a monkey can pick MSFT 20 years ago, and be a millionaire.  It may be luck, but he’s got a million bucks up his tree.

***

Question: if you knew what JJ Putz would have done this season (and presuming he continues to do what he does), how much would you pay for that performance?

You may not like that he’s only pitched 46 innings, but, how much would you have paid for him?  Would he be one of the 5 best-paid pitchers in baseball (given that you knew what you’d get)?  How you answer that question gives you the perspective you need to follow.

Tango, I wanted to thank you for this thread.  I’m a WPA detractor, I suppose, and I posted negative comments about the stat in the WPA thread but I never understood why people can get so rabidly pro- or anti- this stat.  It’s a matter of perspective and there are good arguments for why that perspective is right and why it’s wrong. 

(Even as a WPA detractor I’d still have Putz no lower than 5th on my MVP ballot)

Ror/8: it seems that half of the posters at BTF have taken one position and will argue that position, much like lawyers would in a trial. 

It would seem to me that it would make a better debate, and one that would give us a resolution, if both sides would make factual claims, acknowledging what is true and what is not, and then stating their opinion as to why some facts are more relevant than others.

***

Post 131:

The solution is to formulate a statistic that still has the desirable zero sum properties of WPA without the distortion of context due to incomplete information at the time of the game events. In other words, the state would examine the outcome of the game and distribute credit and blame appropriately among participants. This would continue to place each player’s contributions in the context of actual wins and losses while incorporating all of the information the we have about the outcome of the game.

This is what David Smyth (creator of BaseRuns) did with his AWP.  He basically gives out wins to the winning team’s players, and losses to the losing team’s players in a reasonable fashion.  The VLAD game for example, our main man would get no portion of his team’s losses, while whoever went 0-fer would get their fair share.

Post 136:

I think we can learn some things from WPA, and that the WP framework is an amazing tool for the evaluation of strategies. But I’m highly suspicious of using it, even to describe “what happened”, due to the above, for hitters, and also for pitchers, with the added reason that, as mentioned in a previous post, ...

There is no question that a proper Win Expectancy implementation is exactly what you need to evaluate in-game strategies.  That the change in Win Expectancy (WPA) tells you exactly what is happening at that moment.

The only issue is whether aggregating the WPA on the hitter and pitcher level has any value in-and-of-itself.  Clearly, given enough PA, the best hitters will have the best WPA, and the best pitchers will have the best WPA.  The career leaders in WPA, LWTS, etc are mostly the same.  The reason however is because alot of the leveraged-aspect has been wiped clean, so that, in essence, a player’s performance was randomly spread across the various leverage ranges.

So, it comes down to whether you believe that aggregating WPA for a game, or even a season or two, has any value.

And is the reason you don’t see the value because of the leveraged aspect for relievers; and is the reason because of the average baseline being used, instead of some sort of replacement.

...there is no room in WPA as currently presented for the contribution of fielders. If we have learned anything from the DIPS debate, it’s that fielders can have a huge impact on a pitcher’s numbers—and yet our current WPA presentation ignores this! Of course, it is difficult to measure, that’s not a slam on anyone; but the WPA figures we have now for pitchers (and position players, as fielding is not credited or debited to them, either) are incomplete, even if one accepts the rest of the framework.

The Fangraphs implementation doesn’t have fielding.  But, it’s very easy to create an implementation for fielding if you have the data.

However, I have done WPA when I give the pitchers 100% of the credit for balls in play (BIP), and when I give the pitchers 0% of the credit for BIP.  Guess what?  You get a virtually identically ordered list.  DIPS anyone?  So, it doesn’t really matter.

***

By the way, I forgot to add another win to ARod for the replacement scale.  That puts ARod at +6 WAR and Putz at +4.25, both above replacement using WPA.  It’s not even close, but the only reason is because ARod is having a season for the ages.  That aside, Putz is definitely in the second class of MVP players (and remember, the first class only has ARod in it at the moment).

Post 137:

But “probability” itself is hypothetical. It’s not an actual victory; it’s a possible one. One can’t actually win x% of a game; one either wins or one doesn’t.

A single probably has more value than a walk.  Sometimes they have an identical value (bases empty).  In fact, in this case, a walk has more value, because a single is sometimes a single-out-trying-to-stretch-into-a-double.  And a walk with the bases loaded has probably more value than a single with no men on.  Everything is about probability.

A bottom of the ninth inning, bases loaded, tied, a HR and a walk are actually (not probably) identical.  I don’t see anyone get worked up about giving 4 total bases for the HR in the records.

It’s all about probability.  The reason that we can say that a HR is worth 1.40 runs is because that’s what the average change in run probability states says it’s worth.  But, the HR is also worth .1234 wins, because that’s the average change in WIN probability states.

If Tim Raines gets on base all the time, and is always grounded into a DP, then in reality he did nothing for his team.  He may have as well grounded out.

And if the Yanks don’t make the playoffs, then in reality, ARod’s performance was useless.

But, in probability, if you believe that the evaluation of performance should be based on whether a player gives his team a *chance* to win, then what Raines and ARod did in these illustrations is very valuable.  In reality, they did give their teams a chance to win.

Post 142:

My problem is that you can’t directly convert these into won-loss records.

That is his problem.  They are not supposed to be directly convertible into W/L records, anymore than OBP, SLG, or OPS are.

Post 143:

isn’t it true that pitching a scoreless ninth against Boston’s 3-4-5 hitters is a heck of a lot more valuable than pitching a scoreless ninth against KC’s 7-8-9 hitters, given the same game score? It bugs me that WPA makes no attempt to account for this…

Ror, read a recent blog entry here about “preallocation” of wins.  WPA DOES make an attempt to account for this.  You need to distinguish between the WPA framework, and various implementations of that framework.

Post 150:

And, anyway, the real question is, do you think a player who hits a decisive single in the ninth to give his team a 2-1 victory should get more cerdit than a player who hits a decisive single in the first of an eventual 2-1 game? WPA does, and that’s why people have a problem with it.

Every fan in attendance actually watching the game does in fact give more credit to that ninth inning single.  You can measure it by their cheers, their applause, their heart rate, and however else you want to measure their feelings.

And they still feel this way in retrospect.

It is only the fans who did not watch the game, and who look back on it months later that don’t make this distinction.

That’s why you have two perspectives, and you have to choose the one you want.  It doesn’t invalidate the other perspective.

Ugh.  Why would anyone use this stat?  Fangraphs displays LI perfectly fine on their site.  Why choose something clearly inferior?

BP: I’m begging you, please, stop the insanity!  Don’t confuse your readers.

Help me out here.  They look pretty similar to me.  I’ve always used LI in the past but is there a difference in implementation (which isn’t particularly obvious from the glossary) that makes LI superior?  The only thing that jumps out is that lev appears to be a reliever stat. . .

Post 158:

That doesn’t make the performance of high leverage relievers less valuable - it makes them more volatile. A winning powerball ticket is worth $100 million or whatever that specific jackpot is, regardless of the fact that winning powerball is about one of the flukiest things that could ever happen to someone. The tangible value of a winning ticket doesn’t change because of the surplus of losing tickets.

Right.  That’s the same as my monkey picking stocks example.  We are simply distributing portions of actual wins and losses to players.  If Mariano Rivera is shutting down the opposition more then the average whatever, then he’s going to get his fair share of that win.

On the other hand, if you simply see Rivera’s 100 complete games as nothing more than 100 games of say .800 pitching, say the equivalent of an 80-20 pitcher (Pedro was 82-21 from 1999-2003), or simply being +33 wins above replacement (replacement level pitcher is .470 as a reliever), then fine.

Then you have to ask why is Mariano not being inserted randomly?  The answer is because of the probability of the impact of his performance.  That has to mean something.

***

Don’t forget that Ozzie Smith doesn’t decide how many balls get hit to SS.  But, his fantastic talent puts him in a position to see far more balls at SS than at 1B.  Same for Mariano. 

Are you going to remove 25-35% of Ozzie’s performance at SS because it’s “unfair” that he got to field more balls even though he can’t control how many balls are hit to him?

Guy/159:

Yes, he hit his HR at the “right” time (w/ men on base). But didn’t A and B also draw their walks at the “right” time (i.e. in front of a HR)? Surely A and B couldn’t know a HR was coming, but WPA defenders say that it measures value even if ‘clutch’ performance is not a skill. So, what players knew or could have known is irrelevant to value. If neither one of these fortuitous timings is a real skill, and both really do bring more value to the team (a BB in front of a HR clearly has more value than other BBs), by what possible standard do we assign all of the credit for this fortunate sequence to the last guy in line?

The standard is that this is what actually happened.  Imagine that a reliever was brought in after the first two guys got on base.  And that reliever hits the HR.  Bad timing on the new relievers part, but even with two men on, he shouldn’t be allowed 3 runs to score after 1 batter.  That reliever is subsequently removed, and a new reliever comes in to retire the side.

If you are going to debit the second reliever an enormous amount of wins for being Jeff Weavereque, you’ve got to give the exact same credit to the guy who did the damage.  A guy sells a million dollar debt-free house for 1$ to somebody means that the guy who happened to be standing in line when the moron does this gets the credit, even if he did nothing than other be in the right place at the right time.

It doesn’t measure him as a person, or his skills.  But, he’s got a million bucks of assets to his name.  And that’s real.

Post 169:

There’s definitely a decision making component present for a hitter when he’s at the plate. In that sense, if we are trying to evaluate the hitter’s decisions, it makes sense to incorporate only the information that the player had at the time of making the decision. WPA is fine, if that’s what we’re trying to do.

I am disappointed that no one talks about WPA/LI (that is WPA divided by LI) since this is *exactly* what we care about, with the leveraged portion deflated.  If there’s a runner on 3B and less than 2 outs, and a batter strikesout, that’s a *huge* enormous out.  Moreso if the game is real tight. 

Only WPA/LI will tell you that.  (That is, Linear Weights by Game State.)

Not a single peep in the whole thread.

(MGL has argued elsewhere on this site that if the batter can’t really control this, then we shouldn’t care about this either.  I believe that the batter and pitcher are aware and they are pitching/hitting to that.)

I’ve always thought that the best way to determine MVPs would be -

(park-adjusted((WPA/LI)ORP))+/-defense

Takes care of clutch, positional vagaries, etc.  Murray Chass has shown interest.

azru/14:

Read part 2 here:
http://www.insidethebook.com/articles.shtml

They are not at all similar (other than the name).  There’s no one at BP that would dare challenge me on the relevancy of their LEV.  Heck, I’ll even double-dare them!

"The standard is that this is what actually happened....  It doesn’t measure him as a person, or his skills.  But, he’s got a million bucks of assets to his name.  And that’s real.”

We all agree this isn’t about skill, so let’s leave that aside.  The question is what’s “real.” The two guys who had the clairvoyance to draw their walks at the correct time are just as “real.” Their BBs “actually happened” and they actually scored runs that helped their team win.  They, too, were (literally) “in the right place at the right time.” So why won’t you give them some of the million bucks?

Well, I tried to help, but it’s degraded into a typical BBTF ranting board.

Guy/20: because at the time they got their walk, they didn’t know they were about to score.  This accounting is soley based on the presumption that our expectation of future events will happen based on a set probabilistic distribution.

If that presumption is not agreed to as a matter of perspective, then of course WPA is hogwash.

Again, it’s just a matter of deciding first what perspective you want, and then follow through on that.

***

That aside, why is no one championing Linear Weights by Game State (WPA/LI)?

Agreed.  But at least there were some interesting posts (on both sides)—even a few by “weekly journalist” before he melted down.

I was trying to agree with Studes/21.....

I’d like to ask a few questions now:
1. If you were guaranteed Mariano Rivera for 6 years at his prime, 81 innings, ERA+ or componentERA+ of 200 (i.e., allows half as many runs as the league average), how much would you pay for him each year?

2. What kind of ERA+ would you expect from your #1 or #2 starter (225 IP per year for 6 years) to pay him the exact same amount?

So, I’d really like to see where I’m wrong here, but I am relatively sure that this is just the biggest fallacy currently found in all of sabermetrics. 

You’ll have to forgive me, as I’m not familiar with the BBTF thread, but even the argument proposed in your note above is circular, Tango.  You’re stating that since everyone thinks that relievers are more valuable, they actually are.  Now, if we’re talking about value in terms of, say, market value, that’s true.  However, I’m relatively sure that this is a huge inefficiency in the market.

Runs scored in the first inning absolutely do have the same value as runs scored in the 9th.  This can’t even be argued.  The fact that WPA ignores its complete contradiction with this is a sign that it needs to be taken with a grain of salt.  It’s an excellent tool for looking at a game and seeing who contributed in a particular situation, but a horrible indicator of value (much less a predictor of future value).

As for unfairness, your comparison to a SS vs. 1b argument is a straw man.  A SS does see more balls, make more plays, etc. than a 1b.  A closer does not get more outs (the only thing that matters for a pitcher) than a starter, and to just assume that leverage and WPA are meaningful when that is the subject of argument is silly.

I am the one fan that believes a run is-a-run is-a-run.

Sweet Moses. 

I’m glad I’ve never bothered with the lev stat previously.  Thanks tango.

I am the one fan that believes a run is-a-run is-a-run.

I agree with you after the game is over, but not during the game. When a run is given up in the first inning, there is still time to come back.  When it is given up at the end, there isn’t.

Why else would managers save their best relievers for the end of a game?  Why not use them immediately after the starter leaves?

Water/26: if you believe that there is an inefficiency in the market, insofar as how relievers are being paid (specifically closers), please answer my questions in post 25, and let’s continue.

***

The SS faces about 1.3 time more balls than the average position.  The ace reliever faces situations that are 1.8 times more impactful than random.  My argument in this example is that neither control the environment, other than their sheer talent allows them to see that environment.

Why else would managers save their best relievers for the end of a game?

Because they are foolish?

To be more serious - I believe it is because they want to be able to use that reliever later.  This is only another reason that the starter, pitching low leverage innings, is *more* valuable.  They are not misused out of fear of future situations.

I think the only time the argument about having time to re-take a lead comes into play is if one believes that there is some huge psychological difference in approach based on game situation.  I don’t happen to think that’s the case.  Or, if it is the case, it should almost always not be, and it is another example of foolish decision making (e.g. excessive hit and runs, sac bunts).

An excellent article by Rob Wood several years ago:
http://www.baseballthinkfactory.org/files/primate_studies/discussion/robw_ood_2003-02-04_0/

He ran simulations using ace relievers in various roles to see how much impact they can have, dependent on various game states.

Further to my post/4:

If the Seattle Mariners had the typical closer on the team instead of JJ Putz, they would have won about 3.6 less games than they actually did.

If you take this statement as fact, can’t we accept that Putz performance was worth +3.6 more wins than the average closer’s performance?  If not Putz, who else gets the wins?  Like I said, the Mariners would have won 3.6 less games with an average closer.  They bring in Putz instead of the average closer, and boom, they win 3.6 more games.

To me, it seems logical to attribute those +3.6 wins to Putz.

Is there no run-is-a-run advocate willing to provide answers to post 25?

Tango/19: but would you triple dog dare them?

But seriously, I understand their Leverage number is bunk; how about the replacement and lineup-adjusted aspects of WXRL (or whatever those wacky kids are calling it these days)? Is there any value in those adjustments they’re making, and how does it compare to WPA above Bench?

To me, it seems logical to attribute those +3.6 wins to Putz.

That’s the thing that amazes about that thread - how no one over there even bothered to go near this, but people still bothered to get all up in arms about the value of a high leverage reliever.  I don’t even think this point is arguable: Mariners - Putz + average closer = Mariners - 3ish wins.

I apologize, because I don’t have access to a lot of good stats.  I’m pretty thoroughly outgunned here, and have no experience.  I’m just a firm believer in skill-based metrics, not value based metrics, especially because of the large usage of leverage, etc. that seems to be popping up over the last few years.  However some things to consider in answer to post 25. 

1) I have no idea how to calculate the value of a player that is guaranteed to produce, as all market salaries are based on risk.

2) 2007 stats:
Matt Guerrier ERA+: 263 Salary: 407.5k
Pat Neshek ERA+: 220 (and 206 in ‘06) Salary: 380k
etc.

I’m sure there are a ton of other middle relievers that are comparable.

Re: post 34.  You are saying, essentially, based on WPA, Putz added 3.6 wins, therefore we should attribute 3.6 wins to Putz.  You’re not actually defending the basis on which those wins are calculated. 

What if Putz had pitched the 1st few outs of all of those games instead of the last few.  How much different would the M’s record be?  Not very.  How much different would his WPA look?  A lot (say, 1.8 times less).  Now, unless there is a huge psychological component to performance based on game state, there is no reason for this discrepancy.

I realize, too, that using non-free agents, or even arbitration eligible players, is really useless.  If someone can tell me how to find a list of non-minimum salary middle relievers, though, I’d love to look at it.

Re: post 34.  You are saying, essentially, based on WPA, Putz added 3.6 wins, therefore we should attribute 3.6 wins to Putz.  You’re not actually defending the basis on which those wins are calculated.

If you read the first page of the BTF thread, you’ll notice that you can come to the exact same +3 win conclusion without ever using WPA. 

What if Putz had pitched the 1st few outs of all of those games instead of the last few.  How much different would the M’s record be?  Not very.  How much different would his WPA look?  A lot (say, 1.8 times less).  Now, unless there is a huge psychological component to performance based on game state, there is no reason for this discrepancy.

What if there were green men on the moon that threw laser bolts from their eyeballs and smote the Angels dead? Then the Mariners would win the AL West by default.  Green Men for MVP!

I don’t care about hypotheticals that didn’t happen.  I’m talking about what actually did happen, and how valuable it was.

Water/36:
1. I’m asking for your evalation of two players (a starter and a reliever) and asking you what would make them equivalent.  Whatever degree of risk you apply to one (none, little, alot) apply the same degee of risk to the other.

2. I’m referring to two free agents (or two arb-elgible, or two of whatever is the same).  All things equal, given one guy who pitches in high-leverage 81 IP at a 2.25 ERA in a league of 4.50 ERA, what ERA from a starter with 225 IP would make you want to value the two the same?

3. The Mariners record would be EXTREMELY different.  Putz in random games compared to Putz in high-leverage games would be around a 2 win difference.  2 wins is enormous, especially after only half a season.

Tango/39:
Aha.  Ok, so I see something that makes a large difference that I was not considering.  Rather, I may have considered it, but downplayed it. 

The value of a reliever then, comes from not only their abilities, but the fact that they may be used when the manager desires, that is, in a high leverage situation (or game, which I had discounted).  Alright.

Now, if Putz is in random games, that does lower his value.  What I would offer, though, is that if for a moment we talk only about high leverage games, and not high leverage situations, how big is that difference in value?  I ask this because that should be the only measure that matters if we are considering the value of Putz to a win or loss (if you’ll concede that given a certain “game leverage” it does not matter which inning he pitches).

Taking all of this at face value then, the question as to how valuable a reliever is compared to a starter is extremely hard.  How much is the ability to liberally apply your best players to a situation worth?  The corollary is then, how much does it *cost* a team when the best reliever is not used in a “low leverage” situation, and the result is a comeback, or a barely failed later comeback.

David/38:
If you only want to talk about what did happen, without using hypotheticals, probabilities, etc, then you have to see the 1st and last innings as equal.  The only way a closer has more value than an equally effective (at getting outs) middle reliever, or a starter’s first inning, is because of the hypothetical.  As tango just reminded me - the possibility that he could just not pitch that night, if the game wasn’t close, and *maybe* pitch the next, when it’s a 1 run game and they need more skill on the mound.

I selected the top 30 relievers in Saves, and removed Putz.  The average non-Putz closer, compared to Putz:

HR: same
walks: 6 more than JJ
hits: 14 more than JJ
outs: 17 less than JJ

In terms of runs, that makes JJ 15 runs better than the average closer, or +1.5 wins, if they all pitched in random innings.

If we stick to actual runs allowed, JJ has allowed 4 in 46 innings, while the non-JJ closer has allowed 14.8 in 40.8 innings.  That makes JJ 12.7 runs better than the average closer, or about +1.3 wins better.

So, that’s what we know about their random performances, than JJ is already around +1.4 wins better than the average closer.

***

The average non-JJ closer has an ERA of 3.07 (component ERA also very similar), compared to the league average for relievers of 4.09.  So, the average closer is about 0.5 wins better than the average reliever, over 46 IP.

The average reliever is worth around 0.3 runs more than the replacement-level pitcher.

Add it up, and we get:
JJ Putz: +2.2 wins above replacement
non-JJ closer: +0.8 wins above replacement

And this has nothing at all to do with win probability or leverage.  It’s simply a context-neutral view of their performance.

The average closer has an LI of 1.90.  We know that if you insert a closer into a standard closer role that their performance will impact 1.90 times more than expected.  Using Guy’s process, we do NOT credit the average closer with their standard performance times 1.90, since they had nothing to do with the 1.90 (other than their sheer talent allowing them to be used like that).  But, Putz’ performance is +1.4 wins above the average closer.  And that performance *should* be magnified by the leverage.  That adds another +1.2 leveraged wins over and above his actual context-neutral wins.

Add it up, we now have:
JJ: +3.4 leveraged wins above replacement
non-JJ closer: still +0.8 leveraged wins above replacement

That’s what JJ’s performance to-date has been worth.

But wouldn’t the average closer get put into the same situations leverage-wise?  Why don’t they get the same 1.90 boost?

I should point out that JJ performance is actually more, since he’s performed exceptionally well, even more than his standard line, in the highest of leverages.

Using the Guy method, he comes in at +3.5 wins above the closer level.

jrh/43:

You probably missed Guy’s point on this.  Neither Putz nor the average closer gets the 1.90 boost for average closer performance.  So, if Putz would have performed like an average closer, he simply gets his +0 wins compared to the average closer, plus the +0.8 wins compared to the replacement pitcher.

But, any performance *above the closer line* does get magnified.  That is, that performance is providing real wins over and above the closer line.  And therefore that needs to be magnified.  That’s why Mariano gets paid as much as he does.  It’s not that he’s a closer, but that he’s (a) far better than the average closer and (b) he gets to magnify that extra oomph.

Or, if you don’t like all this math, just go look through his gamelogs and check the distribution of his runs allowed.  All four of them. 

They came in situations where the run did not hurt the team. 

April 13th, he entered the 9th inning against Texas with the team trailing 4-2.  He gave up a solo home run.  The team lost 5-2.  They’re losing that game whether Putz gives up that run or not.

April 15th, he entered the 9th inning against Texas with a 14-5 lead.  He gave up a solo home run.  The team won 14-6.  They’re winning that game whether Putz gives up that run or not.

May 17th, he enters the 9th inning of against Anaheim trailing 6-3.  He gives up one run and the team loses 7-3.  They were losing that game whether he gave up that run or not.

June 2nd, he enters the 8th inning of a tied game, keeps it tied, and the Mariners go on to take a 2 run lead in the top of the 9th inning.  He gives up a run in the bottom of the 9th, but keeps the lead, and the team still wins by 1. 

He has given up one run of any consequence this season, and the Mariners still won the game in which he allowed that run.  He has yet to surrender a lead, or give up the go-ahead run, in any game this season.  At the job he’s given, he has been essentially perfect. 

You can argue that the distrubition of runs isn’t predictive (in fact, people just did the other day).  But you can’t argue that the distribution of runs isn’t valuable.  Putz’s runs have been distributed into situations where the runs didn’t have an impact on the teams wins and losses - in every game where it mattered if he gave up a run or not, he has shut the opponent out. 

That’s not normal.  That’s not average.  That’s remarkably valualbe, and it’s one of the main reasons the Mariners are 10 games over .500 despite outscoring their opponents by 1 run all year.

I’ve got to say, that’s an elegant way of doing it.

Water raises an interesting point:

What I would offer, though, is that if for a moment we talk only about high leverage games, and not high leverage situations, how big is that difference in value?  I ask this because that should be the only measure that matters if we are considering the value of Putz to a win or loss (if you’ll concede that given a certain “game leverage” it does not matter which inning he pitches).

What if we established a “game leverage” score and applied that, retrospectively, to each game’s basic stats?  Interesting approach.

Now, I don’t think a player should get a WPA boost if he singles in a 5-0 game in the fifth that eventually turns into a 5-4 game.  But if a game stays close throughout, his approach would yield different results.

To me, the use of WPA in mvp discussions is very akin to the use of RBIs by traditionalists. It accounts for something that happened that is not necessarily the fault of the player.

Sure, you can argue that the closer is put in the highest leverage situations because he demonstrated that he is good, but how is that different from saying that the cleanup hitter has the most rbis because he has the most chances because he is (or was perceived as) the best hitter in the game and was therefore batting fourth?

The way statisticians loved to degrade the RBI in mvp discussions is what makes the embrace of wpa so odd to me. But I don’t really have a position on this. I can see it both ways. You can’t disbelieve the actual, though.

All you need to do is divide by LI.

andy, that’s the breakthrough of WPA.  It does add back in context, but in a way that’s rigorous and consistent.

Context does matter.  Sabermetricians pooh poohed context because of incomplete stats like saves and RBI’s.  WPA comes closer to being a valid way of measuring context than those stats do.

Can WPA be applied to cricket, where all a batsman’s “plate appearances” occur in sequence and all a team’s runs are scored before the other team gets its shot? If not, why not?

Does it really matter what score the other team has when determining how valuable a hitter’s performance was in a given game?

Does it really matter what score the other team has when determining how valuable a hitter’s performance was in a given game?

Sure.  A hit when losing 20-5 loss doesn’t help.  A hit when the score is tied makes a difference.

Don’t know the answer to your cricket question.

Sure that hit helps.  It just didn’t help enough. 

If Alex Rodriguez, say, playing a cricketized version of baseball, hit five home runs in five at bats before the other team got its innings, you’d call that performance quite valuable, even if the other team went on to score twenty and Alex’s teammates did squat.  The same would be true of a a game played as baseball is now except with the hitting team blind to the other team’s score. 

A hitter doesn’t fundamentally change his approach, which is to get the most possible bases without making an out, no matter the inning or score or other situation.  Sacrifices or choking up with two strikes or whatever come into play at times, but they don’t change the basic, overall approach.  Given that (does anyone not agree?) hitters can’t choose or be chosen by their managers to leverage themselves, I can’t see how WPA is relevant to them.

Hmm, I’ve tried making two posts today, to no avail.  So this is a test.

Well, I was still unable to post what I wanted.  From another computer:

I’d like to ask a few questions now:

1. If you were guaranteed Mariano Rivera for 6 years at his prime, 81 innings, ERA+ or componentERA+ of 200 (i.e., allows half as many runs as the league average), how much would you pay for him each year?

2. What kind of ERA+ would you expect from your #1 or #2 starter (225 IP per year for 6 years) to pay him the exact same amount?

Here’s an initial crak at the problem.  Let’s start with the major league splits for this year:  relievers have an ERA of 4.09, starters 4.50, a league average of 4.36.

“Mariano” has an ERA of 2.18.  He is +17 runs against the average reliever over the course of a season.

In order for a starting pitcher to be +17 runs over the average SP over the course of 225 IP, he’d need an ERA of 3.82, an ERA+ of 114.

Here’>http://www.baseball-reference.com/pi/shareit/MGaH">Here are some starters from 2006 who had an ERA+ in that range.  Free agent starters in that category are Curt Schilling, Barry Zito, and Tom Glavine (well, not really Zito).  These pitchers made an average of $10.3M last year (even though Zito didn’t become a FA until after 2006); Rivera made $10.5M.  None of those three pitchers reached 225 IP (Zito had 221), but on the other hand Rivera’s ERA+ was far better than 200, as it was for the first four years of his current contract (though he also didn’t reach 81 IP).

As best as I can tell, last year Rivera was +19 against the average reliever, and Schilling, Zito, and Glavine were +18, +19, and +14 runs (respectively) against the average starer.

So it appears that MLB GMs (and the market) see the ~115 ERA+ starter to be roughly equivalent to the >200 ERA+ reliever.

It would probably be more ideal to examine these pitchers relative to replacement; the 3.82 starter against a 4.50 average in 225 IP is, what, 32 runs above replacement?  That’s using LgERA(SP)+.60 as replacement level.  Not sure what the replacement level is for RP; to get the 2.18 ERA against 4.36 to be +32 RAR, replacement level would have to be a 5.73 ERA.

Given that (does anyone not agree?) hitters can’t choose or be chosen by their managers to leverage themselves, I can’t see how WPA is relevant to them.

This has already been discussed on this thread.  The analogy is with Ozzie Smith at shortstop.  He can’t control that fact that more balls are hit to shortstop than first base.  Does that mean he shouldn’t be credited with extra fielding plays?

By the way, I do believe that certain people and players react differently to high leverage situations.  Basic human nature.

Let’s say that the ERA+ of 114 is your feeling, and I’ll provide my mathematical basis for a solution.

So, what you are suggesting then is the following:
- league average ERA of 4.36 implies a replacement level pitcher of around 5.26 (4.66 if as a reliever and 5.56 if as a starter)
- an ERA+ of 114 implies an ERA of 3.82

That makes this pitcher worth (5.56-3.82)*25=+44 runs above replacement

The stud reliever is (4.66-2.18)*9=+22 runs above replacement

You are saying that the two are equivalent.  The only way for that to happen is to give Mariano double the impact, or a Leverage Index of 2.0.

In fact, if a run is a run is a run, then you’d want a starter with an ERA of 4.70 to be equal to Mariano Rivera.  A show of hands please for all those who believe this.

Right, no one believes that.  I’m happy calling an ERA+ of 200 for a reliever as equivalent to an ERA+ of 114 for a starter.  That means you must double the impact of the reliever performance for this to be true.

Maybe I should have knocked on wood.  Ramon freaking Vazquez?

Way to back me up today, J.J.

James: can you explain the rules of cricket?  Or better yet, create a new version of baseball that is based on the cricket scenario you are envisioning.

So, what you are suggesting then is the following:
- league average ERA of 4.36 implies a replacement level pitcher of around 5.26 (4.66 if as a reliever and 5.56 if as a starter)

Does a 4.36 ERA imply a 5.56 replacement level for a starter, and merely a 4.66 replacement level for a reliever?  I’m sure you’ve shown the research on this somewhere, though I don’t recall it.  That seems high to me for a starter, but low for the reliever.  That’s, what, a .467 win% for the replacement level reliever?

Incidentally, VORP would put the starter RL at 5.31 and the reliever RL at 5.14.  Our 3.82 starter in 225 would be at +37 against replacement and our 2.18 RL in 81 would be at +27. 

I don’t know which RL is more correct.

I’m also uncertain that examining salaries in this way really tells us that much.  For one, we have little reason to believe that teams are proficient at valuing their players.  For another, we must realize that the Yankees not only signed Mariano Rivera the pitcher, but Mariano Rivera the person, the postseason hero, the guy who sells the #42 jerseys, the hero in the Bronx, and someone whose departure may lead to some dismay amongst fans and fellow players.  For a third, we might also guess that the Yankees are willing to pay more for the security that having Rivera brings.  For yet more, wins are highly valuable to the Yankees as contenders.  For yet another thing, he may also be being paid for his postseason successes, which again carry great (and unique) weight for the Yankees.

Now, Eric Gagne, in his last year in Chavez Ravine, also made $10M.  The postseason caveats don’t come into account for him, but his team probably saw themselves as contenders, and Gagne did move merchandise in a way most pitchers do not.  And we must be willing to acknowledge that possibility that both of these teams made poor evaluations.

There is one more issue:  that of scarcity.

In 2006, there were 84 qualified starting pitchers; 28 had an ERA+ of 114 or better.  There were 98 relief pitchers with 60+ IP; only 8 had an ERA+ of 200 or better.

Over 2003-2006, there were 131 SP who had 350+ IP; 26 had an ERA+ of 114 or better over those years.  There were 143 relievers with 150+ IP; only 4 had a cumulative ERA+ of 200 or better.

The scarcity of the 200 ERA+ RP vis-a-vis the 114 ERA+ SP also contributes to the relatively high salary paid to the RP.

Well, baseball has extra base runners that cricket doesn’t, which makes for somewhat more complicated game situations, but leaving that aside, I was thinking of something like this:

The visiting team gets all its at bats at once.  It keeps sending guys to the plate until 27 outs are made.  Then the home team gets its turn; if it beats the visitors’ run total, the game is over.  You don’t know which run is key until the all the batting is done.

That’s basically how cricket works, although each batter stays at the “plate” until he makes one out, and they go through the batting order just once.  Baseball is essentially the same; the name of the game is accumulating as many runs as you can in 27 outs.  This is slightly disguised by the switching from offense to defense after each three outs, but it’s still all about stacking your runs against the other guys.  The order in which the runs are scored is irrelevant to the ultimate outcome.

Or envision a game in which the hitters and fielders are different people, and the hitters are sequestered in the dugout so that they don’t know what happens when the opposing hitters are at bat.  Each time a batter reaches the plate, he has no idea what the other team’s score is, and he concentrates only on maximizing his own team’s run total.  The so-called game situation is irrelevant to the approach he takes at the plate.

Ok, it sounds like there’s no strategy at all based on the game state.  Regardless of score, or where you are in the lineup, etc, the fielders will always stand in the same spot and the bowler will always throw the same way.

I would even suspect that fans don’t get too excited each team a run is scored, except at the very end? 

Sounds like basketball.

I don’t have a good answer for you at the moment.

***

Hawk: I’m sure there is a scarcity of good DHs too, but teams don’t overspend there.  AFter all, some of the really good DHs are playing 1B and RF.  The scarcity of good closers is because Pedro, Schilling, and Smoltz are in the rotation.

***

The Book shows that the impact of the same pitcher pitching both as a starter and reliever is around a 1 run per game difference. 

So, what I do is start with my replacement level pitcher: 0.410 winning record.  That pitcher, as a starter, will win 0.380 games, and that same pitcher, as a reliever, will win 0.470 games.

So, it’s on that basis that I compare pitchers to the appropriate baseline level.

If BP has their replacement levels as virtually identical, then they are wrong, plain and simple.  I would imagine that the lifespan of a 5.30 (component) ERA starter is far longers than the lifespan of a 5.30 (component) ERA reliever.

I agree with BlackHawk that Mariano’s salary may not be a helpful reference point.  Even leaving aside Rivera’s unique status, there’s reason to think the Yanks often overpay.  But comparing salaries for a pool of relievers would be interesting.  My perception is that for contemporaneous contracts, top relievers generally make less than top starter.

If RL is around 5.50 for starters and 4.50 for relievers, then I would break down a 2.18 closer’s value this way:
3.50 is “closer replacement level” (it shouldn’t be hard to find someone who’s .60 better than a lg avg reliever).  So:
9*(4.50-3.50) = 9 RAR
9 * (3.50-2.18) * 2.0 LI = 23.8 RAR
Total = 32.8 RAR (3 wins)

A starter, over 25 games, would need an ERA of 4.20 to produce the same RAR.  So an ERA+ of something like 105+ (and about .30 better than an average starter). 

I think this is about right:  a pitcher who you could count on for 225 IP at league average ERA is worth as much as an elite closer.  And I think they get paid about the same ($8-10M).

Also, I don’t think “scarcity” adds any value.  How scarce the ability is is already reflected in the league average benchmarks we use.  After that, all we care about is runs/wins.

"Way to back me up today, J.J.”

So in 2/3 of an inning last night, Putz lost about 16% of his value above an average closer—and just gave up 1 run.  Easy come, easy go.  Live by leverage, die by leverage....

* *

“If BP has their replacement levels as virtually identical, then they are wrong, plain and simple.”

Agreed.  What’s odd is that in Woolner’s original essay, in which he defines these replacement levels, he also identifies the starter-reliever gap as being about 1 run (years before The Book reached a similar conclusion).  Yet he didn’t see the obvious point that a replacement pitcher couldn’t possibly then have roughly the same ERA in both roles, since a repl starter would be about 1 run better if converted to relief (or conversely, the starters he was calling replacement were really better than that).

If fact, I think his RL for starters is roughly correct (about 1 run above lg avg), while his reliever level is much too high.  By looking at the pool of reliever with fewest IP, he has huge selective sampling problems.  Relievers will of course post 5.00+ ERAs in their bad years, but I don’t think any reliever consistently at that level can keep a job.

Ok, forget about Mariano.  How about someone who is your standard closer? 

Standard closer (presuming 3.30 ERA) is 1.36 ERA better than replacement or +12 RAR.

A starter with 225 IP (25 full games) needs to be just 0.48 runs above replacement per game, or an ERA of 5.08.

That is, the suggestion is that a standard closer with an ERA+ of 132 is equal in value (money, wins) to a starter with an ERA+ of 86.

While I’m keen on the basic approach to Guy’s method, it flies in the face of what anyone out there is ready to accept.  There is simply no way that any team or fan would trade a 132 ERA+ closer for a 86 ERA+ starter.

Though the Mariners came awfully close:
http://www.baseball-reference.com/s/soriara01.shtml

http://www.baseball-reference.com/r/ramirho01.shtml

I don’t think were as far from current reality as you’re suggesting.  Let’s do this:
Lg avg ERA:  4.35
RL starter:  5.35
RL reliever: 4.35
RL closer:  3.80
Avg closer:  3.30

Using my method, the avg closer is now 14 RAR over 81 IP.  If we expect a more realistic 198 IP from our starter (22 G), then he needs an ERA of 4.71, or ERA+ of 92, to equal the closer.  Is that unrealistic?  I may not have found the best salary info, but I think the Cubs pay Dempster 5.3M, Marquis 7M.  Izzy makes $8.7M, Jeff Suppan makes $10M.  Joe Nathan makes $5M.  I don’t think the idea that a 90-95 ERA+ starter = average ML closer is at all out of whack with the actual salary market.

An aside:  Looking at current closers, it’s surprisingly hard to identify “average closers” who are on FA salaries—it’s really a mix of 1) veteran superstars (Mario, Wagner), 2) veteran mediocrities who really shouldn’t be closing (Wickman), and 3) kids (many of whom will burn out before free agency).

Also, I don’t think “scarcity” adds any value.  How scarce the ability is is already reflected in the league average benchmarks we use.  After that, all we care about is runs/wins.

That may be all we care about, but I’m not sure that it’s all that major league GMs care about.

The Book shows that the impact of the same pitcher pitching both as a starter and reliever is around a 1 run per game difference.

So, what I do is start with my replacement level pitcher: 0.410 winning record.  That pitcher, as a starter, will win 0.380 games, and that same pitcher, as a reliever, will win 0.470 games.

So, it’s on that basis that I compare pitchers to the appropriate baseline level.

That sounds reasonable, but it strikes me that if, say, the RL for a SP is 5.55, then the floor for RP RP is 4.55, but I don’t know if it’s necessarily true that that’s where it lands.  After all, there are SP who are worse than RL who get moved to the bullpen.  But I’m certainly willing to concede that it should be, in this example, 4.55.

I may not have found the best salary info, but I think the Cubs pay Dempster 5.3M, Marquis 7M.  Izzy makes $8.7M, Jeff Suppan makes $10M.  Joe Nathan makes $5M.  I don’t think the idea that a 90-95 ERA+ starter = average ML closer is at all out of whack with the actual salary market.

Lessee, Wagner makes $10.75M per, BJ Ryan $9.4M, Hoffman $6.75M (Maddux got a 1-year deal for $10M from the same team, which may provide an interesting comparison), Benitez (average closer?) $7M, Isprinhausen $8.58M, F. Cordero $4.25M.

Gil Meche had a 92 ERA+ from 2003 through 2006, and he makes $11M per.  Brett Tomko’s at $4.35M.  Kyle Lohse $4.2M.  Meche is an outlier here.

"because at the time they got their walk, they didn’t know they were about to score.  This accounting is soley based on the presumption that our expectation of future events will happen based on a set probabilistic distribution.” (Tango/22)

Going back to the larger WPA discussion, I think this is the central issue.  We can all agree that WPA measures the expected value of each event at the time it occurs.  It tells us how valuable an event appeared to be at that moment, given the most likely subsequent events.  As such, it is an excellent tool—arguably the only one—for analyzing strategic decisions (which must be made without benefit of foresight). 

What is debatable is whether the sum of these real-time value estimates should be considered the best, or even a good, measure of value once the game is over and we have additional information.  Personally, I don’t care if a hitter drawing a walk knows that the next guy will hit a HR.  The timing still made it more valuable, and we should take that into account if we’re going to do that for the HR hitter (or for neither of them).  But I understand others will disagree. 

One thing that creates unnecessary polarization over WPA, I think, is comments like this from David/32:  “I don’t care about hypotheticals that didn’t happen.  I’m talking about what actually did happen, and how valuable it was.” I don’t mean to pick on David—WPA is often described as being “real,” in contrast to context-neutral stats.  But, to go back to my BB-BB-HR example, the only thing that is real is that 3 runs scored.  How we apportion credit for that is a matter of interpretation, not objective reality (I can think of 3 or 4 reasonable ways to do it off the top of my head).  So when we debate WPA, it’s a debate over competing interpretations of reality, not between the “real” and the “hypothetical.”

If we consider me as a moderate pro-WPA and Guy as a moderate anti-WPA, then Guy/71 is about a succinct summary of the positions of each side.

Both sides agree that a win probabilty framework is necessary to analyze strategic decisions.

What is in question is whether we can simply add up these discrete changes and attribute it to the player.  VLAD hit that slam in the 7th inning of a game the Angels lost:
http://www.fangraphs.com/wins.aspx?date=2004-10-08&team=Angels&dh=0&season=2004

There’s no question that that chart captured how we felt at the time.  That what Vlad did was monstrous.  The bases were loaded with 2 outs.  The Angels were down by 4 in the 7th inning.  They had a roughly 9% chance of winning the game, if everything afterwards would have happened “on average”.  Following VLAD’s slam, their odds jumped to 43%.  Our love at that moment was completely centered on VLAD.  And, at the very least, WPA at the discrete player level, measures our love for the player, even today.

But, does all that love really equate to the impact that VLAD had at the game level (as opposed to the play level).  That’s what’s in question.  The Angels lost that game.

What is it about VLAD’s performance that we want to attribute for that game? 
- That he was 1-4, with a HR and IBB (pure linear weights)? 
- That it was a HR with the bases loaded and two outs (linear weights by the 24 base/out states)? 
- That is was a HR with the team down by 4 (linear weights by game state, or WPA/LI)? 
- That it was a PA when the LI was 2.15 and therefore had double the impact that it would otherwise have had (WPA)?

And, how do we want to attribute VLAD’s performance, if the Angels had won the game instead?

In my 4 question example, the win value of the HR was:
context-neutral
+0.13 wins

base/out state
+0.11 wins (if we deflate the extra leverage)
+0.29 wins (if we include the leverage aspect)

game state
+0.16 wins (deflating the extra leverage)
+0.34 wins (includes leverage aspect)

Which of these do we want to give to Vlad.  And how much do we want to change this (if at all) based on whether the Angels eventually win or lose the game?

***

The pure anti-WPA will either choose +0.16 or +0.13.

Tango, I think there’s (at least) one more way of assigning value to Vlad’s tremendous HR, though one I haven’t really seen in practice.  It would be the flipside of Mike Wolverton’s old Support-Neutral system for SP.  In that metric, a pitcher who gave up, say, 1 R in 8 IP would receive credit for .990 wins and .010 losses, as an average team would win a game in which their SP allowed 1 R in 8 IP 99% of the time (numbers made up, of course, I don’t know what it is).  A guy who gave up 3 R in 6 IP might get .550 wins.  8 R in 1 IP might be .005 wins.

I would imagine that there is some way to do that for hitters, some method that ignores how many runs are actually scored against a team and says “an average team that gets a performance of 1-4 with a GSHR and an IBB wins 60% of the time” or whatever it is.  I don’t know if this sort of thing would provide much utility, but it’s an additional way of looking at things.

***

And how much do we want to change this (if at all) based on whether the Angels eventually win or lose the game?

As you might guess, I don’t think it should be changed at all based on this, unless we establish that it was Vlad who forced Scioscia to put in Jarrod &!()^@* Washburn to face David Ortiz instead of leaving in a just-fine Francisco Rodriguez or bringing in Troy Percival.  Beyond his defensive contribution (likely very minimal), he didn’t have anything to do with the number of runs Angel pitchers allowed, and thus should not be penalized for their ineptitude.

http://sports.espn.go.com/mlb/features/freeagents?positionId=15

Among the “3 years free agent” starters (career ERA+ in parens):
Batista (103), Eaton (90), Marquis (96), Padilla (101) signed between 7 and 11 per year. So, starter with ERA+ of 100 signs for 8.5MM per year.

(Note: these guys are worth around 2.5 wins above replacement, which if we put them through our salary calculator http://www.tangotiger.net/salary.html means they should have signed for around 3/26.  And they did.)

How about the relievers?  Of the recent highest paid relievers, we have:
Wagner, 4/43, 2006 season (equivalent to 3/34)
Izzy, 3/26, 2006 season
Hoffman, 3/19, 2006 season
BJ Ryan, 5/47, 2006 season (equivalent to 3/33)

The average of these four guys is 3/28, and add in inflation gets us to 3/31. 

These are the higher-end closers, so a 3/26 might be safe to say as the “average” closer value, which is comparable to our ERA+ 100 starters.

If we treat our average closers as being true ERA+ of 140, that gives them an ERA of 3.11 (close to what I found this year) in a league of 4.36 (and repl level of 4.66).  That puts them as being +1.4 wins above replacement.

Like I said, our starters above are closer to the 2.5 WAR level.

2.5/1.4 = 1.80, which is your standard closer leverage.

LAWoBH/74:

That wouldn’t really be any different than linear weights. The reason it “works” for pitchers is that pitchers have a huge impact on every game, and therefore whether they are consistent or not will have a real impact on how games their team wins above or below what we would expect based on their seasonal numbers. The same is not true for hitters.

Tango/75: 
These comparisons are hard, because ideally you would control for (perceived) durability.  And looking only at 3-year contracts limits the pool.  I think 95-100 ERA+ starters who are perceived as durable are generally in the $10M range today, similar to your group of closers—guys like Suppan (100), Milton (94).

Also, consider these 4 closers’ ERA+ in the several years prior to signing these contracts:  Wagner, Hoffman, and Ryan are all 180+, and Izzy 160-170.  Let’s call them 175.  That’s another 5.7 RAR/.6 win, so they are 2 WAR w/o leverage.  Now the ratio is 1.25:1, not 1.8:1.  Bottom line:  I think the fact that teams could sign Izzy and Hoffman—future HOFer, all-time saves leader—for under $9M/year says a lot.

The important point, though, is measuring relievers against a significantly higher baseline than starting pitchers.  We can quibble about where to draw that line for closers vs other relievers.  But metrics that use one baseline for all pitchers—including WPA and WXRL—will always overvalue closers.

Remember that I said that those heavyweights signed for (effectively) 3/31, which is more than the starters (3/26). 

And even if you “show” ERA+ of 180, you are not a true 180. 

Therefore, a star closer signing for 3/26 is likely closer to a true 140 ERA+.

As for Hoffman, he gave the Padres a home town discount.  I don’t think it says anything about the marketplace… just Hoffman.  BJ Ryan says more about the marketplace, as does Wagner.

At the very least, I think all must concede that there is not a fan or executive that operates on the principle of “a run is a run”.  If we want to say that the average closer is getting 1.4x, 1.6x or 1.8x the value of a run, I’m ok.  I’m not really hung up on the exact number.  I’m just hung up on the issue of those who want to say that a run is a run is a run, when no one believes it.

Your method (leveraged-closer runs above replacement-closer level, plus unleveraged replacement-closer runs above replacement-pitcher level) is I think likely the operating principle of fans and maybe executives.

I agree with #79, and I don’t think we’re really far apart.

“And even if you “show” ERA+ of 180, you are not a true 180.  Therefore, a star closer signing for 3/26 is likely closer to a true 140 ERA+.”

Wagner is career 185.  Hoffman averaged 186 over the five seasons prior to the contract.  Izzy in the 5 years before his contract was 164/158/176/146/200.  Ryan averaged 190 over the two years prior to his contract (I’ll agree that doesn’t prove he’s truly a 175-180 talent, but I believe the Blue Jays thought so).  These guys weren’t being hired to deliver a 140 ERA+. 

I also think it’s interesting that there really aren’t very many veteran 130-140 ERA+ closers.  If a team can’t sign one of the elite 170+ guys, and can’t generate a wage slave from their system, they seem content to use sign a Wickman-type cheap or convert a competent reliever (Weathers) and hope for the best.

Ok, we agree on the basics, so let’s get down to the minutiae.  For relievers, ERA is often misleading, though on a career level, it usually works out.  However, to the extent that a pitcher doesn’t control the sequencing of events, we might as well use OPS+.

Here are the career leaders in OPS+ among relievers:
http://www.bb-ref.com/pi/shareit/8avN

No surprise among the top 5 (Rivera 47, Wagner 49, Percival 57, Hoffman 62, Ryan 63).

(Note: darn that Scot Shields is underappreciated.)

That is their career totals, and Wagner and Hoffman did sign their contract in their late 30s.

I think it’s reasonable to call “65” the expected talent level that teams were signing for these guys.

If I take all recent starter seasons with an OPS+ of 75 or better (edit: average OPS+ of 65):
http://www.bb-ref.com/pi/shareit/7Jqj

I get a starter win% of .668.  So, it’s fair to call a top-level closer a .670 pitcher.

Let’s call a “replacement level closer” an OPS+ of 85.  Brader Looper in his 20s was an 87.  Here’s the full list of relievers in their 20s:
http://www.bb-ref.com/pi/shareit/UxvY

An OPS+ of 85 would be roughly a win% of .570.  That would be our replacement-level closer line.  And of course, our replacement level pitcher as a reliever has a win% of .470.

On to our calculations.  Our high-level closer (.670) is +.100 wins above the replacement-level closer line of .570.  With 9 full games (81 innings), that’d be worth +0.9 random wins.  Leveraged at 2x, that makes it +1.8 leveraged wins.

The replacement-level closer line (.570) is also +.100 wins above the replacement-level pitcher line (.470), making it worth +0.9 random wins, which we DON’T leveage, using Guy’s process.

Adding it up, that gives our top-end relievers a worth of +2.7 semi-leveraged wins above replacement.  The salary calculator says 3 years (starting in 2007) is 29 MM:
http://www.tangotiger.net/salary.html

That’d be 3/26 in 2006, which is a bit lower than they actually got.

I’m fine that Guy’s method works, and that it satisfies everyone that:
1. you get the leveraged impact
2. but that you don’t give the reliever the leveraged portion that he doesn’t deserve

"That’d be 3/26 in 2006, which is a bit lower than they actually got.”

Just throw in Joe Nathan (68 OPS+) and you’ll be right on the money.  (I’m guessing Twins will exercise his $6M option next year.)

After reading tango’s #1 comment and BBTF thread, an seemingly comparable example came into my head. I don’t know if it’s appropriate to post it here right now since initial discussions were over for a while… anyway I’ll just share it.

Suppose Team A & B are in a relay race, the relay order and speed (the higher, the faster) of each runner are listed below:

Team A: 5-3-3-2 (=13)
Team B: 3-4-4-1 (=12)

(BTW, a real relay team won’t set its order like this)

Race start. Runner A#1 gains a +2 advantage after finishing his segment, then his teammates jointly give it back to make race even. Who’ll win it all for his team? A#4 the slowest. (on his team)

I guess if there is a WPA in relay race, it’ll give highest score to A#4—just like who hits a walk-off single in a baseball game earns the highest score. Since A#1 surely is the fastest runner (of all) and makes the biggest speed-wise “contribution” to his team in this situation, it really makes me puzzled…

BTW, I hope tango won’t feel offended but I found this hilarious:

BBTF post #43: (reply to someone’s challenge to the concept of WPA)

“I could be completely wrong, but I don’t think that’s how WPA works. I base this only on the fact that tango isn’t idiot.”