THE BOOK--Playing The Percentages In Baseball

I understand where he’s coming from.

Each site/organization has their “own” version of a player value system VORP, PECOTA, fWAR, rWAR, Win Shares, whatever.

Furhtermore, there’s FIP, xFIP, DIPS, blah, blah, blah.

I really enjoy all of the stats and their variances, but I’m one of the few.

It really is amazing that there isn;t a unification of the advanced metrics, in hopes that one gets established as a stronghold and valuable metric in baseball.

But, it’s also reality that each site/author/organization is looking to make a name for itself, so we have the various “clans” battling over a small scrap of land, instead of uniting to conquor the world.

Sabermatician realize that it’s basically the same 100 guys at all the “stats sites”, right? It’s the same 40 guys reading about all the calculations and differences, it’sd not as if BP has their own “nerds” and TT has different nerds, and FG hasd their own geeks, etc.

Settle on the most important and (more, uh importantly) reliable advanced metrics, whether they be VORP, UZR, WAR, xFIP, etc (I know those metrics measure different things, I’m just trying to represent the variety of acronyms out there), and present the 2 or 3 best as a unified front, and explain them to “the people”, and get the stats to become accepted and relevant to the sport.

I know that won’t happen, because it will be more important for each author to show that their ‘version’ of the stat is 2% better than the other 6 versions that purport to measure the same thing.

Of course, I also understand that once certain advanced metrics become accepted and common, the community will have to come up with opther advanced metrics, if for no other reason than to be counter to the mainstream. *grin*

I’d like to hear what 3 metrics people think could define a player’s worth.

We talked about this.  rWAR and fWAR differ in a few respects, notably the handling of hits allowed by pitchers.

Fangraphs completely ignores it.  While Rally adjusts it based on the fielding numbers of the team.  There’s really no unification on this front.  Each side is completely correct.

The best you can do is to handle things that could lead to absurd results (like how the DH replacement level is handled).  Or getting the wrong relative values for walks and HR, etc.  Construction errors or unreasonable assumptions.

After we get through that, there’s latitude for the builder to implement things.  We don’t want every building to look like the CN Tower do we? 

You adhere to construction principles, and then let your creativity take over.

Since I have no interest in whether the public adopts sabermetrics, I have no interest in metric standardization.  I’m all for hashing out methodological differences, to the extent that’s possible, but I’m always going to enjoy figuring my own stuff, and the same is true I’d bet for a lot of serious saberists.  Unless someone gives me the exact WAR that I want (replacement level set to what I use, using the run estimator that I use, using the park factors that I use, ...), I’m always going to have sufficient motivation to calculate my own WAR.

In any event, I think it’s a net positive that there are multiple approaches, even for the purpose of appealing to mainstream fans.  It emphasizes that these metrics are not etched in stone, but rather are based on certain assumptions that will change when the evidence demands them to.  The worst thing that could happen is for sabermetric stats to be accepted as the new BA/HR/RBI, used unquestioningly. 

In fact, to the extent that sabermetrics will have a public relations problem in the future, I think it’s going to come more from what you could call “Fangraphs babies” (no offense to Fangraphs)--people who are exposed to sabermetrics from a young age, but never actually get their hands dirty themselves.  I’m confident people in that group will have a higher arrogance to knowledge ratio than those that came before.

"Fangraphs completely ignores it.  While Rally adjusts it based on the fielding numbers of the team.  There’s really no unification on this front.  Each side is completely correct.”

My take on this one is that my implementation better describes what happened, and theirs is more predictive.  If you’re answering different questions it’s hard to come to a unified conclusion.  Also impossible to say one is 2% more accurate or whatever.  You’d have as much difference in opinion on what standard to measure your stat by as there is in the construction itself.

I don’t care to rule the world, I don’t care if other people publish a similar stat with a different interpretation.  I just do what I do.

The more I think about it, the more misplaced I think my comments are.

The SABR guys should come up with as many stats as possible, test them, and see which ones have the highest value (relative), reliability, etc. In short the mad scientists should do what they do best.

It would be up to the TV and sports *cough* journalist to learn about the metrics and inform the public. That type of thing may never catch on, as so much commentary seems to be about fans feelings and opinions, and goody things said by Steve Phillips and Eduardo Perez.

I think one of the beauties of modern sabermetrics is to free us from standardized, globally accepted metrics.  Something as sophisticated as WAR should probably never have a standard accepted formula because there are so many elements that *could* be included in it that we don’t even know how to measure yet.  It can always be made better.  Once we say that we have a formula that everyone agrees on, we are sort of admitting that we have nothing left to learn.

We talked about the differences between rWAR and fWAR here: http://bit.ly/8l2e5L

Among the differences were

—for pitchers, fWAR is DIPS compliant “while rWAR teases out the fielding part with his team fielding stats”;

—fWAR assumes all catcher’s are the same defensively while rWAR uses the available data to rate catchers;

—fWAR uses UZR and rWAR uses TotalZone;

—rWAR includes baserunning;

—rWAR accounts for park factors;

—differences in position adjustments and replacement-level baseline; and

—fWAR makes no league adjustments.

I’m certainly pretty ignorant of the math behind both of these statistics (and others!), but I walked away from the above-linked thread seeing no reason I should ever use fWAR instead of rWAR.

As a person who is a fan of what sabermetrics offers, but in no way a contributor to the work being done, I’m not qualified to speak to the differences. It’s informative to see Rally up above (comment #4) note that fWAR is more predictive and rWAR more historically-focused.

But this gets to an issue which was being discussed here a couple days ago: Having multiple statistics with the same name or acronym leads to confusion. It inhibits understanding and adoption by only adding confusion and uncertainty. Outside of a handful of writers or sites, when someone references a player’s WAR, I then go to Baseball-Reference to see if the numbers there align with what the writer cited.

I’m not complaining, but, frankly, the onus should not be on the reader to check which data a writing is using. Heck, prior to the thread I linked at the outset of this comment, I’d never seen a thorough discussion of the differences between the two WARs. And it was only in the course of doing some basic research that I discovered rWAR and fWAR are not the same. Thankfully, when I questioned Tom about the differences, he made a post of it here.

As a fan of the sport and the statistical advancements taking place, this is annoying and unfortunate, and something I hope will be resolved. A real-world equivalent would be like going into a restaurant and ordering the veal parmigiana.  When it comes out, there’s breaded veal topped with sauce and cheese, but it’s surrounded by bread. You tell the server, “No, I wanted the entrée, with the veal served over pasta.”

Like rWAR and fWAR, the dishes are similar and surrounded by starches, but the differences are significant enough to change your perception of what you received.

fWAR accounts for park factors too, they just use different park factors from rWAR.  fWAR has also added catcher defense since that discussion took place.

I’m starting to wonder if Posnanski’s writing about sabermetrics isn’t actually pernicious.  Tom’s third quote from the Poz article shows that he appears to completely misunderstand the concept of WAR-as-framework.  It took me a while to internalize this understanding myself, but I don’t have an audience of millions who will grow up THINKING they know things that they don’t.

Fundamentally, you can group disagreements between the three (if you go ahead and include WARP, as Tango alluded to - four, if you want to include VORP as its own line item, and you can grow the list from there) as follows:

1) Small things, and
2) Big things.

Small things are things like park factors, for instance - everybody pretty much agrees conceptually on what we’re doing here, we just disagree slightly on execution. I think if you made a compelling case that there was a high value on being consistent in how these are done, we could probably hash out a lot of these in short order. They aren’t necessarily battles worth fighting.

But the big things - those really come down to how you define the proposition of defining value. And let’s be honest - you’re trying to come up with a single number to represent every aspect of baseball value, for all players. This is hard, and this is uncertain work, and a lot of it comes down to the foundational assumptions that you make. That is worth fighting for, I think - at least, in so much as any of this is worth fighting for. And sometimes there isn’t a single right answer for any of it.

So I don’t think you’re ever going to see a case where everyone agrees on every detail of figuring WAR(P). And I agree with those who don’t necessarily find it desirable.

I know there are people who seem to want us to come up with “sabermetric RBIs” or what have you - for us to be just as certain of our conclusions as people were in the past, and for our work to be done and history locked down, if you will. I think that overlooks the fundamental revelation of sabermetrics - that nobody will ever know everything there is to know about baseball. That’s a great and terrible and wonderful thing, I think.

Dan Rosenhack has his own WAR which he has used in Hall of Merit threads at BTF.  I believe it is largely independent of the others.

kds, yes, Dan Rosenheck has his own flavor (he uses the WARP appellation, as opposed to WAR). Tango actually hosts it on his site:

http://www.tangotiger.net/rosenheck/

There’s a ton of others you can throw on the fire - THT used to publish Win Shares Above Bench. Justin Inaz did his Total Value numbers for ‘08. I did a WAR implementation based on THT stats for ‘09. Patriot, as noted, figures his own values as well. If you’re willing to consider metrics baselined to average, you can throw in TPR, Super LTWS and Dial’s OPD as well. There’s probably a half-dozen I’m forgetting to mention as well as more I’ve never heard of.

I agree with Poz—having multiple versions hurts the brand and slows adoption.  The mainstream media don’t want to have to figure out which is which and what the difference is and whether they are using the “right” one (even if there is no right one).  If we want Wins and RBIs to someday become secondary stats, the consistent use of a few key new stats such as WAR, wOBA etc. will be paramount.

"Dan Rosenhack has his own WAR which he has used in Hall of Merit threads at BTF.  I believe it is largely independent of the others.”

Dan doesn’t have his own fielding metric, so he’s borrowing from others.  Same for baserunning.  But he’s pretty unique on how he handles the position adjustments, and using standard deviations to adjust for competition levels.

I like the “small things / big things” categorization that Colin noted.

From that standpoint, and using the summary from NaOH:

fWAR v rWAR:
- SMALL things
1. for catchers, fWAR uses SB, CS, while rWAR includes also adds in WP, PB

2. rWAR includes baserunning

3. different ways to account for park factors

4. differences in position adjustments

5. replacement-level baseline

6. fWAR makes no league adjustments

- BIG things
1. for pitchers, fWAR is DIPS compliant “while rWAR teases out the fielding part with his team fielding stats”;

2. fWAR uses UZR and rWAR uses TotalZone;

3. rWAR makes sure all the runs scored and allowed are allocated to the players (sum of runs-parts equals runs-whole); fWAR focuses on components

- HUGE things
0. None.  They both use the Tango-championed WAR framework.

***

So, I think if someone wanted to create a chart, with a list of items going down a column on the left, the various “Total” metrics going across a row on top, and then a box that explains each item, that would be great.

Say, something like this:

Component rWAR tWAR WARP WinSh WSAB
--------- ---- ---- ---- ----- ----
Catching
Fielding
Position
Pitching
Baserunning
Offense
Replacement
Park
SumToRuns
SumToWins
Other?

This way, we can see what each one does, and we’ll be in a better position to understand how each system differs.

Seems like a good Google Docs project for someone to initiate.

Speaking for myself, I *totally* forgot that Dan posted his system on my site.

Fangraphs WAR doesn’t account for park factors?

Fangraphs WAR is park adjusted:

http://www.fangraphs.com/blogs/index.php/win-values-explained-part-one/

However, you may notice that a player’s Batting value doesn’t match his wRAA value on the player cards. That’s because the wRAA numbers on the site are not park adjusted, but to build a proper win value, you have to include the effects of a player’s home environment. Getting back to Beltre, he plays in a park that depresses run scoring, so the runs that he creates are more valuable than they would be if they came in a park where runs were more plentiful. So, while his raw offensive line may have only been worth 3.9 runs, when we adjust for Safeco Field, his Batting value goes up to 5.9 runs.

Actually, someone else noted that it does.  Might be my bad.

Cross-posted with Colin.  I’ll fix the post listing the rWAR v fWAR differences.

I think it’s fairest to say that both take a player’s run environment into account. I don’t believe that Rally WAR is explicitly park adjusted, at least not with “park factors” - IIRC, it uses a custom runs-to-wins conversion based upon the team’s offensive environment, which should come out to about the same.

From briefly looking at the differences in rWAR and fWAR for specific players, it seems that more often than not the bulk of the discrepancy arises from fielding runs:

Justin Morneau:
fWAR: 5.0
rWAR: 5.3
Rfield diff.: 4

David Dejesus:
fWAR: 2.7
rWAR: 3.1
Rfield diff.: 4.3

....etc.

As a novice, I don’t quite understand the nuts and bolts of the differences between Total Zone and UZR. Does anyone have a link to someplace that easily summarizes the difference? Or is there a fairly simple way to think about it? Are they generally thought of to be in agreement in terms of good/bad, but just different in scale? Would it be a dumb idea to somehow average the two? (ok, it’s a dumb idea, I know, but I do think that having a unified number somewhere could be valuable...)

The difference is mostly in the granularity of the data.

TZ sees each data plotted to one of these zones:
http://www.retrosheet.org/location.htm

UZR sees the data plotted to the closest 5 foot point.

At the career level, on the assumption that the Retro zones will come out in the wash to match the more granular data, there should be little difference between the two.

Other potential differences is how park factors are handled, the base/out situation affecting out rates, the GB/FB tendency of the pitcher beind handled differently, the handedness of the batter.  Alot of little things done differently.

We’ve spent a lot of time discussing that recently, David:

http://www.insidethebook.com/ee/index.php/site/comments/war_or_warp_if_you_must_but_not_vorp/

I think that fielding is really where you tend to see the largest area of disagreement in the measurement of player value here - in the case of position players, it’s how to measure it; in the case of pitchers, it’s how to separate what a pitcher did from what his defense did.

(The second largest area of disagreement is probably in the treatment of relievers.)

"I don’t believe that Rally WAR is explicitly park adjusted, at least not with “park factors” - IIRC, it uses a custom runs-to-wins conversion based upon the team’s offensive environment, which should come out to about the same.”

No, I have park factors.  The runs to wins conversion just says you need 11 runs to get a win in 1996, but only 8 in 1968, or something like that.

They will not get you to the same place.  Say player Ecks is -5 runs in San Diego, best pitcher’s park in the game, but a park adjustment makes that +2.  A different runs to wins conversion can’t change the sign of his runs above/below average.

The runs to wins figure is done at the league level, after the numbers are park adjusted.

Whew! Well that helps. I am dizzied by that comment thread but ultimately edified. Thanks for taking a moment to steer me in the right direction.

One quick point of clarification/question: which of UZR, TZ, and Dewan are counting stats? Are they all? If so, what’s the reasoning behind that?

All stats can be both counting and rate stats.  It is just that you typically see certain stats represented as one or the other.  UZR can be presented as either.  I don’t like ANY stat presented as a rate stat!  And the more luck/less skill inherent in the stat, the less I like it presented as a rate stat. 

For my taste, all stats should be represented as regressed.  What players actually “did” does not interest me at all.  And not all stats tell you what a player actually “did” anyway, at least not necessarily what you want to know or what you think it tells you.  And what they “did” might be based on either arbitrary or subjective classifications or not-so-accurate data.

Also, this site:

http://www.thebaseballgauge.com/

which I like a lot, has IT’S own unique implementation of WAR.

Kevin, that site is nuts!  Do you know if they have listed their calculations anywhere?  In addition to WAR, I’d like to know how they calculated WSAB.

Studes:

I have the owner’s email address, as I’ve been asking him about the calculations.  If you want to email me, I’ll give it to you.

He’s using 75% of expected win shares for position players offense and relievers, and using 60% for starting pitchers.  He does something for fielding that I’m not clear on.

His WAR is very much Baseruns based, but I’m still not clear on his exact methodology.

Yeah, we’d have to know about the methodology.

Also:
http://www.thebaseballgauge.com/player.php?ID=120891

Near the bottom.  He has Tim Raines for “WSN”.  You know how I feel about that.

Impressive.  There goes my motivation to update my site.  Other than to put a link to baseballgauge so I don’t forget where it is.

Who’s behind that site?  Got to be someone we know of.  I have a hard time imagining somebody doing that much sabermetric work and not joining any of the online discussions.

Rally/31, the website is registered to Daniel Hirsch.

I’m very curious to know how he figured hits and other components above replacement.  You need to do some kind of “Willie Davis” thing, or at least square roots.  It’s tough to put any stock into the results with absolutely no explanation.

I asked a few weeks ago on Twitter if anyone knew who was behind that site and I didn’t get any responses.

I invited Dan to join this thread so hopefully he’ll explain some of his calculations.

The replacement level for the individual offensive components should be easy enough to explain. 

You start with a replacement-level player, figure his offense at say -20 runs per 700 PA, and then figure out the hitting line that gets you to there, while keeping the PA constant.

This is so that if you add up the individual components with LWTS coefficients, it exactly matches the runs shown.

Well, that’s how I would do it.  Whether Daniel did that, let’s see.

His fielding wins are above replacement, not above average.  The leader is about 20 wins, the worst is about -3.  That’s less variance in fielding than you’ll see from other WAR methods.

I suspect what he’s doing is deriving it from fielding win shares.

Ahhh… that would explain why Ozzie is so low.

Hi guys,

KJOK told me that there were some questions about my calculations, so I’ll try to explain them as best I can.

For WAR, I’ll just copy and paste what I sent KJOK:
---------------------------------------
1. For offensive WAR, I’ll use “player X” as an example.  I calculate BaseRuns for each player and figure the league averages (per 27 outs) for pitchers and non-pitchers.  Let’s say that player X has 104 BaseRuns, 400 outs and the leaguer’s average is 4.50 BaseRuns per 27 outs.  Next, I take the number of games played for each player (because a player can only influence the outcome of the games in which they play).  Player X has 150 Games played.  I multiply the games played by the league average BaseRuns per 27 outs (150 x 4.5 = 675).  This is the total number of runs an “average” team would score in 150 games.  Next, I take the player’s total outs (400) and figure how many BaseRuns an average player would have with 400 outs (400 / 27 * 4.5 = 66.67).  I also figure what a replacement player would have with those 400 outs.  I use .4136 W% for replacement level or 67 wins in 162 games.  Working backwards with the pythagorean record formula, a 67 win team would score 82.5% as many runs as they would allow.  So I multiply .825 by 4.5 to find replacement level (.825 x 4.5 = 3.7125 BaseRuns per 27 outs).  A Repl player would have 55 BaseRuns with 400 outs (400 / 27 * 3.7125 = 55).  Now that I have BaseRuns for player X (104), average player (66.67) and Repl player (55), i can figure offensive WAR.

Take 675 (Avg Team’s runs) and subtract the average player and add player X (675 - 66.67 + 104 = 712,33).  Do the same for a replacement player (675 - 66.67 + 55 = 663,33).  Then use the pythagorean record formula to figure how many wins player X’s team would win in 150 games and do the same for the Repacement Player’s team.  Runs Allowed for each of these formulas is 675 (assuming they are playing average teams).  I’ll skip showing the pythag calculation.  Player X’s team that scored 712.33 and allowed 675 has a winning pct of .524 or 78.65 wins in 150 games.  Repl Player’s team that scored/allowed 663.33 / 675 has a winning pct of .492 or 73.82 wins in 150 games.  78.65 - 73.82 = 4.83 offensive wins above replacement player.  That’s pretty much it for oWAR, although I use pythagPat for pythagorean formula and I also use Park Adjustments.  Offensive WAR for Pitchers is slightly different b/c you have to assume 8 position players and 1 pitcher and you compare pitcher’s to other pitchers.

Pitching WAR is essentially the same as calculating offensive WAR, using games played, innings pitched and Defensive Independent Runs.  I’m not going to go through it b/c I’m sure you can gather the process by using the oWAR calc.

2. Converting Fielding WS into WAR is actually pretty simple.  Let’s say player X is a 2B and has 5 Fielding WS in 1200 Innings or 4.167 WS per 1000 Innings.  Let’s say the league avg 2B has 3 WS per 1000 Innings.  Remember, a Win Share is a Team Win multiplied by 3.  The Replacement level is 67 Wins out of 162.  So we calculate ((67 / 81) x 3 = 2.48).  So, a Repl 2B would have 2.48 Win Shares per 1000 Innings or 2.98 in 1200 Innings.  Player X Has 5 Win Shares minus Repl (2.98), which is 2.02 Win Shares Above Replacement or (2.02 / 3 = .67) .67 Fielding Wins Above Replacement.

I’m not 100% satisfied with using Win Shares but It is the best I have for now until I can calculate Total Zone FRAA with play by play data.

I’m also working on implementing a standard deviation adjustment for leagues/eras with a larger variance in talent.
---------------------------------------------
@28
Studes, I want to thank you for your WSAB method on THT as I tried to mirror it for my calculations.  I’d like to discuss it further with you if at all possible.
---------------------------------------------
@30
I’m using databank’s db, which has the Expos/Nationals franchID as WSN.  I’m not implying that he played for the Nationals, that just happens to be the franchID for that particular organization.  If the franchID happened to be “MON”, then it would look like Ryan Zimmerman played for the Expos.
---------------------------------------------
@31
As mentioned above, I’m not particularly happy with my use of Win Shares for my fielding WAR.  Do you happen to make your TZ calculations public?

Also, I don’t mean to steal any thunder from the great work of yourself, FanGraphs, or Prospectus with the use of “WAR”.  I contemplated using another term for it, but ultimately decided upon WAR.

I’ve been a long time follower of these discussions on different sites (THT, Fever, etc), although I don’t chime in very often
---------------------------------------------
@33
I do use the “Willie Davis Method” for my components above replacement.  I find the league average and then adjust those totals to each park. 

For replacement level, I find the weighted Standard Deviation for each stat (BA, OBP, etc) throughout the history of baseball.  I found the # of SD’s below average for my replacement level BsRns/27.  Then I use that # of SD’s below average for each stat to figure each replacement level.

Railsplitter’s replacement level (.414) is higher than the levels used in the other systems, although this is somewhat offset by using offense and fielding replacement levels rather than one. 

I don’t think the Win Shares math is kosher, since fielding WS already subtract out a .200 level player.  Your effective replacement level is now 82.5% of value over .200 (~50%).  This is going to compress the range of fielding value, beyond anything Win Shares already does on its own.

Also, are you plugging individuals directly into BsR, or using a theoretical team or linear weights generated from BsR?

That’s a good point about the .200 level player.  I’ll have to look into an adjustment. 

As far as BsR, I do it on a “theoretical team” basis, subtracting each player’s stats and finding the difference.  I also find B coef for each league

Right,he splits the replacement level out to offense and defense (much like I presume the replacement level of offense is them split between hits, HR and walks).

Railsplitter… I’ve seen you posting on Fever right?  Were you also there at Fanhome?

Railsplitter’s procedure is interesting, but am I the only one who prefers a simpler linear weights approach for this?

I don’t think the Win Shares math is kosher, since fielding WS already subtract out a .200 level player.  Your effective replacement level is now 82.5% of value over .200 (~50%).  This is going to compress the range of fielding value, beyond anything Win Shares already does on its own.

Patriot, we’ve talked about this before.  Win Shares inserts a replacement level, but then starts all calculations, such as Runs Created and even total Win Shares, at zero, and not at the replacement level.

When you apply replacement level to a player’s total Shares, based on *expected* Win Shares (that is, normalized to the number of Win Shares an average player would achieve), and not on individual components, I don’t think compression is much of an issue.  Of course, it’s a goofy way to do it, but that’s a different issue.

but am I the only one who prefers a simpler linear weights approach for this

The older I get, the simpler I prefer things.  I don’t know if this is wisdom or laziness on my part. 

As long as the construction is basically correct, then I’m ok with it.  I don’t care what run value you give the double, as long as it’s about +.30 more than a single.  And keep the HR value at 1.40.  Just some simple basic rules, and I’m good to go.

Studes, I do recall having this discussion before.  I do not recall ever admitting defeat

I’m not saying that you can’t apply replacement level to actual Win Shares.  What I’m saying is that you can’t apply the normal replacement level (.825 in this case).  IIRC, your WSAB recognizes this.

If you do what Railsplitter does, the linear differences in WARabvRepl between players will be the same, but the scale will be off--a true replacement level player will appear to be below replacement level, and everyone else will be less valuable than they actually would be if compared to the replacement level before going through the WS process.

If you do what Railsplitter does, the linear differences in WARabvRepl between players will be the same, but the scale will be off

OK, agreed that the way Railsplitter is pulling out fielding WS and applying a replacement level is indeed off.

I always presume victory when I can’t remember the actual discussion. 

I make my case in the linked post (I don’t like to do that but it’s easier that way).

If I can offer an unsolicited suggestion, Railsplitter, I’d strongly encourage you to add a glossary.  The more detail the better, as far as I’m concerned, but it doesn’t necessarily have to get into the nitty-gritty.  The glossary on Rally’s WAR site doesn’t give every formula, but it does provide a good overview on what his process is.

@42
I do post on fever and I also was a member of fanhome.  I can’t believe that was 7 or 8 years ago already

Patriot (and others): you are encouraged to link to your own work at will.

And, yeah, some glossary would be nice.