THE BOOK--Playing The Percentages In Baseball

He’s also 64 & 10 on SB & CS the past 3 years.

I would LOVE it if someone at BPro would take Joe to task for what he said.  I mean, you get decent blog posts like mine, or good ones like Colin or JC, and readers benefit, when you set aside partisanship, and simply look at the merits of an argument.

Here is a perfect opportunity where a BPro reader would benefit from Eric or Tim or Matt or Ken or Shawn or the other bright bulbs there exposing a light on Joe’s statement.  Are they going to do that? 

If CBS news affiliates can hunt down Letterman and Halderman, I don’t see any reason why you guys can’t shine a light on Joe.

Don’t disappoint me guys.  Be honest here.

That’s good for another 0.25 wins or so per year.

Sigh.

Many people also say this of David DeJesus… without doing a projection, FanGraphs has him at 3.8, 2.7, 2.7, and 3.4 WAR the last four seasons… and that doesn’t asccount for him being in the AL.

People like this are the kinds of people who think that Nick Swisher is an “average” MLBer.

Or maybe they use VORP, I dunno.

Tom, I don’t think you’re wrong for criticizing the comment itself, but I think you’re holding Joe to a different standard—there are things I disagree with (or are even flat out wrong) on BP and THT and FG everyday. If it’s something that I really believe in, I’ll write a comment or a full response on SB, but I’m not gonna take a guy to task for every mistake. Chat sessions especially are pretty dangerous territory—you’re answering hundreds of questions as quickly as you can over the course of 2-3 hours.

I’m sure Joe would probably like that one back (I don’t know that for sure, and I don’t want to put words in his mouth, but I just have a feeling...), but at the same time, if you look through any chat session on any site, you’re going to find mistakes, many of which will be worse than that one. This isn’t out of partisanship—I honestly think those kinds of things just come with the territory.

Shawn, that’s a fair enough position.

At the same time, this isn’t the first time that Joe’s done this.  So, it’s not like I was trying to pick on him specifically.  I used to email him corrections where he was factually wrong (not even unsupportable on opinion, like here), and he didn’t correct himself.  Anyway, I stopped doing that.

(He used to be more receptive, but with me, either I get on your nerves eventually, or you really appreciate what I do.  Joe is just another in a long-line of people who is part of the former.  Typically, those who are more powerful end up in this camp.  Guys like Neyer and Poz, true top-of-their-game guys, still are appreciative.)

Shawn, if that’s the case, then why does he keep saying this?

For instance:

http://www.baseballprospectus.com/article.php?articleid=9000

This isn’t a casual slip-up, this is a repeated claim that Joe makes in spite of the available evidence. (I even used BPro’s own WARP stats to come to the same conclusion!)

Fish in a barrel guys.  Joe is not nearly an analyst, is not careful about what he thinks or says, and says a lot of dumb, “mainstream sports journalistic” things. If you corrected every mis-step of his, it would be a full time job.  That being said, he is 10 times better than 99% of the mainstream sports journalists out there.  I know that some of us still think or at least hope (against hope) that BP purveys mostly accurate, insightful baseball analysis. That just isn’t the case.

In all fairness to them (BP), the do probably purvey “mostly” accurate and insightful information.  Just not as “mostly” as some of us would like and expect from the most widely-known (I guess) sabermetric site.

Agree with MGL. Joe Sheehan fancies himself a “stat guy” but he is nothing of the short. He regularly misuses, misunderstands, or misinterprets data and comes up looking dumb.

all that said, he at least tries, which makes him paper than almost every MSM writer out there

Guys, if you want to criticize BP, or something written on BP, or something that’s been written anywhere else, that’s perfectly fine—in this case, I think it’s been overdone, and I’ve explained why. But still, fair game.

But MGL, whether you intended for this or not, that comes off as a personal attack on Joe, which, given the context of the things we’re discussing (i.e. baseball), I really don’t think is necessary.

Shawn, you keep bandying about words like “overdone” and “different standard.” I don’t agree in the least. I think there’s a consistent standard, and the BPro singularly fails to meet that standard over a long period of time.

Is the Joe Sheehan line about McLouth that important, in the grand scheme of things? Probably not. Is this an isolated incident? I don’t think so.

As a for-instance, your article a while back about QERA:

http://www.baseballprospectus.com/article.php?articleid=9342

Imagine if the entire baseball blogosphere started using the original Runs Created formula—the one Bill James developed circa Off The Wall—as our primary way of valuing a player’s offensive contribution. Forget run environments, linear weights, league adjustments, and all of the other things we’ve learned over the past thirty years; instead, for the sake of efficiency, we went back to (H + BB) * (TB) / (PA). Maybe it’s not perfect, but hell, it’s easy, and it’s not like Willie Bloomquist is going to come out better than Adam Dunn.

Sounds ridiculous, right? But that’s more or less what’s happening right now with defense independent pitching stats.

That second sentence of the last graph is probalby more accurate if it reads:

But that’s more or less what’s happening right now with MLVr and VORP.

Because that’s exactly the run estimator that forms the basis of those crown jewels at BPro. And in spite of people like Tom and I campaigning about it, nothing’s been done about it. I tried sending you an e-mail about this, but no response.

Or, um, the Wieters PECOTA projection. I sent several people at BPro all of my research on the severe problems with the MLEs before I went to press at THT with it. And STILL nobody at BPro has uttered a word publicly about how PECOTA could miss so badly on Wieters.

I could go on and on an on here - the inseason updated PECOTA overweighting current season performance, the fact that WARP1 was “fixed” from a .150 rep-level baseline to a .200 rep-level baseline… I guess I am going on and on and on so I’ll stop.

But please, Shawn, tell us. What standard of accuracy and accountability should we hold Baseball Prospectus to? How much credibility should we expect them to have? Tell us the standard that BPro expects to be held to, and we’ll go with that. But if it isn’t “Deadly Accurate” than stop putting that on the cover of the annual, please.

Shawn/11: I don’t think it’s been overdone.  I think you made a fair point, but Colin made just as good a point by showing that Joe believes what he said, rather than a slip-up:

There’s still a sense that he’s a very good fourth outfielder who is a marginal starter, but even as I type that, I wonder if the concept is dead. How many “good fourth outfielders” exist any longer? The players I tag with that label all seem to end up starting; Shane Victorino, Gary Matthews Jr., Eric Byrnes all come to mind.

I don’t think it’s fair of you (or me) to say that one side has done more (or less) than they need to.  There’s no point to making a conclusion that simply ends the conversation.

***

I also don’t see what MGL did as a personal attack on Joe.  It was a professional gripe, and possibly professional attack.  There was nothing personal there.

There is a reason I don’t subscribe to BP anymore (and likely will break a long running streak by finally choosing to not buy their annual this year-despite spending much moola on baseball-related books/resources).

BP just isn’t a leader anymore.

Actually, one can get along quite nicely without them.

Wow, if that is a personal attack on Joe, you don’t read many of my posts.  I met Joe a while back at one of the SABR conventions I think. He seemed like a really nice fellow.  I have no reason to attack him personally. There is obviously a fine line (and some overlap) between a personal attack and a professional one, and one person’s personal attack may be another person’s professional one, but that is another story…

Colin, shoot me an email whenever and we can discuss, my squawking baseball address should be linked on my name. I do remember your post about Wieters / the eastern league, but if something went through the BP list, it was before I was on it.

Does it amuse anyone else that BP’s fantasy writer, Normandin, seems to have a better grip on real baseball than their two main writers (Kahrl/Sheehan)? Normandin actually seemed to acknowledge Colin’s points about Wieters projections during his preview pieces too. Imagine that.

Shawn: how about we start with something small:

http://www.baseballprospectus.com/glossary/index.php?mode=viewstat&stat=136

Clay has deprecated Pythagenport in favor of PathegenPat.  He says so, right there, on the glossary page.

And yet, any time it’s referenced in an article (except maybe once or twice) or in the annual, it’s referenced as Pythagenport.  I brought this up to Clay 2 years ago, and he agreed with me that it should be corrected.  He alerted whoever, and he cc:ed me.  Yet last year, the annual STILL showed the wrong thing.

Basically, I see this as BPro being sloppy.  This is one example in a list of examples.  How many years does it take to make sure that Patriot and/or David Smyth are properly acknowledged?

After you do that, let’s talk about Lev v LI.

I gotta say that I read the chat in which the McClouth comment was made.  It is an excellent chat.  Joe is quite knowledgeable and most of his answers are pretty darn good.  I give him lots of props.

In defense of his McLouth comment, he gets and answers a lot of questions in a small window of time (I assume), and is therefore bound to make some mistakes and say some incorrect things.

Is that a personal or professional accolade?

MGL, read the quote in post 13.  It wasn’t a slip.

Joe is having another chat today:

http://baseballprospectus.com/chat/chat.php?chatId=657

I’ve asked him for clarification on the issue; let’s see if any is forthcoming.

In one of his mailbag questions a while back, Bill James, when asked why he still used RC instead of BsR when time and time again people have shown that BaseRuns better reflect reality, said [paraphrasing] that he knew that but he’s more comfortable with RC and it’s what he likes to use because he came up with it.

Maybe BPro is just stubborn like this.

At least Bill James admitted to being stubborn.  Was that question from me, by the way?  Cause I asked him a similar question…

At this point you can’t view BPro the way you look at sabermetricians in the non-profit arena (you guys, Fangraphs, THT, etc.).  They are not part of an intellectual community in which ideas are proposed and subject to criticism, and then everyone is expected to adopt best practices (once it’s clear which is best) when conducting future research.  That is the way (more or less) the academic world works, and also the non-profit saber world.  But BPro is a business, and their business is selling (allegedly) unique insights to people.  Emphasis on UNIQUE.  If they aren’t, why would anyone pay for stuff that is available free elsewhere?  They just aren’t playing by the same rules you guys are, and it seems pointless to expect that to change. 

James, also, mostly stands apart from the community.  He does occasionally critique something done by others, or join a debate (I remember him tackling the issue of whether teams’ overperforming their pythag was a repeatable skill, a year when the DBacks were huge overperformers).  But mostly he seems determined to use his own methods, even if that requires reinventing the wheel in some cases (and at times an inferior wheel at that).  I assume in his case this has little or nothing to do with profit.  But the result is basically the same.

Tango/#23:

I don’t think it was you, though I remember you bringing BaseRuns up several times. I haven’t had a subscription for months, so I can’t go back and get the particulars.

Colin/21: did you get any response?

Nope.

At this point, we might as well label Joe as a “mooch”, like JC, and treat them as such.  Ted Tevan fans will appreciate the label.

All (without exception) the regular posters of this blog, in contrast, are “straight arrows”.

Excellent, I’m a straight arrow! Sounds good to me!

I know that there’s issues with BP’s WARP stats and the replacement level they use (I suppose that there’s also an issue with methodology that differs from WAR’s calculation), but I went and checked McLouth compared to some other players near him in WARP1.

Nate combined for 4.1 WARP1 between Pittsburgh and Atlanta. Not knowing the context for these stats, I took a look around at the players around WARP1, which included Dustin Pedroia, Rafael Furcal, Todd Helton, Curtis Granderson and David Wright. Not the best seasons for most of those players, but their production I would say is most certainly above average. Even with their own stats, McLouth looks like a solid player, even at his worst outfield position.

Michael: right you are echoing Colin’s point.  It seems that not a single person at BPro cares that Sheehan is contradicting their own stats, and that he is offering no supporting evidence.

That is the very definition of what I have in the past called: b-llsh!t.

Michael: In order to convert BPro WARP to WAR in a quick-and-dirty way, simply divide PA by 1000 and subtract that from WARP.

The top ten in WARP, so adjusted:

Eh.

Casey Blake?

I’m getting so confused here. If ANYONE thinks they know why Clay might be using a runs-per-win of 8.8 to calculate WARP, please let me know!

Colin: how do you know which version of WARP they are in?

I have saved on a computer at home old WARP values pre-BP ‘09 and have checked them against what’s on BP’s website in the past. The old values (call it OWARP) have an observed replacement level of about .000 win% - literally some years you would see a negative replacement level. The new values (call it NWARP) have an observed win% of about .200.

I haven’t kept up… did they announce any changes, beyond in their annual?

They haven’t. I think they may have made some but at this point I can’t confirm that or tell you what they might have been.

So far as I can tell, WARP1 currently is approximately:

(BRAR+Pos_Adj+FRAA)/8.8

Which gives you an average 2.90 WARP for a position player in 650 PA.

If you do:

(BRAR+Pos_Adj+FRAA)/10.5

you get an average 2.45 WARP for a position player in 650 PA, which gets you in the ballpark. That (and a similar adjustment for pitchers - that I’m really ballparking here) would I think get you to an observed replacement level of .250, still lower than where I think it should be but a drastic improvement over an observed .200 rep-level.

IIRC, Clay figures WARP by running everything through Pythagorean.  So there should not be a straight-line conversion between RAR and WARP.  And since he uses Pyth with x=2 in the “idealized league”, the standard RPW (for a .500 team) should be equal to RPG.  I think his idealized league sets RPG = 9.  But don’t quote me on that.

I don’t know if that is the cause of this discrepancy or not.  I doubt it, but it might be a factor.

Patriot, things make a lot more sense now, although I’m not persuaded they’re RIGHT. In order to make the exponent of 2 correct you need to be using an RPG of 11.18. Something’s screwy here.

Just so everyone knows, in 2006 apparently the Pirates had a player named:

—see #48291

He’s 25 years old, had 0 PA and -1 PRAR1, for a total of -0.1 WARP1.

Looking at all years 1996-2008, I get the following results at the team level:

AVG(WARP) 52.99508
AVG(RPW) 8.17320829

RPW is:

(SUM(BRAR1) + SUM(FRAA1) + SUM(PRAR1))/SUM(WARP1)

Obviously something of an approximation but it should be close enough.

That’s, what, a .180 rep-level? (And these aren’t the OWARP values; compare what’s currently on the site to say, these values:

http://www.firejoemorgan.com/2005/12/warp-goes-mainstream-gardenhire.html

and it’s pretty apparent they’re different.)

I just tested it using BRAR/FRAA/PRAR with Pythag (exponent of two) on a 9 RPG team and I got results far more in line with what I would expect. I really don’t know how they’re getting from the listed values for batting, fielding and pitching to what they have for WARP1.

It turns out the correct answer can be found here:

http://www.baseballthinkfactory.org/files/newsstand/discussion/inside_the_book_tangotiger_baseball_prospectus_warp1_is_wrong/

Two, and more importantly, WARP numbers for individuals don’t add up to the same values that I would get if I evaluated them from their team numbers. Consider the 2004 Orioles. This team actually went 78-84. Since the replacement level win% is .153, their actual wins above replacement is 78-162*.153=53.2. Call it 53.

The sum of their individuals is 81, a big difference to be sure from the desired 53. But consider the team totals of batting rar (217), fielding rar (164) and pitching rar (381). These numbers have been adjusted to the standardized league of 9.0 runs per game, pythagorean exponent of 2. In the standard, a replacement level team scores 537 runs per 162 (replacement level EQA of .230, divided by league average of .260, raised to 2.5 power to convert it to runs, times 162 times 4.5 runs per team per game = 537) and allows 1262 (replacement pitching and fielding together yield a standard 7.79 replacement ERA, times 162 games = 1262).

A team that scores 537 and allows 1262 has a Pythagorean win pct of .153. The Orioles adjust to 537+217 brar=754 runs scored and 1262-164 frar-381 prar=717 runs allowed. That gives them Pyth% of .525 (85.1 wins), which makes them (.525-.153)*162=60 wins above replacement.

So we estimate the team as being 60 warp, and in real life they had 53 warp. That’s not so bad, considering that they underperformed their real-world Pythagorean estimate by about 4 wins, and they allowed a few more runs than expected based on their pitching statistics.

Some of those figures have changed with the move from FRAR to FRAA; the equation now looks something like:

537^2/(537^2+990^2)

In other words, WARP1 is measuring “how many wins is a player worth to a .227 Win% team?” Which means that we end up with an observed Win% of around .170. (Ignore when I said .200 earlier - apparently they haven’t updated their WARP1 data for the entire season of ‘09 yet; I now have over a decade’s worth of WARP1 data to play with.)

That’s why the run per win values seem so low; you have a diminishing returns on marginal runs as Win% goes up - a run for a rep-level team is worth significantly more than a run for an average team.

So we’ve moved rep-level from the 1899 Spiders to the 1916 Athletics… is this progress?

Or maybe even more to the point, is it progress worth paying for?

Am I reading that right?  It sounds like they are justifying the Orioles’ 81 cumulative WARP being so high by saying that the team WARP is really only 60.  I guess it probably has to do with Colin’s point that they are looking at each player’s contributions to a replacement level team so that everyone’s effects are overstated, but are they really justifying the stat by saying that even testing it against the exact same measure applied differently, it’s wrong by over 20 wins, so the amount its wrong by brings it closer to reality?

Clay seems to have fixed the issues addressed above with Pythag. 2009 WARP1 seems to add up to a .220 win%, which is about where it “should” be, given Clay’s assumption of a .230 EqA offense and a 6.11 RA defense for replacement level.

He does not, on the other hand, seem to have told anyone at BPro what he did:

http://baseballprospectus.com/rt/rt.php?rtId=23