THE BOOK--Playing The Percentages In Baseball

I don’t know much about this ongoing debate, but Diamond Mind’s methodology is obviously very far off given that sportsbooks think the Dodgers win about 55% of the time.

JB H #1:

How are you deciding that, of two differing predictions of an event that hasn’t happened yet, one is “obviously very far off”?  Is it just that only one of the two prediction “owners” has a big financial stake in being right, and thus you figure they probably worked and thought harder over their prediction? 

If that’s it, I can understand, and knowing what I know about the bookmakers, I’d probably side with them too, just wondering if maybe you knew something particular about DMB that was wrong in your opinion…

It’s the former.  Also remember that bettors will punish a badly set line and the books will correct the mistake.  A sportsbook would never be off by the margin Diamond Mind suggests on a big market like this, not even close.

I don’t know specifically what mistake Diamond Mind is making.  My guess is that they’re not properly regressing platoon splits, but that’s just a shot in the dark.

I don’t know what the inputs are in the DM sims and I don’t know how the engine works either.  I can say with a high degree of confidence (probably around 99%) that the notion that the Dodgers are a 3-1 fave versus the Phillies is preposterous.

The Vegas odds are now around Dodgers 54%.

“A sportsbook would never be off by the margin Diamond Mind suggests on a big market like this, not even close.”

That is 100% correct, although I would change “never” to “almost never,” as in 99.99% of the time.

MGL@4:

Sportsbooks do respond to human biases, do they not?  Aren’t the Yankees over-favored when they’re playing a last-place team?  (Which may be better bias, or just people using the Bill James head-to-head winning percentage equation.) Don’t the lines have some trouble differentiating between long-term and short-term pitcher performance?

Diomond Mind simulations are too “confident” in the estimates of a teams true talent level.  Meaning that the theoretical standard deviations they give on the projections for each team are way too small.  That will tend to overrate a good teams odds of winning a series against a worse team.

Even if you knew the true talent level, you would need the disparity to be quite large to give one team a 76.5% series win probability. 

Sticking with a simple binomial model (which obviously has its flaws), you would need a .630 team matched up with a .500 team or a .600 team matched up with a .470 team. 

So while I have no reason to doubt vivaelpujols when he says Diamond Mind is too confident, I think the problem goes a lot deeper.  Their estimate of the relative strength of the Dodgers and Phillies just don’t make sense.

That makes a lot of sense Patriot.  I doubt that their variance around the projected mean is actually less than that of a binomial distribution, so it must be something wrong with their projections.  Does anybody know how they are calculating them?

And BPro has the Philles at 67% to beat the Dodgers…

http://baseballprospectus.com/statistics/postseasonodds.php

BPro’s projection is equally preposterous.

#5, that is true to a small extent.  It is much less true these days than 5 or 10 (or more) years ago.  There are too many smart betters these days that know the difference between the predictive value of short and long term performance and do not have the biases you are talking about and they “correct” a bad line very quickly.

There is virtually no chance that Vegas would have two teams around even when one team should be a 3-1 fave (or even a 2-1 fave).

Not to mention the fact that there is little doubt that both the Phillies and Dodgers have very good but not great teams.  It doesn’t take a genius to figure that out.  There simply can’t be a great disparity between these two teams in a 7-game series.  As Patriot shows, it would take one team being .630 and the other being .500.  There is virtually no chance that either team is a true .630 team or more AND that either team is a true .500 team or less.  If you multiply no chance by no chance, you get something like no freaking chance, which is the probability that one team is a 3-1 fave over the other in a 7-game series. I can’t believe that a proposition that the Dodgers are a true 3-1 fave over the Phillies is even open for discussion.  Seriously.

The fact that DM has the Dodgers at 3-1 and BP has the Phillies at 2-1 tells you that one or both of them is completeley insane.  In this case, it is both…

BPro is probably using the in-season PECOTA projections to fuel the playoff odds. Can anyone look up the ERA projections for the playoff starters for each team and post them here, please?

Obviously, both the BP and DMB systems are breaking down.  Both appear to be fundamentally flawed and cannot be trusted at this point.
vr, Xei

MGL @ 10 - I wasn’t suggesting that the BP or DM lines made any sense.  Just saying that crowds are sometimes a little unwise…

My friends at Diamond Mind Baseball have played the NLCS 1,000 times, and the results are even more lopsided than I guessed: Dodgers 765, Phillies 235. Oh, and the most common result? Dodgers in five.

In order for a disparity like that to make sense, you’d have to assume the Phillies are around an 89 win team as currently constituted and the Dodgers are around a 109 win team.  While it’s possible platoon advantages may cause a disparity between what seem like two fairly even teams, there’s no way they can account for that type of disparity, I don’t think.

And BPro has the Philles at 67% to beat the Dodgers…

A little more reasonable, but it still doesn’t pass the sniff test to me. Looks like they’re assuming a Dodgers team of around 102 wins if the Phillies are an 89 win team.

FWIW, I got odds of Phillies: 54.3%, Dodgers: 45.7% running 10,000 iterations of Monte Carlo simulations after re-projecting the Dodgers and Phillies and estimating playing time distribution in the NLCS (click on my name to see the post).

Sorry, messed up the link. This one should work.

http://www.replacementlevel.com/index.php/RLYW/comments/2009_nlcs_preview_dodgers_vs_phillies

I happen to know a lot about DM.  I ran each game with each pitching matchup at least 1500 times.  DM likes the home team to win every game, but most of them are close.  Thus DM likes LA in seven.

"Thus DM likes LA in seven.”

So where does this “LAD 3-1 fave” that Rob Neyer mentions come from?

I did a search on the internet, including the Diamond Mind and Imagine Sports web sites and I have been unable to find any reference for DM simulating the LAD/PHI series, other than that mentioned by Rob and one other web site.  Anyone know where that Neyer citation came from?

Both Bpro and DMind are making incredible claims here.  They would have to provide some serious evidence for me to take either seriously.

It’s been a long time since I’ve worked out probability problems.  From the examples above, it looks like a 20 win talent gap between 2 team means a 70% chance of the better one winning.  Is it a reasonable rule of thumb that ever 1 win difference = 1% chance of the better team coming out on top?

Obviously it breaks down somewhere, as a 110 win team, even if playing a 60 win team, is not going to win 100% of the time, but for most matchups we’d actually see in the playoffs, would that formula get me to within 2-3% of the rigorously calculated odds?

I wonder if a simulation that had the Dodgers winning that much put in 2009 stats instead of projections, and kept Brad Lidge in the closer role.  If he were a true talent 7.21 pitcher, and the Phillies rode him like it was 2008, that might guarantee a Dodger win.

I have no doubt whatsoever that DMB used 2009 stats as the “true rates”, and the 2009 usage pattern (meaning Lidge as ace reliever).

What BPro did, I don’t know.  And, once again, I’d like to see someone, anyone, at BPro speak up.

That would make Randy Wolf a DMB true talent a full run better than Cole Hamels.  Though it would make J.A. Happ even better than that.

Diamond Mind uses 2009 stats, and they probably only used ‘time with team’ stats, so Vincente Padilla will be a 3.20 ERA pitcher, John Garland will be a 2.72 ERA pitcher, and George Sherrill willbe an 0.65 ERA (!) pitcher in their simulations.  Oh, and Ron Belliard will be a .351 hitter....

That shouldn’t doom the Phillies then.  They can start Paul Hoover (.750) at catcher and find some time for Andy Tracy (.417) at 3rd base.

All of BPro’s numbers seem that radical:

http://baseballprospectus.com/statistics/postseasonodds.php

Between that and Vegas Watch’s numbers about the preseason predictions, I can’t help but wonder if Clay did something to his Monte Carlo sim that breaks it.

Between that and Vegas Watch’s numbers about the preseason predictions, I can’t help but wonder if Clay did something to his Monte Carlo sim that breaks it.

They had Boston at 50% odds of winning the ALDS when down 0-1, and 30% when down 0-2.  It sure seems like something’s not right there.

To have a 30% chance to win a best of five series down 0-2, you need, what, a .670 expected win percentage versus your opponent? Utter madness. They were only .586 in the regular season.

So the Angels have only a 30% chance according to them?  I wish a casino would give me such odds.  I’m feeling lucky.

These projection guys don’t seem to care what they put out because there are no negative ramifications if they are wrong—meaning they aren’t putting their money where there numbers are.  If they did, I’d have to imagine (hope) that they were smart enough not to believe their own numbers.  Whenever I read these numbers, I take them with a grain of salt knowing that most likely the person putting them out has little reason to try to get them right...and also does not think he will get called out on being wrong.

Here’s the numbers I made regarding likelihood of winning

LA W 4/4 7%
LA W 4/5 10%
LA W 4/6 18%
LA W 4/7 22%

PHI W 4/4 5%
PHI W 4/5 9%
PHI W 4/6 14%
PHI W 4/7 15%

Here’s the percentages of each game in the series, guessing for the starting pitchers of course.

Philadelphia (Hamels,Cole) 42% vs. LA Dodgers (Kershaw,Clayton) 58%
Philadelphia (Martinez,Pedro) 45% vs. LA Dodgers (Padilla,Vicente) 55%
LA Dodgers (Kuroda,Hiroki) 43% vs. Philadelphia (Lee,Cliff) 57%
LA Dodgers (Wolf,Randy) 53% vs. Philadelphia (Happ,J.A.) 47%
LA Dodgers (Kershaw,Clayton) 51% vs. Philadelphia (Hamels,Cole) 49%
Philadelphia (Lee,Cliff) 48% vs. LA Dodgers (Padilla,Vicente) 52%
Philadelphia (Martinez,Pedro) 40% vs. LA Dodgers (Kuroda,Hiroki) 60%

King/29 is absolutely right.  Unless you are going to stand behind your product, there’s no reason to even talk about their numbers.

Mike, there is inconsistency between your numbers.  Assuming you think the pitchers and odds of each individual game stays constant despite what happened in previous games (my guess is this is probably accurate for MLB, but not for some other sports like NBA), then given your individual game odds, you should think the chances of the series ending should be:

LA in 4 7.3%
LA in 5 13.9%
LA in 6 16.9%
LA in 7 18.8%

PHI in 4 5.1%
PHI in 5 11.1%
PHI in 6 14.4%
PHI in 7 12.5%

Thanks King. I’ll double check and see where I went wrong. How did you calculate that out?

I figured out the probability of each possible outcome and then added them up for the given result.  Nothing elegant, just brute force.

King’s numbers are exactly right.

Clay corrected himself:
http://www.baseballprospectus.com/unfiltered/?p=1405

His post also reminded me what he did last year about the L/R platoon splits, and we were all over on him about that as well.  It looks like he decided to remove those splits from his program, as well as fixing his other bugs.

Clay is a straight arrow.

Now what about DM, which had an even more preposterous result - although I have yet to see who or what ran the DM simulation that made the Dodgers a 3-1 fave?

Is BPro’s 73% forecast for NYY plausible?  That means you think Yanks are about .610 vs. LAA.  Seems hard to believe, given Yanks’ pythag winning % against the league was .586.  I realize there are other considerations, but 73% seems high.

Seems hard to believe, given Yanks’ pythag winning % against the league was .586.

I think it’s high, but the Yankees having a .586 pythag against the league isn’t particularly useful in assessing this series.  Unless the weather screws things up, they’re going to get 3 starts from C.C. Sabathia and 2 starts from Burnett and Pettitte.  No Wang, no Mitre, no Joba. 

They’ll also be playing their best players as much as possible compared to the regular season where they had people like Cody Ransom and Angel Berroa playing big chunks of time at third base among other issues.

Of course, the same applies for the Angels.

If the Yankees use Sabathia three times, I’ve got them as around .543 against the Angels.  If one of those Sabathia starts goes to Gaudin, it’s more like .535.

SG, I did a simply straight-line approx. for the log5 method with actual win percentage:

.5+.636-.599 = .537

About the dumbest analysis possible, but it’s pretty close to what you’re getting. The BPro estimate for the Yankees series is puzzling.

I don’t have any problem with BP’s Yankee/Angels forecast.

Rather than looking at pythag, actual records, etc., how about “just” looking at each player in the lineup’s defensive, offensive, and base running projection, plus the pitching projections for at least the starting pitchers.  THAT is going to give you a decent forecast for the series, not those other things.

I agree that the Vegas odds are likely to be more accurate than the outlandish simulation results, but let’s not give them too much credit.  These are the same books who offered 125-1 on Tampa to win the AL pennant in 2008, and who made Japan the favorite over the USA in this year’s WBC semifinal.

"THAT is going to give you a decent forecast for the series, not those other things.”

Has anyone actually shown this to be true?  I know player projections should be better, but has this ever been demonstrated?  And lets say you calculated pythag based on base runs and component ERA, and adjusted for HFA.  I wonder if player projections would improve your forecast enough to be worth the trouble....

"I wonder if player projections would improve your forecast enough to be worth the trouble....”

As opposed to what other method?

Sometimes whether something is appreciably different (better) than something else overall does not tell you what you need to know.

Let me give you a germane example:

Let’s say that we have two forecast methods and that if we test both methods on a large sample of players or teams, they come out reasonably close to one another, even though method A is a little better.

Now let’s say that method A is much better in a small proportion of cases.  This would be the case, for example, if we used, say two years of historical data on a player in order to forecast his future performance, as opposed to 4 years.  Or anything like that.  Both methods would be pretty close to one another in the aggregate, but occasionally we will find a player who got really lucky or unlucky (or hurt) in those two years and using 4 or 5 years gives us a much better forecast.

Now, if, when I see that occasional player forecast where method A (using more than 2 years of data) gives much better results, I complain about your method B, and say that it is wrong, you can easily counter with, “Well, I can show you that your method is not much better than mine, therefore your complaint about THIS forecast is hardly warranted.” You would be wrong of course.

Same thing with forecasting team strength.  If you used, say, a teams run scored and allowed to forecast their their true talent, 95% of the time you would get around the same answer as if you used individual forecasts for their players using the usual forecast methodology for those players.  But, you would be able to identify the 5% (or whatever it is) of the time when they had a really lucky or unlucky season in terms of team runs scored and allowed, not to mention the times when team personnel significantly changed, of course, due to injuries, trades, etc.

OK, assume we do care about being better 5% of the time. Has anyone shown that using individual players is in fact any better at all (compared to a “smart” pythag analysis)?

And why are you OK with the 73% Yankee projection?  Is that about what your projections show, or what you think they would show?

yes, I have the Yankees at 72%.

“Has anyone shown that using individual players is in fact any better at all (compared to a “smart” pythag analysis)?”

Why would any projection be better than some backhanded way of projecting performance?  Of course individual player projections are going to be MUCH better than a team’s pythag record or actual record, if for nothing else you know who is going to be playing in the post-season.  A team’s seasonal record or pythag record is poor indication of who is going to be pitching and playing in the post-season (obviously).