THE BOOK--Playing The Percentages In Baseball

Friday, January 15, 2010

How does a sim do in projecting team wins?

Over the years, I have mentioned many times that I can run scenarios through my “sim” to analyze game strategies and the like.  Basically, the sim takes my batter and pitcher projections and tries to simulate a bona fide game, PA by PA, using an odds ratio matchup between the pitcher and batter for each PA. It also uses reliever, pinch hitters, etc.  The projections are component ones, and they include defense (UZR), base running, catcher ratings, SB/CS projections, etc.

How good are these sims? I don’t really know to tell you the truth.  I think they are very good for analyzing in-game strategies.

To test how good they are at the whole-game level, here is what I did:

I simulated the entire 2009 season for every team using projections for all the players which are updated every 6 weeks.  I did not account for injuries and I only updated the basic batter and pitcher projections.  For base running, defense, and the rest of the inputs (other than pitching and hitting) I used pre-season projections for the whole year.  I also had no way of accounting for a player playing injured or some such thing.  To simulate each game, I used the exact lineups for each game.  For pinch hitters, and relievers, I used a file which includes the likely pinch hitters and relievers for that month.  The way I did that was I recorded the batters with the most pinch hit appearances for that month, and the pitchers with the most 8th, 9th and earlier relief appearances.  I assumed that the pitcher with the most 9th inning appearances for that month was the closer (so that the sim used him as the closer), the one with the most 8th inning appearances was the set-up guy, and I used the two pitchers who had the most pre-8th inning appearances as my long men for that month (the sim only uses two long relievers).

So what I did when all was said and done was record each team’s first half and second half win percentage according to the sim.  We can compare that to the actual first and second half w/l precentages for each team.  Remember that I am actually simming each game in the schedule for each team, with the pitchers and batters who actually faced each other, at least the starters that is.  It takes a lot of computer time to do this, so I only simmed each game in the schedule 3000 times, which is not a lot for a simmed game but it will have to do.

After the 1st half and 2nd half data, I’ll show the expected wp (according to the sim) and actual wp for each team FOR THE WHOLE SEASON, again, with the sim updating the projections every 6 weeks as the season goes on.  Remember that it uses the exact lineups for every game and an approximation for the pinch hitters and relievers based on actual usage for each month of the season.  The closers and set-up guys are pretty accurate. 
Here are the data with no commentary:

Read More