THE BOOK--Playing The Percentages In Baseball

Q: “What’s the next traditional strategy that will be vindicated by proper sabermetric analysis?”

A: “If we can get the run environment down by half a run, it’ll be the stolen base. And in fact, that environment currently exists in the playoffs, and so, stolen bases should be more prevalent there.”

I doubt that a change in run environment is even required.  If you removed all the hit-and-run plays (SBAs on which batter swung)—which likely produce a disproportionate number of CSs --- and took into account the pitch count (as well as score and base/run state), I think we’d discover that teams are already rational in their use of the SB weapon.  (In the aggregate; individual teams might run too little or too much.)

My answer to this would be the 5-man rotation, which The Book already provided evidence in support of, but I don’t think is generally supported by the sabermetric community.

I was thinking more that fans don’t necessarily appreciate the cost of the CS, and therefore may not realize that the high SB totals of 20 years ago may have been justified.  That if they were more exposed to differing run environments, they’d have a different view of the SB/CS tradeoff.  And the playoffs should give them that view.

That makes sense, Tango.  I’m just hoping to prod someone into doing a SB study that accounts for the busted H&R factor and other factors that make it appear that teams run “too much” (which I don’t think is true).  I thought Dan Fox might do it, but now we’ve lost him to the Pirates.

Similarly, I think teams are right to give starters 4+ days of rest, but sabermetric conventional wisdom (to the extent such a beast exists) is that this just means giving more innings to inferior pitchers with no offsetting benefit in performance or injury reduction.

I have done the study.  I am preparing it for publication in a week or so.  Preliminary results have the break even point at about 62 to 65%.

Peter/6: there are two issues: (1) what’s the breakeven point and (2) what is the league’s actual success rate

Guy was talking about (2), and saying that because of the artificial CS, the league success rate is too low.  That is you remove the busted H&R, the success rate would rise.  I don’t disagree.

For (1), Table 137 of The Book shows the breakeven rates at various scores, with an overall average of 68.7% for stealing 2B (for the 5.0 runs per game environment).

Now, your post seems to be tackling (at least) issue #1, and you are saying that the breakeven SB rate is 62% to 65%.  Is this a contradiction from Table 137?  And, are you also looking at what Guy is talking about?

Looking forward to whatever it is you are going to publish!

Great.  Where should we look for it?

Its pretty long and involved. I was thinking the Retrosheet research papers area but I am open to suggestions.

Here are several things about current MLB stolen base strategy:

When analyzing anything about SB rates, you MUST include EXACTLY what happens on the field, which should be obvious, but for some reason was the gigantic and fatal flaw in much of the previous sac bunt analysis:

So, in addition to only looking at true SBA (removing hit and runs and whatever), you need to include stolen base errors by the catcher or middle infielder whereby the runner advances another base or two, OR the runner is safe even though the official scoring is a CS (only on a middle infielder error).

You also might want to restrict the analysis to less than a certain score differential to eliminate the SB that are NOT scored as defensive indifference, BUT the defensive team clearly did not care about.

You must include pickoffs, balks, and pickoff errors!  Another gigantic mistake made in some analyses.  That is especially true against left handed pitchers, where there are a lot of pickoffs.  Then you also have to distinguish between pickoffs where the runner is safe (which usually includes an error) and pickoffs where he is out.

Finally: We have to try and figure out how often the batter hits into a line drive and sometimes short (hang time) fly ball that he otherwise would not have had the SBA not occurred.  And how often the runner advances on a hit when he is running on the pitch, and how often (if at all) the batter gets extra ground bal base hits with the runner going and the infield a little out of position.

Without all of those things included, the SB analysis is worthless!

There is one more thing which is a critical point.  It really makes no difference whether in the aggregate teams/managers are above or below the average BE point.  To figure out whether teams are running too much or too little, in the aggregate, tells us practically nothing about how many runs they are “giving away” as compared to the theoretical optimal SB strategy, which is really the only thing we care about.

First of all, it is likely that the optimal SB rate is much higher than the overall BE point (I think), so that if the overall BE point was 71% and the actual SB rate for MLB was 71%, it is likely that they are running too much.

But, again, we really don’t care (that much at least) whether overall they are running too much or too little.  What we care about is how often they are running in any particular situation and with any pitcher/catcher combo as compared to how often they should run.  For example, we may find that in a certain situation, say early in a game with 1 out, they are running the correct overall amount of time, based only on the SB rate, which is higher than the BE point.  BUT, it may be that in that situation, they are running too little with their best baserunners or against the worst pitcher and catcher combos, or too much against the worst pitchers/catchers, etc., or any such combo.  It is kind of like sac bunts or IBB’s.  We might be able to tell overall whether they are above or below the BE point, but what we really want to know is how often they make a mistake by either not running when they should (in the case of the SBA) or running when they shouldn’t.  All kinds of combinations of wrong individual strategies can make up the same overall rate.

I mean, what if all teams only ran 4 times all year and had a 75% SB rate.  Does that mean that they are running optimally?  Probably not, but we would have to try and figure out what profitable opps they missed in order to do any kind of interesting and worthwhile analysis. If they had only 3 SBA per year and were 2 for 3, would that mean that they are running too much, because their overall SB rate (67%) was less than the BE point?  Of course not.

So let’s get away from this notion of looking at overall rates and then making conclusions about whether teams and managers “get it right!” One simply does not follow from the other.

Gosh, I included all that and without your help too!  Although I calculated a break even point for the three year aggregated data because it is useful to compare with the break even points calculated without running on the pitch hit balls, I used total runs gained as the most relavant measure of success.  The one thing I did not do that you suggested is try and separate running on the pitch that was part of a hit and run and running on the pitch that was part of a straight steal.  The retrosheet data doesn’t allow for an easy distiction between the two.  Since the runner suffers the negative consequences of a swing and a miss in either case it seemed that all the positive gains should be his as well.  I did look to see if there were more swings or misses or more called strikes in certain situations and I could find no pattern that would indicate that the batter was acting any differently on a steal than a hit and run.

Oh, I also included the run value of all the unsuccessful pitchouts.  Something you forgot to mention.

I’d suggest Hardball Times, if you can break it out into parts.

The Retrosheet area is fine too, if you want it as one “paper”.

You can also try Birnbaum’s By The Numbers, or BEpress.com, or I can create a folder on tangotiger.net

If you need help getting to any one of these people, let me know.

Hey Peter,

We’d be happy to take a look.  Sounds great.

In case you’ve forgotten it, I’ve included my email address in the link.

Peter, wherever you post it, I’m eager to read it. 

And as a side note, I’m curious how one would determine which plays were hit and runs versus a straight steal, even in theory.

I just remembered, Ted Turocy had a paper about the SB.  He wanted me to read it, but I put it off until after I wrote my chapter.  Time to read it.  He’s got it here, along with a few others:
http://ideas.repec.org/e/ptu12.html

Oh, I also included the run value of all the unsuccessful pitchouts.  Something you forgot to mention.

Yup, I forgot that one!  I think you have to assume that a certain percent of “steals” are hit and runs and try and remove them, even if you don’t know which is which.  And I think you have to assume that on a hit and run the runner gets caught 40-50% of the time and do the appropriate seat of the pants adjustments.

If you do some digging on other people’s research, you can probably find out the approximate frequency of hit and runs and go from there.

I’ll be very interested in reading your paper!

Peter:  does the data indicate whether the batter swung on the pitch on which the SBA took place?  If so, you can just treat all of those attempts as H&Rs, and exclude them from your analysis of “true” SBAs. 

Can you even identify H&Rs on which the batter does put the ball in play?  That is, does your data indicate when a runner broke on the pitch?

Retrosheet data indicates when the runner was running on the pitch.  It is those plays that I am evaluating and adding to the total of stolen bases, caught stealings, pickoffs, pickoff errors, and balks.  I am not eliminating any of the caught stealings where the batter swung on the pitch.  Essentially I am evaluating every play where the runner began running on the pitch or where a play was made on him because it was thought he was going to be running on the pitch.  The tough part was properly evaluating how many runs would have scored had he not run.

So your analysis will cover all plays on which the runner takes off, essentially combining straight steals and hit-and-runs.  That should be very interesting to see. 

Still, if you’re so inclined, I think it would be worthwhile at some point to also do the analysis separately for straight steals (no swing) and hit-and-runs, to see how well teams are using each strategy.

Guy - I don’t think you can separate the two on the basis of swing, no swing.  There are so many base stealers that are given a green light to steal any time they think they can be successful that many of the swings, either swing and miss, foul ball or in play, will occur on non called hit and run plays.

Another possible thing to take into account/do separate analysis of is running on 3-2 counts, which are a somewhat similar to hit and runs, and would presumably have a lower overall success rate since teams often try that with runners/pitchers/catchers that they wouldn’t otherwise with a different count.

Yes, you can’t separate based on whether the batter swung or not!  You can do one of two things (or a combination).  As I said, try and find out the approximate percentage of true hit and runs.  Other database companies track these things, and I think that you can find some numbers on the number of hit and runs somewhere (maybe even the BJ Handbooks).  Then you can separate out some percentage of all steals where the runner was running and assume that they would get caught 40%-60% of the time if the batter misses.  OR, you can look at the runners and the count when the batter swings. If the runner is not typically a base stealer and the count is 2-1, you can assume that this is a hit and run.

You definitely should eliminate the 3-2 count completely.  That is a completely different animal from a steal or a hit and run, and that situation deserves a separate analysis. Including it will only screw up the base stealing analysis.

http://ideas.repec.org/e/ptu12.html

With all due respect to Mr. Turocey, good luck!  For one thing, there is NOT much game theory involved in base stealing (a tiny bit with respect to pitch-outs and pickoff-attempts), and I THINK this is a paper that involves game theory analysis. In any case, I don’t have the time to read it, nor the expertise to understand it.

In the part where he describes baseball in general and base stealing in particular, I would have just said, “If you need me to explain baseball and base stealing, you probably have no interest in reading this paper.”

MGL - My criterion for which running events were included was any event where a non swing or a swing and a miss could result in a CS.  That meant including all 3-2 counts with less than 2 outs and 3-2 counts with 2 outs and a man on 2nd but no runner on first.  I am only using the Retrosheet data for better or worse.  I don’t know how other companies could track actual hit and runs unless they were privy to internal club information.  Aggregate percentages of hit and runs would not be helpful to my study.  I would need to differentiate which individual plays or which defined classes of plays are hit and runs.  If my decisions on these matters end up making the information in my study useless to you that is a result I can live with, and perhaps a conclusion that you should wait and see the study before making.

Peter/19:  I take your point that not all SBAs on which batter swings are hit-and-runs.  And there is certainly real value in analyzing all these plays together, as you’re doing.  So I look forward to seeing that. 

That said, I think making some effort to distinguish H&Rs from straight steals could be very useful. The H&R really is a different animal, often involving players who aren’t true basestealers, and of course the success/failure of the play has many more dimensions (hitter’s success in putting ball in play, DP frequency, base advancement, etc.).  And while not all SBAs on which batter swings are hit-and-runs, it is true that virtually all non-swing SBAs are straight steals, and nearly all H&Rs are captured in the batter-swings category. But I’ll reserve judgment until I see your paper.

One question:  have you factored in the role of the count on the batter when a CS is 3rd out of the inning?  A lot of SBAs occur with 2 outs, and also on 2-strike counts.  I assume a non-trivial # of CSs occur with 2 outs when batter is in the hole.  The cost of that CS is much less than the out/base state alone would suggest.

Guy - Your last point is a good one.  The role of the count should be considered in a complete reexamination of the nuances of SB, CS, and H&R evaluation.  But it shouldn’t just be done for some CS’s.  If it is done for some running on Pitch events it should be done for them all.  That is another major undertaking.  I have limited myself mainly to the role of the extra runs that come from running on hit ball events.

If others want to try and break out H&R events they should do so.  I basically feel that the decision to put the runner in motion is a separate strategic decision from whether to have the batter swing or not.  Sometimes the manager makes both decisions, sometimes he makes neither, and sometimes he makes one or the other.  I have no way to know which is which and I am not trying to second guess the decisions.  I feel by analyzing the results on a team basis you can say that whatever the decision making process was some teams gained runs aned others lost runs.

Peter, why so defensive?  If we want to know about the strategic use of the stolen base, surely we want to make some attempt, if we can, at separating out the hit and run.  And surely we want to exclude the 3-2 count, since that is NOT a stolen base attempt and will only confound the data.  What is the problem with that?

As far as other companies tracking the hit and run, they definitely do that, it is NOT that difficult to figure out which plays are hit and runs when you are actually watching a game, and yes, they will have a small percentage of errors, because some plays are ambiguous.

If you don’t make some attempt to factor out the hit and run and eliminate the 3-2 count, I just don’t know how the results are applicable to a “strategy” for the stolen base.  Since the only real problem is an increased CS rate, you CAN make a “manual” adjustment if you can approximate how often the hit and run is executed AND about how often the runner is cut down when the batter does not swing or swings and misses.

The 3-2 count and 2 outs is probably no big deal (in fact, that is probably a good situation to see how often the runner gets an extra base by virtue of running), since the runner obviously cannot get a SB or CS, but the 3-2 count with 0 or 1 out is pretty much like a hit and run from the perspective of the runner, where you are going to get an inordinately high number of CS when the last pitch is strike 3, since non-base stealers are often running on the pitch AND they often don’t run very hard, even though they should.

Maybe I am overstating the effect.  I’m not really sure without creating or looking at the data myself.  But it sounds like you are doing a thorough and comprehensive job, which is great, and a long time in coming, so why not at least attempt to “go all the way.” It would be a little bit of a shame if you did a comprehensive job, and then had to include a disclaimer, something to the effect of, “The real CS rate, and thus my overall conclusions about manager SB strategy being god or bad, is lower, because of the hit and runs and 3-2 counts.”

But, I certainly look forward to reading what you have, and then I’ll make any comments when I have.

There’s an article in the new Sports Illustrated called “The Art of the Steal”. Here are a couple quotes:

1) “According to Baseball Prospectus, a hitter’s BAvg jumps 15 points when a basestealing threat is on first, with a slight bump in power as well.”

2) “Well, consider that every SB increases a team’s expected runs by .25 per game according to BPro, whereas getting thrown out reduces them by .64.

Seems to contradict what we ‘know’. I didn’t subscribe to BPro for 2008, so I don’t know if that’s what they actually purported, or whether this writer is out to lunch.

The breakeven point seems awfully high. Chris Ballard, author of the article, says that the breakeven point is 75%. From 1999-2002, in a 5.0 RPG run enviornment, Tango said that the breakeven point was 69%.

These are the links being discussed:

sportsillustrated.cnn.com/2008/writers/chris_ballard/09/10/art.of.the.steal/

sportsillustrated.cnn.com/2008/baseball/mlb/09/10/ballard.hendersonrollins/

***

I looked at when a steal was attempted in The Book.  In a late/close game, the breakeven points starts to go down alot.  And, if you are smart, you steal more when the breakeven point goes down.  I think I said the breakeven point was .687 and the actual basestealing rate averaged .678.  Naturally, if you steal only when you are at .687 or above your success rate should be higher than that, say at .72 or .73.

If you are going to discuss stealing, you have to know the inning and score.

And if you are going to discuss the art of stealing, you should make Tim Raines and Carlos Beltran your centerpieces.

Those numbers look similar to what you’d get from BP’s 2008 RE chart. If you assumed that the only basestealing that was happening was a runner on first with no outs in the inning.

Which is, in point of fact, not how it works in the real world. But it’s how Wolverton and Sheehan explained the concept:

http://espn.go.com/mlb/columns/bp/1202793.html

http://www.baseballprospectus.com/article.php?articleid=2607

(The only other thing I can think of is that they’re counting pickoffs seperately, which is because that’s how I did my LWTS recently and that produces similar values. Of course, that then means that you can’t, y’know, apply them to traditional statistics.)

Either way, it’s a mess.

Great find Colin.  Nothing worse for analysts than to be misquoted by the mainstream.

The quote in the SI article about the batter is troubling as it implies that having a base stealer on first is a big benefit to the batter, which confirms conventional wisdom.  However, most of the analysts, including myself and Tango, have found that while the batting line of the batter may change, the overall result is a net of around zero (for all batters, although we both found a great difference between young and old batters).  I don’t remember off the top of my head if the BA was increased (and then obviously other positive things are decreased).

Anyway…

Even if you are assuming a SB attempt (of second) with 0 outs and no on else on base, or across all outs and base runner configurations, and you used the RE tables, you have to include the errors on the throw where the runner gets to third (I think around 1 out of 30 throws), the runner is safe at second when he gets credited with a CS, pickoffs at first base (without SB attempts, in general, there would be few pickoffs), and balks (this is controversial - whether to include them), since you can say the same thing about them (as you do with pickoffs), and the extra value of the runner going on the pitch and the ball is put into play (extra bases on a hit plus fewer GDP’s minus a few extra line drive and short fly ball DP’s).

I think that covers everything.

Then you can compute the overall BE point.

But even then, as several people have already pointed out, this number means nothing for three reasons:  One, it is different depending on score, inning, outs, etc.  Two, steals are attempted at different times, generally more when the BE point is lower (high leverage situations). Three, if you want to “compare” a BE point to the actual success rate and generate a statement or conclusion, you have to remember that the BE point is the minimum percentage at which a steal should be attempted in any given situation, Tango likes to point out.  This means that the actual success rate should be quite a bit higher.  For example, if the BE points in a certain situation were 75% and the league average were 75% in that same situation, then lots of runners are going when their success chance is a lot less than 75%, since there are clearly plenty of fast, excellent runners whose success rate is going to be higher than 75% (and there should be no players with a true success chance of less than 75%), and since even for an individual runner, he is sometimes going when his success chance is higher than 75%, but he should never be going when it is less than 75%, yielding an average much higher than 75%.

One way to look at it (to, say, evaluate manager or team strategy) is to look at individual players’ success rates in various situations and compare them to the BE point.  Although you are going to have random fluctuation making it look like some players are stealing too much or too little (for example, a player might have a theoretical success rate of 80%, but by chance, he got thrown out 30% of the time), you still want to have a team wherein a high percentage of players are above the BE point.  Also the success rate for some players who don’t have a lot of attempts should not be too much higher than the BE point, otherwise he might not be stealing enough (against the tougher pitchers and catchers), although I think that is rarely going to be the case.  You can also use the binomial theorem to figure out how many players you expect to be below the BE point even if their true success rates were above the BE point, and incorporate that into your analysis.

On of the projects I would love to do sometime down the road (and hopefully someone will beat me to it) is to take every player’s estimated true SB success rate (by using their sample one and regressing it to some mean, using their speed score to establish the mean) and then to estimate what it would be against every pitcher/catcher combination, using the catcher and the pitcher’s estimated success rates (probably with a “WOWY” methodology, but ideally with a “time to home plate” for pitchers and I am not sure for catchers), and in every game situation.  Then go through the PBP data and see how often they attempt a steal when they should and when they should not and see how many runs each baserunner gets added or docked as compared to a perfect strategy (for them) and as compared to not running at all.

Of course one of the problems that would have to be handled is that even if a certain baserunner “should” be attempting a steal, given the game situation and the pitcher/catcher combo, he cannot do so every time for several reasons:  One, game theory - the defense would pitch out a lot against him if he was going too frequently in any given situation.  Two, he may have been “planning” to go but the batter put the ball in play or walked or whatever.  Three, he may have been planning on going but could not get a good jump or just did not feel comfortable for whatever reason.  So, even if a situation calls for a steal, given the runner, pitcher, catcher, and game situation, you would have to assume something like a maximum percentage of 25-40% I would think, or something like that.  You could probably just look at good, prolific base stealers in obvious steal situations (against pitchers and catchers that are reasonably easy to steal against) and see around what their maximum percentage is.

Dan has an article on the breakeven points for SB using empirical deltaWE, relative to the run environment:
http://www.beyondtheboxscore.com/2009/6/22/920350/graph-of-the-day-breaking-even-on

The shorthand I’ve been now using is: RPG/2 as the ratio of SB/CS.

For example, if you have 5 RPG, then that means the breakeven points is 2.5 SB per CS, or a rate of .714.  If you have 4 RPG, then it’s .667.  If it was 3, then it’s .600.

Eyeballing Dan’s chart, it’s seems to hold kinda well.  Maybe instead of RPG*.5, it should be RPG*.48 or something…

Thanks Dan, nice chart

I posted this in a comment to my article at BP

“Two main things should be considered when deciding to steal or not - how important is the run to winning the game, and what are the chances of being successful? I want to add win expectancy table a to my steals database. As Matt Swartz showed in his article this week, WE gives a weight on each event based on how important it is to winning the game, looking at inning, outs, bases occupied and score. Then look at the ability of each pitcher and catcher to prevent a successful steal. The combination of the two would give the expected win probability added with an attempt in that particular situation.

It would be a safe assumption that most steal attempts would be in the situations with the highest eWE (and this can be tested, looking for the average Att% at different levels across time). If the Mariners decide to run Ichiro at a 50% rate instead of 15%, we could look at the 50% of the times he was on 1st that offered the best eWE, and then sum the expected steals, caught stealing and pickoffs given those runner, pitcher and catcher matchups.”

I would like to,
1. as Dan has done, assign a WE for each SB opportunity. Group by these to see how the overall rates vary.
2. Use WOWY, as I have done for catchers, to normalize pitcher’s effect on CS%, then for each opportunity calculate the expected CS% and PO% considering the runner, pitcher and catcher.
3. for each runner, then be able to rank his opportunities by expected win value, hopefully being able to identify that runner’s optimal attempt rate, or at least find missed opportunites and attempts that shouldn’t have been

Off the top of my head I tend towards agreeing with Dan that attempts today are probably more intelligent than they were in 1976-1992.

For those who have The Book, I’d like to remind you of Table 137.

For those who don’t have it, you can
1. go to Amazon (click the link/image at the top left of any page),
2. click Look Inside,
3. Type: Table 137
4. Navigate the search hits until you get to Page 336

Hmmm… I just tried that, but the page didn’t come back for me.  Maybe that page is blocked.

Based on a suggestion from David Pinto, I added actual SB success rate to the graph

Teams were well below the breakeven point until the mid 1960s, and have been well above it since the strike.

This suggests they probably could run more (ignoring things like injury concerns, game theory, etc.)

Peter, did this SB article you mentioned last year ever get on the web?  I checked under your name on THT but didn’t see it.

No, it didn’t.  Because of comments in this thread I tried to see if I could separate actual SB attempts from Hit and Run attempts.  I couldn’t.  I then put it aside for the research I was doing on Hit f/x.  I still have the SB research.  It is still interesting, but very complicated since it involves estimating what the run value would have been if the runner had not been running on the pitch.

I hope you decide to publish it anyway.  No matter what you do, there will always be someone to suggest something else to control for, etc.

But in a world where a whole website is set up for base/out percentage, I think your research would be well worth the time to read.