THE BOOK--Playing The Percentages In Baseball

"All you have to do is merge the two (weighted of course by the number of pitches in each pool).”

Right, but the study doesn’t give the number of pitches in each pool.  And, actually, that’s not enough—you also need the number of pitches in each pool broken down by fastball and non-fastball.

Number of pitches in each pool is easy enough.

And number of pitches in each pool broken down by FB, non-FB follows.

For example, the at-bat ends on a first-pitch at bat around 12% of the time.  This is easy enough to calculate:
http://www.baseball-reference.com/leagues/split.cgi?t=b&year=2008&lg=MLB#count

There were 22021 PA that ended on 0-0, and 187614 in all.  That’s 11.7%.

Just go through it all 12 times, and you’ll have your numbers.

I think I recall Levitt having a rather dim view of sabermetrics, so the odds of him engaging with Tango are sadly rather slim.

Tango/2: for those 187614 pitches, how do you find out how many were fastballs?  You can’t just assume it’s the same proportion as for the other 22021 pitches ...

Phil, I see your dilemna. 

I am sure that one of the PITCHf/x-ers can give you a hand.  Here’s the official request:

1. For each count, split up the plate appearance into whether there was a fastball thrown or not.  This will give us 24 slices of data.
2. Calculate Linear Weights by the 24 slices, based on how the plate appearance ended (regardless of the plate appearance ended on that pitch)

So, if a plate appearance went 4 pitches as 0-0 fastball, 1-0 fastball, 1-1 curve, 2-1 fastball double

You will count as a double for each of the 4 pitches:
0-0 FB - 2B
1-0 FB - 2B
1-1 CV - 2B
2-1 FB - 2B

What game theory would suggest is that you’ve got the correct proportion of fastballs thrown at each count, if the wOBA (or EqA or Linear Weights) at each count, for each pitch type, is the same.

If, for example, on a 2-1 pitch, you throw 30% fastballs, and 70% not, and the wOBA for all those PA (whether the PA ended on 2-1 or not) ends up as .330 fastball and .360 non-fast, then you threw not enough 2-1 fastballs.  If you get to the same wOBA, then you know you’ve got the correct mix.

In the Levitt data, he shows, for example, how on 1-0 counts that the OPS, on the pitch that ended that PA, was .877 on fastballs and .861 on non-fastballs, making you think that maybe you shouldn’t throw too many fastballs.  But, on the pitches that did NOT end the 1-0 count, a fastball thrown at 1-0 led to an eventual PA where the OPS was .733 and the non-fast was .744.  So, overall, the fastball and non-fast had the same impact.  Equilibrium.

His interesting results all are with the PA that ended on a 2-strike count.

Agreed, that’s how Kovash and Levitt should have done it.  In fact, all they have to do is change their programming to use the “through” count instead of the “at” count on all pitches—that is, consider *all* pitches at 0-0, not just the ones that ended the AB.  It could, conceivably, be a 30-second change to their programming.

Well, that, and the fact that they seem to have got some of their numbers wrong.  I wouldn’t put any stock in any of their results, even the interesting ones, until they fix the numbers.

BTW, it just occurred to me that their OPS numbers are smaller than the “real” ones because they omitted IBBs.  Fair enough, but that doesn’t explain the 3-2 discrepancy, where they have an OPS in the mid 7s, where the actual number is in the 8s.

I’m reading about his research assistant, and the paper seems to at least have been inspired by him, and he is co-author.  He seems to be into sabermetrics as well.  So, I’ll e-mail him directly.  He sounds like the kind of guy who fits in here, like Shane Jensen before him.

Tango/5:  What’s the argument for using the end result of the PA to establish run values, as opposed to Joe Sheehan’s method of using the change in RE for each pitch?  (I’m not disagreeing, just wondering why you’d approach it that way.)

I’m not sure that the methods are necessarily different.

This is the way I do it:
http://www.insidethebook.com/ee/index.php/site/comments/hitting_by_count/

The “through count” at 3-0 is a wOBA of .570, compared to an average (at 0-0) of .332.  That makes the run value potential at 3-0 as .570 minus .332 divided by 1.15, or +.21 runs (more or less).  If you get a called ball (i.e., walk), that pitch is worth +.11 runs.  If you get a strike (putting you to 3-1) that makes that strike worth -.07 runs.  If the batter gets a HR, that pitch is worth 1.2 runs, etc, etc.

What does Joe propose?

Ken also works at Mozilla (FireFox).

Here’s some of his data analysis work:
http://blog.mozilla.com/metrics/2007/11/26/mozilla-online-advertising-–-an-experiment/

http://blog.mozilla.com/metrics/2007/12/20/mozilla-online-advertising-part-2-experiment-findings-and-marginal-cost/

Tango: Joe S’s article is here: http://baseballanalysts.com/archives/2008/02/writing_about_t.php. Seems different than the method you suggest in #5, based on outcome of that PA.  The 3rd pitch CB in your example would be considered a good pitch by Joe’s method (evening the count at 1-1), but not if evaluated solely by the outcome (double).

Oic.

Either way works.  Let’s forget about the count, and only think about the 24 base/out states.

Basically, you can look at a HR hit with 1 out and bases empty, and call that “1.000” runs using the delta RE method (the run expectancy for future batters stays the same, plus you get the run that actually scored).  Or, you can look at all runs that scores from that play to the end of the inning, and subtract from that the number of runs expected to be scored with 1 out and bases empty.

So, yes, the preferred method is usually the delta method, which presumes that all future events will be random, and not based on the current event.  However, there’s reason to believe that things are not so random, and in this study specifically, they are asking about future events following a fastball thrown at a particular count.

In any case, as I said, either method would work.  You might as well do it both ways. 

In The Book, in Chapter 1, I actually do it both ways, and you can see there’s little difference between the Markov delta method and the empirical results.

Thanks, Tango.

It occurs to me that we might still see some disparity in run values using “through” counts, based on differences among pitchers and/or hitters, even if all pitchers individually used an ideal mix of pitch types.  Let’s take a simplified model of “good” and “bad” pitchers, in which the good pitchers are much more successful on non-FBs, but the gap is smaller on FBs.  Then both groups of pitchers might reach equilbrium on a different FB/NFB distribution:
Good: 
FB 50% -.1
NFB 50%, -.1
Bad:
FB 70% +.1
NFB 30% +.1
The average value of FBs in this scenario will be much lower, because they are thrown more often by weaker pitchers.  The same results would obtain if weak hitters had a harder time with breaking balls, and so a disproportionate number of NFBs went to weak hitters.

In real life these distributions wouldn’t be hugely different.  But they might be different enough, especially in high-leverage counts, that a certain pitch type would appear to be better or worse overall, even though each individual pitcher had reached equilibrium.

Levitt wrote this joke of a blog post over four years ago, and it soured me on him:

http://freakonomics.blogs.nytimes.com/2005/04/06/lets-at-least-argue-about-moneyball-using-data/

“What I am arguing is that they were not successful for the reasons that were most prominently trotted out in Moneyball, namely the ability to find good offensive players cheap.”

Not only is he a disaster analytically, but he childishly brags about his knowledge:

“And for all the talk about me not knowing anything about sabermetrics I actually do know quite a bit”

So I wrote him off - his analysis is bad, and then he tells us “I do so know what I’m doing!”

Ken wrote back, and he’s ok with the “public option”, but he has to get permission.  I’ll keep you posted.

Guy,

Right, and let’s not forget that “fastball” for Moyer and Felix is not really the same thing.  A Felix changeup is faster than a Moyer fastball.

Ideally, you create a chart as I’m discussing BY PITCHER, and then average it out so that all pitchers are equally represented at each count.

Wow, that is a bad blog post entry from Leavitt.  Not even written well.

What I am arguing is that they were not successful for the reasons that were most prominently trotted out in Moneyball, namely the ability to find good offensive players cheap.

Cheap is a noun, Steven, and cheaply is a verb.  7th grade English I think.

He falls into the “amateurish” trap of thinking that because a team (or player) has better stats at home in a pitchers park, that the stats don’t have to be adjusted or somehow that means that the park is not really a pitchers park.  (Not to mention the fact that in a neutral park, a team will hit much better at home than on the road, of course.)

And for all these comments about how I’m an idiot for not controlling for Oakland’s tough-to-hit-in home stadium, Oakland had an OBP of .350 at home in 2004 and .336 when on the road. They also scored more runs at home than away. The Oakland ERA was not much different home or away either (4.06 vs. 4.29).

I think I figured out why the paper’s OPS data is off.  They are effectively using PA, rather than AB, as the denominator for SLG.  Here’s how they describe OPS in the paper:

“If a batter makes an out, his OPS for that at bat is zero. If the batter walks, his OPS is one. A single earns an OPS of two, a double an OPS of
three, a triple an OPS of four, and a home run an OPS of five.”

All true, but if you use these values as your numerator and PA as your denominator, you won’t get OPS.  If this is what they did, in counts with a high walk rate the authors’ OPS should be much lower than the actual OPS—and that’s the pattern we see.  And if you create “LevittOPS” from the B-Ref splits, you get results pretty similar to the paper’s.

The key assumption the authors are making on comparing FB to NFB (non fastballs) is that if the run value on FBs is higher when the pitch ends the AB, AND when it doesn’t end the AB (as is true for quite a few counts), then the overall run value of the FB must be higher.  But that isn’t necessarily true.  For example, at 3-1 the “OPS” for FB is 1.005 on the pitch and .725 after, and for NFB the respective “OPS” are .949 and .703.  85% of the pitches are FB.  But, if 80% of the AB-ending pitches are FB while 90% of the continuing ABs are FB, then non-fastballs will have a higher average run value.

So we have to know the distribution of FB and NFB separately for ending and continuing ABs in each count to know if there really is a disparity.  As we often see, grouping by the outcome can get you into trouble.

Guy, great points!

Basically, they tried to do a wOBA-type of equation, but did it wrong.  wOBA, if we remember, gives the following weights for the Big5:
.72, .90, 1.24, 1.56, 1.95

They used:
1,2,3,4,5

If we multiply all their numbers by .45, we get:
.45, .90, 1.35, 1.80, 2.25

As you can see, they SEVERELY undervalue walks, while overvaluing extrabase hits, and especially HR.

And just to close the loop, I would think that undervaluing walks would make non-FBs look better than FBs, just as the study finds.  Non-FBs presumably result in more BBs but lower SLG than FBs. 

So to really answer this question, the authors have a lot of work to do:  1) replace their modified OPS with wOBA or another metric that properly values offensive outcomes, 2) combine the “at” and “after” data into a total through-count measure for each count, and 3) control for possible hitter and pitcher quality effects (including handedness).

Joe Sheehan’s article (link in post #11) provides run values for different pitches, controlling for count, by handedness of batter and pitcher.  His data does suggest a slightly above-average run value for FBs overall.  So there may be something to this idea.  It does seem surprising that the FB frequency is virtually identical in all 4 matchups, even as other pitch frequencies change dramatically.  Still, quality of hitter could explain this:  I imagine weak hitters see more non-FBs (because they hit in more pitcher counts, and perhaps because many weak hitters can’t handle breaking balls).  For example, my impression is that pitchers-as-hitters see almost exclusively breaking balls unless the pitcher falls behind in the count.  (Note:  another problem is that Joe’s data is a limited subset of parks/players with pitch/fx data.)

Like MGL, I find the idea that pitchers fail to randomly mix their pitch selection much more plausible than the too-many-FBs theory.  It would be interesting to see if good pitchers, controlling for other talents, tend to do a better job of randomizing pitch selection.  Was Maddux especially good at this?  Seems like an interesting issue for study.

I could quit my full-time job, and do nothing but analyze the ball-strike count data for 12 hours a day, and after 3 years, I’d still only be half-way done.

This is the gold, the thing that makes baseball go.

I think you’re absolutely right, tango.  I’ve read a lot lately, notably in that great post by Disco Hayes, about how a “real” baseball fan enjoys watching a game more because of knowledge about the battle between the hitter and the pitcher.  This is opposed to the “casual” fan, or non-fan, that finds baseball boring because they only see the results of at-bats.

However, even we (and I hesitate to include myself) knowledgeable fans know so little about what is going on during that battle.  We know what happens on average, sure, but why did Mauer flail at Greinke’s slider (I think) low and in two at bats in a row this weekend when he normally lays off that pitch?  And why did Greinke then give him a meatball to hit the other way in his next at bat?  Did Mauer just not see the pitch, or was it set up with pitch sequence?  Was Greinke trying to sneak one past him in the next at bat, and would he have been better off randomly selecting a pitch there?

We’re just scratching the surface (and given that, there’s really no excuse to go about it the wrong way, as Levitt is here).

Over at Phil’s blog, Rodney Fort raised the question of whether the BIS pitch coding could be biased in some way, such an breaking balls that fail to break being coded as FBs.  Has there been any good evaluations of the BIS pitch classifications?  Any reason to suspect bias?

Levitt and Kovach note that they compared BIS to STATS classifications on a subset of the data, and found 94% agreement on FB vs. non-FB distinction (less agreement on what kind of non-FB a pitch is, as you’d expect).  So any bias would have to be true of both firms’ scorers.

Another factor that is missed in that article is the importance of pitch counts. As starters are generally better than relievers, it is optimal to have the starter in to pitch to more hitters. As fastballs are more likely to be put in play, reducing the need for further pitches, fastballs are more valuable in the context of the game, rather than the PA. Ignoring the game situation may seriously affect the results.

One way to try and get at this would be to contrast starters, and especially starters in the first 3 innings, with relievers. As relievers generally don’t worry about pitch counts, I would expect that they would be closer to the ideal mix that Levitt is looking for.

I would say it would be improbable to mix a fastball with any other kind of pitch. 

Let’s take Felix Hernandez, he of the occasional 91mph changeup (!).

In his career, his average fastball speed is 7 to 10 mph faster than his average slider and 5-10 mph faster than his average changeup.

That’s on average.  Let’s say that he’s at 94mph for his fastball, 89 for his changeup and 87 for his slider.  In a game, his fastball will range 91-97, and his changeup would range 87-91, and his slider would be 85-89. 

As you can see, we are really talking about the fringes here of a ball that one might call a fastball, that another would call a changeup.  And that’s based strictly looking at the speed of the pitch.  Once you add movement, it should be a lock.

Unless Felix starts throwing his fastball intentionally with less speed and more movement (and thereby creating even more overlap), I would suspect that you can get at least 95% of his fastballs labelled as fastball strictly by using pitch speed as the sole determinant.

So, I agree, bias would not be found in the FB v non-FB.  If you want to say that you can find bias among the non-FB pitches, sure, I agree.

"I imagine weak hitters see more non-FBs (because they hit in more pitcher counts, and perhaps because many weak hitters can’t handle breaking balls).  For example, my impression is that pitchers-as-hitters see almost exclusively breaking balls unless the pitcher falls behind in the count.  (Note:  another problem is that Joe’s data is a limited subset of parks/players with pitch/fx data.)”

I think you’ll find just the opposite, Guy, simply because pitchers don’t want to walk the weak hitters.

MGL:  you’re right, slightly.  But mostly, we’re both wrong.  According to Fangraphs data, the correlation btwn wOBA and fastball % is -.14 this year.  Really, not much relationship.  Albert Pujols gets very few FBs (53%), but Ty Wigginton has the same rate.  Joe Mauer and Randy Winn both have a 63% FB rate.  It seems to depend more on the hitter’s individual strengths/weaknesses than overall talent.

One of the surprising findings in the paper, to me, was that FBs (36.4%) and non-FBs (38.1%) have almost the same rate of being called balls.  Yet pitchers clearly have far more control over FBs, or else they wouldn’t rely on them so heavily in hitters’ counts.  Assuming the data is correct, batters must swing at so many out-of-zone breaking balls in hitter’s counts that nearly as many strikes get recorded on non-FBs.

Pitchers have more control over FB, so they try to spot them in the corners.

They have little control over curve balls, so they aim closer to the center of the plate, and “hope” they don’t let it hang there.

I am sure that one of the PITCHf/x-ers can give you a hand.  Here’s the official request:

1. For each count, split up the plate appearance into whether there was a fastball thrown or not.  This will give us 24 slices of data.
2. Calculate Linear Weights by the 24 slices, based on how the plate appearance ended (regardless of the plate appearance ended on that pitch)

I have run these numbers, and I’ll try to make a post on THT Live later tonight with the full table.  However, the basic conclusion that I find is that pitchers should throw more fastballs at the 0-strike and 1-strike counts and more off-speed pitches at the 2-strike counts.

"Assuming the data is correct, batters must swing at so many out-of-zone breaking balls in hitter’s counts that nearly as many strikes get recorded on non-FBs.”

That is exactly correct.

“But mostly, we’re both wrong.”

Either they throw more fastballs to weaker hitters or fewer. Has to be one or the other (or else it is exactly the same, which I doubt).  We cannot both be correct or both be wrong.  This isn’t an issue of correlation.  Simply split batters into 2 categories, with equal representation (by PA or number of batters) in each category. Below a certain wOBA and above a certain wOBA.  Then look at percentage of fastballs in each category.  That at least answers the general question.  Obviously on an individual basis, it depends on the batter.  Generally, it has to do with power.  If a batter has power, you tend to throw him fewer fastballs.  It also tends to be about batting eye in conjunction with power.  The better the eye, the more fastballs.

Mike:
Very interesting data.  How do you handle 2-strike fouls?

Perhaps hitter quality accounts for some of the pattern?  It could be that on hitters’ counts, pitchers throw relatively more non-FBs to good hitters (on theory they’re willing to walk them if necessary, rather than let them tee off on a 2-0 or 3-1 FB).  Conversely, in pitcher counts they may throw more non-FBs to weak hitters, as these hitters are more likely to swing at a pitch outside the zone and giving them another ball is less costly.  Just a theory, of course.

I’m not finding the time to a writeup tonight, so I’ll just do a data dump instead.

I left out 9405 pitches that either had unknown or erroneous pitch type or count, and they had a total linear weights of +1411 runs.  Some of that is just bad data from Gameday.

Two-strike fouls were treated just like other pitches, e.g., if a fastball was fouled off at 3-2 and the next pitch was off-speed and hit for a double, I accounted for that as one fastball at 3-2 for +0.8 runs and one off-speed at 3-2 for +0.8 runs.

I think it might be better to do the run value accounting one pitch at a time, taking into account the average value of adding a ball or strike to the count, but Tango’s suggestion of applying the final outcome of the plate appearance to each pitch was easier to implement quickly.

I did not make any adjustments for or measurements of hitter quality.

I forgot to mention that the 9405 pitches that I removed included intentional balls and pitchouts, which are obviously going to have a positive linear weights value.

Also, my total runs/pitch for the events in my database (2007-27sep2009) is +0.0015.  I’m off a little bit somewhere, but I’m not going to bother investigating where until the season is over.  That’s a small enough error I think it’s fairly safe to neglect for now.

I realize that Mike did not have the time to adjust for batter and pitcher quality, but without doing that, I think that the numbers are meaningless as far as whether pitchers are optimizing their pitch selection.

As an extreme example, let’s suppose that we have only two batters in the data sample, a pitcher and Pujols.  At all 3-2 counts, the pitcher is going to get mostly fastballs and Pujols will get, say, half FS and half OS.  So that data will say that on a 3-2 count, the OPS on a fastball is around 2/3 .500 (or whatever it is for a pitcher) and 1/3 1.000 (or whatever it is for Pujols) or .667.  Only Pujols gets the OS, so the OPS on an OS is going to be 1.000 (or whatever).

Mike:  great data.  Thanks for sharing it.

I agree that we need to control for pitcher and hitter to reach firm conclusions.  But taken in the aggregate, Mike’s data seem to indicate that fastballs are likely NOT thrown too often by pitchers (the weighted delta shows that fastballs are a tiny bit more effective overall).  It does appear that fastballs are thrown too frequently in some counts and too infrequently in others.  But given the overall parity it does seem likely that those differences will largely disappear once we control for hitter and pitcher quality.  It would be surprising if pitchers over-utilized the FB in some counts, but then under-utilized it in other counts in almost perfectly offsetting amounts. 

*

MGL/31:  I looked at all hitters with more than 50 PAs in 2009 (including pitchers), and ranked them by wOBA.  I then calculated a straight average of fastball % and wOBA for each quintile (not weighted by PA, so this is crude).  This is what I get:
FB% / wOBA
0.58 / .380
0.60 / .341
0.58 / .316
0.59 / .289
0.66 / .210
It looks to me like the very weakest hitters (pitchers and a handful of position players) do get more FBs, as you expected.  But other than that, the FB% seems pretty flat.

I can’t help but wonder here:

If you replaced Fastball/Offspeed with strike/ball, would you come to the conclusion that pitchers are just throwing too many strikes?

Guy 13 and Tango 16

The authors do use pitcher and hitter fixed effects and in some specifications pitcher and hitter interactions.  So for those cases it wouldn’t be the case that individuals could mix optimally but the group as a whole would appear suboptimal, right?

TJ:  there are two big problems (at least) with the regressions:  1) as discussed above, their version of OPS significantly distorts the real value of offensive outcomes (and in a way that likely overstates the value of non-fastballs), and 2) they fail to combine the outcomes “at” and “after” the pitch, which you have to do to properly assess the impact of the pitch.  For example, as I showed above it could be the case that a fastball appears to be worse both when the AB ends and when it doesn’t, and yet still be the better pitch overall (because it ends the AB more or less often).

But if you fixed those problems, and then controlled for pitcher and hitter quality (and their interaction), then I would agree you shouldn’t get divergent results. 

BTW, strictly speaking an optimal mix would mean that the marginal value, not average value, of each pitch type should be the same (for a given count, pitcher/batter mix).  But in this case, I think the two are effectively the same thing.

Mike/33: fantastic data!

From that data, we see that there are way too many fastballs thrown at 2-strikes.  What you can do to control for the quality is simply calculate the wOBA of each batter/pitcher, and report back the average wOBA at each count, for each FB/OS pitch, exactly as I did in The Book, when controlling for the quality of players in the batting order chapter.

Boom, done.

I agree on those issues I was speaking specifically to that earlier post and controlling for pitcher and hitter effects.

Another thing that occurs to me is that their data span 5 seasons and their fixed effects do not appear to be on a per season basis.  So it would just be Barry Bonds for a hitter fixed effect, not separate effects for Barry Bonds-2002, Barry Bonds-2003 etc.  This to me is making a rather dubious assumption that the effects are actually “fixed” for this time period.

In most of the hitters’ counts, more offspeed pitches are thrown to better quality hitters.

You may notice that the sample sizes are slightly different than those in Post 33.  There are a few hitters for whom I don’t have a wOBA, so I dropped their data from this sample.

I assume we could probably take this data, convert wOBA to Lwts, and apply it to the table in Post 33.

wOBA minus average, divided by 1.2, equals LWTS.  In this case, average is .330.

Equivalently, you can do wOBA/1.2 - .330/1.2, or wOBA/1.2 - .275

You will note that the “.275” is not an accident, but the run value of an out.

wOBA is LWTS.

On the 3-2 count, we see that above-average hitters are in both pools, the FB and off-speed pools.  The .3338 hitters means +.003 runs per PA, and the .3422 hitters means +.010 runs per PA.  So, we’ve got +.007 runs per PA better hitters seeing off-speed pitches.

Unreported in Mike’s data (yet) is the pools of pitchers.

Just eyeballing the two tables, it looks like hitter quality alone explains a good portion of the apparent inefficiencies.  An exception is the full count, where the FB is much less effective despite being thrown to a weaker pool of hitters.  Maybe good pitchers are much more likely to have the skill/guts to throw 3-2 breaking balls?  If not, we may have found a place where pitchers could benefit from changing the mix.  (Although I also worry that counting the 2-strike fouls might distort things here.)

Tango:  Any reply from Ken Kovash?

Ken had replied right away that he was going to get a hold of Levitt, but that Levitt is not the easiest guy to reach.

Nonetheless, Mike Fast’s data is what we needed.  The best they can do is follow our lead, and show things in terms of Linear Weights.

More exposure on the study:

http://www.newscientist.com/article/dn17901-sports-jocks-are-ohso-predictable.html?full=true&print=true

Glove-slap: Col

I may have mentioned this before, but if each batter in general was a little stronger against the FB or NFB, we would expect to see a fastball followed more frequently by a fastball and an off-speed pitch followed more frequently by an off-speed pitch.  Did the authors control for that? 

IOW, if they found that fastballs were followed by off-speed 4% too often, that might represent a mistake of 6% or something like that, right?

Let’s take an extreme example.  Let’s say that half of all hitters are great fastballs hitters and half are great off-speed hitters.  And let’s say that to the great fastball hitters you throw 90% off-speed and to the other hitters, you throw 90% fastballs.

If pitchers were perfectly randomizing their pitches, you would see fastballs after a fastball 82% of the time and off-speed after an off-speed 82% of the time, even though each pitch was thrown 50% of the time.

So basically what I am saying is that if fastballs are thrown 60% of the time overall, you would expect, if pitchers were perfectly randomizing their pitches, to see more than 60% fastballs after a fastball and more than 40% off-speed after an off-speed, since clearly pitchers throw more fastballs to some batters and more off-speed to other batters.  IOW, the “mistake” that the authors found is even more of a mistake than it appears, if they didn’t control for the fact that different batters have different percentages of fast/non-fast thrown to them.

Of course, as someone already mentioned, it may be correct to change your pitch more often than “randomness” would dictate because of the familiarity factor for the batter.  Or if the batter expects the same pitch too often, then it is also correct for me to throw that same pitch less frequently.  So the authors cannot call that a mistake by the pitchers without knowing how the batters are acting (thinking).

mgl/48: absolutely, good point.  Should have thought of that myself.

The study did control for batter/pitcher, though, in at least one of the regressions.

Pitchers’ wOBA by count and pitch type:

Let me ask a basic question about an assumption we’re making:  why does equilbrium necessarily require equal wOBA for both pitches (holding count and batter/pitcher constant)?  I know the theory:  if there’s a disparity, the pitcher could then throw that pitch more frequently—reducing the effectiveness of that pitch and increasing the effectiveness of the other—until the two wOBAs are equal.  But why would that necessarily be a good strategy for the pitcher? 

Let’s say he throws 60% FB, 40% NFB, and wOBA is FB .320 and NFB .340.  And let’s say for each additional 1% he devotes to FBs, his FB wOBA climbs .001 and his NFB wOBA declines .001. The wOBAs will be equal at .330 when he throws 70% FB.

But, here’s the problem:  the pitcher is now worse off.  His wOBA has increased from .328 to .330.  Why should/would he do that? 

Equilibrium requires that a change in distribution not benefit either party.  But I’m not sure that means each strategy has to yield the same average return.  That should depend on the steepness of the tradeoff curve for each strategy at a given frequency.

Think about Kobe and the Lakers.  He delivers more points per possession than the rest of the team.  But does that mean the Lakers must give Kobe more shots?  I don’t think so.  There is evidence that NBA shooters become less efficient as they shoot more.  And almost certainly the dropoff becomes larger as usage grows (at 100%, a player would obvioulsy be extremely ineffective). So giving Kobe 31 shots/game instead of 30 likely reduces his eFG%.  And not just on the 31st shot, but on all 31 shots.  That will become a bad tradeoff for the Lakers long before Kobe’s eFG% reaches the rest of the team. 

Thoughts?  I’m sure the game theory experts have worked this through. What am I missing?

Guy, I agree with you, and it’s the same argument I put forth on the Mariano thread.  That even though his outside-outside and outside-inside performances are severely skewed, that perhaps it needs to be that for him to be that effective.

We reach equilibrium when no matter what the pitcher does, his wOBA can’t go any lower, and no matter what the batter does, his wOBA can’t go any higher.

Of course, all this is tough, considering how players are humans, and can change their approach on a whim.

Guy, if you just throw some numbers out there, you can always make it not work, but the basic tenets of Game Theory dictate that both the pitcher and batter play (if the game is in equilibrium) in such a way that it doesn’t matter what the other one does - they cannot get an advantage.  That means that the results for the FB and NFB have to be the same. Otherwise the batter would look more for the pitch that gives the pitcher the better result.  And if that pitch still gives the pitcher the better result even though the batter is looking for it more, the pitcher would throw it more. So your example with your numbers could not happen unless one or the other of the pitcher and batter is not playing optimally. The only flaw in the authors’ assumptions is that they would expect to find the same result on FB and NFB pitches if the pitchers were acting optimally only if the batters were acting optimally. So if he finds that the results are not equivalent overall or at any one count, then either the pitchers are not acting optimally or they are and the batters are the ones making the mistake (or both).

But, as I said in my last post, one of the big flaws is that this is not one game.  This is a series of mini games between individual pitchers and individual batters and you cannot treat it is one big game.  And each of those mini games has their own equilibrium point such that it is not necessarily true that overall the FB and NFB results will be the same.

Say there are two classes of batters equally represented, similar to my other example. One class gets 90% FB and 10% NFB and their wOBA on both is .300. The other class gets 90% NFB and 10% FB and their wOBA is .400.  The overall results for FB and NFB are not even close even though each mini-game is in equilibrium.

Guy/51: I think mgl is right ... in equilibrium, the numbers just can’t work out that way.  I’ll try to think of a way to explain it, once I figure out how to explain it properly to myself. 

One thing to keep in mind is that the ine quality this disproves only that BOTH the hitter and pitcher are in equilibrium.  It’s possible that the hitter has the wrong strategy, and pitchers know this and are taking advantage, even though the pitchers are fully aware of what the equilibrium strategy might be.

For instance, in rock/paper/scissors, if I see you never pick Rock, I just pick scissors every time and never lose a round.  But once you switch to equilibrium, then we start winning equally.

So it’s very hard to tell whether it’s the pitcher’s “fault” that it’s not in equilibrium, the batter’s “fault,” or both—or, if it’s both, what proportion of the “fault” belongs to each.

”...the basic tenets of Game Theory dictate that both the pitcher and batter play (if the game is in equilibrium) in such a way that it doesn’t matter what the other one does - they cannot get an advantage.  That means that the results for the FB and NFB have to be the same.”

I agree with the first statement.  But I’m not sure the second follows.  I readily admit I may just be missing something, but your answer seems tautological to me—it can’t happen because game theory says it can’t happen. 

Which of my assumptions is impossible?

Assumption #1:  A pitcher throws 60% FB/40% NFB, and wOBA is FB .320 and NFB .340 (controlling for count and hitters).  Are you saying this is impossible because hitters will “look” fastball until they raise their wOBA?  But what if this is how well hitters perform when they know p=.6 for FBs?  Or what if they had to “look” FB so much to bring their FB wOBA up to .330, that their NFB wOBA then fell to .290?  Then hitters wouldn’t do that.  I don’t see why this is impossible.

Assumption #2 For each additional 1% he devotes to FBs, his FB wOBA climbs .001 and his NFB wOBA declines .001.  Is this implausible?  Certainly there is some tradeoff between frequency and success.  Otherwise, the correct strategy is either 100% FB or 100% NFB for any given count. 

Surely in the NBA, we see that the Kobe/non-Kobe, or LeBron/non-LeBron options have very different payoffs for the Lakers and Cavs.  Is anyone prepared to argue that this represented some massive strategic failure by the teams, and that these players “should” be taking 50% or 60% or 70% of their team’s shots?  That seems totally implausible.  I think it must be true that at some level of usage, maybe around 30%, the penalty a star player suffers for taking more shots becomes greater.  And because that penaly applies to all his shots, not just his “extra” shots, it makes sense for him to stop shooting long before his success rate equals that of his mates.

Perhaps the difference here is that in game theory constructs, no particular strategy in INHERENTLY better than any other.  And any possible strategy can be blunted by an option available to the other side? 

But in sports that isn’t true.  If I have LeBron and give him the ball 20% of the time, he’s going to score a LOT more points than on my other 80% of possessions—even if I give the other team a sworn promise to use LeBron exactly 20% of the time.  The other team simply can’t stop this, or rather, the steps they would have to take to make LeBron an average player would allow the other players on the team to become well above-average. 

Similarly, a pitcher’s FB might simply be a better pitch than his NFB, even if hitters know how often to expect it. Equilibrium then depends on the usage/effectiveness tradeoff for each option.  (I think.)

Guy/55: Try a 2x2 matrix where the pitcher has only two strategies (say, FB and CB), and the batter has only two strategies (say, expect FB and expect CB).  Plug in whatever wOBA numbers you want for each of the four cells.  Figure out what each player’s optimal strategy should be (which will be a certain percentage of strategy 1, and the rest strategy 2).  (I’m not sure how to do this, but I’m sure we can find a method for solving a 2x2 game on the web.)

I guarantee you that, if each player chooses optimally based on the other’s optimal choice, the expected wOBA will be the same regardless of whether an FB is thrown or a CB is thrown.

That doesn’t answer your question, but it will make it easier to address if you do it in 2x2 format.

Guy/56: giving LeBron the ball 20% of the time is NOT the optimal strategy, which is why the two strategies don’t seem equal.

Giving LeBron the ball 80% of the time might be optimal.  And for the defense, the strategy might be to triple-cover LeBron, and 1/2 cover the other four guys.  That might equalize the chance of scoring no matter who has the ball. 

You have to think: intuitively, what strategy makes the chances equal regardless of which one you choose?  LeBron at 80% might be that strategy.  You can’t arbitrarily say 20%.

It’s also important to note the difference between a “strategy” (throw 67% fastballs) and a specific “iteration selection” (the die came up 1 or 2, so I’m going to throw a curveball).

Only if you choose the right *strategy* (67% FB) and the opponent chooses the right strategy (look for fastball Y% of the time) will the outcomes of the two different *iteration selections* (fastball, curveball for the defense, “look fastball,” “look curveball” for the offense) come out the same.

P.S.  does anyone know the actual game theory term for what I call “iteration selection”?

The Kovash paper has a “part 2” that deals with football (pass or run).  Brian Burke gives us his thoughts:

http://www.advancednflstats.com/2009/10/full-review-of-game-theory-run-pass.html

I especially agree with him about the “regression” using the time and score.  You can’t just throw it in there, thinking it will capture what you think it’s going to capture.  The impact of the time and score acts differently at different points of the game.  For example, I would not simply throw in inning and score in baseball in a regression equation for Leverage Index or WE.  It won’t work at all.

His idea around this, to focus on a subset of plays, is a good one.  Going back to baseball, I could create an LI function, if I focused strictly on the bottom of the 7th, down by 1.  Then, my function would only use base/outs, and I could get you a good LI equation.

There’s regression, and then there’s regression.

http://www.advancednflstats.com/2009/10/full-review-of-game-theory-run-pass.html

Phil/MGL:
I took Phil’s advice and tried a 2x2 matrix, and I can see that there will indeed be an equibrium where neither side can gain by changing strategies.  For fastball vs. non-fastball, this model seems like a good approximation of reality.  And my guess is that once we account for count and hitter/pitcher matchup, we’ll find MLB is in fact pretty close to such as equilibrium. 

However, I do think basketball may be different.  The difference there is that the offense (and to some extent the defense) can select their play with knowledge of the other side’s decision.  That is, if Cleveland’s opponents double-team LeBron, Cleveland can elect to have another player take a shot.  And even if the defensive strategy remains static, in the nature of the game there will be plays where LeBron is more or less open on any given play.  Presumably, the Cavs try to have LeBron shoot when his expected points is higher than his teammates’ (at that moment), and vice-versa. 

So even though LeBron is more productive than his mates on average, it may not follow that he will be just as productive if you force him to take more shots—even if the defensive strategy remains unchanged.  You would be asking LeBron to take these extra shots not in random circumstances, but specifically those when the team thought another player had a greater likelihood of success given the situation at that moment. 

Maybe equilibrium still requires equal outcomes for both strategies under these conditions, but I’m not sure it would.

Let’s say we have this chart:

Feet Swing Bunt
70 0.470 0.140
80 0.460 0.240
90 0.450 0.340
100 0.440 0.440
110 0.430 0.540
120 0.420 0.640
130 0.410 0.740

It reads like this: when the 1B/3B play Pujols 70 feet from home plate, his wOBA on swinging is .470, and his wOBA on bunting is .140.  When they play him 130 feet, his wOBA on swinging is .410 and on bunting it’s .740.

If he swings 91% of the time, and bunts 9% of the time, his overall wOBA is .440, at every spot.  That is, regardless of where they play him, as long as he always swings 91% of the time and bunts 9% of the time, he will always have a wOBA of .440.

However, if Pujols sees the fielders at 70 feet, he can get a higher wOBA.  Indeed, if the fielders always stay at 70 feet, Pujols never has a need to bunt, and he’ll get a wOBA of .470.  Similarly, if the fielders are always playing Pujols at 130 feet, he doesn’t have to swing ever, as he’ll just bunt, and get his .740 wOBA.

(All numbers for illustration only.)

The optimal point for the fielders is to make sure to position themselves so that Pujols gets a .440 wOBA.  In order to do that, they play him at 100 feet.  At that point, Pujols can choose to never even bunt, and the defense has been positioned optimally.  However, if the defense knows that Pujols will never bunt, they will play him at 110 feet, trying to get his wOBA down to .430.  Pujols therefore is forced to bunt at least 9% of the time to get his wOBA back to .440.  Indeed, knowing the fielders are at 110, he should bunt 100% of the time (all other things equal).

(Based on the completely made up wOBA numbers presented, which is only for illustration.  I just know someone’s going to take that out of context and say I propose Pujols bunt 9% of the time.)

Similarly, the suggestion is that Mariano should go outside-outside and outside-inside such that the resulting wOBA is the same.

But, are players really that tuned to playing that optimally.  Say that Pujols sees the fielders at 110 feet (where he gets a wOBA of only .430).  They are doing that because they don’t think he will bunt any more than 9% of the time.  They think he’s a 2% bunter.  That makes Pujols a .432 player, not a .440 player.  Pujols, on the other hand, doesn’t think it’s worth trying to maximize his value, if it means he has to do something different (bunting). 

So, perhaps the outside-outside and outside-inside split for Mo is such that it’s too much of a bother for Mariano to switch.  He thinks he’s optimal, and if he changes things, it may turn out worse for him.

Is this possible?

Ok, now let’s try football:

All numbers are made-up.  This is the chart for 1st and 10 at your own 30, 5 minutes into the game.

The first column is the expected frequency of passes, the second one is the number of yards per pass, the third is the number of yards per run.  The last is the overall yards per play, based on the frequencies of pass and run.

We see here that if your expectation for pass or run is 50/50 (or 60/40 pass), and that is the actual frequency, then the yards per play is 6.

If you expect a higher frequency of passing, say 90%, and that is what you get, then only 4.2 yards per play will result.

Therefore, this is an example of where you should not expect the same payoff for pass and run.  That if you expect 60% passes, and the offense shows 60% passes, and the offense gets 7 yards per pass and 4.5 yards per run, then the offense has maximized its production.

And therefore, as long as the offense keeps the balance 60/40, and the defense guesses 60/40, then we’re at an equilibrium.  The payoff for each decision doesn’t have to be the same.

So, I think this is what we’re up against with Mariano’s data.  They payoff of each doesn’t have to be the same.  Otherwise, it would mean we’d need 76.67% passes attempted to get a 5.33 yards on pass and 5.33 yards on run.

Tell me where I’m wrong.

Tango:
Phil’s suggestion is helpful.  Pick outcomes for these 4 conditions for Offense/Defense:
Pass/Pass Defense
Pass/Run defense
Run/Pass defense
Run/Run defense
Then you can calculate the total gain for any combination of Pass% and PassDefense%.  You’ll find that there is a solution where neither side can gain by changing it’s allocation.

Thought we should close the loop on Mike’s data.  If you adjust his pitch outcomes for quality of hitters, I get the following results.  (Apologies for format. The last column is the important one.)

Count NFB FB Delta
0 0 0.000 -0.002 -0.003
0 1 -0.042 -0.043 -0.001
0 2 -0.106 -0.095 0.011
1 0 0.035 0.032 -0.003
1 1 -0.014 -0.017 -0.004
1 2 -0.084 -0.073 0.011
2 0 0.090 0.092 0.001
2 1 0.028 0.033 0.006
2 2 -0.047 -0.035 0.012
3 0 0.195 0.190 -0.006
3 1 0.136 0.129 -0.007
3 2 0.033 0.065 0.032

Generally we see very small difference within the range of sampling error, except for the three 2-strike counts.  On these, the fastball appears less effective than the offspeed pitches, and the difference is quite large at full count.

I haven’t adjusted for pitcher quality, because the differences were very small.  But one more element could be very important:  batter/pitcher handedness.  Overall, FBs are thrown more often when hitters have the platoon edge, so that might have some impact.  And we know that pitch type varies a lot based on platoon advantage (sliders are used mainly when pitcher has advantage, changeups when hitter has edge).  So it seems possible that on 2-strike counts the FB/NFB mix could vary a lot based on handedness, with pitchers throwing a lot of NFB when they have platoon advantage (especially LHP/LHH), but relatively more FBs to opposite-handed hitters.  If so, that would make the FBs appear less effective.

So, it may be that pitchers should throw more NFBs with 2 strikes.  But my guess is handedness will explain much/all of the remaining spread.

BTW, Brian Burke did nice analysis of the football data in the paper here:  http://www.advancednflstats.com/2009/10/full-review-of-game-theory-run-pass.html.

Good point about the handedness!

My microeconomics professor emailed me to ask what I thought of this paper (it’s actually a working paper), and I emailed him back echoing a lot of what was said here and at Phil’s blog (giving credit, of course). He responded back saying:

“Interesting.  I’m disappointed but not surprised to hear this.  This is the problem with writing almost every paper on a different topic - you simply don’t have the relevant subject matter expertise to write a useful, first-rate paper.”

Just thought I’d share.

There was a short article in the Washington Post this weekend about the paper, and The Economist also has a write up in the current issue.  In those venues, the findings just get reported uncritically, of course.  (Not a knock on them.)

I think this highlights the problem of going public with work that hasn’t been subject to serious review.  Dan (or others):  what is the view within the profession of promoting a working paper like this to the general media?  Is that considered appropriate? 

And what obligations are Kovash and Levitt under at this point to set the record straight, if any?  Presumably, they’ve read the discussion here, at Phil’s blog, and Brian Burke’s review of the football analysis.  They have to know now that their data does not actually support their conclusions (and that, at least on the baseball side, their conclusions are very likely wrong).  Should they pull the working paper?  Circulate a correction?  Just let it continue to circulate and misinform people, knowing that few will ever run across these critiques?

Game Theory probably has use in a baseball context, and the authors’ paper may be interesting in an academic sense, but it seems to be of little practical baseball value. The authors assume pitch type to be the key variable; pitch location, however, is often more important. Three consecutive fastballs in the same location will likely produce a different outcome than three fastballs on both sides of the plate, up and down in the zone. Moreover, “fastball” is a broad category including multiple subtypes like 4-seam, 2-seam, cutter and sinker. Skilled pitchers add and subtract velocity on their fastball, move the ball around, and vary its action to create greater variety than the catch-all fastball classification suggests. For starting pitchers, a times-through-the-batting-order effect may also be present, e.g. establishing fastballs to a batter his first time up for the purpose of setting up a different pitch in a subsequent and perhaps more critical PA.

Keith, they didn’t “assume” it.  That’s the only parameter they controlled for.  They “assumed” that all other things were equal (meaning the rest of noise randomly distributed, with no bias).  I think that assumption is fair.

Tango/69 - Understood. By “assume” I referred to the study design which led to a plausible but possibly incorrect explanation. Implicit in a belief that all other things are equal is an assumption that fastballs and non-fastballs are thrown for strikes at equal rates. OPS is roughly 150 points higher for PA ending on in-zone pitches; consequently any pitch thrown more frequently for strikes is bound to result in higher OPS regardless of any other factors. (Fastballs, of course, being thrown for a higher strike percentage, aggregated across all pitchers.) Perhaps the study should have controlled for this. OPS on PA ending on fastballs is roughly 200 points higher than those ending on non-fastballs but a large portion of that difference goes away when comparing only in-zone pitches.

Continuation of #70: Strike percentage is the complication with simply tossing more non-fastballs. As the non-fastball strike percentage is approximately 95% that of fastballs, overall strike load would drop if more non-fastballs were used, leading to more hitter’s counts and reduced pitcher efficiency. Those secondary factors may negate any primary benefit from using more non-fastballs on PA-ending pitches. The only way for a pitcher to guarantee the benefit is infallible foreknowledge of which pitch will end the PA. Nonetheless, good pitchers likely do optimize even if they are unaware of it. An out-pitch is not just an effective pitch thrown at random times. A skilled pitcher calculates the correct occasion to attempt to finish off the batter with his out-pitch (often a non-fastball). Guesswork and chance are at play, but the out-pitch phenomenon seems to demonstrate the ability of pitchers to optimize.

Keith: unless you are going to provide evidence for this out-pitch “phenomenon”, it’s really just something people say.

Are you suggesting that a pitcher will be able to get more value from his “out pitch” when deployed in out-pitch scenarios, more than we’d expect if there was no “phenomenon”?

Tango: To cite a specific example, Lincecum’s most-used pitch type on 2-strike counts is his changeup, with which he recorded outs on 91% of PA ending with 2 strikes. On PA ending on other counts, the effectiveness of his changeup was similar to that of his other pitches. It was considerably more effective at inducing outs than the rest of his repertoire, though, on 2-strike counts. He’s an exceptional specimen, perhaps, but seems to represent a case of getting more value from a particular pitch in “out pitch” situations:

PA ending before 2 strikes
Changeup - 62% out pct.
Other pitches - 61% out pct.

PA ending with 2 strikes
Changeup - 91% out pct.
Other pitches - 80% out pct.

Keith, I have yet to see one piece of research that I find credible when we look at “pitch that ends the PA”.  I cannot stand those articles, those that start with the endpoint.  I rail about this all the time, and I always encourage people to look at the “through” counts, not the “at” counts.

Tell me you are new around here, otherwise, this means I haven’t been doing a good enough job in explaining this.

Tango; Yes, of course, but looking at endpoint seems appropriate for a notion of out-pitch. Anyway, returning to the Lincecum example:

Statistic: OPS on and after pitch
7003 PA

BEFORE 2 STRIKES
Changeup .614
Other pit. .621

WITH 2 STRIKES
Changeup .400
Other pit. .532

Regardless of whether PA continues, his changeup is more effective when used on 2-strike counts.

It’s nice of Kovash to let you post the paper.  At the same time, I wonder why the paper continues to be promoted.  I assume Ken acknowledges that he miscalculated OPS, and in a way that potentially has a large impact in evaluating impact of fastballs and non-fastballs.  And I would guess he would agree that the analysis requires looking at the combined “through” counts, not looking separately at the “at” and “after” counts.  Leaving everything else aside, those 2 problems mean the baseball conclusions really have no supporting evidence at this point.  (There are also problems with the football analysis, but Brian Burke seems to feel the conclusion of “not enough passing” may still be valid.)

Now, circulation only among economists and sabermetricians is fine, and will produce useful feedback for the authors. But this is being promoted to the MSM:  the Wash Post, Boston Globe, The Economist, etc.  I just don’t see how that’s appropriate or responsible at this point.  Am I off base?

And if Mike Fast stops by:  any chance you could create 4 separate versions of the table in post 33 for LHP/LHH, LHP/RHH, RHP/RHH, RHP/LHH?  (I know, I’m getting greedy).  Or if that’s too much work, maybe just tell us if the platoon advantage frequency differs much between FBs and NFBs on 2-strike counts?  It’s only on the 2-strike counts where any possible disequillibrium appears to occur.

Ken acknowledged our two main problems (OPS instead of LWTS, and “at” instead of “through” counts).

It also sounds like he’s at the mercy of the process.

Tango, what do you mean by “the mercy of the process?”

It sounds like there are some things that he has control over, and some he doesn’t.  Even what makes it into his own paper.  If Ken stops by, I’ll let him explain whatever he wants.