THE BOOK--Playing The Percentages In Baseball

I have not read the study or the rebuttal yet, but the principal reason why football teams don’t opt for the play that yields the most average yards (other than the yards needed for a first down, which is a critical determining factor of course) is game theory.  You can’t pass or run too much or the defense will key on that kind of play and eventually it won’t be worth nearly as much as starts out.

For example, if a pass, given normal defense for the game situation, is worth 5 yards and a run is worth 3.5 yards, if you opted to pass all the time, the defense would change their defense, key in on the pass only, and the pass would now be worth 4 yards and the run 4.5 yards (or whatever).  THAT is why teams can’t choose to pass or run according to how many yards each is worth given the defense and given the amount of time they pass and run typically.

Without first figuring out the correct balance between running and passing according to game theory (and that is not easy), any analysis is useless.

I assume that the author addressed this issue, but you don’t mention it in your critique.

Actually, I did mention it originally, then deleted it ... probably should have left it in. The author actually never mentions the game theory aspect, which is a curious omission—however, since he did come up with a mix of running/passing for all situations, maybe he thought it was obvious.

Here’s a question for game theorists reading this ...

My understanding of a solution to a game is that it occurs when neither party can do better by unilaterally altering his strategy, even if the opponent doesn’t change his own.

So let’s say that in equilibrium, a pass is worth 5 yards, and a run is 3.  My rational strategy might be to pass more.  But since we’ve assumed equilibrium, I should do WORSE even if the opponent doesn’t change his strategy (by defending against the pass more and the run less).

But that doesn’t make sense, does it?  In equilibrium, shouldn’t the run and the pass have equal expectations?  Doesn’t the fact that they are different mean that we DON’T have equilibrium?

Of course, it could be that on first and 10, the expectation is equal, but on third and 2, the pass is worth more because the defense is looking to stop the run and get the ball back.  But wouldn’t you still have to have some kind of equal expectation (perhaps including the punt probability) for both the run and the pass?

The whole thing is predicated on the correct game theory mix.  How you can’t mention that, I don’t know.  (I still have not read the study and rebuttal).  It may be obvious that game theory says you must mix up your pass/run ratios even if one averages more yards per play than the other, but it is far from obvious what that correct mix should be.

In order to figure that out, you would have to explicitly know how much the defense’s expectations can affect yardage.  I’m not sure THAT is even doable, although I suppose you could just guess.  One way to make an estimate, I suppose, would be to look at average yards gained when the defense expects it and when the defense doesn’t, for a pass and for a run.  That gives you some idea as to how much the defensive expectation affects the yardage. That is sort of what I did with the sac bunt.  I assumed that late in a close game in a sac situation with a weak hitter at the plate that the defense was breathing down his neck. I also assumed that early in a game with a batter who does not bunt that often, the defense was agnostic at best.  And of course with the pitcher at bat, we know that the defense is breathing down their necks.  So we can then attribute the differences in results to the position of the defense.

To some extent we can do that in football.  We can choose a situation where the offense runs almost all the time (say, 4th and .5 yard) and one in which the offense almost always passes.  The problem with that is that the offense does things differently.  In baseball, a bunt is a bunt (by a certain batter).  The only thing that changes is the defense.  In football, with 4th and .5 yard, not only is the defense expecting a run, but the offense is only trying to get a yard or so.  When the defense is not expecting a run, the offense is calling a completely different play (like a draw) if they run.  So in order to try and quantify the effect of the defense on a run or pass, you have to try and control for the type of play.  For example, compare an end around by the tailback when the defense expects a run with the same exact play when the defense expects a pass.

In any case, as I said, the game theory element is a critical part of the analysis.  Without the author explaining that and telling you how he derived the proper mix of pass/run and how a change in that mix affects the average yards per play, there is no way to follow or verify his analysis.

But again, I have not read the article.

Phil, if one party is using a perfect strategy then no matter what the other party does, nothing changes, as you say.  However, if that party deviates from their perfect strategy, then of course the first party must change his, and the second party that changed loses.

We have to be careful about the terms though.  A “perfect strategy” is one in which you maximize your returns.  That depends on what your opponent does.  However, when I say “if one party is using a perfect strategy” above, I mean that he is assuming that his opponent is a perfect game theorist. That is just semantics.

As far as whether the expectancies of all the strategic options have to be the same when the two parties are in equilibrium, I would have to think about that some more.  I guess if it doesn’t matter what I do when you are playing perfect “defense,” then the expectancies have to be the same, as you are assuming too.  So yes, for every given situation (down, score, yard to go, etc.) in football, if the defense is playing perfectly (with the assumption that I am an expert theorist), then the WE of the pass should be exactly equal to the WE of the run. If it is not, then the defense is not playing properly as if you are playing with perfect game theory, or you (the offense) are not playing properly and the defense either is or is not (it is possible for both players to be playing improperly, one player, or neither).

Too confusing for this late at night.  And although I have studied game theory a fair amount and think about it a lot in terms of baseball and other games like poker, I am by no means an expert.

Crap, I just last my post!  I’ll do it again.

After thinking about it some more, Phil is right that the expectancies have to be equal when the game is in equilibrium.  That assumes that one side or the other has not reached the “end of their rope.” For example, when a poor hitting pitcher comes up in a sac situation, no matter where the defense plays, it still is probably correct to sac 100% of the time, in which case, the sac is worth more than swinging away. But in a game where each side still
has strategy options when they reach equilibrium, then the expectancies are definitely the same.

That means that in football, for any given situation or all situations combined, the WE for running and passing should be the same.  If it is not, then someone is making a mistake.  It does not have to be the offense’s mistake, BTW.  If the defense plays for the run too much, then I will pass more and the pass will be worth more than the run.  Maybe that is what is happening in football, and not the the offense is not passing enough.  Either alternative is possible if the pass is worth more than the run.

Of course it is the WE and the yards per play that we are interested in and that should be the same.  And if you are using yards per play as a proxy for WE (I don’t know how good that assumption is), you need to include the fumbles and interceptions, so you need to use some kind of “net yards” that includes that.

I would think that the yards per play should be around the same but I am not sure.  The WE should definitely be the same though.

I think you can model this for football.

There are situations, primarily in the 2nd half when the score is not so close, that the winning team always runs (say Steelers running on 20 consecutive plays) and the losing team always passes. The types of runs or passes may vary, but everyone knows if it’s a run or pass.

In those situations, determine for each team, and then for the league, what the offensive stats are when the defense knows if it’s a run or pass. Then compare to say 1st half/close score situations when teams are most likely to mix their play calling, and see how much the defensive keying effects the outcomes.

I think it should be pretty obvious that using average yards gained is very problematic given that different yards logicly must have very different values.

Excatly how the value function (i.e. Win-EV) looks as a function of yards gained in each situation is most likely very complex but we should at least be able to say that gains beyond the first down distance are likely worth significantly less per yard.

I think that the reason why teams run as much as they do even if the average yards are significantly lower than for passing is because a much larger fraction of running gains are “premium yards” and a much higher fraction of passing gains are “junk yards”.

#7 and #8 good points.  Especially about the pass yardage being more likely to be non-premium (more than needed for a first down).  So it would make sense that even if the WE were the same, the yards per play would be more for the pass.

Then, to complicate matter even further, you have lot of situations where even though, for example, a pass (or run) may be worth considerably more in yards or even in WE, not including the clock, you have a team that is ahead that wants to run out the clock (and therefor run every play), or a team that is behind that needs to stop the clock with incomplete passes and passes to the sidelines.

So I think that the assumption that yards per play should be about the same when the two teams are in equilibrium may be a poor one.

#7 - for the team running out the clock, I think they’ll be mixing in some of their non-premier backs to take some of the abuse.

I guess you could control for this?  And for the passing team that is trying to catch up, there are bigger issues.  Mainly, defenses will play the “prevent” defense, which is distinctly different from any type of defense they’d play when game theory would apply.

I agree with #8 and #9.  I’ve never understood why it’s considered a “paradox” that the average yardage gains for passess and runs isn’t the same. 

Let’s take an extreme example:  every run gains 3.5 yards, while pass plays yield a 10-yd completion 45% of the time and an incomplete 55% of the time.  Passes have a much higher average gain (4.5 vs. 3.5).  Yet the correct strategy is obviusly to run every play, which will aways yield a first down after 3 plays.  In contrast, a team that always passed would give up possession 17% of the time. 

Clearly, the probability of reaching a first down within 3 plays is a huge component of the WE in football.  And there’s no reason to think average yardage (even if interceptions and fumbles were properly accounted for) is an adequate proxy for that.

#11, that is a good example that absolutely sums up the deal.  You pass when you need big yardage and/or few downs and you run when you need small yardage and/or lots of downs.  You mix it up close to 50/50 when it is in between and you mix it up with a more extreme percentage when you have big yardage/few downs or small yardage/many downs.

So sure, there is absolutely no reason to think that the correct yardage split would be even (between passing and running).  Now, that still leaves us with the question of whether teams pass or run too much overall or in any given situation, but it sounds like a bear to figure out what the correct percentages should be!

I agree completely that running and passing should be equivalent in terms of WE, not necessarily in terms of yards, for the reasons everyone has pointed out. 

It seems to me that the only time the WE will NOT be equivalent is when one strategy dominates the other—on most fourth and 1 situations, you always want to run even if the defense knows for sure that you’re running.

Now, in some situations, like, say, first and 10, when making the 10 yards right away isn’t an issue, wouldn’t the change in WE be roughly proportional to the yardage?  In that case, shouldn’t we expect the expected yards to be roughly the same for a pass as a run?

I think first and 20 would probably give you the equivalent proportional yards/deltaWE you are discussing.  At first and 10, a 12 yard gain is a big difference from a 8 yard gain, certainly more than the difference between 8 and 4 yard gains are.

At first and 20, the number of 20+ yard gains occur infrequently enough that it’s not really an issue.

10-15 yard gains occur frequently enough that it is a concern for first and 10.

That makes sense ... but then, if 10-15 yard gains happen significantly more on a pass than a run, then, on first-and-10, the run should gain MORE yards than the pass, right?  To make up for the fact that you’re less likely to make first down.

Offsetting this is that a 9.5 yard gain might actually be better than 10, because you get a free shot at extra yards before running and almost certainly making first down.

I’m not sure I agree with Tango’s premise.  The difference btwn 8- and 12-yard gains on 1st down strikes me as rather small, since the 1st-down conversion rate from 2nd-and-two must be very high.  In contrast, I’d guess the conversion rate from 2nd-and-6 is much lower, making the spread between 4- and 8-yard gains large. 

* *

The other factor to consider is the impact of the yardage gain on one play on the likely gains from later downs.  I would guess (but don’t know) that the average gain on a pass at 3rd-and-9 is lower than average, because the defense knows (virtually) that a pass is coming.  So you want to avoid 3rd-and-longs.  That in turn makes runs on early downs more valuable, because the likelihood of a zero gain is much lower than with a pass.

Good point Guy.  The general wins/deltaWE stands, but not the way I thought!

But, 1st and 20 likely has the proportional relationship we are seeking.

How much of this stuff has already been dealt with by the Football Prospectus people? They have their own PBP system which takes into account many of these factors such as down/distance. I’m not sure, but I think it also tales the time into account.

They have a very complex passer rating system, which takes all this stuff into account on a PBP basis. Sort of a wpa for QBs, and probably more involved than pbp for baseball. Yet I found that the correlation between that, and the simple TD/INC ratio, is about +.80. I mentioned that to the site guru (Aaron something), and he never responded.

The relationship between running and passing was studied by Ignatin/Barra, and I think Palmer/Thorn, and they found little to no correlation between passing effectiveness and rushing effectiveness. So they concluded that the run does not really set up the pass, and vice versa. The problem is that this approach does not really remove the game theory aspect, as they assumed it did.