THE BOOK cover
The Unwritten Book is Finally Written!
An in-depth analysis of: The sacrifice bunt, batter/pitcher matchups, the intentional base on balls, optimizing a batting lineup, hot and cold streaks, clutch performance, platooning strategies, and much more.
Read Excerpts & Customer Reviews
If you are a media member and would like a review copy of The Book, please contact Kevin Cuddihy of Potomac Books.

Buy The Book from Amazon

MOST RECENT ARTICLES
MAIL : You ask | We say

Advanced


THE BOOK--Playing The Percentages In Baseball

<< Back to main

Wednesday, September 12, 2007

QB rating

By Tangotiger, 03:52 PM

When I was a kid, I used to do QB ratings for CFL players.  I did something like Yards minus 5*incompletes minus sacks minus 50*interceptions, all divided by pass attempts plus sacks.  I didn’t really have anything to back me up.  I just sort of reasoned that getting an incomplete pass (losing a down) would be as costly as the difference between gaining 15 and 10 yards.  And ths was 25 years ago, so I would have hoped that we can be clearer on the answer. The Wages of Wins tries.  In their case, they do minus 3*plays, rather than minus 5*incompletes.  I’m not sure I agree that you look at plays (PA in baseball talk) and not incompletes (outs in baseball talk). Also, for whatever reason, they round “35*interceptions” down to “30*interceptions”.  What’s wrong with leaving it at 35?


#1    Xeifrank      (see all posts) 2007/09/12 (Wed) @ 19:57

Any idea where one could find detailed game stats of NFL and College football games.  The typical Yahoo and ESPN box scores only carry the standard stuff.  Thanks.  vr, Xei


#2    Patriot      (see all posts) 2007/09/12 (Wed) @ 20:27

My personal invention QB rating as a kid followed the Runs Created model of multiplication, and was Completion %*Yards/Attempt, which of course if you expand it is completions*yards/attempts^2, which doesn’t make a whole lot of sense.

I remember on the few occasions where we’ve (we being the old FanHome crowd, or Primer, etc.) discussed this in the past, there were some people who thought incompletions were irrelevant, and that attempts were the only opportunity factor.  I still have a hard time with this.  It seems to me that there has to be some value in completing passes and extending drives.  Leaving time out of it, you have two basic elements in football: trying to advance down the field to the end zone, and trying to advance down the field to the first down marker.  The two go together, but not perfectly.

However, I don’t see the problem in their case with subtracting plays instead of completions.  I mean, if I ran a regression on runs from H, W, TB, and AB, and got .6H + .25TB + .33W - .1AB, I could rewrite it as .5H + .25TB + .33W - .1Outs.

I also wonder why touchdowns should be completely ignored.  They are of course team-dependent, probably more strongly then the other categories even, and yet I think they probably deserve a small weight.  A 3rd-and-goal TD from the 4 may lower your yards/attempt, but it is a very positive play.

Pete Palmer and co. in the Hidden Game of Football used (Yards - Sack Yards + 10*TD - 45*INT)/(Attempt + Sack)


#3    auntbea      (see all posts) 2007/09/12 (Wed) @ 20:35

Tango:  Not sure how complex you want these ratings to be, but not all interceptions are remotely equal.
If you watch Peyton Manning enough, you realize he takes far greater risks on 3rd down, especially 3rd and long, as an interception downfield in that situation is far less costly than one on 1st down.  Jay Cutler, the young QB for the Broncos, has yet to figure out when to force the ball into coverage, as evidenced by some extremely questionable decisions on first down against the Bills last weekend, only one of which (luckily for him, IIRC) was intercepted.  Also, Cutler may have cost his team a shot at the playoffs with a very poor decision to throw into coverage in the flat (on first down) leading to an interception and TD against the 49ers in the final game of last season.

While a play-by-play, value-added approach such as the one Football Outsiders uses to measure QB performance is inherently tainted by random fluctuation that is interpreted as “clutchiness”, such a ranking does underscore the incredible difference in value for interceptions at different times/downs/yards to go in the game.  Quarterbacks have a lot of control over when to force a ball into coverage, and some of that control is reflected in these value added stats.


#4    Jim A      (see all posts) 2007/09/12 (Wed) @ 22:01

Interceptions are very difficult to weigh properly in QB rating systems.  Not only are they very dependent on down/distance situations as auntbea suggests, their frequency also varies greatly according to the current score.  QBs on poor teams throw a lot of interceptions, not only because they are passing more often, but also on a per attempt basis because they are taking more risks in an effort to catch up from behind.  I don’t know whether systems based on regression really account for that.


#5          (see all posts) 2007/09/14 (Fri) @ 10:55

The guy who runs that site also insists that basketball players are “more consistent” than baseball players, because his win score rating (or whatever it’s called) is more stable from year to year than batting average and RBIs.  Drives me crazy, I’ve pointed out to him why this is silly to write, but he keeps saying it.

Tango, you’re a hockey fan right?  Is anyone trying things like this with hockey?  I tried a couple years ago to make something like the original 82games.com “roland rating”, which was essentially a comparison of the +/- of a player relative to his team.  I scrapped it when it showed David Tanabe as one of the top 5 players in the NHL grin But I fear that goals are so infrequent during a season that it would take 4 or 5 seasons to even get a valid sample size from which to do that sort of analysis.  Just curious if you’ve found (or developed) anything that works reasonably well…


#6    tangotiger      (see all posts) 2007/09/14 (Fri) @ 14:17

Mike, can you provide a reference for the WoW claim?

As to your second point: no comment.


#7          (see all posts) 2007/09/14 (Fri) @ 16:39

Well yesterday’s post:
http://dberri.wordpress.com/2007/09/13/not-exactly-good-news-for-greg-oden/
“As often stated in this forum, NBA players are quite consistent.  At least, relative to what we see in football and baseball, basketball players are amazingly consistent across time.”

http://dberri.wordpress.com/2007/09/09/a-new-qb-score/
http://dberri.wordpress.com/2007/08/28/how-to-lose-your-best-player-and-win-a-title/
http://dberri.wordpress.com/2007/03/19/but-you-are-consistent/

A while ago, I commented on it a couple times, but I can’t find what I wrote and I didn’t want to beat a dead horse.

Best of luck with any hockey project that may or may not exist grin


#8    tangotiger      (see all posts) 2007/09/14 (Fri) @ 16:52

I posted.  I’ll be ignored of course.

What’s the point of having a blog, if it’s one-way?


#9          (see all posts) 2007/09/14 (Fri) @ 20:44

Interesting, you point out sample size… I think my comments to him were regarding the stats he’s using.  I couldn’t have told you that Ordonez and Posada would have such great BA this year.  But I’m pretty sure I could have predicted that Dunn and Howard would be near the top of the league in K rate, Johan would be near the top in K/BB, and Wang wouldn’t give up a lot of HRs.

If we believe “skill” remains pretty constant in players, barring changes in philosophy, injury, bulking up, etc… I think the “consistent” stats are pretty good measures of skill. So I applaud him for coming up with a way to measure basketball players that’s consistent from year to year.  That makes me think it’s a good indicator of some inherent skill.  Totally agree on blogs.


#10    tangotiger      (see all posts) 2007/09/14 (Fri) @ 22:08

And the reason that K rate is so consistent is that it’s not an important skill to have, and therefore, there is great variance among players.  But, getting hits on BIP is important for all hitters to have, and therefore, less variance (i.e., selective sample).  And, the lower the variance, the smaller the correlation.  Furthermore, with BIP, you also involve fielders, and therefore, introducing more noise.

It’s basic: the larger the sample and the larger the variance, the higher the correlation.  Even something with almost no true talent will have a correlation approaching 1, if the sample size approaches infinity.


#11    David Arnott      (see all posts) 2007/09/16 (Sun) @ 15:09

I’ve read the WoW book, and there is a lot lacking. They take shortcuts that make no sense. In their basketball evaluations, specifically, of course the total wins valuations will work out when they equally distribute defense based on playing time, but individual players will not be valued properly.

http://dberri.wordpress.com/2007/03/26/incorporating-defense/

As for football, while not open-source, FootballOutsiders.com seems to have the right idea with play-by-play evaluation of play values. A two yard run on third and one is not worth the same as a two yard run on first and twenty, and that’s before taking into account field position. The same can be said of passers.


Page 1 of 1 pages


Name (required)
E-Mail (optional)
Website (optional)

<< Back to main


Latest...

COMMENTS

Jan 09 16:41
Sabermetric Moves of the 2009 Pre-Season

Jan 09 19:56
Modeling Baseball Player Ability with a Nested Dirichlet Distribution

Jan 09 18:08
Line Drives

Jan 09 18:04
Challenging Nate Silver (and all other forecasters)

Jan 09 17:31
Cheers

Jan 09 17:14
Teaching sabermetrics at school

Jan 09 16:51
The first Hardball Times Annual available for download!

Jan 09 14:44
Vote for the Worst Player in MLB

Jan 09 12:29
Clint Eastwood is Archie Bunker

Jan 09 12:16
Mailbags on Parade