Buy The Book
Pizza Cutter offers a study in By The Numbers on measuring plate discipline. I’m afraid it’s going to take me several reads to understand it. It looks cool. You can get a link to his article, and talk about it on his blog:
http://mvn.com/mlb-stats/2007/05/24/the-adam-dunn-debate-defining-plate-discipline/
(Comments closed.)
Wow, what a clever way of looking at plate discipline. I never thought of this, and now it seems entirely obvious. That, to me, is a hallmark of great analysis. Thanks for posting this. It’s fabulous.
JB, “sensitivity” and “bias” are the terms of art from detection theory. You don’t want to change terms of art. This analysis works straight off the canonical approach. Its nomenclature should reflect that.
To me, what you miss is that the batter does not test location. A swinging strike on a fastball down the middle is still a Type I error. The batter can’t get a hit if he doesn’t make contact at all. He can, however, occasionally get a called ball deep in the zone.
On interpretation, go back to detection theory. A sensitive test is one that detects the target. Vlad is very ‘sensitive’ because he detects the target, hittable pitches. He also shows tremendous ‘bias’, as his errors bias towards swinging and missing, not taking called strikes.
Rubbish. I don’t know detection theory. I would certainly like to, but it would take a very long time for me to pick up all the mathematical concepts I’m unfamiliar with in my free time. For me to understand it, I need someone to explain it in layman’s terms. Bill James isn’t capable of brilliant feats of mathematizing, but he was the best sabermetrician because he made his points easily understandable.
Terms of art are great for people in the know, but I agree with John that the important thing is to communicate the results effectively, not to comply with industry-speak.
One thing that is worth bearing in mind is that By The Numbers is for a more statistically minded audience than many sabermetric websites so I understand why PC choose to write the way he did—no criticism of him whatsoever. For what it’s worth I thoroughly enjoy all of his writing over at stats speak .... but I did struggle with this paper. Though as I said that could be more my fault than anything else
First off, sensitivity and response bias were a little over my head, but made sense what was being measured.
I see a couple issues with the methodology, but given the available data, it’s not an issue with using it, just that it could be more accurate using something like TLV pitch data, say like enhanced gameday.
One issue would be, as pointed out, that some pitchers might get more swinging strikes out of the zone, so maybe the data needs to be adjusted based on the pitcher. Next, one problem may be that hitters do have some “cold” zone and lay off pitches there on purpose, or some area outside the strike zone they can hit, so they swing there, which I assume the methodology accounts for. Another problem would be, along the same lines, a hitter who waits for the right pitch, a patient pitcher that decides he couldn’t do anything with a certian pitch and waits for another one.
All in all, I think it’s much better way to measure plate discipline than K/BB or the likes, and something similar could be done for pitchers, how many “missed bats”, ie, missed swings in the strike zones, or “fooled batters”, swings outside the strike zone, as well as how many aren’t swung at outside the zone and how much contact is made.
I liked the article on OBP and SLG relationship. Makes sense that it’s dependant on run environment.
Should have read “a patient HITTER that decides ....
Y’all have hit on the one methodological problem that I wish I could have done differently. My data set was from Retrosheet and I only had pitch results, and not pitch locations. Pitch location data would make these metrics 10x better, but I didn’t (and still don’t) have it available. Over the summer, maybe I can cobble it together. Phil Birnbaum (editor of BTN) and I also went back and forth on the batter who lays off a specific pitch figuring that even though it’s a strike, he can’t do anything with it. I consider this metric something of a work in progress. I’m happy to share my databases with anyone who wants to try to improve it.
I also struggled with whether to stick with the “industry” terms used (response bias and sensitivity) or to come up with new more intuitive terms. As John/4 pointed out, I did write it at a fairly high level stat-wise based on the readership of BTN. Perhaps a companion piece explaining the concept is layman’s terms is warranted over on StatSpeak.
On the Vlad example, the guy swings at everything. If you look at the ratio of his swinging strikes to called strikes, it’s very very unbalanced. But he’s also a high contact hitter and making contact with the ball is the surest way to avoid striking out. In fact, the guy doesn’t have a lot of strikes on him (called or swinging). So, if he does make a mistake (a strike), it’s most likely going to be of the swinging type (because he swings so much). That’s what his response bias tells us. But he’s very good at avoiding strikes (mistakes) in general, which is what his sensitivity rating tells us.
PC
Thanks for the clarification on Vlad.
Just to make sure I get this: Sensitivity is a measure of how often a player avoids taking a called strike ?? Of course, a swinging strike is a mistake .. but I guess that is what you are capturing in response bias?
By the way, if you do have the inclination to do a layman’s piece of Stats speak I, for one, would appreciate it.
Sensitivity is essentially a measure of how often a player avoids any sort of strike, called or swinging. He may do this by either taking a ball or putting the ball in play. Higher numbers mean you are better at doing something with the ball other than having it be a strike, even if it’s just grounding out to short.
Response bias is how likely a player is to swing. So, players with a high response bias (Vlad) swing a lot (whether they swing and hit it or swing and miss is irrelevant to the response bias statistic). Players with a low response bias (Brian Giles) don’t swing much (whether it’s a called strike or a ball is irrelevant for response bias.) At a response bias of 1.00, you are mathematically minimizing the number of strikes on you (whether called or swinging), given your level of sensitivity. If you’re above 1.00, you swing more than you should. Some of those swings will result in balls in play, but you purchase those at the cost of a disproportionately greater number of swinging strikes. The higher you go, the more swinging strikes you have to endure to “purchase” another ball in play. Going the other way, below 1.00, you will take more pitches than you should. Some of those will be balls, but a larger proportion will be called strikes.
Just think: Vlad could stop swinging so much and probably not be the worse off for it. In fact, he might actually be a better hitter for it.
I’ll hopefully have the explanation piece for signal detection this weekend.
The statistical details of this are over my head. But if I’m understanding this correctly, the basic idea is that batters should swing at all strikes, and take all balls. PC does say that this might not be exactly correct, and that refinement is possible, given better data. But to me, the idea that a batter should swing at all strikes is so far from true, that I question any results or conclusions springing from that as largely worthless. IOW, if you don’t have the data to do it right, then maybe you shouldn’t do it al all.
The assumption that all strikes should be swung at is suspect and a weakness of the method, I am happy to admit. But even those situations can tell us something about the player. Given, there are times that it’s prudent to lay off a pitch (and with the data I have, it’s hard to tell when those are exactly.) In that case does the batter take the pitch or does he swing anyway? This tells us a little bit about whether he’s a free-swinger or a taker (response bias). It will affect the sensitivity measure (a batter is penalized for what is essentially an unhittable pitch thrown by the pitcher… not the batter’s fault, just the pitcher’s good work), and that’s the part that needs refined a bit. The reason, even realizing the troubles with the data sources available, that I did the study is that the field was stuck in an errant supposition that K/BB = plate discipline. My goal was to offer a different framework, where none had existed before.
Well, I’m still waiting for your definition of plate discipline, given in plain English.
I still argue back and forth with myself whether plate “discipline” is the right word. I suppose the sensitivity measure could be boiled down to “strike = bad”, but it allows for the fact that there’s more than one way to not take a strike (either a ball or putting the ball in play) as K/BB doesn’t. My definition of plate discipine, at least in these metrics is that the hitter must do two things on every pitch. He must first figure out if the pitch can be hit, then decide whether or not to swing. A disciplined hitter is one that can tell the hittable (would-be strikes) apart from the non-hittable (presumably balls) and respond appropriately, neither being too anxious nor too reluctant to swing. These are two separate skills that I believe form what is commonly known as plate discipline.
Great post from JinAZ on this
http://jinaz-reds.blogspot.com/2007/06/adam-dunns-plate-discipline.html
Hopefully this works: All 2006 hitters with more than 100 PA in 2006.
http://spreadsheets.google.com/pub?key=p7lF8U3MSyYPhcvrcQuz3ZA
Players who debuted in 2006 (or perhaps 2005) don’t have their name given on the sheet, but are listed under their Retrosheet and possibly their Lahman ID.
By the way, does anyone have a Lahman/Retrosheet ID patch that goes all the way through 2006 and/or 2007?
Thanks for posting those data! -justin
In terms of plate approach, I’m essentially interested in 2 categories: 1) what is the “take%” on pitches out of the strike zone, and 2) what is the “swing%” on pitches down the heart of the strike zone.
The data PC uses is not all that helpful, because it 1) does not distinguish between strikes which were actually strikes, and those which were bad pitches swung at, and 2) does not distinguish between “fat fastballs” and “pitcher’s strikes” (strikes which have a low probability of being hit safely and/or far).
And that may be why we get strange looking results such as N Perez having great “sensitivity” and B Giles doing poorly in “response bias”.
Sorry, but if I have to judge a batter’s approach, I’d rather look at BB/PA.
I think conceptually Pizza system could be interesting but i agree with David that the data he uses isn’t granular enough to allow us to test plate discipline.
Ultimately what we want to know is what are a batter’s hot and cold spots and based on that define what his own “strike zone” is and based on this understand how well he understands plate discipline.
I imagine in the ultimate implementation we use some sort of continuous function based on babip by strike zone location
If you’re above 1.00, you swing more than you should. Some of those swings will result in balls in play, but you purchase those at the cost of a disproportionately greater number of swinging strikes. The higher you go, the more swinging strikes you have to endure to “purchase” another ball in play.
Of course, another issue to deal with here is what the cost of a strike is, and whether that cost is the same for all batters. A strike would seem to cost less to Vlad (who has a contact rate around the league average, which is actually amazing given his his large “swing zone") than to Adam Dunn (whose contact rate is far below the average).
There is certainly a way to figure out how much strikes cost different batters ...
Beamer - The “ultimate implementation” that you mention in post #18 would be good at measuring one of the skills that PizzaCutter is trying to measure, the ability to hit a strike, but would say nothing about the other skill involved in his study, strike zone discrimination. The problem with PC’s study is not just lack of pitch location data. It is in the design of the study. To use his own terminology, it is not sensitive enough in measuring either specific skill because it includes both. The results are also muddled by the different strategies that both pitchers and batters use on different counts in the pitch sequence. The result is that you get a lot of type II error, batters that know a pitch is a strike but either let it go without swinging because they want to “see a pitch” or employ a low percentage of contact strategy by taking a big cut hoping for an extra base hit. You also have type I error with players like Vlad whose extraordinary ability to put balls in play that are out of the strike zone
masks his lack of strike zone discrimination.
PizzaCutter’s experimental design would be better if his study were limited to just full counts. This would eliminate much (but not all)of the strategies employed by the pitcher and batter. Also, on a full count the batter has a very high incentive to not swing at a hittable ball out of the strike zone, since he gets a walk if he lets it go. The number of full count fouls in relation to a players full count BABIP would be purely a measure of bat control; the ability to spoil a strike in a location that the batter didn’t feel he could hit well or the ability to put his bat on a pitch outside the strike zone that fooled him.
Peter, I considered doing the full count study (I could still do it fairly easily) for the very reason you bring up although I’m concerned about the effect that it will have on sample sizes. I’m a little on the busy side in my real life, but if I have a chance, I will take a look.
Using 2004-2006 data you get 270 batters who had more than 100 PAs with full counts. That seems respectable for a study. Besides, large sample size is only a virtue if it is contributing to what you want to know. In your case it is not.
I think you’d also want to consider the game state.
For example (and ignoring inning/score for now): http://www.insidethebook.com/ee/index.php/site/comments/leverage_index_by_base_out_states#1
With 2 outs and runner on 2B only, the pitcher will fear a HR alot more than a walk (relative to his normal fear of HR and walks). With bases loaded and 0 outs, the pitcher will avoid, more than usual, to throw a pitch outside the strike zone.
Of course, you’d want to include the inning/score as well. In essence, you need to create a gamestateLI for the walk, and use that as an additional parameter.
Peter, that’s a great suggestion. Looking at full counts would remove many confounding factors. Even looking at all 2-strike counts might be interesting (larger sample).
Still, I think it’s much more useful to think of plate discipline in terms of the swing/no swing decision. How often a hitter makes contact when swinging is, to me, a separate issue. And it’s intertwined with what a hitter does when he makes contact—Dunn may make contact much less than Hatteberg, but when they do make contact Dunn has a .768 SLG compared to Hatteberg’s .464 (and 60 points higher BA).
One variation on Peter’s idea would be to compare a hitter’s rates for BBs and strikeouts-called (SOc). These represent the principal good and bad outcomes of the decision not to swing, and should give us a sense if a hitter makes that decision well. A free swinger will have few SOc but also few BBs; an overly patient hitter should be high on both. B-Ref now has this data available for hitters. Dunn, for example, from 2001-2006 had 291 SOc and 574 BBs, or +283 (as we’ve discussed elsewhere, subtraction is a better approach than ratis when looking at BBs and Ks). Hatteberg, often viewed as a very disciplined hitter, is +267 over the same period (97 SOc, 364 BB), and they are about the same per 162 G (Dunn +56, Hatteberg +54).
Obviously, Dunn has far more swinging strikeouts than Hatteberg. But those are a cost paid for his power, so we can’t simply call those a “mistake.” Allowing a called strike 3, however, is always a mistake.
(B-Ref doesn’t post league averages for SOc—anyone know where to find that?)
Peter/22, One minor methodological issue that I could see with only including full counts is that it might load the sample sizes up for those players who have really long at-bats. When I refer to sample size, I’m not worried about having enough players, but having unequal sample sizes between players. If I’m a hitter who swings at everything and is fairly good at making contact, how many full counts would I have? A 3-2 count requires at least five pitches, but it may also be a special type of at-bat for some players. (Vlad has a cold and decides not to swing as much that day?) What if I don’t like hanging around that long in the batter’s box in general? Will some of the more fringe-y players who fit that mold be excluded if I use a cutoff of 100 PA? I think that the methodological strength that your approach gains outweighs the drawbacks, but it should be approached as something of a selective (and biased?) sample. Ah, the joys of research methodology!
In his career, Hatteburg has gone to a full count in 14% of his PA. Vlad is 6.5% (but still with 418 PA).
I think you’ll have more than enough PA for each hitter to figure out how he approaches a 3-2 count.
As I mentioned, I’d also want to include the game state parameter.
Tango - If you try and add too many variables then sample size does become a problem. Although base-out state, game-state, slugging percentage, and batting order position are all important factors of when a walk occurs, meaningful results can still be found in a study like PC’s if limited to full counts. Of the factors listed above the most important for this study is slugging percentage and or batting order position as base-out and game-states tend to even out over three year aggregated data. And the best way to control for slugging percentage or batting order position is to compare players with similar values.
For example:
--------3_2PA--K_S--K_C--BB--H---BB/K---BA
Griffey---179----41---14--49--22--.89--.172
Guerrero-131----15----3--49--22-2.72--.275
Its clear that even though Vlad has a reputation for being a free swinger, he actually has excellent strike zone discrimination, as evidenced by his BB/K on full counts. Additionally, he has excellent bat control as shown by his low K_S total and good BA. Griffey, not so much.
A few more results for the Patience Metric (BB minus SOc). I think it’s interesting, though I’m not sure it tells us any more than (BB - K).
Per 162 G:
Soriano +21
Dunn +56
Vlad +55
Pujols +68
Bonds +107
(Multiply by .3 if you want a rough estimate of run impact.)
Fun fact: Vlad has just 67 career SOc (compared to 564 for Barry)
And here’s the always great Dan Fox with his take:
http://danagonistes.blogspot.com/2007/06/quantifying-plate-discipline.html
I’m still struggling to fully wrap my head around this.
I generally like the idea but I stuggle to really understand sensitivity and response bias. I understand what is being measured but the words chosen aren’t that intuitive—they need to be renamed.
Anyway, a concern is whether you can actually do this analysis without pitch by pitch data, such as that from Enhanced Gameday.
I’m not sure how much faith I have in the false alarm rate SS/(SS+BB). What this is trying to measure is how often hitters swing at balls out of the zone (ie, the pitches they should leave). The assumption is that if you swing and miss then the ball is out of the strike zone.
I’m not sure this is true. Cause and effect are difficult to tease apart. For instance, there is no doubt that pitchers throw Frenchy more balls because he is more free swinging.
Anyway, I had a look at Kevin Millwood with the Enhanced Gameday data (data was to hand—this is not a scientific study) and I found that 25% of the swinging strikes that he induced were in the strikezone.
Pizza—any thoughts on what impact this may have on the metrics?
The other thing I don’t quite get yet is the diffence between response amd sensitivity. I still don’t 100% follow PC’s example with Guerro. His response bias is poor because he swings and misses ... but he has the best sensitivity in the game? According to the BTN article
I can’t see how Vald would be top by that definition.
I can only assume I am missing something ...