THE BOOK--Playing The Percentages In Baseball

Given Mike’s map and the BIS map, it seems that the difference may be stark enough to rule out smoothing at a level of simple density plots.  But, here’s what I’m imagining they might do (pure guess): plot the probability of striking out.

Mike uses density maps in his article, which simply plot location as a function of the number of pitches.  However, if BIS is plotting something like the probability of striking out on a given pitch, the probability will be VERY low within the strike zone and extremely high in areas where there are very few pitches thrown in general.

For example, say he strikes out on a pitch waist high 4 feet outside of the zone.  The rest of the time, it’s relatively random throughout the strike zone.  You’ll get a big red splotch way out at the 4 feet outside mark, but everything else will look relatively cool colored.  I’ve run into this problem a lot with heat maps, which is one reason I haven’t done too much with these (Albert Lyu has noted similar difficulties).

If these are in fact simple density maps, then there is a real problem with the data.

(begin caveat)
I’m not wild about the colors that BIS chose for their heat map, but that aside, if you want me to, I can speculate about what is happening with the pitch charting and why it ends up looking like it does.  I didn’t speculate in my article because I think to some extent it doesn’t matter to the user why the data is bad; it’s just bad.  And it’s possible that BIS can cast doubt on my speculation about HOW the data comes to be bad.  I left the speculation out of the article because I wanted the article to stand on facts.  But here I’ll speculate.
(end caveat)

Taking Crawford, my guess is that when the stringer sees that Crawford bent over to reach a pitch low and away, he marks the pitch low and away, in approximately the same spot every time, regardless of actual variance in where exactly low and away the pitch was located.  So you get a bright spot on the heat map low and away. 

If the stringer doesn’t get the obvious cue from Crawford’s posture, he has much less information about the pitch location, particularly in absence of an umpire call, so he marks the pitch at the nearest strike zone boundary.  If you take all the clusters from the PITCHf/x heat map and move them to the nearest strike zone boundary, unless they’re low and away, move them to the low-and-away spot, and I think you’ll get something like the BIS heat map.

You can do the same thing for Pena, but since he doesn’t have any obvious low-and-away cluster like Crawford did where his body gives a cue, most of the pitches end up marked in two big clusters right on the outside edge and the bottom edge.

I don’t think what you’re seeing is so much a function of the BIS heat mapping process (though that didn’t help) as it is a function of human behavior by the stringers given the limitations in the information they have.

Millsy, they clearly say they plotted pitch locations, not probability of striking out, unless you can see something differently in what they said:

Let’s take the Bill James Baseball IQ app for a test drive.  One of the chart selections on the app is called the “K Zone” and it enables the user to view a heat map displaying the location of pitches where a player commonly strikes out.  You can select a batter or a pitcher and there are several different filter options available, such as pitch type, the count and the season (the data currently stretches back to 2007).

As a comparison, let’s look at two left-handed hitters that changed teams this off-season, Carlos Pena and Carl Crawford.  We’ll keep it simple and look at 2010 only and we won’t specify a pitch type.

The first result set we’ll look at is Carl Crawford.

While there were a handful of instances where Crawford strikes out low and in, up and away or high and out of the zone (lighter blue areas), an overwhelming number of his strikeouts in 2010 came on the pitch down and away where he chased a ball out of the strike zone and possibly in the dirt.

http://www.actasports.com/stats_detail/?StatId=281

I put an UPDATE in the main post as well, calling into question what the scale even means.

Hmm.  Like I said, if that’s truly the case, then the data must be a mess. 

Let me take quick look to see if the prob. of a strike type of plot comes up similar to BIS.  If so, then perhaps it’s a mis-communication or simplification in the explanation of what’s going on there.  But from the sounds of things, your conclusion seems to be more plausible.

crap...don’t have the data on my laptop right now (on ‘vacation’.  Will have to see another time.

Millsy, you can download the PITCHf/x data for Crawford from Joe Lefkowitz’s site, or alternatively, I can email it to you.

http://www.joelefkowitz.com/batter_card.php?pid=408307

Millsy, these are not density maps.  If you do a density map using the BIS data it looks nothing like these heat maps.

I am seriously confused as to this conclusion:  “The FanGraphs plate discipline statistics (as currently constituted) would seem to have little basis in fact other than possibly as a record of umpire calls on taken pitches and of when the batter swung.”

You look at a heatmap that in my opinion isn’t particularly good and you compare it to your much better heatmap and you draw that conclusion?

I don’t think anyone is debating that pitchf/x data is going to be more accurate than BIS data, but I just don’t think you can draw any real conclusions by looking at these heatmaps and I certainly don’t see how you can go about attacking FanGraphs because of these heatmaps.  I know you don’t like the plate discipline data, but I think this is a bit much.

I can certainly believe that the data was plotted from an analog scale (i.e, to the millimeter) and converted to a more digital scale (say a 3-inch x 3-inch square).  So, then things look more clustered.

But then when you plot it out, it smooths everything out, so instead of having squares all over, the shapes converts back to something that looks analog.

Millsy, here is the PITCHf/x data for the strikeout locations vs. all pitch locations for Crawford in graphical form.

David/8, I don’t see how the quality of the heat map is the issue here.  Despite the poor quality of the heat map color choice, you can still tell a lot about the underlying BIS location data. 

Are you telling me that you believe that the BIS location data for Crawford actually has a lot of strikeout pitches located in the heart of the zone, but somehow the heat map obscured that fact completely?  You have access to the BIS raw data, so tell me where they actually have the pitches located, if for some reason you believe that their heat maps are telling us a different story than their raw data.

If you can’t produce your own heat map for licensing reasons, tell us how many strikeout pitches BIS had charted within the box of 2.0 < pz < 3.0 and -0.5 < px < 0.5 in the heart of the zone.  PITCHf/x has 21 strikeout pitches in that box.

Also, David, can you clarify what you mean by “density”?

Millsy, these are not density maps.  If you do a density map using the BIS data it looks nothing like these heat maps.

Did you mean “spatial density of strikeout pitches” or “probability that a given pitch was a strikeout”?

More applicable, of course, than the population of all pitches that I showed in #10, if one is mapping strikeout probability, is the population of 2-strike pitches.

Whoops, my post vanished and I forgot to save it!

Anyway, the short version:  I don’t work with BIS location data that much anymore so converting the coordinates is going to be more than I can do right now (on a train to Sloan, anyone in this thread going?).  But there were 36 pitches in what they define as the strike zone.  Not exactly what you’re looking for, but maybe useful.

Some more info is that this is not a BIS in house application, it is done by a third party.  I don’t know which company actually wrote it, but it might be worth finding out.

Mike, so I do my heatmaps by creating points on an image, setting the radius, setting the intensity and then creating an overlapping “heat map” based on the combined density of the points.  It looks like you do something similar because you get tight isolated areas and rounded areas.

The heatmaps in this application have jagged edges.  They are doing something different and I’ve seen them before and used to make them with some excel add-on.  I never could figure out what they were doing and abandoned making them, but someone who knows more about this than I do probably will have some idea.

David, I used Millsy’s technique almost directly lifted from his web page, which I linked at the bottom of my article.  The only changes I made from his code was to use the location data for Crawford and Pena, to change the color scheme to match the BIS color scheme as closely as I could, and to move the boundaries of the zone to where they should be for these hitters.

I’m still curious, though, what you meant when you said these weren’t density plots.  Are you referring to the different smoothing techniques as in #15, or are you saying that BIS plotted something like the probability of strikeout / (all two-strike pitches)?

Dave,

I suspect you are right that they aren’t standard density maps.  I’m in the process of seeing if I can reproduce them using R.  Here is my approach:

Subset the data into 2-strike counts.

Use a loess (gam with loess really is optimal for binomial data, but has difficulty with outliers in this small a data set) to smooth, which essentially calculates the probability that Crawford will strike out on that pitch, given its location.

Map it onto a grid, which gives us the heat map.

I’m already in trouble for looking at baseball stuff (on vacation with my sig. other), but I’m going to try and squeeze it in!

Okay, so using the above methodology, I get a map very close to the one that BIS produces (I think).

Essentially it’s the probability that Crawford will strike out in a 2-strike count, given the pitch location.  I posted it at my site. Keep in mind it’s a quick and dirty version, and I DID NOT rescale the probabilities from the loess smoother to a 0-1 representation.  So essentially ignore the ‘probability’ on the right, thought the image should represent what is going on in the data fairly well:

http://princeofslides.blogspot.com/2011/03/follow-up-to-comments-to-mike-fast-at-b.html

My Internet on my computer died on the train, but I’ll see if I can post a BIS heatmap that I made using the same stuff as the fangraphs tool.  It looks a lot more like Mike’s than the other one.

Millsy, would you be willing to share the R script that you used to produce that LOESS graph?

This is a BIS heatmap version, though I didn’t quite match the colors to yours, but looks pretty close to me.

Thanks, David!  I’ll agree that looks more or less like the heat map from the PITCHf/x data.  There are some differences there but overall not dramatic like the other graph suggested.

So the question remains what BIS was plotting on their heat maps.  Ben indicated that he thought it was simply pitch location density, as the article indicated, but said he would look into it.  Based on David’s data, I tend to believe it’s not pitch location density.

I’m not convinced from Millsy’s graph that it’s strikeout probability on two-strike pitches, but I’m willing to believe that with different parameters perhaps it could be.  A scale of 0 to 15 also doesn’t make much sense for strikeout probability.

Anyhow, Ben said he would get back to me on Monday with further information.

This may be nothing (i.e., just a visual trick), but it looks, to me, like the area outside of the strike zone is smaller in the BIS heat map. If the pitches that are further away than the cut-off get piled up at the cut-off then that might skew the rest of the heat map in weird ways.

"A pitcher relying on counsel coming from a baseball operations analyst who used this BIS data might believe that he should avoid the plate completely with two strikes because Crawford almost never misses a pitch there. “

If you looked at Crawford hit chart, you would probably find out you are as likely to get hammered if you pitch in the zone with 2 strikes, than if you pitch low and away outside the zone. 

Pitchers need to balance risk vs reward.

So pitching outside the zone (low and away) to get a K is a low risk and relatively higher probability strategy. If Crawford chases it, he will do little damage, or strike out. If not, it’s a ball (hopefully), and you try something else. 

If you pitch in the zone in the hopes of getting a strike out, thats a high risk, relatively low probability strategy.  You may get that K, but you are more likely to see Crawford standing on 2B or 3B after the pitch than if you stayed off the plate. 

I suspect that with 2 strikes Crawford would see more pitches in the zone with 3 balls.  With a full count Crawford went 273/.474/.527/1.002.  Those who did not get a k or walk got hammered, probably because they were in the zone. 

With an 0-2 count, Crawford went 120/.120/.120 .240.  He had a 34% k rate, and I would bet most of these pitches were outside the zone.

I’m glad that the map in #21 looks like Mike’s map and I would have expected it.  Dismissing pitch location (or hit location) data that uses video because of the problems with parallax, video distortion, camera angles, and such, is not something I am a big fan of.

I have always felt that it is not too difficult to come reasonably close to pitch f/x data just by watching TV.  And I’ll put my money where my mouth is if anyone wants to challenge that statement (of course we would have to define beforehand what “reasonably close” is and how to measure it).

Yes, it will be less accurate and precise and there will be more systematic error (bias) using video and the eye, but on the whole I expect that a heat map of 100 or more pitches using pitch f/x data and one that uses data from video should look pretty darn similar, as long as the method for producing the heat map is the same.

To start out by saying that there is something wrong with the data because the heat maps looked completely different (which they did) was both unfair and an unfounded and unreasonable assumption.  I am glad that that was so far proven correct (that it was the heat map itself and not the data that was vastly different between pitch f/x and BIS).

Let me be clear.  I was wrong about what the data was.  David’s heat map has made that plain.  Therefore, I was also wrong about what I said about FanGraphs.  I’m curious about what the data actually was, and I’m waiting on BIS to get back to me about that, but I don’t need to wait to say that the conclusions I made about FanGraphs were wrong.  Some or all of what I wrote about BIS is probably also wrong, and that’s what is waiting to be sorted out.

I was wrong, and I apologize to David and FanGraphs.

Mike/26: My kind of guy.

Mike, no worries, glad I could help clear things up!

Yup, it is rare that someone admits a mistake with no waffling or qualifications AND apologizes.  I applaud Mike for that!