THE BOOK--Playing The Percentages In Baseball

One of the things ive noticed is that there’s alot of errors from stadium to stadium, some rather large.

When writing articles and using movement numbers and mph, it probably makes sense to use home starts only, although I’ve heard that sometimes there is even variation within the same stadium on a game by game basis.

Hopefully in the future the data will be more accurate.

It’s known that there are potentially large errors in the movement variables and, to a lesser degree, in the speed both from one stadium to the next and from one year to the next in the same stadium.  These seemed to be worse in 2008 than in 2007. I think the same holds true for 2009, although I haven’t worked with the ‘09 data yet.

I have not explicitly looked at the release point information, but I think Harry convincingly shows that it jumps around a lot, too.

Some information about Pitch f/x that people probably don’t realize.  In calcuating the Pitch f/x data the Sportvision computers don’t use images of the ball from their cameras until the ball gets almost halfway to home plate and then they stop capturing ball images about 10 to 12 feet from home.  So the “release point” isn’t actually measured, in is inferred from the path of the ball that is estimated from the ball positions measured between around 38 to 10 feet from home plate.  This is necessary because the program that recognizes the ball position and calculates the flight path of the ball would be confused by the movement of the pitcher or the batter if the digitized images included them.

What this means is that the “release point” is extrapolated from the measured ball positions and extrapolations are much more subject to error than interpolations.  The chance of error on the release point is even greater because the cameras are calibrated on points closer to home plate in order to ensure that the data close to home is as accurate as possible. Most of the path of the ball from the pitcher to home home plate is estimated very accurately.  The Pitch f/x operator monitors the difference between the estimated ball positions and the actual measured ball positions during every pitch and is alerted when they are off by more than .5” and will make in game recalibrations.  But the release point is probably the least accurate piece of data given by Pitch f/x and nothing should be inferred from it.

Peter, great info.

If I am a researcher, what I would like is to see:
1. the actual data points being recorded
2. the actual alterations to the calibrations being done

I will take and accept all the inferences and extrapolations that MLB.com’s braintrust is giving us (data analysis).  But, I’d also want to see what is actually being recorded, too (data recording).

Tango - I sent you the raw data for one pitch via email so you could see what it looks like.  Fun to look at once or twice but I’ll settle for the finished product.

Peter, agreed!

And I have to say I continue to be super-impressed with Sportvision as a company.  Hopefully, they won’t turn Fredo on us.

Re Peter #3:  A few comments on Peter’s remarks:

1.  I agree that release point is determined by an extrapolation of the measured points to the 50 ft point.  The tracking region varies from park to park because of the differences in camera location.

2.  However, I certainly don’t agree with the statement that “the release point is probably the least accurate piece of data given by Pitch f/x and nothing should be inferred from it.” Based on simulations I did last year, I would guess that the uncertainty in the release point due to random measurement error (as opposed to calibration error) is of order a couple of inches at the most.  Certainly nothing approaching 1 ft.  BTW, I already discussed this point with Peter over the phone, and he agrees he was probably too strong in his statement.  Also, many of you may recall the C. C. Sabathia story from last year.

3.  The recalibration that is done “on the fly” is mainly making slight changes in the orientation of the cameras, which can change as the stands fill up with people.  Given the long lever arm, a very tiny change in the orientation of the camera axis can lead to a big change in the inferred positions.  Generally, the pfx operator is able to view on his screen the actual and the inferred foul lines superimposed on each other, and that provides guidance in doing the on-the-fly recalibration.  The other piece of information used is the error in the intersection of the images from the two cameras.  Each camera actually determines a “line of position (lop)” connecting the camera (whose position is known) to the ball.  The intersection of the lop’s for the two cameras determines where the ball actually is in space.  In an ideal world (i.e., perfect calibration), the two lop’s actually intersect.  In practice, they do not.  The location that minimizes the distance between the two lop’s is taken as the ball position.  And the separation between the two lop’s at that location is taken as some measure of the “lop error”.  The operator monitors this “lop error” pitch by pitch, and if it gets too large (say, greater than 0.5” or perhaps a bit larger), then the operator knows he needs to recalibrate.

One of the issues with pitchf/x, in my opinion, that can lead to this is that while I am relatively convinced that the systems are very accurate around the plate, they potential for calibration errors away from the plate is much larger.  There are just fewer natural calibation points away from home plate for both on the fly calibration, and even for the calibration done before games/series.  For the calibrations that are done well before the game starts, they do put markers between the plate and the mound, but IMO, not nearly enough of them.

This can mean that slight miscalibrations near home can translate into large miscalibrations near the mound…

When you see the movement parameters varying from park to park, you can bet that the release point is off by an amount directly related to it....

Maybe thats a plot I should have made...difference from “average” for release point vs differences from average for movement (or acceleration, or velocity) in a given direction...I haven’t made it yet, but I can almost bet that there is a relationship between these…

I’ll see if I can scratch something like this together this weekend.

Ike - Good to see you posting here.  We can use your input.  Are you going to make it to the Summit this year?

Ike #8

I’m working on a park “effect” primer for BtBS, doing something similar to, if not exactly like, what you’re describing.

Unfortunately, I won’t be able to make it to the summit...new job, new kid...these things are keeping me busy and at home these days.  I will be trying to attend via this fantastic series of tubes however.

I’ll be there, Peter.  Are you going?  Anyone else?

Eric - I’m hoping to.  I might have an obligation that would interfere.  Are you going to present a paper?

I don’t think I’m presenting anything this year.  There are a few pretty interesting things I just started to research and look into but I won’t be nearly done in time to present.  I think I’d rather take it in this year and start presenting next year.  In any event, would be great to meet you and everyone else if they can attend.

Getting back to the original topic, Alan and I did talk about why there might be variation in the pitcher’s horizontal release point and I agree that I overreached when I stated that “nothing should be inferred from it”.  I also agree with Alan that any difference due to the extrapolation issues that I mentioned would probably be on the order of 2 to 3 inches maximum.

Now I have a new theory.  I was thumbing through the major league rule book yesterday (don’t ask why) and I happened to notice that there is no rule stating that the pitcher’s rubber has to be placed with its center exactly on the center line between home and second base.  As a matter of fact the illustration in the rule book shows it OFF center, closer to first than third.  Page 5 at: http://mlb.mlb.com/mlb/downloads/y2008/official_rules/01_objectives_of_the_game.pdf

If each home team is free to have the rubber in a different position and if pitchers are working off the rubber with their foot in the same position, then Pitch f/x would be correctly measuring a different horizontal release point from park to park.  Six inches to the left of center in one park and six inches to the right in another could account for the foot variation in Harry’s graph. 

It would be intersting to look closely at these changes in release point to see if they are consistent for each park which would indicate that the rubber is put down in the same position each time, or whether it varies randomly, or if it varies according to who the home team pitcher is.

Peter, great insight!

***

Mike Fast’s article trying to correct the release point:

http://www.hardballtimes.com/main/blog_article/chien-ming-wangs-release-points/

Mike, well done. Have you run that with a LHH/RHH split, to control for pitcher’s shifting on the rubber? Not a lot do it, but I’m curious if that would show any effect.

Mike - Can you easily check the release points of Yankee left handers to see if they exhibit the same shift in direction as the right handers?  If so, it might help to support my shifting rubber theory rather than Pitch f/x error.

Peter, the lefthanders consist of one Andy Pettitte, and his data shows a shift of about 0.6 feet in the -x direction during the April-July 2009 period.

Damaso Marte pitched a couple games with the Yanks right at the very end of this period, but I didn’t query his data.

Make that April-July 2008 in post #19.

Mike - So that is in the same direction as the right handers but about not as much?  It looks like you transposed the portion of the graph for that period for the right handers about .75 feet, is that correct?

What do you think was going on?  Can you imagine how the Pitch f/x cameras could be out of adjustment for that long a period in such a consistent manner?

Here’s Andy and Wang on the same time line. y axis is average horizontal release point by date (y0=55). Lines are marked b’twn seasons. The dots are the same color, but Wang’s are the negative and Andy’s the positive.

http://sites.google.com/site/harrypitchfx/Home/wangpettite.PNG

Thanks Harry.  Your graph looks to me like the offset for Wang and Pettite was about the same and that something was changed between 7/20 and 7/31.  Either the rubber got moved or Pitch f/x made some change in how they calculated things.  More than just the usual before game recalibrations.

But I believe there’s a shift in ax, as well - see here:

http://www.beyondtheboxscore.com/2009/5/30/893513/graph-of-the-day-wangs-spin

The spin movement graph is for Wang only, but the “bonus” ax graph is all Yankee home games. Sorry for the lack of time markings on the x-axis there, but the shift should correspond to what we’re all pointing at, IIRC

Yes, that certainly points to a Pitch f/x problem that got changed. Or perhaps not.  I don’t know what one would have to change as a pitcher if you were actually starting your pitch 8” to 12” further to the right and yet still pitching to the same spot on the plate.  I’ll see if I can find out from Marv if Sportvision made a correction at Yankee Stadium in that period of July last year.

The change occurred between July 28 and 29, 2008.  I emailed Rand Pendleton to ask him if he could find anything in the camera calibrations that was reset at that point.  That was in the middle of a July 28-30 series with Baltimore.  I can’t imagine them moving the pitching rubber by almost a foot in the middle of a series and no one noticing and complaining.  I have to think it is something that changed in the distortion correction values for one or both of the Pfx cameras at that point.

No, moving the rubber in the middle of a series doesn’t sound plausible.  I’ll wait to see what you learn from Rand.  If Sportvision does make a change that has a big effect, it seems like they should have a place on their web site that could be accessed by analysts that would list the change, the date and time it was made, and what needs to be done to normalize all the values.  It would be to everybody’s benefit to be working off the same data, and not have any errors in research that could be attributed to faulty data.

It indeed the camera calibration changed, it might be possible to “replay” the data prior to the change with the new calibrations and obtain corrected trajectory information.  I suspect that it is possible to do that, but it will take a lot of effort on someone’s part.

Alan - I would think that if both the old calibrations and new calibrations were known that there would be a mathematical transformation that could give the corrected values directly from the incorrect values without having to recalculate from the original raw data.  If that is so, each researcher could use the transformation to correct his own database as necessary.  Am I incorrect in my assumptions?

Peter...I guess you are suggesting a transformation to relate corrected to uncorrected values for the 9 parameters.  I doubt that is possible.  But I may be wrong.  I would be interested in learning what actually changed in the calibration.  Was it simply applying distortion factors?

Is there any possibility that instead of just the rubber, that the whole mound was moved? Back in the days of artifical surfaces, and I know for sure at Three Rivers, when there was a non-baseball event at the stadium, they could pick up and move the entire mound. When it was returned, they would need to get it back to the same location on the field.

Brian, I can’t imagine they would have a major non-baseball event that would require moving the mound right in the middle of the Monday-Tuesday-Wednesday series with Baltimore.

Time for a moment of parsimony.  Don’t waste time looking for explanations like “the rubber/mound” moved for one park, when there are similar issues elsewhere. In addition to Yankee Stadium (which is calibrated now to match the incorrect calibration from last year, pushing pfx_x about 3 inches left) we have seen problems in Cincy, Philly and Detroit.

Plus, home plate moved at Wrigley after the initial install. Petco runs hot and with extra backspin. Texas is a foot high. And there are lots of subtle issues around. Toronto and Tampa stick out in my mind, too.

I’m cataloging the park effects I’ve found and will send them to Sportvision when I’m done. Rand already knew about Cincy, I mentioned it at the Summit last year, and they did adjust (multiple times).

Its interesting because the other day Dan Warthen talked about Johan giving up 9 runs at Yankee Stadium.  He said that he changed his grip on the ball to compensating for a blister problem he had been having.  Warthen said thats why his fastball is cutting and losing a few mph.  So I checked Pitch FX....saw Johan average x movement yesterday was 3.66, compared to his overall 2009 x movement of 6.40.  I concluded that he was indeed cutting the fastball.  But now I’m wondering if it isnt just Yankee stadium’s pitch fx figures just being miscalculated.

John 34, check Hardball Times tomorrow morning for my piece on it. You also have to split his two- and four-seam fastballs and match up some other pitchers .... you’ll see. I’m not done yet, but I think there’s a slight truth to what Warthen is saying.

Harry #33 - I don’t doubt that there are errors that are the result of calibration inaccuracies in the Pitch f/x system.  That may be the main or even the only cause of the park to park differences that you are noting.  The only reason that I mention that the rubber might not be in the same location in each park and/or may change loacation at different times is because I didn’t realize that the location was ALLOWED to vary under MLB rules, and the actual location would have to be known before any corrections to the data could be made either by Sportvision or by normalization by analysts.  Otherwise we would be in danger of over correcting.

If Ike’s idea that the release point errors are correlated with the pfx errors, then one could get a handle on the position of the pitching rubber.  There should be no sizable effect on pfx values if the pitching rubber is moved.

Harry/35 - I’ll have to check that out.

Also I was gonna wait cause I knew you had planned on doing a Pitch Classification Series at Beyond the Boxscore but I was wondering how do you separate 2 seamers and 4 seamers.  Are you using spin angle?  And if so, whats the cutoff point to separate the two pitches?

Peter, I think we are in agreement. As a general concept, I am concerned about correcting/removing meaningful variance.

I’m hoping, with some degree of confidence, we can reverse engineer, or better, retroactively apply Sportvision’s corrections to the data and then tease out actual effects.

John 38 - To split two- and four-seamers I use both pfx and spin axis. But mainly spin axis. Unfortunately, it varies by park (go figure) and even by pitcher.  220 degrees is a good rough marker to start from, or 140 for lefties. That’s an adjusted figure, subtracting the tilt caused by the pitcher’s arm angle, so you’re just dealing with the grip related spin, for lack of a better term.

Ahh thanks.
I was using 225 without any type of adjustment tho.

Another thing people may look for is a correlation between x0 (the horizontal release point) and vx0 and ax.  For example, if x0 is more negative, for whatever reason, then some combination of vx0 and ax must be more positive to compensate so that the ball gets over the plate.  All of x0, vx0, and ax are needed to determine pfx_x, since the first two determine the no-spin trajectory and ax determines the deviation from that trajectory.  If a pitcher purposely changed his horizontal release point, he would almost surely compensate by changing vx0 (in the opposite direction), leaving ax (and pfx_x) unchanged.  However, if the change in x0 is the result of a miscalibration, then it is likely that both vx0 and ax will change.  I look forward to reading Harry’s THT piece tomorrow to learn how he makes corrections.  I don’t really know how to do that myself.  In particular, if the x coordinate is wrong (due to a miscalibration), then I would expect y and z to be wrong also, since the cameras jumble up all three coordinates.

Alan 42 - Yep, you’re correct, you have to carefully tease out each component.

For tomorrow’s THT piece, I’m going to show something that’s exceedingly simplified, but I’m hoping it sparks some ideas and conversations around how to really attack the problem. Consider my approach to be pasta against the wall, we’ll see what sticks.