THE BOOK--Playing The Percentages In Baseball

The attendees at the Pitch f/x conference were made aware that MLB was negotiating with the umpires to replace Questec with Pitch f/x.  The sensitiviy of the negotiations was why we were asked not to publish articles using Pitch f/x to rate umpires.  Umpires have made minor improvements in their game calling under Questec and are already at a very high standard of calling balls and strikes.  I don’t expect anything will change much under Pitch f/x.

“A lot of people are of the impression that a particular umpire not in a QuesTec park would run wild and become an outlaw,” Port said. “But there is virtually no difference. And that was the purpose of the system.”

Talk about dumb quotes!  That is the purpose of the system?  To make sure that umpires call balls and strikes the same in parks with and without Questec?  Huh?

The whole thing is a joke.  First of all, to let an employee have a say (within reason) in whether and how they get evaluated is ridiculous.  Employees don’t want to get evaluated, period.

The other thing is that it is pretty clear from watching games that no one has been monitoring Questec for a long time.  And if they are, no one is telling the umpires what size strike zone to use. There are clearly large differences among umpires as far as the overall size and shape of their strike zone.  Yes, it is true that most umpires are fairly consistent within their own strike zone.  But MLB is not using Questec to make sure that all umpires have around the same shape and dimension of strike zone.  They are also not using it to make umpires call the rule book strike zone. The strike zone has risen a little since 5, 10 and 20 years ago, although that is the one thing that varies a lot with umpires - the height of the zone.  But on the average it is still nowhere near the rule book zone - height-wise.  Which I have no problem with, BTW…

MGL had a great Questec article a few years ago:
http://tangotiger.net/mgl/

Pitchers don’t like seeing umpiring normalized because they have invested time to learn the strike zone of each umpire.

I don’t really understand why it’s an issue to have all of umpiring reduced to computers.  How do you think they measure the distance from home to first and make sure its exactly 90 feet?  Do they eyeball it?  Then why shouldn’t the dimensions of the strike zone be as specific as the dimensions of the diamond?

Rob Neyer thinks that the replacement-level for umpires is pretty low:
http://sports.espn.go.com/espn/blog/index?entryID=4034138&name=Neyer_Rob

I don’t think that’s correct.  In hockey, there are two types of officials: referees and linesmen (i.e., doctors and nurses).

The bottom refs make as much as the top linesmen.  Why would a linesman be a career linesman, when he’s just one step away from being a referee?

As one person put it: .  Refs have b@lls and linesmen don’t.  That’s how it was described.

I don’t know if that’s accurate, but it sure seems valid.

I think the same applies here.  Yes, anyone can be an ump with training.  I think though that the pressure to perform well with so much riding on each decision must be a huge stress-maker.

I pretty much agree with Rob that MLB umpiring is not thatdifficult. I would venture a guess that if they replaced all MLB umpires with those from AA, AAA or even college that hardly anyone would notice or care.  The average fan cannot name one umpires and barely gives them a thought unless they blow a call or two, which happens to everyone.

Rob’s main point is 100% true I think - that the umpires union has virtually no leverage in that MLB realizes that it would not be all that difficult or detrimental to MLB if they had to replace all the umpires.

His other point which is the same thing I said in my last post is that it is ridiculous to think that you need the permission of the umpire’s union in order to evaluate the umpires.  No one wants to be evaluated - especially ego-driven, old-school people like MLB umpires.  Their talk about these systems being “not accurate” is merely smokescreen for “How dare you evaluate me...”

As Peter Jensen said (#1), QuesTec is being replaced by PITChf/x, and I think the change will take place this year.  PFX offers several distinct advantages over QuesTec.  First, it is installed in every ballpark.  Second, it is a completely transparent system, as we all know.  The Sportvision people have been incredibly open with the baseball analyst community (e.g., people who contribute to this blog) about how the system works and how well it works.  Many people have had an opportunity to evaluate the data and (correct me if I am wrong), the consensus seems to be that it works very well, minor issues aside.

Long ago, perhaps in 2001, I tried to obtain data from QuesTec, which I recognized as being a excellent source of data on baseball aerodynamics.  The company was completely uncooperative with me.  The very first time I asked Sportvision for data, they sent some to me, and this was before the data were available to the public.  So, good riddance to QuesTec.

A quote from the Schwarz NYT article:

But, umpires have pointed out, the accuracy of the system suffers once a pitch enters the strike zone — because the zone hovers above the five-sided plate as more of a three-dimensional prism, not the rectangle that television viewers see. They have maintained that although QuesTec (like Zone Evaluation) collects data in three dimensions, a hitter’s position in the batter’s box or distractions like bat movement can cloud the information, making it unfit for evaluative decisions about umpires.

There is no problem of principle for the Zone Evaluation (ZE) system (essentially pitchf/x) to deal with a five-sided plate.  Unlike QuesTec, the ZE system tracks the entire flight of the ball, sampling points between mound and home plate.  As we all know, the measured points are fitted to a simple function to get the full trajectory.  It is not necessary to actually measure points right at home plate since the trajectory can be easily extrapolated into the home plate region.  So the question about interference from the batter or bat movement is simply not an issue.  I am not sure whether the same was true of QuesTec, which I think only measured points in the home plate region.  There is a real advantage to tracking the full pitch, since it lessens the sensitivity to distractions in the home plate region.  And it is a very straightforward matter to figure out if the ball passes through the 3d-five sided volume that defines the strike zone.  Having said that, I don’t know if ZE will actually do that.  I do know that it is easy to do.

Just to be accurate, Sportvision stops its sampling between 8 and 15 away from home plate so the extrapolation of the flight of the ball is greater than Alan implied.  However, as Alan also noted, the extrapolation to the home plate area is still very accurate as there are no additional forces acting on the ball to alter its path.

Umpires must hate their reports when Tim Wakefield is pitching.

Please send me all your questions regarding calibration of the strike zone, and I will try to get you answers from an insider.

Two things:
1) A “+1” to echo what Alan said above, that Sportvision is really a fantastic company in terms of openness, etc. They really are a good bunch of people.

2) Does this mean that the “embargo” on strikezone related info is over? I’ve been rather quietly posting strikezone maps for each game, and I know there are a bunch of analyses of aggregate zones that happened last year. But, I was joking with a friend last year that “I provide something like 100 different ways to break down a start… and the graph that gets 100x more pageviews than anything else is the half-hidden strikezone plot.”

If it’s suddenly become “okay” to publish, I will make it more prominent. Because fans obviously want to see it.

I think as long as you present things in an honest and objective manner, I don’t see how anyone can take issue with it.

***

Dan Fox’s article was here:
http://danagonistes.blogspot.com/2007/08/umpires-and-questec.html

Another good read is here:

http://bucco-blog.com/Pittsburgh-Pirates/2009/05/29/mlb-umpires-when-a-strike-is-called-a-ball/

That is an interesting article with interesting data, which kind of flew under the radar.  It is probably worthy of its own thread.  Nice job by the author in compiling the data.

(Does that mean that bloggers are allowed to publish umpire data now, or is it just that “the public” is under no obligation one way or another?)

Anyway, a few thoughts came to mind:

Are the differences among pitchers and teams even statistically significant?  I have no idea.  It could all or mostly be noise, but certainly some kind of statistical test has to be done before we can even begin to discuss the “whys and hows.”

If most or some of these differences are “real” there are many reasons why, other than umpires “squeezing” or not certain pitchers or teams, some of them discussed by the author.  Some of these are “stringer biases” from park to park, the types of pitches that each pitcher throws (for example, an umpire is MUCH more likely to “miss” a curve ball or slider than a straighter pitch like a fastball or change up).  Park biases having to do with the lighting and things like that.  Umpires biases - IOW, a pitcher may have happened to have a large or small proportion of pitchers or hitters umpires in general.  The catcher, as he mentions.  Some catchers are better than others at framing pitches.  There are probably more.

I track umpires balls to called strike ratios every year.  His list is right on the money as compared to mine.  Other than Cooper, his top 10 “pitchers’ umpires” are all on my pitchers’ umpires list.  Pretty much the same thing with the bottom of his list, the hitters’ umpires, although there are more on his bottom 10 that are not on my list (of hitters’ umpires) at all.

No one has put me in MLB umpire jail for this article yet:

http://www.beyondtheboxscore.com/2009/4/20/846320/using-pitch-f-x-to-determine-an

I plan on doing heat maps for each umpire as time comes up.