THE BOOK--Playing The Percentages In Baseball

I wonder if he’ll give me the 2004-2006 data he has…

Did he really say, “In the last five or six years?” Pitch f/x data started in 07, right?

And it is just the opposite of what he is saying! 

Because of the internet (and the widespread and easy availability of the data), and the number of analysts out there, almost as soon as data becomes available, it starts getting churned.

Ah, what the heck.  In another 10 years, I’ll probably be saying stuff like that…

PITCHf/x adds amount of movement, which we never had, and release points. 

Prior to that though, we had pitch speed (to some degree), we had pitch type (fastball, curve, etc to some degree), we had pitch location (to some general degree). 

PITCHf/x adds a consistent system, levels of precision, and gobs of data.

So, general pitch tracking has been around for as long as James is saying, if not even longer.

That said, to say that we don’t know what to do with it… it’s a strange thing to say or take on its face.  There’s something lost in the translation.

For that matter, when (if?) FieldFx data comes out with accurate hit and fielder location data, I know exactly what I’m going to do with it.  I’m not saying that the analysis of said data won’t continue to get better and better, but I’m not going to sit there for two or three years and scratch my head in bewilderment, that’s for sure…

I didn’t understand him to be saying that literally no one had a single idea what to do with it and we were all sitting around on our hands staring at the mountain of data in befuddlement.

I believe it’s fair to say I’ve looked at the PITCHf/x data as much as anyone has, and I still don’t get the feeling I’ve come close to fathoming its depths.  There’s a lot (or ALOT!) left in it that I don’t grasp yet.

When Bill James talks about not knowing what to do with the data, that’s how I understood it.  I pull on a thread, and it just keeps unraveling and unraveling, and I don’t think I’ve found the end of a single thread yet.  I’m nowhere close to having my arms around the data, as they say in the business world.

Now that’s a far cry from saying I haven’t learned anything yet, but that’s not what I heard Bill saying.

And I agree with him 100% that the challenge is only going to grow exponentially with the data from FIELDf/x.  The mountain of data we have from PITCHf/x is nothing compared to the planet-sized ball of data we are going to get from FIELDf/x.  THAT is going to be overwhelming.

Mike #5:

Well, I think FieldFx is (will be?) a different animal than PitchFx.  Sure, there will be some similarities, but the main difference is that many, perhaps most, of the parameters that FieldFx will track are going to be unequivocal, while many (most?) of the parameters PitchFx tracks are equivocal in their relationship to outcomes.

What I mean is this: with PitchFx, you can’t say unequivocally that higher pitch velocity is better: sometimes lower velocity gets the job done better in pitching.  You can’t say unequivocally that greater movement in a particular direction is better: sometimes less movement makes a hitter miss more often, if they are accustomed to more movement, for example.  Same with release points: you can’t just say that a particular release point is better.  Now, you might be able to determine that a consistent release point for different pitch types works better, I don’t know.  But, my point is, for PitchFx, a lot of the things it tracks are equivocal.

Now let’s consider FieldFx.  If it fulfills its promise, it ought to be able to give us such things as fielder reaction time, fielder speed, fielder positioning (maybe average distance from landing point, or a weighted average based on hit value, could be used to express how well a player is positioned), etc.  If we’re getting “GameFx”, we’d also have throw speed, release time (glove to release), throw accuracy, etc. 

If you described every one of these parameters as unequivocal, you’d be very, very close to the truth, I think.  Reaction time?  Faster is better.  Fielder speed?  Faster is better.  Positioning?  Closer to the optimal point is better (acknowledging some potential difference in how thsi optimal point is determined).  Throw speed?  Faster is better.  Release time?  Shorter is better.  Throw accuracy?  More accurate is better.

This is why I don’t think FieldFx/GameFx is going to be as much of a deep, dark mystery as PitchFx has been.  The surface information is going to be very accessible.  The deeper stuff such as new metrics for outfielder range, will probably come in otu in several forms before one emerges as the best method, but overall, I don’t think there will be such a learning curve as for PitchFx…

Greg, I agree with most of what you say in #6, and that is a good point and something I hadn’t thought much about. 

However, having had access to one game’s worth of FIELDf/x data, let me say that getting from the data to the quantities you mention is far from trivial.  I don’t anticipate that the data is going to be presented to the user with the quantities you mention.  Deriving them is going to be a huge task.  That’s where the mountain of data comes into play as an impediment, much moreso than it did/does with PITCHf/x.

It makes a big difference how well they are able to track the ball (in time and space), and that wasn’t implemented in the version I saw.  In fact, I imagine (without any inside knowledge) that is the largest technological hurdle they have to overcome before the system can go into full-scale testing/production.

If you know precisely (within 10 milliseconds and within 1 foot) where the ball is at all times, the task of parsing the data becomes much simpler.  However, I’ll believe that when I see it and not any sooner.  I’m very skeptical they can do that.

Mike, good point on the derivations from the raw position/time data.  As for the skepticism on the arrival of this glorious future, that’s why I keep adding “if?” whenever I find myself using the word “when” around FieldFx.  I think we’re on the same page as far as concern over whether this eventually gets done…

"nobody’s been able to analyze it yet.”

Mike, what someone says and what someone means can sometimes be two different things - in any case, his words are unambiguous.  Is BJ your brother-in-law?

I believe field f/x will make things easier as today’s defensive metrics have to make too many assumptions where data is missing. We expect those holes to be filled in with the new data.

MGL, don’t tell me you’re a Scalia fan.  Words have contexts; they are not unambiguous entities on their own.

The question right after the one about FIELDf/x was this:

CTR: Is our understanding of the game - and I see where so many people are trying to understand the game in new ways - do we understand the game any more than we did in 1906?

BJ: No. We don’t. Absolutely not. Our knowledge is much larger and our ignorance is much larger. Is one larger relative than the other?

I assumed he was saying exactly the same thing about PITCHf/x.  It’s increased our knowledge but also revealed the depths of our ignorance.

I also assume that Bill James is probably ignorant of the vast majority of PITCHf/x analysis that has been done.  I’m not sure that invalidates a philosophical statement about what a huge increase in available information does to our understanding of the game.

I’m hardly a Bill James apologist.  If I want cutting edge research, I come here, not to Bill James.  I don’t even subscribe to BJOnline because I don’t think it’s worth my time/money.  But he has been and continues to be entertaining and often has a unique perspective that can be helpful.  He’s wrong as often as he’s right, but I’m fine with that. 

I also try not to take things like “every” or “never” too literally in an interview.  I doubt the hyperbole was intended.  In a written article where the author has time to consider the best way to carefully construct a phrase to make a point, I take such assertions more seriously than from something like this.

For my part, I totally disagree with the part about how we don’t understand the game better now than “we” did in 1906.

I get that he’s making a point about how now we know enough to realize how much more knowledge there is to pursue, but come on.

Do we know more about the weather and climate now than we did in 1906?  Obviously yes, even though we have learned enough in the past 104 years to realize that there is some seriously complicated stuff going on in the center of a tornado, or in the ocean off of Peru, or in the spots on the surface of the sun…

I agree with Greg/12.  I pointed out that quote just as part of the context for his quote about FIELDf/x and PITCHf/x.

I’m sure I don’t agree with everything Bill James believes about the subject, nor do I think that’s totally clear from this single interview question.

I did think that he had an interesting point that when we get a vast new quantity of data, it takes us time as a community to begin to understand what to do with that.  Obviously the knowledge is not a step function from 0 to 100% at some given point in time.  But the arrival of the new data does not magically increase our understanding right away.  We have to wade in and get our feet wet and incrementally begin increasing our understanding, often by trial and error, and this process takes a few years.

PITCHf/x data first came online publicly in October 2006.  We were into midseason 2007 before we had any real grasp of what we were doing, and well into early 2008 before I’d say our knowledge began to accumulate significantly.  Even still, I’d say we’re at much less than 100% of what we can get from it.  I’d say closer to 50% than 0% but closer to 0% than 100%.  Not that those percentages mean anything, really.

He gave the example of the computerization of the data.  How long did it take before we got to B-Ref and Retrosheet and the Lahman database and things like that where we could really exploit the power of computers on large sets of data?

Yes, his “1906” statement is even more ridiculous.  He seems to like hyperbole more and more as he gets older (and wiser?).

I don’t like hyperbole.  I hate it.  I think it is insidious.  The entire debate about health care was hyperbole and I think it was and is terrible for the country.

You ARE a James apologist!  If you have to listen to someone speak and decide what he “meant” as opposed to what he said, what is the point?  That is the very definition “spin.” You get to decide what someone else thinks?  That is dangerous.  If someone wants to convey an idea it is their responsibility to convey it in a way that is clear and unambiguous.  If they do not, then they run the risk of people misinterpreting them, which can lead to all kinds of problems.

And, for what it is worth, while I don’t dislike everything about Scalia, I don’t much like him at all.  We do the best we can to interpret and understand the constitution.  They had to use succinct words to convey some very complex ideas.  To say that we should either be constitutional purists or not is a false dichotomy.  Try taking the Second Amendment and being a purist.  You can’t, because no one knows WHAT it means.  Now, there is legitimate debate over whether we should follow what we think the framers meant at the time or whether because things change radically in society whether we should mold or change their original meaning, but that is different from what we are talking about here…

Ultimately, the important question is how useful PITCHf/x analysis is—and can be—to big-league teams. That includes for the front office, players and coaching staff.

Bill James has worked with the Red Sox for a while. He’s at least somewhat aware of how much this data is employed. Consider THAT might be where he’s coming from.

We can look at flight paths and other plots all day, but we have not yet figured out how compelling this data is for actual decision-making. Mike appears to get this ... and I think his humility is important as we continue to search for how PITCHf/x can be useful.

Bill James gave much the same answer in an interview two years ago.  By his own admission, he doesn’t tend to keep up on others’ baseball research very well.

From http://freakonomics.blogs.nytimes.com/2008/04/01/bill-james-answers-all-your-baseball-questions/

“The pitch by pitch data — the pitch fx and similar data from Baseball Info Solutions — gives us dramatically better detail about what pitches pitchers are throwing how often and how effectively. It will take us twenty years to figure out what some of this stuff means, but it is clearly generating a lot of excitement.”

I’m done defending Bill’s thought process on this subject.  Apparently he does think the worst that people were perceiving him to have said.

“We’ve had these cameras pointed at pitchers for several years now and we haven’t really learned a damn thing that is useful,’’ James said. “We have tons of data but in terms of comparing one pitcher to another we’ve learned nothing. I’d suspect the same thing would be true with respect to fielding. We have accurate measures of fielders now. I don’t think they will be significantly more accurate by increasing the cameras, frankly.’’

http://sportsillustrated.cnn.com/2010/baseball/mlb/wires/04/01/2010.ap.bbo.the.stats.revolution.adv03.2030/

That’s just ignorant.

I can’t help but think he’s hurting the Sox by maintaining that attitude.  Fortunately for them, Tom Tippett isn’t stuck in the mud that way.

On the other hand, if he were saying “[Bill James hasn’t] learned a damn thing that is useful,” I’m sure that’s true, and if he were saying “[The Red Sox] haven’t really learned a damn thing that is useful, that’s probably an exaggeration but maybe not too far from true.  But there are other teams who have learned many useful things from the PITCHf/x and HITf/x data, and many, many in the public sabermetric community who have learned a lot of useful things.

So I think in some ways his comment can be turned around as him simply saying that he personally and the Red Sox as a team are ignorant.  If that’s what he’s claiming, I won’t choose to argue with him.  Otherwise, he really is trying desperately to plant his tombstone as a useful analyst.

What I think we’ve learned about James over the past decade, unfortunately, is that for him the statements “we’ve learned nothing” and “I’ve learned nothing” are synonyms.  If he hasn’t learned it, from his own research, the knowledge simply doesn’t exist.

Mike - It all comes down to how one defines significantly, doesn’t it?  If you believe that today’s best defensive metrics are able to estimate a player’s defensive value to within +-7 runs a year and Field f/x can get you to +-3 runs per year you may or may not think that extra 4 runs is “significant”.  I think that it will be useful to have the best data possible and that Field f/x or some competing technology will give us that eventually.  But I am not necessarily convinced that there will be a “revolutionary” reordering of players’ defensive ability.  The other possibility, of course, is that James is privy to a proprietary system that Boston already has in place that is already a step above the public metrics that are available.

As far as Pitch f/x is concerned, I admire some of the research that independent analysts have done, and apparently so do some teams as they have hired many of the better analysts.  But I think that much of the analyses fall into the category of confirmations and explanations of things that we were pretty sure were happening but didn’t previously have the data to prove were true.  There has been relatively little information that would actually help a team to make better decisions about their pitchers on a contractual, coaching or strategic level.  I don’t think this is an indictment of Pitch f/x or the people who have been doing research using Pitch f/x data, I think that many of the most interesting questions that Pitch f/x will eventually help us answer just need a larger body of data over a longer period of time than we presently have.

One person’s hyperbole is another person’s ignorance…

That being said, James is creating quite the fine line between the two…

Mike - It all comes down to how one defines significantly, doesn’t it?

Yes, it does, Peter.  But I’m not one who thinks all the significance is wrapped up in how sabermetrics can value a free agent.  I think there’s a lot of value that the Morgan Ensbergs of the world can take away about how to play the game smarter.  Does that ultimately end up in the win column, too?  I suppose so.  But I don’t intend to measure my success as a saberist by that.

I’m far more fascinated by being able to quantify the jump that different outfielders get on balls, or whether backspin/topspin/sidespin makes fly balls harder to fielder, or what affects how the umpire calls balls and strikes. 

In some ways it’s hard to have this conversation because almost all I know about what various teams are doing or not doing with the data is from off-the-record conversations that I consider confidential.  I have ideas about what may lead Mr. James to some of his conclusions, but I have to leave it there.

I don’t think that most teams in general (with a few exceptions, again mostly obvious by who has hired the PITCHf/x experts) are yet making nearly as much of the new data as we are in the public domain.  Just because teams aren’t using the data, though, doesn’t mean that it’s not “useful”.