THE BOOK--Playing The Percentages In Baseball

I am trying to wrap my head around one thing you said.  But any single event (save or goal) is luck.

I agree, but only to a degree.  Talent plays a significant role in the one event to.  It isn’t all luck that I can’t save a goal but a professional can.

If we talk about a pool of professional goalies all trying to save 1 goal in 1 chance, that has a lot more luck.  But i wonder 200 professional goalies all get 1 chance.  Will there be a higher percentage of the ‘better’ professionals saving more goals than the lower end professionals?  I think so.

So how is it all luck?

Lou, when you say “Will there be a higher percentage of the ‘better’ professionals saving more goals than the lower end professionals?  I think so.” this is exactly what TangoTiger is talking about when he says that everything is luck about the true mean - better goalies will have a better true mean save percentage. This true mean is different for different players and different types of situations, and it’s the same thing that is referrenced when we talk about regression. Like in The Book where they talk about players regressing to the mean, but that doesn’t necessarily mean league average. Finding that true mean is generally the problematic part of things.
Of course there are some philosophical/existential problems that can be raised - there are certian things that will happen with a frequency of 1 or 0, at least in certain reckonings. But generally this is a problem of how narrowly you define an event. For instance, you can say that given that it’s 2010, the probability that the Blackhawks win the cup is 1. But then you aren’t saying anything useful.

Luck is a word used by those who presume to have insight into events and, paradoxically, to claim there is no need to understand the cause of those same events, since it is simply due to luck.

The unexpected occurrence of an event, good or bad, assuming one has a reasonable basis for expectation, is probably the best definition of luck.  But this “surprise” may have more to do with our lack of understanding of that which is the basis for our expectations.

From Jaynes-The Logic of Science

“ Quantum physicists have only probability laws because for two generations we have been indoctrinated not to believe in causes, and so we have stopped looking for them. Indeed, any attempt to search for the causes of microphenomena is met with scorn and a charge of professional incompetence and ‘obsolete mechanistic materialism’.

....In current quantum theory, probabilities express our own ignorance due to our failure to search for the real causes of physical phenomena, and worse, our failure even to think seriously about the problem. This ignorance may be unavoidable in practice, but in our present state of knowledge we do not know whether it is unavoidable in principle; the central “dogma” simply asserts this, and draws the conclusion that belief in causes, and searching for them, is philosophically naive...... 

If everybody accepted this and abided by it, no further advances in understanding of physical law
would ever be made; indeed, no such advance has been made since the 1927 Solvay Congress in
which this mentality became solidied into physics. But it seems to us that this attitude places a premium on stupidity; to lack the ingenuity to think of a rational physical explanation is to support the supernatural view.”

Lou/1: WW/2 answered it.  I said this:

“but predicated on the true mean”

The true mean is specific for some point in time and space.

Without getting into the quantum aspects of this, I’m not sure I agree from an everyday level that, given perfect knowledge of all involved factors, a macro event’s frequency will still revolve around it’s true mean.

It seems to me that the ‘revolving around the true mean’ is noted precisely because we *don’t* know all the involved factors. Given perfect knowledge of that (which was what Tango specified) why wouldn’t the outcome always occur at the same frequency--the true mean?

Thanks WanderingWinder and Tango.

Dave: because any single event either does, or does not, happen.  If Shaq were to ALWAYS maintain the same free throw skill at all times (say it’s 50%), then his next free throw he either does, or does not, make it.  Whether he does or not is luck… centered around his true mean.

Even if he throws 2 million shots, he won’t get exactly 1 million baskets.

But Tango, you are choosing to call luck the very real things which cause Shaq to either make or miss any particular free throw. It’s not as if he is an automaton, shooting exactly the same way every time, and some god of fate intervenes and says “in” or “out”. There are physical factors involved which vary with each attempt. These variables are what I assumed you were referring to as “full information”. I guess you meant only his result over an infinite number of trial as “full information”.

If Shaq misses a free throw because he spins the ball 5% too fast, that’s not luck, that’s simply an unmeasured variable.

pft/3:
The statement from Jaynes you cite is incorrect.  Work is ongoing into the fundamentals of quantum mechanics.  The slow progress is due to:
(1) The difficulty of the problem.  (For that matter, the difficulty of even _stating_ the problem in a non-circular manner.)
(2) The apparent lack of practical consequences.  (Quantum mechanics is usually pretty clear about _what_ it predicts, which makes designing experiments to find out _why_ both difficult and presumably unrewarding.)
(3) The abundance of other, easier targets.
(4) Lack of funding caused by #1-3.  The only person I could name working on the problem is also working on a dozen other projects.  He had a large research group; but he refused to take students to work on the fundamentals, because he would not be able to support them appropriately.

Dave/8
“Luck” is a working hypothesis.  There are advantages to assuming random variation about a mean.  This hypothesis provides a robust prediction about future behavior.  The prediction is also falsifiable.

What you are proposing is a hidden variable hypothesis, that there is something other than a baseline skill and a random factor that influences the result of an individual event.  If your hypothesis is purely phenomenological (e.g.: odd-numbered free throws are better than even ones), you need to _vastly_ outperform the luck hypothesis.  If your hypothesis has a physical mechanism behind it (e.g.: free throws are less accurate late in the game when players are tired), you can beat the luck hypothesis by only a fair amount.

Somehow, I feel like we’ve covered this all in great detail before.  Now where could that have been?

David, I don’t think I was clear enough, or perhaps I may have confused you.  I will repeat this (with the bold as added emphasis):

To further add: the “true mean” is based on everything we know, and don’t know, about the environment.  That is, god herself told you the odds of something happening based on the properties of each entity. 

The question then is, with *full* information, is the true mean for a single attempt anything but 1 or 0? I think no: once you have full information from God about that particular free throw, then the true mean for that attempt is either 1 and it’s going to go in, or 0 and it’s going to miss.

And outside of individual attempts, a true mean does not exist, in the sense that we’re using it here of an unchangeable, fixed number.

I said I’m not talking about fate.

With full information, you still can’t predict a person’s choice.  If you are talking about “full” information as including what a person is going to do and how he’s going to do, such that all that is left is a physics problem, then you are talking about fate, the 1 or 0.

But, I’m talking about the point(s) prior to the decision points, where there’s at least one variable in play.

So, if we are talking about free throws, it would have to be (no later than) just prior to the ball being released, such that we don’t know how or where the final finger contact is made.

Again, no fate.  You can’t predict the future.  As long as you can’t predict the future, then the result has no choice but to be random around some true mean (a true mean that will change for any time-space point).

--"As long as you can’t predict the future, then the result has no choice but to be random around some true mean (a true mean that will change for any time-space point).”
*****

So, for me. there is nothing new here, it just boils down to semantics, and maybe philosophy. And that is why I reacted to the initial post the way I did, as though Tango was answering a question as a saber expert, when in reality there’s much more going on than that, and it’s not quite so cut and dried. A bit less certainty would have been more called for, IMO.

Re-reading my post, there’s nothing I would change.  I could have expanded it for clarity, but not for anything else.

Dave/15, I’m with you.  I’m in complete disagreement with Tango on this subject, but we’ve discussed in such great detail here before, I’m not sure I have anything to add to what I’ve said before.

/14/ seems like a really weird stance to take.

Presuming you know everything detail about arm and hand movement, then you can obviously determine whether a free throw will go in or not with newtonian physics. In most cases, that last little finger touch you’re talking about as some sort of decision point, won’t even matter and so while your understandings is partly probabilistic, it’s also partly deterministic. If, on the other hand, you want to move your “full information” point be when he steps to the line, then you’d run afoul of the idea that you have full information.
A more rational view, which accords better with modern philosophy and physics is that you cannot have full information—not that with full information there’s no way to determine subsequent state.

I don’t know if this helps but we can not know the true mean and therefore in practice we can not actually figure out if deviation from a prediction happened because of luck or a shift in the true talent (odds of the event happening).

In theory I think everybody understands luck when it relates to predicting the the outcome of a dice roll.  In practice for baseball its tougher because the numbers on the dice may be changing as a function of time and we can’t know what they are for certain.  That’s why whenever a player is struggling there is a constant argument between bad luck or a shift in true talent.

I think this has all been said before but maybe this will help a little more clear up confusion.

Presuming you know everything detail about arm and hand movement, then you can obviously determine whether a free throw will go in or not with newtonian physics.

No, that’s not true.  That can only be true, maybe, for a machine.  Just because you know how something behaves independently, you won’t know how something interacts.

You can know everything about Shaq throwing motion, but you won’t know how he’s going to put it together at any point in time-space.  Even at the very last moment, you still won’t know how much pressure he’ll put, how he’ll lift himself up, how everything.

From the moment he steps up to the line to the instant to just before the ball is released his true mean changes all the time, and this is based on us knowing everything about him.

At no point while the ball is in his hand will the chance of him making or not the basket ever, ever, be 1 or 0.

You guys can disagree with me all you like.

At no point while the ball is in his hand will the chance of him making or not the basket ever, ever, be 1 or 0.

While I consider this a true statement, I don’t think defining luck in this manner is very productive way of approaching the problem.  What you are calling luck I would just call a lack of consistancy.  A player’s “true talent” is not just the mean of the outcomes, it also includes the standard deviation of the attempts.  In the case of Shaq’s free throw shooting, Shaq’s true talent is not an ability to make his free throw’s at a 60% rate, it is an ability to shoot a free throw to the point at which he is aiming at some much lower rate with a standard deviation of say +- 4 inches.

You guys can disagree with me all you like.

This seems uncharacteristically closed minded of you.  Yes, by your personal definition of luck the statement will always be true, but it pretty much ends your participation in the discussion of whether that is the best definition of luck.

A player’s “true talent” is not just the mean of the outcomes, it also includes the standard deviation of the attempts.

A player’s true talent is the mean of the outcomes if he were to repeat an infinite number of times exactly all his steps up to that particular point in time-space.

So, Shaq steps to the line on April 1, 2011, at 20:45:43.324578.  His chance of making the basket is 52.3%.

He bounces the ball twice, and now the ball is in a new position in his hand.  His chance is now 54.8%.

He squats down, and now his chance if at 48.9%.  He extends his arm, and lifts himself, and his chance is now 58.4%.

He flicks his wrist, and the ball is still on his fingers, and his chance is now 68.4%.

He moves slightly, and at the nanosecond prior to release, his chance is now 32.9%.

The ball is now in the air.  Now, it’s a physics problem.  The chance is now 100%. 

In all steps while the ball is still in his hand, the chance of him making the basket or not is luck (but centered around the mean at that point in time-space).  It’s like flipping the weighted coin.  Shaq controls the mean of the coin.  He doesn’t control the final outcome.

I’ve gotten into the philosophical arguments a lot in the past, but here’s the thing: it doesn’t really make much difference which interpretation we choose (unless we are in the limiting case).  We can interpret randomness as either representing our own ignorance of the deep underlying process *or* in the more quantum, “true random” sense (or both!).  In either case, we have finite sample data from an unknown distribution; the key is that there is uncertainty in results.  We can apply statistical techniques to that data to try to better understand past results and possibly to predict future results.  The philosophical interpretation of the uncertainty doesn’t make much difference in doing this as long as we are at least a little bit removed from the process (so there are still several sources of uncertainty).

Yes Tango, I know that is your way of looking at the problem.  But its not the only way, and in my opinion as I said above, not the best way.

Tango/22, on its face, that’s an absurd position to take.  What happened in the nanosecond before release that took the chances from 32.9% to either 0% or 100%?  And who controlled that?  Obviously it was something that Shaq controlled, except for the one time in a billion that a ceiling tile fell from from the roof and struck the ball in flight, or something like that.  I won’t be as kind as Peter here.  Your argument is flat-out wrong.

Moreover, it’s irrelevant.

The relevant question is what can we measure that will improve our estimate from the 52.3% that we have for Shaq’s long-term mean, and how much difference does that make in our estimate?  What measurements could we potentially make, and how much would those improve our knowledge?

Baseball becomes more complex than free throws with, in the simplest model, two players--pitcher and batter--involved.  The opportunity to improve our measurement over the baseline “Pujols is a .420 wOBA” is immense.

There is a very strong current in “sabermetrics” to call anything different than a player performing at his Marcel (or ZiPS, CHONE, PECOTA, etc.) as luck and to belittle attempts to understand what caused things to happen at a finer-grained level as equivalent to and as far-fetched as trying to determine the effect of what Pujols had for breakfast.  That line of thinking leaves us intellectually poorer in sabermetrics, and you are encouraging it.

Does the opposite error happen, i.e., the one described by the XKCD comic that you added to the sidebar, of commentators finding patterns and narratives where they don’t exist?

Of course it does.  There is plenty of error made in that direction, too, though I would argue that the vast majority of that is outside the sabermetric community and thus mostly out of the reach of this blog.

However, I find that the sort of stubborn “luck is what doesn’t match my simplified model” thinking as expressed in this thread actually spawns a lot of resistance to sabermetric thinking in the broader baseball world, and rightly so.

Acknowledging the limitations of our models (1) helps us discover better and more powerful models and (2) encourages outsiders to believe that we are being intellectually honest rather than making them think we are twisting numbers to support our pre-ordained point of view.

"What happened in the nanosecond before release that took the chances from 32.9% to either 0% or 100%?  And who controlled that?  Obviously it was something that Shaq controlled,...”

This is really an issue of semantics, which is a large part of why these discussions never get anywhere.  From your point of view, obviously it was Shaq who was in control; he was still in contact with the basketball so he could exert forces in different ways.  From my point of view (and I’m guessing Tango’s), Shaq is not perceiving himself as doing anything differently here in that last nanosecond to change from 32.9% to either 100% or 0%.  Sure when we measure microscopic differences at the nanosecond time scale we will observe some things that Shaq “controlled” in different ways.  However, I suppose I would argue that Shaq was not in “control” of the subtle differences caused by Shaq’s reflexes taking an extra nanosecond (please don’t take this to say that I don’t think athletes can practice to make their skills more repeatable).  It’s all boils down to semantics, parsing, and what factors we want to account for.

Mickeyg13/27, I mostly agree with what you said, and that’s why I said the more important questions revolve around figuring out the limitations of our models and how much improvement we can obtain in exchange for added complexity. 

When we’re talking about the plate-appearance level of analysis, rather than the season-level, and comparing the wOBA binomial model to finer-grained models, I’m pretty sure that answer is that added complexity will buy us a lot.  We’ll be much closer to 1 or 0 than we are to where the season-level model (Marcel, etc.) started us.

Moreover, though the question about what the player controls, whether consciously in the moment or unconsciously but through practice and coaching, interests me, the much more tangible question for an analyst is about what we can measure.

Let me state my position more clearly:

The model that uses season/career-level “true talent” wOBA of batter and pitcher (current Marcels, etc.) to predict the outcome of a single plate appearance is a deeply flawed model.

There are many things that we can measure that will greatly improve our ability to understand the outcome of an individual plate appearance over what the aforementioned model tells us.  “Luck” is a very misleading label for those things.

There is a very strong current in “sabermetrics” to call anything different than a player performing at his Marcel (or ZiPS, CHONE, PECOTA, etc.) as luck and to belittle attempts to understand what caused things to happen at a finer-grained level as equivalent to and as far-fetched as trying to determine the effect of what Pujols had for breakfast....

Somehow, you went from that (and I agree that it is NOT luck when that happens) ...

That line of thinking leaves us intellectually poorer in sabermetrics, and you are encouraging it.

... to this.  Those two things have nothing to do with each other, and in no way at all do I accept whatsoever that I am encouraging such behaviour.

I reject your conclusion.

You can argue that I’m doing a poor job of explaining myself, which is a perfectly defensible position.  Like “Tango, your argument is really confusing, and just makes no sense to me.” The best you can say is that you don’t understand what I am saying.  You can’t say you understand what I am saying, and then come to that conclusion.

***

What happened in the nanosecond before release that took the chances from 32.9% to either 0% or 100%?  And who controlled that?

Luck!

To argue against luck is to accept fate.  There’s really only two choices.  Either there’s a non-zero chance anytime an entity is involved in something, and so when there’s a manifestation of something (0 or 1), it’s luck that it happens.  Or, there’s a predetermined chance of something happening (0 or 1), in which case it’s fate.

***

However, I find that the sort of stubborn “luck is what doesn’t match my simplified model” thinking as expressed in this thread actually spawns a lot of resistance to sabermetric thinking in the broader baseball world, and rightly so.

That’s not at all what I am saying.  I am saying that if you know absolutely everything there is to know about the behaviour and property of every single entity in question, that god herself told you all that, then when those entities collide with each other to produce an event (result of 0 or 1), that that result is due only to luck.

***

Acknowledging the limitations of our models (1) helps us discover better and more powerful models and (2) encourages outsiders to believe that we are being intellectually honest rather than making them think we are twisting numbers to support our pre-ordained point of view.

I don’t know who this is addressed to, but I hope it’s not me.

***

“Luck” is a very misleading label for those things.

Luck is a completely wrong label for that.  So, I agree with you.

I get the feeling we agree, but there’s something you think I am saying that you think we are disagreeing on.

In my scenario, Shaq’s true talent level hovered between 32% and 68% in the matter of less than ONE second.  And that had NOTHING to do with luck.  Nothing at all.  That was his true talent level at those moments in time.

Indeed, in reading your last post, I am surprised you are disagreeing with me as much as you are.

***

To put it another way, there are three different things to think about:
1. Our estimate of the true mean at a point in time-space

2. God’s complete knowledge of the true mean at a point in time-space

3. The manifestation of an event at some point after the various entities collide at a point in time-space

When you go from 1 to 2, that’s the uncertainty level of the true mean going from something to zero.  That has nothing to do with luck.  It’s an uncertainty level.  Like my Shaq example, his true mean is 52%, give or take (between 32% to 68%).

When you go from 2 to 3, that’s luck.  Something has to happen.  Shaq won’t get 0.32 baskets or 0.68 baskets or whatnot.  He has to get to 0 or 1.  That’s either through luck or fate.

When we look at the results of a single plate appearance, we may have two questions:

1. What caused the outcome?
2. To what extent do those causes have predictive power for other plate appearances?

You can extend the same questions up from the plate appearance level to other small samples that are still well short of the season level (or other large sample, depending on the stat).

If the answer to question #2 appears to the analyst to be “none, or insignificantly little”, it is very common to see the analyst say that the answer to question #1 is “we don’t care” or “it was luck.”

However, I believe that question #1 has to be answered before question #2 can be answered, and the better you answer question #1, the better prepared you will be to answer question #2.  It’s very easy to make mistakes about question #2 if you’ve done a shoddy job of answering question #1 (or if the data you had available for tackling question #1 was poor).

When it comes to a single plate appearance, and by that, I mean a plate appearance whose participants are in no way linked to other plate appearances, then our estimate of the true mean will have a pretty wide range.  We won’t be able to distinguish Pujols from Ryan Howard from any strong minor leaguer.  We won’t be able to distinguish any pitcher.  All we can do is look at what we can see prior to that plate appearance, and make a decision as the true mean of the outcome, and we’ll come out with something like a .200 to .500 OBP.

After that PA occurs, you can include how the batter approached the PA, how the pitcher attacked the batter, and maybe you can reduce your estimate, based on a single PA, to .225 to .475 OBP or something.

Now, if you have 200 PA, you get to reduce the uncertainty of your mean estimate far more.  You rely not only on outcome, but approach and scouting.  Whatever you can record.

None of that has to do with luck.  That’s the uncertainty level of your true mean estimate.

The manifestation of the behaviour and property of all the entities involved into a single event, where something either did or did not happen, that’s luck.  Luck centered around a true mean, or luck centered around an estimate of a true mean (with an uncertainty level around that estimate).

So, I really don’t know what we are disagreeing about.

Tango/30, most of what I said was directed at you, and I don’t see how I’m misunderstanding you.  We hashed this out pretty thoroughly previously.  Then, you said this:

That’s not totally true.  If Pujols was a “True” .440 OBP guy, and always was .440 and never wavered from .440, we would get a certain distribution.  One explained by the binomial distribution.

Now, there are other forces at work in reality.  The batter himself is a human being.  The conditions he faces are not static.  This adds to the spread we would expect from a pure binomial.

And what do we actually see?  Well, we see something that is almost a perfect binomial, as if he were a pure .440 unchanging as a player and facing static conditions.

Not exactly of course.  But close enough that, for all intents and purposes, that we can get away with it.  To treat him as a true human with all the extra conditions does nothing but slow us down considerably in the analysis.  WE can get 99% of the way there by using the binomial.

The theoretical objection is fine.  But practically speaking, it’s an almost non-issue.

--edited out link to previous thread because I’m having trouble getting this post to submit--

You claimed that binomial model with a unchanging prior (i.e., Shaq at 52% free throws, Pujols at .440 wOBA) tells us 99% of what we can know about an individual plate appearance.

I don’t believe that’s anywhere close to correct.

If you’re allowing now that the binomial model based upon batter and pitcher true talents determined from large samples is woefully inadequate at the single PA, and that we need to make substantial adjustments to it (or discard it altogether) in order to understand the causes of the result of an individual PA, then I’m with you.

To put it another way, there are three different things to think about:
1. Our estimate of the true mean at a point in time-space

2. God’s complete knowledge of the true mean at a point in time-space

3. The manifestation of an event at some point after the various entities collide at a point in time-space

I believe that the gap between 2 and 3, by definition, is basically zero.  You say it’s either fate or luck.  Well, sure, but you’re lumping player choice in with fate, in which case I’ll say it’s mostly fate.  But we’re probably arguing semantics on that point, and I don’t think that’s profitable.

I’m far more interested in how big you think the gap is between different types of knowledge #1.  How close can we get to true knowledge of what would have happened?  Whether that’s your knowledge level #2 or #3, I don’t know--to me they are the same thing, but for sake of argument, let’s say that knowledge #2 is somehow removed from #3.  In that case, do you think the binomial wOBA model with seasonal (or multi-seasonal) inputs gets us very close to knowledge level #2?  And do other models, with data we can humanly obtain, move us much closer to knowledge level #2?  My answers to those questions are “no”, and “yes”.

It seems to me that it’s a very common sabermetric practice, including yours, to label anything between knowledge level #1 as provided by the binomial wOBA model with seasonal (or multi-seasonal) inputs and knowledge level #3 as “luck”.  That would indicate to me that either you think the difference between knowledge level #1 as provided by the binomial wOBA model with seasonal (or multi-seasonal) inputs and knowledge level #2 is small, or that you are generally using “luck” more expansively than in post #30 here.

Imagine a black box, unadorned except for a button and an LED mounted on the box.

With some time on your hands, you push the button, and the light comes on.  You push it again, it stays dark.  Push again a number of times, and you get a mixture of light and no light results.  Do this for a thousand trials, and you get 310 lights, and 690 no lights.  You might conclude that there is a 31% chance of the light coming on every time you push the button, and go on to do something else. 

However, another person might come by, and start doing things like pushing the button at specified intervals, or pushing it with varying degrees of force, or pushing it with different fingers or toes, or standing very close to it when you push vs. as far away as you can reach, etc.  This person might get different results than 31% lights - perhaps a statistically significant difference that suggests that the button/light relationship isn’t as simple as it seems.

Person #1 might argue that light vs. no light is a matter of luck, while person #2 might argue that there are poorly understood, or even unsuspected factors that determine the result.

Who’s right?

The situation under discussion here is similar to this scenario, with one major difference: in baseball, we KNOW there are other factors that matter, and matter profoundly.  The pitcher/hitter confrontation is not governed by a random number fed into a binomial distribution.

Now, if we are not able to completely figure out the transfer function for the pitcher/hitter confrontation (and that’s the case), it may serve us well to *model* that confrontation as random numbers plucked from a binomial distribution.  However, we have to remember that this is a simplification.

The “perfect knowledge” angle holds no interest for me.  Maybe quantum theory holds that the up/down spin of electrons can exert a macroscopic influence on a confrontation between two human beings each made up of gumbillions of atoms, but I don’t buy it.  Physics theory also says that there is a non-zero likelihood of a pitched baseball flying right through the catcher without touching him, but why waste time thinking about it that way?

You claimed that binomial model with a unchanging prior (i.e., Shaq at 52% free throws, Pujols at .440 wOBA) tells us 99% of what we can know about an individual plate appearance.

I don’t believe that’s anywhere close to correct.

You would use the Odds Ratio method in the case of looking at a man v man scenario, but otherwise, yes, we get most of the way there by presuming that Pujols is a fixed entity.  Our uncertainty level of Pujols is something like a .440 wOBA +/- 1 SD = .020 or something, but there’s very very little we can do to reduce that level of uncertainty.

You however seem to be arguing that perhaps we can reduce our level of uncertainty of Pujols down to 1 SD = .010 or .005 or perhaps even less?  And so, rather than him being .440 +/- .020, you might be able to get him down to .430 +/- .005, if you know even more about him (how he woke up, how he’s feeling, how batting practice went, that he matches up disproportionately well with this pitcher, etc).

***

If you’re allowing now that the binomial model based upon batter and pitcher true talents determined from large samples is woefully inadequate at the single PA, and that we need to make substantial adjustments to it (or discard it altogether) in order to understand the causes of the result of an individual PA, then I’m with you.

There’s a single PA with no prior information, and a single PA with tons of prior information.

You are quoting me by conflating the two.

***

I believe that the gap between 2 and 3, by definition, is basically zero.  You say it’s either fate or luck.  Well, sure, but you’re lumping player choice in with fate, in which case I’ll say it’s mostly fate.  But we’re probably arguing semantics on that point, and I don’t think that’s profitable.

Correct that it’s luck or fate. The gap between 2 and 3 is huge, not zero.

Luck only enters AFTER the determination of the true mean estimate has been made, not as part of the estimate or prior to the estimate.

***

I’m far more interested in how big you think the gap is between different types of knowledge #1.  How close can we get to true knowledge of what would have happened?  Whether that’s your knowledge level #2 or #3, I don’t know--to me they are the same thing, but for sake of argument, let’s say that knowledge #2 is somehow removed from #3.  In that case, do you think the binomial wOBA model with seasonal (or multi-seasonal) inputs gets us very close to knowledge level #2?  And do other models, with data we can humanly obtain, move us much closer to knowledge level #2?  My answers to those questions are “no”, and “yes”.

Gap in different knowledge in #1?  Probably not much.  We’re pretty limited in our estimates of true talent of various entities.

How close can we get?  Not much closer than where we already are.  Marcel is a leading indicator in telling us that there’s not much more we can know.

How close to knowledge level #2?  Since #2 is a true mean estimate with 0 uncertainty, we’ll never get there, and we’re not close to getting there.

And no, humanly-potentially-obtainable data won’t get us much closer.

***

It seems to me that it’s a very common sabermetric practice, including yours, to label anything between knowledge level #1 as provided by the binomial wOBA model with seasonal (or multi-seasonal) inputs and knowledge level #3 as “luck”. 

I wish you’d stop mischaracterizing my position, and instead, just ask me.  I’ve already said that luck is going from true mean estimate to a manifestation of an event.

I keep saying that luck is the random variation around a true mean or estimate of a true mean.  Luck is NOT the uncertainty around that estimate of the true mean.

Going from a current-human-level uncertainty level of whatever it is to the god-level uncertainty level of zero is not luck.  It’s an uncertainty level of the true mean estimate.  That has nothing to do with luck as I’ve described it in this thread.

***

Again, what I am saying about luck is that after you know, or have estimated, the property and behaviour of all entities that are about to collide, such that you’ve got some idea of the different frequency of possible outcomes, that the manifestation of the single outcome that occurs is due to luck.

So, if Pujols has a 10% chance of hitting a HR with 0% uncertainty, based on god-knowledge, or has a 9% +/- 2% chance of hitting a HR based on human-knowledge, that from that point of determination to whether he happens to hit a HR is pure luck.

If Pujols is in the middle of his swing, God would say 15% +/- 0%, and humans might say 17% +/- 1%, then from that point of determination to whether he happens to hit a HR is pure luck.

If at the moment of impact, God would say 3% +/- 0%, and humans might say 5% +/- 0.5%, then from that point of determination to whether he happens to hit a HR is pure luck.

Once the ball is in the air, it becomes a physics problem (to the extent that there’s no wind, it won’t be close to the fence for a outfieler grab, or any other outside force).  That is fate.  The decision has been made there that it’s either 0 or 1.

"Again, what I am saying about luck is that after you know, or have estimated, the property and behaviour of all entities that are about to collide, such that you’ve got some idea of the different frequency of possible outcomes, that the manifestation of the single outcome that occurs is due to luck.”

*If* the world happens to follow our binomial *model* of it then yes of course this is true.  My take on this argument is that Mike Fast and others are saying sabermetricians are quick to dismiss factors outside of their model.  I *think* Mike Fast is saying that if only we knew what God knows, we’d have a lot more 100% +/- 0% and 0% +/- 0% (or at least results closer to 0 and 100%).  Instead of trying to keep looking even deeper, we stop with the binomial model that gets does a reasonable job in aggregate.

Am I accurately portraying the two camps here?  If not, please correct me, but I felt like something wasn’t being properly communicated.

Statistician George Box famously said, “Essentially, all models are wrong, but some are useful.” I believe Mike Fast and others are accusing sabermetricians of sometimes ignoring the first part of the quote.

I *think* Mike Fast is saying that if only we knew what God knows, we’d have a lot more 100% +/- 0% and 0% +/- 0% (or at least results closer to 0 and 100%).

Is that the argument? That if we knew more, we’d have estimated or true means much closer to 0% or 100%, rather than, say, an OBP expectation of Pujols of 30% to 50%, depending on pitchers, park, etc?

If that is the argument, then we should see more clumping than we do.  Suppose for example Pujols can take more advantage of a particular pitcher if we knew something extra, like Pujols really disproportionately loves pitchers who throw to the outside corner, with fastballs at 92, and the pitcher having a bad changeup, and a curve they can’t locate, such that his true OBP, in this isolated case, would be say 70%.  And for example Pujols simply cannot hit pitchers who have a fastball-changeup combination, such that his OBP is, in this isolates case, would be say 20%.  And so on.

If you suppose all that, then we’d see more clumping.  A level of clumping that would suggest that presuming that Pujols has a true mean estimate of .420 +/- .020 is not supportable.  That instead we’d have to presume his mean estimate to be .420 +/- .200.  Because doing so would at least support clumping.

It’s all fine and well to suggest that his true mean estimate will change radically under different scenarios.  The problem is that the results we observe are far more consistent with presuming that his talent level is fairly tight over the course of a season.

Random variation centered around an estimated (fairly tight through time-space) true mean explains the observed data much better than any other theory.

Yes, mickey/36 has a decent summary of my argument, and I think you pretty much grasp what I’m saying based upon what you said in #37.

I agree that in order for me to be correct, there has to be some kind of “clumping”.  You have to look the right way to find the “clumping”, though.  I don’t think that the way you looked at types of pitchers in the Book is adequate to the task (not because of any fault on your part, your analysis was quite good, but because your data was not adequate).  You have to know how to select things that are in common in order to see clumping.  I think you are suggesting that we should see clumping in time, or maybe you are also including looking out of the time sequence at similar pitchers. 

I’m suggesting we need a more sophisticated model.  I have one in mind, specifically, here.  That is a physics model of the ball-bat collision.  With HITf/x or TrackMan data, we have enough parameters to make pretty good estimates of the inputs to such a model.  Then we can tell what types of pitches make a batter swing early or late or under or over the ball, and what pitchers do to throw these types of pitches (or sequences of pitches).

I agree with Tango that treating players as fixed entities explains almost all of their performance.

But we all know that players are not fixed entities, that players get injured and can develop problems with their swing etc.

We also know that players performance increases with experience and then declines with age but as the season is six months long is this decline detecatble within season or do we assume it happens in the off season.

Has anyone tested the distribtuion of OBP results for a player e.g. turning their season into a string of 1s and 0s representing reaching or failing to reach base and tested it for true randomness over the course of a season.

Or do the same for a creeer (fusing all seasons together) and tried to detect the rise and then decline in ability with age. Are there more or less streaks of good and weak performance than we would expect if a player was a fixed entity.

James

james/39 We can’t do a test for “true randomness” unless we know what to control for.  Suppose we did such a test and found the 1’s and 0’s did appear to be randomly distributed temporally (I think this might have actually been done already anyway).  This result is consistent with the binomial model, but it’s also consistent with Mike Fast’s contention that we just don’t yet know the right way to look at the data to see the clumping.

For a silly example, maybe on days when Pujols eats his Wheaties he’s actually a true .600 wOBA hitter while he’s only a true .300 wOBA hitter otherwise.  Since we aren’t looking for this, there’s a risk that we could get results that would be much better explained by some other model.

For a more realistic example of the type of thing I think Mike Fast is concerned with, it might be that the swing of Pujols happens to work particularly well against pitches arriving at a certain particular angle.  Until we control for that we would be missing out on some of the story.  My hunch is that there won’t be much particularly interesting to find here, but really we don’t know what we don’t know.

Good job in the end to all involved in getting to some sort of stable discussion platform.  Thanks guys…

Yes to mickeyg13/40.

For a more realistic example of the type of thing I think Mike Fast is concerned with, it might be that the swing of Pujols happens to work particularly well against pitches arriving at a certain particular angle.  Until we control for that we would be missing out on some of the story.  My hunch is that there won’t be much particularly interesting to find here, but really we don’t know what we don’t know.

I’d say, though, not just the swing of Pujols vs the trajectory of the pitch but better framed as the brain of Pujols that powers his swing, vs. the brain of the pitcher that produces the trajectory.

I see the results of a plate appearance much more like a chess game than a lottery.

We can take Elo ratings or something similar that will tell us, hypothetically, that Garry Kasparov will beat Bobby Fischer 75% of the time.  But when they play an actual game, it’s the actual moves that they make that determine who won that game.  If Kasparov won the game, it wasn’t 75% his performance and 25% luck.  It was 100% his performance.  That performance may not be completely repeatable next game, but it wasn’t random.

One way a baseball plate appearance differs from the chess game is in repeatability from “game” to “game”.  Because baseball is carried out with physical actions, it’s difficult to repeat them exactly every time.

We don’t actually know what makes some players better at that than others.  It’s tough to quantify it when we don’t understand the causes.

The ball and player tracking data that is being collected these days is going to go a long way toward helping us answer those questions.

The baseball-as-lottery model says that the player talent measured from a multi-year sample determines how many tickets he can buy, and from there only the luck of the draw determines who holds the winning ticket.  That works fairly well as a model for a large sample.  I say fairly well because no one has yet figured out how to beat it.

I think a model that is built from the bottom up, understanding the physics of the bat-ball collision and the chess game between batter and pitcher, has a good chance to blow the baseball-as-lottery model out of the water.

Partly that is a philosophical statement on my part because of what I believe about the nature of the game, and partly it is my belief based upon what I have seen in the data.

Time will tell whose model is better.

Because baseball is carried out with physical actions, it’s difficult to repeat them exactly every time.

This is the most important part, that your choice will have different outcomes if you try to repeat it.  This is true in golf, in free throws, in archery, in beer pong, in whatever.

It’s not like in chess you can “drop” a piece on the board, and wherever it lands it lands.  It’s not like if your piece is not perfectly centered, it takes on a new property.  You make a choice, and there is nothing in the universe that can affect the impact of your choice when you make that choice.

It’s not “luck” that I chose computer science as my major.  I made a choice, and I created a new state for myself.  And if I remade my choice at the same time, I’d still be in the exact same state.

I think a model that is built from the bottom up, understanding the physics of the bat-ball collision and the chess game between batter and pitcher, has a good chance to blow the baseball-as-lottery model out of the water..... partly it is my belief based upon what I have seen in the data.

What kind of data are you thinking of here, Mike?  The data that comes immediately to mind for me—hitter vs. pitcher matchups, results for “hot” pitchers—pushes me in the opposite direction.  Indeed, it would almost suggest that the best you could hope for is that the chess game model will be marginally better.  What kinds of data give you this confidence?

What kind of data are you thinking of here, Mike?

With Alan Nathan’s assistance, I put the April 2009 HITf/x data and PITCHf/x data through a ball-bat collision model based upon work done by (Dr.) Alan and Dr. Rod Cross.  I looked at the results through the lens of the batter’s swing being early/late and under/over.  The sample size has limited somewhat what I have been able to do with the data, as well as some improvements needed in the code, as well as the time that Alan and I have had to devote it.  But I think even in that limited sample of data there is huge potential for discovery based upon what I’ve seen so far.

I should note that what I can do with the HITf/x data is also limited by the lack of data on foul balls in the public data set.  There’s no technical obstacle to collecting that data, but it wasn’t in the data set that was released.  I’d also love to know the timing and location of the batter’s swing relative to the ball when he completely missed it, but that probably involves overcoming some technical challenges.  Just having the foul ball data would add a lot.

Sounds very interesting, Mike.  For now, I’ll remain skeptical about the “blowing the boat out of the water” potential of such data.  But perhaps you can prove my skepticism wrong.....

Mike may very well be able to explain why a batter got a hit or not, retrospectively speaking.  That by itself is a worthwhile endeavour(*).

(*) F-ck you Google spellchecker.

I remain hugely skeptical that we can infer much more as to the player’s true talent level at that moment in time-space, or any time in the future.  The gains, whatever they will be, and they will be there, will be very incremental.  The value will be more with players who have established a new level of talent (via injury or change in approach).

I agree with Tango’s last point.  If we see large breakthroughs, I think it will be in determining when a player is really “done” (Posada) or has really flipped a switch (Bautista).  Even there, I would expect modest progress.

The other possibility is that this kind of data is diagnostically useful for players and coaches.  Maybe it will help some hitters to fix “holes” in their swing (and pitchers to find some).  But again, I’d expect modest gains, because I assume most players are close to the limits of their physical ability by the time they get to MLB (probably earlier in most cases).

mickeyg13/40:

I am almost certain that you would find that such a test would result in failure of the hypothesis that PAs are independent trials and clump as you would expect for a binomial with p = OBP.  There’s more than enough data to get a very tight test on this, and we know there are effects like the platoon advantage that really do exist and would change p from one PA to another. 

Furthermore, wOBA isn’t modeling a binomial process!  It’s modeling a discrete multi-outcome process.  So, the model here is pretty clearly wrong, even if rather useful.  Accounting for the distribution of outcomes with a model that has more than a single parameter seems likely to be important to me. 

In any case, I maintain that the probability of a single event is a pretty useless number.  Probability is only meaningful for collections of events.  I think this puts me essentially in Mike’s camp.  Tango is trying to call the difference between an individual prediction and outcome as “luck,” my thinking is that an individual outcome doesn’t have a probabalistic prediction in any meaningful sense.  You can only predict a distribution of outcomes which will come true over many trials.

Larry/51:

I don’t think anyone is proposing a static p that is equal to OBP.  The p would change according to pitcher identity, pitcher handedness, ballpark, etc.  Tango mentioned the odds ratio earlier in passing, which means putting the batter skill and pitcher skill into a specific formula to get the matchup-specific p for that particular instance.  I’m not positive but I think the work has been done already for the well known factors like handedness.  We don’t know what other factors might be out there that we haven’t thought to test though.

Tango et al designed wOBA specifically because they wanted a binomial process, even though of course it is modeling a discrete multi-outcome process.  I think they pretty clearly understood this was “wrong” but they did amazing things with it in The Book anyway.  If you want to use a multinomial model (yes that’s a thing!), go right ahead.  If I were doing this and had no time/computing restraints then that’s what I would use.

we know there are effects like the platoon advantage that really do exist and would change p from one PA to another

This is what gets me with Tango and Guy’s view here, that anything additional data gives us would be relatively small.

We use a really blunt tool like L/R handedness, and we see a pretty big effect over a whole population.  And it turns out that if you look at another pretty blunt tool like ground/fly tendencies, you also see a fairly significant effect over the whole population.

If blunt tools like that can show us fairly large effects, why would we think that much more precise scalpels would show us things that are much less significant?

I think the chess analogy is a great one for determining luck.  It removes the physics question and obvious quantum restrictions to perfect information.  If you have two players, equally skilled, you would expect each to win half the games (of those that are not drawn).  However, what determines whether a player wins or loses a particular game?  I played competitive chess for a number of years.  Sometimes I saw brilliant combinations.  Sometimes I made really stupid mistakes.  I think either of those deviations from my true talent level could be described as luck.  I suppose you could say that my talent level was constantly changing and occasionally my ability to focus was better than others.  The last part is almost certainly true, but it can’t possibly explain all of the variation.  Given that a player is always evaluating a subset of the total possible moves, whether or not that subset includes the best possible move or a deceptive blunder may largely be random.  Each move I made was conscious and at the time, seemed optimal.  However, I highly doubt I would make the same choice every time if presented with the same situation and possessing the same skill level.

What I should have written in 54 is “the chess analogy is a great one for determining exactly what one means by luck”.

Mike: 
One reason I think gains will be small is that what you call “blunt” and “precise” could also be called “large” and “small”.  That is, whether a ball is hit in the air or on the ground is a BIG difference in outcomes.  Breaking it down into 90 different 1-degree slices is more precise, but it’s also very small—and it may be that GB/LD/FB already captures 93% of what matters. 

The other reason is historical.  In general, the more we learn, the smaller the subsequent discoveries are likely to be.  You can see that in the discussions of the various ERA-style metrics, or comparisons of different projection systems.  The fact that these are invariably angel-on-head-of-pin discussions suggests that we are already approaching the limits of predictive/analytical power.

Many saberists seem to think that micro data will unlock the games hidden secrets.  But my expectation is that micro data will produce mainly micro insights.  Which isn’t bad....

mickeyg13/52:

I was just responding to the proposed experiment in 40, which seems to indicate you think clumping would not be found.  I’m pretty confident it would.  But, more to the point, I agree that Tango doesn’t provide good enough evidence to rebut Mike’s contention that with better knowledge we might be able to show this clumping is a lot more predictable than we currently are doing.

I was under the impression that the authors of The Book created wOBA in order to properly value PAs, not to get a binomial variable.  Certainly they’ve done great things with it, I’m not objecting to it in general.  It definitely behaves closely enough to a binomial variable when you look at sizeable collections of trials.  But, when you get to this kind of philosophical argument over a single trial, the analogy from Free Throws or saves to wOBA didn’t seem like a good one to me due to the multi-outcome nature of the process but I concede that any discrete process has the same philosphical issue.

Guy/56, the GB/FB platoon differential is a relatively new discovery.  Or at least I think it was first discovered in The Book.  DIPS is a relatively new discovery.  I don’t see the trend you’re claiming for the diminishing impact of new discoveries.

I think micro data will produce discoveries that may not have universal application to every player.  If they had universal application to every player in every game situation, we probably would have found them already.  Then again, when you think about the recency of the two discoveries I mentioned above, maybe not.  Maybe we just haven’t been looking in the right place, even for macro, large-population effects.

But I believe it’s very possible, even likely, that from the micro data we will learn many things that will apply only to a few players in a few situations, but added together, will have a very significant impact.

Mike: That’s a good distinction, between few players and selected players, and the micro data may produce important insights for/about some players.  How much it will add up to, I don’t know.

I would distinguish between insights that enhance our understanding of the game, and those that provide actionable information.  I think the micro data will do a lot of the former, less of the latter.  Even DIPS is mainly in that category, as it was already understood that K:BB ratio was the most important metric of pitcher talent.

It would be interesting to try to catalogue research insights that have really impacted team/player behavior.  I would guess the last one that really had a big impact on the sport was James’ discovery that minor league stats were highly predictive of MLB performance (which I’m assuming teams didn’t fully understand).  But that was, what, 25 years ago?

Agree with Guy.

(Not actually Gary Huckabay)

I would guess the last one that really had a big impact on the sport was James’ discovery that minor league stats were highly predictive of MLB performance (which I’m assuming teams didn’t fully understand).

Guy - You are joking aren’t you?  Minor league stats are only predictive of MLB performance for those players that MLB teams decide deserve promotion. There is a huge selection bias since we have no information about how players with similar stats that don’t get promoted by an MLB team would have performed.

Peter:  Is your hypothesis that the minor leagues are full of above-replacement players?  I don’t find that remotely plausible.  So no, I’m not joking (which of course you already knew).  And even in your wildly implausible scenario of significant selection bias, James’ discovery would still be valuable information to teams—it would allow them to project performance among the subset of players they were likely to consider promoting (based on whatever criteria teams traditionally employ).

I don’t know about Peter, but I would think the opposite.  That guys with similar stats, with one getting promoted and another not, that the “not” guy is actually worse than his numbers indicate.

(Traditional)MLEs would more likely indicate that there are tons of AAAA players.

Guy: Is your hypothesis that the minor leagues were full of above-average replacement players before James had his “insight” about minor league equivalency?  I don’t find that remotely plausible.  You are the one that claimed that James’ “insight” had a big impact on the sport.  The biggest impact in the last 25 years, you said.  I say that James’ only provable claim was that minor league players that were promoted to MLB and played well enough to stick in the MLB also had performed well at the minor league level. My hypothesis is that that revelation had little or no impact on the sport because it was already well understood by the teams.

Mike/58 -
Groundball/Flyball platooning was examined back in 1990 in The Stats Baseball Scoreboard, looking at STATS’ first 3 years of proprietary batted ball type data.

Anyway I agree with you in this debate; there is a possibility that we can arrive at more detailed models of the batter/pitcher confrontation which will lead to significantly improved ability to predict the outcomes of specific pairings. There is also a possibility that the improvement would be marginal. I certainly think it is worth your effort to try to come up with an improved model.

As far as the side issue of MLEs, James’ hypothesis was testable on future players who hadn’t entered professional baseball yet. What impact his discovery had on MLB decision making is an inside story; I don’t know that we have much information about it. James may only have caught a wave, but baseball has evolved toward greater use of the type of players he championed. And contra Peter, I don’t see in principal why we can’t look at AAA confrontations between promoted and un-promoted players in situ to get around potential selection bias issues. Jeff Sackman took a stab at a related question with this type of method; his samples were too small to be meaningful.

Good lord, Peter.  This is what I said (emphasis added):

I would GUESS the last one that really had a big impact on the sport was James’ discovery that minor league stats were highly predictive of MLB performance (which I’m ASSUMING teams didn’t FULLY understand).

Like Joe, I don’t know what teams did and didn’t understand in 1985 (or whenever James wrote that essay).  As a fan, I certainly believed the then-conventional wisdom that MLB was in some sense a “different game” than what was played in the minors, and that even a very good performer in the minors might fail if promoted.  So James’ discovery changed my way of thinking about the game.  If it did the same thing for teams, it might have led them to do things like 1) adjust minor league stats to more accurately assess young players, or 2) promote good young players to the majors more aggressively.  (Hard to know if latter has happened, given countervailing impact of free agency clock concerns, and greater longevity from medical advancements.)

And I don’t see any reason whatsoever to think selection bias is more than a totally trivial concern.

Here are the names of the OPS leaders in international league in 2005:

NAME
Brian Daubach
Kevin Barker
Shane Victorino
J.J. Jurries
Eric Munson
Alejandro Freire
B.J. Upton
Ryan Garko
Andy Marte
Curtis Granderson

I believe all of them except Jurries got at least a few MLB plate appearances.  Four of them played at least one full season as an MLB regular.  They all have played at least somewhat close to their MLEs.  The other 5 that played at the MLB level for a short time did not. How can you claim that a)"minor league stats were highly predictive of MLB performance” or b)"I don’t see any reason whatsoever to think selection bias is more than a totally trivial concern.”?

Peter:  Can you provide the post-2005 minor league stats (PA and OPS), and ages, for the 6 non-MLB players?  That may help address the issue.  And can I assume that these 6 had comparable MLEs (adjusting for 6 park/league), both in 2005 and in prior years, to the four who were promoted?  I assume you wouldn’t even present this evidence otherwise.....