THE BOOK--Playing The Percentages In Baseball

There are certain things that scouts can see for which we have no data, or our data is at such a level of abstraction and/or requires such a large sample size that it’s hard to apply to individual players in short time periods.  Our data is catching up quickly on that front, though.

But there are another large set of things where the scout’s value is seeing quickly with his trained eyes what the analyst can see in his data but doesn’t think to look for.  The analyst can act as a force multiplier for the scout in this case. 

Eyes and training and brains and numbers and technical skills all work best when they are working in concert.

Agree completely Mike.  And the front offices know and practice this.

The Buster Olney’s however simply miss the boat.  Confirmation bias is a scary thing when given to someone with so much reach.

I don’t mean the following to be offensive, so please don’t take offense; it may come off as an accusation of hypocrisy, which is not what I intend. I’m only trying to highlight what I see as real tensions in thinking about the value of scouting information.

I’m puzzled by the recent trend among saber-writers and thinkers to embrace scouting. How a guy can believe that what the scorer counts as an error has no merit whatsoever because it is a subjective opinion finds a scout’s assessment evidentially worthwhile? The two seem very similar to me. Of course, in the case of scouting, we can corroborate scouting opinions with actual results, but have we ever tested these against base rates of success for other simple metrics to prognosticate young players, or even older ones? Honestly, the embrace of scouting among sabermetrically inclined thinkers seems to me more like an attempt at something ecumenical among baseball thinkers rather than an honest attempt to sort out what we know about baseball.

My own view of scouts is more guarded: they are a valuable source of information that cannot be written up in a box score, but like any information, we must understand it’s value when we use it, both as a matter of its faults and its merits. However, I’m not sure those merits and faults have really been assessed.

Philosofool, I’ll speak for myself here, though I don’t know how much of what you wrote was directed at Tango and how much at me.

I’m pretty sure I addressed this in an earlier thread here, but it may have been a response I wrote up and then deleted without posting.

All of my investigation into baseball data has convinced me of how little I really know about the game.  The more I learn, the more frontiers and avenues I realize I have never traveled down or across.

Scouts and coaches and players know a lot more about the game than I do.  I can learn a ton from them.  Most of it is in the form of ideas that I never would have thought to think about.  Even if half those ideas are dead wrong and another 30% only contain a grain of truth, they can send me down paths of investigation that are far more fruitful than I ever would have come up with on my own or from simply talking to other saber folks who mostly think like me.

That’s why I said it’s the interaction that is important.  Presumably it’s valuable for the baseball insiders, also.  They can check their ideas against the facts and quantify the impact by checking with people like me, thus weeding out the false ideas from their thinking and prioritizing the importance of the ones that are shown to be true.

The baseball analysis world is not simply one of data.  It’s also a world of theories and perspectives, and continual learning is the key to getting anywhere.  Some people may be able to learn by going off in the corner with a good database and reporting back in six months.  I can’t do that.  I have to interact with other people with good ideas in order to make progress.

Those people don’t have to have the label of “scout” for their information to be valuable.  I find that whoever has spent a lot of time carefully observing the game and thinking about the reasons for success and failure has something helpful to offer me.  Scouts and coaches tend to be those kind of people, but by no means is it limited to those two professions.

For example, why was I taking Rick Peterson to task only yesterday if I’m in the thrall of field personnel and scouts?
http://www.baseballprospectus.com/article.php?articleid=13030

I would still LOVE to have an hour with Rick Peterson to pick his brain.  I could learn so much.  That doesn’t mean that I would think all his ideas were correct or that he would think all my ideas were correct, either.

philo/3: I think your perspective is not well-represented by the facts.

The issue I had with earned runs is the supplanting of runs allowed by the reconstructed earned runs. 

I have no issue with a scorer marking a play as “Error”.  Indeed, I like it.  He’s recording what he sees.  He should stop there.  I don’t know how many times I’ve said I want the scorer to mark hang times, to mark number of bounces the ball takes to get to the fielder, if the 1B scoops a ball, and so on.  These are all subjective observations, they have biases, but it’s data, and it’s valuable.

The idea of a recent embrace to scouting is also faulty.  I ran the Fans scouting report for 8 straight years.  Would I like to have the pro scouts from all the 30 teams as well?  You bet.  But, the teams aren’t going to give them to me.  I’ve got the next best thing.  I had a couple of teams verify the results against their scouts and it came to strong agreement.  So, I know my stuff is not useless.

I’d like Kevin Goldstein and John Sickels and Baseball America to give me their rankings… without ever looking at a performance number.  I don’t want them to analyze what Dustin Ackley’s strikeout rate is in the fall league and what it means for the Mariners in 2013.  I want them to tell me what the numbers DON’T show.

Once I’ve got the two independent sources of information, the scouting lens and the performance lens, then *I* will be in a position to combine them.  This is a position I’ve held for many years.

"Honestly, the embrace of scouting among sabermetrically inclined thinkers seems to me more like an attempt at something ecumenical among baseball thinkers rather than an honest attempt to sort out what we know about baseball.”

I am not exactly sure what that sentence means, but if it means what I think it means (that some saberists are trying to be politcially correct, not be the subject of so much scorn and derision, and are being overly solicitous, in their “embracing” of scouts), I agree.  Sure, there are plenty of things to learn from them, just like there are plenty of things to learn from Olbermann, Limbaugh, Hannity, Joe Morgan, and lots of other talking head, bloviating commentators, but I’m just saying.

Philosofool, please try and write for others and not for yourself, on this blog at least.  I don’t mind a word or two that I have to look up, but come on.  What is the point of writing something that is very difficult to understand?  For example, this sentence is an abomination:

“How a guy can believe that what the scorer counts as an error has no merit whatsoever because it is a subjective opinion finds a scout’s assessment evidentially worthwhile?”

I don’t know if there is a grammatical typo in there or it is just unintelligble, but I don’t have the time to figure it out.

Nothing personal, but it is a bugaboo of mine (one of many) when someone writes in pedantic, hard to understand fashion.  That is fine for a novel, but when you trying to have a discourse with people, it is not fine.  Try and write like you are speaking to an (intelligent) 8 year old.  Seriously.  Or not. 

7 I was not trying to be difficult to understand. Sorry for the typos, and for the jargon. I often read this blog on breaks from my job, which is in a university, and making the shift from academic reading and writing, which is badly laden with jargon, to another audience can be difficult.

4 These seem like good thoughts, Mike. On the whole, I was mostly directing my thoughts at Tango, but I think I misunderstood him when he said he’d take ten scouts for every numbers guy.

6 I’m sympathetic to this view that in a perfect world we’d have the two sources of information separately, so I guess we’re in agreement. When you wrote “i love scouts” I read that to mean “I trust scouts at face value”, but your opinion is clearly more nuanced. The face value trust sentiment is one that seems to be said a lot in stuff I’ve read recently, though not really here (except in the post here.). One of the things I like about this blog is that people usually confront challenges honestly, and so I thought I’d ask why this faith in scouts seemed to be gaining currency, evn though it hasn’t been backed by mathematical rigor and evidence the same way we demand, for example, of SIERA and FIP.

Philosofool, now that I can understand!  I agree wholeheartedly with your last paragraph.  As I said, it seems that we (some saberists, not me) have gone overboard with the “We need scouts” meme.  I think it is fear of being ridiculed and not being accepted by some mainstream folks.  Things that I don’t give a hoot about.  While it is OK to give an “expert” in a field (and scouts are indeed experts in their field) initial deference, anything they assert should be subject to the same critical scrutiny that we give to anyone else.  IOW, you make an assertion without evidence, it is B.S.  BTW, if you have a certain level of trust in a source, it is not always necessary to demand that evidence, but even then, what that expert says must be taken with some degree of skepticism, depending on what they are asserting of course…

The scouts said his arm angle dropped down the stretch, without defining what is stretch.  Per BP’s article, Mike indicated the arm angle was stable from June 9th, and it apears around June 9th is when the arm angle dropped from earlier in the year. 

Evidence is valid even if obtained by observation, provided the observation is true and accepted by all. So if you review Wainrights tapes from last year, you may confirm or reject the scouts observations. Without doing so, you either accept the scouts observation as a matter of faith, or reject it as a matter of faith in pitch f/x accuracy and a rigid definition of “down the stretch” to mean September.

An observation should be regarded as an evidence for a proposition when it is:

1. Compatible with the proposition.
2. It is incompatible with other plausible states of affairs.

So the proposition that Wainright’s drop in arm angle (say from June 9th) was a bad sign, or a sign that he would need TJ surgery is dependent on the truth of observation that his arm angle dropped, that such a drop would be expected by a pitcher with a sore elbow, and that there is no other plausible reason for the arm angle dropping.

I think even if one were to accept the observation as true, and certainly a drop in arm angle from someone having an elbow issue is plausible, there could be other reasons for the drop having nothing to do with his elbow, especially if his performance was not adversely effected.

Since he finished the season rather strong, unless there was a FB velocity drop off at the end or he threw less CB, or Wainright or someone says his elbow was bothering him last year, I would be skeptical the injury occurred in what was his best season ever.

However, I did notice he was shut down after game 153, and threw only 84 pitches in his final start.
Cards out of the race maybe, wanting to keep down his IP, etc, or it could mean something more.

And sure enough, here it is.

http://www.sbnation.com/mlb/2010/9/28/1717687/adam-wainwright-cardinals-arm-injury-strain

Forearm strain was the reason, which is usually the first sign of a torn ligament, and they don;t always show up on MRI (eg Liriano).

So there is some evidence Wainrights injury may have occurred last September, after dropping his arm angle slightly in June (if the pitch f/x data is to be believed), and noticed by 2 scouts in September? (maybe not every pitch). Perhaps coincidental, or not.

pft/10, if you’re arguing that there was some evidence that something was wrong at the end of September, yes, of course.  His velocity dropped a lot in his last start, and his final scheduled start was skipped.

We also don’t know exactly what scouts said to Buster Olney.  All we have is his interpretation of their comments and then what he could fit into 140 characters of Twitter from that.

However, in your argument, you’re assuming that a scout could see the slight variation that occurred early in 2010.  I don’t think that’s true.  Even though PITCHf/x can detect a little change there, that’s a movement of about two inches and would have required that the scouts see him at the peak arm angle right on May 9, 2010, and not again until September.  Pitchers vary back and forth that much all the time.  All the time.  That’s truly a minor variation.  From June 9, 2010, and on, Wainwright was one of the most consistent pitchers in baseball in terms of arm angle. 

You talk about reviewing the tapes as if that would provide some sort of check on the PITCHf/x data.  It won’t.  PITCHf/x is by far the more fine-grained source here.  It’s showing things that you would never have the resolution to identify either with your eye or from video.

I do believe that scouts (and a careful video analysis) could detect the bigger change that occurred on May 16, 2009.  But even that change is not an uncommon thing for healthy pitchers to do.  Not all all.  It’s certainly possible that a scout who hadn’t seen him since early 2009 saw him again in late 2010 and noticed the arm angle change at that point.  Again, we don’t know what exactly was told to Olney.  But that’s not what his tweet implied.  There is simply no indicator of an injury in the game-to-game release point data.

Right, I love scouts because they have the opportunity to give me data that I can’t get otherwise (or as easily available).

To take an extreme example: how fast was Tony Gwynn able to say that Stephen Strasburg was a can’t miss kid?  Did Strasburg even need to pitch in a real-game at all for him to say it?  But, us, we need data.  And that data needs to regress to a population mean.

But, if I have Gwynn and whoever Jered’ Weaver’s coach was and Prior’s coach was (and Brien Taylor’s coach) tell me that these kids are “can’t miss”, then I have a much different population mean to regress their performances.  All of a sudden, a 12K performance by these guys in college is treated far different from a 12K performance by a no-name pitcher.

Same applies in MLB.  I have a .380 wOBA by Pujols and by Bloomquist after 200 PA.  And that’s all the data I have.  But the scouts tell me that Pujols is awesome and Willie is terrible.  Now, I can regress Pujols up to a .410 pop mean and Willie is regressed down to a .300 pop mean (or whatever it is that I can infer from the scouts).

The issue with the scouts is how reliable are they, and how independent are their views from the data. 

I certainly cannot take their observations at 100% face value.  Nor can I take the fact that Joe Mauer went on a HR tear two years ago at face value either.

These are all observations, sample data points, to be treated with a certain level of uncertainty.  I love scouts because they give me data points that I simply can’t get anywhere else.

#12, 100% agreed and it should go without saying…

mgl/13: It’s probably apparent to us because we think about it all the time.  But, it’s possible that others misinterpret what we do or say based on limited readings of what we’ve posted in the past. 

philosofool obviously came to a wrong conclusion, and if he’s like that, I presume at least another 100 people are like that.

In any case, if some media person asks me about scouts v stats, then I have a handy post that I can just refer him to, so he can rewrite it as scouts + stats!

It seems that a lot could be gained by the sabermetric folk doing more directly with scouts to help quantify more directly what the scouts are seeing, to standardize observation and so forth.  The collaboration need not occur just at the organizational level, but at the personal level.

Could we be modeling “intangibles”?  Can we create standards around reaction time, acceleration, swing angle, etc.?  We can bring more scientific methods to scouting itself, rather than keeping sabermetrics solely within the purview of performance analysis.

#15, no question and an excellent point.  The ideal team front office would be doing that…

#11, you wrote

“However, in your argument, you’re assuming that a scout could see the slight variation that occurred early in 2010.”

If a scout says he saw the variation, I assume he is telling the truth.  Not being a scout, I do not know their detection limits. 

“Even though PITCHf/x can detect a little change there, that’s a movement of about two inches and would have required that the scouts see him at the peak arm angle right on May 9, 2010, and not again until September.”

Scouts do not watch every game.  It may have been a team that played the Cardinals/Wainright early in the season and not again until later in the season, and they send their advance scouts before the games. We don’t know which scouts, so we don’t know the teams.

With video, you can slow mo, freeze frame, and other amazing things, and can certainly detect a change in arm angle if the games you are comparing are at the same park and camera angles (typical of home games)

“PITCHf/x is by far the more fine-grained source here.  It’s showing things that you would never have the resolution to identify either with your eye or from video.”

But there is some error and calibration issues that may make it difficult to see such changes (or show changes that are not there) over a small number of games.  Also, correct me if I am wrong, but they don’t actually detect the release point, but have an algorithm that “estimates” the release point based on where pitch f/x first picks up the ball (which is not the release point, seems I have read somehwere it is like 20 ft from the mound).

If a scout says he saw the variation, I assume he is telling the truth.

We don’t know what the scouts told Buster Olney.  All we have is the 140-character summary that Olney made. 

There is also a strong incentive to remember things differently than they actually happened after an injury occurs.  Every time a pitcher has Tommy John, someone goes back and points out how they saw something.  That’s way different than saying ahead of time who does and who doesn’t have problems.

Scouts do not watch every game.  It may have been a team that played the Cardinals/Wainright early in the season and not again until later in the season, and they send their advance scouts before the games. We don’t know which scouts, so we don’t know the teams.

You’re telling me that the only two scouts that Olney talked to randomly happened to both see Wainwright on May 9 and not again until September.  Possible?  Sure.  Likely?

With video, you can slow mo, freeze frame, and other amazing things, and can certainly detect a change in arm angle if the games you are comparing are at the same park and camera angles (typical of home games)

I have the distinct feeling you’ve not actually tried doing that.  A pitcher’s hand moves four feet in between video frames.  It’s basically impossible to do what you’re suggesting from video unless the change is huge (on the order of feet, not on the order of inches).

But there is some error and calibration issues that may make it difficult to see such changes (or show changes that are not there) over a small number of games.  Also, correct me if I am wrong, but they don’t actually detect the release point, but have an algorithm that “estimates” the release point based on where pitch f/x first picks up the ball (which is not the release point, seems I have read somehwere it is like 20 ft from the mound).

The PITCHf/x system fits to the data using Levenberg-Marquardt and then solves the equations of motion using the Runge-Kutta method.  They assume a constant acceleration fit, which actually turns out to be quite accurate.  From the equations of motion, you can find any point on the trajectory that you want, from the pitcher’s hand to the catcher’s glove.

As I wrote in the article that Tango linked, I’m using data that was corrected for calibration issues.

I go into more detail in the article I wrote here:
http://www.baseballprospectus.com/article.php?articleid=12432

"If a scout says he saw the variation, I assume he is telling the truth.”

pft, I have read a ton of books on cognitive, behavior psychology, and prospect theory.  Unfortunately, I don’t remember much of what I read, but I can assure you that there is no simple dichotomy between “telling the truth” and “telling a lie.” What a person thinks is “the truth” or not is extremely complex..