THE BOOK--Playing The Percentages In Baseball

Maybe yes, maybe no.  It is certainly possible, perhaps even more likely than not, that a scout can do a better job with his subjective evaluation of players by looking at their stats.

I would want a scout to report based on his observations than by trying to base his reports on a player’s prior stats, but at the same time I’d like my scouts to be grounded enough in sabermetrics to understand what skills are needed to win at the big league level.

If he looks at his stats, it’s more likely than not that he will be biased.  What you can do is have someone else recommend to the scout players to watch for.  While this introduces a general bias, the scout still won’t know why he’s looking at the player (no stats).

***

I don’t see any need for the scout to know anything about sabermetrics.  What exactly do you think he should know?

I just want him to tell me about the guy’s physical and mental tools, how they are used in game situations, and how he approaches each count.  And how he thinks he’s going to mature, physically, mentally.

Just like in hockey or football, where you don’t have too many stats to help you out.

The last thing you want is a scout looking at a stat-sheet and saying that a guy “hit .315, so he must be a good hitter.” The scout needs to tell us whether he will be able to translate whatever contact ability (or power, fielding, etc.) he has into the upper levels of the minor leagues and to the majors.

For example, a scout would (probably) be able to tell us that Juan Miranda is better than his .260 average in single A without looking at his batting average. But if he sees that Miranda hit .260 against younger competition, some doubt would almost certainly enter into his mind about Miranda’s ability.

OK, I’m sold.  Makes sense.  Now, is there any scout in professional baseball that either purposely does not or is required not to look at players’ stats?  I would say no.

As somewhat of an aside, how much of the “top prospects” list, especially for hitters, is dictated by stats (and non-adjusted ones at that)?  Just by random fluctuation alone, there should be some players who are highly regarded without great stats and vice versa.  How often do these come up on these lists?  Not that there is anything wrong with making top prospect lists from stats or a combination of stats and scouting - as long as the readers are told from whence they came (the lists, that is).

I think top prospect lists are a different animal altogether than scouting.  I don’t see any reason not to include some statistical analysis when putting together a top prospect list whether it’s for each organization, each league in the minors, or all of minor league baseball.  The scouts for organizations aren’t going to be putting these lists together anyway. 

I’d much rather prefer my scout to use his observational analysis than be influenced by the player’s stats.  Let the statisticians do their jobs and the scouts do their own.

I can understand Tango’s request, but I’d like to see scouts do a better job of recognizing a ballplayer instead of pure athletes.  Separate the Dustin Pedroias from the Denard Spans.  Scouts seem to do a pretty good job of differentiating among 3 of the 5 tools - raw power, speed, and throwing ability.  Maybe the others, hitting ability and fielding are subtle skills that take a long time to observe, maybe you just need stats to do it properly anyway.

It does annoy me when a guy like Pedroia is described as an overachiever without tools.  It’s BS.  When it comes to putting the bat on the ball, he’s a 75 if not an 80.  He’s a 2 tool player (very good glove), and excels in the one that is most important to MLB success.  Denard Span, well, he’s really fast.  He’ll make a great pinch runner and defensive replacement.

I would think there’s a place for both.  A scout who tells me what the players’ physical and mental tools are and how they might develop and a scout who knows that the pitcher struck out a batter an inning and can tell me how he did that, which pitches he used, and what might need development before he can perform in the majors.

PITCHf/x and video analysis verge on being able to do some of the second role, but they can be informed by someone who watches the pitcher in person and vice versa.

Rally/8: I don’t disagree with that.

My sole point is that sample data will bias a scout, just as much as it biases any fan.  Anyone can have a great two months of counting stats.  It’s the baseball equivalent of the May-December romance.

Tango, I am going to have to strongly disagree with your contention.  I agree that there is a real chance that stats will bias a scout, such that the evaluation is not truly independent of the stats evaluation.  However, that is more than outweighed (IMO) by the improved evaluation of the game that comes from understanding the relationship between stats and observational analysis.  Put another way - you want your scout to say “In the vacuum I live in, I think player X is good for these reasons”.  That evaluation may contain all sorts of personal biases about what constitutes a good vs. bad player in that particular scout’s mind.  I want my scout to say “The stats for this guy are not impressive, but they specifically miss these factors that I see in this player”.  The second piece of analysis starts with a much better context, and so is much more valuable.

Additionally, I think teams have a moral obligation to provide professional development for their employees, which would include learning skills such as stats.

EA

Erik, I definitely don’t agree with you. 

When you say:

I want my scout to say “The stats for this guy are not impressive, but they specifically miss these factors that I see in this player”.  The second piece of analysis starts with a much better context, and so is much more valuable.

Why can’t it be:

“These are the factors that I see in this player”. 

Let the sabermetrician then combine what the scouts sees and what how he interprets the stats.  Let the sabermetrician come back and say:

“You know, the size of the sample of performance data is virtually useless. I’m giving 90% of the weight to the scout”,
or
“Man, the sample size on this guy is tremendous, and the results are at great odds with the scout.... let me talk to the scout and see if we can find the reason there’s such a gap here”.

Tango, your points are well taken.  However, I would question why the issue of resolving discrepancies falls to the sabermetrician.  Your last quote:

“Man, the sample size on this guy is tremendous, and the results are at great odds with the scout.... let me talk to the scout and see if we can find the reason there’s such a gap here”

could have just as easily been uttered by a scout.  It seems like you are saying one group (sabermetricians) is qualified to have knowledge of both stats and observation, but another group (scouts) can’t be trusted to simultaneously have knowledge of both.

Again, I don’t mean to suggest that the risk of bias introduced by looking at stats isn’t important.  I think it’s a serious concern, and I think we can agree with that.  But smart decision makers have to come to grips with their potential biases every day - shouldn’t teams be training their scouts to do just that?

I agree that a scout’s initial evaluation of a player should be made without any statistical input. If after he files a scouting report, he wants to look at statistics to find areas to focus on in a followup evaluation, that’s fine.

The way I see it is just like in Corporate America.  The scout is your subject matter expert (SME). The SME knows all about scouting, what to look for, how things fit, what depends on what else.  He has total knowledge.  I, as a sabermetrician, having just enough knowledge of the subject to follow along.  I have no value-added to scouting.

However, as a data designer, I can create a function model for all that knowledge.  I can track historical data, I can make it easier to store his knowledge, quantify it, be able to have it used in a proactive, rather than reactive sense.  The data will be a driver.

All this, without the scout ever looking to see if the ball hit was a HR or caught for an out.  After all, how does he know if the ball was a high pop, or a liner that was snagged at the shoestring if he looks at a stat line from a month ago for a game he didn’t attend?  What does he know about sample size and uncertainty levels?

The sabermetrician’s task is to gather data, be it Observational Scouting, or Performance Results.  He will create the model to bring the two together.  He will determine the relationship and relevancy of all the data.

The scout may say “looks good in jeans”, and it’s the sabermetrician’s job to figure out if that means anything.

Scouting and Performance Results need to be kept separate by the people recording those pieces of data.  I don’t want the Official Scorer (OS) to say “out, but would have been a hit if Mike Cameron wasn’t the CF”.  I want the OS to simply record his data.  That’s what I expect from my scout.

I don’t need their biases polluting the data anymore than it’s already biased.

A good example of a SME is Will Carroll.  He knows what he knows.  But, insofar as to how that applies to baseball, you need a sabermetrician to make use of his data.  His Light scheme likely is not good, and requires someone to design a system for it. 

I don’t want Will to look at someone’s IP totals, or ERA, and pitch count totals to tell me anything more.  (Whether he does or doesn’t, I don’t know.)

I do want him to tell me what the various injuries are, what causes them to happen, how they are linked to other injuries, how long they last, how they affect a pitcher’s mechanics etc.

The sabermetrician will tell him what it all means in the grand scheme of things, because he will have created the model for it.