THE BOOK--Playing The Percentages In Baseball

Wow.  I’ve wanted to do a similar study for a long time!  I can’t wait to read this one.

I have also always wanted to take it a step further and see if the hard throwers (who are usually wild as compared to the soft-tossers) have a better chance of turning into a good pitcher (usually by improving their walk rates) than the soft tossers. 

Rally?

I guess the first question is, obviously, are there any differences between the “soft-tossers” who do end up in the majors and the ones that don’t? As with any study of this nature, there’s a selective sampling issue.

Great study!  I love it!  First of all, why say that projecting pitchers is an exercise in futility.  As you show in your study, we can even project pitchers with almost no major league experience fairly well!  Obviously on an individual level, there is less reliability with pitchers than with batters, but we can still project which groups of pitchers will be great, very good, good, mediocre, bad, and terrible.  Of course all we are doing is just taking pitchers with great historical numbers and projecting them to be great, pitchers with good historical numbers and projecting them to be good, etc.  But who said that a decent projection had to be rocket science?

Also, I agree that projecting pitchers from minor league numbers is a lot harder than from major league numbers.  There are a number of reasons for this, not the least of which is the difficulty in doing park and league adjustments and strength of schedule (opponents) adjustments.

Bill James said a long time ago that projecting major league performance from minor league performance is just about the same as projecting major league performance from major league performance.  He still sticks to this theory I think.  I have disagreed with that for a long time, based on my own projections I have been doing from minor and major league data for 20 years.  Projecting batters from minor league data is not as good as projecting batters from major league data.  And projecting pitchers from minor league data is not even close to being as good as from major league data.

The other thing that Rally mentions is the concept of certain pitchers (usually soft-tossers) being able to get out minor leaguers but not major leaguers. He is correct that his study to some extent debunks that theory.

I have believed that that theory was always bunk.  It makes no sense other than in “announcer-speak logic” (which is something that everyone says because it sounds about right, but no one ever really thinks about it).  There is absolutely no “bright line” between the minor and major leagues that would lend itself to supporting that theory.  All major leaguers were once minor leaguers, just a little better (and not all of them of course).  There are plenty of minor leaguers that are better than major leaguers.  There is lots of overlap. Obviously plenty of minor league first basemen and corner outfielders are better hitters than lots if not most major league catchers and SS.  In addition, we find that batters and pitchers in the majors do not generally have “abilities” to be particularly good against either good or bad opponents.  We present that research in The Book, I think.  In other words, pitchers and batters do best against the weakest opponents, do a little worse against slightly stronger opponents, etc.  Given all of that, there is simply no logic to the idea that a batter or pitcher has the “skills” to be good in the minor leagues but bad in the majors, given that there is a smooth continuum of talent in both the minors and majors, and that that talent significantly overlaps and that batters and pitchers do not appear to have an ability to have “staggered” or “uneven” performance levels against various levels of opponents.

Anyway, back to his study.  Given what he found, I wonder how the prospect lists would stand up to this kind of analysis.  IOW, are the top of the lists (Sickels, Goldstein, BA, McKamey, etc.) populated by the hard throwers, no matter what their MLE?  We talked about this before, but I would love to see the same study done with prospect lists rather than speed of fastball.  I would be a little surprised if the scouts beat the numbers!

One argument I’ve heard that I believe has merit is that pitchers with mediocre fastballs but good off-speed stuff will dominate young hitters in the low minors who haven’t yet learned how to hit a good breaking ball. But when they move up and face more experienced batters, their off-speed pitches aren’t fooling batters as much.

To me, this makes sense. The fastball is the first pitch you’ll learn as a kid, and the one you’ll use most. It stands to reason that by the time you’re in A-ball, your fastball is what it is (or at least has less room for improvement than other pitches).

Changeups, curves, sliders, etc. will be learned later and thrown less often. Most high school batters will really struggle against these, so a pitcher can get away with a mediocre breaking ball. So when a pitcher is in A-ball, there is a lot of room for improvement on his off-speed pitches.

Now take a pitcher like Tyler Clippard. He has a mediocre fastball but a polished change and curve. When he’s in A-ball, his off-speed pitches are already close to as good as they’re ever going to get, so he’s not going to improve as much as a typical pitcher will (according to this theory anyway). He’ll succeed as long as he’s facing young batters still learning to hit the breaking ball, but against more mature batters (even in AAA), he’ll be mediocre (at least according to scouts).

I use Clippard as an example because this is exactly what Kevin Goldstein said about him while Clippard was doing very well in A and AA. Obviously we don’t yet know what kind of pitcher Clippard will end up as, and even if Goldstein ends up right, it’s still only one guy. But I think the logic behind the theory makes sense.

This really has more to do with the quality of a pitcher’s secondary pitches early in his pro career than with his fastball velocity. Not really sure how we would test this theory, other than reading past prospect reviews from Baseball America to find the scouting opinion of early breaking balls.

It’s also quite likely that there are enough mature hitters in AAA (between demoted MLB vets and legit prospects with a couple thousand pro PA) that the type of pitcher you are doesn’t matter at that point: if you can get out AAA hitters, you can get out MLB hitters, junkballers and flamethrowers alike. The type of pitcher you are may only matter in A and AA.

"As you show in your study, we can even project pitchers with almost no major league experience fairly well!”

I don’t think my study shows that.  If you take a large enough group of pitchers you can project what the group will do, but we don’t have a great idea what the individuals will do, at least nowhere near as good as we can do with hitters.

This was a pretty basic study.  Maybe someone will find something in pitch f/x that will improve pitcher projections, maybe detailed injury analysis will help.

Bill James sticks to the theory as far as batters are concerned.  I think his position on pitcher projections is that it can’t be done, and the pitcher projections from the Bill James handbook are not his, they are the work of other guys at BIS.

Colin, yes selective sampling once again rears it’s ugly head.  There may be some soft tossers with good numbers who never get the chance to pitch in the majors.

First, I think it is somewhat dangerous to mix up everything from rookie ball to AAA simply as “the minors”.

But if we limit ourselves to AAA and to some extent AA it seems very strange that there should be any “sharp” line that divides what works in AAA and what works in the major leagues.

Most average or above average AAA players are players that are either likely to play in MLB soon or players that with great “being in the right place at the right time” luck could be playing in MLB or could’ve played in MLB.

How many games would the best AAA teams be projected to win in a 162 game MLB season anyway. My guess is that it is not neccesarilly that far behind what the worst MLB teams do?

I know we love to analyze on the micro level, but how about this one - you keep promoting the guy until he proves he can’t get the batters out any longer

If you take a large enough group of pitchers you can project what the group will do

Isn’t that all you can do with any projections?  Obviously individual projections are going to be all over the map.  The better you can project, the less of the map they will cover.  But the answer to the question, “Can we project or can’t we?” is most certainly answered by looking at our projections in aggregate. If we can project players in aggregate and the range of groups is not de minimus, then we can most certainly project those players.  Now what you are I or anyone considers “good” is another matter altogether.  Bill James is most certainly wrong when he says that, “We cannot project pitchers.” In fact, that is an absurd statement.  What, we can’t project that Peavy, Webb, Santana, et al. will, as a group, have ERA’s 1 to 1.5 runs below average, and that Ponson, Marquis, Robertson, et al. won’t have ERA’s .5 to 1 run worse than average?  Of course we can!  To Bill James, what is that - wood?

If we can project a group of pitcher to have an ERA of 3.90 and they do, another group to have an ERA of 4.5 and they do, and another group to have an ERA of 5.10 and they do, then we are doing at least a decent job.  If we can only project a group of pitchers to have an ERA of 4.30 and they do, 4.50, and they do, and 4.70 and they do (IOW, we can only tell who is slightly better or slightly worse than average, or around average), then I will concede that we are barely able to project this group of pitchers, but that is not the case, even with minor leaguers I don’t think.

Let’s see.  Of the 128 pitchers who made their debut in 2008, I have 73 of them that I mapped and that MGL forecasted.

Of the top 15 pitchers forecasted: MGL forecasted them for a 4.45 ERA, and they were actually 4.43.

Of the bottom 15 pitchers forecasted: MGL forecasted them for a 6.15 ERA, and they were actually 5.16.

All weighted by the number of actual innings pitched.

If I take the bottom 25 instead, the forecast is 5.92 against an actual of 5.40.

It’s possible that simply by virtue of being called (scouting report), that the teams know more than MGL’s data.

I’ve got 56 PECOTA pitchers.  Bottom 15: forecasted 5.81, actual 5.83.

Top 15: forecasted 4.04, actual 4.45.

So, opposite problem of MGL.

These are 2007, not 2008, right? 

How is that the “opposite problem”?

What do you mean “being called?” Why do you say that the teams may know more than I?  Don’t I identify the good ones and the bad ones to some degree?

What is wrong with my and Pecota’s projections?

I am able to distinguish the best from the worst by either .73 or .97 runs in ERA and Pecota is able to distinguish between the good and bad to the tune of almost 1.4 runs!  What is wrong with that?

Right, 2007, since you never gave me your 2008!

You gave them a gap of 1.70 runs and they performed at 0.73 runs difference.

PECOTA did a bit better, forecasting 1.77 runs difference and performed at 1.38 runs difference.

Sure you distinguish them, but not at the scale implied.  You guys are showing a much bigger gap than they actually performed (albeit at such a small sample).

I can easily correct my bottom performers just by scaling them.  So can Pecota just scale their top 15.  It doesn’t matter that I am almost a run off.  What matters is that both of us can easily identify the good and bad pitchers.  How “good” the projections are is really just a function of how much gap there is.  If the gap is small, then you are not doing much better than random.  If it is large, then you are doing a pretty good job!  I’d say a gap of 1 or 1.5 runs, in Pecota’s case, is an extremely good job for pitchers who have never thrown a pitch in the major leagues.

Again, why are you suggesting that the teams/scouts may know better than I do (it is not me of course, just my MLE’s) or than Pecota?

At the very least, we want to take the scouts’ (like BA) list of prospects and see what the spread is between their top and bottom 15.

I am not saying that they will do better.  I have no idea.  In fact, I would say they would do around the same.  My guess is that if you combine them, you would do much better.

As Rally has shown, “stuff,” at least in terms of velocity is NOT any more of a predictor than the stats alone.  There is no doubt that scouts/teams are making mistakes in overvaluing stuff.  At the same time, we are making lots of mistakes by just using stats because we have never seen any of these pitchers and we are not scouts.  Plus we are not even using draft status and the like to regress then towards (which we should).  Those things are built in to the scout evaluations.  So I think they know a lot more things than we do (of course), but they also make some mistakes that we don’t.  I think it will be pretty close to a wash, but I am not by any means certain about that. Maybe they would kill us.  I don’t know.  It would certainly be interesting to see how players that we rate well but are not rated highly by the scouts performed, and vice versa.  That is often the best way to compare two systems - to see what happens when they disagree.