THE BOOK--Playing The Percentages In Baseball

An interesting contrast to Sackman is Jaffe’s article on Starting Rotation Analysis at BTF.

http://www.baseballthinkfactory.org/files/primate_studies/discussion/starting_rotation_analysis/

I’ll comment as I’m reading it.  Good so far, in that Jaffe selects the starters before they pile up their performance.  I don’t like the idea of the 6th starter being slotted into whoever’s spot he takes over.  Clearly, if the 4th starter gets removed from the rotation, the 5th guy slides up to the 4th and the new guy is the 5th guy.  Probably not a big deal right now, but let’s see how it shakes out.

Incidentally, my short rule has been to use these as the average win% level for the 5 starters:
.580, .540, .500, .460, .420. 
All the other starters are .380.  In addition to it being nice and symmetric, it sticks to the idea that the replacement level is .380.  This spectrum is not a hard and fast rule, but it is (seemingly) something reasonable.

Anyway, back to his article.

Not sure why he’s using pythagenport, since even Clay doesn’t use that any more.  Using pythagenPat, Jaffe’s data gives us:
.590, .525, .490, .440, .383

Note that with Jaffe’s data, his 6th and 7th starters are actually redistributed into the pool, while I kept mine separate.  Let’s say that I redistribute my 6th/7th starters into the 5th slot, and then reshake the remaining 5 starters into 4 slots.  This is what I get:
.572, .524, .476, .428, .380

For a quick and dirty, that’s reasonably close to Jaffe’s 05/06 data.

Now, for the interesting part.  Jaffe shows how the Whitesox 5th starters only pitched 4.5 IP per start, compared to the average for that slot of 5.4.  This is a GOOD thing!  If you have a replacement-level pitcher in your 5th slot, and you have a replacement-level pitcher in your bullpen, get the ball to the bullpen guy!  Because of his role, he will pitch better.  True, you are depleting your bullpen, but it’s probably a wash.  I wouldn’t get too hung up about it.  In particular, I argue in The Book that teams would be better off not being so strict in the starter role for their bottom of the rotation.  They should be treated more like swingmen.  Tell them they’re going to go out for 2 or 3 innings, not 5 or 6.

If I instead use:
.595, .555, .515, .475, .435
and the next two starters are: .395, .355

(A drop of .040 wins between each class of starters.)

Then, let’s assume that our top 5 starters only make 80% of the starts, and then as the replacement starters are brought in, our starters are slotted over.

So, slot 1 is 80% 1st starter, and 20% 2nd starter.
Slot 2 is 60% 2nd starter, and 40% 3rd starter.
Slot 3: 40% 3rd, 60% 4th
Slot 4: 20% 4th, 80% 5th
Slot 5: 67% 6th, 33% 7th

This gives us a slot 1 to 5 win% of:
.587, .539, .491, .443, .382
for an overall average of .488.  Compared to the overall average of Jaffe’s .486, and 5 slots of:
.590, .525, .490, .440, .383

So, I think I’ve hit on a fairly reasonable model.  I’d probably be better off making the drop-offs between each class of starter different.  Something like:
.610, .550, .500, .460, .430
.410, .400
Which produces slots 1 through 5 of:
.598, .530, .476, .436, .407

Man, that Jaffe is verbose!

FWIW, Jeff is aware of Tom’s points and agrees.  I think he plans to incorporate them into a future article.

Man, that Jaffe is verbose

I did an interesting (but completely irrelevant) experiment. Fully expanded, the Jaffe article take s17 “page downs” until you are at the end. 3 page-downs is all you need to understand what he did and what the results are.

I’ve got to be honest I still find the allocation of slots a little arbitrary. I can’t tell for a 100% but I think Jaffe allocates slots based on who he thinks is #1 in terms of the order they started. While that is fine if pitchers 1-5 continue to rotate in the correct sequence but as Jaffe acknowledges there is plenty of ambiguity.

For instance, he has Liriano as a #3 starter. I’m sure if you talk to Gardenhire he’d have pegged him at #2, possibly #1 at one point.

As long as you peg him before his subsequent start. 

Considering that my off-the-cuff matches so well to Jaffe’s, pretty much shows it doesn’t matter too much.

The allocation of slots is sorta strange.  As Tango pointed out, Kyle Gabbard should never count in the Red Sox “ace” column, as he did in Jay’s analysis. 

So I did the same analysis, only for the Red Sox, with a different allocation method.  I looked at where starters began the year, and then bumped everyone up when a starter was lost, and a replacement brought in, and bumped everyone down when a starter returned. All new starters entered at the 5th spot, except for David Wells, who was slotted in as a 3rd starter. Replacement-level starters could get past the 5th slot if another replacement-level guy joined the force. They were ranked in the order of their use.  The highest a replacement-level starter reached was the 2nd slot.  Kyle Snyder pitched once out of the 2nd spot when Beckett, Wakefield, Lester, Clement and Wells were hurt or traded.

The Red Sox put up an abysmal
4.04, 4.82 5.04 5.27 6.07 for 1st through 5th starters respectively. Not sure how that compares to Jays results.

The fifth spot had 1, 2, 5, 5, 6, 2, 2, 3, 3, and 2 starts respectively from Jon Lester, Matt Clement, Kyle Snyder, Lenny DiNardo, Jason Johnson, Julian Tavarez, Kason Gabbard, David Pauley, Kevin Jarvis, and Devern Hansack.

Save us Matsuzaka!

Great work!

Baselining to Jaffe’s numbers so that the average Redsox starter is equal to the average MLB starter, this is how CDM’s numbers translate for the Redsox:
.586, .505, .484, .462, .393

Yeah—CDM, I like that approach better; it seems more logical. However, as Tango points out it doesn’t look as though it makes too much difference to the results.

Love the shortcut approach by the way Tango

Jeff gets it right for the #1 through #5:
http://www.hardballtimes.com/main/article/another-look-at-starting-rotations/

His #6 is worthless because of the selective sampling.  It would have been far easier to simply say “the rest”, and be done with it.  That, in essence, is your “replacement-level starter”.  He’s not a replacement level pitcher, since that’d be your 12th or whatnot best pitcher.  Your replacement-level starter is better than the worst reliever on the team.

Sticking with just the top 5, Jeff’s numbers become:
0.571, 0.495, 0.482, 0.474, 0.460
(I forced the weighted average to .500.  Not correct, but ok for this.)

I’m assuming the rest of the starts came out to .430 (which gives us an overall starter of .486).

So, that’s what you get from your replacement-level starters: a .430 winning percentage.  They also make up around 23% of your starts. 

Please remember that a replacement-level starter and a replacement-level pitcher as a starter is not the same thing.  A replacement-level starter has value.  A replacement-level pitcher does not.

I think there could be a selective sampling problem even in slots 1 thru 5, if Jeff’s average ERA in each slot is weighted by IP.  That is, one #4 April pitcher pitches well, stays in the rotation, and delivers 220 IP.  Another guy bombs in April and is out of the rotation after 50 IP.  I think that probably makes the #4 and #5 pitchers (who had lower IP on average) look better than they really are.

Guy, great point!  I was trying to figure out why the #2 through #5 were so close, and there you have it.

I think therefore that the best approach is a bit of Jeff, a bit of Chris, and a bit of mine, which is:
1. preselect pitchers prior to start of season, using forecasts (not just ERA, but probably ERA and IP forecasts… maybe runs above repl?)

2. Ensure you always have five starters, so as one starter comes in, a chaining effect takes place based on the preseason forecast

However, that also has some sampling issue, since a guy who comes in as replacement, but is subsequently kept for the whole season (say Liriano) will continue to be a #5 pitcher (maybe #4 at times).

And the chaining idea makes sense more when the pitcher leaves because of injury.  Otherwise, if it’s some bad outings, and everyone slots up 1, there’s a bias there too.

So, I suppose therefore that the best way is to simply use the forecasted numbers, and that’s it.  Of course, that can be biased too, since they are forecasted with a regression toward the mean, and that mean may be a league-average starter (as opposed to say a #5 starter).

This is harder than I thought.

I suppose if we had the hand of God to tell us who was the #1 through #5 before each game, that’s what we are really after.  And therefore, the best way to approximate this is to have a forecast before each game.  Therefore, someone like Liriano can come in as a #4 or #3, and move up to #2 or even #1 (on a non-Twins team anyway).  This really mimics reality the most.  They key is to have a good regression toward the mean component, as obviously a 21-yr old in a starting rotation is likely to be a better pitcher than a rookie 25-yr old.  Scouting would be really needed in these cases.  Knowledge of injury history would be critical to know (see: Pedro).

I’m not sure I see why you need to do complex chaining.  I would just say we need 180 IP from each slot, and then calculate what we get if we take the average pitcher in that slot and add enough replacement starter innings to get to 180.  Jeff doesn’t tell us what the result is for “all others” (after top 5), but let’s guess it’s 5.50.  Then, if the average #5 gives us 127 IP at 4.96, we add 53 IP at 5.50 and say the #5 slot will give a team 180 IP at 5.12.  And as you say, we should not make the mistake of thinking that pool of “replacement starters” has no value.

What’s missing here is some measure of variability or risk for the team.  That is, a team may get 4.96 and 127 IP from slot 5 “on average,” but that average includes a few nice upside surprises (210 IP at 4.70) and a whole bunch of serious failures that required the team to make changes (50 IP at 6.50).  So hidden in that average might be a 40% chance of getting an ERA of 5.50 out of that slot, which means a lot of losses that year.  As someone once said, you can drown in a river with an average depth of 2 feet. 

I think this is an important issue for sabermetrics, which focuses a lot on average/mean but rarely takes account of risk.  Since division championships, pennants, and WS titles occur only in a single season, it matters a lot what happens over 162 specific games.

It would be worthwhile to take a step back and look at a very specific question, which may be different than what Guy is asking.

The question that Jeff may be asking, and that I think I would like to ask is:
what does a #1 starter look like?
what does a #4/#5 starter look like?

So, perhaps all my convoluted talk misses the real point that I’m interested in.  And, Jeff lays it out pretty nicely: a #1 starter (or, the #1 starter on each of the 30 teams) throws around 180 innings, and pitches around 6.4 IP per start.  A #4/#5 starter throws around 125 innings, and 6.0 IP per start.

I think Guy’s original point that the #5 starter’s ERA is weighted by IP stands.

***

You can also make the case that if a guy’s “tools” is that of a #1 starter, that his regression point is a 3.9 ERA, and not 4.6 or 5.0.  You can also make the case that if the talent of the guy is of a 3.9 ERA, then the number of starts forecast may be higher than a regression number would give us.

"Jeff lays it out pretty nicely: a #1 starter (or, the #1 starter on each of the 30 teams) throws around 180 innings, and pitches around 6.4 IP per start.  A #4/#5 starter throws around 125 innings, and 6.0 IP per start.”

I think that’s right, with one important caveat.  I believe Jeff used season stats for each pitcher, rather than just stats as a starting pitcher (that’s certainly the case for the figures he posted for Minn starters in his first article).  This means that the #4 and #5 pitchers (in particular) actually threw fewer innings in a starting role, and may have posted somewhat higher ERAs as well given the reliever advantage.  So if he reruns the numbers with starter data only, we’ll have what we’re looking for.

(What made me think to check this was that his numbers indicate that #1 starters averaged 6.36 IP/start, while #5 starters averaged 6.15.  The gap can’t be that narrow.)

Even framed that way, you beg the question, “What is a #1 starter.” For my analysis, my definition was: “your #1 starter is the best pitcher you have that is going to pitch in the next 5 games.”

If you use Jeff or Guy’s method of defining a 5th starter, you no longer mean hes at the back-end of the rotation.  At any given time, he’s likely to be the 2nd or 3rd best pitcher in the rotation.

Like Jeff points out, the average team used 12 starters.  If you use Jeff or Guy’s definition of starters, one could reasonably say, “This pitcher was a good acquisition. He should slot into the 9th pitcher slot, and ensure that J. Crappy McPitcher in the #12th slot doesn’t see the mound”

In contrast, if you use my definition, saying “This guy is a fringe #5th starter at best” really means that he is going to see the mound when everyone else is hurt.

cdm: yes, and that goes back to the Liriano situation.  He may come in as a #3/#4, and bubble his way to #1.  It would be rather inconceivable for him to be #5 for all of 2006 and then be #1 for all of 2007.  So, you really want a forecast of some sort, but always *prior* to the actual performance.  And you certainly don’t want to keep bumping someone up and down from 5 to 1 to 5, etc, just based on 1 or 2 performances.

***

Guy, good insight again.  The average IP per start is right around 6.0.  So, if the #1 and #2 are in the 6.3-6.4 range, then the rest must be pretty low, in the 5.5 or under range.  I was wondering how he could have had 6.0 for those guys as well.  We’ll wait to hear from Jeff.

***