from Amazon
Thursday, February 11, 2010
What does SIERA think of walks?
I started with this baseline data:
PA BB SO BatBall GrnB LinB FlyB
1000 90 180 730 314 146 270
That is, you have 1000 batters, of which 90 are walks, 180 are strikeouts, and 730 are batted balls. And the batted balls are broken down as 314 GB, 146 LD, 270 FB. SIERA comes in with an ERA of 4.31. Estimating 234.7 IP, and .92 ER per R, that means 122.31 runs allowed.
I created a matching line for FIP, using the same PA, BB, SO, IP data, and putting in 27 HR (10% of FB). Adding in a constant +3.1995 (in place of the “3.2"), and I get an identical ERA and runs allowed of 4.31 and 122.31.
I created a crude BaseRuns equation to give me the same result.
Finally, I put the above in the Markov calculator as:
http://tangotiger.net/markov.html
910 AB, 235 H, 54 2B, 5 3B, 27 HR, 90 BB, 180 SO
This gets me 4.688 runs per game. Multiplying by .92 and I get 4.31 as an ERA.
All equations are now calibrated to the same baseline.
I then added 1 walk, 1 PA to see what would happen:
SIERA: 122.68 runs, or +.373 runs
FIP: +.375 runs
BsR: +.340 runs
Markov: +0.3%, or +.365 runs
Some basic agreement here. What happens if I add 10 walks and 10 PA instead? Here are the marginal changes:
SIERA: +.365 runs
FIP: +.375 runs
BsR: +.341 runs
Markov: +.388 runs
Here’s the pattern for SIERA, FIP, BsR, as I add 10 walks each time:
BB SIERA FIP BsR
100 0.370 0.375 0.341
110 0.365 0.375 0.342
120 0.359 0.375 0.344
130 0.354 0.375 0.345
140 0.348 0.375 0.346
150 0.343 0.375 0.348
160 0.338 0.375 0.349
170 0.333 0.375 0.350
180 0.329 0.375 0.351
190 0.324 0.375 0.353
200 0.319 0.375 0.354
As you can see, FIP keeps it constant, which is wrong. BsR goes up, and SIERA goes down. Markov is +.483. Markov suggest therefore that the marginal run value of the walk should go up, as it should. SIERA suggests the opposite.
In isolation, this means that SIERA has an issue. However, if there is an relationship between extra walks and other events (say a lower HR rate or lower hit rate per batted ball), then SIERA would capture this. This can only be shown through empirical testing, not this general testing.
If we go backwards, from 90 walks down to 0 walks, we get this:
BB SIERA FIP BsR
80 0.376 0.375 0.340
70 0.382 0.375 0.338
60 0.389 0.375 0.337
50 0.395 0.375 0.336
40 0.402 0.375 0.334
30 0.409 0.375 0.333
20 0.416 0.375 0.332
10 0.423 0.375 0.330
0 0.431 0.375 0.329
It follows the same pattern, such that the fewer walks you have, the more run impact each walk represents. Though, in isolation this is untrue, it may be true if there is a dependency to other events, and presumably this is what SIERA is trying to capture.
Is this true? Well, I guess we’ll have to test that.