THE BOOK cover
The Unwritten Book is Finally Written!
An in-depth analysis of: The sacrifice bunt, batter/pitcher matchups, the intentional base on balls, optimizing a batting lineup, hot and cold streaks, clutch performance, platooning strategies, and much more.
Read Excerpts & Customer Reviews

Buy The Book from Amazon


SABR101 required reading if you enter this site. Check out the Sabermetric Wiki. And interesting baseball books.
MOST RECENT ARTICLES
MAIL : You ask | We say

Advanced


THE BOOK--Playing The Percentages In Baseball

<< Back to main

Monday, February 27, 2012

Value of knowing performance from 4 years ago?

By Tangotiger, 11:37 PM

Great study here.  Go read it, because it came off a challenge I put out.

Now, we expect to see some difference, some tiny difference.  It’s not like data from year T-4 is irrelevant.  But, HOW MUCH does it mean is always the question.

Here is what Rob got:
1. He started with his two matched groups, and he had one group at 0.06 FIP lower than the other.
2. In the out-of-sample, we expected to have the same difference, but instead, the gap was now 0.19 FIP

That is, the gap was 0.13 more than expected.

Why is there a difference?  Because in T-4, one group had a FIP of 2.75, and the other had 4.55 for a gap of 1.80 runs.

Therefore, knowing there is a 1.80 larger delta in T-4 will yield a 0.13 larger delta in year T.  That is, you need to weight year T-4 at 0.13/1.80 = 7%

That is shockingly low.  I was expecting an even higher weight.  I use a decay rate of 30% each year.

So, in year T-1, I give a weight of 100%, year T-2, I give 70%, in year T-3, I give 50%, and year T-4, I give 35%.  As you can see, year T-4 is 35/(100+70+50+35) = 14% of the total weight.  (There’s also some regression which would bring it down.)

But perhaps a 40% decay rate makes more sense.  In that case, the weights would be 100%, 60%, 36%, 22%.  In that case, T-4 is 10% of all the data.  Add in regression, and, well, that explains Rob’s findings.

Conclusion: year T-4 should be severely underweighted, and its weight is consistent with a 40% annual decay rate.

Great job Rob!  I’ll put this as one of the top research pieces of the year already.


#1    MGL      (see all posts) 2012/02/28 (Tue) @ 05:33

Great work.

In all fairness to BP, Colin was talking about batters and not pitchers when he said that they used almost equal weights for prior years. He said nothing about pitchers, I don’t think.

Surely they don’t think that pitchers should have their prior years get equal weight.

Hopefully, Rob will do the same thing with batters. I’m not sure why he chose pitchers when BP only discussed batters, but it is great work nonetheless.

Also, I’m not sure, but I think that if pitchers in the “high” group had higher FIP’s in year x-4, they would also tend to have higher FIP’s in year x-3 (and x-2, etc.). I think.

So, even though both groups had a similar 3 year FIP, their 3-year Marcel or other projection would have a smaller gap between the groups.

IOW, Rob got this:

Group FIP IP K/9 BB/9 HR/9
Low 3.46 162 9.0 3.4 0.8
High 3.52 160 8.3 3.5 0.8

However, if the high group had higher FIP’s in the earlier years, a forecast based on only 3 prior years might be something like this:

Low 3.5
High 3.48

Since he found that the higher group was .19 more in FIP in year X, it would be actually .21 more than expected rather than .13, giving even more weight to year x-4.

Again, I am not sure if a higher FIP in year x-4 means a higher FIP in year x-3 (and x-2 and x-1), but if Rob is reading this, perhaps he can tell us the average FIP in each year for each group…


#2    Tangotiger      (see all posts) 2012/02/28 (Tue) @ 08:52

I’d bet that Colin has something similar for pitchers.  His top ranked starter was Tim Lincecum, and Kershaw was much lower than Marcel had him.  That’s what happens when you count Lincecum’s two Cys so high, and you include the very young Kershaw’s high BB rates.


#3    Guy      (see all posts) 2012/02/28 (Tue) @ 09:53

Nice work.  I think the component breakdown is interesting.  It seems that x-4 HR and BB rates have a small amount of predictive power for year X, in the expected direction.  However, x-4 K rate adds nothing (as we would have guessed), and may even be a ‘reverse predictor’—those with a low x-4 K rate actually improve in year X.  Because the pitcher has improved his K rate in recent years, his expected K rate in year X may actually be HIGHER than that of the pitcher who was worse in x-4.  Has anyone (Brian?) studied whether the trajectory of K rate adds any predictive power?  (Alternatively, this may just be a function of age.)


#4    Rob      (see all posts) 2012/02/28 (Tue) @ 20:07

MGL/1: I think the Kershaw example got stuck in my head, so I was thinking only about pitchers. (I also didn’t read Colin’s article fully enough, clearly.)

As for the FIP breakdowns by year:

Group   t-4    t-3    t-2    t-1      t
Low    2.75   3.40   3.48   3.54   3.47
High   4.55   3.38   3.36   3.67   3.66

(Remember t-3 to t-1 are the in-sample years.)


#5    MGL      (see all posts) 2012/02/28 (Tue) @ 23:05

I guess I was wrong about the fact that we might see a higher t-3 FIP for the high group. 

BTW, you said that the average FIP (for 3 years) for the low group was 3.45 and the high group was 3.51.  I get 3.47 for both groups just taking the simple average of all 3 years. Can different number of IP in each year make that much difference?

Also, you base line (or expected FIP) needs to be a weighted average of the 3 years, not an un-weighted average (assuming that a weighted average is correct for pitchers, which I doubt that Colin would argue with).  The reason is this:

What happens if we happen to get these numbers (just by chance) for the two groups:

Group t-4 t-3 t-2 t-1 t
Low 2.75 3.43 3.51 3.47 3.47
High 4.55 3.18 3.56 3.67 3.53

Even though both groups have the same simple average FIP in years t-1 to t-3, the higher FIP in year t in the “high” group would be explainable by the higher FIP in year t-1 and t-2, even without including year t-4…


#6    Tangotiger      (see all posts) 2012/02/29 (Wed) @ 01:55

Given the number of pitchers he got, I think it was a foregone conclusion that he was not going to get that kind of result.

And he didn’t get that kind of result.

So, I’m not sure why we need to incorporate more complexity into something that gives us such starkingly clear results.


#7    MGL      (see all posts) 2012/02/29 (Wed) @ 02:11

I brought that up because I was concerned that the high group was 13 point higher in year t-1 and thus their expected FIP in year t without year t-4 might be higher than a simple average of the 3 years would suggest (and the simple average numbers do not seem to jive with the individual year numbers as well). Although it is the opposite in year t-2 which somewhat mitigates the weighting versus simple average problem…


Page 1 of 1 pages


Name (required)
E-Mail (optional; WILL be published)
Website (optional)

<< Back to main


Latest...

COMMENTS

May 25 15:28
Largest demonstration in Canadian history?

May 25 15:12
Do pitcher’s reach back for velocity when needed?

May 25 15:02
Pete Palmer’s new book: Basic Ball

May 25 14:44
What sabermetrics is NOT

May 25 13:04
“Why Kickstarter works”

May 25 12:51
Chad Curtis

May 25 11:32
Howard Stern

May 25 11:26
Lack of hustle during a game

May 25 10:58
Rooting for laundry

May 25 02:38
NFLPA lawsuit against collusion