THE BOOK cover
The Unwritten Book is Finally Written!
An in-depth analysis of: The sacrifice bunt, batter/pitcher matchups, the intentional base on balls, optimizing a batting lineup, hot and cold streaks, clutch performance, platooning strategies, and much more.
Read Excerpts & Customer Reviews

Buy The Book from Amazon


SABR101 required reading if you enter this site. Check out the Sabermetric Wiki. And interesting baseball books.
MOST RECENT ARTICLES
MAIL : You ask | We say

Advanced


THE BOOK--Playing The Percentages In Baseball

<< Back to main

Saturday, January 30, 2010

Idea for a study…

By , 01:21 AM

I am thinking of making this a permanent thread, although I have not discussed this with Tango yet.  The idea is to have a section where anyone can post a suggestion for a study that they can’t do themselves for whatever reasons.  Then maybe someone else can take these ideas and run with them, perhaps writing an article on one of the various sabermetric web sites.  (Of course, if there is already some research in that area, someone can point that out too.)

What do you guys think?

Anyway, the idea for a study that I presently had was this:

Take all the FA signings in the last X years, and split them into 2 (or more) buckets:  The ones that were overpaid according to some projection system and the ones that were underpaid. 

Then look at their performance (either rate or counting) in the subsequent year or years and see if teams/GM’s have any skills that enable them to project performance better than the projection system.  After all, any GM or baseball insider you speak to will say something like, “Sabermetric projections are nice, but....” implying that they know some things that the projections don’t, like work habits, injury status, etc. Well, this kind of a study should shed some serious light onto that hypothesis!

You can also break it down by team/GM, although you would be getting into some sample size issues.

Has this ever been done?

Anyone up to it?  It shouldn’t be that difficult to do.  Basically look at projected WARP (or whatever metric you want to use that reflects value) from some projection system versus actual WARP and compare that to the salary paid to a FA minus their “fair” salary given their projected WARP.

Personally, I would use a rate stat for this kind of study to remove some of the fluctuations associated with playing time and injury, but in doing so, you are removing any possible skill at projecting playing time/injury by the team/GM.  On the other hand, a GM/team, especially for their own players, probably SHOULD have better knowledge regarding playing time than a projection system would, so maybe using a counting stat (for example, WARP) would not be so interesting.

Probably both ways (doing it with a rate stats AND a counting stat) is the best way to go. 

You can use the “buckets” method (what I like to call a “poor man’s regression"), or, since you have lots of data points, you can do a linear regression (X is dollars over or under “fair dollars” and Y is performance over or under projection in total runs/wins for each season or runs/wins per PA or some number of “games,” also for each season).  Presenting both would probably be a good idea.


#1    Brad at Cubs Stats      (see all posts) 2010/01/30 (Sat) @ 14:44

Ooh! This sounds really interesting. It would be great to have some sort of compendium of fresh research ideas. Additionally, it may make for some fund historical reading if an amazing, groundbreaking study comes from it all!


#2          (see all posts) 2010/01/30 (Sat) @ 15:12

The one study no one has done completely is the impact of a manager decisions over the year.  I tried last year and they just make too many and they are tough to measure sometimes.  Not all of them are much each time, but they each would need to added.

One problem I had was watching/listening/tracking each game.  It would almost have to be a joint effort of 6-8 people from a team site so someone can follow game and make decision on it.


#3    MGL      (see all posts) 2010/01/30 (Sat) @ 17:29

JeffZ, yes, I’ve wanted to do that for a long time, and yes, it would be difficult and time consuming.  The question is if you could do it from retrosheet data rather than having to watch all the games.  Another of the difficulties is quantifying the value of a move by a manager as compared to the optimal move.


#4    Alex Krolewski      (see all posts) 2010/01/30 (Sat) @ 17:45

What about a position adjustments study using the Fans’ Scouting Report?  I tried to do something like that a couple months ago, but I got some strange results due to small sample sizes and as a result I haven’t done anything else.
My idea was to use the individual categories (arm, reactions, etc.) to find similar players, and then compare their UZRs.  Of course, UZR would have to be regressed, because the FSR measures “true-talent” fielding; also, some players, like Torii Hunter, would have to be removed because UZR has always liked them less than the fans.


#5    Jeff Z      (see all posts) 2010/01/30 (Sat) @ 18:32

#4 On the same lines, I would like to see if the FSR is biased towards better hitters.  Steve Sommers and I compared defensive projections with and without the FSR.  By just doing an eyeball look, the players that deviated to a better value seemed good offensive players and those that deviated to the worse were worst offensive players.

http://www.beyondtheboxscore.com/2009/12/4/1181838/comparision-of-uzr-150-projections


#6    Nathaniel Dawson      (see all posts) 2010/01/31 (Sun) @ 02:35

This isn’t actually an idea for a study, but it’s somewhat related. Oh, and does anybody know if HitFX will be made public? I read that there was some doubt whether it would be accesible to the public a la PitchFX.

HitFX promises to offer incredible opportunities to evaluate defense much more thoroughly and in different ways than anything that’s been developed to date. Reliable, accurate defensive evaluation has always lagged well behind our understanding of offense. How about a sort of “open source” defensive evaluation system based on it? You are all aware of the many great minds that are involved in Sabermetric research right now; certainly pooling their abilities could offer huge benefits to everyone.

Someone would of course have to manage all that input and create an environment where people could work together to put something like this together. I can’t think of any better people to do this than Tango and MGL.....

The scope of this project would be huge—managing all the personalities and filtering all the ideas would be a task approaching something monumental, and I can see many reasons why it might not work. But most of the people that make it their business (I mean hobby, really) to analyze baseball aren’t doing it for the profit—they mostly want to advance the field of baseball analysis and provide greater understanding for the public. Well, that and a little acclaim for themselves once in a while, of course. An intelligent, comprehensive application of HitFX data could revolutionize our knowledge of defensive ability, and the insights and contributions from many different points of view could create something greater than any one person could hope for. Great ideas come from many places.


#7          (see all posts) 2010/01/31 (Sun) @ 06:20

I’ve always wanted to know if there was a strong correlation between position and proneness to injury:

e.g. do SS miss more game to injuries? do CF get injured quicker than RF? do Catchers get injured more than 3B?

Basically, if you have a bat like Mauer at Catcher, but don’t want to lose him for many games, it might make more sense to simply move him to a less injury-prone position where his ridiculous-bat would be in the lineup everyday with less chance of him going down to injury.


#8    Hizouse      (see all posts) 2010/01/31 (Sun) @ 14:03

There probably isn’t enough value-add to justify the legwork, but I would be interested to see Tango, Rally, et al. change the age adjustments in their projection systems to something approximating JC’s age adjustments.  And then we could look back and see whether actual Marcel and CHONE in past years were more accurate than “JCMarcel” or “JCCHONE.”


#9    Greg F.      (see all posts) 2010/02/03 (Wed) @ 02:04

I would like to do a study looking to whether players with poor plate discipline struggle in high leverage situations.

Ideally, I’d use Pitch F/X data like O-Swing % and others along with high leverage tOPS+ and see if these types of players actually do struggle a bit in high leverage situations.

I’ve read the book’s chapter on clutch hitting, and I’m not sure that it totally rules out this possibility.

I don’t know how to go about researching this, though.


#10    Nick Steiner      (see all posts) 2010/02/03 (Wed) @ 02:10

Greg/9

Why would guys with poor plate discipline struggle in high-leverage situations?  I’m not giving an opinion on the matter, I just haven’t heard of that before.


#11    Colin Wyers      (see all posts) 2010/02/03 (Wed) @ 02:11

Hizouse: Clay and I have discussed doing that with PECOTA, as well as the Marcels age adjustment. We’re going to have to wait until the season starts for that project, though.


#12    tangotiger      (see all posts) 2010/02/03 (Wed) @ 08:33

Greg.Nick: actually, in the “Color of Clutch” project, one of the findings is that fans think of clutch of “get the ball on the bat”, and those players may end up doing better.  You have somewhat of a similar finding in BPro’s BBTN.


#13          (see all posts) 2010/02/16 (Tue) @ 01:15

Here is my idea:

I would be interested in seeing a study on how different types of batters are affected by differences in the quality of the pitchers they face.

My guess is that high TTO guys will be more sensitive, and low TTO guys less sensitive. By way of example, if I’m correct, an Adam Dunn would murder poor pitchers but be not so good against top pitchers.  Conversely, Ichiro will hit everyone similarly.  I think this ought to follow to the extent that DIPS is true.

If this is correct, I think it would have interesting implications in terms of player usage, predicting post-season match-ups, etc.

If one were to study this, one could, say, take the top and bottom 10% of batters in (BB+K+HR)/PA and look at the change in their performance (using wOBA) vs. the top and bottom 20% of pitchers (maybe ranked by FIP).

Just an idea.

I missed this thread the first time through, so thanks to Sean for suggesting I post here.


#14    MGL      (see all posts) 2010/02/16 (Tue) @ 02:53

"My guess is that high TTO guys will be more sensitive, and low TTO guys less sensitive. By way of example, if I’m correct, an Adam Dunn would murder poor pitchers but be not so good against top pitchers.  Conversely, Ichiro will hit everyone similarly.  I think this ought to follow to the extent that DIPS is true.”

I will lay 2-1 that that is not true.  Want to make a friendly wager?


#15    Tangotiger      (see all posts) 2010/02/16 (Tue) @ 08:18

"I would be interested in seeing a study on how different types of batters are affected by differences in the quality of the pitchers they face. “

It’s in The Book.


#16    Peter Jensen      (see all posts) 2010/02/16 (Tue) @ 12:56

I would be interested in seeing a study on how different types of batters are affected by differences in the quality of the pitchers they face. “

It’s in The Book.

A study on how different levels of wOBA batters fare against different levels of wOBA pitchers is in The Book (tables 34 and 35), but I cannot find a study that is even close to the one that the poster describes.  Tango, you are usually the most encouraging person to those with new ideas.  So why the quick dismissal of this one?


#17    Tangotiger      (see all posts) 2010/02/16 (Tue) @ 13:43

The study I was thinking about was Table 32, where I break up pitchers and batters based K per BB ratio.

Clearly, pitchers with a high K per BB are going to be really good pitchers.  For example, in that table, I showed that the neutral batters had a .374 wOBA against pitchers with a poor K per BB and a .312 against pitchers with a great K per BB.

So, when Craig says to rank the pitchers by FIP, well, K per BB is pretty close.

And, the batters were ranked somewhat similarly, though not necessarily what Craig was saying.  Against neutral pitchers, batters with a great BB per K were .383 and pitchers with a bad BB per K (or a “great” K per BB) were .325 wOBA.  So, again, a decent way to split quality based on something to do with the strike zone.

And the conclusion to that study was that there was no platoon effect in this regard.

There was also a similar type study in Table 33 based on BB+K per PA (or contactedPA per PA).  And again no platoon effect.

Finally, Tables 34/35 deal with wOBA specifically, and again no platoon effect.

If Craig’s point is specific that a study would be to combine say one or two of the ideas (so say quality v TTO, etc), well, I don’t think you’ll find anything.  Basically, I’ve identified three different profiles for a player, and I matched up the profiles for pitchers and batters based on those profiles.  But, you could have 9 different such studies to get all the possible combinations.  And I don’t see that you’re going to find anything.

***

Also note, any post made prior to 8:30 AM means I’m writing while eating breakfast and I’m in a rush to get dressed and get my kid ready for school.  If this means that my abbreviated comments should not be made, then I will accept that as something wrong that I do.  (And unfortunately will continue to do.)


#18    Rally      (see all posts) 2010/02/16 (Tue) @ 14:22

Ichiro and Jack Cust are probably very close in hitter quality, though their styles are very different.  It would be interesting to see a study like Craig suggested, controlling for hitter quality but with great contrast in style to see how they are affected by quality of pitchers.  But not something I have time to do at this time.


#19    Jim P      (see all posts) 2010/02/16 (Tue) @ 14:33

How much of the difference in “player development” is due to identifying the best players pre-draft and how much is due to working with the players after they’ve been drafted?  Organizations with good scouting would see their future major leaguers immediately overperform their draft position, while good developers would see similar performance initially and better performances higher in the minor league chain.

At what stage in the development process is the most value added to an organization?  (Perhaps compare MLEs for each minor league team at the beginning and end of each season.)


#20    kcdc      (see all posts) 2010/02/16 (Tue) @ 17:39

Although I can’t find it at the moment, I remember reading an analysis a while back about why ground balls are often hit to a hitter’s pull-side and fly balls are often hit the other way.  The idea was essentially that the bat is not level as it is swung--it extends at a downward angle from the batters shoulders--and so if the batter misses such that the ball hits the top half of the bat, it will tend to drift upward and away from the batter, while if the batter misses such that the ball hits the bottom half of the bat, it will tend to move downward and to his pull-side.

This batted ball distribution would likely be advantageous for a pitcher for at least two reasons.  Most importantly, home runs (and to a lesser degree, doubles and triples) are overwhelming hit to a batter’s pull side.  If a pitcher could decrease the ratio of fly balls that are pulled, he would likely significantly decrease his opponents’ slugging percentage.  Secondly, if a pitcher achieves a high ratio, it is likely because hitters are not squaring up the ball well on the bat, and may be hitting the ball with less velocity.  This could limit hits on both flies and grounders.

What I’d like to do if I had the time/data would be to determine if the pull/push split on ground balls and fly balls is a repeatable skill for pitchers, and how it correlates with opponent slugging percentage and pitching BABIP.  A second study could be conducted for hitters.  Thoughts?  Advice?


#21    Mike Fast      (see all posts) 2010/02/16 (Tue) @ 17:44

Kcdc/20, I believe you are referring to Matt Lentzner’s article:
http://www.hardballtimes.com/main/article/why-flies-go-one-way-and-grounders-go-the-other/

You might also be interested in an article that I wrote that addresses some of your questions:
http://www.hardballtimes.com/main/article/confessions-of-a-dips-apostate/


#22          (see all posts) 2010/02/17 (Wed) @ 00:31

Tom/17 --

OK, I reviewed the tables. and I don’t think either quite does it.

Table 32 uses K/BB—but under my thesis batters should be ranked by K+BB (actually +HR too).

Table 33 does use (I think) K+BB, but for pitchers I’d actually want something more like K/BB.

So, I guess if you ran batters grouped by K+BB (or K+BB+HR) vs. pitchers grouped by K/BB (or FIP), this would probably answer it.


#23    kcdc      (see all posts) 2010/02/18 (Thu) @ 13:36

Thanks for responding, Mike.  Your article was exactly on topic, and your results--that there is some evidence of a repeatable, but weak tendency for pitchers to control what field fly balls are hit to--is more or less what I would have guessed.  Coincidentally, your article breaking down Brian Bannister’s pitching repertoire as it related to his unusually low BABIP in his first season with KC was what led me to this question.  I remember looking at his batted ball data, and noticing that almost none of the fly balls he gave up went to the pull side.  Reading Lentzner’s article led me to suspect that this might be a repeatable skill for some pitchers.  I’m a DIPS apostate myself, and I’ll be interested to read any of your future work on the subject.  Here’s to hoping hit f/x comes along soon.


#24    Ari Berkowitz      (see all posts) 2010/02/24 (Wed) @ 13:32

1)Is the average age of the injured players (on the DL) higher than the average age in the MLB? 
2)Even if there are other players on the DL, is most of the salary going to older guys?
3)Are older players more likely to go straight to the DL while younger players may first go DtD?


#25    Mike Fast      (see all posts) 2010/02/24 (Wed) @ 18:00

I’ve posted some incomplete thoughts on catcher defense and identifying hit-and-run plays at THT Live.  People are welcome to pursue this research further.

http://www.hardballtimes.com/main/blog_article/yadier-molina-hit-and-run-plays-and-ideas-for-a-study/


#26    Mike Fast      (see all posts) 2010/03/12 (Fri) @ 16:34

From the Cleveland Plain Dealer on Wednesday, quoting from the handbook of received baseball wisdom:

The Twins have no proven replacement for Nathan. A championship-caliber team without a closer is a team asking to get its heart broken. Losing games late on a consistent basis ruins teams, no matter how good or bad they are.

Is that so subjective it would be impossible to affirm or refute, or is there an objective question in there that could be studied?  I think it’s fairly clear that “Losing games...ruins teams”, but how would you measure whether losing games late has a negative effect over and above the obviously negative effect of losing the game itself?


#27    Rally      (see all posts) 2010/03/12 (Fri) @ 17:36

Losing a closer before the season often means a relative unknown winds up with a 2.75 ERA and 35 saves.


#28    MGL      (see all posts) 2010/03/12 (Fri) @ 18:42

"I think it’s fairly clear that “Losing games...ruins teams”, but how would you measure whether losing games late has a negative effect over and above the obviously negative effect of losing the game itself?”

Obviously, you can look at the results of the next game or two after “losing a close game.” I think someone already did that.

But....

I doubt that a non-proven closer loses many more close games than a proven one, other than based on the difference between their overall quality.  IOW, a great closer like Nathan is simply going to convert 90-93% of his save opps and a not-so-great one will convert 85% (or whatever), which comes out to about 3 fewer conversions per season, and in 1 of those, they will still win the game. So is 2 extra blown wins in a season going to send the team into a tailspin? I hardly think so.

The conventional thinking is that the proven closer comes in and slams the door shut in almost every game he pitches, and that the non-proven one blows close games left and right.  That is ridiculous of course.


#29          (see all posts) 2010/05/28 (Fri) @ 09:15

This is more of a stat, but create a similarity score like stat for players using WAR.  See how close they are at all ages and all the components of WAR.


Page 1 of 1 pages


Name (required)
E-Mail (optional; WILL be published)
Website (optional)

<< Back to main


Latest...

COMMENTS

Feb 11 20:29
Who is Jeremy Lin?

Feb 11 20:11
Clutch analogy

Feb 11 20:11
Fighting leads to goals?

Feb 11 19:55
Why do players get crappy caps?

Feb 11 19:12
Hero of the month: Brittney Baxter

Feb 11 17:59
MGL: Today on Clubhouse Confidential

Feb 11 16:48
Reader Mail of the Day: Why do we need X years of fielding data?  And what about outliers?

Feb 11 10:29
Dwight Evans

Feb 11 02:12
Performance through the ages

Feb 10 23:01
For Your Soul