THE BOOK--Playing The Percentages In Baseball

We were looking into projecting team fielding b/c we need it for our 2010 pitcher projections.  Dash took projected UZR’s (I think this is Jeff Zimmerman’s work) and summed up the starters by team.  He got +37 for the Mariners which was a close 2nd to the +42 for the Rays but all teams averaged something like +4 so saying +33 and +38 might be more accurate given our system.

This still needs a some work but do you think the projected player UZR’s are too conservative?  They would have to be considerably more aggressive to have teams in the +50 to +110 range.

Are you confidence that the variance in true talent offense is the same as the variance in true talent for defense?  Also, variance in pitching + variance in defense = variance in defense only works if pitching and defense have no correlation, but they probably do, right?

How are the team win totals calculated?  There is no mention of this at the link that was given.
vr, Xei

Ok, I see there is some kind of spreadsheet.  Must be in there.  Can’t download from this computer.
vr, Xei

What I don’t get is why the M’s are above average in each category, and have a winning percentage of .537 in column F of the spreadsheet (team totals tab).  That’d get you 87 wins, not 77.

So how do these winning percentages get translated into the ‘standings?’ What’s docking the M’s 10 games - it can’t be schedule, looking at the projections for the other AL West teams....

4/These are optimistic projections so essentially every team is over .500. The average AL team is .562, so Seattle’s .475 makes perfect sense once you put them in a league with 29 other optimistic teams.

The .537 W% is the “optimistic” win percentage—every team’s, I believe, is higher than their projected. But the schedule adjustments take care of this and make the total projected wins add up to 2430.

As far as I can tell, the best single season team defense by BIS UZR for the years we have has been +70.  Looking at zone rating since 1987 shows about the same for a maximum.  Is +110 really realistic?

I’ve got the M’s projected around +50, but that includes Bard as the primary starter, and my defensive projection for him is pretty bad (-8).  If I swap in Rob Johnson instead, who I have rated closer to average they get to about +55.

Guess I should have looked at 2009, the Mariners were around +86.  Still not sure if projected +110 is realistic though.

The spreadsheet uses the odds ratio to generate team by team W-L records (what are the odds a .637 team beats a .542 team?, etc.) You can turn the protect worksheet off if you want to see the formuli.  I just put that in there because the only thing you have to change if you use different team assumptions is the win % in column B, the rest works automatically.

J Cross, if you are using Fangraphs UZR then you probably don’t have a catcher figure, I’ve got Rob Johnson at -5 so other than that, we’re close to exact match.

I think the upper range you could expect is the unregressed career UZR for this bunch, above average fielders all:

Kotch +4
Lopez +1
Wilson +6
Figgy +8
Ichiro +11
Gutierrez +26
Bradley 0

For a total of +56.  But if you don’t do any regression, keep it a secret from MGL

Comparing the unregressed career UZR to the TZ projections on my site, excluding catcher and CF UZR has +30 and I’ve got +29.  The difference is Gutierrez, +26 on UZR and only +6 for TZ.

I said anything between +50 and +110 I can believe.

And I said all the fielding numbers should be doubled.

And that Chone was at +28.

So, if you were to revise to +56, I’d have no issue.

The spreadsheet is there if you want to put your own numbers in and give us your team predictions.  I’ll stick with some regression.

I am curious that you would find 110 more believable than 28.  That would be double the unregressed career UZR for all these players.  If They should be at +56, how does that look at the player level?

Right, I don’t see how 110 could be remotely believable.  If no team’s observed performance has come close to matching that in the BIS era then I find it extremely unlikely that a team’s true talent is anywhere near that.

I did say 50 to 110, meaning 80 +/- 30.

I don’t think it’s crazy to look at Gold-Glove candidates at 5 positions, with the other 3 positions being at worst neutral, and think that the mean for that should be +80, is it?

If the mean is +80, then the uncertainty range should be… something, and I say it should be +/- 30.

It doesn’t matter if any team has ever approached it, if no team was specifically built for it.  That is, there were lots of +50 observed UZR, but those teams weren’t designed to have necessarily at worst an average fielder at each position, and have stars at 4 or 5 positions, right?

So, you have to be careful with regression.  If you know nothign at all, you regress observations as you would do anything.  But, Mariners players have obviously been targetted differently.

Looking at it, maybe it should be +70, +/- 30.  But, it’s pretty gosh-darn high.

Instead of looking at career UZR, if we look at a 4/3/2/1 weighted average of 2006 through 2009 with no regression, here’s how the projected Seattle starters at the seven non-catcher positions look in terms of UZR/150:

Ichiro Suzuki, RF: 11 UZR/150
Franklin Gutierrez, CF: 26 UZR/150
Chone Figgins, 3B: 12 UZR/150
Jose Lopez, 2B: -1 UZR/150
Jack Wilson, SS: 11 UZR/150
Casey Kotchman, 1B: 8 UZR/150
Milton Bradley, LF: -13 UZR/150
Total, : 54 UZR/150

Bradley has SEVERE sample size issues in LF, he’s been a hair above average in CF and RF, so we should probably assume he’d be at least average in LF.  So as a team, they’d project around +67.  I guess the +70 +/- 30 range is reasonable, although my preference would still be to include some regression, especially for Gutierrez, who also has sample size issues.

Having Langerhans and Hannahan on the bench is also good for the defense, as both look like plus defenders.

I looked at the teams I had projected as the best defenses last year.  TB, Tor, Sea, Phi, Det, and Oak averaged about +30 in my projections.  The 2009 UZR of that group averaged +33.  Don’t see any reason to change the formula here.

If my defensive UZR values were used, they are basically Marcels for defense.  Maybe I should call them the Georges.

Steve Sommer did some projections using the FSR, that may not be regressed as much.

I think I come in closer to +40 or +50.  Tom, you know I like you, but +70 to +80 seems… really optimistic, even with these players. 

Kotchman’s good, sure.  But Garko wasn’t brought in to sit around, and he’s not so hot at first base.  A straight platoon takes 25% of the first base innings away from Kotchman and gives them to a below average defender.  That makes a real high projection at first base tough. 

Lopez - UZR likes him way more than fans/scouts/body mass index.  There’s no way around the fact that he’s rather out of shape for a middle infielder, and he makes more than his fair share of errors.  He’s the kind of guy who could easily post a -10 UZR and not shock anyone with it.  We have to regress his UZR harder than we would a more athletic player. 

Wilson has had a bunch of nagging injuries the past two years, limiting him to ~120 games per season.  He’s also in his 30s now.  Even if we’re not heavily regressing his abilities, we have to be conservative with his playing time, and the backup SS is Jack Hannahan, who has all of one game experience as a professional at the position. 

There are scenarios where I could see this team playing +70 or +80 defense in 2010, but they involve a lot of health from Jack Wilson and Eric Byrnes, a lot of DH’ing from Milton Bradley, and Jose Lopez continuing to put up good defensive numbers in spite of looking like a tub of goo.

I don’t think I want to count on any of those things happening.

SG, with corner outfielders I treat it as the same position, so if you’ve got a guy who’s mostly played right, a few games in left, but you know he’s playing left next year, just add up his defensive runs and chances for both positions and project him as a corner outfielder.  Even better is to use the CF data (adjusted of course) but that is a little trickier.

If you use unregressed career UZRs, the Ms still come out as the worst team in the division. The As’ unregressed career UZRs are actually better than the Mariners’. You could also just use the 2009 UZR numbers. The Mariners still finish last. You could use Jeff’s UZR projections. You could, as Tango suggests, double Rally’s defensive numbers and increase his pitching numbers by 40%. The Mariners project as the worst team in the division using all of these scenarios.

Tango’s way smarter than me, so I’ll accept his assertion that the variances in offense should match the variances in pitching+hitting, which makes sense, but if there are flaws in Rally’s projections, underrating the Mariners doesn’t seem to be one of them.

I think there may have been some miscommunication.  Tango, when you say 80 +/- 30, are you expressing uncertainty in the mean (true talent) or uncertainty in what the observed result will be given a mean of 80?  I parsed your original as stating the true talent may be anywhere from 50 to 110.  My main objection was that it’s exceedingly unlikely that the true talent is that high, even though they may perform that well.  Without having checked it, I’m not even sure if taking the best defender at every position would yield a true talent above 100…

SG’s numbers is along the lines of what I was thinking, with LF being at least 0.  So, +70.

Add in injuries, limited playing time, more Garko, etc, and I see a +40.  Add 5 runs to everyone, and you are at +100.

So, yes, true talent.

FWIW, here are the top players at each position by UZR using the aforementioned 4/3/2/1 weighing for 2009->2006 with no regression.

Player, Pos: UZR
Evan Longoria, 3B: 17
Chase Utley, 2B: 14
Carl Crawford, LF: 14
Franklin Gutierrez, CF: 12
Ryan Sweeney, RF: 12
Elvis Andrus, SS: 11
Albert Pujols, 1B: 7
Total: + 87

These are the weighted average UZRs with actual playing time, not adjusted to 150 games, but aside from Gutierrez and Sweeney they’re basically full-season projections.

Just to be clear, you’re fine with calling franklin Gutierrez a “true talent” +26 defender in CF?

I love Death To Flying Things as much as anyone, but that makes him the center field version of Ozzie Smith. 

His UZR/150 in CF is +26 in just over a season’s worth of playing time.  In about equivalent time in the corners, he’s +20 UZR/150, which would make him something like a +10 CF.

Even if we just add them together, that brings him down to +17 UZR/150, no regression.  I’d probably call him a +15 CF, which makes him the best defensive outfielder in baseball, but I don’t think I can see anything higher than that.

I’m gonna have to go with Rally here.  Even assuming that the Mariner’s will have a higher mean to regress to, given what the front office is looking for in players, the max I could see projecting them to is +40 or so. 

BTW, from 2002-2008, team UZR has a standard deviation of 31 runs.  Team runs scored have an SD of 74 runs.  Team FIP runs allowed…

(((HR*13 + (BB*3) - SO*2)/(IPouts/3)+3.2)/9)*(IPouts/3) AS runs

...have an SD of 58 runs.  Those should all be deflated for projections for obvious reasons, so I don’t see anything inherently wrong with projecting the SD of offense to be higher than that of pitching and defense - although I think Rally’s are probably a bit too high for offense:pitching:fielding ratio.

Actually, I misread what you wrote in the OP.  Although the variances don’t actually add up historically, with offense being 1151 square runs higher than FIPruns + UZRruns.

I don’t know enough to say whether I think the projections are regressed too heavily or not, but I can say, as I twins fan, I really like the looks of them.  If the Twins are really a -9 defensive team without counting Nick Punto, then they are probably about average overall.  I expect him to play about half the games at third, and replacing a -8 with at least a +8 (his career UZR is +20 at 3rd, and +18 at SS), should make up those runs. 

Personally, though, I just can’t imagine that Delmon and Cuddyer end up only 12 runs below average, even if Delmon lost 30 pounds over the offseason.  Their numbers match what I see on a daily basis, that they are bad outfielders. 

I expect the Twins to be somewhere in the -15 to -25 range, depending on how much Punto plays, and if they sign a good defender to be a defensive replacement.  If they were league average, I would be ecstatic.

Nick, I think you would want to compare the variance of team runs scored to runs allowed instead of to FIP+defense, because FIP isn’t going to have as much variance as actual runs allowed adjusted for defense since it is also neutralizing things like sequencing and BIP placement that will add variance to the team totals.  I think you would also want to put FIP on the scale of runs, not earned runs, by dividing the FIPruns by .92 if you were going to do it that way.  Using team UZR will also deflate run prevention variance a bit because it doesn’t include catcher defense, so FIPruns+UZR runs won’t cover all of the variance of observed pitching+defense.  Looking at actual runs scored for offense and removing some of the varying factors for pitching will inflate the variance of the hitting relative to pitching/defense and make the offense look like it has more variance even if it doesn’t.

Since the whole point of adding up pitching and defense is to get the total variance of run prevention and compare that to the variance of run scoring, just using runs allowed is simpler anyway, and it gives you a check to make sure that the variances of pitching and defense actually should add up.  I agree that they don’t have to, though.

Even if they did add up historically, it wouldn’t necessarily mean the variance of run scoring and run prevention for the projections have to because of the uncertainty talked about in the original post.  If projections for pitching and/or defense are more uncertain, then you can’t project as much variance even if you know they’ll end up with the same total variance as offense.  I don’t think that means that a team built with a focus on run prevention is necessarily underrated by the projections, though.  If a team actually is top-notch all around at run prevention, it would be underrated, but the uncertainty means that you can’t tell as well who the top teams/players are on defense as you can on offense.  If defensive projections inherently have more variance than offensive projections, and the Mariners are trying to build a great defensive team, they are subject to the same uncertainty as the projections (unless found some tablets carved in stone from god that say exactly what everyone’s true talent is).  They can only know with as much certainty as their projections who the best defenders are, which means that they are less likely to have identified the best defenders than they would be to identify the best hitters.

Dave/24: my statement can’t be interpreted that I agree with a line-item by line-item acceptance of SG/15.  I don’t know how to say that I generally agree with them, without being individually specific, but that overall, it’s close.  I said that he’s along the lines I was thinking, and I’ll leave it at that.

If you think that SG/15 is accurate overall (including the assumption that Bradley is at least average in left), but that Gutierrez’s true talent is less than +26, then you would have to believe that at least one other person has a true talent level better than even the unregressed UZR numbers, i.e., the opposite of regression to the mean. That doesn’t seem right. Maybe you just think that the catchers will make up the difference between +26 and whatever Gutierrez’s true talent is? Total zone doesn’t like the defense of Johnson or Bard or Moore, but I guess you could argue that since the Mariners are building for defense, we should assume that they wouldn’t be comfortable using a below-average defensive catcher. That’s too many assumptions for my taste.

When will people realize that because of flaws in the methodology TotalZone is not capable of producing run values that are accurate enough to be useful for analysis.

"even the unregressed UZR numbers, i.e., the opposite of regression to the mean.”

Only if you think the “mean” is zero.  You regress performance of players toward the mean of the population they belong to.  If you have reason to believe that Gutierrez or Jack Wilson or Figgins belong to a different mean, then you need to include that as a parameter as well.

As a fer instance, let’s say that all the scouts agree that Elvis Andrus is the best fielding SS in baseball, and perhaps the best since Adam Everett or Ozzie Smith or Mark Belanger.

His UZR is +12, and based on limited playing time.  What do you do?  Well, that UZR needs to be regressed toward the mean of all SS (0) and also toward the mean of all highly-rates SS (say a mean of +25 or +30).  So, you might end up with say a +19 for Andrus.

That’s how you can regress away from 0.

So how much if any does the lowering of the Citi Field fence affect CHONE’s predictions for the Mets crew?

(trying to decide whether to keep Wright or Andre Ethier in my fantasy league! One’s got to go)

This discussion about the Mariners defense seems applicable to the Reds as well, who dropped back 4 games in the simplified methodology.

In terms of specific defensive values, I was quite surprised to see Brandon Phillips at -3 and Orlando Cabrera as the defensive contributor worst fielder in baseball at -15.  The Reds must have the worst defensive middle IF in baseball per CHONE…

The approach also seems to do harm to teams where a position will likely be manned by a platoon.  Again, the Reds are harmed here due to their LF.  It’s highly unlikely any single LF gets a majority of the PA and the Reds talent is highly conducive to an effective platoon.

"So how much if any does the lowering of the Citi Field fence affect CHONE’s predictions for the Mets crew?”

I’m just using last year’s park factors.  I don’t know how that will affect things at Citi field, and I’m not going to try to guess.

"When will people realize that because of flaws in the methodology TotalZone is not capable of producing run values that are accurate enough to be useful for analysis.”

I’ve found when using multiyear data combined into projections, it does about as well as using UZR projections or Fan scouting reports to predict next season’s defense.

My methods were to use preseason projections applied to actual playing time, and compare to park adjusted DER for the next season.  The different methods were so close to be insignificant.  Using Zone rating runs was in the same ballpark as well.

IIRC, we figured that TZ was good,but the methodology depressed things more than it should.  That’s why the standard deviation in TZ was something like 2/3 or 3/4 that of UZR.  Wasn’t that the conclusion we reached?

Yes, we concluded that TZ will tend to understate very good (or bad) defensive performances.  And the way it does that is by essentially sharing credit/blame for the performance with adjoining teammates.  As a result, I don’t think Rally’s team level validation answers the question.  Because if TZ gives 40% of the blame for Jeter’s missed balls to other Yankees (as it does), and those same players are with the Yankees next season, then it will all balance out and the prediction will look fine.

I think Tango’s initial estimate of a fielding SD of 15 runs is far too high.  That would imply a team SD of over 40 runs in true talent, which is more that what we observe (which must be much larger than spread of actual talent).  I think a within-position SD of 5-7 runs is probably closer to the mark, 10 at the very outside.

Guy #38 - Exactly correct, an elegantly concise description of the problem.

Guy, the 15 runs is the sd at the team level that Chone is forecasting.

The within-position SD is 10 runs for UZR (it used to be anyway).  UZR seems tighter with BIS data.  And if you have 8 positions, then the team SD is 28 runs. 

In order to have a team-level 15 runs sd, then that means each of the 7 positions would have an SD of 5.7.

Following Tango’s suggesting, I tried projecting the Mariners’ defense through player-specific regression based on scouting reports. Basically, I just regressed a weighted 2007-2009 UZR rate for each player toward a regressed UZR rate for his most similar players (according to the fans’ scouting report). I doubt the standards of similarity I used in determining each players’ peers is really helpful. All of the peers I used were within ~.3 points of the players they were compared to (except in Ichiro’s case, where I had to go to ~.6 disparity), which is probably too fine a distinction, and it might be more worthwhile just to have like five big bins for elite, above-average, average, below-average, or bad (which stevesommer5 did at STL sports scene, which article you can read by clicking on my name). I did it this way, though, mostly because I wanted to see whether Gutierrez, Ichiro, Wilson, and Figgins could possibly improve upon their UZRs by being regressed to the level of the absolute best at their positions. I didn’t find that to be the case, though Figgins barely changed at all when regressed toward Rolen, Beltre, Inge, Longoria, Zimmerman, and Ian Stewart. Anyway, here are the results:

Gutierrez: +12.6
Figgins: +10.6
Ichiro: +9.7
Wilson: +9.7
Kotchman: +4.8
Lopez: -2.5
Bradley: -2.9
total: 42.0

My outfield projections had too much regression.  I did that because I wanted to lower the spread of the minor leaguers (some crazy numbers in there, data quality issues and lack of park adjustments).  But I hadn’t fixed it to leave the major leaguers alone.  Fixed now.  Changes a few players by a pretty good margin, but for teams doesn’t change much.  Projected standings have been updated and no team changed by more than a game.  Improved the defensive runs for the Mariners, but not the projected win total, partly due to rounding, partly because the net effect was positive for the majority of teams.

Also, My projections for those 7 match #42’s UZR + scouting reports almost exactly.

Gutierrez: +11
Figgins: +7
Ichiro: +9
Wilson: +9
Kotchman: +6
Lopez: +2
Bradley: -3
total: 41

OK, here are my projections
(5/4/3 seasonal weighting)
Gutierrez: +16.9
Wilson: +9.5
Figgins: +7.4
Kotchman: +6.0
Ichiro: +3.1
Bradley: +2.2
Lopez: +0.9
total: 46.0

and I have Michael Saunders at +15.6, but he’s -4.3 batting...defensive replacement for Bradley?

Summing by team all the players who I have projected for 2010 who appeared in mlb, with total projected (not mlb) playing time

defense
OAK +51
PHI +48
CLE +45
SEA +45
BOS +37
...
SDN -12
FLO -14
WAS -19
ATL -22
TOR -26

Tango/41:  sorry, misread your original post.  Still, if Nick’s data above (21) is correct, then the observed team SD is about 30.  That includes both random variation in actual fielding performance, and measurement error.  I would think that means the team true talent SD is what, in the 20 run range, maybe less?  And that would mean a player SD of about 7.  That seems much more plausible than 10 (I’d guess it’s really 5-6 runs).

I missed Nick/25:

BTW, from 2002-2008, team UZR has a standard deviation of 31 runs.  Team runs scored have an SD of 74 runs.  Team FIP runs allowed…

(((HR*13 + (BB*3) - SO*2)/(IPouts/3)+3.2)/9)*(IPouts/3) AS runs

...have an SD of 58 runs.

First to see if it makes sense:
off^2 = pit^2 + fld^2
off^2 = 58^2 + 31^2 = 66^2

So, that’s not right.  It has to be much closer.  I’ve been using a 5x ^ 2 = 4x ^ 2 + 3x ^ 2 kind of model.  so, that would mean:
75^2 = 60^2 + 45^2

The offense matches to what Nick said as does the pitching.  So, something is not adding up somewhere.

***

I revised it to 70 +/- 30.  All the forecast by you guys are at 40-60.  So, I don’t know that we’re that far off.

I had a huge problem with +28.  Since +42 is inside my 40-100 range, I really don’t have much of an issue.

First to see if it makes sense:
off^2 = pit^2 + fld^2
************
Is this a result from somewhere that you’re using?  b/c this doesn’t *have* to be true.

If you look at historically runs scored and runs allowed per game, you will find that the SD is roughly the same.

So:
off^2 = def^2

And, def^2 = pit^2+fld^2

All these equations work to the extent that they are all independent.  Which they pretty much are.

ok, so:

var(off) = var(def) from the data

so var(true tal. off) + var (luck off) = var(true tal. def) + var (luck def)

and I guess it’s not such a leap to say that var(luck def) = var(luck off), in fact, maybe *that* does *have* to be true, so that you can say:

var(true talent off) = var(true talent def)

it does surprise me, somewhat, that true talent pitching and true talent defense aren’t correlated but maybe it shouldn’t.

sd(FIP) + sd(UZR) may not equal SD team runs scored.  You also have to account for ballparks and pitcher hit prevention ability.

The new SD in my projection for fielding is 17.  If you split that into what the expected SD per position is, it’s not going to be uniform. You’ll get a much larger SD for positions with a lot of chances like CF and SS than 1B.

Part of the difference we had is whether people were counting catcher or not - Rob Johnson rates slightly below average, doesn’t look like they have a good defensive catcher on the roster.

The variance in real runs scored = runs allowed, but that doesn’t mean you should get the same thing on projected totals.  Pitchers are just much harder to predict than hitters - Chance of injury is greater and injuries can affect performance as well.  A team that builds on pitching would be underrated by projections if everything goes right, but probably have a greater chance of things blowing up - their seasons hinge on delicate tendons.

but that doesn’t mean you should get the same thing on projected totals.  Pitchers are just much harder to predict than hitters

Agreed, which is why I started off this thread as:

Now, Rally could say that there is so much more uncertainty in fielding and pitching than hitting that that’s why it looks like that. If so, then his forecast will undervalue a team you really think is pitching+fielding heavy… hence, why I and other would think his Mariners forecast is way too low.

So, I actually predicted exactly what you were going to say!

Doesn’t mean a forecast undervalues a pitching/defense heavy team.  They have more upside, but also more downside.

Clearly, next year’s projection analysis will have to include fielding projections.  If it’s fairer to have everyone predict a “stat” rather than a “metric” we could try to convince forecasters to project a team BIPr.  Otherwise, we could compare projected runs saved to UZR’s for individual players.

Sorry, Rally, but you are wrong here. 

Let’s say that you have no idea how to forecast fielding.  And you are just as good at forecasting hitting as you are pitching.

The spread in your forecasts will be this:
50 hitting
40 pitching
0 fielding

Therefore, any team that is fielding-heavy will be underforecasted, because the spread should be 50/40/30.

It’s the same principle in your example, just not as much.

Tom,

Actually you are wrong here. In your example, it is not that you have no idea how to forecast fielding—it is that fielding has a much smaller true talent spread (in your case, 0), but a high random variance. Any decent projection system will, in aggregate, correct identify true talent. You’ve showed that with the Marcels, and Rally showed that for his fielding system earlier in the thread.

So if Rally projects a low true talent spread compared to what is observed in the data, that simply means that there’s a lot more random variance in year-end fielding or pitching numbers than there is in year-end hitting numbers. Which makes total sense, by the way. If you want your projected pitching + fielding variance to equal your projected hitting variance because that’s how it works out in the seasonal data, go ahead, but in that case, I’d be happy to bet some money and take you to the cleaners.

Tango/56, that’s only true if the Mariners (or whatever other team) are significantly better at projecting defense than Rally.  Maybe they are because of their scouting services or whatever, if they are using those things properly.  But unless their forecasts for defense are legitimately more certain than Rally’s, then they can’t take advantage signing the best defenders any more than they can tell who they are from their projections.  So I don’t think you can say a fielding-heavy team will necessarily be under-forecast just because there is more uncertainty in the fielding projections.  For that to be true, Rally’s fielding projections have to be less certain than the team fielding projections.  Maybe they are, but that has nothing to do with how the variances add up.

I also think if the variances do add up, then you are probably forecasting with too much certainty, because you can’t forecast as much variance for pitching and fielding as for hitting if there is more uncertainty in those areas.  Shouldn’t the variances add up more like:

Var(Hit-forecasts)+Var(true talents around mean of forecast =
Var(Pitch-forecasts)+Var(true talents around mean of forecast +
Var(Field-forecasts)+Var(true talents around mean of forecast

For every forecast, there is some uncertainty, and there will be a variance of actual true talent levels for the players around the mean that you forecast for them.  Some of the players forecast at +5 runs will really be +10 talents and some will be +0 talents, but they should average +5 (and I’m not just talking about observed performance varying from projections, but about the actual true talents varying from the true talent estimated by the projection).  If the variance of true talents around the forecast mean is higher for pitching and fielding, then you can’t forecast as much variance, because then when you add in the variance from the uncertainty of the projections, you will have over-forecast the variance of run-prevention.

Just read Gassko/57, and I agree.  So maybe even the variance actual true talents shouldn’t add up, and it should be more like:

Var(Hit-forecasts)+Var(true talents around mean of forecast) + Var(random variation from true talent in observed performance)=
Var(Pitch-forecasts)+Var(true talents around mean of forecast) + Var(random variation from true talent in observed performance) +
Var(Field-forecasts)+Var(true talents around mean of forecast) + Var(random variation from true talent in observed performance)

because it is the variances of the observed performance in RS and RA that add up, and the observed performance also includes the random variation from true talent that Gassko talked about.  If the both the uncertainty of the forecasts and the random variation around true talent are higher for pitching and/or defense, then there’s no way you can forecast as much variance from them.

I’m right, so let me try to explain, because obviously I didn’t explain something.

We start wiht a truism:

off^2 = pit^2 + fld^2

If we know perfectly how to forecast from god, we’d get:
60 = off
48 = pit
36 = fld

If we have SOME uncertainty in off and pit, and total uncertainty (in our estimate) of fld we’d get:
50 = off
40 = pit
0 = fld

So, we KNOW the spread is 36 runs, but we can only estimate it as 0.

So, any system that has a spread in FLD that is too low will by definition estimate a team that is fielding-heavy as too low.

The problem is we don’t know which team actually is the fielding-heavy team.

My last line in post 60 is exactly what Kincaid is saying here:

Tango/56, that’s only true if the Mariners (or whatever other team) are significantly better at projecting defense than Rally.  Maybe they are because of their scouting services or whatever, if they are using those things properly.  But unless their forecasts for defense are legitimately more certain than Rally’s, then they can’t take advantage signing the best defenders any more than they can tell who they are from their projections.

I also think if the variances do add up, then you are probably forecasting with too much certainty,

Right, the TRUE variances will add up.  The forecasted variances will not, exactly for the reasons we’ve noted above.

Tango, are you saying that teams that are fielding-heavy are under-forecast, or teams that are built to be fielding-heavy are under-forecast (or specifically that the Mariners are under-forecast because they are fielding-heavy)?  The former is fine, but the latter two are not, because of what you said in post 60 about being unable to identify which teams actually are fielding-heavy.  You can say in general teams that actually are elite defensive teams are under-forecast, but you can’t apply that to any specific team because you can’t identify them.

We start wiht a truism:

off^2 = pit^2 + fld^2

If we know perfectly how to forecast from god, we’d get:
60 = off
48 = pit
36 = fld

I don’t think that is true, because of Gassko’s point.  The observed variances of RS and RA are close to the same, but the observed variances include the variance of true talent plus the variance due to random variation from true talent levels.  If fielding and/or pitching have more random variance from true talent levels over a season, then the variance of true talent levels can’t be as high for run prevention because there is more variance coming from variation outside the spread in true talent, and your forecasts from god would still have less variance for pitching plus fielding than for offense.

"You can say in general teams that actually are elite defensive teams are under-forecast, but you can’t apply that to any specific team because you can’t identify them. “

Yes, exactly.

***

“If fielding and/or pitching have more random variance from true talent levels over a season”

But they don’t, so the rest of the sentence doesn’t apply.

""You can say in general teams that actually are elite defensive teams are under-forecast, but you can’t apply that to any specific team because you can’t identify them. “

Yes, exactly.”

I’m confused. Isn’t this exactly what Rally was saying in #54 when you called him wrong? And doesn’t this directly contradict your original post, where you say “his forecast will undervalue a team you really think is pitching+fielding heavy”? You are saying now that if you pretend to know which teams are the elite defensive teams, you will *overvalue* those teams, because you can’t really identify which ones are elite.

Rally, it occurs to me that the way you did team projections might (if I’m understanding it correctly) be a good way to find a “league factor” (how much better the AL will be than the NL).  Did you have two teams swap leagues and seeing what the difference in projected wins was?

But they don’t, so the rest of the sentence doesn’t apply.

Is that an assumption, or do you know for a fact that is true (and if so, how)?  I can’t really think of a way to separate the variance of actual true talent around the estimated true talent from random variance of the observed performance around the actual true talent to test that (or even how you would define which is which if you could do that; for example, if a fielder feels sluggish one week and gets poor jumps that week and then feels great the next and gets great jumps, is that random variation around his overall true talent level, or is it just changing true talent levels?).  Although, I guess it’s not really relevant, since all we know is a player’s estimated true talent level, and what we care about is how much variance there is around that estimate, regardless of how much of it is because of variance of true talent around our estimate and how much is because of variance of performance around true talent, or regardless of how we define one vs. the other.

"Right, the TRUE variances will add up.  The forecasted variances will not, exactly for the reasons we’ve noted above.”

You simply do not know this. The only evidence you’ve given thus far is that the observed variances add up, which tells us nothing about the true variances.

“But they don’t, so the rest of the sentence doesn’t apply.”

If this is not the case, then the y-t-y correlation (or observed minus random variance—to-may-to, to-mah-to) of OPS and RA at the same number of plate appearances should be equal, which they are not.

"You can say in general teams that actually are elite defensive teams are under-forecast, but you can’t apply that to any specific team because you can’t identify them. “

“Yes, exactly.”

I think we’re on the same page with that statement.

J Cross, I can try that very easily.  Or anybody else can who downloads my spreadsheet.

I downloaded it so I can try to figure out what you’ve got going on there and give it a shot when I get the chance later.

I switched the Angels and Dodgers.  Angels go from 83 to 89 win in the NL West, Dodgers from 84 to 78.

Also, you can add up the records to see I’m forecasting a 135-117 AL record in interleague.

Good stuff, thanks.

You simply do not know this. The only evidence you’ve given thus far is that the observed variances add up, which tells us nothing about the true variances.

I think it would be kinda hard to create something with the flip-side of the same events, where the observed variances equal each other but that the underlying true of each don’t add up.

and, here’s the strength of schedule for each team (wins with neutral schedule divided wins with real schedule):

Team SoS
Cardinals 0.95
Cubs 0.96
Reds 0.96
Brewers 0.96
Rockies 0.96
Braves 0.96
Pirates 0.97
Padres 0.97
Dodgers 0.97
Giants 0.97
Astros 0.97
D-Backs 0.97
Nationals 0.97
Marlins 0.97
Phillies 0.98
Mets 0.98
Twins 1.02
White Sox 1.02
Indians 1.02
Rangers 1.02
Yankees 1.02
Red Sox 1.03
Angels 1.03
Tigers 1.03
Rays 1.04
A’s 1.04
Royals 1.04
Mariners 1.05
Orioles 1.06
Blue Jays 1.08

Sorry in advance for the table formatting.

Their team UZR is not great, but it is still +11 runs, which is around +13 runs for the season.

***

Brian’s Strasburg projection looks a lot better than Tom’s Mariners defense projection:

http://www.insidethebook.com/ee/index.php/site/comments/chone_team_forecasts/

Sorry, Tom…

David, did you post that to have fun, or did you post that with contempt?

Sorry if that wasn’t clear, it was out of fun. I’ve made plenty of wrong predictions in the past—no way I would want to be mean-spirited about that kind of thing. There’s nothing wrong with being wrong about a projection—it tends to add a little humility to the debate the next time around. I just happened to remember that thread, and figured it was worth bringing up again, since there was quite a bit of debate then.

Ok, good. 

Yes, in that one, it was important to note that the discussion point was not only about the mean but the uncertainty of the mean.  I think I ended up saying +70 +/- 30 or something, and most people were forecasting the M’s in the +40 to +50 range, and Rally (TZ) I think was lower than that.

UZR has them in 2010 at +11, and Dewan has him (after rebaselining to zero) at around +33.  So, they’re probably going to come in close to the +40, which is right where most of the readers had them as their mean, and I had them on the very low side.

Whether the Dewan numbers reflect the Mariners true fielding as the readers here expected them, or whether they reflect a disappointment relative to what I was thinking they should be, well, that’s the real discussion point, and one I can’t have.

The other one is the big gap that UZR has with Dewan here.