THE BOOK--Playing The Percentages In Baseball

Bottom of the ninth, with a 1 run lead?  The % might work out but it seems like there’s a much bigger lead but with a lot of time left.  Maybe its more like top of the 7th inning with a 3 run lead, or something like that.  McCain needs to draw a couple walks or get a couple dribblers to go through and then hit a home run.  A solo homer isn’t going to even things up.

It feels like McCain is down by more than that because he’s such an inferior natural politician. That’s not a partisan remark. He may be a better president if elected, but I think it’s pretty clear that Obama is the far superior politician.

So it’s like bottom of the 9th, 1 out, down by 1 run, McCain batting, Obama pitching. But McCain has Mario Mendoza battitng with Tony Pena Jr. on deck and Obama has Mariano Rivera on the mound.

OK that’s maybe a little harsh, but you get the point. Maybe it’s more like Paul Lo Duca facing Kerry Wood, but regardless Obama has the better match-up.

Bottom 7th, 1 out (to continue with the Palin being picked off 1B), down by 3 is an 89% win%.

We actually need to know the Leverage Index to know which inning we are in. 

Bottom of 7th in that situation has an LI of just 0.8.  Bottom of the 9th, it’s 2.8.

So, how much does news move the needle?  That’s how you know whether you are in the 7th or 9th inning, if the win% is 89%.  Maybe someone can ask Nate about it.

It seems like we are closer to the 7th than the 9th.  Good call…

I’m not sure about WE’s for football or basketball, but those seem like better analogies than baseball for the presidential election. Maybe a ten-point lead in basketball with four minutes remaining or a two touchdown lead with seven minutes for football?

Oh, just read over #3 again. That’s what I was trying to say

Steve, I wonder whether the political talents you identify in Obama and McCain actually reflect an illusion similar to the one that has many people perceiving, for example, Jack Morris to have been a better pitcher than Bert Blyleven. It is hardly conventional wisdom, but I wonder whether if you switched Obama in time with Kerry, Gore, Mondale or Dukakis (like Obama all bright, worldly, articulate liberal Democrats)it would change any outcomes.  Sometimes historical context distorts the way we perceive individual talents, and while we may perceive Obama as a more talented politican that Kerry, Gore, et al, or McCain for that matter, that may simply be a function of Obama’s being the benficiary of a particularly pro-Democratic set of current historical circumstances [full disclosure—I consider myself a liberal Democrat and am an ardent Obama supporter in the current campaign].

I don’t think Dukakis would draw 200,000 people in Germany.

In hindsight, and based on more recent events, I don’t think I’d call Gore “bright”, though I suspect things might have been different when he was VP (during my childhood).

I thought this was going to be about Nate Colbert . . .

But Germans don’t vote, and they had free beer and a concert. Sarah did draw 70,000 in central Flroida. The family and I are going to see her Saturday, but the hockey arena only fits 4000 or so (obligatory hockey reference).

To me, it feels like Palin doubled, and McCains taken a couple called strikes.

In 2004 I dabbled with basically the same model that Nate is using, in analyzing the polls to derive a win pct in each district, then run a simulation a thousand times or so, and come up with a histogram. The Dems achieved well over what I derived from the polls, so I got depressed and didn’t do anything else with the project. Now Nate’s on TV (honestly, although I’ve heard of Colbert, I’ve never seen him).

One thing that Nate does at 538 is after weighting the polls for sample size, age and past accuracy of pollster, he then also mixes in historical voting in each state. Except for DC (arguably most liberal district) & HI (Obama’s home state) the 49 others all had a projection 2-6% more in favor of Obama than the straight polling weighted mean. Looking at only the polls (as of Mon 6 Oct) McCain needed FL (-0.5) and VA (-1.2) to go ahead in electoral votes, assuming he holds all the others. That was the mimimum needed to flip the Electoral College. The big but is that the past two weeks the polling trend has been steadily towards Obama. Will Obama continue gaining ground, or will he peak at some point before the election?

The computer I was on at work just upgraded to IE8, and didn’t display every post...now at my old computer, I get a bette context of the thread

I agree, Dukakis wouldn’t draw 200,000 anywhere, even with free beer and a concert. Dukakis is boring, Obama definitely is not. Obama, Bill Clinton and Ronald Reagan are/were very charismatic and can touch voters on that level, which Mondale, Dukakis, Bush Sr, Gore and Kerry didn’t have.

I want my leaders to be have ideology, knowledge, judgement and at least a minimum of experience running big things, but if they cannot communicate with the voters they will likely lose the election.

Brian/10 -

I don’t think historical voting figures have more than a tiny effect, which would be included in the “538 Regression” he uses for each state. Those regressions only have a significant effect on states that haven’t been polled much, so it’s not much of an issue.

The main cause for the discrepancy between the straight polling mean and the projections on Nate’s site is that he’s making a “trend” adjustment based on when the poll was conducted. Basically, he’s assuming that the state polling is moving proportionately with the national polling. So, if a mid-September Virginia poll was taken that showed McCain +2 when the national race was even and the national race today is Obama +6, he’s adjusting for that national swing of 6 points, making it Obama +4 today.

So, overall, his weighted polling average for Virginia is Obama+2.6, but his “trend-adjusted” number is Obama+7.9.

I am an almost daily visitor to 538

I thought the age of the poll was one of the factors going into the weight, and thus would should up in the polling average line for each state.

I perceived a bias of sime sort when comparing the projection with the polling average, as 49 of 51 electoral districts show it in the same direction, in roughly the same amount. It’s a matter of determining what caused the bias, and if it’s accurate, or a flaw in the modeling.

As this election is done by district, I would not regress it to the national average, which IMO means nothing. CA can vote 100% Democrat and skew the national total, but every vote over 50% is an excess vote and counts nothing additional towards who wins in that district or nationally.

If CA votes 100% Democrat it certainly means something, insofar as you can INFER how the other 49 states may vote.  Not 100% obviously, but that’s what regression is all about. If all you knew was CA votes 100% Democrat, then you may be able to infer that NY would would at least 75% Democrat, and some heavy republican state might be at least 35% Democrat… all depending on the relationship as to what one can infer from the other.

That is what regression is all about, and NEVER can you say that one piece of data means nothing to inferring something else.  It’s all a matter of degrees.

Tom, I understand your point, when there’s a lack of data you need to find some kind of proxy. But you also know that political affiliation is not uniformly distributed geographicly. That’s the whole idea of “packing” during redistricting.

It is true that the two largest states have Dem majorities, and that creates a larger number of excess votes there. Should that be part of the formula in analyzing poll results in FL or VA? Right now, I don’t have as much confidence in the projection numbers as I do in the polling means. I don’t know everything that goes into the projection, and I’m observing a bias in the model. Not saying it’s intentional, just pointing it out as a possible desgin flaw. I think there’s a lot of similarities between what Nate has done with his 5238 projections and what we do here.

So now the ad at the bottom of the page is “Get Stephen Colbert Rongtones!” LOL

Nate goes into tremendous detail regarding the methodology here:

http://www.fivethirtyeight.com/2008/03/frequently-asked-questions-last-revised.html

If you scroll down, you’ll get to “Step 2. The Trendline Adjustment.”
I didn’t read all the previous entries he links to, but it seems that the trendline adjustment varies somewhat state-to-state based on demographics.

I had read these before, but it probably had been a couple months ago. Steps 2 thru 5 are further massaging of the polling data, and this trend analysis currently gives Obama another 2-6% in virtually all the states. The questiuon I had in my first post is “will the trend continue?”. I love what he’s done with combining polls to give us a more accurate picture of today’s real vote pct. It’s the trend analysis I am not yet sold on. “If things continue like they are now” then it’s probably “accurate”, but anything can happen between now and then to upset that.

1. Polling Average: Aggregate polling data, and weight it according to our reliability scores.

2. Trend Adjustment: Adjust the polling data for current trends.

3. Regression: Analyze demographic data in each state by means of regression analysis.

4. Snapshot: Combine the polling data with the regression analysis to produce an electoral snapshot. This is our estimate of what would happen if the election were held today.

5. Projection: Translate the snapshot into a projection of what will happen in November, by allocating out undecided voters and applying a discount to current polling leads based on historical trends.

6. Simulation: Simulate our results 10,000 times based on the results of the projection to account for the uncertainty in our estimates. The end result is a robust probabilistic assessment of what will happen in each state as well as in the nation as a whole.

My column last week at MVN was about aging curves. I used a Marcel based system with MLEs, and have no problem doing a one year adjustment to correct age bias in the projections (guys in their 20’s project low, so bump them up for next year, while guys in their 30’s project high, so bump them down). Clay Davenport’s peak translations say that Neil Walker, a 21 year old, 4 year pro in AAA, will peak as a 27 HR guy. I look at his year to year normalized numbers, and the standard aging curves, and say Bull! even though I’m still not comfortable projecting more than one year at a time. Clay has his formula that says Walker will change that much from now (13-15 HR) til 6 years in the future. At least in this example, I just don’t see it. It’s a matter of each of us designing different models, with different assumptions.

cannatar/12 is basically right about how Nate does it.

So, if a mid-September Virginia poll was taken that showed McCain +2 when the national race was even and the national race today is Obama +6, he’s adjusting for that national swing of 6 points, making it Obama +4 today.

The details are complicated, though. Each state is given a sensitivity rating between 0 and 2. Pretend that VA has a sensitivity of 0.8. That means a national swing of 6 points is supposed to correspond to a swing of 6 * 0.8 = 4.8 points in VA. So, the McCain +2 poll would be adjusted to Obama +2.8.

This raises the question of how Nate computes the sensitivity ratings. He explains that on this page:

http://www.fivethirtyeight.com/2008/06/construction-season-over-technical.html

The ‘m’ parameter in that link is what I have been calling the sensitivity rating.

Long story short, it is a kludge but it probably works reasonably well. Reading 538 has given me insight into Nate’s overall approach towards modeling, which appears to be:

1. Come up with a model.
2. Look around for something that the model systematically gets wrong.
3. Create an adjustment to deal with the systematic error.
4. Layer the adjustment on top of the model’s other calculations.
5. Go back to step 2.

The result is something that’s quite accurate but rather ugly in the details.

Another link that is interesting to read:

http://www.fivethirtyeight.com/2008/05/michigan-what-would-have-happened.html

because Nate later commented that it is “basically a lo-fi version of PECOTA.” Indeed, that link really gave me a good idea of how PECOTA works. (For all I know, there’s something much better on BP, but I’m not a subscriber.)

I have a quite similar approach to modeling, and I’m fine tuning my Marcel based with MLE projections.

The “battleground” or “tipping point” states are each getting polled like ten times a week. We do not have a lack of data in those places. Where there is a large volume of polls, new ones every week, I would probably want to age the older ones rather quickly as we have enough newer info to replace them.

I’m still not comfortable with trending into the future. All this CO2 will make it so much hotter 100 years from now, except that the sun stopped making spots a few months ago. Maybe a mini ice age coming. Obama peaked in June, and was crashing in August, until the banking system went under and McCain couldn’t find a way to explain that it was government mandates and not deregulation that created subprime loans. Now Obama’s higher than he was in June. Will he keep climbing? Will the polls stay where they’re at? Will something else happen to swing things back McCain’s way? No way in hell I think I know the answers. I’m more comfortable looking at where we are now. Guess that’s why I’m so depressed.

Brian, I still think you’re misstating what Nate is doing. The process you’re taking issue with isn’t Nate projecting past polls into the future, it’s him projecting past polls to the present.

He does do a separate final step that projects the future (from “snapshot” to “projection"), but that just assumes that the race will tighten slightly (it’s currently a pro-McCain adjustment).

The analogy of 7th inning, down by 3 seems reasonable, but I find it depressing.  My opinion on these candidates is that I’d prefer it to be before the 5th inning, with a heavy chance of rain.

Apropos of #20’s third paragraph, tracking political polling may be unlike baseball win expectancy in that the answers that prospective voters give to pre-election poll-takers have no binding effect and have no rules-based cumulative effect on the outcome of the election.  If Boston leads the Yankees 5-2 in the seventh inning, Boston can never lose those five runs, they are Boston’s to keep.  The Yankees can catch Boston only by scoring runs of its own.  In the pre-election polling of Obama/McCain, however, if on October 8 50% of respondents say they would vote for Obama (as against say 42% who say McCain), Obama does not get to “keep” that 50%, or indeed any portion of it.  Any event that persuades people to change their minds affects not only McCain’s total but Obama’s as well.  Baseball games are cumulative—the results of each inning cumulate to the final score.  Election polling is not cumulative in that sense—every poll before Election Day is a sort of late September game for two teams that have already clinched the playoffs: interesting as an indication of the talent of the teams but without binding impact on who will win the championship when the time finally arrives.

Re: my comment about Obama’s superiority as a politician, you have to consider that this is a guy who was nowhere 4 years ago, got a bit lucky by having all the opposition implode in his senate race, but since then has used a single convention speech to catapult himself over the Clinton machine to be the top Democrat in America. And he’s black too, you know. It’s an unbelievable achievement. Comparing Obama’s raw ability as a politician to Dukakis or Kerry or Gore just seems crazy to me.

As for McCain, he’s ok, but his grumpy old man demeanor and inability to put together anything resembling a strategy puts him in the lower tier of presidential candidates in my lifetime. I could see an argument for him as a pretty skilled campaigner in certain settings, but I can’t see how anyone could argue that he’s as good or better natural politician than Obama.

Gore, BTW, had enormous advantages in 2000. Strong economy, popular incumbent, less than overpowering opponent--it took an incredibly weak performance on Gore’s part for that election to ever be close.

Steve, my guess is Gore would have won easily in 2000 but for two words that had absolutely nothing to do with him: Monica Lewinsky. Clinton’s embarrassing personal habits, combined with a virtual coup attempt by the GOP trying to taking advantage of such habits, tarnished everyone associated with Clinton, to some extent much the same as W.’s eight years of deception,incompetence and ideological extremism have tarnished McCain, to a degree unfairly.  Clinton was deservedly popular in 2000, but he was also, as a result of the Lewinsky affair, a joke, and that handicapped Gore, who had been tied directly to him for eight years.  And of course Gore won anyway.

I still suspect (though of course none of us have any way of knowing) that Obama would not have gotten as far as Dukakis, Mondale, Kerry or Gore if you were somehow able to lift him out of time and have him run in the years they ran.  But that is kind of like asking how well Joe Torre would have done managing the Dodgers of the 1950s, or the Giants of the 1960s—there’s just no objective way to assess such matters.

21 - That’s the way I thought it was, working in past historical data, when I started commenting in this thread, but I was told I was wrong.

24 - I am a lifelong conservative Republican, and I agree with absolutely everything you said.

Since we’re tossing around political opinions, why not weigh in.  I think Gore lost in 2000 because he played it too safe.  He’s a really smart guy, but he was too interested in not ruffling feathers to show voters his intelligence and passion.

Of course, he also lost because he got completely and totally screwed.  I say this without bias… we’re all “stats guys” here… someone made this amazing scatterplot of the Buchanan votes in Florida counties.  There was a nice picturesque correlation between votes for Buchanan in 1996 and 2000… Except for a few counties that used those butterfly ballots, and apparently had a huge change in attitude towards Mr. Buchanan.

an unbelievably weak performance by Gore, yes, and ALSO by the “liberal media” (scarequotes signal contempt for the ideological bullshitters who think the media is or was actually liberal.) take a look at, eg, media critics on the bullshit"gore sigh” phenomenon--the daily howler, etc. 

there’s a certain irony, then, in the focus on McCain’s alleged reluctance to handshake, refusal to make eye contact, etc.  I’d say, were I sufficiently cynical, that the MSM has decided Obama is in fact a cautious centerist (I would argue that the first black president of the US almost has to be a cautious centerist, but that’s another topic).  having decided such, and realizing he won’t actually raise taxes on nice journalists making $200 K or so, or reverse the American policy of ruinous, ill-conceived foreign wars, then hey presto!  they like him....and they can make fun of McCain (who, as a fake maverick, is indeed eminently mockable) to their heart’s content.....

I was trying to keep my conversation to how campaigns are run, and the the technicalities of analyzing polls.

Yeah, it’s too bad when a good technical discussion degenerates into a political flamewar. The comments section at 538 used to be very good, but for a while now it’s been nearly unreadable.

Really though, I wanted an excuse to post this 538 comment, which is possibly the dumbest argument I have heard from either side:

Perhaps the Obama fan club on this site should remind themselves that we live in a DEMOCRACY - Obama is a SENATOR - that is part of the LEGISLATIVE branch - he should not be running for PRESIDENT (EXECUTIVE BRANCH) - haven’t you heard of SEPARATION OF POWERS???

*MCCAIN 2008*

http://www.fivethirtyeight.com/2008/07/todays-polls-724.html#comment-1532975799084240345

Words fail me.

32 - Yeah, that gets a huge “Huh?” out of me.

But I do wonder, if we have three co-equal branches of government, and the President can veto Congress, and Congress can override the President, and the Court can overturn Congress or the President - why can’t Congress and/or the President overturn the Court?

Because the Court said so.

I would recommend that anyone interested in the separation/balance of branches of government read (or re-read) Federalist Paper 51 by Madison.

http://www.constitution.org/fed/federa51.htm

re #34 (Brian Cartwright):
The legislative branch could overturn the supreme court by ratifying an amendment to the constitution.

[I am not an US citizen and live in Europe, for this reason, I don’t know the technicalities of how that would be done, but I am sure you get my point.]

re #34:

Please refer to Federalist #81 by Hamilton on judicial review.  Among other things, he notes that this role of the courts is not novel and existed in state courts and, prior to that, in English courts and the House of Lords.  So, it was recognized by the authors of the constitution.

The Anti-Federalist Brutus was a lot more prescient about the Court than the Federalists were.

Brian #34: Note that the President and the Congress (the Senate part of it) appoint the members of the Court in the first place.

Also Congress and the President can overrule the Supreme Court easily on most of the issues that the Supreme Court deals with, which are matters of statutory interpretation. In such cases if Congress doesn’t like the Court’s interpretation it need merely change the statutory language and get the President’s signature or override a veto.

Only in the limited number of SCOTUS cases that are based on U.S. Constitutional interpretation can the Court only be “overruled” by Constitutional amendment.  The most important of these cases tend to turn on the Consitution’s protection of minority or personal rights against the rule of the majority.  Were such decisions subject to being overruled by majority decision the very concept of such protections would be at risk.

Part of the confusion raised by your comment #34 may be the use of the terminology “co-equal branches” --the phrase itself is a civics class oversimplification.  The relationship of the three branches is complex, with each playing very different roles; whether these roles are “equal” is really meaningless issue.  Are the roles of the catcher and shortstop “equal”?  Not really an interesting question—they are just different.

One other thought on Brian’s #32.  The sarcasm regarding the concept that in Marbury SCOTUS arrogated to itself the authority to definitely interpret the Constitution is somewhat misplaced.  Marbury was decided more than 200 years ago.  If the concept of judicial review articulated there was somehow inimical to the structure or philosphy of US government it would have been long since overcome or disregarded.  In fact, because it is so critical to the balance of majority and minority rights that is a hallmark—if a continually contested one—of American society the concept of judicial review has stood the test of time and vast swings in the balance of views within the country.  SCOTUS is the ultimate arbiter of Constitutional interpretion not “because it said so” but because the concept has worked for centuries and become an accepted and crucial part of the nature of American government.

I have my preference but I think we all can agree on this point-either Obama or McCain would be a significant upgrade over the replacement level president we currently have in the white house....

I’m sure it weighs the influence of FNC (the faux news channel) on the electorate.... 

Is anyone bothered that a “news” channel, when they run their viewer votes, gets 85-90% “McCain wins debate” or “Palin wins debate”?  Clearly, the news channel is attracting a highly skewed sample of the population.  If this was MTV or VH1 or BET or ESPN, that’s fine.  Those channels specifically target a non-random sample of the population.  But, one would hope that a news channel, while not necessarily completely impartial, would attract a viewership that is not so incredibly skewed.

I imagine CNN.com and MSNBC.com also run online polls.  What kind of numbers do they show?