site banner

Nate Silver: The model exactly predicted the most likely election map

natesilver.net

Key excerpt (But it's worth reading the full thing):

But the real value-add of the model is not just in calculating who’s ahead in the polling average. Rather, it’s in understanding the uncertainties in the data: how accurate polls are in practice, and how these errors are correlated between the states. The final margins on Tuesday were actually quite close to the polling averages in the swing states, though less so in blue states, as I’ll discuss in a moment. But this was more or less a textbook illustration of the normal-sized polling error that we frequently wrote about [paid only; basically says that the polling errors could be correlated be correlated between states]. When polls miss low on Trump in one key state, they probably also will in most or all of the others.

In fact, because polling errors are highly correlated between states — and because Trump was ahead in 5 of the 7 swing states anyway — a Trump sweep of the swing states was actually our most common scenario, occurring in 20 percent of simulations. Following the same logic, the second most common outcome, happening 14 percent of the time, was a Harris swing state sweep.6

[Interactive table]

Relatedly, the final Electoral College tally will be 312 electoral votes for Trump and 226 for Harris. And Trump @ 312 was by far the most common outcome in our simulations, occurring 6 percent of the time. In fact, Trump 312/Harris 226 is the huge spike you see in our electoral vote distribution chart:

[Interactive graph]

The difference between 20 percent (the share of times Trump won all 7 swing states) and 6 percent (his getting exactly 312 electoral votes) is because sometimes, Trump winning all the swing states was part of a complete landslide where he penetrated further into blue territory. Conditional on winning all 7 swing states, for instance, Trump had a 22 percent chance of also winning New Mexico, a 21 percent chance at Minnesota, 19 percent in New Hampshire, 16 percent in Maine, 11 percent in Nebraska’s 2nd Congressional District, and 10 percent in Virginia. Trump won more than 312 electoral votes in 16 percent of our simulations.

But on Tuesday, there weren’t any upsets in the other states. So not only did Trump win with exactly 312 electoral votes, he also won with the exact map that occurred most often in our simulations, counting all 50 states, the District of Columbia and the congressional districts in Nebraska and Maine.

I don't know of an intuitive test for whether a forecast of a non-repeating event was well-reasoned (see, also, the lively debate over the performance of prediction markets), but this is Silver's initial defense of his 50-50 forecast. I'm unconvinced - if the modal outcome of the model was the actual result of the election, does that vindicate its internal correlations, indict its confidence in its output, both, neither... ? But I don't think it's irreconcilable that the model's modal outcome being real vindicates its internal correlations AND that its certainty was limited by the quality of the available data, so this hasn't lowered my opinion of Silver, either.

9
Jump in the discussion.

No email address required.

The sad thing is that the “correlated errors” aren’t based on polling data or past results, they are just an arbitrary adjustment he adds to the model based on feels. Like he literally codes in “the sun belt states are likely to be correlated” etc.

After this election I am totally convinced Silver a fraud. He simply can’t admit that there is a polling industry bias. His techniques make it impossible to account for this accurately because they are based on weighted polling averages, where really he needs to add a bias term, which he refuses to do.

To elaborate on that, if literally all the polls miss left, you can’t fix that with weighting. In reality, he would have needed to put all of the weight on AtlasIntel and Rasmussen and close to 0 on everything else. This shows that weighting is the wrong approach.

Edit: He does have “house effects” but this adjust polls towards the weighted average, not towards historical results. So it doesn’t help.

Yeah that article where he explained the house effects modelling had me screaming at my monitor.

Like, you've noticed that pollsters are herding, and then you correct for house effects by... measuring how different their predictions are from the median pollster!? WTF Nate.

The bias term is the polling error. The reason he treats it as an error rather than a predictable bias is because he doesn't think it's predictable. Assuming it is predictable based on two elections where it was actually pretty different, even if it was in the same direction both times (something that had a 50% chance of happening even if it were completely random) risks over fitting the model.

To elaborate on that, if literally all the polls miss left, you can’t fix that with weighting. In reality, he would have needed to put all of the weight on AtlasIntel and Rasmussen and close to 0 on everything else. This shows that weighting is the wrong approach.

No, it shows there was a polling error. Let's say he follows your advice and the polling error is in favour of the Democrats in the next election. Then his model would have been really really inaccurate.

Of course, if there really is a consistent bias in favour of Republicans, then it would make it more accurate, but there isn't much data to make that assumption.

The reason he treats it as an error rather than a predictable bias is because he doesn't think it's predictable.

And this is why he is an idiot. The pollsters all understand at this point that it is inherently due to a predictable non-response bias. As a fall back, many used the recalled vote to reweight the sample. But given the unusually high turnout for Dems in 2020, the recalled vote begs the question and was a sandbag for Trump.

Understanding this, unlike professional idiot Nate Silver, I made some heavy bets for Trump and won a good chunk of change.

How do you know you didn't just get lucky with a coin flip?

How do I know? I know because I know that my reasoning was solid and strongly supported by the available data.

How do you know that I know? You don’t. But I’m not really here to convince you. I’m here to make fun of Nate Silver for predicting nothing yet declaring victory.