Key excerpt (But it's worth reading the full thing):
But the real value-add of the model is not just in calculating who’s ahead in the polling average. Rather, it’s in understanding the uncertainties in the data: how accurate polls are in practice, and how these errors are correlated between the states. The final margins on Tuesday were actually quite close to the polling averages in the swing states, though less so in blue states, as I’ll discuss in a moment. But this was more or less a textbook illustration of the normal-sized polling error that we frequently wrote about [paid only; basically says that the polling errors could be correlated be correlated between states]. When polls miss low on Trump in one key state, they probably also will in most or all of the others.
In fact, because polling errors are highly correlated between states — and because Trump was ahead in 5 of the 7 swing states anyway — a Trump sweep of the swing states was actually our most common scenario, occurring in 20 percent of simulations. Following the same logic, the second most common outcome, happening 14 percent of the time, was a Harris swing state sweep.6
[Interactive table]
Relatedly, the final Electoral College tally will be 312 electoral votes for Trump and 226 for Harris. And Trump @ 312 was by far the most common outcome in our simulations, occurring 6 percent of the time. In fact, Trump 312/Harris 226 is the huge spike you see in our electoral vote distribution chart:
[Interactive graph]
The difference between 20 percent (the share of times Trump won all 7 swing states) and 6 percent (his getting exactly 312 electoral votes) is because sometimes, Trump winning all the swing states was part of a complete landslide where he penetrated further into blue territory. Conditional on winning all 7 swing states, for instance, Trump had a 22 percent chance of also winning New Mexico, a 21 percent chance at Minnesota, 19 percent in New Hampshire, 16 percent in Maine, 11 percent in Nebraska’s 2nd Congressional District, and 10 percent in Virginia. Trump won more than 312 electoral votes in 16 percent of our simulations.
But on Tuesday, there weren’t any upsets in the other states. So not only did Trump win with exactly 312 electoral votes, he also won with the exact map that occurred most often in our simulations, counting all 50 states, the District of Columbia and the congressional districts in Nebraska and Maine.
I don't know of an intuitive test for whether a forecast of a non-repeating event was well-reasoned (see, also, the lively debate over the performance of prediction markets), but this is Silver's initial defense of his 50-50 forecast. I'm unconvinced - if the modal outcome of the model was the actual result of the election, does that vindicate its internal correlations, indict its confidence in its output, both, neither... ? But I don't think it's irreconcilable that the model's modal outcome being real vindicates its internal correlations AND that its certainty was limited by the quality of the available data, so this hasn't lowered my opinion of Silver, either.
Jump in the discussion.
No email address required.
Notes -
Massive cope. His model got a few small things right, it got the big things people actually care about wrong. As usual he will hide behind the “things with <50% probability still happen!” defense, but this is just sophistry as it was never 50/50
He liked to tout how in 2016 he allowed the errors of different polls to be correlated, rather than being purely independent experiments. But at the end of the day all this does is increase the uncertainty and expand the error bars. If you keep doing this and allowing for more error here or there it tends your “prediction” towards throwing up its hands and saying “idk it’s a coin flip”, which is what happened to Nate and why he had to shut off his model so early on election night. He did plenty of bitching about “herding” of polls while he himself was herding towards coinflip. His big brag in 2016 was ultimately that he had herded towards 50/50 harder than anybody else.
In the prediction thread I called this for Trump with high confidence and said it was an “easy call” because there was ample evidence there for those with eyes to see. 2020 was an extremely close election and by every metric (polls, fundamentals, vibes, registration, mail in voting) Trump was better positioned this year than he was then. Nate can call everything a coin flip and cope all he wants but his credibility is shot
He wasn't herding. "Trump can win this" was a contrarian viewpoint among people who see themselves as nonpartisan observers of public opinion.
More options
Context Copy link
That isn't what herding means. Herding doesn't necessarily imply putting the finger on the scale towards a close race, it implies marshalling your results towards the average of other pollsters. After all, if everyone else is predicting a blow-out win and you predict a close race, you still look not only stupid , but exceptionally stupid if it is in fact a blow-out, whereas if you follow the crowd and predict a blow-out you only look as bad as everyone else if it turns out close.
Nate was doing the opposite of herding, if anything, in 2016. If Hillary wins that election very easily, Nate (possibly unfairly) looks stupid for constantly warning people about the significant possibility of a Trump victory. He looks good from 2016 precisely because he didn't follow the crowd of other modellers and gave Trump better odds than anyone else.
More options
Context Copy link
A child, when introduced to the concept of probability, gives equal weight to the possible outcomes. Two choices means 50/50 (a coin flip.) A pollster that isn't better than a coin flip is useless. You might as well ask a child. (I believe the children's election - 52/48 in favor of Harris - being +2 D, while being wrong, was more accurate than any of the left-leaning pollsters could muster.)
It's not useless if it's actually 50-50.
I get so triggered by this logic because it’s so wrong. Elections are not a football game. They are not actually a random variable. On November 4th the result was already set in stone, unless one of the candidates died or something. You could replay November 5th 1000 times and Trump would win 1000 times. It wasn’t 50/50. It can never be 50/50. It is always 100/0.
Epistemic uncertain is a feature of the model and its inputs, not some inherent feature of the real world. There was enough data to conclude with relatively high certainty that Trump was on pace to win. Nate’s model didn’t pick up on this because it sucks. It has high epidemic uncertainty because it’s a bad model with bad inputs.
There have certainly been elections which were decided by tiny margins. They might well decided by the contrast in weather between the red and the blue part of the state. Now, you can say that Nate's model sucks because it does not sufficiently predict the weather months in advance.
We can score predictors against each other. A predictor who gives you a 50/50 on anything, like 'the sun will rise tomorrow' or 'Angela Merkel will be elected US president' will score rather poorly. ACX had a whole article on predictor scoring. If there is someone who outperforms Nate over sufficiently many elections, then we might listen to them instead. "I bet the farm on Trump, Biden, Trump and doubled my net worth each time" might be a good starting point, especially if their local election prediction results are as impressive.
Unfortunately, I have not encountered a lot of these people.
More options
Context Copy link
More options
Context Copy link
If it was actually 50-50, why did he take down his real time election result projection?
I don't know anything about that or what point you're making.
More options
Context Copy link
He admitted well before E-day that anything more than a very crude real time projection needed far more resources than he had - he borderline told people to just go and look at the NYT needle, and probably only did his thing because it wasn't known until the last minute whether the needle would be active.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Everyone on the right called it with high confidence this time, unlike 2016 and 2020. Everyone on the left seems to call it for their guy with high confidence every election, so Dem/left predictions carry no weight. Nate will maintain his (undeserved) credibility by still being more accurate than most on the left.
Plenty of right-wing figures do this too, and got resultant egg on their faces in 2018, 2020 and 2022. It's hard to quantify but there are definitely a lot of left-wing pessimists and I don't think partisan boosterism is more prevalent on one side compared to the other.
In fact on twitter there were a lot of big right wing accounts predicting a Kamala win (legitimately or otherwise) shortly before the election.
More options
Context Copy link
And everyone who was right was a genius and everyone who was wrong was a fool (or a fraud) apparently.
This is not how probability works.
It's incredibly lazy to say that 'everyone on the right' and 'everyone on the left' called something' to make the specious point that your opponents statements are not meaningful. You might as well be saying 'my ingroup is better and more intelligent' than the outgroup'.
More options
Context Copy link
More options
Context Copy link
Seriously.. Of course he is going to claim credit but he shouldn’t get credit for having his model hedge more than others. But apparently I don’t understand statistics because I think he shouldn’t get credit for hedging more.
Why shouldn't he get credit for hedging his model more than others?
Because absolutely no last-minute polls existed that justified his sudden shift the day prior to Election Day 2016. Nate knew something was wrong with the polling, and put his thumb on the scale to make Trump look better than his model said.
More options
Context Copy link
He should get credit for being well-calibrated. If he is always right with his confident predictions and mostly right with his hedged predictions, then he is doing the right thing.
More options
Context Copy link
It did incentivize him to hedge more and more until we get to this ridiculous point where he tries to take credit for a 50/50 prediction.
The problem is you can't evaluate how well the model did based on just the probability of winning, except for the correct or not question, and he was not correct in 2016. Maybe whatever he is doing in this post is good, but it sure didn't translate to the final prediction, so he doesn't get credit for that either.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link