Which forecasters lean Democratic or Republican?
We compared 15 election forecasters, including ourselves, on the same 711 elections from 2016 to 2024: which way each one leans, how much of that is just the year, and whether leaning costs accuracy.
The short version
- The year swamps everything. In 2020 forecasters on average gave Democrats 13 points more win probability than their results earned; in 2022 they gave them 11 points less. Every forecaster moved together.
- Against each other, on the same elections, the differences are small: most forecasters sit within 2 points of the pack. 6 lean Democratic and 3 lean Republican by more than chance.
- VotePredictor leans Republican against the others, by 3.5 points. On its raw record it is the closest of all to even (R+0.7), because the field overstated Democrats in most of these years. It cost us in 2018.
- Leaning further from the pack, either way, goes with being less accurate (correlation 0.58).
What “lean” means for a forecaster
Most forecasters publish a win probability, not a vote margin. So a forecaster's lean is how much more likely it said Democrats were to win than they turned out to be. If a forecaster gives the Democrat a 70% chance in ten elections and Democrats win six, it leaned Democratic by 10 points: it expected seven wins and got six.
Measured that way, the year decides almost everything. These are the averages across every forecaster, election by election:
Average Democratic win probability minus the outcome, across forecasters, in percentage points. Positive: Democrats did worse than forecast.
Forecasters read the same polls, so when the polls miss, they all miss. In 2020 the polls overstated Democrats by about 5 points and forecasters followed them. In 2022 the expected “red wave” didn't arrive, and forecasters had given Republicans too much credit. The table shows how closely each forecaster tracked the year.
| Forecaster | 2016 | 2018 | 2020 | 2022 | 2024 |
|---|---|---|---|---|---|
| Race to the WH | · | · | D+7.9 | R+7.7 | D+9.4 |
| CNalysis | · | · | D+6.1 | R+12.0 | D+11.8 |
| The Economist | · | R+5.6 | D+15.1 | R+8.6 | D+8.5 |
| Princeton (PEC) | D+8.1 | · | D+7.5 | · | · |
| DDHQ | · | R+0.9 | D+14.2 | R+12.0 | D+6.1 |
| FiveThirtyEight | D+6.5 | R+0.7 | D+13.1 | R+8.9 | D+7.2 |
| Inside Elections | D+7.6 | D+1.5 | D+7.0 | R+9.3 | D+6.0 |
| Sabato | D+8.1 | even | D+12.6 | R+12.8 | D+6.7 |
| JHK Forecasts | · | D+5.4 | D+12.8 | R+12.1 | D+7.5 |
| Split Ticket | · | · | · | R+12.6 | D+7.4 |
| Cook | D+5.0 | R+2.6 | D+13.1 | R+9.5 | D+5.2 |
| Elections Daily | · | · | D+3.1 | R+11.8 | D+8.2 |
| Fox News | · | D+0.6 | · | R+12.4 | D+6.3 |
| VotePredictor | D+3.9 | R+7.1 | D+9.1 | R+9.1 | D+1.3 |
| RealClearPolitics | · | D+0.4 | D+1.1 | R+18.0 | D+2.9 |
Average Democratic win probability minus how often Democrats won, in percentage points, on the elections each forecaster covered that year (at least 30). Hover a cell for how many more or fewer Democratic wins the forecaster expected than happened.
Every forecaster's lean, against the others
To see a forecaster's own lean, we compare it with the other forecasters on the same elections: its Democratic win probability minus their average. The year cancels out. The dot is the average; the bar is a 95% range. Forecasters need at least 90 elections shared with others to appear.
Percentage points of Democratic win probability, relative to the other forecasters' average on each election. Hover a row for its range and number of elections.
RealClearPolitics leans Republican by the most, 5.6 points. Most of that is 2022, when it expected a bigger Republican year than anyone; it was also the least accurate. At the other end, Race to the WH, CNalysis and the Economist lean Democratic by 1.5 to 3.5 points. FiveThirtyEight leans D+1.1, about the middle of the pack.
Where we sit
VotePredictor leans R+3.5 against the others, which sounds like a flaw. But the field leaned Democratic in 3 of these 5 years. In 2024 we were at D+1.3 while the other forecasters ranged from D+2.9 to D+11.8; in 2020 we were at D+9.1 against a field from D+1.1 to D+15.1. Leaning less Democratic than the field was the right call in those years.
It was the wrong call in 2018. We were at R+7.1 while the others ranged from R+5.6 to D+5.4. That was the Democratic House wave, and our House forecasts for that year were trained on 2016 alone, so they underrated it. That year is most of the gap between us and FiveThirtyEight on House elections in the track record.
Does leaning cost accuracy?
Some. The chart puts each forecaster's lean against its Brier score (the standard accuracy score for win probabilities) relative to the others on the same elections. Higher is more accurate.
Brier score is the squared gap between the win probability and what happened; lower is better. Shown here as each forecaster's Brier minus the others' on the same elections, so above the middle line is more accurate than the others. Bigger dots have more elections.
The forecasters furthest from the pack, in either direction, tend to score worse, though the sample is small and RealClearPolitics carries much of the pattern. FiveThirtyEight and VotePredictor are the two most accurate against their peers, and they lean in opposite directions.
How we measured it
- Each forecaster's final win probability for Senate, House, governor and presidential (state) elections, as archived by JHK Forecasts. Rating shops (Cook, Sabato, Inside Elections and others) publish ratings, which the archive converts to probabilities.
- Only Democrat-vs-Republican elections that our backtest also covers, so every forecaster, VotePredictor included, is scored on the same set.
- VotePredictor's numbers are its walk-forward record: each year forecast by a model trained only on earlier years, never the published forecast of that year.
- Lean is a forecaster's Democratic win probability minus the average of the other forecasters on the same election. Elections no other forecaster covered are left out.
- The 95% range treats elections as independent. Elections in one year share a miss, and most forecasters cover only four or five years, so the true ranges are wider than shown.
The full scores are on the track record page. The same question for pollsters is in which pollsters lean Democratic or Republican.