No prediction markets, betting odds, news, scandal or subjective candidate-quality inputs. Any future rule-based adjustment has to improve held-out backtests first.
Each fold was fitted only on earlier elections and scored on contests the model never saw. Log loss is for the winner (lower is better); the comparison is a regression fitted on the same features.
| Held out | Contests | Model | Comparison | Chamber 80% intervals |
|---|---|---|---|---|
| 2014 | 3,292 | 0.376 | 0.412 | 5 of 6 |
| 2018 | 3,820 | 0.375 | 0.387 | 7 of 7 |
| 2022 | 3,586 | 0.433 | 0.425 | 4 of 4 (VT two-chamber total missed) |
| 2024 (untouched) | 2,402 | 0.492 | 0.540 | n/a |
2022 was the hardest test: redistricting reset most district histories and the model did not beat the comparison there. The 2024 test was run once, after the model was frozen; most of its gain came from races with third-party candidates. Production rules for missing data use separate empirical rates from Klarner's legislative returns, 2000–2020.