Post Mortem after the 2014 Senate Election
The discussion below was on the website before the 2014 Senate election. The last data collected, listed first below, was on November 4, 2014. These data are used for the test of the ranking assumption.Of the ten states followed below, the Democrats won only MI and NH. Using the final ranking on November 4, 2014, for Upshot and 538, the ranking assumption says that if the Democrats lost NC, which they did, they should also have lost all the states ranked below NC, which they did. So given these two rankings, the ranking assumption is perfect. Note that both Upshot and 538 had probabilities greater than 50 percent for the Democrats winning NC and KS (counting the independent candidate in KS as a Democrat). Even though they were wrong in the sense that the Democrats lost both states, this has no effect on the performance of the ranking assumption. It was perfect even though Upshot and 538 were not.
PredictWise had NC ranked above NH. Given that the Democrats lost NC, the ranking assumption says that they should also have lost NH, which they did not. So the ranking assumption made one error using the PredictWise ranking.
Loosely speaking, in the Senate elections there was a wave of Republican support not completely captured in the probabilities. This wave of support was across the country, affecting all probabilities in the same direction, which is consistent with the ranking assumption. The probabilities were not independent, nor even close to being independent. Note that the Democrats almost lost Virginia, where the probabilities were close to 100 percent that the Democrats would win (not collected below).
Background
The paper,Interpreting the Predictive Uncertainty of Elections, Journal of Politics, April 2009,
provides an interpretation of the uncertainty that exists on election morning as to who will win. The interpretation is based on the theory that there are a number of possible conditions of nature than can exist on election day, of which one is drawn. Political betting markets and sites like Upshot and 538 provide a way of trying to estimate this uncertainty. (Polling standard errors do not provide estimates of this type of uncertainty. They estimate sample-size uncertainty, which can be driven close to zero with a large enough sample.)
This paper also introduces a "ranking assumption," which puts restrictions on the possible conditions of nature that can exist on election day. Take as an example the vote in each state for the Democratic candidate for Senate. Rank the states by the probability that the Democratic candidate wins the state. The ranking assumption says that if the Democrats win state i, they win every state ranked above state i.
Given some ranking for the Senate elections in 2014, the ranking assumption can be tested by simply looking to see after the fact if the Democrats won a state ranked lower than one they lost (contrary to the ranking assumption). I am going to collect probabilities each Tuesday morning from Upshot, 538, and PredictWise. The November 4, 2014, data will be used to test the ranking assumption. The test is a test of the joint hypothesis that the Upshot or 538 or PredictWise probabilities are right and the ranking assumption is right.
November 4, 2014, 6:00 am
I have been following 10 states (see below), where the independent candidate in KS is counted as a Democrat. The final ranking for Upshot (before voting begins) of the probability that the Democrats win the state is:| state | prob | |
| MI | 96 | |
| NH | 75 | |
| NC | 73 | |
| KS | 54 | |
| GA | 40 | |
| AL | 35 | pivot for 50 Democratic seats |
| IA | 34 | |
| CO | 20 | |
| LA | 16 | |
| AR | 11 | |
30 = Upshot probability that the Democrats get 50 or more seats.
The final ranking for 538 is:
| state | prob | |
| MI | 99 | |
| NH | 79 | |
| NC | 69 | |
| KS | 53 | |
| IA | 30 | |
| CO | 28 | pivot for 50 Democratic seats |
| AL | 26 | |
| GA | 25 | |
| LA | 19 | |
| AR | 4 | |
23.8 = 538 probability that the Democrats get 50 or more seats.
The final ranking for PredictWise is as of
11:00 pm on November 3, 2014, except
for KS, which is as of October 28, 2014. (PredictWise did not
report values for KS on November 3.) The ranking is:
| state | prob | |
| MI | 100.0 | |
| NC | 85.0 | |
| NH | 74.8 | |
| KS | 48.0 | as of October 28, 2014 |
| IA | 38.4 | |
| CO | 23.1 | pivot for 50 Democratic seats |
| AL | 23.0 | |
| GA | 6.9 | |
| LA | 4.5 | |
| AR | 0.2 | |
For PredictWise the probability that the Democrats get 50 or more seats under the assumption that the independent candidate in KS is counted as a Democrat is not computed.
October 28, 2014
All but 10 states have probabilities close to 0 or 1 on Upshot and 538. The 10 states for Upshot ranked by probabilities for the Democrats are (the independent candidate in KS is counted as a Democrat):| state | prob | |
| MI | 97 | |
| NH | 74 | |
| NC | 71 | |
| KS | 58 | |
| GA | 48 | |
| IA | 37 | pivot for 50 Democratic seats |
| AL | 29 | |
| CO | 17 | |
| AR | 15 | |
| LA | 6 | |
30 = Upshot probability that the Democrats get 50 or more seats.
The 10 states for 538 ranked by probabilities for the
Democrats are:
| state | prob | |
| MI | 98 | |
| NH | 82 | |
| NC | 70 | |
| KS | 51 | |
| GA | 50 | |
| IA | 34 | pivot for 50 Democratic seats |
| AL | 34 | |
| CO | 22 | |
| LA | 19 | |
| AR | 15 | |
36.7 = 538 probability that the Democrats get 50 or more seats.
The 10 states for PredictWise ranked by probabilities for the
Democrats are:
| state | prob | |
| MI | 100.0 | |
| NH | 85.4 | |
| NC | 76.6 | |
| KS | 48.0 | |
| GA | 38.7 | |
| IA | 28.5 | pivot for 50 Democratic seats |
| CO | 17.2 | |
| AL | 8.7 | |
| AR | 4.0 | |
| LA | 2.0 | |
For PredictWise the probability that the Democrats get 50 or more seats under the assumption that the independent candidate in KS is counted as a Democrat is not computed.
October 21, 2014
All but 10 states have probabilities close to 0 or 1 on Upshot and 538. The 10 states for Upshot ranked by probabilities for the Democrats are (the independent candidate in KS is counted as a Democrat):| state | prob | |
| MI | 92 | |
| NH | 81 | |
| NC | 81 | |
| KS | 47 | |
| GA | 39 | |
| IA | 37 | pivot for 50 Democratic seats |
| CO | 32 | |
| AR | 23 | |
| AL | 22 | |
| LA | 8 | |
34 = Upshot probability that the Democrats get 50 or more seats.
The 10 states for 538 ranked by probabilities for the
Democrats are:
| state | prob | |
| MI | 93 | |
| NH | 81 | |
| NC | 71 | |
| KS | 54 | |
| GA | 35 | |
| IA | 35 | pivot for 50 Democratic seats |
| CO | 30 | |
| AR | 25 | |
| LA | 22 | |
| AL | 22 | |
36.8 = 538 probability that the Democrats get 50 or more seats.
The 10 states for PredictWise ranked by probabilities for the
Democrats are:
| state | prob | |
| MI | 99.8 | |
| NH | 82.9 | |
| NC | 70.4 | |
| KS | 39.0 | |
| GA | 35.3 | |
| IA | 28.7 | pivot for 50 Democratic seats |
| CO | 24.7 | |
| AL | 11.8 | |
| LA | 8.6 | |
| AR | 5.9 | |
For PredictWise the probability that the Democrats get 50 or more seats under the assumption that the independent candidate in KS is counted as a Democrat is not computed.
October 14, 2014
All but 9 states have probabilities close to 0 or 1 on Upshot and 538. The 9 states for Upshot ranked by probabilities for the Democrats are (the independent candidate in KS is counted as a Democrat):| state | prob | |
| MI | 90 | |
| NH | 86 | |
| NC | 78 | |
| KS | 68 | |
| IA | 38 | |
| CO | 37 | pivot for 50 Democratic seats |
| AL | 19 | |
| LA | 17 | |
| AR | 13 | |
32 = Upshot probability that the Democrats get 50 or more seats.
The 9 states for 538 ranked by probabilities for the
Democrats are:
| state | prob | |
| MI | 91 | |
| NH | 86 | |
| NC | 79 | |
| KS | 58 | |
| IA | 37 | |
| CO | 36 | pivot for 50 Democratic seats |
| AR | 26 | |
| LA | 26 | |
| AL | 22 | |
40.5 = 538 probability that the Democrats get 50 or more seats.
October 7, 2014
All but 9 states have probabilities close to 0 or 1 on Upshot and 538. The 9 states for Upshot ranked by probabilities for the Democrats are (the independent candidate in KS is counted as a Democrat):| state | prob | |
| MI | 95 | |
| NH | 90 | |
| NC | 83 | |
| KS | 64 | |
| CO | 51 | |
| IA | 35 | pivot for 50 Democratic seats |
| AL | 22 | |
| LA | 19 | |
| AR | 16 | |
39 = Upshot probability that the Democrats get 50 or more seats.
The 9 states for 538 ranked by probabilities for the
Democrats are:
| state | prob | |
| MI | 91 | |
| NH | 83 | |
| NC | 81 | |
| KS | 65 | |
| CO | 47 | |
| IA | 35 | pivot for 50 Democratic seats |
| AL | 27 | |
| AR | 27 | |
| LA | 26 | |
42.4 = 538 probability that the Democrats get 50 or more seats.
September 29, 2014
All but 9 states have probabilities close to 0 or 1 on Upshot and 538. The 9 states for Upshot ranked by probabilities for the Democrats are (the independent candidate in KS is counted as a Democrat):| state | prob | |
| NH | 82 | |
| NC | 81 | |
| MI | 81 | |
| KS | 56 | |
| IA | 39 | |
| CO | 39 | pivot for 50 Democratic seats |
| LA | 28 | |
| AL | 28 | |
| AR | 19 | |
33 = Upshot probability that the Democrats get 50 or more seats.
The 9 states for 538 ranked by probabilities for the
Democrats are:
| state | prob | |
| NH | 82 | |
| NC | 81 | |
| MI | 77 | |
| KS | 57 | |
| IA | 43 | |
| CO | 43 | pivot for 50 Democratic seats |
| LA | 30 | |
| AL | 29 | |
| AR | 26 | |
39.0 = 538 probability that the Democrats get 50 or more seats.