Post Mortem for the 2014 Senate Elections: November 5, 2014 |
Of the ten states followed below, the Democrats won only MI and NH. Using
the final ranking on November 4, 2014, for Upshot and 538 (below), the ranking
assumption says that if the Democrats lost NC, which they did, they should
also have lost all the states ranked below NC, which they did. So given these
two rankings, the ranking assumption is perfect. Note that both Upshot and
538 had probabilities greater than 50 percent for the Democrats winning
NC and KS (counting the independent candidate in KS as a Democrat).
Even though they were wrong in the sense that the Democrats lost
both states, this has no effect on the performance of the ranking
assumption. It was perfect even though Upshot and 538 were not.
PredictWise had NC ranked above NH (below). Given that the Democrats lost NC, the ranking assumption says that they should also have lost NH, which they did not. So the ranking assumption made one error using the PredictWise ranking. Loosely speaking, in the Senate elections there was a wave of Republican support not completely captured in the probabilities. This wave of support was across the country, affecting all probabilities in the same direction, which is consistent with the ranking assumption. The probabilities were not independent, nor even close to being independent. Note that the Democrats almost lost Virginia, where the probabilities were close to 100 percent that the Democrats would win (not collected below). |
The Ranking Assumption in 2014 using Upshot, 538, and PredictWise Data |
Background
The paper, Interpreting the Predictive Uncertainty of Elections, Journal of Politics, April 2009, provides an interpretation of the uncertainty that exists on election morning as to who will win. The interpretation is based on the theory that there are a number of possible conditions of nature than can exist on election day, of which one is drawn. Political betting markets and sites like Upshot and 538 provide a way of trying to estimate this uncertainty. (Polling standard errors do not provide estimates of this type of uncertainty. They estimate sample-size uncertainty, which can be driven close to zero with a large enough sample.) This paper also introduces a "ranking assumption," which puts restrictions on the possible conditions of nature that can exist on election day. Take as an example the vote in each state for the Democratic candidate for Senate. Rank the states by the probability that the Democratic candidate wins the state. The ranking assumption says that if the Democrats win state i, they win every state ranked above state i. Given some ranking for the Senate elections in 2014, the ranking assumption can be tested by simply looking to see after the fact if the Democrats won a state ranked lower than one they lost (contrary to the ranking assumption). I am going to collect probabilities each Tuesday morning from Upshot, 538, and PredictWise. The November 4, 2014, data will be used to test the ranking assumption. The test is a test of the joint hypothesis that the Upshot or 538 or PredictWise probabilities are right and the ranking assumption is right. November 4, 2014, 6:00 am I have been following 10 states (see below), where the independent candidate in KS is counted as a Democrat. The final ranking for Upshot (before voting begins) of the probability that the Democrats win the state is:. |
state | prob | |
MI | 96 | |
NH | 75 | |
NC | 73 | |
KS | 54 | |
GA | 40 | |
AL | 35 | pivot for 50 Democratic seats |
IA | 34 | |
CO | 20 | |
LA | 16 | |
AR | 11 | |
30 = Upshot probability that the Democrats get 50 or more seats.
The final ranking for 538 is: |
state | prob | |
MI | 99 | |
NH | 79 | |
NC | 69 | |
KS | 53 | |
IA | 30 | |
CO | 28 | pivot for 50 Democratic seats |
AL | 26 | |
GA | 25 | |
LA | 19 | |
AR | 4 | |
23.8 = 538 probability that the Democrats get 50 or more seats. |
The final ranking for PredictWise is as of
11:00 pm on November 3, 2014, except
for KS, which is as of October 28, 2014. (PredictWise did not
report values for KS on November 3.) The ranking is:
PredictWise
state | prob | |
MI | 100.0 | |
NC | 85.0 | |
NH | 74.8 | |
KS | 48.0 | as of October 28, 2014 |
IA | 38.4 | |
CO | 23.1 | pivot for 50 Democratic seats |
AL | 23.0 | |
GA | 6.9 | |
LA | 4.5 | |
AR | 0.2 | |
For PredictWise the probability that the Democrats get 50 or more seats under the assumption that the independent candidate in KS is counted as a Democrat is not computed. |
October 21, 2014
All but 10 states have probabilities close to 0 or 1 on
Upshot and 538. The 10 states for Upshot ranked by probabilities for the
Democrats are (the independent candidate in KS is counted as a Democrat):
Upshot
state | prob | |
MI | 92 | |
NH | 81 | |
NC | 81 | |
KS | 47 | |
GA | 39 | |
IA | 37 | pivot for 50 Democratic seats |
CO | 32 | |
AR | 23 | |
AL | 22 | |
LA | 8 | |
34 = Upshot probability that the Democrats get 50 or more seats.
The 10 states for 538 ranked by probabilities for the Democrats are: |
state | prob | |
MI | 93 | |
NH | 81 | |
NC | 71 | |
KS | 54 | |
GA | 35 | |
IA | 35 | pivot for 50 Democratic seats |
CO | 30 | |
AR | 25 | |
LA | 22 | |
AL | 22 | |
36.8 = 538 probability that the Democrats get 50 or more seats. |
The 10 states for PredictWise ranked by probabilities for the
Democrats are:
PredictWise
state | prob | |
MI | 99.8 | |
NH | 82.9 | |
NC | 70.4 | |
KS | 39.0 | |
GA | 35.3 | |
IA | 28.7 | pivot for 50 Democratic seats |
CO | 24.7 | |
AL | 11.8 | |
LA | 8.6 | |
AR | 5.9 | |
For PredictWise the probability that the Democrats get 50 or more seats under the assumption that the independent candidate in KS is counted as a Democrat is not computed. |
October 28, 2014
All but 10 states have probabilities close to 0 or 1 on
Upshot and 538. The 10 states for Upshot ranked by probabilities for the
Democrats are (the independent candidate in KS is counted as a Democrat):
Upshot
state | prob | |
MI | 97 | |
NH | 74 | |
NC | 71 | |
KS | 58 | |
GA | 48 | |
IA | 37 | pivot for 50 Democratic seats |
AL | 29 | |
CO | 17 | |
AR | 15 | |
LA | 6 | |
30 = Upshot probability that the Democrats get 50 or more seats.
The 10 states for 538 ranked by probabilities for the Democrats are: |
state | prob | |
MI | 98 | |
NH | 82 | |
NC | 70 | |
KS | 51 | |
GA | 50 | |
IA | 34 | pivot for 50 Democratic seats |
AL | 34 | |
CO | 22 | |
LA | 19 | |
AR | 15 | |
36.7 = 538 probability that the Democrats get 50 or more seats. |
The 10 states for PredictWise ranked by probabilities for the
Democrats are:
PredictWise
state | prob | |
MI | 100.0 | |
NH | 85.4 | |
NC | 76.6 | |
KS | 48.0 | |
GA | 38.7 | |
IA | 28.5 | pivot for 50 Democratic seats |
CO | 17.2 | |
AL | 8.7 | |
AR | 4.0 | |
LA | 2.0 | |
For PredictWise the probability that the Democrats get 50 or more seats under the assumption that the independent candidate in KS is counted as a Democrat is not computed. |
October 21, 2014
All but 10 states have probabilities close to 0 or 1 on
Upshot and 538. The 10 states for Upshot ranked by probabilities for the
Democrats are (the independent candidate in KS is counted as a Democrat):
Upshot
state | prob | |
MI | 92 | |
NH | 81 | |
NC | 81 | |
KS | 47 | |
GA | 39 | |
IA | 37 | pivot for 50 Democratic seats |
CO | 32 | |
AR | 23 | |
AL | 22 | |
LA | 8 | |
34 = Upshot probability that the Democrats get 50 or more seats.
The 10 states for 538 ranked by probabilities for the Democrats are: |
state | prob | |
MI | 93 | |
NH | 81 | |
NC | 71 | |
KS | 54 | |
GA | 35 | |
IA | 35 | pivot for 50 Democratic seats |
CO | 30 | |
AR | 25 | |
LA | 22 | |
AL | 22 | |
36.8 = 538 probability that the Democrats get 50 or more seats. |
The 10 states for PredictWise ranked by probabilities for the
Democrats are:
PredictWise
state | prob | |
MI | 99.8 | |
NH | 82.9 | |
NC | 70.4 | |
KS | 39.0 | |
GA | 35.3 | |
IA | 28.7 | pivot for 50 Democratic seats |
CO | 24.7 | |
AL | 11.8 | |
LA | 8.6 | |
AR | 5.9 | |
For PredictWise the probability that the Democrats get 50 or more seats under the assumption that the independent candidate in KS is counted as a Democrat is not computed. |
October 14, 2014
All but 9 states have probabilities close to 0 or 1 on
Upshot and 538. The 9 states for Upshot ranked by probabilities for the
Democrats are (the independent candidate in KS is counted as a Democrat):
Upshot
state | prob | |
MI | 90 | |
NH | 86 | |
NC | 78 | |
KS | 68 | |
IA | 38 | |
CO | 37 | pivot for 50 Democratic seats |
AL | 19 | |
LA | 17 | |
AR | 13 | |
32 = Upshot probability that the Democrats get 50 or more seats.
The 9 states for 538 ranked by probabilities for the Democrats are: |
state | prob | |
MI | 91 | |
NH | 86 | |
NC | 79 | |
KS | 58 | |
IA | 37 | |
CO | 36 | pivot for 50 Democratic seats |
AR | 26 | |
LA | 26 | |
AL | 22 | |
40.5 = 538 probability that the Democrats get 50 or more seats. |
October 7, 2014
All but 9 states have probabilities close to 0 or 1 on
Upshot and 538. The 9 states for Upshot ranked by probabilities for the
Democrats are (the independent candidate in KS is counted as a Democrat):
Upshot
state | prob | |
MI | 95 | |
NH | 90 | |
NC | 83 | |
KS | 64 | |
CO | 51 | |
IA | 35 | pivot for 50 Democratic seats |
AL | 22 | |
LA | 19 | |
AR | 16 | |
39 = Upshot probability that the Democrats get 50 or more seats.
The 9 states for 538 ranked by probabilities for the Democrats are: |
state | prob | |
MI | 91 | |
NH | 83 | |
NC | 81 | |
KS | 65 | |
CO | 47 | |
IA | 35 | pivot for 50 Democratic seats |
AL | 27 | |
AR | 27 | |
LA | 26 | |
42.4 = 538 probability that the Democrats get 50 or more seats. |
September 29, 2014
All but 9 states have probabilities close to 0 or 1 on
Upshot and 538. The 9 states for Upshot ranked by probabilities for the
Democrats are (the independent candidate in KS is counted as a Democrat):
Upshot
state | prob | |
NH | 82 | |
NC | 81 | |
MI | 81 | |
KS | 56 | |
IA | 39 | |
CO | 39 | pivot for 50 Democratic seats |
LA | 28 | |
AL | 28 | |
AR | 19 | |
33 = Upshot probability that the Democrats get 50 or more seats.
The 9 states for 538 ranked by probabilities for the Democrats are: |
state | prob | |
NH | 82 | |
NC | 81 | |
MI | 77 | |
KS | 57 | |
IA | 43 | |
CO | 43 | pivot for 50 Democratic seats |
LA | 30 | |
AL | 29 | |
AR | 26 | |
39.0 = 538 probability that the Democrats get 50 or more seats. |
Post Mortem for the 2012 Election: November 7, 2012 |
The final ranking at 6am on November 6, 2012, repeated from below (see also Table 1), was: |
state | prob | votes | sumvotes | |
MN | 90.6 | 10 | 217 | |
NV | 88.1 | 6 | 223 | |
PA | 79.0 | 20 | 243 | |
WI | 77.5 | 10 | 253 | |
IA | 70.0 | 6 | 259 | |
OH | 68.8 | 18 | 277 | pivot |
NH | 68.0 | 4 | ||
VA | 55.8 | 13 | ||
CO | 54.1 | 9 | ||
FL | 35.0 | 29 | ||
NC | 20.5 | 15 |
The ranking assumption was perfect. Obama won all these states except NC.
Note that Intrade was wrong about FL regarding the winner. The ranking
assumption is, however, fine because it says that if Obama wins FL, he
wins every state above FL, which he did.
The ranking assumption did well for the Senate races, where it was off by one. The following is the Intrade ranking as of 6am on November 6, 2012, for the Senate races that were in play. The price for each state is the price that the Democratic candidate wins: |
state | prob |
FL | 91.2 |
CT | 88.0 |
MA | 84.0 |
OH | 83.2 |
IN | 80.0 |
VA | 78.2 |
MO | 72.7 |
WI | 60.0 |
MT | 47.4 |
NV | 20.1 |
ND | 19.0 |
The Democrats won every state except NV. Given that the Republicans won NV, the ranking assumption says that the Democrats should have lost ND, and this is the one error. Otherwise, the Democrats won MT and every state above it. Note that Intrade was wrong about the winner in MT, but again this is not a problem for the ranking assumption. Both Intrade and the ranking assumption were wrong about ND. |
The Ranking Assumption in 2012 using Intrade Data |
Background
The paper, Interpreting the Predictive Uncertainty of Elections, Journal of Politics, April 2009, provides an interpretation of the uncertainty that exists on election morning as to who will win. The interpretation is based on the theory that there are a number of possible conditions of nature than can exist on election day, of which one is drawn. Political betting markets like Intrade provide a way of trying to estimate this uncertainty. (Polling standard errors do not provide estimates of this type of uncertainty. They estimate sample-size uncertainty, which can be driven close to zero with a large enough sample.) This paper also introduces a "ranking assumption," which puts restrictions on the possible conditions of nature that can exist on election day. Take as an example the vote in each state for the Democratic candidate for president. Rank the states by the probability that the candidate wins the state. The ranking assumption says that if the candidate wins state i, he or she wins every state ranked above state i. Given some ranking, the ranking assumption can be tested by simply looking to see after the fact if the candidate won a state ranked lower than one he or she lost. In the paper the assumption was tested for the 2004 and 2008 presidential elections using Intrade probabilities at 6am Eastern standard time on the day of the election. The test is thus a test of the joint hypothesis that the Intrade probabilities are right and the ranking assumption is right. Using the Intrade probabilities, the ranking assumption was perfect in 2004 and off by one in 2008. In 2008 Missouri was ranked above Indiana, and Obama won Indiana and lost Missouri. Both of these elections were very close. Obama won Indiana with 50.477 percent of the two-party vote and lost Missouri with 49.937 percent of the two-party vote. Otherwise, 2008 was perfect. Evidence is also presented in the paper, although this is not a test of the ranking assumption, that Intrade traders use the ranking assumption to price various contracts. 2012 Election I plan to collect Intrade probabilities at 6am Eastern standard time on six days: September 11, 25, October 9, 23, 30, and November 6. The November 6 data will be used to test the ranking assumption. The data as they are collected are presented in Table 1. September 11, 2012 All but 11 states on Intrade either have probabilities close to 0 or 1 or have essentially no trading. The 11 states, ranked by probabilities for the Democratic candidate, are: |
state | prob | votes | sumvotes | |
MN | 82.5 | 10 | 217 | |
PA | 80.1 | 20 | 237 | |
NH | 71.0 | 4 | 241 | |
WI | 68.0 | 10 | 251 | |
NV | 67.9 | 6 | 257 | |
CO | 64.8 | 9 | 266 | |
OH | 61.9 | 18 | 284 | pivot |
IA | 59.7 | 6 | ||
VA | 58.9 | 13 | ||
FL | 47.0 | 29 | ||
NC | 28.3 | 15 |
61.0 = Intrade probability that the Democratic candidate
wins the presidential election.
"sumvotes" is the sum of the electoral votes of all the states ranked above the state plus the state's vote. 270 votes are needed to win. You can see that Ohio is the pivot state. If Obama takes Ohio and all the states ranked above it, he gets 284 votes. Of the states ranked below Ohio, he could also win by not taking Ohio and taking any one of the others. There is evidence that the Intrade traders are using the ranking assumption. According to the ranking assumption, the probability that Obama wins overall is the probability that he wins the pivot state, Ohio, which is 61.9. The price of the contract on Intrade that the Democratic candidate wins the presidential election (in the electoral college) is 61.0, close to 61.9. If the ranking assumption were not being used, the overall probablilty would be much higher if, say, the probabilities were thought to be independent. The ranking assumption says that Obama is not going to take, say, Iowa or Virginia unless he also takes Ohio, so the probabilities for Iowa and Virginia do not matter. The probability he wins overall is just the probability he wins Ohio. If the probabilities were independent, the probability of an overall win would be 82.3 percent (computed numerically). September 25, 2012 The results are: |
state | prob | votes | sumvotes | |
MN | 92.2 | 10 | 217 | |
PA | 87.0 | 20 | 237 | |
NV | 79.8 | 6 | 243 | |
WI | 79.0 | 10 | 253 | |
NH | 75.6 | 4 | 257 | |
OH | 72.6 | 18 | 275 | pivot |
IA | 65.1 | 6 | ||
VA | 64.3 | 13 | ||
CO | 61.1 | 9 | ||
FL | 55.0 | 29 | ||
NC | 42.6 | 15 |
72.4 = Intrade probability that the Democratic candidate
wins the presidential election.
Ohio is still pivot, and it seems clear that the Intrade traders are using the ranking assumption. If the probabilities were thought to be independent, the overall probability would be 93.1 percent (computed numerically). October 9, 2012 The results are: |
state | prob | votes | sumvotes | |
MN | 90.0 | 10 | 217 | |
PA | 87.4 | 20 | 237 | |
NV | 76.7 | 6 | 243 | |
NH | 74.8 | 4 | 247 | |
WI | 70.0 | 10 | 257 | |
IA | 67.7 | 6 | 263 | |
OH | 66.6 | 18 | 281 | pivot |
VA | 63.0 | 13 | ||
CO | 61.0 | 9 | ||
FL | 49.9 | 29 | ||
NC | 30.0 | 15 |
62.7 = Intrade probability that the Democratic candidate
wins the presidential election.
Ohio is still pivot. If the ranking assumption were being used by the Intrade traders, the probability of winning overall should be 66.6, the probability for Ohio. It is in fact 62.7, so a discrepancy of 3.9. In the two earlier samples above the discrepancies were 0.9 and 0.2. If the probabilities were thought to be independent, the overall probability would be 89.0 percent (computed numerically). It is thus surprising that the overall probability of 62.7 is smaller than both the probability for Ohio and the overall probability if the state probabilities were independent. Could be a thin market problem or market manipulation of the overall probability contract. October 23, 2012 The results are: |
state | prob | votes | sumvotes | |
NV | 85.0 | 6 | 213 | |
MN | 81.0 | 10 | 223 | |
PA | 75.2 | 20 | 243 | |
WI | 71.0 | 10 | 253 | |
IA | 57.8 | 6 | 259 | |
OH | 57.0 | 18 | 277 | pivot |
NH | 53.5 | 4 | ||
VA | 48.0 | 13 | ||
CO | 45.5 | 9 | ||
FL | 30.0 | 29 | ||
NC | 18.4 | 15 |
61.7 = Intrade probability that the Democratic candidate
wins the presidential election.
Ohio is still pivot. If the ranking assumption were being used by the Intrade traders, the probability of winning overall should be 57.0, the probability for Ohio. It is in fact 61.7, so a discrepancy of -4.7. This compares to a positive 3.9 on October 9, 2012. If the probabilities were thought to be independent, the overall probability would be 68.1 percent (computed numerically). So in this case the overall probability is in between what the ranking assumption would imply and what the independence assumption would imply. October 30, 2012 The results are: |
state | prob | votes | sumvotes | |
MN | 92.0 | 10 | 217 | |
PA | 82.0 | 20 | 237 | |
NV | 80.0 | 6 | 243 | |
WI | 71.2 | 10 | 253 | |
IA | 61.6 | 6 | 259 | |
OH | 58.8 | 18 | 277 | pivot |
NH | 58.3 | 4 | ||
CO | 47.8 | 9 | ||
VA | 45.1 | 13 | ||
FL | 30.1 | 29 | ||
NC | 22.0 | 15 |
62.0 = Intrade probability that the Democratic candidate
wins the presidential election.
Ohio is still pivot. If the ranking assumption were being used by the Intrade traders, the probability of winning overall should be 58.8, the probability for Ohio. It is in fact 62.0, so a discrepancy of -3.2. If the probabilities were thought to be independent, the overall probability would be 73.9 percent (computed numerically). November 6, 2012 The results are: |
state | prob | votes | sumvotes | |
MN | 90.6 | 10 | 217 | |
NV | 88.1 | 6 | 223 | |
PA | 79.0 | 20 | 243 | |
WI | 77.5 | 10 | 253 | |
IA | 70.0 | 6 | 259 | |
OH | 68.8 | 18 | 277 | pivot |
NH | 68.0 | 4 | ||
VA | 55.8 | 13 | ||
CO | 54.1 | 9 | ||
FL | 35.0 | 29 | ||
NC | 20.5 | 15 |
67.9 = Intrade probability that the Democratic candidate
wins the presidential election.
Ohio is still pivot. If the ranking assumption were being used by the Intrade traders, the probability of winning overall should be 68.8, the probability for Ohio. It is in fact 67.9, so a discrepancy of 0.9. If the probabilities were thought to be independent, the overall probability would be 82.7 percent (computed numerically). This ranking will be used to test the ranking assumption. The probabilities are at 6am on the day of the election. The ranking assumption (combined with the Intrade probabilities) says, for example, that if Obama wins Colorado, Romney will not win Virginia, New Hampshire, or Ohio. If, on the other hand, Romney wins New Hampshire, the ranking assumption says that he will also win Virginia and Colorado. Table 1 gives more details. |