Poll Aggregation Methodology
Using a lot of polls from a lot of pollsters to try to get a better idea of what those polls are measuring is nothing new. This model, however, takes advantage of modern computing power to try to get a better idea of what the public really thinks. FiveThirtyEight’s model, and its derivative operated by Silver Bulletin used a combination of of an exponentially weighted moving average and a polynomial spline regression to try to aggregate polls, and while it has proven to be quite good historically, I think that the this Bayesian updating model does a better job for reasons that will become apparent later in this section.
Polling Data
The raw polling results I collect are from VoteHub. Unlike other polling aggregators, I combine polls by the week they were primarily conducted in, and do not provide daily updates. This is to ensure that I have a decent sample of polls to work with.
Like Silver, I aggregate these polls by weighing according to the square root of their sample size, but my adjustments for pollster bias and the universe surveyed are slightly different. While my adjustments are derived from his work, I treat those adjustments as distributions rather than point estimates. For the adjustments for registered and likely voters, the distribution started as normal with a standard deviation of forty percent of the value he derived, but those adjust dynamically as more data is accumulated. For the adjustments based on house effects and biases of pollsters, I use Silver’s estimates as the mean, and a standard deviation of forty percent of that mean.
How the model Interprets the Data
The way this model works is that it starts out with a distribution predicting how the universe of people feel about what is being polled. Then it updates that distribution based on the data it receives for the current distribution derived from the polling aggregation. This returns what is called the posterior distribution, which is what the model thinks is really going on.
Then when the next week of data is available, the posterior is used as the prior, and the process repeats. Or it would if this didn’t lead to the model tightening unduly. Instead, the posterior from the previous week is widened beforehand.
This happens in two ways, and in order to understand them, it is necessary to explain what type of distribution is being used. Rather than using a Normal distribution, which has a mean and a standard deviation, this model uses a T distribution with a mean, standard deviation, and a number of degrees of freedom.
Each week, before it is used as a prior, the standard deviation of the posterior is expanded by a constant factor representing the decay of the model’s confidence in the data, and the number of degrees of freedom are reduced in relation to the natural log of the number and size of the polls in each week’s aggregate, which means that the more and better polling data available in a week, the more movement it can generate.
Why this Model is Better
The exponential moving average is a good way to smooth out the noise in polling data, especially when there isn’t much data to work with, but older data drops off consistently, which can be a bit of a restriction in understanding the current situation, given that different drivers of public opinion can have different staying power.
A polynomial spline regression on the other hand, is vulnerable to overfitting, and setting the number of knots is as much an art as a science.
Rather than trying to fit a curve to the data, this model is comfortable with uncertainty, and allows the data to speak for itself.