There will be much hand wringing, considerable hyperbole, and lots of pseudo theories/ explanations for why the polls did not foresee the Conservative majority coming in the 2012 Alberta provincial election. Some attention will inevitably be placed on the methodological rigour of the polling methods but the size of the difference between the polls and the outcome is stunning.
On the Thursday and Friday before the election, three polls were released publicly (for a summary of some of these see here) and they all showed the Wildrose Party had a decisive lead (its share of the decided voters was 41%) but on election day only 34% of Albertans voted for the party.
Reportedly, some polling that was in field later showed the Wildrose Party with a smaller advantage.
1. Election Polling is a Contact Sport
There was a time, and I believe that I am not viewing the past through rose-coloured glasses, that election polling was not like it is now. A relatively small number of companies offered their polls to the media and their respective talking heads offered interpretation and insight. Companies were not doing it out of pure public interest — there was a perceived brand impact of being in the news but a common, agreed upon framework meant that there was little to be bitter about when it came to the actual polls. The relative paucity of polls also meant that the media was not constantly chasing a poll story.
My quick count is that there were at least 7 companies that offered polls on the provincial election in Alberta. And they weren’t all doing it the same way — there were IVR only polls; telephone polls; online polls, and probably others — and all this means is that there is more competition and more at stake as companies seek to prove themselves or their method. Inevitably this has led to accusations and criticism and it means that the concept of a “poll” is not shared by everyone.
2. Election Polling is Becoming Inherently Difficult to Get Right in the Face of Disruptive Change
The idea of using polling to predict election outcomes has always been a dangerous game. The following things can impact the relationship between a poll result and the election result:
- Timing: people can and do change their mind.
- Turnout; with less than 50% of Canadians voting in most elections it is impossible to create a perfect random sample of voters who will actually vote.
- Methodological impacts: each methodology has advantages and disadvantages when it comes to research. The telephone has coverage and response biases while online is not truly random (in almost all cases).
- Conformity bias: what people tell researchers is related to what they expect researchers want to hear so some parties are advantages/ disadvantaged and this is not easy to adjust for.
With all of these things going on, any campaign that is potentially disruptive to the political system is likely to be hard to understand just on the basis of the “who do you intend to vote for question”.
It is Time for a Change but What Change?
It is hard to resist a call for the following two things (1) less horserace coverage and (2) fewer polls since these would likely improve the quality of election campaign polls. The number of firms competing is unlikely to change and neither are journalistic norms so these seem like red herrings.
We could also argue as an industry for higher standards for polls but as I have written elsewhere, this is unlikely to be a compelling solution.
At the end of the day we will have some form of consensus about why the polls got it wrong but each poll failure diminishes the industry because we wrongly imply we have a window into the future. Rather than hanging on our hat on accuracy, we need to:
- ask more questions about the how and why of support so we can tell more compelling stories rather than relying on the horserace numbers.
- place more attention on where we might be wrong and why.