Just this week, we got another pseudo apologetic letter about why election polling is sometimes (perhaps even often) inaccurate. This time it was from Peter Loewen (an Assistant Professor of UofT). His article in the Ottawa Citizen is here but, it like many similiar pieces completely misses the point.
Pollsters are in a unique an perilous position in the days leading up to an election because as much as they want o believe it it not true; voters are allowed to change their mind.
Peter’s article which is largely about the Quebec provincial election is a disservice to the industry. His focus is on the problem of getting a good sample. In his own words:
Pollsters simply do not know enough about who responds to polls via some media, who replies through others, and what kinds of people ignore polling requests entirely. The problem isn’t getting sample, it’s getting good sample. (Peter Loewen)
Does sample matter? Yes. Is their a challenge with getting a representative sample given response rates and new technology? Yes. These are not even close to being the issues that challenge pollsters in being more accurate about the vote. They are simplistic and naive assertions which miss the fundamental challenges of election polling.
1. Not Everyone Votes
Public opinion polls at there best are uncomplicated because everyone has an opinion or a non opinion. When it comes to elections, what people say is their vote choice (or non choice) is a spur of the moment decision that is divorced from actually voting. While pollsters can expect that most people who express an opinion will actually vote, we know that there is a gap when 75% give us a vote preference but only 50% or so turnout. What we lack is a guaranteed method of identifying those who will actually vote. The best sample in the world won’t address the gap between those who have a preference and those who will vote.
2. Voters can Change their Mind
I am sure that many would prefer that people did not change their mind but they do. And, given the importance of a democratic franchise, perhaps we should applaud those ballot box reconsiderations. In either case, voters are allowed to be coy — about their intention — and ultimately deceive pollsters. Good pollsters and good media know when this is a possibility.
So, sure, the sample matters but in reality, it could easily be argued that many of failures of polling are simply a reflection of a misguided focus on predicting a future election outcomes (the result) and not enough focus on the meaning inherent in what people tell us. There is very little science that can tell us how people (rather than atoms) will behave in a particular situation. Perhaps it is time to stop expecting polling to do that.