Anyone looking for a good glimpse into the state of the market research industry need look no further than the Open Letter penned by two senior members of Ipsos Reid and the coverage of how other professionals have responded. It shows that (a) the accuracy of political polls continues to be central to both our own and others’ evaluation of us as a industry; and, (b) that the industry has evolved so much that we no longer have a shared understanding of what a poll is.
For more information about the controversy: an early news story appeared in the Globe and Mail and was covered in the The Agenda with Steve Paikin Blog (the latest Canadian Press story appeared on September 26). The letter openly criticizes the methods used by some pollsters and the reporting of polls by the media in the name of democracy. The MRIA responded to the debate and coverage with their own response about the validity of research methods.
It is not a stretch to say that Darrell Bricker and John Wright of Ipsos Reid care deeply about democracy and believe that public opinion research can contribute to a healthy polity.
Certainly, as a believer that information is always preferable to speculation and guesswork, I also believe that knowing how other citizens feel about the political landscape is interesting for the rest of us even if translating that knowledge into our own decision-making is challenging.
Election Polling as Industry Bellwether
There is something inherently dangerous about our focus on election polling as evidence of our ability to be
good researchers. I have written about this idea elsewhere in the context of the chance that one of these times we will not get it right and what it would mean for the industry. In fact, it is interesting that we get it right so often given that: (1) we try to get a representative sample (using imperfect techniques); approximately only half of the people will actually vote; and, (3) people are free to change their mind right up until election day.
A Higher Standard for Media Polls
The open-letter can be read as a call for a higher standard of research for publicly released polls and this argument makes sense from the point of view of the industry-insider who is looking at election polling as a reflection of our overall quality. It is also not hard to argue that the media could do a better job of reporting polls (or maybe even the news more generally) and that in a democracy we should be striving to provide citizens with the best (sic more accurate) information possible.
The problem with the higher standard idea is two pronged. The first is that we won’t agree on what the high standard looks like beyond the fact that it “must” somehow be accurate on election day. The second is that it really raises the question of the standard for non-political polling.
As much as I appreciate the idea that there is something inherently important about elections, clients who are looking for value may rightly question why there is one standard for them and another for political polls.
The higher standard that is referenced in the original letter is a standard based on the RDD telephone survey and while the telephone is not yet dead as a research method most market research has moved online. The industry has both responded to and encouraged the trend so we are not idle bystanders. In this context, the letter is partially a lament for a golden age that is now passed.