Survey mode is an interesting variable in the current market research industry landscape. Historically we moved from in-person to mail to telephone and online as the principle methods of polling. The latest entry into the survey mode toolkit is the IVR or interactive voice response survey. Several market research firms in Canada release polls into the public domain and, it is clear, that the use of IVR has a significant impact on the total amount of policy coverage that polls can have. IVR survey releases are shorter so there are fewer questions.

Between April 1, 2016 and June 30, 2017 (15 months), there were 905 public opinion releases including all geographies that the number of questions reported on was reasonably determined (265 releases or 23% did not have this information or it was unclear).

For the purpose of this analysis, a question is any time a person is required to offer a response. A four item agree battery is therefore treated as four questions, while a multi-response question is considered a single question since only one reply is required. Demographic and screening questions, since these are not generally included in releases are not counted.

  • On average, each public opinion release had 9.96 questions on it.
  • Online survey releases are generally longer (13.6 questions) than IVR releases (5.0 questions) by a factor of 2.7.

When we consider only those national surveys about public policy issues (n=224), the impact of mode on content is clear. The average number of questions on a national policy release is 12.8. Online survey released have on average 14.2 questions compared with 3.5 for IVR. Notably, the telephone survey (n=5) tend to be long (41.2 questions) suggesting that when people invest in a national telephone survey they make it count in terms of the policy content.

On average an IVR national policy survey release has only 3.5 questions, which significantly reduces the total content that the public, if they see the poll, are provided.