Bias has considerable negative consequences and the easiest critique of a public opinion survey is to suggest that the questions are biased. But bias is inherent in asking questions and relying on truly unbiased questions risks stripping the questions of meaning, leaving them stale and irrelevant.
While probably everyone could recognize extreme bias, bias can be subtle or obvious. Endorsements often fall into the subtle category.
An endorsement is any aspect of the question that links one side of the question being asked to a group, individual or institution. References for example, to political parties, specific groups (e.g. anti-abortionists, liberals) provide respondents with cues that respondents can use as a proxy for their true opinions.
I like/dislike that group therefore my opinion is ‘X’ or I usually agree/ disagree with the group therefore my opinion is ‘X’.
The risks exist because people construct an answer to the question based on their values, beliefs and recent information. By including information about endorsements we are encouraging people to weigh the relevance of that endorsement in their answer.
As a result, many questions in the public domain are strikingly bland. It is harder to call a simple question biased. A recent CBS/ NY Times poll question captures this effectively, “Do you think it should be legal or not legal for same-sex couples to marry?” It is a question that goes to the core of the values at play but it ignores the subtlety of the issue and the actors involved (Supreme Court; Congress; groups) and their arguments.
Of course, this is not to say that all questions are devoid of endorsements. Sometimes the endorsements is directly related to a political fact. For example, “Do you support or oppose the Government’s plan to increase deficit spending?” In this case, the plan is a real plan so the inclusion of it is arguably relevant. Taking it out, which leaves the question, “Do you support or oppose deficit spending by the Federal Government? is more neutral but it does not capture the fact of an existing plan.
The use of endorsements in survey questions is fraught with risk. Questions that will be released in the public domain must use them judiciously. They are, however, a powerful tool for a researcher who can use endorsements to understand how to move public opinion.
While simple questions without endorsements offer the least bias, they are also the least like how people hear about and discuss these issues in real life.