Biased Questions

by Craig Shrives

What Are Biased Questions?

Biased questions are questions designed to elicit the required answer. In other words, it is possible to get the answer you want by the way you ask the question. Let's start with an extreme example of a biased or unfair question:

Answer this question with either "yes" or "no":

• "Have you stopped shoplifting?"

It doesn't matter how you answer this question, you're condemned as either an active shoplifter or an ex-shoplifter. (The old gag of shouting across a crowded room to ask your mate if he's all clear at the clinic is another example of an unfair question.)

Now, the shoplifting and clinic questions are not so much biased as unfair, but some trials and polls feature questions that are more subtly unfair. These are more accurately described as "biased questions". Asking biased questions is another way to twist a result. Look at these two questions:

• Version 1 of the Question:"Should the UK leave the European Union?"
• Version 2 of the Question:"Should the UK remain part of the European Union?"

People Like to Say "Yes"

Because of an inner desire to avoid confrontation, we are far more inclined to "agree" than "disagree" when presented with questions of this nature. In other words, we are more likely to answer "yes" to these questions. This means that Version 1 was far more likely to see the UK leaving than Version 2.

The human desire to "agree" with such questions is so strong that campaigners on both sides believed the way the question was posed would trump the British people's objective thoughts on whether they wanted to be in or out. As a result, for the both remain campaigners and leave campaigners, the poll question was a highly contentious issue in the run-up to the referendum.

Our inherent bias to answer "yes" is so strong that the government seriously considered presenting both questions to the public, requiring people to answer "yes" to one and "no" to the other. But that would have been too messy.

Eventually, the UK Government settled on the following question, which does not require a yes/no answer: In the end, it may not have been the poll question itself that influenced voters, but the general question everyone was asking themselves: "Should we leave the EU?" Across the nation, this was the question on people's lips, and this - as a biased question - will have played its role in the referendum result.

People Like to Avoid Confrontation

The inner desire to avoid confrontation does not just manifest itself with people tending to answer "yes" to questions. It also sees them answering questions in a way that pleases the asker. For example, in Afghanistan, US pollsters trying to gauge the sentiment of local Afghans found that they could not deliver the polls themselves because the Afghan people answered the questions to please them. To counter this, US pollsters used locals to conduct the polls, but these were assessed to be too biased in the other direction. It transpired that the hardest part about conducting a poll was to find a pollster that wouldn't influence the answers. (The differences in sentiment towards foreign troops on US-led polls, Afghan-led polls, and neutral polls were marked.) Of course, that's just the start point. Once you've found a neutral person to ask your question, don't make it a yes/no question.

Critical Thinking Test

Are you good at spotting the biases, fallacies, and other cognitive effects? Can you spot when statistics have been manipulated? Can you read body language? Well, let's see! Show Instructions

See Also