How do you know if a survey is biased?

Abhijeet Modi/ September 14, 2021/ Blog

A survey is a primary research tool that researchers used to collect the public’s views. It helps in profiling the target audience, finding people’s preferences and needs, etc. Users also share their feedback by filling surveys. However, just coming from the horse’s mouth does not guarantee its absoluteness. The crispness and unbiased nature of the survey are ideally dependent on how the researchers design the questions to elicit responses.

If the survey is biased, it can affect the quality of research. So, reviewers should affirm that the questions don’t have the following characteristics:

  1. Questions that skew people’s opinion towards a particular answer

The leading questions tend to lead to skewed responses. It would help if you asked questions that don’t put answers in the respondent’s mouth. They should get the cue to share their experience or opinion as is. Some leading questions are: How happy were you to attend our annual gala this December? It forces the respondent to center their thoughts around one feeling only – happiness. This needs correction to ensure an unbiased survey.

  1. Use of loaded questions

The questions like, “Will you buy our product online or offline?” can bring bias in the survey. Such questions are assumptive and do not address the possibility that the respondent may not be interested in buying the product at all. “How often you wake up late at night?” It is another bad question and may end in an empty answer slot because the person may not have a habit of waking up at night.

  1. Use of technical jargon

No matter how technical your product is, your end-user may be just a layman. Posing questions that contain technical jargon like OKRs, tech support executives, face-time, etc., may leave people confused, and they may tend to avoid answering it. Or, they may not answer it at all, leading to bad survey quality.

  1. Putting ‘negative words and not’ together in the question

“Was the product not unaffordable?” Frankly, this question scores poorly on sensibility. The researchers could have easily asked, “Was the product affordable?” Using not and negative words together affect the survey quality in two ways. First, it may lead to wrong answers due to oversight. It means the buyer may miss reading ‘not’ and give an opinion accordingly. Secondly, such a question may sound like a brain-teaser to people who are in a hurry. So, it may lead to no response too.

  1. Mismatched scale with choices

If you have asked the question, “How easy was our ordering system?” and use the option ‘yes’ or ‘no,’ it may defeat the survey’s purpose altogether. Consumers expect some sort of scale as a response recording method and may leave the question unattended, thinking it to be some kind of error.

These are some of the common mistakes to look for in a survey questionnaire. The respondents can feel confused, distracted, or completely demotivated to answer such questions. Thus, the survey can attain an unbiased quality if the questions do not have the flaws mentioned above.

Also, be thorough with your questionnaire planning and revise it, keeping yourself in the respondents’ shoes. If you feel uncomfortable, the response is most likely to be the same in actual conditions too.

Share this Post