Examples of bad survey questions (and 3 easy tips to avoid them)

If you’re an experienced market researcher, you know you can design survey questions that make people say anything. For instance, if you’re not careful, you might end up creating a poll that gets 5% of African Americans to be against freeing slaves. Below are some common examples of bad survey questions, and 3 top tips on how you start designing surveys like a pro.

Examples of bad survey questions: 

Assumptions

Bad: “Please tell us why you hate Trump so much:”

Assuming that you know how the respondents are thinking is a very common faux-pas. It colors the results and leads to poor research design. The example assumes the respondent hates Trump, and even if they did, you wouldn’t get a complete picture. To get a more objective opinion on Trump, you could ask: 

Good: “Please tell us your opinion of Trump, and explain your response:” 

Bias

Bad: “How heroic is Christopher Columbus?”

As a rule of thumb, it is best to avoid taking a stance on anything when designing research, especially when it comes to historical and political figures and events. Let the research participants express their own opinions using a more neutrally-worded question. Rather than asking this prime example of a bad survey question, structure the question to elicit their opinion of Columbus. 

Good: Do you agree or disagree with the following characteristics of Christopher Columbus? (then provide a list of adjective & answer scale 1-5, Strongly disagree – Strongly agree)

Double negatives

Bad: “Is it wrong not to use double negatives?”

Confusing, isn’t it? Double negatives confuse participants and muddle your data. As survey designers, your job is to make it as easy and as clear to the participants as possible. There’s a special place in hell for researchers who use double negatives!

Good: “Should we use double negatives?

Using absolutes

Bad: How would you rate Donald Trump’s performance as a president?

• He’s the best president EVER!
• He’s the worst president EVER!

Asking questions with absolutes limits the quality of the responses. It makes the respondent feel that their responses have to be black or white. Give your participants a variety of answer options so that you can have better insights.

Yes/No questions are OK for survey screeners if you’re looking for participants with certain characteristics. However, it’s good practice to be nuanced. Avoid any word that signifies an absolute, terms like “ever” and “always.” 

Good: How would you rate Donald Trump’s performance as a president? (On a scale of 1 to 5)?

1. Very Poor
2.  
3.  
4.  
5. Excellent

Combining questions

Bad: How often do you purchase and eat candy bars?

You may think that you are saving time by asking a question that serves a dual purpose. However, this mistake can be costly. Whatever you do, do not try to cram two or more things into one. While you may think that you’re efficient and effective, it could kill your survey. The responses will be fuzzy, and you will not have a clear answer for either question. The solution is simple, just break it up into two questions. 

Good:
How often do you purchase candy bars? 
Never
A few times a year
Every few months o Once a month
Once a week
A few times a week
Everyday

How often do you eat candy bars?
Never
A few times a year
Every few months o Once a month
Once a week
A few times a week
Everyday 

Using complex language, acronyms

Bad:

• How many times a week do you suffer from cephalalgia?
• Do you habitually imbibe in alcoholic beverages?

These questions are unnecessarily complex. Survey participants may not know the meaning of the words “cephalalgia” or “imbibe.” If you must use complex words, always leave the definition accessible in the question. Otherwise, it’s good practice to use the simplest form of the word when possible. Follow this maxim: Always Be Clear!

Good:

• How many times a week do you experience headaches?
• Which of the following best describes your alcohol drinking habits?

Here are 3 top tips to design surveys like a pro:

1. Always Be Clear: This is the ABC of survey design. Pretend your respondents are 6 year-olds and write your survey accordingly.
2. Stay Objective: Be aware of your potential biases and remove them from the survey.
3. Make life easy for respondents: Make it easy for them to answer your questions. Remember, they’re there to take a survey, not a test. Don’t make them think!

Are you looking to create a survey? Check out how we can help!