How to improve your survey question design
The Internet, and survey software, has made the technical process of survey question design and gathering the answers easy, but developing the content can still be a challenge.
We work with a wide variety of clients doing surveys. Clients who are new to the process benefit from assistance defining the project and creating survey content that supports their goals.
And for clients who are old hands at creating surveys, it’s easy to get “too close” to the topic, and to end up with content that doesn’t lead to reliable results.
Here are seven tips that even experienced survey professionals can use to improve their survey questions.
1: Have clear goals for using the results
Your goals for the survey will help inform what questions to include. Don’t waste your respondents’ time by fishing for answers for which you have no immediate use.
2: Define the Terms Clearly in your Survey Question Design
Your survey question design and responses won’t be reliable unless you and the respondent agree on definitions.
If your survey is about “organic coffee” asking “What coffee do you consider to be organic?” helps determine your respondent’s current understanding of the coffee market.
But follow up with your definition of “organic coffee” for the purposes of the rest of the survey. If you don’t, and they think “organic” is a meaningless marketing word, their subsequent answers will reflect that.
3: Give respondents some wiggle room
Too often, surveys force respondents into either/or answers, when reality isn’t quite so black and white.
By adding a qualifier in the survey question design and then giving some additional options for the answer, you can give your respondents some wiggle room. It will help their answers to be more realistic.
Here is an example. By adding the qualifier, it gives the question a better fit.
Q. Do you MAINLY make your own coffee or purchase it already made?* [Select one]
- Make own
- Purchase it
- Both
- I don’t drink coffee
4: Check your bias
Chances are you are creating survey questions around a topic area on which you are an expert, and your respondents aren’t.
It’s easy to forget that and to form questions that reflect your “expert bias”. Having your survey reviewed by an impartial person can help find places where the responses can be rounded out for richer data.
5: Limit “Required” answers
You might be tempted to require that respondents answer all of your survey questions. This isn’t necessary, and often annoys them. It can also lead to inaccurate data as respondents choose “something, anything” to get through the survey.
So, make most responses optional. Required answers should only be used if essential.
6: Avoid ranking questions
Don’t give respondents long lists of options to rank. Ranking is time-consuming, and will lead to bad responses. Many give up the process halfway through.
Instead, provide checkboxes and ask respondents to select their “top choices”, or a specific number or to select a range (such as between 3 and 5).
7: Don’t lead the jury
Make sure that your surveys are properly finished, and don’t give people the impression that you ran out of time or questions.
You will know yourself that many surveys have a final question that’s open-ended. An example is:
Q. Please provide any additional comments you think would help us.
To a respondent, this can come off as “We were in a hurry while making this survey and probably didn’t think it through completely. Can you help us out?”
This isn’t a good way to finish a well-designed survey. It’s better to ask a definite final question, one that has a purposeful outcome.
Get more reliable results, all the time
No matter how many surveys you have created, you can still create biased or tiring questions.
To avoid it, have an impartial third party review your questions for every survey that you create. It’s a great way to ensure you’re getting the reliable results you are after.
Spark Chart can help
Spark Chart has expert survey consultants and the ultimate survey software. Contact us if you would like advice.
