Avoid these survey pitfalls
It’s common for people who design surveys to focus on writing engaging questions. In the process they don’t avoid survey pitfalls. The focus can be on making the survey look fantastic. The piece that is often missed is the all-important pre-launch work that helps make surveys successful.
In this article we look at four common survey pitfalls, and how you can avoid them.
Survey Pitfall 1: Not testing the survey
As much as we like to think we are, nobody is perfect. With your survey, you only get one chance to make a good impression.
No matter how interactive your survey is, if your survey is filled with errors or vagueness, then you have just dramatically decreased the chances of getting a good response rate.
Testing for errors by yourself is not encouraged: When we build things, we often can’t see the wood for the trees!
To avoid this problem, get other people to take your survey before you launch it. Let them make notes about things they see as being problems. Use this information to refresh how you see the survey.
Tips for good tests:
- Ask as many colleagues as you can to help them, and encourage them three or four times.
- Ask for feedback as though they were respondents
- Ask them to test different survey response combinations
- Engage a group of respondents and pilot the survey with them.
And then, if time permits, do the test again.
Survey Pitfall 2: Not giving enough thought to response values
Response values are the values that you assign to survey options. What you want to do is make sure that they are crystal clear to you. Your respondents may not see them, but you will – often much later.
Your reporting values can help you with data analysis, if you consider them properly.
To avoid this problem, think carefully about whether you need response values to help you with your data reporting. Find out if there are any requirements that you might not already know.
Here’s an example. Let’s look at this Likert Scale of possible responses:
- Strongly disagree
- Disagree
- Neutral
- Agree
- Strongly agree
In this survey rating scale, your response values might be assigned as follows:
- Strongly disagree
- Disagree
- Neutral
- Agree
- Strongly agree
Then, when you get your results, you will have a number that quantifies your responses, and allows you to analyse results.
If you set your response values, remember to test them to make sure they work!
Survey Pitfall 3: Not thinking about how you intend to report the data
Sometimes we get really creative, building surveys that have many different question types. For the respondent they can be fun and interesting to complete.
But a warning: If you haven’t clearly considered your goals for the survey and the data, being creative like this can put roadblocks in front of you.
To avoid this problem, think about what you would like a summary of the data to look like.
Question responses can be reported in different ways.
Any time you use a new survey question type, make sure you test the reporting. After all, it’s not the survey that you will use, but the reported data once it is completed.
Survey Pitfall 4: Using open-response questions when you don’t really want to read them
If you don’t want to read 1000 essay responses, don’t put essay responses into your survey. While there may be a text analysis tool available to you, it is far better to assume that there isn’t.
When you ask for open responses, someone (usually you) is going to have to go through every response. Manually. And you’re going to have to do something with them so that the data can be used meaningfully.
So, if this isn’t something you really want to do, don’t ask open questions and find a way of getting the data you need from closed questions instead.
Avoid these pitfalls ensure success
Taking the time at the beginning to think about how to avoid survey mistakes can solve a lot of stress for you. Well designed surveys are worth every second you spend on them. Data analysis is already difficult; don’t make it unnecessarily so.
