How to design great surveys to get the best data

As humans we are lucky to be able to picture the results of what we do, before we even do them. And when we do, we find that we are more successful. Surveys are no different. When you begin the design process knowing your desired outcomes, you are more likely to reach your goals.

The basics of good survey design

Designing a good survey means starting with the right questions. The answers to the questions below will help you start thinking about the best way to get the information that you need.

The questions are:

• What are your survey goals? Write down the goals for your survey project – three is a good number. Then, write some notes to yourself about what would help to get you the data to support your goals.
• How will the data be used? Will the analysis just be of data from this survey, or will it be combined with other data? Will you use it internally or publish it? Is it just for your use, or are the results for a client?
• What kind of questions do you need to ask? Think about what you need to support your goals. What is absolutely required for you to reach your goals? (And then, what is not necessary at all?)
• Who are you targeting? Think about their ages, genders, locations, languages, the devices they are most likely to use. How will you engage your survey respondents?
• What’s the best way to distribute the survey? Thinking about all the previous questions, how will you send it? Will it be by email, via a website, or an in-person request? Or by post?
• What sort of reports do you need? It makes a difference whether you are reporting just for yourself, or for other people? And whether you need to deliver reports in other formats. If you do, think about what they are: Word, SPSS, PDF, Excel, PowerPoint, or something else).

Designing great questions

Now you know the answers to your questions, you can start designing the survey itself. The questions you create must support your survey goal, and they shouldn’t ask anything that is irrelevant.

Here are some guidelines for writing good questions:

1. Keep all of your questions short and your language simple
2. Be specific in what you ask
3. Phrase your questions in a direct way
4. Keep all questions directly relevant.

These four guidelines will help you to stick to the point, and to get only the data that supports your goal. There is no point asking about location, for example, when don’t intend to use that information for anything later.

Select the right question types

There is a range of different types of questions that you can use. Given you are reading this online, it’s likely you will be using a web-based survey platform. Once you understand the different question types, you can use only those that will get you the data that you need.

Quantitative questions are directly measurable. This means that you set up a list of answers and your respondents will choose from those. They include:

• Radio buttons
• Checkboxes
• Dropdown menus
• Rating scales

These questions will give you clean reports, easy-to-analyse charts, and will help you to identify patterns and trends.

Qualitative questions are those that let respondents tell you the answer in their own words. They are:

• Text boxes
• Essay questions.

Even though they can be more difficult to analyse, qualitative questions will show you exactly how your respondents are thinking.

Some of the ways that you can analyse qualitative questions include creating word clouds to identify common words. You can also read each one yourself, noting important elements of the response.

To get the best results, you need to use a combination of quantitative and qualitative survey questions. But remember: If you ask an essay-type question, don’t ask it up-front. Get buy-in from your respondents early with easy quantitative questions, and leave the essay questions to last.

Think about your reporting early

It’s a common problem that surveys are designed and completed, and then reports are difficult to pull together. You can avoid this by thinking about your reporting during the design stage.

This gives you the benefit of being able to include custom values that you will see in your reports, but which are hidden to the survey-takers. You might need to do this when you have to report specific values (which don’t mean anything to your respondents), or you have to analyse data in a particular way, or with a particular tool.

When you set your reporting values, go back to the purpose of your survey. Think about the values that are going to allow you to achieve your goal, and that will fit the purpose. That way, you will design questions that give you effective and useful reports.

Consider the languages of your respondents

Great surveys are designed with the target audience in mind. So, if your audience is more comfortable in a language other than English, make sure that you give them the option of completing the survey in that language.

For multi-language surveys, it’s critical that you test and evaluate the meaning of the question and all of the answer options. It is a mistake to assume that what works in English works in all other languages, because it’s simply not true. Languages include cultural considerations, so what seems simple in English may cause cultural offense.

Make sure that you work with good translators, too. Relying on Google Translate is one of the fastest ways to creating surveys that don’t make sense, or are just plain incorrect.

Well designed surveys = great data

By following this guide, you will avoid common mistakes in survey design. Paying careful attention to all elements of the design will improve your response rates, improve the quality of your data, and make your reporting  a breeze.