How to create survey rating scales that guide respondents effectively
Survey Rating Scales are one essential component of survey survey design. Key questions to ask when designing rating scales include:
- How many scale points should my survey have?
- How should response options be labelled?
- Is a middle response necessary?
Goals when creating survey rating scale points and labels
When creating rating scales, the goal should be to make the scale:
- Easy to interpret
- Interpreted similarly by every respondent
- Able to differentiate respondent views as validly as possible
- Return reliable data (meaning, if it was asked again, respondents would give you the same response)
- Has points that map its underlying idea as closely as possible.
Together, these goals can help to make your scale is valid, reliable, and easy to complete.
How many rating scale points should my survey have?
The short answer to this question is, ‘it depends’. With each scale, the aim is to give respondents a way to differentiate themselves without cluttering the scale.
Five point scales are most popular followed by seven point scales. These scale ranges tend to work best. Beyond a 7 point scale participants have trouble responding the same way if they repeated the survey, so we recommend staying away from anything above 7 points.
Here is an example of a 5 point Likert Rating Scale:
- Strongly disagree
- Disagee
- Neutral
- Agree
- Strongly Agree
You may also consider providing a “Don’t know” option or “Prefer not to answer” option with a value of say “0” which is excluded from any response calculations.
Is a middle response necessary?
If you listen to your common sense, it might suggest to you that giving people a middle option allows them to avoid taking a real position. But this is not necessarily true. Some research has shown that even when forced to pick a side, middle-ranking respondents wouldn’t necessarily respond like the others would.
If you decide to eliminate a middle option, you may be forcing responses. When you do this, you introduce bias into the research, and run the risk of the research not being valid.
How should response options be labelled?
You will no doubt have seen all manner of option labels. Some people label only either end. Some people label every point. Some people only label the ends and middle. Then, there’s the question of whether or not to use just text, or just numbers, or some kind of blend.
So, what’s the right way to do this?
The best (read: Most accurate) survey rating scales have labels that are clear, specific, and say exactly what that point means. When you remember that we want every respondent to be able to interpret the scale option in the same way without difficulty, this makes complete sense.
Scales that are partially labelled have been shown to be less effective than those that have a full set of labels.
A good comparison is a road with roadworks. A well-signed road gives you confidence about which lanes are merging, and where you need to be to avoid either collision or stand-still. But a poorly-signed set of roadworks leads to confused motorists, incomplete movement, and high levels of frustration.
This is what poorly labelled rating scale questions can be like. Your survey labels are the signposts that your respondents need to know exactly where they are, and exactly how it’s best for them to answer the question.
Some final tips
There are some other things you can do to make sure that your data is consistently high quality. As you practice labels for your scales, make sure that:
- Every survey rating scale option is labelled with words, so the meaning is clear
- Every option is relevant to the question
- Language is consistent throughout the whole survey
- Use scales that balance well (have the same number of options on each side)
- Consider providing a “Don’t know” option or “Prefer not to answer” option
By doing these things, and following the guides above, you will consistently create survey rating scales that guide your respondents effectively.
