Sometimes it is easier to spot someone else’s mistakes than to spot your own. Maybe this is why so many awful drivers seem to be zealous “back seat drivers.”
The same applies to survey design. Sometimes you are so close to your own survey designs that you don’t see the possible potholes until you hit them. So to help you learn how to write greats surveys, here are a few sample survey questions. Each one has at least one error. Can you find them?
Did you spot the error?
What is “significant? Five percent? Twenty? It is totally subjective. So how will you interpret the results? The only thing you will learn is that some percentage of people considers their change “significant”, but you won’t know what that really means. Maybe that is fine for you—maybe you only care about the perception of significance. But usually, you will need something more precise in order for the data to be useful.
So what did you think of that one?
There are a couple of issues. While asking for recall over ten days is a lot more realistic than, say, had it asked about thirty, it is an odd unit of measurement. People tend to think in terms of weeks. Asking for 10 days means people have to take the time to think which specific days that includes.
Secondly, it is onerous. It could have been made a lot simpler by offering ranges for the spending amount (instead of having it be a write-in response).
Review the following example, which I saw recently in an online survey from a very well-known travel company:
Did you catch that one? Pretty egregious. The text anchoring the ends of the scale do not match the question wording. This is just sloppy. Does it matter? Yes; respondents get turned off when they can see that the survey designer didn’t care enough to do basic editing. It leads people to wonder, “How much do they value my opinion if they didn’t care enough to do basic proof reading?”
Not sure if you can edit your own work as objectively as you critiqued these three examples? Have a colleague proof your work; sometimes you need that clean set of eyes to spot the flaws. Yes, sometimes you actually want a “backseat driver”.