This is a Test of your Online Survey Critiquing Skills. This is Only a Test. Part 2

aytm logo icon
Posted Aug 30, 2012
Kathryn Korostoff

Sometimes it is easier to spot someone else’s mistakes than to spot your own. Maybe this is why so many awful drivers seem to be zealous “back seat drivers.”The same applies to survey design. Sometimes you are so close to your own survey designs that you don’t see the possible potholes until you hit them. So to help you learn how to write greats surveys, here are a few sample survey questions. Each one has at least one error. Can you find them?Designing a survey? Create a new market research study, and get the valuable answers you need, with beautiful survey design built in automatically. As a market research company, we at AYTM are here to help you quickly and easily test ideas for your new company, product, service, or homework assignment. Learn more

Example 1

Did you spot the error?What is “significant? Five percent? Twenty? It is totally subjective. So how will you interpret the results? The only thing you will learn is that some percentage of people considers their change “significant”, but you won’t know what that really means. Maybe that is fine for you—maybe you only care about the perception of significance. But usually, you will need something more precise in order for the data to be useful.

Example 2

So what did you think of that one?There are a couple of issues. While asking for recall over ten days is a lot more realistic than, say, had it asked about thirty, it is an odd unit of measurement. People tend to think in terms of weeks. Asking for 10 days means people have to take the time to think which specific days that includes.Secondly, it is onerous. It could have been made a lot simpler by offering ranges for the spending amount (instead of having it be a write-in response).

Example 3

Review the following example, which I saw recently in an online survey from a very well-known travel company:Did you catch that one? Pretty egregious. The text anchoring the ends of the scale do not match the question wording. This is just sloppy. Does it matter? Yes; respondents get turned off when they can see that the survey designer didn’t care enough to do basic editing. It leads people to wonder, “How much do they value my opinion if they didn’t care enough to do basic proof reading?”

Conclusion

Not sure if you can edit your own work as objectively as you critiqued these three examples? Have a colleague proof your work; sometimes you need that clean set of eyes to spot the flaws. Yes, sometimes you actually want a "backseat driver".Still confused about how to design your survey?Easily create a new market research study right now on our Target Market page to test your concept(s), where your survey design will already be optimized for online and mobile surveytaking. If you have any questions about how to construct your study, email support@aytm.com or call us at +1 (415) 364-8601 (8am-6pm Pacific Time).