Extreme Makeover... for Survey Questions!

Avocado colored appliances and wood-grain paneling from the ‘80’s. Bold plaids and shag carpet from the ‘70’s. Worn linoleum and tarnished chrome from the ’60’s… What this house needs is a makeover! And while we’re at it, let’s take a look at your survey questions. Far too often they’re in the same dire need of an update; following are two areas that frequently need attention.

Online Survey Questions to Gauge Willingness to Pay

It happens all the time. “How much are you willing to pay for product X?” (or, “How much do you think product X is worth?”), followed by a list of price options from low to high. “X” may be an application, a food product or a game system, but the effect is the same. People tend to pick the lowest price when they see this type of list, and you’ve learned nothing of value with your question.

• Offer a tradeoff—a better approach is to give them a tradeoff: “Would you prefer an application with feature “A” that costs $50, or one with feature “B” that costs $65?” To gain even more insight, the answer options can be “A”, “B”, or “neither.”
• Form subgroups—alternatively, (and assuming your sample size is big enough); split your survey into 3 versions. Using the AYTM cloning feature makes this easy. Now you can ask each group a different price point and get an honest answer to, “How likely are you to purchase application X at price Y?”
• Ask a series—if you don’t have a large enough sample group, you might ask a series of price questions in the same survey. Each question will appear on a separate page, of course beginning with your highest price. If 30% will buy at your high price, and another 10% jump on at the next price lower, that’s very interesting. And what if 5% accepted your hypothetical offer at the highest price, but another 25% find the next lower price agreeable? Now you’ve really learned something about your target market’s price sensitivity.

Just remember that pricing research isn’t perfect. You still need to consider competitor and substitute pricing so that when you come to market with your fantastic new product, potential customers won’t balk.

Seeking Priorities, Getting Wish Lists

We often need to gauge priorities. For example, imagine that you’re testing an idea for a new iPad game, and you want to prioritize several possible features for Release 1. Too often researchers in this situation frame their questions using a multiple check option format, showing a list of feature options and asking the respondent to check their top 3. Unfortunately a multiple choice question format doesn’t enforce the limitation of “top 3”, and even though you’ve asked nicely, respondents will make mistakes. This leads to inconsistent data because they’re not being "forced" to choose their top three. Some people will even like all the features and check them all! And even when they do choose three, the data doesn't tell you which is the first, second and third choice —it doesn’t prioritize. The solution is to use a rank order question type, which provides answers in order of preference.

Online Survey Design With The Latest Style

It’s time to take a look at your question style and format, and ask yourself the hard questions. Are my surveys out of date, out of style, showing wear? It might just be time for an extreme makeover.Image: williamcho