Extreme Makeover… for Survey Questions!

Avocado colored appliances and wood-grain paneling from the ‘80’s. Bold plaids and shag carpet from the ‘70’s. Worn linoleum and tarnished chrome from the ’60’s… What this house needs is a makeover! And while we’re at it, let’s take a look at your survey questions. Far too often they’re in the same dire need of an update; following are two areas that frequently need attention.

Online Survey Questions to Gauge Willingness to Pay

Survey Questions: Makeover EditionIt happens all the time. “How much are you willing to pay for product X?” (or, “How much do you think product X is worth?”), followed by a list of price options from low to high. “X” may be an application, a food product or a game system, but the effect is the same. People tend to pick the lowest price when they see this type of list, and you’ve learned nothing of value with your question.

  • Offer a tradeoff—a better approach is to give them a tradeoff: “Would you prefer an application with feature “A” that costs $50, or one with feature “B” that costs $65?” To gain even more insight, the answer options can be “A”, “B”, or “neither.”
  • Form subgroups—alternatively, (and assuming your sample size is big enough); split your survey into 3 versions. Using the AYTM cloning feature makes this easy. Now you can ask each group a different price point and get an honest answer to, “How likely are you to purchase application X at price Y?”
  • Ask a series—if you don’t have a large enough sample group, you might ask a series of price questions in the same survey. Each question will appear on a separate page, of course beginning with your highest price. If 30% will buy at your high price, and another 10% jump on at the next price lower, that’s very interesting. And what if 5% accepted your hypothetical offer at the highest price, but another 25% find the next lower price agreeable? Now you’ve really learned something about your target market’s price sensitivity.

Just remember that pricing research isn’t perfect. You still need to consider competitor and substitute pricing so that when you come to market with your fantastic new product, potential customers won’t balk.

Seeking Priorities, Getting Wish Lists

We often need to gauge priorities. For example, imagine that you’re testing an idea for a new iPad game, and you want to prioritize several possible features for Release 1. Too often researchers in this situation frame their questions using a multiple check option format, showing a list of feature options and asking the respondent to check their top 3. Unfortunately a multiple choice question format doesn’t enforce the limitation of “top 3”, and even though you’ve asked nicely, respondents will make mistakes. This leads to inconsistent data because they’re not being “forced” to choose their top three. Some people will even like all the features and check them all! And even when they do choose three, the data doesn’t tell you which is the first, second and third choice —it doesn’t prioritize. The solution is to use a rank order question type, which provides answers in order of preference.

Online Survey Design With The Latest Style

It’s time to take a look at your question style and format, and ask yourself the hard questions. Are my surveys out of date, out of style, showing wear? It might just be time for an extreme makeover.

Image: williamcho

ABOUT THE AUTHOR: Kathryn Korostoff
Kathryn Korostoff teaches Market Research best practices at AYTM.com for Ask Your Target Market, and is the president of Research Rockstar, delivering Market Research training and support services. She can be reached at KKorostoff AT ResearchRockstar DOT com.
  • Elle Morse

    GREAT article Kathryn! I’m going to be spinning at least two of these into my own blogging world. 
    Thanks AYTM for sharing!

  • Anonymous

    These
    all seem like awesome ways to revamp boring surveys. I’m sure we’ve all
    been there, staring down at “strongly agree” through “strongly
    disagree,” not really able to compress our opinion into a check-box. The
    questions are almost always too broad, but then ask for a yes or no kind
    of answer. If you want a specific answer, ask a specific question. If
    you want a broad answer, leave a big blank so I write down how I feel!

  • http://pulse.yahoo.com/_NTHCXABE3SIIXGB5FLDCXMOIUU Julee

    Survey questions have to be carefully formed. Even though you want honest
    opinions, you have to make sure you’re getting that opinion about
    exactly what you’re researching. There are some great suggestions in
    this article, particularly the “trade-off” question. It has to be an
    exercise in careful language use, to ask concise and well-directed
    questions that don’t give the survey-taker a chance to wander.