Do’s and Don’ts of Question Wording

Deciding how to word your survey questions may seem like one of the easier tasks of survey writing, but it is actually one of the most difficult and critical tasks. Poorly worded questions can result in bad data: respondents may refuse to answer the question entirely or misinterpret what the question is asking, and as a result, answer incorrectly. These lead to response errors and can make data analysis more challenging. It is key that both the researcher and respondent interpret the questions identically, otherwise you wind up with biased data. Consider the following six tips when writing your survey questions to help optimize wording and, ultimately, obtain more accurate data.

Be as specific as possible by using the 5 W’s

Think like a journalist and consider your question in terms of who, what, when, where, and why. Consider these two questions: “Which brand of toothpaste do you use?” and “Which brand(s) of toothpaste have you personally used in the past month? (Select all that apply)”. Although the first question is concise, it is not well-defined: “who” exactly is being addressed? Should the respondent be answering for herself, personally, or for her entire household? Additionally, the timeframe (the “when”) within which the respondent should be answering is unclear: should the respondent be thinking about all of the toothpastes she’s ever used, the toothpaste she purchased most recently, or perhaps her favorite – regardless of when it was purchased? The second question is very specific and will force all respondents to interpret the question the same way.

Use ordinary, unambiguous words and phrases – not corporate jargon

It’s important that the question wording matches the vocabulary level of respondents, so keep in mind that the average U.S. respondent has a high school education – not college — and it could be even lower in certain regions. Instead of saying “Please indicate your level of purchase interest regarding this new-to-market lotion with deep conditioning properties.”, ask “How likely are you to purchase this new deep-conditioning lotion?”. It’s also important that you select words that have a single meaning and cannot be interpreted differently by different people. Words such as “usually”, “normally”, “frequently”, “often”, “regularly”, “occasionally”, and “sometimes” can lead to response bias. Consider using a concrete frame of reference like, “Less than once”, “1-2 times”, “3-4 times”, and “5 times or more” when appropriate.

Make a dictionary and thesaurus your new best friends

When deciding how to word questions (and answers), don’t be afraid to consult a dictionary and thesaurus. Ask yourself the following questions to gut-check word choice:

       
• Does the word mean what I intended?
       
• Does the word have any other meanings? If yes, does the context make the intended meaning clear?
       
• Can the word be pronounced more than one way? If yes, are there other similarly pronounced words that might be confused with this word?
       
• Is there a simpler word or phrase that can be used?

Avoid leading and biasing questions

A leading question gives respondents a clue to the desired answer or leads respondents to answer a certain way, resulting in acquiescence bias – or the tendency for respondents to agree with the direction of the leading question. Bias can also occur if the sponsor of the survey is revealed too early because respondents tend to respond favorably to the sponsoring brand.

Don’t make respondents work too hard

Respondents have a difficult time thinking in percentages, so instead of asking them to estimate the percentage of time they spend each week participating in certain activities, have them tell you how many hours per day they spend dedicated to each activity. Likewise, don’t require respondents to compute complex calculations – save that for your analysis. For example, if you want to know how much respondents spend on household groceries per year, ask them how much they spend, on average, per week and calculate their annual spend yourself during analysis.

Present both positive and negative statements

Questions that ask for respondents’ level of agreement, such as those measuring attitudes and lifestyles, are influenced by the “direction” of the statement – either positive or negative. It’s best practice to use a mix of positive and negative statements, such as “high quality” and “poor design”.

The Takeaway

Precisely wording the questions in your survey is a vital step towards collecting correct data. Questions and answers should only be interpreted one way by all business stakeholders and respondents – that way there can be no debate on the intended meaning and resulting findings. It’s great practice to have a colleague who isn’t necessarily familiar with the research review your questionnaire draft for comprehension. Additionally, if you have the time and resources available, consider conducting an in-person pre-launch with a small, representative sample of your target market. After they take your survey, you can ask for immediate feedback on how they interpreted your questions and answers and learn if any phrasing is confusing or needs revision before launching to a larger sample. Carefully constructing your survey will help ensure your data is insightful and business resources were used strategically.