Overcoming Respondent Inability and Unwillingness to Answer

A structured survey is the most common quantitative method for collecting primary data from a sample of a population. The sample of respondents is asked a variety of questions regarding past or future behavior, attitudes, motivations, lifestyle, and demographic traits. There are several advantages to using a survey methodology: the questionnaire is easy to administer, the data are reliable, and using fixed answer choices reduces any interviewer bias or variability that may have resulted in a phone or in-person interview, for example. A disadvantage; however, is that respondents may be unable or unwilling to provide the information you’re looking for. When writing your survey, consider the following information and techniques to help encourage more informed and accurate answers from respondents.

Understand the survey taking experience from the respondents’ perspective

A few key factors affecting the respondents’ experience include perceived anonymity, social desirability, and control.

‍Perceived anonymity refers to the respondents’ perception that their identities – also known as Personally Identifiable Information (PII) in the Market Research industry – will not be revealed. This not only includes information like full name, home address, and phone number, but their specific responses to survey questions as well. In internet surveys, perceived anonymity is high because there is no contact with an interviewer while answering questions. Data security is very important, and Market Research agencies may take additional measures in managing the collection, usage, and storage of data by following the Information Security standards outlined by the Insights Association.

‍Social desirability refers to the tendency of respondents to provide socially acceptable answers, regardless of accuracy. Because perceived anonymity is high in internet surveys, evidence suggests they are less susceptible to social desirability bias and, therefore, are a good way to obtain sensitive information.

‍Control refers to how much or little control respondents have in the interview process, specifically, when and where they take the survey, and how long they can take to complete the survey. Internet surveys provide respondents with a lot of flexibility in these areas, offering them a great deal of control over the process.

Are respondents informed, can they remember, and can they articulate a response?

Certain information may be difficult for respondents to report: they may not know the answer, may not remember, or may have a hard time articulating. Using screening or prequalification questions at the beginning of your survey can help ensure the sample you are interviewing are familiar with the topic. You can ask additional filtering or branching questions within the main survey to measure awareness and past experiences before following up with more specific questions. Skip pattern logic is useful if respondents aren’t aware or informed about a subject and you want them to advance to a different section in the survey. Also, including a “don’t know” or “not applicable” answer option can further reduce uninformed responses without reducing overall response rate. You want to be careful when including “don’t know” or “not applicable” options, however; try to include an exhaustive list of answer options whenever possible, or add an “other (please specify)” option that respondents can select and write in their answers, so they don’t default to selecting “don’t know”.

Respondents have a difficult time remembering specific quantities of products they’ve used/consumed in the past. For example, if you want to know how many cases of water respondents purchase each month, consider utilizing actual purchase behavior data captured at point of sale instead of relying on self-reported behavior. People are more likely to remember events or behaviors that are important to them, occur frequently, and/or more recently, such as reporting behavior from their latest shopping trip.

Likewise, relying only on open-ended questions to collect certain information may not provide you with useful insights because not everyone can articulate their ideas or opinions if asked about them generally. While it’s often useful to have respondents share unaided information (like brands they can recall within a particular category), consider following up with a question that presents an aided list that respondents select from. If you include too many questions that respondents are unable to answer, they may be discouraged and abandon the survey entirely, or it may affect their responses to subsequent questions. Provide pictures and descriptions when applicable.

Reduce respondent effort and include contextual information

Even if respondents are able to answer a question, it does not mean they are willing to. Unwillingness can occur if too much effort is required of them, or they perceive the topic or question to be inappropriate, illegitimate, or overly sensitive. To reduce respondent fatigue, use structured questions that specify a limited list of available answer choices whenever possible. Survey context is also influential; given the context, respondents may or may not be willing to share certain information. They may not deem the purpose of the survey to be legitimate because they don’t understand how their responses will be used. Explaining why the data are being collected or how they will be used in context will help overcome this. Similarly, respondents can be unwilling to disclose sensitive information that may cause embarrassment or be controversial, such as information related to finances, family life, and political and religious beliefs. Consider asking sensitive questions at the end of the survey. By then, respondents should be more comfortable answering because rapport and legitimacy have already been established. You can also preface a question with a statement that demonstrates the particular behavior being asked about is common, so they don’t feel embarrassed or discouraged.

The Takeaway: Respect Your Respondents

Surveys are excellent tools for collecting information from a sample population, but it’s important to respect respondents’ privacy, ability to remember, and time.  Providing context whenever possible establishes legitimacy and encourages honest responses. In recall questions, focusing on the most recent-past enables respondents to provide you with more accurate data (and they aren’t frustrated trying to remember). Limiting the number of open-ended questions, as well as providing complete lists of answer options in structured questions, reduces the effort needed to complete your survey. Respecting your respondents by following these tips will help ensure your survey captures the insights you need to answer your research and business objectives.