Including open-ended questions in your surveys can result in richer insights and deeper understanding about respondents' perspectives, attitudes, and/or behaviors. But not all open-ended questions are created equal. In this article, we offer some best practices about how to best go about incorporating open-ended questions into your market research efforts.
What are open-ended questions?
To get started, let’s take a look at the basics: What are open-ended questions? The most important thing to know is that they’re qualitative—meaning they ask respondents to provide free-form text, as opposed to quantitative questions where respondents select a response option from a list or scale provided to them.
Let’s take a closer look at open-ended versus closed-ended questions. The following table provides a comparison of the different question types.
Why should you use open-ended questions?
Moving on, there are a few important reasons for including open-ended questions in a survey—we’ll go over them below:
- You may uncover unexpected findings
In closed-ended questions, we provide a list of response options that we assume are applicable to most respondents. But very few lists are exhaustive. By including open-ends, respondents can provide answers that are most appealing or relevant to them.
- They can provide context to responses to quantitative questions
Here is a very simple example. If you ask respondents, “what is your favorite ice cream flavor?” and have them choose from a list of 5–10 popular flavors—you’ll be able to see how popular a flavor is amongst your sample but will not understand why. In addition, you have likely only provided a subset of all of the possible ice cream flavors.
If you ask this question as an open-end, then the range of response options will likely be greater.
In addition, a benefit of open-ended questions is that they can be used as follow-ups to closed-ended questions. Going back to our ice cream example, the ice cream flavor selection from this question provides the opportunity to ask a follow-up open-ended question to obtain more information from the respondents. Therefore, the open-ended follow up would be, “why is [insert flavor] your favorite flavor of ice cream?
- Open-ends add flexibility to your research
They can do this by helping you validate or test some new ideas. Perhaps you are doing some preliminary, exploratory research about a new product and need some guidance for the next phase of product development. By asking open-ended questions, you can see what ideas or themes emerge from consumers and then use those responses (also known as verbatims) to help formulate and focus quantitative research that may follow.
What are best practices when it comes to open-ended questions?
Because open-ended questions are free-form text, they are, by nature, difficult to quantify. It is important to frame these questions in a way that generates useful and is helpful to your research. Here are a few suggested best practices to help craft questions that generate usable insights.
- Be specific
Avoid being too broad or vague. For example, sweeping, general questions will not result in responses that help you make decisions. Questions that ask respondents to consider big time frames, such as the last decade, will test the limits of respondent recall. If you want to know about a specific date in time or event, then reference it in the question.
- Only ask one question at a time
Asking more than one open-end at a time may feel burdensome to respondents, plus you likely won’t get a complete answer to both questions. Here’s an example of a situation to avoid: “How was your visit at our hotel? What was your favorite part of your stay?”
- Consider if it should be a close-ended question
If the question can be answered with a yes or no, perhaps rephrase the question.
In addition, be wary of asking respondents where they live as an open-ended question. Respondents may enter various answers that may be difficult to recode, such as country, state, city, or zip code. Instead perhaps use a drop down question type with pre-populated states, cities, zip codes, or regions.
- Do not require a minimum word count
Minimum word counts can backfire on researchers because they may cause respondents to give up or enter a nonsense response simply to fulfill the requirement.
- Keep open-ended questions optional for respondents
Sometimes respondents do not have anything additional to say as a response. Forcing a response could lead to bias with respondents including ideas or feedback that was not necessarily top of mind for them.
- Limit how many open-ended questions you include
Remember, responding to open-ended questions can be more mentally taxing on respondents. It does require more effort than a closed-ended question. So be judicious about how many you ask in any one survey. Including too many open-ends can significantly lengthen the survey. Therefore we suggest limiting the number of open-ended questions to no more than one-third of your survey questions. This approach will keep the length of interview reasonable and help maintain respondent engagement.
Closing remarks on open-ends
To wrap-up, open-ended questions are a nice addition to your survey as they allow respondents to contribute, in their own words, valuable feedback. Because open-ended responses are unstructured data, analyzing the text can take extra time. It’s easy to give these responses less attention because we often focus on the data generated by the quantitative survey questions. However, examining the responses to your open-ends can add valuable context to your data analysis and further enrich the insights you share with your stakeholders.
.jpg)
