Want to obtain more qualitative data from your next survey? Open-ended questions can help lead the way. Because they're exploratory in nature, OEs offer insights professionals rich, qual data, incorporating a human element into your quantitative research.

The Quali/Quant Sweet Spot

Quotes you get from open-ended questions can put a human voice to the objective numbers and trends revealed from your results. It often helps to hear your customers describe your brand or its products in their own words to uncover any blind spots you hadn't even considered when formulating your survey questions.

For example, let's say you want to run an ad test to get consumer feedback on a new campaign. You can easily measure many things with quantitative questions, including like vs. dislike, believability, relevance, uniqueness, and purchase intent, to name a few.

But you could also offer a couple of open-ended, qualitative research questions to uncover things you may have overlooked. You could ask:

• What did you enjoy most about the ad?
• In your own words, what is the main message of the ad?
• Describe how the ad made you feel.

If you notice any common themes running through the answers to your open-ended questions, you may need to research them more in-depth or immediately make changes to the next draft of your ad.

Imagine if several panelists said that the ad's central message was something completely different from what you intended. That's a clear indication that you need to either tweak the copy, the creative, or both. 

Why Use Open-Ended Questions in Your Surveys?

When it comes to the respondent experience, there are a couple of reasons why it can be beneficial to offer up a few open-ended questions. First, OEs are shorter, because they don't include a list of answers. Giving panelists fewer words to read and answer choices to consider reduces the likelihood of survey fatigue.  

Open-ended questions also eliminate the risk of primacy bias – the tendency for respondents to pick the first option presented to them in a closed question. Instead, OEs push respondents to think about their answers. 

Another great thing about open-ended questions is that they allow panelists to include additional information that can reveal their hidden feelings, attitudes, and understanding of the subject. OEs also allow respondents to answer in their own words, without any influence from the survey designer.

Use Cases for Open-Ended Survey Questions

Open-ended questions can often lead to extremely valuable feedback and rich insights when added to a well-designed quant test. Here are a few examples of questions you can add to your next survey to add a little qual to your quant.

What did you enjoy about the product? - Uncovering your strengths helps you highlight your value to more users. For example, if your early creative concepts are heavily focused on calling out feature A, but very few consumers mention it as something they enjoyed — you may want to reconsider centering your marketing messages around it. 

What can we improve? - This simple open-ended question can serve as an open platform for consumers to tell you their pain points – which is infinitely more valuable than pulling out your crystal ball and guessing what product improvements you should make before going to market. 

Where did we fall short? - Perhaps your product promised a certain benefit that the consumer didn't experience. Knowing exactly how and where you missed the mark, can be invaluable in ensuring you improve at setting and meeting customer expectations in the future. You want to under-promise and over-deliver, not the other way around.

Open-ended questions can lead to insightful answers that can help you empathize with users and how they experience your products. Use them to compliment or get context for your closed-ended responses. You can add OEs as an optional response in single or multi-choice questions, as a text box asking for a short response or an essay question, requesting a more extended response from panelists.

Open-Ended Data Analysis

Because there's an unlimited range of potential answers respondents could give to open-ended questions, it's incredibly important to have an intuitive way to analyze them. When reviewing open-ended text, you want to identify word themes and patterns that offer more in-depth insights into the overall survey. You can categorize and calculate the open-ended text based on respondent profile, custom variables, or responses.

Filtering allows you to break down your survey results and focus on specific parts of the data. For instance, filtering by question and answer will enable you to view respondents who answered a question a specific way.

With an automated solution like aytm, we take OEs one step further with an intuitive interface, which makes it easier to analyze the data – allowing you to sort, filter, and search for specific words or phrases while seeing respondent traits – all in one place.

Once your survey is live, you will be able to access the statistics page and watch your respondents' answers stream to the page in real-time. Here's how to access and navigate your open-ended responses.

Filters allow you to search through your data and dig deeper into correlating what respondents say to the mass survey data. Aytm allows you to: 

• Filter by questions 
• Sort responses by time, length, or kudos
• See the number of responses per page 
• Toggle between pages of comments 
• Show Traits for each individual
• Search for keywords or phrases 
• Highlight search terms 
• Choose question order

Let’s say you need to quickly find every response that includes the word "cat." Simply type in your search term and press enter or click on the magnifying glass, then see how people who responded with the word cat answered your survey.

When you use the search field, you can select to have the search terms entered highlighted. In the example above, you'd see the word "cat" highlighted in every relevant response, making it incredibly easy to locate precisely what you're looking for.

Data Visualization for Open-Ended Responses

Open-ended answers are presented as an interactive word cloud on your Stats page. The aytm platform automatically creates a visually appealing graphic, where the size of each word corresponds to how frequently it appears in your responses. 

Word clouds are a simple, yet powerful graphical representation of qualitative data. They create a very compelling visual display that tells a story, showing you what words are coming up the most versus the least.

Our Word Cloud functionality now includes answer variants with open-ended text fields. To populate a Word Cloud, click to expand, and then load the word cloud, repeating for each answer option. You can also populate OE comments with a Word Cloud in a Virtual Question.

Export & Print Open-Ended Data

Want to take and filter your data offline? You can export a raw data file with only the verbatim responses. While the Open-Ended page is active, click on the Export tab, and a file containing only questions with verbatim responses will populate. This file will also include the Response ID and Date/Time column. 

When you have the Open-Ended response page open, simply click the print tab to print just the verbatims. Keep in mind that if you apply filters to the verbatims, only what’s filtered will export and print.

Inject Qualitative Data into Your Quantitative Research

Open-ended questions can add a little flavor to your quantitative studies, leading to deeper insights, a better understanding of your target customer, and a wealth of feedback that translates into smarter strategic decisions. 

Aytm's new OE interface makes searching through, filtering, and sorting your open-ended data easy and intuitive, while the visualizations on your Stats page allow you to quickly understand the data, at-a-glance, and share it with key stakeholders in one click. 

See how you can use open-ended questions in your next survey.