The Open-ended Question

Pop quiz: What’s a common open-ended question you hear and say a dozen times a day, and don’t even realize it?How about, “How are you?”Basically, it’s a question without prompts or obvious expectations, and no list of pre-scripted answer options. It’s a wild card. In response to, “How are you?” Katharine Hepburn once quipped; “I’m alright, unless you want the details.”You just never know what reply you’ll get.

To apply the concept to online surveys, imagine you’re doing a survey about eating habits. You might ask, “Please indicate how overeating has affected your health”, and follow with a list of answer options such as weight gain, lowered self-image, diabetes, etc., expecting people to check one or more answers.But alternatively you might ask, “How has overeating affected you?” with an open text box for the response. The former is a close-ended question, the latter open-ended.

A useful tool for online surveys…

In spite of the potential for surprises—or more accurately because of that potential—open-ended questions can be very useful for a couple of reasons:• Discovery — Open-ended questions are wonderful for discovery purposes. Because people get to write whatever comes to mind, you may get responses you’d never have thought to include in your list.• Language — because they use their own words to respond, you have the opportunity to discover reoccurring themes in the language they choose. Knowing what word choices your customers use gives you the chance to leverage it into marketing that will resonate with them.

…With a down-side

Open-ended questions do have disadvantages for you as a researcher, however:• Effort for them — not everybody likes to type, so they require more effort from your respondents.• Effort for you — Open-ended questions are harder to analyze. Sorting and coding hundreds of open-ended responses so you can see which ones occur most versus least frequently is a lot of work.

Use sparingly and well

A few well placed, well-structured open-ended questions can provide a wealth of information. For instance:1. Good & Bad — in a survey for customer feedback you might ask, “…in your most-recent visit to our store what did you like most?” Of course, to have a balanced perspective you should follow up with “…at your last visit to our store what did you like least?” Or, to allow for broader feedback, you might ask, “If anything, what could we improve about our store?” This gives them an opportunity to offer feedback and suggestions without “leading” them in any particular direction.2. Get at the WHY — Imagine you’re doing a survey to understand customer interest in a specific feature of a new software application. You might ask, “How important is Feature X?”, and then ask them to rank it on a Likert scale ranging from “not at all” to “extremely” important. Now, you can set up the online survey to ask a subset of those people a follow up “why.” For those who answer “extremely” you might have an open-ended question. “Can you help me understand why this is extremely important to you?” This ability to get at the “why” can add a lot of richness to your data analysis.3. Do a wrap-up — Say you are doing an online survey about Facebook activity — what people like about Facebook, how they use it, etc. As a final question you might ask, “Are there any issues about Facebook not addressed in this survey that you think we should know about?” This “catchall” question creates an opportunity to discover issues you might not have thought of.

Unconventional Examples

Finally, try to think outside the box. Much of the value in a flexible tool like the open-ended question lies in how you wield it. Here are some other examples I’ve used that have yielded great and unexpected results:• “If you could have dinner with the CEO of any company, who would it be and why?” (from a study on brand perceptions)• “Please complete this sentence: “I’d spend more time online if….” (for a study on online behaviors)• “What, if anything, is most annoying to you about using product ‘ABC’?” (following a survey exploring customer needs)

Bottom line

Open-ended questions have disadvantages, yes, but they can provide the “seasoning” that makes your survey special. Sure, closed-ended questions are better for quantitative analysis. But they don't always get at the story behind the data. Sometimes, Ms. Hepburn, we really do want to know the details.