.jpeg)
My father was the joker in my family growing up, and he had a way of answering a question that used to drive me mad. Asked to make a choice between two possible options, he would unfailingly respond, “Yes”. It would go like this:“Would you like blueberry or apple pie, Dad?”“Yes, please!”Accurate, yes. Truthful, yes. Helpful? Not so much.
In our quest to keep online survey questions and answer options as simple as possible, it’s tempting to frame our questions in a “yes or no” format:• Do you like our new logo?• Do you plan to download any MP3 files in the next seven days?• Do you like chocolate-flavored ice cream?Obviously these are very simple examples, but they show that some questions can indeed be asked in a yes or no format. But should they be? You’ve received a little information, yes, but there’s no filling to the pie. There’s no “why”. There’s no comparison, nor context. Take the logo example. Maybe 80% say “yes.” Great. But did they dislike the old one? Maybe like the old one even better? Did they like the new one a little, or like it a lot?Sometimes simple is fine. But be sure that the information your next online survey collects will be useful.
Building in Bias
One reason we sometimes reject the yes/no option is because it can build bias into your question. Let’s go back to that first example again:“Do you like our new logo? Yes or no.”As a respondent, how would you be inclined to answer that question? People are actually pretty polite, and a “yes” answer seems much more agreeable. It takes a lot more strength to say, “no.”There are many alternate ways to tackle the logo example. Here is just one: “Which of the following words best describes our new logo?”, followed by a list of options such as “fresh, fun, clever, complex, confusing, boring...”This approach could give you the filling for your pie. By including both positive and negative choices in our list, we’re inviting a little more thought about our question. It’s not that we’re fishing for the negative news. We just want to make sure that the respondent feels comfortable giving us an honest response.
Watch Out for Social Expectations
There are clearly cases where yes/no works, but it usually works best for collecting factual information. “Do you currently own a 3D TV? Yes or No”, for example. It’s factual, so it works.We need to be careful, however, about the issue of social desirability. Let’s take another example. “Are you currently employed? Yes or No.” Some people simply feel self-conscious saying “No.” So they may lie, or drop out of the survey. Anywhere there’s the potential for social or peer pressure, there’s a potential inclination to respond “yes” even though “no” would be more truthful. Asking about voting behavior is a classic example of this effect; many people will say they vote, even if they don’t.The bottom line: in cases where social desirability is a risk, avoid yes/no questions and use alternate approaches.
Know when to say “No” — or “Yes”
When can you use yes/no questions? If it’s factual information, absolutely. If you're confident your answers won’t inject a bias, certainly. Yes/no answers are a very clean, simple format. For those people who are designing primarily online surveys, as we are here at AYTM, the simpler the screen looks, the better. There’s no need to clutter up our user interface with scale options if a yes/no answer will do. But if you’re asking someone like my father what he’d like for dessert—or what he’s likely to buy—you’ll need to dig a little deeper than that to get any real insight.
