Welcome to the third installment of our series on DIY survey design! We’ve had some great discussions centered around focusing our intentions and building questions that drive precision. If you’ve yet to read previous parts of the series, take a moment and start from the top! Today, we’re looking at how to map logical survey routes, returning to themes of empathy and thoughtfulness that drive better responses and boost data quality. We’ll talk about helping your respondents understand what you have in store for them, planning to ask and listen efficiently, and how to use quotas to your advantage. Y’all ready? Let’s get started!

Be Thoughtful About the Flow

As always, let’s take a minute to imagine our survey as a real-world conversation. We’ve said it before, but it bears repeating the ways in which a well-structured conversation resembles a well-designed survey—so consider that when mapping out the flow for your respondents. 

Start your survey out with basic questions, moving anything that isn’t intended to screen or orient your respondents towards the end. As the conversation progresses, build gracefully into deeper questions and give places for your respondents to relax, and take a breather from the heavy-hitting questions. The idea here is to avoid saving the deeper probing questions for the end—conversations that end in really personal questions often leave both parties in an awkward spot. 

When it comes to question types like MaxDiff, Conjoint, or repetitive evaluations, your best bet is to sprinkle them throughout the middle of the survey. Considerations like these will make sure respondents aren’t exhausted with the home-stretch of the survey, and almost certainly lead to a more enjoyable experience—and better data quality!

Help Them Know the Path (Part 2 of 4)

Have you ever done a group workout where the leader of the exercise doesn’t give you any indication of how long you’ll be doing each section? It’s insane. Oh, you’ll get a workout, but it’s far less enjoyable when you have no idea how to pace yourself. Think of how differently you’d go about the experience if you knew the path. When you’re creating a survey—especially if it’s a complicated one—it’s very helpful to give your respondents an understanding of what to expect.

Another way to help your respondents know the path is to create guideposts as they travel along the route. A great application of this technique is to build in meaningful breaks in your questions—even a message that lets them know they’re halfway there can help respondents stay focused. You can also create groupings in questions with section names like  “Help Them Know the Path (Part 2 of 4).” 

The more you can help respondents understand where they’re at in the survey process, the less they’ll feel disoriented and able to give their best. For obvious reasons, knowing they’re not endlessly spiraling into an abyss of questions is a good thing. Taking these measures can help you reduce the bounce rates of your surveys and provide you with the quality data you need rather than several half-finished surveys. Ok, onto the next section—you’re halfway through!

Plan to Ask and Listen Efficiently

It’s important to note that planning isn’t all about respondent experience. As you map out your survey, build a plan that makes things easier for you as well. Start with the way you choose your questions. Although it seems easier to ask a lot of open-ended questions—especially if your survey has several iterations, try to be selective and choose your most meaningful questions. Getting hundreds of open-ended responses that read “none” or “nothing” isn’t always the most helpful data. Prioritize your meaningful questions. It pays off in analysis. 

Next, consider how you can use logic to help you ask and listen more efficiently. Depending on your approach, you can improve efficiency with thoughtful routing. For example, asking likes and dislikes on a series of concepts gets really hard to answer once you’re fatigued, but using “reduce by relevance” is a great way to keep the number of questions down for any given respondent. When it comes to questions revolving around likes and dislikes, if there are strong “top 2” box scores, use logic to focus on what they like—if they’re on the lower end of a concept rating, focus on what could be different, changed, or improved about the product. If there’s not something that stands out to your respondents, you’ll get hundreds of open-ends with “none” or “nothing,” and those won’t be helpful. 

The more you can think about getting to the meaningful content upfront, the better off you’ll be when you analyze your data. The aytm survey platform also allows you to randomly reduce questions, which makes you able to probe deeply on a few areas per respondent. That’s right, it’s totally possible to focus in on what you need while keeping your response quality high.

Use Quotas to Your Advantage

As you consider your survey route, really mull over what you’re expecting to see in the world of your respondents. You probably have an idea of the incidents or frequency of certain behaviors types or categories of users, but have you thought through how your respondents might fall out? If you’re not careful, you can paint yourself into a corner by narrowing your search a little too much. If you need a significant amount of a certain group, build it into the quotas upfront. If you’re unsure how to do that, let us know—we can help!

Quotas can be super powerful. Sometimes they can be so powerful that they hinder you from achieving a robust dataset. When it comes to setting quotas, our advice is to try and avoid over-engineering the sample. Instead, use them thoughtfully and sparingly in order to ensure that get the data you need. If you know what you’re looking for, then work to ensure you have enough data—especially if you’re planning on cutting your data in certain ways. 

One Last Reminder

Before we wrap things up, let’s return to the analogy of the exercise class. As the designer of the survey, you are the one leading the workout, and if your respondents are new to the subject matter, they might perform better if you help them understand the survey path. As the leader, you can encourage them to keep going and give their best effort—proper guidance will help them deliver the best data possible. 

As you do this, listen and ask your questions efficiently. Don’t push them too hard too fast—give them breaks, and use the tools at your disposal to help reduce some of the more strenuous stretches of work. They’ll be more inclined to finish strong!

Listen to their feedback effectively—focusing on top 2 responses for open-ended questions about what they like, or, alternately, asking what could be improved for lower ratings. And remember, if you require a certain number of responses or types of respondents, make sure to build that into your survey ahead of time. Just make sure your quota will provide you with the desired amount of data. Just don’t overuse quotas, that way your survey data will remain as realistic as possible.

Interested in learning more go ahead and check out Level Up Your Survey Game: Part One and Level Up Your Survey Game: Part Two of this four part series. Be sure to join us next time where we’ll look at ways to simplify the survey journey.