Surveys are valuable tools for gathering insights, but their effectiveness hinges and proper execution from a number of angles. If not conducted correctly, surveys can produce inaccurate or incomplete results, wasting both time and resources. While survey design often receives a lot of attention, survey launch, or the process of fielding a survey is often overlooked, despite being equally as important.
Don’t let the success of your surveys be compromised by overlooking the importance of a well-executed survey fielding strategy.
We recently hosted a webinar on survey fielding called “Survey fielding best practices: Advice from the experts.” Joining us were two experts: Cody Silver, Senior Consultant in Customer Success at Cint and Rossi Dobrikova, Senior Panel Operations Director at aytm. During the webinar, our experts explored topics such as optimizing survey launch times, managing holiday impacts, executing multi-country projects and utilizing census balancing effectively in the hopes of equipping participants with the knowledge and strategies for more effective and impactful surveys.
Survey timing factors
“If you launch a study about candy consumption or purchasing behavior right before Halloween your data will be very different from any other time during the year.” - Rossi Dobrikova, aytm
In the dynamic landscape of survey research, the timing of survey launch serves a critical role in ensuring high-quality data. Beyond just the time of day, variables such as the season, month and even proximity to holidays can have a significant impact on survey results. Examining the timing of survey launches reveals several factors that can play a role in the survey results, including considerations like the country’s time zone and traditional work hours.
Launching a survey during a specific time can impact the audience available to actually take that survey. For example, when targeting a teenage audience, it is important to consider school hours when selecting a launch time. Launching the survey during school hours may not only result in fewer respondents during that period but also may attract a different demographic, such as students who use their phones during class or those skipping school.
The day of the week is another essential factor to consider when survey fielding. If you are hoping to target parents, launching a weekend study may yield more responses, leveraging the increased availability that many parents have on the weekends.
It’s also important to consider holidays in the locations where the survey is being conducted and how they may affect survey results. Holidays not only affect response rates due to fluctuations in individuals’ available free times but can also impact the nature of their responses. For instance, launching a survey on candy consumption just before Halloween or exploring consumer spending between Thanksgiving and Christmas can yield unique insights not attainable at other times of the year. A thoughtful approach to survey launching times, considering factors like day, time, and holiday relevance is imperative for obtaining accurate and meaningful survey data.
Strategies for building your sample
“I don’t recommend using sample census balancing if the incident rate of your survey is lower than 70–75%. If 30% or more of all respondents on your survey terminate on your qualification questions it indicates that your audience is likely not representative of your census.” - Rossi Dobrikova, aytm
Census balancing
Let’s start with one of the more nuanced approaches to sample, census balancing. According to experts Rossi Dobrikova, aytm and Cody Silver of Cint, its application should be carefully considered. Both Rossi and Cody feel census balancing is best reserved for studies where demographic accuracy is crucial, such as market sizing or gaining a true understanding of a target audience. While census balancing can mitigate sample biases and provide a more accurate reflection of the population, there are potential challenges and limitations.
The recommendation is to only employ census balancing for surveys with incidence rates above 70–75%, emphasizing the focus on demographic criteria rather than census data when a significant portion of the population terminates on screening criteria. Rossi suggests that census balancing makes sense when researching entirely new topics, where the population of interest is unknown, allowing the incoming sample to align with census data organically.
Overall, this underscores the importance of strategic decision-making when deploying census balancing, tailoring its use to the specific demands and characteristics of the study at hand.
Estimating IR
When faced with the challenge of estimating survey incident rate (IR) inaccurately, there’s no need to panic. If the estimated IR falls below expectations, it’s advisable to adjust fielding expectations, and if possible, potentially revisit budget considerations. Cody Silver recommends incorporating “soft launches” as a tactical approach to gauge the actual IR allowing for a more accurate assessment before diving into a full launch. Building a “soft launch” into your current process ensures a more informed decision-making process and enhances the overall effectiveness of survey implementation.
Complex survey practices
“Be as consistent in the levers that you can pull so that when the data comes out, you can look at it without concerns of other variables.” - Cody Silver, Cint
Multi-country projects
Navigating the complexities of market research across multiple countries requires a nuanced approach. There are many different factors to consider when fielding a survey in a different country. Arguably the most important factor is language. It’s critical to field the survey in the native language of each country whenever possible. Other factors to consider include time zone variations, local holidays, cultural customs and sensitivity to certain demographic questions that may have cultural nuances such as relationship status questions.
Tracking surveys
For optimal results in fielding tracking studies, adhering to best practices is essential. Consistency is a fundamental principle, and conducting these studies at a similar time of the month or week ensures that any observed changes are reflective of actual trends rather than external variables.
Rossi Dobrikova of aytm stresses the importance of knowing from the very beginning what the duration of the tracking survey is so a corresponding sample blend can be maintained by the sample provider. Different panels have different recruiting methodologies, and changes in the sample blend may lead to data shifts. Rossi notes that it’s also important to assess metrics after the first wave. If IR is lower than anticipated, for example, feasibility may need to be re-assessed and sample blend may need to be re-done to account for the lower IR.
Media-enhanced surveys
“First and foremost, you’ve got to set the expectation. You’ve got to tell the respondent as they enter the study what the goal and the purpose is and give them an opportunity to opt-out right then and there.” - Cody Silver, Cint
In the evolving landscape of survey design, the integration of diverse response options, such as photo or video, provides a new level of insights for researchers. However, for many respondents, this additional requirement may lead to an incomplete survey. Rossi and Cody suggest adding a disclaimer at the beginning of the survey to ensure that respondents are aware of the requirements necessary to receive compensation for their survey response.
Clear communication from as early on as the survey invitation stage allows participants to make informed decisions about their involvement and understand what they are being asked for from the very beginning. For both photo and video responses, it’s important to provide clear instructions on aspects like lighting, external factors and even dress code. Specifically for video, it’s suggested to not make responses mandatory due to potential discomfort with being on camera. Incorporating these recommendations can ensure a smoother and more effective addition of photo and video responses into survey research.
Participant engagement
“Let’s make sure our surveys are well-designed and thoughtful. We want it to be an enjoyable experience for them, not just a regurgitation of questions.” - Cody Silver, Cint
Crafting an exceptional respondent experience goes beyond simply having an interesting topic and offering appropriate compensation, although both elements are undeniably crucial. A positive survey-taking experience plays a pivotal role in ensuring respondents provide thoughtful, accurate answers and more importantly, actually complete the survey.
One factor to focus on is survey length. Lengthy surveys with a 20–30 minute LOI (length of interview) are becoming obsolete. It’s not just about the length of surveys though; the way people take surveys is evolving as well. Shifting towards mobile-friendly surveys is key. This aligns with the modern lifestyle and makes the survey-taking process more convenient and enjoyable, allowing respondents to take surveys anywhere and everywhere.
Thanks for joining us!
We just went through a quick recap of the best practices we covered in our most recent webinar on survey fielding. But if you’re looking for more in-depth details on these concepts, including more details from the experts, check out the full recording of the webinar.
.jpg)
