The perfect survey length: Do you know what it is?

The length of a survey is well known to impact survey completion rates, but do you know the ideal length of a survey? Surprisingly, there has been very little research done on what effect length has on survey completion rates. Here, we take a closer look at LOI and its impact on data quality. When we use the term data quality,  we refer to its “fitness for use,” which means the data is unbiased and achieves the intended purposes of the research study. The goal of any survey is to extract the highest possible quality of data every time so that organizations can get the actionable insights they need to make data-driven decisions. Let’s dive in.

What is an LOI?

LOI stands for length of interview. People use this phrase to describe the length of a survey measured in minutes. This is the total time it takes for a survey respondent to complete the survey from the beginning to the end. Nowadays, most surveys are online. The interview length for an online survey would be the time it takes from the moment the respondent accesses the survey’s web page to the moment the survey is completed and submitted to the research firm. For most surveys, this is usually only a few minutes; however, some can extend into taking a half-hour to complete. Often, organizations will seek to reduce costs by attempting to extract as much data as possible from a single survey, driving a longer LOI.  

LOI is important because research (explored in detail below) demonstrates that it impacts whether or not survey participants complete a survey. Generally, a longer LOI means a lower response rate from specific demographic groups, thereby reducing the quality of the data collected from that survey. To make accurate conclusions, leaders need precise market data. That’s why it’s vital to find the ideal LOI that doesn’t bias the results. A lot of time and money has been spent researching the perfect LOI for the average survey. Let’s take a closer look at the data and see what it says about the perfect survey length. 

What the research says

The question of how LOI impacts data quality and data degradation was the subject of research conducted in 2021. They set out to answer what happened to data quality as the length of a survey was increased. They found a couple of important conclusions. Firstly, after 10 minutes long, completion rates and data quality drop. What’s happening here? This means that people (specific demographics are more likely to do this) are getting tired of filling out responses and just abandoning the survey halfway through. 

This abandonment results in an unrealistic skewing of the data in favor of only those who don’t mind longer survey times. This study looked at the specific time of response per question in seconds to measure data degradation. They found that respondents tend to spend less and less time per question as the survey continues, a phenomenon they referred to as the Data Degradation Factor (DDF). This means that the data becomes more degraded as the survey gets longer. 

Open-ended questions experienced a particularly severe data degradation factor. These questions can be very valuable as they attempt to elicit a genuinely personal response. However, respondents tend to give them very little thought as the survey gets longer. Looking at the response times for these questions is essential because they represent a richer type of data. 

That was what they concluded after 10 minutes, but what about 7 minutes? Here, you find a more modest 25% drop in response times for multiple-choice questions and a decrease of over 50% for open-ended questions. All this data could lead you to conclude that the shortest survey is best. However, shorter surveys provide less return on investment, and the data, though very high quality, will not be in very high quantity. Finding the ideal length for a survey is more complicated than it seems. 

How aytm manages survey LOI

So, that’s the research. But how do we manage survey length here at aytm? At the end of the day, respondents are people, not machines. This means that they will get distracted and disinterested, but it also means that we can inspire them. That’s why we at aytm have focused a great deal of effort on empathizing with our global panel of over 100,000,000 respondents to get high-quality data every time. The numbers don’t lie. Our panel has consistently been one of the highest-rated on SurveyPolice for five years in a row. 

Empathy is a core value for our company, and it's a value that we always extend to our panelists. Looking at the reviews, you can see that our panelists are grateful to be a part of our team. They keep coming back because they know that we care about their time and experience, so we incorporate LOI into our quotes.  Our respondents are compensated for their time and are further rewarded for continuing to provide honest responses. We have a system called Traitscore, which analyzes the quality of responses and rewards respondents who consistently provide great answers. These features put us in the position to offer you cost-effective surveys that do not experience the data degradation effect described in the research. 

So, what’s the best survey length?

The conclusion of the research we looked at earlier was that 5-7 minutes is the ideal survey length and consistently provides the best data quality in research studies. However,  that doesn’t mean that you should always avoid longer surveys. What this research really shows is that LOI is an essential factor to consider when sending out surveys. However, it’s important to remember that LOI is not the end all be all of survey completion rates and data quality. 

For example, the quality of your survey questions can also seriously impact data quality. Some providers will also have limited sample sizes or have a limited number of demographics in their panel. This means that any responses you get from them will be of poor quality. All of these elements should be considered when building your surveys and questionnaires. 

At aytm, we feature a diverse, global panel with millions of respondents in over 40 countries and speaking 20 different languages. Furthermore, we continue to look for new ways to expand our panel into other regions. Our system allows you to design and build your own survey, making the whole process easy and fun. This also ensures that all surveys are built in the same high-quality format that provides our panelists with a great experience. With such a large panel, we give you unique targeting capabilities to focus on just the respondents you’re looking for. At aytm, we can also give you real-time quotes and same-day delivery options for your surveys. It’s truly never been this easy, so drop us a message to learn more today.