Panels, Sampling and Survey Design: An Introductory Guide

Surveys are vital research tools, and the insights gathered from market research surveys drive business decisions that can have far-reaching impacts on a company’s future. When so much is riding on your data, getting high-quality survey results will be the difference between effective, accurate insights and untrustworthy noise.

The components required for high-quality survey results are – sound methodology, effective survey, expert analysis, and quality sample. For most projects, the most effective way to reach quality sample is with panels.

Panels 101

Panel companies collect and manage groups of people who are interested in taking surveys and participating in studies. AYTM owns and operates our own proprietary global consumer panel.

These companies track the demographic data of their respondents, so they are able to work with you to define the best audience for your survey. Then, they are able to serve it directly to that audience. At AYTM in addition to ten demographic traits, we track and maintain over 2,000 psychographic data points on our respondents. Allowing you precisely target your audience.

Individuals that participate in surveys for panels are usually rewarded for their time, either with cash incentives or other rewards.

The costs for utilizing panels vary depending on the specificity of your audience and the length of the survey.

Why you should use panels

While there are many benefits to using online panels, here are

  • Guaranteed responses to your survey. Panel companies cultivate and manage large databases of people who are willing and motivated to answer your surveys. This is particularly helpful when you have a very specific audience in mind, like people who own a dog or people who use Lyft. Panel companies allow you to pinpoint your audience and can guarantee respondents.  
  • Access to specific populations. However niche your target demographic, there is a panel company who has access to that group. Panel companies can provide you with an amazing level of access to a population that is pre-profiled to be relevant to your survey.
  • Fast turnaround time. Because panel companies have a database of confirmed survey takers, they are able to field respondents much more quickly than any other distribution method.
  • There is a lot of variation in panel companies. Many try to create a representative audience, which means their respondents will reflect the demographics of the larger population.

Do your research on the panel companies you are considering, and don’t be shy in reaching out to a contact at the panel company. They will often help you to identify your target market and ensure that they will be able to fulfill your needs.


How to Determine your Sample Size

Once you determine your target market and decide on a sampling source you have to determine how many respondents you need. Many factors can determine how many respondents are appropriate for your survey: budget, analysis and data objectives to name a few.

Fortunately, there is a simple calculation for determining sample size.

  • Population. What is the size of the entire population that you want to survey?
  • Margin of error. What margin of error are you comfortable with, for most survey projects, a margin of error of 5% is acceptable.
  • Confidence. How sure are you that the sample of the population that responds to the survey reflects the larger population? The most commonly accepted confidence level is 95%. 

If you are unsure of the size of the total population, a good sample to work with is 400 respondents – it’s the industry standard for statistical significance. You can also use this handy calculator.

For a full guide to determining the best sample size for your survey project, see our [insert sample size article.]

Survey Design When You’re Working With Panels

In addition to our best practices for Selecting the Best Research Design, there are a few additional guidelines we recommend for survey authors intending to use online panels.

The first is to set up proper screening at the start of your survey to make sure that the people who are taking your survey are the people you are trying to reach. This will save you money, as you’ll only be charged for completed surveys, and will result in clearer, more relevant data.

If you need demographic data about your respondents, check with the panel company to see what’s provided. While many companies will offer some data you might need additional data points they don’t include. Any data points not provided by the panel company will need to be added to your survey.

Lastly, every year we notice that more and more panelists choose to take surveys on mobile devices. It’s important to ensure your survey is optimized for all browsers and devices. Make sure to test your survey on a mobile device prior to launching.

The Takeaways

While you have many things to consider for your next survey project quality sample will be an important factor. With many sampling options to choose from, online panels can offer specific target markets, guaranteed responses in the blink of an eye.