With ever-evolving research solutions most surveys are fielded online nowadays, which, although expedient, generates risks associated with successfully evaluating the quality of respondent’s answers. However, there are numerous, useful, indicators that will help assess whether or not respondents are providing candid responses. Collecting and analyzing bad data can be worse than having no data. With no data, one can rely on experiences, but interpreting bad data will likely spoil important business decisions.

Interpreting the Signs

For each of the following scenarios below, it is advisable to flag all potential violations. Typically, a single flag, or even two, wouldn’t be grounds for discharge. Some participants can read through a survey more quickly than others, a straight line through a matrix grid can often be legitimate, and it is not improbable for a respondent to misinterpret the phrasing of a survey questions. However, a good rule of thumb to abide by is if there are three or more violations or failed quality checks, the respondent should be removed from the data set and replaced with a new one.

Daily Data Assessments

This is something that should become a daily ritual while a survey is live in the field. Data sets must be reviewed as often as possible to confirm that your respondents are providing valuable and insightful information that aligns with your research objectives.

Inspect Multiple Select Questions & Answers

It is vital to flag a respondent as suspicious if the participant selects all answer options, especially during screening questions, or selects exactly one or two answer options in all multiple select questions throughout the survey.

Identify Straight-lining or Patterning

Straight-lining or patterning is a case in which respondents are rushing through a survey, clicking on the same response every time. It’s especially important to look out for these violations in survey questions that are formatted as a grid.

Recognize Demographic Inconsistencies

Identifying numeric inconsistencies, especially with respect to age, can be a valuable strategy for flagging potential violators. For example, if a respondent states that they’ve had a job for 25 years but was born in 1991 there is obvious room for a research to flag with concern.

Examine Open-End Responses

Including free form questions in a survey will allow you to scan the open-end responses and other, “please specify” responses for gibberish or excessively vague answers.

Flag the Speeders

It is prudent to test your survey and determine the length of time it takes to complete the survey entirely. If a respondent’s completion time is grossly under the actual survey time, then it would be a justified flag.

Include Quality-Check Questions

Quality check questions will ask the respondent to select a specific item. If the respondent fails any of the quality check questions the data should be flagged.

Review of data by research analysts

Cleaning up open-ended responses is ideal, but to truly guarantee the best quality data, it is worthwhile to call in the professionals. Reviewing data must go beyond simple grammar or typos checks. When a research analyst reviews data, they can determine if the data is providing enough information for proper analysis and conclusions. Every survey is different, and the cut-off decisions vary. With simple quality-check processes in place, this bad data should never happen.

The Takeaway

Outside of refining projects that are already fielded, monitoring data will improve your market research skills because you are gaining insight from the ground level. Studying and learning how participants respond to questions will provide a better understanding on how minor adjustments to question phrasing will impact the collected data. Reviewing data will also aid in avoiding respondent fatigue and frustration. Overall, conducting consistent data reviews is an important process from multiple perspectives as it is the core foundation for your research project.