Most researchers never get a clear look at what's actually happening inside their own studies. The data comes back, the charts get built, the decisions get made—but where the problems entered, how they were caught, and what the survey design did to the results usually stays invisible. That blind spot is exactly what Jonathan Goodbread, Head of Data Quality Strategy at aytm, set out to close at Quirk's Virtual.
In his session, “What your data quality report reveals: A framework for smarter survey research,” Jonathan walked through a way to think about data quality across the entire arc of a study—not as a single checkpoint at the end, but as something that takes shape from the moment a respondent enters your survey.
The core idea: quality is visible if you know where to look
Data quality issues are pervasive across market research, but they rarely announce themselves. The difference between a study you can defend and one you can't often comes down to transparency: being able to see where quality breaks down, how it gets resolved, and what that means for the decisions riding on the data.
Jonathan organized the framework around four P's—Prevent, Protect, Purify, and Prove—a way of evaluating data quality across the full arc of a study so you can see which stage is leaking before a stakeholder meeting, not after.
The stakes: what 4 billion surveys reveal
Jonathan opened with a research-on-research scan of every survey attempt across the panel ecosystem from January through August 2025 (Greenbook + Rep Data, State of Survey Fraud 2025). Across roughly 4.1 billion survey attempts, about 33% were flagged as fraudulent and about 27% as inattentive. On B2C traffic, roughly half was contaminated, net of overlap. The number that reframes everything: about 70% of fraud slips through standard data cleaning—so the “cleaned N” most teams report isn't what they think it is.
His core argument: data quality isn't a filter you apply at the end. It's a discipline you build in from the start. Filter-first accepts contamination and optimizes for removal—and misses most of it. Discipline-first builds the conditions where good data is the norm. Same cost to run, very different business outcomes.
The 4 P's
Prevent—start with a survey that doesn't hurt your data. A long survey on a phone is a fatigue factory, and a meaningful share of inattentive responses is manufactured by survey design itself. Grading a survey before launch flags fatigue risk and structural noise so you're not designing problems into the data.
Protect—your sample is the foundation, so make sure it's real. With roughly a third of survey attempts fraudulent, blocking bad traffic at the source (rather than relying on opt-in or single-signal checks) keeps a study from paying for fraud twice: once to field it, again to clean it out.
Purify—catch what gets through. Three categories matter in 2026: AI-generated responses (especially in open-ends), response farms (coordinated real humans mimicking authentic respondents), and inattentive respondents (real, targeted people who are distracted). In-field scoring across behavioral signals pulls contamination before it lands in the dataset—so open-ends are trustworthy and there's no mid-analysis surprise.
Prove—turn the claim into a measurement. Every study comes back with a data quality report and a comparable score. Without a record, you can't defend your numbers when challenged; without comparability, you can't tell whether your program is getting better or just bigger.
The bigger picture
Jonathan closed on where this is heading. AI is raising the stakes on both sides of the survey. Transparency is becoming a buying criterion—the vendors who can show their work will win. And the credibility gap is widening: the insights teams that can defend their numbers become the indispensable ones.
What you can do about it
The session's closing advice was refreshingly concrete: ask your suppliers tougher questions and refuse opaque numbers, treat survey design as a quality investment rather than a creative exercise, read the quality report before the results, and push your program toward discipline-first instead of filter-only.
Press play on the full session below.
Want to talk through what data quality transparency could look like for your own research program? Request a demo.
.webp)
.jpeg)
