Are You Up to the Challenge?

For 2016 AYTM is laying down a challenge to the research industry. It’s time for everyone to take responsibility for their panels and the research conducted on them. Read more about the gauntlet we’ve thrown down.

Our challenge to the market research industry

Online panel and sample providers took a bit of a beating in 2015. Between the doubts cast by the disastrous polling results in the UK elections and the public session at the MRMW conference where live respondents admitted openly to cruising mindlessly through surveys and even lying deliberately in qualifying questions, the industry’s quality fell into shadow. The year was book ended with an article in the New York Times that described significant concerns surrounding online polling in the US Republican polls.

The implications are clear: there’s a serious problem with polling and online research and it’s no longer an internal discussion, it’s mainstream news.  Concerns about panel quality are hardly new. But they are now boiling over into the public domain.

Research buyers are slow to change. It’s hard work, with increasingly smaller budgets, and they are struggling to discern the signal from the noise among substitute and complementary methodologies. Traditional suppliers, whose fees are a direct function of length of interview and who count on annual revenue streams from inflexible tracking and normed studies, are still trying to stave off financial challenges (rather unsuccessfully) to buy time to retool. Panels are starved of respondents under the age of 35 because none of them wants to take long punishing desktop-only surveys, which as we’ve known for years cause them to disengage and respond with bogus data.

It doesn’t have to be this way

Blame for the industry’s problems can be placed at every point in the chain. But it doesn’t have to be this way.

It is an indisputable truth that survey research can be done far more inexpensively, far more quickly, and far more efficiently with online platforms. These platforms are device agnostic and mobile friendly by design. They are automated and don’t require a slow, costly army of people to run them. AYTM is one of these platforms.

It is also common knowledge that shorter, well written surveys improve data quality and participant engagement. And we understand that people who aren’t subjected to long, poorly written desktop-only surveys are better responders who complete more surveys, more accurately, on average in their “lifetimes”.

The technology exists to tackle all these issues and test and retest people’s answers, using analytics and game mechanics to reward accurate responders and to eliminate those over time who give suspicious answers.

So why aren’t doing it?

Taking responsibility

There’s really no reason for the current situation to persist. It no longer requires management courage to reach the conclusion that the time to change is now. If anything, those who aren’t articulating concrete plans should be challenged by their employers, boards, and shareholders.

Here’s how we see the challenge.

Research buyers need to take responsibility for the studies they insist on running. If you’re still running long desktop-only studies, you are a fundamental cause of this problem. Blaming your suppliers for the quality of their panelists is like blaming the bartender for your hangover.

Sample companies and research agencies need to take responsibility for educating their clients. Discussing solutions to these issues inside the MR conference circuit is all well and good, but if you’re taking the work that’s killing your panels without ensuring you’re educating your clients on best practices, you’re also part of the problem.


There are many things happening in and around the research industry that are out of anyone’s control. The pace of technological change is breath-taking. New sources of data and new methodologies are changing the way we think. It’s a fantastic time to be working in this space.

We can, however, control our own actions. It is time for us all to “walk the walk.” We need to follow the principles we espouse even if they are commercially inconvenient.

  • We will be device agnostic (device aware, in AYTM’s case) and optimized for mobile by design.
  • We won’t subject research participants, ours or anyone else’s, to long crappy desktop-only surveys.
  • We will talk to clients and research users in plain language about what surveys are good at and what they aren’t and how that impacts their studies.

This is the gauntlet we’re throwing down to the industry. AYTM is, as of this writing, the only company I know of that is able and willing to make this commitment.