Are DIY Platforms Ready for Prime Time?

Article by JD Deitch, PhD
In 2013 I attended the CASRO Online Research Conference (now CASRO Digital) in San Francisco. One of the sessions that caught my eye was a roundtable discussion on do-it-yourself (DIY) survey tools. The program notes set the stage as follows: “Much has been written about the recent proliferation of companies offering DIY/Instant Research methodologies, which are typically inexpensive. Are these methodologies able to produce representative results? What conditions are necessary to use such offerings effectively?

Representing the traditional industry on the panel were two respected and well-known researchers. The two biggest DIY firms (then and now) were at the table represented by senior officers. The typical firmographic composition of a CASRO event meant that the DIYers would be heavily outnumbered and ensured skepticism from the off. It was therefore hardly surprising that the discussion never got beyond topline rhetoric. The researchers (on the panel and in the crowd) observed that the existing DIY solutions didn’t offer quality controls or oversight that one got from full service companies, which thus intrinsically and greatly increased the likelihood that people would do bad research. That these tools were being deliberately targeted at nonresearchers was, to some, like giving kids booze and cigarettes; it was morally reprehensible and threatened to bring the whole industry into disrepute. The DIYers rather unapologetically replied that people were already making choices, and that these were neither spendthrifts nor the feeble-minded but rather some of the traditional industry’s best and most astute clients. There were a few questions from the audience that tried to strike more thoughtful chords, but there was little room for common ground.Two very momentous years have passed. What used to be a tiny insurgency has become a viable and consequential sector of the market. There are those who would view commercial success as the surest sign that self-service tools are ready for prime time. Are they though? Are they good enough? The answer is an emphatic yes.

Their tools are complete, integrated, automated, and fast.

While there’s variation in both form and function across the space, DIY solutions are built from the ground-up to be maximally automated and fast. The best are horizontally-integrated and offer an all-in-one package of questionnaire programming, sample supply, data processing, and reporting that just leaves the industry’s traditional solutions in the dust. They are cheaper to run and don’t require days or weeks to produce ugly PowerPoints. They are indisputably better.

Their question formats are better.

There. I said it. With much of the industry desperately late in transitioning to a mobile-first, device agnostic world, the respondent experience for most DIY tools is downright refreshing. Many of these companies started as technology plays and thus understood implicitly that mobile friendliness was nonnegotiable. With no installed base of trackers to worry about or existing factories that needed to be retooled, they faced neither of the major obstacles that traditional firms did/do. Most have spent time on user experience and understand its implications for data collection. Some go so far as to hard-code character and response option limits so as to prevent the sort of questionnaire design problems that even the traditional industry seems unable to regulate with its clients. Even the interfaces for the survey creator are simple and fluid. Are legacy firms making progress? Yes, their tools are getting better, but their execution remains mired in the massive number of projects that are still desktop only.

They are tackling and solving sampling problems.

Yes, you read that correctly. DIY companies are leading the way into programmatic sample, and in doing so taking out needless cost, enabling greater diversity, and still using reputable sources. That the questionnaires are short, well designed, and device agnostic means even broader coverage of audiences that are abandoning the traditional industry’s studies and panels. Some have their own panels, which they’re protecting precisely by not fielding the long, boring, desktop-only surveys that plague the legacy industry.

They are educating…

Whether they’ve come to the conclusion for commercial reasons or out of a belief that they need better quality (probably both), DIY companies are acquiring research competence and putting it to work. They’re hiring people who know research. They’re offering pre-packaged question modules and sample design recommendations to address the areas of greatest risk.

…and are attracting conscientious clients.

Amongst the researchers at the CASRO conference, the prevailing wisdom was that the potential for harm was great, especially for nonresearchers. Implied in this concern was the belief that (a) DIY firms were going to prioritize business growth over quality and (b) their clients were lemmings blithely jumping off the cliff. My conversations with people across the industry—with clients, DIY firms and observers—suggest that this hasn’t happened. To the contrary, it appears that clients are not only conscious of issues, many are asking for help. This is driving DIY firms to adapt and offer consultation.

Is there still a quality risk?

Are there people still doing bad surveys on DIY platforms? For sure, the answer is yes. But two things are abundantly clear. First, either by design or through deliberate action, DIY companies are taking clear steps to ensure quality while retaining their attractive price and speed. Second, they have resolved the quality problems that continue to vex legacy research firms, namely the glacial transition of long, punishing desktop-only surveys that are stressing panels and failing to cover a large swath of very desirable respondents. Put differently, there are plenty of quality risks that remain in working with traditional firms.

All grown up

In my original article on disruption in the insights industry, I stated that the winners of the revolution that continues to sweep through the sector would have a few central characteristics. They would make their clients’ money by getting closer to their execution. They’d build new assets. They would breed quality by design. They would ooze efficiency. Finally, as a result of these factors, they’d be able to create a new model to achieve profitability.As the saying goes, I now have a horse in this race. After many years in the traditional industry, I’ve joined one of these DIY companies because I think this is the future of research. Don’t take my word for it, though. See it with your own eyes. DIY firms are solving problems that research agencies are not, or are slow to solve. They’re starting to change the way people think about the impossible “research triangle” where, given the essential components of quality, speed, and cost, optimization is supposedly only possible across two. They face their own challenges about building and sustaining business, but they’re fundamentally doing what Clayton Christenson said in his seminal article about disruption. Because they’re not locked into legacy business models that militate against big change, they’re building the next generation and, in turn, leapfrogging the incumbents. They—we—are here to stay.Article now also featured in GreenBook!

Featured photo: “Launch Button -- SMASH Rocket Club 5-9-09 4” by Steven Depolo, used under CC BY 2.0