AYTM philosophy or why we implemented certain limitations on the platform:
As a general principle one of our top priorities is to provide the best experience for all involved parties. We believe that by keeping everyone happy we'll facilitate better research speed, quality and overall satisfaction.
When we launched the company we saw that people suffer on both sides of the process: survey takers are being taken advantage of, asked to spend hours of their time struggling through pages and pages of boring research questionnaires often to find out 50 minutes later that survey was closed. This practice aggravates the survey takers and they start playing the system, finding shortcuts and lying their way in or out of the survey to get the incentive with minimal effort. As a result, researchers end up with garbage answers on very comprehensive and well-thought-out surveys. Business people base their decisions on such data and lead their companies into crisis. The fallout from the steep prices for data collection and market research consulting so high that only the wealthiest companies were able to afford it.
To make the service affordable for startups, small and medium size companies, students, entrepreneurs and agencies we had to lighten the process and create our own panels. We made some promises to the people we asked to participate in answering surveys and we stick to those promises even if it means asking our clients to find more creative & compact ways of designing their surveys. Our survey takers enjoy the best interface and survey taking experience available in the industry. Our short format and length of the interview (2-7 minutes on average) is very appealing, including rich media illustrations and interactive elements. We put ourselves into their shoes and take surveys ourselves all the time to make sure we stay true to our promise.
As a result – surveys are getting completed in a matter of hours, sometimes minutes, survey takers stay happy and engaged, and their attention stays focused on the researched topic, not the incentive, confusing UI or finding the quickest way out. Our clients get valid actionable data and can confidently act upon it while paying a fraction of the industry cost. Now anyone can take advantage of the public insights.
AYTM is a DIY platform which means that we have to rely on our clients to adopt our philosophy. The only way for us to ensure that they do is to put hard limits into the system.
25 questions max
We started with five questions 3 years ago. We gradually and reluctantly increased the max number over the years to find the compromise number. A 25 question long study gives a huge amount of data to analyze while taking only about 7-15 minutes to complete. Research shows that if you interview a person on a topic they are not very interested in, on average they are able to give you 100% concentration for the first 5-7 minutes. At the 10 to 15 minute mark they are still with you but their attention is starting to wander, and after 15-25 minutes most of the people are becoming distracted and losing their concentration. In a focus group it's up to the moderators to get their attention back, shake them up a bit with jokes and a quick break and keep them engaged. With online surveys we don't have that luxury and need to stay within the proven time frame to get the most meaningful insights for our clients.
7 answers max
Have you ever seen a survey where at some point you are presented with a page with hundreds of radio buttons or check boxes, and dozens of columns and rows (sometimes even with horizontal scrolling). It asks you to go through every line and select a level of satisfaction or indicate ownership for a group of items. Such question types are called "matrix". The problem with matrix question types is that they are not very user-friendly. After the 5th to 7th line you often lose the header and get lost in the meaning of columns. It's also hard to concentrate on more than 7 entities at once for most people. So while the numbers may look innocent, in the end most of it could be pure rubbish when desperate survey takers start blindly clicking any options just to get through this nightmare. By limiting every question to 7 answer choices we balance on the edge of the attention span of the average consumer. You probably heard that 7 is about the maximum number of people you can effectively interact with in a meeting. When this number gets higher it becomes harder to keep everyone on your radar and participants tend to become less engaged in the conversation. A similar thing happens with more than 7 answer choices.
Researchers often don't put themselves in the survey takers' shoes. In fact, they rarely complete their own surveys or even ask their spouses to go through this struggle. They are more concerned about including all the questions that may be interesting to analyze. Most of them are pretty conservative and can't adopt the twitter-style of Internet social communication. They don't realize that people don't like reading long amounts of text on the screen and prefer to exchange quick blurbs on social media networks, blog comments, SMS etc.
The proliferation of mobile devices only makes it more critical to be compact. With smaller screens and shorter time slots to fill, information has to be cut into smaller chunks in order to fit the lifestyle and devices of average consumers.
The current style of online and mobile communication dictates the survey taking format and logically leads to the character restrictions that we implemented on AYTM. As a result, shorter questions and answers look nicer in the survey widgets (answers don't usually bleed to the second and third lines, questions don't take too long to read and understand). If you ever have written a boilerplate for a press release or an ad, you know that by spending some extra time you can communicate your point in fewer words and make it easier to read and capture core points. Same here – you usually have only one shot at getting respondents' attention and by making texts shorter you are increasing the chances of clear and effective communication.
So we implemented below-Twitter-limits of 120 characters max for the questions and 90 for the answers. Yes, it may be challenging at times, but it is well worth the effort. It focuses you as a researcher and helps you to avoid making one of the most common mistakes – asking for more than anyone would be willing to read and provide feedback on.
No more than 30% of Open Ends
Coming up with a custom text reply probably requires survey takers the highest level of engagement – they need to think and write a meaningful answer. They also have this notion of a DIALOG. It's your topic, not theirs, and when you invite them to share their feedback, they expect some input from you before giving their opinion. We found out that the more open-ended answers you include, the fewer quality answers you'll get. Based on thousands of surveys processed, we determined that if more than 1/3 of the answers requested are open-ended, your actionable answers returned diminish exponentially. Thus we implemented the limitation.
No more than 50% of instructional questions
Instructional questions don't ask anything, they simply give respondents context, prepare them for the questions, or explain the task. The 50% limit makes it possible to have a dedicated instruction for every single question in your survey which should be more than enough in most cases. By adding more text than necessary you risk losing respondents' attention and make them skip further instructions altogether.
No more than 7 video clips in survey
Well, this is a simple one. A video clip that you append to the survey can be up to 4 minutes long. While it's usually fun to watch a video and easier than writing an open-ended answer, it may become quite time consuming (7*4 = 28 minutes). It provides researchers with a great way to compare a few commercials, communicate any point, or provide background for their study. You don't want to show an entire movie, just enough to collect their feedback.