Imagine you’re the product manager for a line of athleisure wear. You’re hoping to launch your newest product line at a major department store to round out the brand’s total offering, increase sales, and grow your brand’s floor and shelf presence. Unfortunately, the sportswear department’s buyer isn’t convinced this new line is needed, fearing it will cannibalize sales from your brand’s current assortment. To support your theory that the new line will yield incremental sales, you suggest conducting an in-store test by launching the new product line at a handful of stores representative of the total market and comparing sales and basket data pre- and post- launch. The sportswear buyer agrees to a 5 week in-store test, and you’re eager for the opportunity to win the additional business. Now, it’s time to start carefully planning for this in-field experimental research and learn if your hypothesis (the new line will create incremental sales) can be inferred based on the test market findings.

What is Causal Research?

Research designs can be exploratory (when the objective is to discover new ideas and insights), descriptive (when the objective is to describe market characteristics), or causal. Causal research seeks to determine cause-and-effect relationships. Through an experimental research design, one or more independent variables is manipulated and the effect on the dependent variable(s) is measured while simultaneously controlling any extraneous variables. You have two goals when conducting experimental research: draw valid conclusions about the effects of the independent variables on the dependent variables (suggesting internal validity) and make valid generalizations to a larger population (suggesting external validity).As the product manager in the athleisure example, during your experiment you’ll measure any change in sales (dependent variable) based on the introduction of the new product line (independent variable). For this to be a controlled experimental design, you will need to control/account for other influential variables that could weaken or invalidate the results – such as your brand’s advertising, competitive advertising, discounts, store size/location/design, and flow of in-store traffic.

Four Methods for Controlling Extraneous Variables

Extraneous variables pose an enormous threat to the validity of your experimental research findings, so it’s critical that you understand how to control them.You’re probably already familiar with the use of randomization in survey design (randomized answer choices, rotating concepts so they’re shown to respondents in a random order, etc.). Randomization is also a method for controlling extraneous variables in experiments. It involves randomly assigning test units (such as stores or consumers) and independent variables to different experimental groups. This helps ensure extraneous variables are represented equally across groups.Another method involves matching or comparing test units on a set of key background variables before assigning them to an experimental group. For example, department stores can be matched up based on annual sales, size, and location. Then, stores from the matched groups of stores can be selected and assigned to an experimental group.You can also use statistical control by measuring extraneous variables and adjusting for their effects during analysis using statistical techniques.Finally, you could use experiments specifically designed to control for extraneous variables, also known as design control. For instance, with the athleisure wear in-store test you can control distribution volume, store placement, pricing, promotion, and stock for a high degree of internal validity.

The Difference Between Lab and Field Experiments

There are both advantages and disadvantages for using either lab or field experiments, so you’ll need to weigh the tradeoffs and determine which is best for fulfilling your research objectives, or if you want to employ both in a complimentary design. Conducting experiments in laboratory or artificial settings offers a great deal of control over extraneous variables due to the carefully monitored environment, and as a result, increases internal validity; however, the findings may not be generalizable to the real world. Laboratory experiments tend to use a smaller number of test units, last for a shorter period, and be more geographically restricted making them less expensive and easier to conduct than field experiments. However, due to the artificial environment, respondents may react to the staged situation rather than the independent variable(s). They may also try to guess the purpose of the experiment and respond accordingly rather than naturally. Field experiments, on the other hand, occur in actual market conditions. Internal validity is lower in field experiments, but the results are more generalizable across the larger market. One example is test marketing.

Test Marketing Basics

Test marketing replicates a planned national marketing program completed on a small scale in a carefully selected limited number of test markets. Some common U.S. cities used for standard market testing include Charlotte, Indianapolis, Kansas City, Nashville, Rochester, and Sacramento because each is considered representative of a larger segment of the United States. Once the test markets are chosen, product is sold through normal distribution channels with no special considerations given simply because the products are being used for test marketing. One or more marketing mix variables (product, price, distribution, and promotion; i.e. the independent variables) is varied, and sales (dependent variable) are monitored. Other kinds of test marketing include controlled test marketing in which the entire program is conducted by an outside research company – guaranteeing distribution in a predetermined percentage of the market – and simulated test marketing in which consumers are intercepted in high-traffic locations like malls, prescreened for product usage, given the opportunity to purchase a product in either a real or lab setting, and interviewed post-product usage. One advantage of simulated test marketing is that it’s much less expensive than standard test marketing.

The Takeaway

Experimentation in market research is becoming increasingly popular across companies in a variety of industries. Consider using a causal research design when you want to assess the effects of changing one or more of the marketing mix variables. When designing your research, it is critical to control extraneous variables that may invalidate your research findings. You also need to decide if you want to do an artificial lab experiment that will likely result in higher internal validity or an in-field experiment in which the results are likely more generalizable to the larger market. As there are advantages and disadvantages associated with each, you want to be sure your experiment is designed to achieve your research objectives and answer your business questions.