Primary Data Collection: Experimentation and Test Markets

1. Chapter Eight Primary Data Collection: Experimentation and Test Markets

2. Learning Objectives Chapter Eight Primary Data Collection: Experimentation Understand the nature of experiments; Gain insights into the requirements for proving causation; Learn about the experimental setting; Examine experimental validity; Learn the limitations of experimentation in marketing research; Compare types of experimental designs; Gain insight into test marketing. Chapter 8

3. What is an Experiment? An Experiment: • A research approach in which one variable is manipulated and the effect on another variable is observed. Key Variables: • Independent: variables one controls directly such as price, packaging, distribution, product features, etc.; • Dependent: variables one does not directly control such as sales or customer satisfaction - (might control them by manipulating the independent variable); • Treatment: the independent variable manipulated during and experiment to measure its effect on the dependent variable; • Extraneous: Factors one does not control but has to live with such as the weather. Chapter 8

4. Demonstrating Causation Causal Research: Research designed to determine whether a change in one variable likely caused an observed change in another. A causal relationships must demonstrate three things: • Concomitant Variation; • Appropriate Time Order of Occurrence; • Elimination of Other Possible Causal Factors. Chapter 8

5. Demonstrating Causation Concomitant Variation: A statistical relationship between variables; Appropriate Time Order of Occurrence: Change in an independent variable occurred before an observed change in the dependent variable; Elimination of Other Possible Causal Factors: “If you eliminate the impossible, whatever remains, however improbable, must be the truth.” Sherlock Holmes Chapter 8

6. Experimental Setting Laboratory: Experiments conducted in a controlled setting. Field: Tests conducted outside the laboratory in an actual environment, such as a marketplace. Chapter 8

7. Demonstrating Validity Internal Validity: The extent to which competing explanations for the experimental results observed can be ruled-out. External Validity: The extent to which causal relationships measured in an experiment can be generalized to outside persons, settings, and times. Chapter 8

8. Demonstrating Notation “X” = Independent Variable: • Indicates the exposure of an individual or a group to an experimental treatment. This variable is something the researcher can change and manipulate. It is hoped that the change in the independent variable will cause a change in the dependent variable. “O” = Dependent Variable: • Indicates a variable the researcher cannot change directly. It is hoped that changing the independent variable will cause changes in the dependent variable. Thus the dependent variable is “dependent” on what the researcher does with the independent variable. Chapter 8

9. Extraneous Variables History: • Intervention, between the beginning and end of an experiment, of outside variables that might change the dependent variable. Maturation: • Changes in subjects occurring during the experiment that are not related to the experiment but which might affect subjects’ response to the treatment factor. Instrument Variation: • Changes in measurement instruments (e.g., interviews or observers) that might affect measurements. Selections Bias: • Systematic differences between the test group and the control group due to a biased selection process. Chapter 8

10. Extraneous Variables Mortality: • Loss of test units or subjects during the course of an experiment - which might result in a nonrepresentativeness. Testing Effect: • An effect that is a by-product of the research process itself. Regression to the Mean: • Tendency of subjects with extreme behavior to move towards the average for that behavior during the course of the experiment. Chapter 8

11. Controlling Extraneous Variables Randomization: The random assignment of subjects to treatment conditions to ensure equal representation of subject characteristics. Holding constant the value or level of extraneous variables throughout the course of an experiment. Use of experimental design to control extraneous causal factors. Adjusting for the effects of extraneous variables by statistically adjusting the value or the dependant variable for each treatment condition. Physical Control: Design Control: Statistical Control: Chapter 8

12. Experimental Design, Treatment, and Effects Experimental Design: A test in which the researcher has control over and manipulates one or more independent variables. Treatment Variable: The independent variable that is manipulated in an experiment. Experimental Effect: The effect of the treatment variable on the dependent variable. Chapter 8

13. Experimental Design, Treatment, and Effects High Cost: • Is the research affordable? • Will the research be beneficial & help solve problems? • Has a cost & benefit analysis been done? Security Issues: • Particularly critical with field experiments • The competition might be “tipped-off” • Are the data and findings secure? Process Contamination: • People who unwittingly get caught into the survey. • Outside factors unnaturally affecting the experiment. • Participants who intentionally try to skew the results. Chapter 8

14. Experimental Design Examples Pre-Experimental Design: • Designs that offer little or no control over extraneous factors. Three Key Design Types: One-Shot Case Study One-Group Pretest-Posttest Static-Group Comparison Chapter 8

15. Experimental Design Examples O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable Given: Chapter 8

16. Experimental Design Examples O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable Given: Chapter 8

17. Experimental Design Examples O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable Given: Chapter 8

18. True Experimental Design True Experimental Design: • Research using an experimental group and a control group, to which test units are randomly assigned. Three Key Design Types: Before and After With Control Group Solomon Four Group Design After Only With Control Group Chapter 8

19. Experimental Design Examples O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable Given: Chapter 8

20. Experimental Design Examples O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable Given: Chapter 8

21. Experimental Design Examples O = The Measurement of the Dependent Variable X = The Manipulation/Change of Independent Variable E = Experimental Effect - Change in Dependent Variable due to Change in the Independent Variable Given: Chapter 8

22. Quasi Experiments Quasi-Experiments: Studies in which the researcher lacks complete control over the scheduling of treatments or must assign respondents to treatments in a nonrandom manner. Interrupted Time-Series & Multiple Time-Series Chapter 8

23. Quasi Experiments Interrupted Time-Series: Research in which the repeated measurement of an effect “interrupts” previous data patterns. Multiple Time-Series: Interrupted time-series design with a control group. Chapter 8

24. Cities as Test Markets Test Markets Test Market: Testing of new product/service, or some element of the marketing mix, using an experimental or quasi experimental design. • Advertising expenses; • Point-of-purchase materials; • Coupons and sampling; • Travel and set-up expenses; • Need for customized research; • Possible diversion of sales from your other products; • Potentially bad press / public reaction if experiment fails; • Letting competitors know what your company is doing; • Falsely thinking the sample results are always representative of the population. Cost Issue: Chapter 8

25. Test Market Steps 1. Define the Objective: • What do you hope to learn? • What are the characteristics of the people/products of interest? 2. Select a Basic Approach: • Simulated, controlled, or standard test? 3. Develop Detailed Test Procedures: • How will you execute the study? • Who will be involved? • How long will it take and how much can you spend? Chapter 8

26. Test Market Steps 4. Select the Test Market: • Market should not be over tested; • Should have little media spillover; • Demographics should be similar to your target population; • Market should be large enough to provide useful results; • Distribution and other patterns should be similar to the nation. 5. Execute The Plan: • How long should the test run? • Who should execute it? 6. Analyze the results: Use qualitative and quantitative techniques when possible. Chapter 8