Appendix B

1. Appendix B Conducting Experiments

2. Experimentation and Causality • Survey Research • Inferences are based on what respondents say about the topic • Experimental Research • Inferences are based on what subjects do regarding the treatment • Experiments manipulate conditions and measure the effects on behavior

3. Types of Experiments • Laboratory Experiments • Behavioral science laboratories • Test kitchens and tasting rooms • Special experimental facilities • Field Experiments • Advertising and marketing situations • Human resources and work settings • Public facilities and services

4. The Con’s Small number of subjects demands precise sampling Artificiality threatens external validity Special facilities or equipment may be needed Subjects must travel to the laboratory or facility Disruption of subjects’ daily lives may create problems Knowing they are being tested may sensitize them The Pro’s Small number of subjects permits precise sampling Laboratories provides great control Subjects can be randomly assigned Instructing subjects is more thorough “Take” measures can be obtained Conditions permit more elaborate designs Pro’s and Con’s of Laboratory Experiments

5. Experimental Terminology • Experiment • A procedure for measuring the effect of some treatment or condition on subjects • Treatment or Treatment Condition • The experimental manipulation of some factor or condition to determine its effect on subjects • Control Group or Condition • Subjects manipulated identically to treatment groups except for the experimental treatment • Confound • Anything other than the experimental manipulation that might cause a difference between the treatment and control group measurements

6. Types of Experimental Designs • Pseudoexperiments • Designs too flawed to be regarded as legitimate • Genuine Experiments • Designs that control threats to validity • Extended Genuine Experiments • Designs that include multiple treatment groups • Nonequivalent Groups Experiments • Designs lacking random assignment to groups • Times Series Experiments • Designs that compare before and after effects

7. Threats to Internal Validity • External Events Over Time • Internal Processes Over Time • Premeasurement Sensitization • Measurement Instability • Systematic Selection • Experimental Attrition • Regression to the Mean

8. Postmeasure Only, Experimental Group Only Time Line Treatment Postmeasure A2 Pseudoexperimental Designs Treatment Group Group A

9. Pre- and Postmeasurement, Experimental Group Only Time Line Premeasure A1 Treatment Postmeasure A2 Pseudoexperimental Designs Treatment Group Group A

10. Pre- and Postmeasure, Experimental and Control Group Group A Group B Random Random Premeasure A1 Premeasure B1 Time Line Treatment Control Postmeasure A2 Postmeasure B2 Genuine Experimental Designs Treatment Group Control Group

11. Postmeasure Only, Experimental and Control Group Group A Group B Random Random Time Line Treatment Control Postmeasure A2 Postmeasure B2 Genuine Experimental Designs Treatment Group Control Group

12. Four-Group, Six-Measure Design Pre- and Postmeasure Postmeasure Only Group A Group B Group C Group D Random Random Random Random Premeasure B1 No Measure Premeasure A1 No Measure Time Line Treatment Control Treatment Control Postmeasure A2 Postmeasure B2 Postmeasure C2 Postmeasure D2 Genuine Experimental Designs Treatment Group Control Group Treatment Group Control Group

13. Pre- and Postmeasure, Extended Design Group A Group B Group C Group D Treatment Group Treatment Group Treatment Group Treatment Group Random Random Random Random Premeasure B1 Premeasure D1 Premeasure A1 Premeasure C1 Time Line Treatment A Treatment B Treatment C Treatment D Postmeasure A2 Postmeasure B2 Postmeasure C2 Postmeasure D2 Extended Genuine Experimental Designs

14. Postmeasure Only, Extended Design Group A Group B Group C Group D Treatment Group Treatment Group Treatment Group Treatment Group Random Random Random Random Time Line Treatment A Treatment B Treatment C Treatment D Postmeasure A2 Postmeasure B2 Postmeasure C2 Postmeasure D2 Extended Genuine Experimental Designs

15. Pre- and Postmeasure, Matched Groups Group A Group B Matched Matched Time Line Premeasure A1 Premeasure B1 Treatment Control Postmeasure A2 Postmeasure B2 Nonequivalent Groups Experimental Designs Treatment Group Control Group

16. Postmeasure Only, Matched Groups Group A Group B Matched Matched Time Line Treatment Control Postmeasure A2 Postmeasure B2 Nonequivalent Groups Experimental Designs Treatment Group Control Group

17. Single Group, Pre- and Postmeasurement Series Measurement A1 Measurement A2 Premeasure Series Measurement A3 Measurement A4 Time Line Treatment A Measurement A5 Measurement A6 Postmeasure Series Measurement A7 Measurement A8 Time Series Experimental Designs Group A

18. Two Groups, Pre- and Postmeasurement Series Measurement B1 Measurement A1 Measurement A2 Measurement B2 Premeasure Series Measurement A3 Measurement B3 Measurement A4 Measurement B4 Time Line Treatment B Treatment A Measurement B5 Measurement A5 Measurement B6 Measurement A6 Postmeasure Series Measurement B7 Measurement A7 Measurement B8 Measurement A8 Time Series Experimental Designs Group B Group A

19. Threats to External Validity • External Validity • The degree to which results can be generalized to the population • Artificiality • The manner in which the experimental setting differs from the subjects’ daily environment • Reactivity • The tendency for the experimental situation to effect subjects’ responses to treatments

20. Blind Testing in Experiments • Blind Testing • The procedure does not reveal to the subjects what is being measured or tested • Double-Blind Testing • Neither the subjects nor those running the experiment are told what is being measured • The Ultimate Blind Test • A field experiment where subjects are not aware they are participants in an experiment • Test marketing of new products • “Split run” direct response advertising • Selective modification of public facilities

21. Experimentation/Survey Commonalities • Project Initiation • Definition of information needs • Project planning procedures • Sampling design • Measurement Instrumentation • Questionnaires, instructions, and scales • Observation and recording • Analysis and Reporting • Data editing and processing • Statistical analysis and interpretation • Report composition and presentations

22. End of Appendix B