Chapter 9

1. Chapter 9 Experiments 課堂教授:洪新原老師 組員:602530020邱炳諵602556026張易揚602556026蔡明學 報告日期:12/2

2. Learning Objectives • Uses for experimentation. • Advantages and disadvantages of the experimental method. • Seven steps of a well-planned experiment. • Internal and external validity with experimental research designs. • Three types of experimental designs and the variations of each.

3. What Is Experimentation? • An experiment is a study involving intervention by the researcher beyond that required for measurement. • The usual intervention is to manipulate some variable in a setting and observe how it affects the participants or subjects being studied.

4. The researcher manipulates the independent or explanatory variable and then observes whether the hypothesized dependent variable is affected by the intervention. • There is at least one independent variable (IV) and one dependent variable (DV) in a causal relationship. We hypothesize that in some way the IV causes the DV to occur.

5. There are three types of evidence necessary to support causality. Agreement between IVs and DVs Time order of occurrence Extraneous variables did not influence DVs

6. First, there must be an agreement between independent and dependent variables.The presence or absence of one is associated with the presence or absence of the other.

7. Second, beyond the correlation of independent and dependent variables, the time order of the occurrence of the variablesmust be considered. The effect on the dependent variable should not precede the manipulation of the independent variable. The effect and manipulation may occur simultaneously or the manipulation may occur before the effect.

8. The third source of support comes when researchers are confident that other extraneous variables did not influence the dependent variable. • To ensure that these other variables are not the source of influence, researchers control their ability to con-found the planned comparison. • Under laboratory conditions, standardized conditions for control can be arranged.

9. An Evaluation of Experiments

10. Conducting an Experiment Exhibit 9-1 depicts the use of experiments in the research process.

11. We discuss seven activities the researcher must accomplish to make the endeavor successful:1. Select relevant variables.2. Specify the treatment levels3. Control the experimental environment4. Choose the experimental design.5. Select and assign the subjects.6.Pilot test, revise, and test.7. Analyze the data.

12. Select relevant variables • Hypothesis is a relational statement because it describes a relationship between two or more variables. It is must also be operationalized.Does a sales presentation that describes product benefits in the introductionof the message lead to improved retention of product knowledge. • Since a hypothesis is a tentative statement a speculation about the outcome of the study, it might take this form:Sales presentation in which the benefits module is placed in the introduction of a 12 minute message produce better retention of product knowledge than those where the benefits module is placed in the conclusion.

13. The researcher is challenged to 1. Select variables that are the best operational definitions of the original concepts.2. Determine how many variables to test.3. Select or design appropriate measures for the chosen variables. The selection of measures for testing requires a thorough review of the available literature and instruments.

14. Specifying Treatment Levels • In an experiment, participants experience a manipulation of the independent variable, called the experimental treatment.The treatment levels of the independent variable are arbitrary or natural groups the researcher makes within the independent variable of an experiment.

16. A control group can provide a base level for comparison. The control group is composed of subjects who are not exposed to the independent variables, in contrast to those who receive the experimental treatment.

17. Controlling the Experimental Environment • Environmental control means holding the physical environment of the experiment constant. The introduction of the experiment to the subjects and instructions would likely be videotaped for consistency.When participants do not know if they are receiving the experimental treatment, they are said to be blind. When neither the participant nor the researcher knows, the experiment is said to be double-blind. Both approaches control unwanted complications.

18. Choosing the Experimental Design • Experimental Design serve a positional and statistical plans to designate relationships between experimental treatments and the experimenter’s observations or measurement points in the temporal scheme of the study. In the conduct of the experiment, the researchers apply their knowledge to select one design that is best suited to the goals of the research.

19. Selecting and Assigning Participants Matching Random assignment

20. Random assignment • Random assignment uses a randomized sample frame for assigning participants to experimental and control groups.

21. Matching • Matching is an equalizing process for assigning participants to experimental and control groups. • Matching employs a nonprobability quota sampling approach. • The object of matching is to have each experimental and control participant matched on every characteristic used in the research.

22. Quota Matrix Example

23. Pilot Testing, Revising, and Testing • Pilot Testing is intended to reveal errors in the design and improper control of extraneous or environmental conditions. • The experiment should be timed so that subjects are not sensitized to the IV by factors in the environment.

24. Analyzing the Data • If adequate planning and pretesting have occurred , the experimental data will take an order and structure uncommon to surveys and unstructured observational studies. • It is not that data from experiments are easy to analyze ; they are simply more conveniently arranged because of the levels of the treatment condition, pretests and posttests, and the group structure.

25. Measurement Options

26. Validity in Experimentation Internal External • Internal validity exists when the conclusions drawn about a demonstrated experimental relationship truly implies cause. • External validity exists when an observed causal relationship can be generalized across persons, settings, and times.

27. Maturation History Testing Experimental mortality Statistical regression Instrumentation Selection Threats to Internal Validity Threats

28. History • History: During the time that an experiment is taking place, some events may occur that confuse the relationship being studied. One Group Pretest-Posttest

29. Maturation • Maturation: Changes may also occur within the participant that are a function of the passage of time and are not specific to any particular event. A participant may become hungry, bored, or tired and these conditions can affect response results.

30. Testing • Testing: The process of taking a test can affect the scores of a second test.

31. Instrumentation • Instrumentation: This threat to internal validity results from changes between observation in either the measuring instrument or the observer.

32. Selection • Selection: An important threat to internal validity is the selection of participants for experimental and control groups. The groups should be equivalent in every respect.

33. Statistical regression • Statistical regression: This factor operates especially when groups have been selected by their extreme scores.

34. Experimental mortality • Experimental mortality: This occurs when the composition of the study groups changes during the test.

35. Additional Threats to Internal Validity

36. External Validity • External validity is concerned with the interaction of the experimental treatment with other factors and the resulting impact on the ability to generalize to (and across) times, settings, or persons. Three major threats to external validity are presented in the slide.

37. Threats to External Validity

38. Reactivity of testing on X • The reactive effect refers to sensitizing participants via a pretest so that they respond to the experimental stimulus (X) in a different way. • For instance, people who participate in a web survey like the one from Kohl’s shown in the slide may then be sensitized to store displays and organization.

39. Interaction of selection and X • The process by which test participants are selected for an experiment may be a threat to external validity. • The population from which one selects participants may not be the same as the population to which one wishes to generalize the results.

40. Other reactive factors • The experimental settings themselves may have a biasing effect on a participant’s response to X. • An artificial setting can produce results that are not representative of larger populations. • If participants know they are participating in an experiment, there may be a tendency to role-play in a way that distorts the effects of X. • Another reactive effect is the possible interaction between X and participant characteristics.

41. Experimental Research Designs • Experimental designs vary widely in their power to control contamination of the relationship between the independent and dependent variables Pre-experiments True experiments Field experiments

42. Pre-experiments • After-Only Case Study X O • X refers to the treatment or manipulation of the independent. O refers to the observation or measurement of the dependent variable. The lack of a pretest and control group

43. Pre-experiments(Cont.) • One Group Pretest-Posttest O1 X O2 • This design meets the threats to internal validity better than the one-shot case study, but it is still a weak design.

44. Pre-experiments(Cont.) • Static Group Comparison X O1 O2 • This design provides for two groups, one of which receives the experimental stimulus while the other serves as a control. there is no way to be certain that the two groups are equivalent or that the individuals are representative.

45. True experiments • Pretest-Posttest Control Group Design R O1 X O2 R O3 O4 • This design deals with many of the threats to internal validity, but local history, maturation, and communication among groups can still lead to problems. • External validity is threatened because there is a chance for a reactive effect from testing.

46. True experiments(Cont.) • Posttest-Only with Control Group R X O1 R O2 X refers to the treatment or manipulation of the independent. O refers to the observation or measurement of the dependent variable.

47. Field experiments • Nonequivalent Control Group Design O1 X O2 O3 O4 X refers to the treatment or manipulation of the independent. O refers to the observation or measurement of the dependent variable.

48. Field experiments(Cont.) • Separate Sample Pretest-Posttest R O1 (X) R X O2 X refers to the treatment or manipulation of the independent. O refers to the observation or measurement of the dependent variable.

49. Field experiments(Cont.) • Nonequivalent Control Group Design R O1 O2 O3 X O4 O5 O6 R O7 O8 O9 O10 O11 O12 X refers to the treatment or manipulation of the independent. O refers to the observation or measurement of the dependent variable.