Business Research Methods Donald R. Cooper and Pamela S. Schindler

1. Business Research MethodsDonald R. Cooper and Pamela S. Schindler Chapter 12.1.: Basics of Experimental Research

2. Chapter 12.1.: Experimental Research 1. Basics of Experiment & Causality 2. Advantages and disadvantages of the experimental method 3. Steps of a well-planned experiment 4. Validity in experiments

3. 1.Basics of Experiment & Causality 1.1. Definition of Experiment: An experiment is a study involving intervention by the researcher beyond that required for measurement. The usual intervention is to manipulate some variable in a setting and observe how it affects the participants or subjects being studied. There is at least one independent variable and one dependent variable in a causal relationship.

4. 1.2. Causal EvidenceThere are three types of evidence necessary to support causality. Agreement between Independent and Dependent Variables Time order of occurrence Extraneous variables did not influence Dependent Variables

5. 1.2.1. Agreement between Independent and Dependent Variables First, there must be an agreement between independent and dependent variables. The presence or absence of one is associated with the presence or absence of the other.

6. 1.2.2. Time order of occurrence Second, beyond the correlation of independent and dependent variables, we consider the time order of the occurrence of the variables. The effect on the dependent variable should not precede the manipulation of the independent variable. The effect and manipulation may occur simultaneously or the manipulation may occur before the effect.

7. 1.2.3. Extraneous variables did not influence Dependent Variables The third source of support comes when researchers are confident that other extraneous variables did not influence the dependent variable. To ensure that these other variables are not the source of influence, researchers control their ability to confound the planned comparison.

8. 2. Advantages and disadvantages of the experimental method Advantages • Ability to manipulate Independent Variable • Use of control group • Control of extraneous variables • Replication possible • Field experiments possible Disadvantages • Artificiality of labs • Non-representative sample • Expensive • Focus on present and immediate future • Ethical limitations

9. 2.1.Explanation of Some Advantages of Experiments • Replication: is the process of repeating an experiment with different participant groups and conditions to determine the average effect of the Independent Variables across people, situations, and times. • A field experiment: is a study of the dependent variable in actual environmental conditions.

10. 2.2.Explanation of Some Disadvantages of Experiments • The artificiality of a lab is possibly the greatest disadvantage of experiments. • Also, experiments typically use small convenience samples which cannot be generalized to a larger population. • Compared to surveys, they are expensive. • They also cannot deal with past events or predict events in the far-off future. • Finally, marketing research is often concerned with the study of people and there are limits to the types of manipulation and controls that are ethical.

11. 3. Steps of a well-planned experiment Specify treatment variables Specify treatment levels Control environment Choose experimental design Select and assign participants Pilot-test, revise, and test Collect data Analyze data

13. Steps of a well-planned experiment The activities the researcher must accomplish to make an experiment a success: 3.1. Specify treatment variables: a) select variables that are the best operational definitions of the original concepts, b) determine how many variables to test, c) select or design appropriate measures for the chosen variables. The selection of measures for testing requires a thorough review of the available literature and instruments.

14. 3.2. Specify treatment levels: In an experiment, participants experience a manipulation of the independent variable, called the experimental treatment. The treatment levels are the arbitrary or natural groups the researcher makes within the independent variable. A control group is a group of participants that is measured but not exposed the independent variable being studied. A control group can provide a base level for comparison.

15. 3.3. Control environment: Environmental control means holding the physical environment of the experiment constant. When participants do not know if they are receiving the experimental treatment, they are said to be blind. When neither the participant nor the researcher knows, the experiment is said to be double-blind. 3.4. Choose experimental design: The design is then selected. Several designs are discussed on the next several slides.

16. 3.5. Select and assign participants: The participants selected for the experiment should be representative of the population to which the researcher wishes to generalize the study’s results. Random assignment is required to make the groups as comparable as possible. Random assignment uses a randomized sample frame for assigning participants to experimental and control groups. Matching is an equalizing process for assigning participants to experimental and control groups.

17. 3.5.1. Random assignment : • The sampling frame is often small for experiments and the participants may be self-selected. • However, if randomization is used, those assigned to the experimental group are likely to be similar to those assigned to the control group. • Random assignment allows one to make the groups as comparable as possible. • It means that participants have an equal and known chance of being assigned to any of the groups in the experiment.

18. 3.5.2. Matching : Matching is a control procedure to ensure that experimental and control groups are equated on one or more variables before the experiment. The object of matching is to have each experimental and control participant matched on every characteristic used in the research.Matching employs a nonprobability quota sampling approach. Quota matrix is a means of visualizing the matching process. If matching does not alleviate assignment problems, a combination of matching, randomization, and increasing the sample size may be useful.

19. Quota Matrix Example Exhibit 10-3 presents an example of a quota matrix. One-third of the participants from each cell of the matrix would be assigned to each of the tree groups.

20. 4. Validity in Experimentation Internal validity exists when the conclusions drawn about a demonstrated experimental relationship truly implies cause. External validity exists when an observed causal relationship can be generalized across persons, settings, and times.

21. 4.1.Threats to Internal Validity There are twelve possible threats to internal validity: • History • Maturation • Testing • Instrumentation • Selection • Statistical regression • Experimental mortality • Diffusion or imitation of treatment • Compensatory equalization • Compensatory rivalry • Resentful Demoralization of the disadvantaged • Local history http://cde.annauniv.edu/CourseMat/mba/sem2/dba1657/val.html

22. Threats to internal validity • History: In the experimental designs a control measurement (O1) of dependent variable is taken before introducing the manipulation (X). After the manipulation an after measurement (O2) of the dependent variable is taken. Then the difference between O1 and O2 is attributed to the manipulation. (See also One Group Pretest-Posttest Design) However some events may occur during the course of the experimental study, which will affect the relationship between the variables under the study.

23. Threats to internal validity • Maturation: Changes may also occur within the participant that are a function of the passage of time and are not specific to any particular event. • A participant may become hungry, bored, or tired and these conditions can affect response results. • Testing: The process of taking a test can affect the scores of a second test. For instance, repeatedly taking (the same or similar) intelligence tests usually leads to score gains.

24. Threats to internal validity • Instrumentation: This threat to internal validity results from changes between observationsin either the measuring instrument or the observer. • Selection: Differential selection of subjects for experimental and control groups affects the validity. Validity considerations require the groups to be equivalent in every aspect. The problem can be overcome by randomly assigning the subjects to experimental and control groups. In addition matching can be done. Matching the members of the groups on key factors also enhances the equivalence of the groups.

25. Threats to internal validity • Statistical regression: This factor operates especially when groups have been selected by their extreme scores. • For example, when children with the worst reading scores are selected to participate in a reading course, improvements at the end of the course might not be due to the course's effectiveness. • Experimental mortality: This occurs when the composition of the study groups changes during the test.Some participants may drop out the experiment.

26. Threats to internal validity • Diffusion or imitation of treatment: If people in the experimental and control groups talk, then those in the control group may learn of the treatment. This eliminates the difference between the groups. • Compensatory equalization: Where the experimental treatment is much more desirable for the experimental group, there may be an administrative reluctance to deprive the control group members. Actions to compensate the control group may confound the experiment.

27. Threats to internal validity • Compensatory rivalry: This may occur when members of the control group know they are in the control group. This may generate competitive pressures, causing the control group members to try harder. (e.g. Hawthorne effect ) • Resentful demoralization of the disadvantaged: When the treatment is desirable and the experiment is conspicuous, control group members may become resentful that they are deprived and lower their cooperation and output.

28. Threats to internal validity • Local history: The regular history effect already mentioned impacts both experimental and control groups alike. When one assigns all experimental persons to one group session and all control group people to another, there is a chance for some peculiar event to confound results.

29. 4.2.Threats to External Validity External validity is concerned with the interaction of the experimental treatment (X) with other factors and the resulting impact on the ability to generalize to (and across) times, settings, or persons. External validity is high when the results of an experiment are applicable to a larger population. Three major threats to external validity are as follows: Reactivity oftesting on X Interaction of selection and X Other reactive factors

30. Reactivity of testing on X • The reactive effect refers to sensitizing participants via a pretest so that they respond to the experimental stimulus (X) in a different way. • For instance, people who participate in a web survey may then be sensitized to store displays and organization.

31. Interaction of selection and X The process by which test participants are selected for an experiment may be a threat to external validity. The population from which one selects participants may not be the same as the population to which one wishes to generalize the results. It limits the generalizability of the findings.

32. Other reactive factors • The experimental settings themselves may have a biasing effect on a participant’s response to X. • An artificial setting can produce results that are not representative of larger populations. • If participants know they are participating in an experiment, there may be a tendency to role-play in a way that distorts the effects of X. • Another reactive effect is the possible interaction between X and participant characteristics.

33. Business Research MethodsDonald R. Cooper and Pamela S. Schindler Chapter 12.2.: Types of Experimental Research Designs

34. Chapter 12.2.: Types of Experimental Research Designs 1. Pre-experiments 2. True experiments 3. Field experiments

35. X refers to the treatment or manipulation of the independent variable (more than one X refers to a different level of treatment). • O refers to the observation or measurement of the dependent variable. • Experimental designs vary widely in their power to control contamination of the relationship between the independent and dependent variables. • Experiments can be categorized as pre-experiments, true experiments, and field experiments based on the characteristic of control.

36. 1. Pre-experiment Pre-experimental research designs are research designs that are characterized by a lack of random selection and assignment. Types of Pre-experiments: • After-Only Case Study • One Group Pretest-Posttest Design • Static Group Comparison

37. 1.1. After-Only Case Study X O • In this type of experimental design only one treatment (X) or manipulation is done on the independent variable. • Then, the dependent variable is measured.

38. X O An example is a media campaign about a product’s features without a prior measurement of consumer knowledge. Results would reveal only how much target consumers know after the media campaign, but there is no way to judge the effectiveness of the campaign. The lack of a pretest and control group makes this design inadequate for establishing causality.

39. 1.2. One Group Pretest-Posttest Design O1 X O2 This design meets the threats to internal validity better than the one-shot case study, but it is still a weak design. For example, a researcher examining the effect of a commercial on brand liking would begin by taking a pre-test to determine current levels of brand liking among the participants.

40. O1 X O2 The commercial would be shown. Then a post-test would measure brand liking after the commercial. A comparison between the post-test and the pre-test shows the change in liking. However, any changes in liking are not necessarily due to the commercial. The act of giving a pre-test could have influenced liking (testing effect).

41. 1.3. Static Group Comparison ExperimentalGroup:X O1 ControlGroup: O2 This design provides for two groups, one of which receives the experimental stimulus while the other serves as a control.

42. For example, imagine that a new type of cheeseburger is being introduced, and an advertisementcampaign is run. After the ad airs, those who remember seeing it would be in the experimental group (X). Those who have no recall of the ad would be in the control group. The intent of each group to purchase the cheeseburger would be measured. The main weakness of this design is that there is no way to be certain that the two groups are equivalent or that the individuals are representative.

43. 2. True experiment • A true experiment is a method of social research in which there are two kinds ofvariables. Theindependentvariable is manipulated by the experimenter, and thedependentvariable is measured. • The signifying characteristic of a true experiment is that it randomly allocates the subjects in order to neutralize the potential to ensure equivalence. • There is alsoa control group for comparison. Types of Trueexperiments: • Pretest-Posttest Control Group Design • Posttest-Only Control Group Design

44. 2.1.Pretest-Posttest Control Group Design Experimental Group: R O1 X O2 Control Group: R O3 O4 • Thesymbol R meansthatthe true experimental designs use randomly assigned groups to ensure equivalence. • The effect of the experimental is: E = (O2-O1) – (O4-O3). • This design deals with many of the threats to internal validity, but local history, maturation, and communication among groups can still lead to problems. • External validity is threatened because there is a chance for a reactive effect from testing.

45. 2.2. Posttest-Only Control Group Design Experimental Group: R X O1 Control Group: R O2 • In this design, the pretest measurements are omitted. • Pretests are well established in classical research design but are not really necessary when it is possible to randomize. • The experimental effect is measured by the difference between O1 and O2. • Internal validity threats from history, maturation, selection, and statistical regression are controlled adequately by the random assignment. • Different mortality rates could cause a problem.

46. Example for Posttest-Only Control Group Design • Buick dealerships wish to determine the effectiveness of a special “test-drive” incentive. • Buick dealerships nationwide are randomly assigned to either the control group or the experimental group. • Those in the experimental group use a promotion to encourage test drives. • The control group does not use any such promotions. • The number of test drives throughout are measured and compared to determine if the promotion resulted in significantly more test drives.

47. 3. Field experiment • Experiment conducted in a natural setting (e.g. on a sports field during play). The conditions of field experiments are usually very difficult to replicate. Types of Field experiments: • Nonequivalent Control Group Design • Separate Sample Pretest-Posttest Design • Group Time Series Design

48. 3.1. Nonequivalent Control Group Design Experimental Group: O1 X O2 Control Group: O3O4 • This is a strong and widely used quasi-experimental design. • It differs from the pretest and posttest control group design because the test and control groups are not randomly assigned. • There are two varieties: intact equivalent design and self-selected experimental group design.

49. Nonequivalent Control Group Design • In the intact equivalent design, the membership of the experimental and control groups is naturally assembled. • The self-selected experimental group design is weaker because volunteers are recruited to form the experimental group, while non-volunteer participants are used for control. • A comparison of the pretest results for each group is one indicator of the degree of equivalence between test and control groups.

50. Example for Nonequivalent Control Group Design • For example, children from two different classes in school may be asked to test a toy. • Participants are pre-tested on their interest in the toy. • The experimental group spends time playing with the toy while the control group is not exposed to the toy. • A post-test then measures interest in the toy.