Chapter 7 Control Techniques in Experimental Design

1. Chapter 7Control Techniques in Experimental Design Control: techniques and design features that serve to eliminate the influences of confounding extraneous variables. ch8(1)

2. Sampling In an ideal world we would randomly select our subjects from the larger population we are interested in studying (ensures External Validity) and then we would randomly assign participants to the control and experimental conditions (ensures Internal Validity). Rarely when studying people is this possible. ch8(1)

3. More emphasis on Internal validity than External Validity. • In psychology we are generally more concerned with whether the IV is truly causing a change in the DV than with the generalizability of the effect. • External Validity is often established by replicating the study with other populations. ch8(1)

4. Controlling for Confounding Extraneous Variables Ensures that the DV is the only variable which has a differential influences on conditions being compared in the study. ch8(1)

5. Randomization(Random Assignment) Each participant in the study has an equal and unbiased chance of being assigned to any of the conditions being compared in the experiment. Equates Groups by controlling for both known and unknown extraneous variables. ch8(1)

6. Between Subjects Designs Each participant serves in only one condition in the study. Comparisons are made between conditions therefore the biggest potential confound is that there may be systematic differences between the individuals being compared other that those created by manipulating the IV. Confound: Individual Differences ch8(1)

7. Best Solution! Random assignment of individuals to conditions. Random processes only work when you have a large enough sample size to work with. In General if Number if participants per condition is greater than 15 Random processes will work. ch8(1)

8. Matching Techniques Techniques for equating subjects assigned to different conditions for specific extraneous variables. - can be used for Smaller Sample Sizes. ch8(1)

9. Other Matching Techniques Matching by Holding Variable Constant E.g., I only include subjects with IQ scores of 100. Differences in IQ are eliminated as an extraneous variable. Weakness – limits external validity because we do not know if the finding generalizes to people of higher and lower intelligences. ch8(1)

10. Matching by Treating Extraneous Variable as a Second Factor Like running 3 separate studies at the same time, each with subjects in different intelligence groups. Allows us to control for intelligence and still determine if the results generalize. ch8(1)

11. Statistical Control - extraneous variables that might be confounds are measured during the study. The effects of these variables are then controlled for (removed) during statistical analysis. More often used with natural manipulation studies (correlational studies) to statistically control for confounds that cannot be otherwise controlled. ch8(1)

12. Matching by Yoked Control The control group mirrors the experimental group on some variable. REM sleep deprivation study. Subject woken up when ever they enter REM. How do you produce a meaningful control group? Example –sleep study (begin at 8.11) ch8(1)

13. Matching by Equating Participants a) Precision Control Matched on sets of participants on case-by-case basis. Participants are matched on a characteristic that the researcher wants to control difference in. Then the pairs are randomly assigned each to one of the two conditions. ch8(1)

14. e.g., If I have a small number of M&Ms (12) and there are 6 colors, if I use random processes I could end up with Two groups that look very different from each other. Creating pairs that are matched for color and assigning 1 member of each pair to each group will insure that the distribution of colors in each group are the same. ch8(1)

15. While we know that the two groups are Matched for color – this does not ensure that they are matched for other variables. Therefore, we randomly assign each member of a pair to the groups. e.g., Twin Studies: Identical Twins are biologically matched for DNA. They differ however on many other factors. If we were to do a twin study we would take sets of twins and then us a random process to determine which twin is assigned to which condition. ch8(1)

16. B) Frequency Distribution Control Mean, and Standard deviation of the Control and Experimental groups are matched. Caution: Groups that have the same means and St. dev can have very different distributions. (See Text Figure 7.5). ch8(1)

17. Within Subjects (Participants) Designs All participants serve in each condition of the study. Advantage: Individual Differences are not a Confound. ch8(1)

18. Types of WS Designs 1) Pre-Post Design with no treatment control group. Used when the purpose is to show a change between conditions (i.e. treatment or an outcome) and the order of the conditions is logically set (i.e., pre test must proceed post test). ch8(1)

19. WS design with counterbalancing for order. Used when comparing conditions and the conditions can be measured in any order. Weakness: Sequencing Effects: Participation in one condition may effects participation in a subsequent condition(s). ch8(1)

20. A) Order Effects Changes in performance due to practice, familiarity, boredom that are due to repeated testing and not to differences in the IV. ch8(1)

21. Carry-over Effects Long-lasting effects participation in one condition that carry-over to effect performance in subsequent conditions. e.g., Learning, Expectation, attitude, physical changes due to experimental condition. E.g., Reward Contrast effects in Rats ch8(1)

22. Example of Carry Over effects: Reward Contrast effects in Rats ch8(1)

23. Controlling for Sequence Effects (WS Design) 1)Counterbalance - controls for order effects and for unexpected carry-over effects. Counterbalancing: Within your study, each condition occurs equally as often in each ordinal position, and each condition occurs equally as often before and after each other condition. ch8(1)

24. Randomized Counterbalancing Each Subject gets a different order. ABC ACB BCA BAC CAB CBA Subjects are randomly assigned to order conditions. ch8(1)

25. Complete Randomized Counterbalancing All possible orders of conditions are used in and participants are randomly assigned so that there are equal numbers in each order condition. ch8(1)

26. Equal numbers of subjects are assigned to each order condition, thus order effects are controlled by equalling the effects of order for each condition. ch8(1)

27. Incomplete Counterbalancing With 4 conditions all orders would total 24. ABCD BACD CABD DABC ABDC BADC CADB DACB ACBD BCAD CBAD DBAC ACDB BCDA CBDA DBCA ADAB BDAC CDAB DCAB ADBA BDCA CDBA DCBA ch8(1)

28. Rules for Incomplete Counterbalancing • Each condition must appear an equal number of times in each ordinal position. • Each condition must precede and be followed by every other condition equal numbers of times. ch8(1)

29. Intrasubject counterbalancing Each subject gets each possible order. ABCCBA ch8(1)

30. Between Subjects Designs – Each participant serves in only one condition in the study. Confound: Individual Differences ch8(1)

31. Control of Individual Differences e.g., IQ, personality, sex, race, age etc. Random Assignment - Each member of the sample has an equal chance of being assigned to any of the conditions. ch8(1)

32. Power Ability to find a difference between conditions when a difference actually exists. When designing a study we want high power!! ch8(1)

33. Power and Designs • WS designs have higher power than BS designs and should ALWAYS be considered first. ASK Can I counterbalance for order? If yes, then counterbalance. If no, use pre-post test with no-treatment control group. ch8(1)

34. BS designs are used only when participants cannot be in more than one condition of the study. • Being in each condition permanently changes a person so that they cannot be in other conditions of the study. ch8(1)

35. Comparison between treatment types here is a BS comparison ch8(1)

36. BS designs are commonly used for Natural Manipulation studies (e.g., sex differences, personality differences). These are not experiments and they have low Internal Validity. ch8(1)