1 / 37

New Complications

New Complications. Adding a control group to our design deals with the confounds discussed so far Unfortunately, however, it also creates new potential confounds. Selection. With a between-subjects variable, one has multiple groups of people

rylee-rice
Download Presentation

New Complications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New Complications • Adding a control group to our design deals with the confounds discussed so far • Unfortunately, however, it also creates new potential confounds

  2. Selection • With a between-subjects variable, one has multiple groups of people • You want the groups of people to be as similar as possible to one another • This is known as having equivalent groups • Notice that I did not say identical groups • The groups will differ to some extent

  3. Group Differences • What differences are acceptable or unacceptable? • Acceptable • The groups differ in terms of an uncontrolled variable that does NOT correlate with the DV • Unacceptable • The groups differ in terms of an uncontrolled variable that CORRELATES with the DV • These are known as systematic differences

  4. Example • Does high vs. low waitperson attentiveness affect tipping? • Acceptable • Groups differ in terms of hair color • High attentiveness = mostly blondes • Low attentiveness = mostly brunettes • Unacceptable • Groups differ in terms of table waiting experience • High attentiveness = mostly waited tables • Low attentiveness = mostly not

  5. Confound • If groups differ in terms of an uncontrolled variable that correlates with the DV, then the study MAY be confounded • To determine whether the study is confounded, one must consider whether the uncontrolled variable provides an alternative explanation for the study’s outcome

  6. Example • Outcome • Tips were higher in the high attentiveness condition than the low attentiveness condition • Not confounded • High attentiveness = mostly not • Low attentiveness = mostly waited tables • Confounded • High attentiveness = mostly waited tables • Low attentiveness = mostly not

  7. Example • Outcome • Tips were equal in the high and low attentiveness conditions • Not confounded • High attentiveness = mostly waited tables • Low attentiveness = mostly not • Confounded • High attentiveness = mostly not • Low attentiveness = mostly waited tables

  8. Solutions • There are a number of ways to mitigate a Selection confound • Make the uncontrolled variable another IV • Create equivalent groups • By randomly placing participants into one’s conditions • This is better than matching when collecting a large sample • By matching participants in one’s conditions • This is better than randomization when collecting a small sample

  9. Example: Make IV • Do waitperson attentiveness (high vs. low) and parton experience (waited tables vs. not) affect tipping?

  10. Example: Randomize • Randomly assign participants to the high and low attentiveness conditions • One can do this purely randomly wherein every participant has an equal chance of being assigned to either condition • One can do this quasi-randomly with the constraint that once a participant has been assigned to a condition no other participants will be assigned to that condition until all conditions have an equal number of participants • This is known as block randomization

  11. Example: Matching • Pair participants who have waited tables • Pair participants who have not waited tables • Randomly assign one member of each pair to the high and low attentiveness conditions • To be successful, one must have a good reason to match participants, as well as a good way to measure the matching variable

  12. Our studies • Experimental group • Pre-Test, Verbal Training w/ Feedback, Post-Test • Control group • Pre-Test, Verbal Training w/o Feedback, Post-Test • Participants were placed randomly into the Experimental (Feedback) and Control (No Feedback) groups

  13. Attrition • Sometimes people in one group quit the study more often than people in the other group • This is known as an attrition problem

  14. Attrition 1. Collect active Pre-Test data 2a. Experimental group: Provide training with feedback • Many people in this group do not complete the study 2b. Control group: Provide training without feedback • Few people in this group do not complete the study 3. Don’t change anything else 4. Collect active Post-Test data 5. Compare Pre and Post-Test data • Is difference due to full training, or who quit the study?

  15. Solution • There is nothing methodological that one can do to prevent an attrition confound • One must be watchful for them, and adjust conclusions accordingly, if one is suspected

  16. Order • Within-Subjects variables expose each participant to every level of the Independent Variable • Accordingly, in most circumstances, the order in which the levels are presented to the participant becomes an issue • This is known as an Order problem

  17. Solution • Order problems can be mitigated by presenting the levels of the Independent Variable in different orders to different people • The orders must be created so that each level occurs (roughly) equally often at each phase of the testing session

  18. Example • Consider a variation of our studies • In this variation, participants will throw the beanbag to a target, with their eyes closed, and with them open • Throwing repeatedly might cause fatigue, so throwing in second half of the study will always be worse than throwing earlier in the study • Solution • Half of the participants throw with their eyes closed first, and then with their eyes open, while the other half throws in the opposite order

  19. Counterbalancing • Complete counterbalancing • Uses each of the possible orders of treatments • The number of possible orders is found by X! • where “X” is the number of conditions and “!” is the factorial • As the number of conditions increases, then the number of potential orders increases markedly • Requires as many participants as possible orders

  20. Counterbalancing • Partial counterbalancing • Uses a subset of the possible orders • The subset may • Be chosen at random • Employ a Latin Square

  21. Latin Square • An example of a Latin Square would be • A, B, C, D • B, C, D, A • C, D, A, B • D, A, B, C • In a standard latin square, each level occurs equally often in every sequential position. • One or more people would be exposed to each of these orders

  22. Our Independent Variables • Between-Subjects • Feedback • Two conditions, i.e., with and without feedback • Within-Subjects • Session • Three levels, i.e., Pre-Test, Training, Post-Test

  23. Our studies • In our studies, we must run the levels of our Within-Subjects variable (Session) in a particular order, so we must not counterbalance

  24. Carry-Over Effects • When employing a within-subjects IV, what happens during one level sometimes affects what happens during subsequent levels • This is known as a carry-over effect

  25. Example • Does high vs. low waitperson attentiveness affect tipping? • High attentiveness  Low attentiveness • In this order, Low attentiveness seems worse than it would have otherwise because of prior exposure to High attentiveness • Low attentiveness  High attentiveness • In this order, High attentiveness seems better than it would have otherwise because of prior exposure to Low attentiveness

  26. Example • Note that each sequence has a unique effect on the data • High  Low = Low seems worse • Low  High = High seems better • Note also that these effects do NOT cancel • The worsening of the Low attentiveness level is not counteracted by anything • The enhancing of the High attentiveness level is not counteracted by anything

  27. Counterbalancing • Given that each sequence has a unique effect on the data, and those unique effects do not cancel one another, counterbalancing does NOT eliminate carry-over effects

  28. Example • High  Low = Low seems worse • Low  High = High seems better • When you combine the data, you get • High + High (better) • Low + Low (worse)

  29. Solution • If one suspects that a carry-over effect may happen, then it is advisable to use a Between-Subjects Independent Variable • This has consequences, though, because within-subjects variables require less participants and have greater statistical power than between-subjects variables

  30. Bias • Lastly, two forms of bias can confound one’s experiment • Experimenter bias • Experimenters influence the participants unintentionally in ways that make the study come out as expected • Participant bias • Participants may act differently because they are in an experiment, which is known as a Hawthorne effect • Participants may try to be too good, which is known as an Orne effect

  31. Solutions • Experimenter bias • Minimize and standardize the interaction between the participant and the experimenter • When possible, design studies so that participants and/or experimenters and participants do not know what condition is being run • These are known as blind, and double-blind studies, respectively

  32. Solutions • Participant bias • Minimize and standardize the interaction between the participant and the experimenter • Provide the participants with just enough information so that they can make an educated judgment about informed consent, but no more • This is known as having naïve participants

  33. Our studies • In our studies, we use a standard consent form and instructions, in order to minimize and standardize the experimenters’ interaction with the participants

More Related