Basics of Experimentation

1. Basics of Experimentation

2. Basics of Experimentation • Variables and Control • Importance of replication e.g. study of alcohol consumption and test performance • Operational definition – IV and DV defined by the operations used to produce or measure them. • Operational definition crucial for replication and for communication of study results.

3. Examples of operational definitions • Alcohol consumption • Bph (beers per hour) • cc injected at 10 percent alcohol • Total volume of liquid consumed at 1 sitting (may or may not have alcohol). • Intoxication • Number of steering wheel changes of direction while driving • Insults recorded on videotape during a conversion with a confederate • Confederate’s subjective assessment of intoxication • Reaction time in msec by stopwatch

4. Example study – aircrew training for beginning pilots • Half the pilots role played a flight with a confederate ATC. The ATC made 2 mistakes, one on the clearance, one a vector into a thunderstorm. • The other half played video games together and did other exercises. • All then flew together in a high fidelity simulator and were given problems in flight. • Two instructor pilots watched videotapes of crews in hifi sim and rated their performance.

5. Desirable qualities for an Independent Variable • Relevant to research question. E.g,. Want to know effects of simulator time on pilot proficiency. What about simulator (experience, duration, etc. Manipulated ATC mistakes.) • Sampled appropriately. Either broad range (sim time) or representative sample (study time). A possible problem for our example. • Manipulability (at least in experiments). Can assign values (e.g., training time; ATC mistakes).

6. Desirable qualities for a Dependent Variable • Relevant to research question (e.g., pilot proficiency) Instructor ratings? Self evaluation? Incidents? Client satisfaction with flight? • Reliable – free from measurement error. Repeatable, consistent. 2 instructor pilots to check for agreement. • Sensitive – capable of showing differences. Icing problem vs. ATC problem.

7. Review • Suppose we want to study the influence of alcohol consumption on decision making. • Design a study by choosing an IV and a DV. Describe the operational definitions of each. • What levels of alcohol did you choose? Why? Is alcohol manipulated in the study? • Why did you choose the DV that you did? Is it relevant? How? Is it likely to be reliable and sensitive to your manipulation?

8. Extraneous Variables • Extraneous variables have unintended influence on the DV. Two flavors: • Nuisance variables increase the variability within groups and make it harder to see effects. Pilot previous experience with ATC. • Confounders (confounding variables) change the difference between groups, either increasing or decreasing treatment effect (e.g., if we let pilots choose their training treatment of video games or PC flight). Experimenter effects.

9. Nuisance Variable Study with Nuisance vbl Study without Nuisance vbl

10. Confounding Variable Confounder Present Confounder Absent

11. Example Studies • Pizza study. IV – promotional. DV – sales. Nuisance – prior sales volume. Confounder – excellence of mgmt. • Clothing study. IV – dress for success. DV – business mgrs evaluations of pictures for management jobs. Nuisance – background of managers. Confounder – sex of people in pictures.

12. Controlling Extraneous Variables • Randomization – assign people to treatments with equal probability. • Elimination – remove the variable entirely. Noise is eliminated in a quiet room. • Constancy. Turn a variable into a constant. E.g., choose only women as participants in a study.

13. Controlling Extraneous Variables (2) • Counterbalancing. Reversing orders of stimuli or conditions. Helps with carry-over and contrast effects. E.g., Pepsi challenge, physical ability testing.

14. Experimenter as Extraneous Variable • Physiological and demographic differences • Sex of experimenter (e.g., clothing, conformity) • Race of experimenter (opinion surveys) • Psychological • Friendliness (e.g., openness & social desirability) • Competence (bumbling experimenter, conformity, compliance)

15. Experimenter as Extraneous variable (2) • Experimenter Expectancy • Classroom performance of “intellectual bloomers” – Rosenthal effect • ESP. Students recruited to help study ESP and told either good or bad things about ESP.

16. Controlling Experimenter Effects • Controlling demographic and psychological effects • Make constant • Balance or counterbalance • Remove experimenter • Controlling experimenter expectancy • Script experimenter contact • Eliminate the person (use a machine) • Use single blind in which experimenter is not told information about participants

17. Choosing Participants • Suitability (easily the most important) • Generalizability to a context (e.g., pilots). Factors like experience, abilities, attitudes, consequences of behavior. • For theoretical relevance (e.g. anorexics). • Precedent • Availability • Number of participants • Time, money, availability, size of treatment effect.

18. Review • Consider your study of the influence of alcohol consumption on decision making. • Describe a nuisance variable and how you might control it. • How could an experimenter be an extraneous variable in your study? How could you fix that? • Who would be good participants in your study?