Revised 9/11/13

1. Psychology 242, Dr. McKirnan Revised 9/11/13 Right click for “full Screen” or “end show”. Left click to proceed, Lectures 3: Developing research questions and hypotheses. Basic experimental design & internal validity. Week 3; Experimental designs

2. Research questions, hypotheses & designs • How do social values affect science? • Where do research questions & hypotheses come from. • Variables in research • Basic experimental designs • True vs. Quasi- Experiments Phenomenon Theory Hypothesis Methods & data Results Discussion & Conclusions Week 3; Experimental designs

3. The big picture: Introductory lectures  • How do social values affect science? • Where do research questions & hypotheses come from. • Variables in research • Basic experimental designs • True vs. Quasi- Experiments Phenomenon Theory Hypothesis Methods & data Results Discussion & Conclusions Week 3; Experimental designs

4. Social Values… Phenomenon …affect what we choose as our research question… Theory Hypothesis Methods & data Specific methods are more standard and objective Results Discussion & Conclusions …and our conclusions Week 3; Experimental designs

5. Values, theory and data in the scientific process. Social values help define a scientific “problem” or question. Norms, values (& data) determine what is credible / fundable. Phenomenon Theory Theory is influenced by norms + empirical background of field. Hypothesis Science hinges on clear, objectively stated hypotheses. Clear hypotheses lessen bias in interpreting results. Methods & data Methods & analyses are most objective, but fields vary in methodological rigor. Results Discussion & Conclusions • The “meaning” of a finding is influenced by cultural & social values or concerns. …for science and, particularly, for society. Week 3; Experimental designs

6. Values and science: The internet and sexual risk. Social values help define a scientific “problem” or question. Norms, values (& data) determine what is credible / fundable. Phenomenon Theory Sexual internet use became a large issue for gay/bisexual men in the 2000s • Internet use exploded, and became a sexual venue. • Men who have sex with Men [MSM] seemed particularly likely to be risky via internet sex. • The HIV prevention field needed to understand how this worked. • HIV made research on MSM a legitimate and important topic Hypothesis Methods & data Results Discussion & Conclusions Week 3; Experimental designs

7. Values, theory and data in the scientific process. Phenomenon Theory Theory is influenced by values & empirical background of field. Hypothesis Several theories may help us explain internet  sexual risk: • Theories of depression: a Ψ factor within the person may interfere with his decision making. • Social structure: The social environment of MSM may influence internet use & risk. Taking an individual v. social perspective can be a value choice. Methods & data Results Discussion & Conclusions Week 3; Experimental designs

8. Values and science: climate change, 1. Phenomenon Theory Hypothesis Science hinges on clear, objectively stated hypotheses. Clear hypotheses lessen bias in interpreting results. Methods & data • Statistical tests of “mediation” can test relatively clear tests of hypotheses about depression and sexual risk Results Internet +sex Depression Risk Discussion & Conclusions   Week 3; Experimental designs

9. Values, theory and data in the scientific process. Widely shared conventions make value judgments less important. However: • Behavioral sciences vary considerably in their rigor. • Choosing, e.g., quantitative vs. qualitative research can be a value choice. • Fields such as literary criticism, history or feminist studies may use substantially different methods. Phenomenon Theory Hypothesis Methods & data Methods & analyses are most objective, but fields vary in methodological rigor. Results Discussion & Conclusions Week 3; Experimental designs

10. Values, theory and data in the scientific process. No scientific finding can completely answer a research question; • Its meaning is affected by existing theory and empirical findings. • Society may not be “ready” for certain findings: e.g., data showing that GLBT people are not “mentally ill” were rejected for many years. • Many important findings – e.g., from economics – have little affect on social policy if they contradict a widely shared ideology. Phenomenon Theory Hypothesis Methods & data Results Discussion & Conclusions • The “meaning” of a finding is influenced by cultural & social values or concerns. …for science and, particularly, for society. Week 3; Experimental designs

11. The big picture: Introductory lectures • How do social values affect science? • Where do research questions & hypotheses come from. • Variables in research • Basic experimental designs • True vs. Quasi- Experiments Phenomenon  Theory Hypothesis Methods & data Results Discussion & Conclusions Week 3; Experimental designs

12. Research questions Where do research questions come from? • Practical questions • Unanswered questions from previous research • Testing theories. Week 3; Experimental designs

13. Sources of research questions Practical / applied questions • Describe or explain an important social process • Evaluate an intervention or policy change Chicago school closings and reassignment are predicted to result in better scores. Do they over 5 years of follow-up? E X A MP L E Week 3; Experimental designs

14. Sources of research questions Unanswered questions from previous research • Clarify conflicting / unclear findings • Who actually uses the internet to meet people… • Do previous findings generalize to… • …different groups • Many Social Psychology studies enroll middle(+) classWhite female undergraduates in research labs. • …different research areas • Can interventions to increase healthy behavior generalize to recycling and energy conservation? • …different research approaches • Do controlled lab studies generalize to less controlled field research? Practical / applied questions Week 3; Experimental designs

15. Theories Practical / applied questions • Use existing theory to explain a new phenomenon • “Sensation seeking” personality is associated with drug use. Might it also explain unsafe sex in adolescents or gay men? • Test contrasting theories of a phenomenon • How much is adolescent problem behavior controlled by psychological variables (depression) vs. peer influence? • Develop new / expanded theory • We discriminate among very subtle differences in smell. Might olfactory cues affect who we are attracted to? Unanswered questions from previous research Testing theories Week 3; Experimental designs

16. Theories How do we go from a research question to an actual study? • Phenomenon • Theory • Hypothesis • Methods Key Terms: • Hypothetical Construct • Operational Definition Week 3; Experimental designs

17. The research process Phenomenon Overall issue or question; What controls emotional states? Why are some people vulnerable to depression? Theory Possible explanation: “How it works” statement Several theories may help explain the phenomenon Theory 1 Emotional stability requires secure emotional attachments. Theory 2 Some brains are genetically disposed to serotonin depletion during stress Week 3; Experimental designs

18. Research process, 2 Theory 1 Emotional attachment  emotional stability. A theory can lead to several hypotheses Hypothesis 1 Fewer parent – child interactions  vulnerability to depression. Hypothesis 2 Emotional support during stress  less depression A given hypotheses can be tested in several ways Methods 1 Survey measurement: assess # of “family meals” per week, correlate it with self-reported depression. Methods 2 Experiment: ½ have structured parent / child interactions, ½ do not, induce stress to both groups, assess depression Week 3; Experimental designs

19. Research process, 3 Theory 1 Emotional attachment  emotional stability. Hypothesis 1 Family interactions  depression. Hypothesis 2 (Non)Support + stress  depression Some hypotheses are best tested in a measurement approach, and some with experimental designs Best tested by a measurementstudy • Family interactions are difficult to bring into the lab, • Possible ethical problems. Can be tested in an experiment: Both support and stress can be controlled or manipulated in the lab. Week 3; Experimental designs

20. Research process: The Big Picture Phenomenon Big picture question. Theory 1 Possible explanation, invoking one set of hypothetical constructs. Theory 2 Alternate explanation, invoking other hypothetical constructs. Hypothesis 1 A prediction that logically flows from – and tests – the theory. Hypothesis 2 Another prediction that tests the same theory. Methods 1 Operationally define the variables & test the hypothesis. Methods 2 An alternate operational definition & way of testing the hypothesis. Week 3; Experimental designs

21. Question 1 A hypothetical construct is… A = A specific prediction about the outcome of an experiment B = A little known band from Muncie Indiana C = A general ψ process that underlies our observations D = A central element in a theory Week 3; Experimental designs

22. Question 2 To be testable, a hypothesis… Must rest on operational definitions. A = true B = False C = I don’t know Week 3; Experimental designs

23. Question 2 An operational definition is A = The procedure(s) we use to measure a study variable B = The way we define our theory C = The procedure(s) we use to manipulate a study variable D = What we use to derive our hypothesis Week 3; Experimental designs

24. Question 3 To be testable, a hypothesis… Must potentially be found to be false. A = true B = false C = I don’t know Week 3; Experimental designs

25. Question 1 A theory… A = Leads to one specific hypothesis B = May be one of several ways to explain something C = Is not as important as simply collecting data D = Is what you make up to explain why you forgot your boy/girl friend’s birthday E = Is not really affected by social or personal values Week 3; Experimental designs

26. The big picture: Introductory lectures • How do social values affect science? • Where do research questions & hypotheses come from. • Variables in research • Basic experimental designs • True vs. Quasi- Experiments Phenomenon Theory  Hypothesis Methods & data Results Discussion & Conclusions Week 3; Experimental designs

27. Theories Types and uses of variables in research: • Experiments • Independent vs. Dependent variables • Measurement / field studies • “Predictor” vs. “Criterion” • Forming variables: • Direct manipulation • Indirect manipulation • Measurement Week 3; Experimental designs

28. Types of Variables:Experimental designs Independent VariableDependent Variable Defines the “contrast space” Models the phenomenon What is compared to what e.g., drug v. placebo • What is being explained; e.g., task performance Hypothetical “cause” • “Effect” Imposed by researcher as experimental conditions • Measured as the outcome Categorical • Continuous Week 3; Experimental designs

29. Types of Variables:Measurement Studies PredictorCriterionorOutcome Defines the “contrast space” Models the phenomenon What predicts the outcome? e.g., age, education, gender • What is being explained; e.g., political attitude Hypothetical “Cause” • “Effect” Measured not imposed by researcher • Measured as the outcome Continuous • Continuous Week 3; Experimental designs

30. Experimental v. Measurement designs Experimental designs • Manipulating the Independent Variable: • Enhances internal validity • May lessen external validity • Participants randomly assigned to experimental v. control groups Measurement (or correlational)designs • Measurement “in the field” • May enhance external validity • Typically lessens internal validity • Sampling very important Week 3; Experimental designs

31. Creating independent variables [IVs] 1. Direct experimental manipulation • Most typical of “true” experiments • Maximum control over IV 2. Indirect manipulation via experimental or research conditions • Less direct control over IV 3. Quasi-Independent variables: forming groups using a measured variable. • Experiments without complete control over variables • Used in measurement studies Week 3; Experimental designs

32. Forming Variables 1.Direct experimental manipulations • Drug or biomedical intervention • Behavioral intervention or treatment • Focused experimental study of, e.g., specific stimuli • System-wide “treatment” (e.g., policy change, school-based) • Simple presence v. absence of the treatment or stimulus • Single v. multiple treatment doses • Type of treatment or stimulus • Structurethe IV vis-à-vis: Week 3; Experimental designs

33. Direct experimental manipulation • Hypothesis: words presented in a semantic context are recalled better than when presented randomly. Independent Variable Dependent Variable Experimental group Target words presented within complete sentences Word recognition task Control group Target words presented randomly Word recognition task Completely controlled by the experimenter Experimental manipulation same as Independent Var. Week 3; Experimental designs

34. Forming Variables 2.Indirect experimental manipulations • Experimental “induction” of a mood or state… • “Stage manage” a social event  • Induce mood via description of the experiment Stress require presentation in front of peers Depression  Write about worst mistake you ever made Stereotype threat  “This test reflects on your group” Anxiety  Stage a robbery or fight Happiness  Lottery winnings? Relaxation  Meditation E X A MP L E Week 3; Experimental designs

35. Forming Variables 2.Indirect experimental manipulations • Experimental “induction” of a mood or state… • “Stage manage” a social event  • Induce mood via description of the experiment • Do a manipulation check to see if you actually manipulated your Independent Variable • Self-report, standard assessment (e.g., of stress) • Observer rating Week 3; Experimental designs

36. Indirect experimental manipulation • Hypothesis: happiness enhances pain resistance. Independent Variable Experimental Manipulation Dependent Variable Imagine you won the lottery – what will you buy first? Happy state Cold Presser Task (ice bucket) Experimental group Control group …what will you need to buy this month? Normal / baseline state Cold Presser Task Directly controlled by experimenter Notdirectlycontrolled by experimenter Our induction of the Independent Variable (happiness) is indirect. Week 3; Experimental designs

37. Indirect experimental manipulation • Hypothesis: happiness enhances pain resistance. Independent Variable Experimental Manipulation Dependent Variable Imagine you won the lottery – what will you buy first? Happy state Cold Presser Task (ice bucket) Experimental group Control group …what will you need to buy this month? Normal / baseline state Cold Presser Task AManipulation Checktests whether the experimental manipulation actually induced the Independent Variable Week 3; Experimental designs

38. Quasi-independent variables 3.Create a quasi-Independent variable using a measured variable. • Categorize participants by measuring (not manipulating) something: • Scores over / under an established “cut point”, • e.g., over 4 depression symptoms on a standard scale. • Scores based on a frequency a distribution: • Median split: top v. bottom half. • Extreme scores: top v. bottom 10% of scores. • Simple self-identification: • e.g., “Republican” v. “Democrat”. • Behavioral index: • Used any drug in previous year v. not. • Voted in 2012 v. not. Not a “True” IV: Participants not randomly assigned to groups. Week 3; Experimental designs

39. Using a measured variable to create groups # of symptoms rated 2 or 3 • Administer depression scale, count the # symptoms rated 2 or 3. Form groups based on a cut point; e.g., > 4 symptoms = quasi-clinical depression. Participants are assigned to groups based on their ratings, not random assignment. Below is a list of different feelings. Circle the number that shows howmanydaysyou felt each of these over the PAST WEEK. Rarely or A Little A moderate Most or all of none of of the Time amount of the time the time the time (less than 1 day) (1 or 2 days) (3 - 4 days) (5 - 7 days) I was bothered by things that usually do 0 1 2 3 not bother me. I felt I could not shake off the blues even 0 1 2 3 with help from my friends or family. I had trouble keeping my mind on what 0 1 2 3 I was doing. I felt depressed. 0 1 2 3 I felt that everything I did was an effort. 0 1 2 3 My sleep was restless. 0 1 2 3 I was happy. 0 1 2 3 I enjoyed life. 0 1 2 3 I felt sad. 0 1 2 3 Week 3; Experimental designs

40. Question 4 An independent variable… A = Is measured on a continuous scale B = Is manipulated by the researcher C = Is the outcome of the experiment D = Is the “phenomenon” you are trying to explain. E = Does not care about other people Week 3; Experimental designs

41. Question 5 An dependent variable… A = Is typically measured on a binary scale B = Is manipulated by the researcher C = Is the putative cause in the theory D = Is the “phenomenon” you are trying to explain. E = Is over-concerned about other people Week 3; Experimental designs

42. The big picture: Introductory lectures • How do social values affect science? • Where do research questions & hypotheses come from. • Variables in research • Basic experimental designs • True vs. Quasi- Experiments Phenomenon Theory Hypothesis  Methods & data Results Discussion & Conclusions Week 3; Experimental designs

43. Overview: Basic Designs • “Pre-experimental” designs: no control group Experimental Observe1 Treatment Observe2 Post-Test Only Design Pre- Post- Test Design Group assignment Pre-test Experimental manipulation Outcome Week 3; Experimental designs

44. Basic Designs • “Pre-experimental” designs: no control group Experimental Observe1 Observe1 Treatment Observe2 Post-Test Only Design Pre- Post- Test Design True (or Quasi-)experimental designs with a control group “After only” Control group design Pre- Post- Group Comparisons Group assignment Pre-test Experimental manipulation Outcome Control Control Observe2

45. Basic Designs • “Pre-experimental” designs: no control group Post-Test Only Design Experimental Observe1 Treatment 1 Observe2 Pre- Post- Test Design Control Experimental Observe1 Treatment 2 Observe2 True (or Quasi-)experimental designs with a control group Control Observe1 Observe2 “After only” Control group design Pre- Post- Group Comparisons Multiple group comparison Group assignment Pre-test Experimental manipulation Outcome

46. “Pre-experimental” designs Post-Test Only Design Group Treatment Measure Only 1 group - typically an existing group: no selection or assignment occurs. Experimental intervention (“Treatment”) may or may not be controlled by the researcher. Use for naturally occurring or system-wide events (e.g., group trauma, government policy change, etc.). Measurement may or may not be controlled by the researcher. Pre- Post- Test Design Group Measure1 Treatment Measure1 Only one group; • only group available? • naturally occurring intervention? Measurements given to all participants at baseline & follow-up All participants get the same treatment, which may or may not be controlled by the researcher. Week 3; Experimental designs

47. “Pre-experimental” Designs (2) Advantage of “Post-” & “Pre- Post-” Designs: Allow us to study naturally occurring interventions. • e.g., test scores before and after some school change, • Crime rates after a policy change, etc. • Having both Pre- and Post measures allows us to examine change. Week 3; Experimental designs

48. “Pre-experimental” Designs (2) Disadvantage of “Post-” & “Pre- Post-” Designs: No control group = many threats to internal validity. Maturation: Participants may be older / wiser by the post-test History; Cultural or historical events may occur between pre- and post-test that change the participants Mortality: Participants may non-randomly drop out of the study Regression to baseline: Participants who are more extreme at baseline look less extreme over time as a statistical confound. Reactive Measurement:Scores may change simply due to being measured twice, not the experimental manipulation. Week 3; Experimental designs

49. Experiments “After only” Control group design Control Experimental Treatment 2 Observe2 Adds a control group. Either… Observed Groups: • Naturally occurring (e.g., Class 1. v. Class 2) or • Self-selected (sought therapy v. did not…). Assigned Groups: • Randomlyassign participants to experimental v. control group, or • Matchparticipants to create equivalent groups. Measure Dependent Variable(s) only at follow-up. Use experimental or standard measures (e.g., grades, census data, crime reports). Control Observe2 Week 3; Experimental designs

50. Advantages of experimental design “After only” Control group design Control Experimental Treatment 2 Observe2 Control Observe2 Advantage: Lessens the likelihood of confounds or threats to internal validity. • Control group • Random assignment Disadvantage: Existing or self-selected groups may have confounds. No baseline or pre- measure available: • We cannot assess change over time. • We cannot assess whether the groups are equivalent at baseline. Week 3; Experimental designs