Finishing up

1. Finishing up Psych 231: Research Methods in Psychology

2. This is the final new content lecture • Thursday (optional) – Course overview, Review, Q&A, etc. • Turn in extra-credit journal summaries • Make sure that your SONA credits are assigned where you want them • Labs this week • Poster presentations • Turn in group ratings sheets • Turn in the GP: results and discussion sections Announcements

3. Sometimes you just can’t perform a fully controlled experiment • Because of the issue of interest • Limited resources (not enough subjects, observations are too costly, etc). • Surveys • Correlational • Quasi-Experiments • Developmental designs • Small-N designs • This does NOT imply that they are bad designs • Just remember the advantages and disadvantages of each Going to finish up after the stats lectures Non-Experimental designs

4. What are quasi-experiments? • Almost “true” experiments, but with an inherent confounding variable • Typically, they lack random assignment for one of the independent variables • General types • An event occurs that the experimenter doesn’t manipulate or have control over • Interested in subject variables • Time is used as a variable • Commonly used with • Developmental Designs • Non-equivalent control group designs • In Program Evaluations Quasi-experiments

5. Independent Variable Dependent Variable Dependent Variable Non-Random Assignment Experimental group Measure Measure participants Control group Measure Measure • Nonequivalent control group designs • with pretest and posttest (most common) (think back to the second control lecture) • But remember that the results may be compromised because of the nonequivalent control group (review threats to internal validity) Quasi-experiments

6. Program evaluation • Systematic research on programs that is conducted to evaluate their effectiveness and efficiency. • e.g., does abstinence from sex program work in schools • Steps in program evaluation • Needs assessment - is there a problem? • Program theory assessment - does program address the needs? • Process evaluation - does it reach the target population? Is it being run correctly? • Outcome evaluation - are the intended outcomes being realized? • Efficiency assessment- was it “worth” it? The the benefits worth the costs? Quasi-experiments

7. Advantages • Allows applied research when experiments not possible • Threats to internal validity can be assessed (sometimes) • Disadvantages • Threats to internal validity may exist • Designs are more complex than traditional experiments • Statistical analysis can be difficult • Most statistical analyses assume randomness Quasi-experiments

8. Sometimes you just can’t perform a fully controlled experiment • Because of the issue of interest • Limited resources (not enough subjects, observations are too costly, etc). • Surveys • Correlational • Quasi-Experiments • Developmental designs • Small-N designs • This does NOT imply that they are bad designs • Just remember the advantages and disadvantages of each Non-Experimental designs

9. What are they? • In contrast to Large N-designs (comparing aggregated performance of large groups of participants) • One or a few participants • Data are typically not analyzed statistically; rather rely on visual interpretation of the data • Historically, these were the typical kind of design used until 1920’s when there was a shift to using larger sample sizes • Even today, in some sub-areas, using small N designs is common place • (e.g., psychophysics, clinical settings, animal studies, expertise, etc.) Small N designs

10. = observation Steady state (baseline) Treatment introduced • Baseline experiments – the basic idea is to show: • Observations begin in the absence of treatment (BASELINE) • Essentially our control/comparison level • Then treatment is implemented and changes in frequency, magnitude, or intensity of behavior are recorded Small N designs

11. = observation • Baseline experiments – the basic idea is to show: • When the IV occurs, you get the effect • When the IV doesn’t occur, you don’t get the effect (reversibility) • This allows other comparisons, to the original baseline as well as to the transition steady state Transition steady state Reversibility Steady state (baseline) Treatment introduced Treatment removed Small N designs

12. Before introducing treatment (IV), baseline needs to be stable • Measure level and trend • Level – how frequent (how intense) is behavior? • Are all the data points high or low? • Trend – does behavior seem to increase (or decrease) • Are data points “flat” or on a slope? Unstable Stable Small N designs

13. ABA design (baseline, treatment, baseline) Steady state (baseline) Transition steady state Reversibility • The reversibility is necessary, otherwise something else may have caused the effect other than the IV (e.g., history, maturation, etc.) • There are other designs as well (e.g., ABAB see figure13.6 in your textbook) ABA design

14. Advantages • Focus on individual performance, not fooled by group averaging effects • Focus is on big effects (small effects typically can’t be seen without using large groups) • Avoid some ethical problems – e.g., with non-treatments • Allows to look at unusual (and rare) types of subjects (e.g., case studies of amnesics, experts vs. novices) • Often used to supplement large N studies, with more observations on fewer subjects Small N designs

15. Disadvantages • Difficult to determine how generalizable the effects are • Effects may be small relative to variability of situation so NEED more observation • Some effects are by definition between subjects • Treatment leads to a lasting change, so you don’t get reversals Small N designs

16. Small vs. Large N debate • Some researchers have argued that Small N designs are the best way to go. • The goal of psychology is to describe behavior of an individual • Looking at data collapsed over groups “looks” in the wrong place • Need to look at the data at the level of the individual Small N designs

17. Course Review: The Research Process

18. Slide from Day 1 Course Review: The Research Process

19. Presenting your work • A set of skills leading to knowledge & understanding • A way of thinking (beware small samples, correlation is not causation, etc.) • A way of life? Get an idea Stats.org: Stats in the news Course Review: The Research Process

20. Get an idea • Often the hardest part • No firm rules for how to do this • Observations • Past research • Review the literature The Research Process

21. Review the literature • What has already been done? • What variables have people looked at • What hasn’t been looked at • How are other experiments in the area done? • What methods are used? • To measure the dependent variable • To manipulate the independent variable • To control extraneous variables The Research Process

22. Formulate a testable hypothesis • What is a hypothesis? • A predicted relationship between variables • What does it mean to be testable? • Must be falsifiable • Can it be replicated • Must be able to observe/measure (and manipulate for experiments) the variables • Directly • Indirectly • Operational definitions The Research Process

23. Design the research • What method? • Experiment, Survey, Developmental designs, … • What kind of comparisons are used • Control groups • Baseline conditions • What are your variables? • How many levels of your Independent variable(s) • How do you measure your dependent variable(s) • What can be done to control for biases and confounds? The Research Process

24. Collect Data • Importance of pilot research • Who do you test? • What is your population? • Your sample? • Your sampling method? The Research Process

25. Analyze the data • Design drives the statistics • Understanding Variables and variability • Descriptive statistics (summarizing) • Means, standard deviations • Graphs, tables • Correlation • Inferential statistics (drawing conclusions) • What kind of analysis is appropriate for your design • T-tests • ANOVA • Between or within versions The Research Process

26. Interpret the results • Correlation versus causation • Reject or fail to reject null hypotheses • Statistical vs. theoretical significance • Support or refute the theory (or revise) • Generalizability of the results The Research Process

27. Present the results • Getting the research “out there” • Conference presentations • Posters • Talks • Written reports • APA style • Supports clarity The Research Process

28. Repeat • Each set of results leads to more research questions • Refine the theory • Test a refined theory • Test alternative explanations The Research Process

29. Wed@ 1:00P • It is cumulative, covers the entire course. The majority is on new material (roughly 65%), the rest is material covered on Exams 1 & 2. • All multiple choice/scantron for the final Reviewing for the final exam

30. Final 1/3 of the course • Non experimental methods • Survey, correlational, & developmental • Statistics • Descriptive • Inferential • Presentations • Papers, Posters, & Talks Reviewing for the final exam

31. First 2/3 of the course • Scientific method • Getting ideas • Developing (good) theories • Reviewing the literature • Psychological Science • Ethics • Basic methodologies • APA style • Underlying reasons for the organization • Parts of a manuscript • Variables • Sampling • Control • Experimental Designs • Vocabulary • Single factor designs • Between & Within • Factorial designs Reviewing for the final exam