Research Methods Review

1. Research Methods Review Advanced Cognitive Psychology PSY 421 - Fall, 2004

2. Overview • The Basics • Experimental Methods • Designs • Within-subjects • Between-subjects • Factorial • Statistical Review • Example of Experiment • Don’t let me forget a simple assignment to give you before you leave

3. What are Research Methods? • Application of the scientific method to studying behavior • Scientific Method = observe, hypothesize/predict, test, conclude • Work with structured observations • Deal with solvable problems • Produce publicly verifiable information • Hypotheses, theories, and explanations MUST BE FALSIFIABLE

4. Watch people in public settings and drawing conclusions = Observational/Descriptive Conducting a survey, questionnaire, interview, poll, etc. and comparing responses on one question to other questions = Relational/Correlational Randomly forming 2 or more groups, treat them all differently, see how the outcomes differ = Experimental Compare two or more groups (that are not randomly formed and are different to begin with) in a variety of ways = Quasi-Experimental General Methods – The Basics

5. Experimental Method • This type of method attempts to answer questions about CAUSE • Variables • Independent = manipulated or changed for various subjects; IV • Dependent = the way to measure a change in behavior (or not change); DV • Control = a potential IV that is held constant at one level – all subjects are exposed to that one level • Manipulation = change

6. Manipulating Variables - Designs • One variable – multiple levels • Univariate or one-way design • Level = one aspect of the variable; condition • Example • Manipulation Types • Between Subjects = ONE • To expose some subjects to one level of the IV and other subjects to another level of the IV – subjects are not exposed to all aspects of the variable • Within Subjects = ALL • To expose all subjects to all the levels of the IV

7. The BOX • Boxes with rows and columns – essential for understanding experiment design and statistics Variable A Level 1 Level 2 Row 1 Level 1 Variable B Row 2 Level 2 Column 1 Column 2

8. Factorial Designs • When more than one variable is manipulated in an experiment • 2 Variables = Two way • 3 or more variables = multivariate • Between subjects design = all variables are manipulated between/across the levels • Within subjects design = all subjects receive all the levels of all the variables • Mixed design = at least one variable is manipulated between subjects and at least one variable is manipulated within subjects

9. Statistical Review • Populations and Samples • Hypothesis Testing • Using methods and statistical tests together

10. Population vs. Sample • Population = everyone that you are interested in studying or at the very least, generalizing your results to • Ex: Women; children; Men over 40 • You can’t possibly hope to study the entire population • Sample = a subset of the population that contains the important characteristics of the population; a sample is representative of its population • Various techniques for sampling from a population • Ex: PSU Women; children at BFC; Male faculty in psychology over 40 • Why does this matter? • Allows research to occur without the impossible task of studying everyone • Important assumptions for statistics

11. Hypothesis Testing • Comparing the Null and Experimental hypotheses to predict the likelihood of one being show to be true • Null Hypothesis = There is no difference; nothing will change; zero • Experimental Hypothesis = There will be a difference; something will change; non-zero

12. Hypothesis Testing • Null Hypothesis There is no difference in GRE scores between males and females • Experimental Hypothesis There is a difference in GRE scores between males and females

13. Hypothesis Testing Example Barbie and Kendall – Chocolate Eaters (to be read in class) • From this, what could we conclude about this contradiction? • According to our assumption, in the real world, H0 is true. Therefore, if Barbie rejected H0 because she thought it was wrong (based on her study's results), what has happened? Did she commit an error or make a correct decision? • If Barbie would have retained H0 (and to do that, her study would have resulted in no differences between the mean exam scores from the two groups), and we assume that in the real world, H0 is true, did she commit an error or make a correct decision? • If we now assume that Kendall is wrong and in the real world, H0 is false, and Barbie rejected H0because she thought it was wrong (based on her study's results), what has happened? Did she commit an error or make a correct decision? • Again, assume that Kendall is wrong, and H0 is false in the real world. If Barbie would have retained H0 (and to do that, her study would have resulted in no differences between the mean exam scores from the two groups), and we assume that in the real world, H0 is true, did she commit an error or make a correct decision?

14. Hypothesis Testing

15. Scales of Measurement • Nominal = used to identify a particular characteristics of the scale; also called categorical (categories are mutually exclusive) • EX: Sex (M/F); ZIP Codes • Ordinal = numbers indicate whether there is more or less of the measured variable; order is important • EX: Levels of education (Freshman, Sophomore, Junior, Senior); Olympic medals • Interval = numbers correspond exactly to changes in the measured variable and there are equal distances between numbers that correspond to equal changes in the measured variable • EX: IQ; Temperature (Fahrenheit, Celcius) • Ratio = like an interval scale (equal intervals) but also includes a true zero point (the absence of the measured variable). This allows for multiplication and division of scale values. • EX: Weight; Height; Temperature (degrees Kelvin)

16. Descriptive Statistics Decide how to summarize and represent data based on the TYPE of data that you have (its scale of measurement) • Measures of Central Tendency • Nominal Scale – Mode • Ordinal Scale – Median • Interval/Ratio Scales - Mean • Measures of Dispersion • Nominal Scale – Range • Ordinal Scale – Absolute Deviation from the Median • Interval/Ratio Scales – Variance, Standard Error, Standard Deviation • Graphical Representations of Data • Nominal/Ordinal Scales (Qualitative Data) – Bar Graph, Pie Chart • Interval/Ratio Scales (Quantitative Data) – Line Graph, Frequency Polygon, Histogram

17. Inferential Statistics Decide which test to use based on the TYPE of data you have and the KIND of outcome you are looking for • Relationships (Correlations) • Nominal Scale – Chi-Square Test of Independence, or Phi • Ordinal Scale – Kendall’s Tau or Spearman’s r • Interval/Ratio Scales – Pearson’s r

18. Differences • One Independent Variable • Between-Subjects manipulation • 2 levels • Nonparametric – Chi-Square Goodness of Fit (nominal) and Mann Whitney U (ordinal) • Parametric – Independent means t-test or one-way ANOVA • 3+ levels • Nonparametric – Chi-Square Goodness of Fit (nominal) and Kruskal Wallace (ordinal) • Parametric – One-way ANOVA • Within-Subjects manipulation • 2 levels • Nonparametric – Chi-Square Goodness of Fit (nominal) and Mann Whitney U (ordinal) • Parametric – Independent means t-test or one-way ANOVA • 3+ levels • Nonparametric – no good tests • Parametric – One way Repeated-Measures ANOVA

19. Differences, continued • Two Independent Variables • Between-Subjects manipulation • Nonparametric – Wilcoxon-Wilcox and Friedman tests • Parametric – Factorial ANOVA • Within-Subjects manipulation • Nonparametric – Friedman test • Parametric – Factorial Repeated-Measures ANOVA • Mixed manipulation – Mixed Factorial ANOVA

20. Putting this all together… • To study behavior, we have to create conditions that are controlled enough to be able to predict an outcome in the controlled conditions • We have to think about how to study the behavior of interest and how to make it change in a predictable fashion • Experimental methodology allows researchers to control the sample and expose the participants to changes that are predicted to influence the outcome • When you construct a particular experiment or use a particular research method, a certain logic applies when choosing the “proper” statistical method to analyze the results/outcome

21. Experiment Example • This will be shown in class • Signal Detection Experiment • IV – Presence of Target • Levels: Present or Absent • Manipulation: Within-Subjects • Measure/DV: Hits, False Alarms, Correct Rejections, Misses