Quantitative Research

1. Quantitative Research

2. Overview

3. Observational • Definition - Directly observing naturally occurring behavior unobtrusively, typically in the field, but can also take place in laboratory settings • Pros • Goal is unobtrusive observation so that your presence does not alter the participant’s affect, behavior, cognition • Allows for continuous measurement • Avoid participant self report error • Other • Construct Validity – depends on usage • External Validity – Excellent b/c naturally occurring • Bias – coding bias • Error – coding errors

4. Observational • Choose “Observational” if… • interested in naturally occurring behavior • interested in studying a phenomenon as it is naturally occurring • interested in gathering information about naturally occurring boundary conditions or moderators • interested in depth/breadth of continuous measurement

5. Survey • Definition • Using self-report measures in any type of collection method ---► in-person, online, telephone, and mail • Pros • Relatively easy to collect data • Multiple collection methods for larger and representative sample sizes • Other • Construct – depends on usage • External – excellent because large sample sizes and representative samples • Bias – social desirability bias (except online) • Error – participant self-report error

6. Survey • Choose “Survey” if… • your topic can be analyzed using self-report • interested in collecting a lot of data – many variables and questions • interesting in collecting a lot of data – many subjects, power • interested in representative sample and generalizability • have little money, time, resources (except for mail/telephone if using Survey Company)

7. Archival • Definition - Using previously collected materials to analyze new research questions by using quantitative (numbers) analysis • Pros • No longer restricted only to present-day people and events so access to larger sets of data • Unobtrusive so reduced chances of experimenter error • Other • Construct – Insensitive measures since not collected for purpose of your study, No control over how information collected so possibly flawed • External – depends on inclusion criteria • Bias – coding bias • Error – coding error

8. Archival • Choose “Archival” if… • Same as with “Historical”, such as • interested in origins and growth • interested in particular historical events • no current data on point so look to past data • want to “generalize” from past events to current or future events • research question can only be answered by previously collected data • Plus… • have resources like coders, time • want to minimize experimenter bias • want to synthesize and compare data quantitatively

9. Meta-analysis • Definition - A meta-analysis statistically combines the results of several studies that address a shared research hypotheses • Pros • Central tendency - whether X affects Y, is the effect significant, and how strong is that effect? • Variability - If there is heterogeneity, then look for moderating variables that explain the variability. Does the effect of X on Y differ with moderator? • Other • Construct – Depends on CV of included works • External – Depends on EV of included works • Bias – Inclusion/Exclusion bias, Interpretation bias • Errors – Inclusion/Exclusion error

10. Meta-analysis • Choose “Meta-analysis” if… • Same as with “Literature Review”, such as • have an argument that can be supported by published research • interested in “summarizing” the literature for a variety of reasons • interested in in “interpreting” the literature, for a variety of reasons • interested in communicating the “quality” of the literature, for a variety of reasons • Plus… • have resources like coders, time • want to minimize experimenter bias • want to synthesize and compare data quantitatively

11. Meta-analysis • What are those “reasons”? • Same as “Literature Review”, such as • No one has previously summarized and/or interpreted the primary articles • The literature has grown to the point that it necessitates guidance or direction • There is a new topic that cross-cuts many previous literatures so new Literature Review needed to synthesize disparate literature relevant to new topic • There are controversies or disagreements that need resolution or support from summarizing/interpreting the literature • Plus… • Interested in “overall effect” of literature • There are new/old “moderators” that you want to test and/or can be tested across studies

12. Experiment • Definition • Testing cause-and-effect relationships by: (1) random assignment of Ss (2) to two or more conditions (3) which differ in terms of (only) IVs • Pros • Can prove causation • Tight controls • Other • Construct – depends on usage • External – experiments are artificial; alternative is conduct field study but then problem is loss of control and influence of extraneous variables • Internal – see the information from the previous PowerPoint slides about internal validity • Bias – experimenter bias • Error – experimenter error

13. Experiment • Choose “Experiment” if… • want to prove causation

14. Quasi-Experiment • Definition • Contains aspects of both experiments and non-experiments because deficient in at least one of the three aspects of experimental designs • Pros • Depends on which aspects of experiment and which aspects of non-experiment are involved in the quasi-experiment • Other • Construct – depends on… • External – depends on… • Bias – depends on… • Error – depends on…

15. Quasi-Experiment • Choose “Quasi-Experiment” if… • want scientific rigor of experiments but can’t satisfy all three requirements for variety of reasons (see next PowerPoint presentation about all the types of quasi-experiments such as hybrid, matched-pairs, within-subjects, mixed-designs, and single-n studies

16. Advanced Sources • Observational Participant Observation: A Methodology for Human Studies, by Jorgensen, Sage Publications • Survey Chapter 9 (Survey Research) of The Handbook of Research Methods in Social and Personality Psychology, Edited by Reis and Judd • Archival Archival Strategies and Techniques, by Hill • Meta-analysis Practical Meta-analysis, by Lipsey and Wilson • Experiment and Quasi-experiment Experimental And Quasi-experimental Designs For Research, by Campbell Experimental and Quasi-Experimental Designs for Generalized Causal Inference, by Shadish et al