Quasi Experiments Non-Experimental Research

1. Quasi ExperimentsNon-Experimental Research Research Methods & Statistics Fall 2011

2. Quasi Experiments • “Research procedure in which the scientist must select subjects for different conditions from preexisting groups” • Research Methods, McBurney & White • “An empirical study used to estimate the causal impact of an intervention on its target population. Quasi-experimental research designs share many similarities with the traditional experimental design or randomized controlled trial, but they specifically lack the element of random assignment to treatment or control. Instead, quasi-experimental designs typically allow the researcher to control the assignment to the treatment condition, but using some criterion other than random assignment. • Wikipedia, Nov. 17, 2011

3. It’s a matter of control True Experiment Quasi Experiment Selection of subjects for the conditions Observe categories of subjects If the subject variable is the IV, it’s a quasi experiment Don’t know whether differences are caused by the IV or differences in the subjects • Random assignment of subjects to condition • Manipulate the IV • Control allows ruling out of alternative hypotheses

4. Other features • In some instances cannot completely control the what, when, where, and how • Need to collect data at a certain time or not at all • Practical limitations to data collection, experimental protocol

5. Validity • Internal validity is reduced due to the presence of controlled/confounded variables • But not necessarily invalid • It’s important for the researcher to evaluate the likelihood that there are alternative hypotheses for observed differences • Need to convince self and audience of the validity

6. External validity • If the experimental setting more closely replicates the setting of interest, external validity can be higher than a true experiment run in a controlled lab setting • Often comes down to what is most important for the research question • Control or ecological validity?

7. Nonequivalent Control Group Designs • 2 groups, non-random allocation of subjects and groups, pre-test, treatment (Y/N), post-test • Desired pattern: • Dependent variables have equal pre-test values, difference seen between experimental and control groups on post-test (Pre: LL, Post: HL) • Want to show that any differences that exist did not impact the value of the variable of interest

8. Exercise • Interpret the following graphs (experimental, control): • Pre: (L, H) Post: (M, H) • Pre: (L, M) Post: (H, ML) • Pre: (L, M) Post: (H, M)

9. Mixed Factorial Design with One Non-manipulated Variable • Example: experiment on pain perception (effect of caffeine, expected differences between men and women) • Protocol: • 25 men/25 women, each takes part in two sessions, one week apart • One session: drink coffee (decaf) and put hand in ice-water until feel pain • Other session: drink coffee (caffeinated) and put hand in ice-water until feel pain • Between subjects variable (male/female) • Within subjects variable (caffeine intake)

10. Non-Experimental Research Read and understand: • Gabriella Belli - Nonexperimental Quantitative Resarch • “Any quantitative study without manipulation of treatments or random assignment is a non-experimental study” • Experimental research shows cause and effect • Non-experimental research studies variables as they exist

11. Purpose • Descriptive: • primary focus for the research is to describe some phenomenon or to document its characteristics. • Predictive: • primary focus for the research is to predict some variable of interest (criterion) using information from other variables (predictors). • Explanatory : • the primary focus for the research is to explain how some phenomenon works or why it operates.

12. Time Frame • Cross-sectional: • data are collected at one point in time, often in order to make comparisons across different types of respondents or participants. • Prospective (longitudinal): • data are collected on multiple occasions starting with the present and going into the future for comparisons across time. • Retrospective: • look back in time using existing or available data to explain or explore an existing occurrence.

13. Techniques • Surveys • http://www.socialresearchmethods.net/kb/survey.php • Interviews • http://www.socialresearchmethods.net/kb/intrview.php • Observations • http://interactionarchitect.com/knowledge/article19991212shd.htm • http://www.edu.plymouth.ac.uk/resined/observation/obshome.htm • Unobtrusive methods: • http://www.socialresearchmethods.net/kb/unobtrus.php