1 / 19

Quasi-Experimental Design

Quasi-Experimental Design. Jung Eun (Jessie) Hong Feb. 23, 2009. Outlines. Experimental Design Definition Process A Key Point Types of Experimental Designs Quasi-Experimental Design Designs Strengths Weaknesses Examples in Geography Ongoing Debate. Definition of Experimental Design.

Download Presentation

Quasi-Experimental Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Quasi-Experimental Design Jung Eun (Jessie) Hong Feb. 23, 2009

  2. Outlines • Experimental Design • Definition • Process • A Key Point • Types of Experimental Designs • Quasi-Experimental Design • Designs • Strengths • Weaknesses • Examples in Geography • Ongoing Debate

  3. Definition of Experimental Design • A structured, organized method • To determine whether some program or treatment causes some outcome or outcomes to occur. • If X, then Y • Because there may be lots of reasons, other than the program, for why you observed the outcome, • If not X, then not Y needs to be addressed, too

  4. Process of Experimental Design • To show that there is a casual relationship, • Two “equivalent” groups • The program or treatment group gets the program • The comparison or control group does not • The groups are treated the same in all other respects • Differences in outcomes between two groups must be due to “the program”

  5. A Key Point of Experimental Design • How do we create two groups that are “equivalent”? • Assign people randomly from a common pool of people into the two groups • The experiment relies on the idea of “random assignment” to obtain two similar groups. • A key to the success of the experiment • Assume that two groups are “probabilistically equivalent”

  6. Types of Designs Is random assignment used? No Yes Randomized or True experiment Is there a control group or multiple measures? No Yes Non-experiment Quasi-experiment

  7. Quasi-Experimental Design • Similar to the experimental design, but lacks the key ingredient, “random assignment” • Easily and more frequently implemented • Extensively used in the social sciences • A useful method for measuring social variables • Two classic quasi-experimental designs • The Nonequivalent Groups Design • The Regression-Discontinuity Design

  8. The Nonequivalent Groups Design • The most frequently used in social research • Try to select groups that are as similar as possible to compare the treated one with the comparison one • e.g. two comparable classrooms or schools • Cannot be sure whether the groups are comparable • The groups may be different prior to the study • Any prior differences between the groups may affect the outcome of the study • Require a pretest and posttest

  9. The Regression-Discontinuity Design • A useful method for determining whether a program of treatment is effective • Participants are assigned to program or comparison groups based on a cutoff score on a pretest • e.g. Evaluating new learning method to children who obtained low scores at the previous test. • Cutoff score = 50 • The treatment group: children who obtained 0 to 50 • The comparison group: children who obtained 51 to 100 • The program (treatment) can be given to those most in need

  10. The Regression-Discontinuity Design With no treatment effect With Ten point treatment effect

  11. The Regression-Discontinuity Design Discontinuity

  12. Strengths of Quasi-Experimental Design • Useful in generating results for general trends in social sciences • Difficult pre-selection and randomization of groups • Easily integrated with individual case studies • Generated results can reinforce the findings in a case study • Allow statistical analysis to take place • Enable to reduce the time and resources required for experimentation • Not required extensive pre-screening and randomization

  13. Weaknesses of Quasi-Experimental Design • Without proper randomization, statistical tests can be meaningless • Do not explain any pre-existing factors and influences outside of the experiment • The researcher needs to control additional factors that may have affected the results • Some form of pre-testing or random selection may be necessary to explain statistical results thoroughly

  14. Quasi-experiments vs. Non-experiments to address similar questions • Both designs are applicable when the subjects are not able to be randomized • Some variables cannot ethically be randomized • e.g. Studying the effect of maternal alcohol use when the mother is pregnant

  15. Example of Quasi-Experimental Design in Geography • Baker and White (2003) • The Effects of GIS on Students’ Attitudes, Self-efficacy, and Achievement in Middle School Science Classrooms • Conducted the Nonequivalent quasi-experimental design • Two eighth grade teachers, across ten classrooms • Total 192 eighth grade students participated • Treatment group: used a Web-based GIS application • Control group: used paper maps

  16. Example of Quasi-Experimental Design in Geography • Impossible to randomly assign each student to a GIS or paper mapping conditions • Randomly assigned whole classes to two conditions • Different instructors affected the results differently • Instructor effect played a substantial role in student attitudes and self-efficacy

  17. Ongoing Debate • Whether true experiments or quasi-experiments represents the superior design • Supporters of true experiments • Difficult to isolate the program effects using quasi-experiments • Quasi-experimental results are biased and sensitive to minor changes • Not sure about whether quasi-experimental designs can adequately control selection bias • Hard to determine better design • True experiments are impossible and impractical in some cases

  18. Any questions???

More Related