1 / 21

Field and Quasi-experiments

Field and Quasi-experiments. Review of main effects/interactions sheets. Field experiments. NOT quasi-experiments NJ Negative Income Tax Experiment ( hsty ) What are potential problems with field experiments ? How can these be encouraged? Evidence-based practice Rubin’s causal model.

urania
Download Presentation

Field and Quasi-experiments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Field and Quasi-experiments

  2. Review of main effects/interactions sheets

  3. Field experiments • NOT quasi-experiments • NJ Negative Income Tax Experiment (hsty) • What are potential problems with field experiments? • How can these be encouraged? • Evidence-based practice • Rubin’s causal model

  4. ITT vs. TOT • What are they, and which should you care about? • Vietnam draft example (instrumental variable and potential outcomes solution) • What is the TOT vs. ITT question? • Problem of comparing those who served vs. not • 35.4% of draft eligible served • 19.4% not eligible served • So 15.9% due to random assignment • 2.04% draft elig vs. 1.95% not died • So (2.04-1.95)/15.9 = .56%

  5. Other field exp problems • Attrition—how can this be decreased? • Nested designs—how can these be dealt with? • How can you improve power in nested designs? • Intraclasscorrelations • Ethical issues?

  6. Why use quasi-experiments? • When are they better?

  7. What is the threat to IV? • You are interested in the stability of people’s attitudes toward campus safety. You give out a questionnaire, then plan to follow up the questionnaire two weeks later. In the meantime, a rape occurs on campus. What threat to internal validity is happening here? How could you prevent it?  • 2. You are comparing a new technique for teaching mathematics to the traditional approach in two elementary school classrooms. The first teacher volunteered to use the new technique, while the other teacher is teaching math as usual. What is the internal validity threat? How could you prevent it? •  3. Your survey of personality and aggressive tendencies is 300 questions long and takes participants a couple of hours to complete. What is the threat to internal validity? How could you prevent it? • 4. In an observation study, the observers code behaviors for hours at a time. What is the threat? How could it be prevented? • 5. You give participants a self-esteem questionnaire, then choose people who are especially low or high in self-esteem to participate in your experiment. In the experiment, you try to modify their self-esteem with test feedback. What is the threat? How could you get around it?

  8. Threats to internal validity • Selection • History • Maturation • Testing • Instrumentation • Mortality • Regression • Social interaction threats

  9. Example #1 • You want to study the effects of 24 hour visitation in the dorms of a college campus on grades. One all-female dorm voted for 24 hour visitation while another did not. You compare the grades of people living in each of these dorms the semester before and the semester after the visitation change. What kind of study have you done? What are some possible threats to internal validity? How could you correct them?

  10. Example #2

  11. Example #3 • A researcher studying developmental changes in intelligence conducted the following cross-sectional study. The Wechsler Adult Intelligence Scale was given to 20 people at each of the following ages: 20, 30, 40, 50, 60, and 70. These are the results: • Age IQ • 20 120 • 30 110 • 40 100 • 50 95 • 60 93 • 70 89 • From these results the researcher concluded that intelligence peaks in the early years and declines steadily thereafter. Do you agree? Why or why not?

  12. Nonequivalent Control Group Design • What is it? • Example • What makes a good control group? • Focal local • Matching • Propensity score analysis • Advantages and disadvantages • Threats to internal validity

  13. Interrupted time series • What is it? • Example • Advantages and disadvantages • Threats to internal validity • ABAB designs (and ABA, ABACBCBC ,etc.) • Pooled time series/multilevel models

  14. Regression Discontinuity Design • What is it? • Example • Advantages and disadvantages • Threats to internal validity • Fuzzy RDD

  15. Pattern matching • What is it? • Example • Advantages and disadvantages • Threats to internal validity

  16. Other quasi-exp designs • Proxy pretest • Separate pre-post samples • Double pretest design • Switching replications • Nonequivalent dependent variables • Regression point displacement

  17. How else can you reduce threats to IV? • Give examples: • By argument • By measurement or observation • By design • By analysis • By preventative action

  18. How to expand a basic design • Across time • Across programs • Across measurements • Across groups • How does Trochim suggest you build a design?

  19. Good design elements • Theory-grounded • Situational • Feasible • Redundant • Efficient

  20. Other design issues • Why is theory important? • How objective is research and how important is objectivity? • How can you know that the treatment was delivered effectively?

  21. Next week • Go straight through to get out by 2:20 to go to talk • Comment on readings AND • Find and evaluate a meta-analysis on a topic that interests you • Rough drafts due post-spring break

More Related