1 / 19

Introduction to Research Design

Introduction to Research Design. External Validity Simple Research Designs Extraneous Variable Control. External Validity. The extent to which the results of the study can generalize beyond the specifics of the experimental situation. Would you get the same results if you

cree
Download Presentation

Introduction to Research Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to Research Design External Validity Simple Research Designs Extraneous Variable Control

  2. External Validity The extent to which the results of the study can generalize beyond the specifics of the experimental situation. • Would you get the same results if you • used different subjects • manipulated the IV in a different way • measured the DV in a different way • The setting was different • etc.

  3. Threats to External Validity:Testing x Treatment Pretest Postest O  X  O Control Group Design O     O • Is the effect of the treatment the same for pretested subjects as for subjects not pretested? • Can use a Solomon Four Group Design • Or a Posttest Only Control Group Design

  4. Threats to External Validity Selection x Treatment : Does the treatment interact with characteristics of the subjects? • Can you generalize your results to different sorts of subjects? • from laboratory rats to feral rats? • from laboratory rats to humans? • from US college students to those in China? • from college students to farmers?

  5. Threats to External Validity:Reactive Effects of Experimental Arrangements Can you generalize the results found with observed subjects to subjects who are not being observed? • Difficult or impossible to observe without affecting. • Heisenberg uncertainty principle. • Demand characteristics.

  6. Threats to External Validity:Multiple Treatment Interference Do results found with subjects used in many experiments generalize to subjects without multiple treatment experience? • Expensive monkeys used in medical research. • Trained human subjects used in perceptual research.

  7. Simple Research Designs:One-Shot Case Study (X)  O • No variable is manipulated. • Just find group that has experienced X. • Example: Post tornado research here. • No pretest data, no comparison group. • Of little if any scientific value.

  8. Randomized Pretest-Posttest Control Group Design R  O  X  O R  O     O • Random assignment to groups – can assume groups equivalent prior to treatment. • Thus, no selection threat and • No Selection x Maturation threat.

  9. Randomized Pretest-Posttest Control Group Design R  O  X  O R  O     O • Effects of history, maturation, testing, instrumentation can be assessed by pre-post comparison in control group. • They are controlled if they exist to the same extent in both treatment and control groups.

  10. Randomized Pretest-Posttest Control Group Design R  O  X  O R  O     O • Testing x Treatment remains a threat to external validity. • Problems with random assignment • Compare the two pretest means • The Elmira College Study

  11. Randomized Pretest-Posttest Control Group Design R  O  X  O R  O     O • Factorial ANOVA: Time x Group • Mixed between subjects and within subjects • Primary interest is in the Time x Group interaction. • Independent t comparing the two groups’ change scores (equivalent to interaction analysis)

  12. Randomized Pretest-Posttest Control Group Design R  O  X  O R  O     O • Analysis of Covariance • pretest scores as covariate • posttest scores as dependent variable • All three of these have more power than independent t on posttest scores.

  13. Randomized Posttest OnlyControl Group Design R  X  O R     O • For controlling threats previously discussed, this is the strongest design. • But it usually has less power than designs that include within subjects comparisons. • This is a threat to statistical conclusion validity.

  14. Extraneous Variable Control • Important to control extraneous variables to • Reduce noise • eliminate confounds • I’ll discuss five methods of controlling extraneous variables.

  15. Constancy • Hold values of EVs constant across subjects and groups. • for example, use only female participants • test all subjects at same time of day, etc. • Eliminates confounds. • Reduces noises, raising power. • May reduce external validity • do results generalize to men, etc?

  16. Balancing • Distribution of EV constant across groups • if 60% female in one group, same in others • if 70% tested during day in one group, same in others • Prevents confounding. • Does not reduce noise.

  17. Randomization Assumed to produce balancing, thus • Confounds are eliminated. • Noise is not reduced.

  18. Matching • Subjects matched up on EV(s) into blocks of k, where k = # of treatments. • Within each block, randomly assign one subject to each treatment. • Matched pairs, randomized blocks, split-plot. • Balances groups, eliminating confounds. • Allows power-enhancing statistical analysis.

  19. Statistical Control • Statistical techniques, such as ANCOV and ANOVA, can be used to remove the effect of extraneous variables and thus increase power. • I’ll discuss these in more detail in the next unit of this class.

More Related