Experimental Research Designs, Part 2

1. Experimental Research Designs, Part 2 Research Process and Design Spring 2006 Class 7

2. Today’s objectives • To answer any questions you have • To further explore quasi-experimental and single-subject designs • To talk about writing the methods section • To apply knowledge to your group project Research Process and Design (Umbach)

3. Characteristics of Experimental Research • Statistical equivalence of subject in different groups (usually by random assignment) • Comparison of 2 or more groups • Manipulation of at least 1 IV • Measurement of DVs • Inferential statistics • Maximum control of extraneous variables Research Process and Design (Umbach)

4. Experimental Research—Strengths and Limitations • Strengths • The only designs capable of examining causal relationships • High degree of control of extraneous variables • Direct manipulation of an independent variable • Weaknesses • High levels of control needed for assuring internal validity mean very restrictive manipulated conditions which in turn imply low external validity • High levels of external validity mean little control and thus low internal validity • Not possible in many situations Research Process and Design (Umbach)

5. Quasi-Experimental Designs • Good designs that are frequently used in education • Lack of control related to randomly assigning subjects to groups (i.e., the use of pre-existing groups) • “Non-equivalent groups” terminology reflects the lack of random assignment and the concern about the selection internal validity threat Research Process and Design (Umbach)

6. Quasi-Experimental Designs • Basic designs and several variations of them • Non-equivalent groups designs • Non-equivalent groups pretest-posttest experimental-control • Non-equivalent groups pretest-posttest experimental-comparison • Non-equivalent groups pretest-posttest experimental-control-comparison • Statistical analyses focus on the comparison of groups and the use of appropriate analyses like ANCOVA Research Process and Design (Umbach)

7. Quasi-Experimental Designs • Time series designs: groups repeatedly measured before and after treatments • Single group interrupted time series (e.g., A OOOOOO X OOOOOOO) • Experimental-control group interrupted time series • Variations of these designs include the use of comparison groups and different treatment times • Statistical analyses examine the patterns of performance over time Research Process and Design (Umbach)

8. Factorial Designs - Definitions • Main effect – the effect of each independent variable on the dependent variable • Interaction effect – the effect of the interaction of two or more independent variables on the dependent variable • Factors – independent variables (e.g., two-factor design means two independent variables being studied) • Levels – independent variables vary, often represented in groups (e.g., high, medium, low) Research Process and Design (Umbach)

9. How many variables? How many IVs? How many groups? • 2 x 2 • 3 x 2 • 3 x 2 x 2 • 2 x 2 x 2 x 2 Research Process and Design (Umbach)

10. Single-Subject Designs • The use of one subject studied over time • Typically one subject, but there are situations where two or three subjects are sampled • Focus on populations where there is a low incidence of specific characteristics (e.g., severe and profound special education students, physically-challenged students, etc.) • Need to draw causal inferences Research Process and Design (Umbach)

11. Characteristics of Single-Subject Designs • Reliable measurement • Repeated measurement • Explicit descriptions of conditions • Baseline and treatment condition: duration and stability • Manipulation of a single variable at one time Research Process and Design (Umbach)

12. Single-Subject Designs • Three types of designs • A-B designs: baseline and treatment • A-B-A or A-B-A-B designs: baseline, treatment, baseline (i.e., reversal or withdraw designs) • Multiple baseline designs Research Process and Design (Umbach)

13. Single-Subject Designs—Multiple Baseline • Use of the A-B logic with the addition of two or more actions, subjects, or situations • Used when it is impossible or undesirable to remove a treatment condition • Types • Multiple baselines across behaviors (i.e., the use of a single treatment across several distinct behaviors) • Multiple baselines across situations (i.e., observation of a single target behavior observed in two or more settings) • Multiple baselines across individuals (i.e., two or more subjects receiving treatment where behavior and the situation are held constant) • Statistical analyses examine the patterns of behavior Research Process and Design (Umbach)

14. Group Activity • Design a study using 2 of the research designs described in today’s or last week’s class. • What are the implications of each design? • What are the pros and cons of each design? Research Process and Design (Umbach)

15. The method section of a quantitative study • Vary somewhat but generally have the following components • Participants • Variables • Instrumentation/materials • Procedures • Data analysis Research Process and Design (Umbach)

16. More specifically, methods section should: • Identify research design • Describe sample, population, and participants • Describe instruments, variables, and materials • Review procedures in detail • Explain plans for preliminary studies or pilot studies (if necessary) • Provide details about data analysis procedures • Identify ethical issues and your approach to dealing with them • Describe limitations or threats to validity Research Process and Design (Umbach)

17. Remember, you need to give enough information so that the study can be replicated Research Process and Design (Umbach)

18. Group project • Begin thinking about the elements of your method section • What design will you employ? • Who do you plan to study? • What and how will you collect data? • What are your independent and dependent variables? Research Process and Design (Umbach)

19. For next week… • Non-experimental research design: Correlational and ex post facto studies • Readings: • Jaeger – Chapter 4 (p. 61-76) • * Wiersma, W. & Jurs, S. G. Chapter 7 • Distribute take-home exam Reminder: Outline of literature review due Research Process and Design (Umbach)