1 / 48

RESEARCH DESIGN PHC 6700/RCS 6740 March 7, 2006

RESEARCH DESIGN PHC 6700/RCS 6740 March 7, 2006. RESEARCH DESIGN. Experimental Designs The specific research designs available to investigators can be divided into two basic types: group designs, and single-subject designs. RESEARCH DESIGN. A Typical Experimental Design

sawyer-eaton
Download Presentation

RESEARCH DESIGN PHC 6700/RCS 6740 March 7, 2006

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RESEARCH DESIGNPHC 6700/RCS 6740March 7, 2006

  2. RESEARCH DESIGN Experimental Designs The specific research designs available to investigators can be divided into two basic types: • group designs, and • single-subject designs.

  3. RESEARCH DESIGN A Typical Experimental Design Pretest-Posttest Control Group Design R O1 X O2 R O3 O4

  4. RESEARCH DESIGN Group Designs The group (multi-subject) designs all include one or more groups of subjects and are classified as either: • between-groups, • within-subjects, • or mixed.

  5. RESEARCH DESIGN Between-Groups Design Between-groups design is used to assess the effects of different levels of an independent variable by administering each level to a different group of subjects and then comparing the status or performance of the group on the dependent variable. • The simplest between-groups designs include a single independent variable with two levels. When using the design, the study includes two groups that each receives a different level of the IV.

  6. RESEARCH DESIGN Example: A psychologist assesses the effects of a “self-control” procedure by comparing the achievement of children who have been trained in the procedure (experimental group) with that of children who have not been trained in the procedure (control group).

  7. RESEARCH DESIGN The simple two-group design can be expanded in two ways. One way is to include more than two levels of a single independent variable. The psychologist in this study could compare three levels of the control procedure • a procedure that includes self-instruction only, • a procedure that includes self-instruction, self-monitoring, and self-reinforcement, and • no procedure. In this situation, the study would involve comparing the average academic achievement test scores of subjects in the three groups.

  8. RESEARCH DESIGN Another way to expand a two-group design is to include two or more independent variables. Whenever a study includes two or more independent variables, it is called a factorial design. The major advantage of a factorial design is that it provides more thorough information about the relationships among variables by allowing an investigator to analyze the main effects of each independent variable as well as the interaction between independent variables.

  9. RESEARCH DESIGN If the psychologist in the self-control study included initial symptom severity (mild, moderate, and severe) as a second independent variable, s/he would be able to determine if there are: • MAIN effects of the self-control procedure, • MAIN effects of initial symptom severity, and/or • an INTERACTION between self-control procedure and initial symptom severity.

  10. RESEARCH DESIGN Main Effect A main effect is the effect of one independent variable on the dependent variable, disregarding the effects of all other independent variables. An interaction refers to the effects of two or more independent variables considered together. • An interaction occurs when the effects of an independent variable differ at different levels of another independent variable.

  11. RESEARCH DESIGN Illustration: Assume the psychologist in the self-control study obtains a sample of 60 children and divides them into three groups on the basis of their initial symptom severity (mild, moderate, or severe). S/he then randomly assigns subjects in each group to either the experimental (self-control procedure) or control (no procedure) group so that there are 10 children in each of the study's now six groups (see table in next slide) • Although the data collected by the psychologist would have to be analyzed with an inferential statistical test to determine if there are significant main and/or interaction effects, tentative conclusions can be drawn by examining the data.

  12. Self-Control Procedure No Procedure Mild Symptoms 52 36 Moderate Symptoms 40 30 Severe Symptoms 34 36 RESEARCH DESIGN As an example, assume that the psychologist obtains the following mean achievement test scores for the six groups of children:

  13. RESEARCH DESIGN To determine if there are main effects of each IV, it is necessary to calculate the marginalmeans. For self-control procedure, the marginal means are 42 and 34. These means were obtained by adding the means in each column and dividing by 3 (the number of means): (52 + 40 + 34)/3 = 42 and (36 + 30 + 36)/3 = 34. • Because the marginal means are different, it is possible to tentatively conclude that there are main effects for the self-control procedure. • Overall, the self-control procedure seems to have beneficial effects on academic achievement test scores.

  14. RESEARCH DESIGN For initial symptom severity the marginal means are 44, 35, and 35. These means were obtained by adding the means in each row and dividing by 2 (the number of means): (52 + 36)/2 = 44; (40 + 30)/2 = 35; and (34 + 36)/2 = 35. • The marginal means indicate that there are also main effects for initial symptom severity. • Although children with moderate and severe symptoms obtained the same mean achievement test score (35), children with mild symptoms obtained a higher mean score (44). (If all three means were identical, there would be no main effects of symptom severity.) • This indicates that, overall, mild symptoms are associated with the highest achievement test scores.

  15. Self-Control Procedure No Procedure Mild Symptoms 52 36 Moderate Symptoms 40 30 Severe Symptoms 34 36 RESEARCH DESIGN

  16. RESEARCH DESIGN To determine if there is an interaction, the cell means are inspected. If there is an interaction, the effects of the self-control procedure will differ at different levels of symptom severity. • As can be seen in the previous table, for children with mild symptoms, the self-control procedure had a very positive effect: • Children who were trained in the procedure obtained a mean of 52, • while children who received no training obtained a mean of 36. • For children with moderate symptoms, the self-control procedure also had positive effects, but the gap between the two groups is somewhat smaller: • Children who received training obtained a mean of 40, • while children who received no training obtained a mean of 30. • Finally, for children with severe symptoms, the self-control procedure did not have a beneficial effect. • Children with severe symptoms obtained a higher achievement test score if they received no training (34 with training versus 36 with no training).

  17. RESEARCH DESIGN These results suggest that there is an interaction between the two IVs. The effects of the self-control procedure differ for different levels of initial symptom severity. (If the effects of the self-control procedure were the identical for all levels of symptom severity, there would be no interaction.)

  18. RESEARCH DESIGN An inspection of the data suggests that there are main effects for both variables as well as an interaction effect. • Note that the presence of the interaction invalidates the conclusion that was drawn on the basis of the main effects alone: • Overall, the self-control procedure seems to be beneficial (main effect), but a closer inspection of the data shows that this is true only for children with mild and moderate symptoms (interaction). • This illustrates the importance of interpreting the main effects (or lack of main effects) with caution whenever there is an interaction: • An interaction is likely to modify or invalidate the conclusions made on the basis of the main effects alone. • Finally, note that a study can have any combination of main and interaction effects. It is possible to have an interaction, for example, without any main effects (or vice versa).

  19. RESEARCH DESIGN Study Tip: Be sure you understand what main and interaction effects are on a conceptual level and be able to determine, from a table of data like the one given, whether it looks like there are main and/or interaction effects. Keep in mind that there can't be an interaction unless the study has at least two IVs and that, when there is an interaction, the main effects must be interpreted in light of the interaction.

  20. RESEARCH DESIGN Within-Subjects Designs When using a within-subjects (AKA: repeated measures) design, all levels of the independent variable are administered sequentially to all subjects. • Consequently, comparisons of different levels of the independent variable are made within subjects rather than between groups of subjects. • Like between-groups designs, within-subjects designs can include only two levels of a single independent variable or can be expanded to include three or more levels of a single IV and/or two or more IVs.

  21. RESEARCH DESIGN The single-group time-series design (AKA the simple interrupted time-series design) is one type of within-subjects design. • When using this design, the effects of a treatment are evaluated by measuring the dependent variable several times at regular intervals both before and after the treatment is applied. • This procedure allows subjects to act as their own no-treatment controls.

  22. RESEARCH DESIGN Example: A psychologist in a study assesses the effects of a low dose of phenothiazines on Brief Psychiatric Rating Scale (BPRS) scores by administering the BPRS to a single group of patients at one week intervals for two months before and two months after patients begin receiving a low dose of the drug.

  23. RESEARCH DESIGN A shortcoming of the single-group time-series design is that its internal validity can be threatened by history: • It's possible that an external event could occur at about the same time the independent variable is applied and account for any observed difference in pre- and posttest scores. • Note, however, that this design does help control maturation since maturational effects tend to occur gradually over time and can usually be detected in the overall pattern of pre- and posttest scores.

  24. RESEARCH DESIGN In another type of within-subjects design, two or more levels of an independent variable are applied sequentially to each subject, and the dependent variable is measured after each level has been applied. Example: A psychologist compares the effects of a low and high dose of phenothiazines on BPRS scores by giving the low dose of the drug to a group of patients for two months and administering the BPRS; and then giving the same patients the high dose of phenothiazines for two months and administering the BPRS again. The effects of the phenothiazines will be analyzed by comparing the BPRS scores obtained by the patients after each dose was administered.

  25. RESEARCH DESIGN A problem with this design is that it is susceptible to carryover effects (multiple treatment interference): • If the psychologist this study finds that the high dose of phenothiazines is more effective than the low dose for improving symptoms, this could be because the high dose followed a period of time during which patients received the low dose. • Counterbalancing can be used to control carryover effects. • Carryover effects could be controlled in the above study by dividing the patients into two groups and administering the two levels of phenothiazines in a different order to each group (i.e., low dose, high dose to patients in Group # l and high dose, low dose to patients in Group #2). If the high dose is more effective for both groups of patients, its effects would not be attributable to carryover effects.

  26. RESEARCH DESIGN Within-subjects designs have at least two advantages over between groups-designs. • First, they require a fewer number of subjects and, consequently, are more "economical." • Second, these designs help control individual differences that can threaten a study's internal validity since subjects are compared with themselves rather than with other subjects. As a result, within-subjects designs can actually be more powerful than between groups-designs (i.e., better able to detect the actual effects of the IV).

  27. RESEARCH DESIGN A disadvantage of the time-series and other within-subjects designs is that the analysis of the data can be confounded by autocorrelation (also known as serial dependency). • In other words, subjects' performance on the post-tests is likely to correlate with their performance on the pretests. • Autocorrelation can inflate the value of the inferential statistic (e.g., the t or F), thereby resulting in an increased probability of a Type I error. • For this reason, a number of experts recommend that special statistical techniques be used to analyze data collected in a study using this type of design.

  28. RESEARCH DESIGN Study Tip: Be sure you have this design linked with the concept of autocorrelation and know that autocorrelation can decrease power and inflate the chance of making a Type I error.

  29. RESEARCH DESIGN Mixed Designs A mixed ("split-plot") design combines between-groups and within-subjects methodologies. • Counterbalanced designs can be considered a type of mixed design because they permit comparisons both between groups and within subjects. • A design is also a mixed design when it includes two or more independent variables and at least one variable is a between-groups variable and another is a within-subjects variable.

  30. RESEARCH DESIGN Example: In the example study, the psychologist would be using a mixed design if therapy approach is treated as a between-groups variable (patients receive only one type of therapy), while phenothiazines is treated as a within-subjects variable (the placebo, low dose, and high dose are administered sequentially to each patient).

  31. RESEARCH DESIGN Mixed designs are common in research studies that involve measuring the dependent variable over time or across trials. In this type of study, time or trials is an additional IV and is considered a within-subjects variable because comparisons on the dependent variable will be made within subjects across time or across trials.

  32. RESEARCH DESIGN Example: In our example study, the psychologist decides to compare the effects of four levels of therapy (family therapy, individual therapy, a combination of the two, and no therapy) by assigning patients to one of the levels and measuring the short- and long-term effects of therapy by administering the BPRS at two-month intervals for 24 months after therapy begins. Because the study includes a between-groups variable (therapy) and a within-subjects variable (time), it is utilizing a mixed design.

  33. RESEARCH DESIGN Single-Subject Designs The single-subject designs were derived primarily from the work of behavioral psychologists, especially those engaged in applied behavioral analysis, which combines behavioral principles with the techniques of experimental psychology to solve socially-relevant problems. While the single-subject designs are often used to investigate the effects of an independent variable on the behavior of one subject or a small number of subjects, they can also be used with groups of subjects.

  34. RESEARCH DESIGN Several characteristics distinguish the single-subject designs from the group designs: • Each single-subject design includes at least one baseline (no treatment) phase and one treatment phase. As a result, each subject acts as his/her own no-treatment control. • The dependent variable is measured repeatedly at regular intervals throughout the baseline and treatment phases. Repeated measurement of the DV helps control any maturational effects that might otherwise threaten the study's internal validity by enabling an investigator to detect those effects in the pattern of performance on the DV measure.

  35. RESEARCH DESIGN There are a large number of single-subject designs, but the most commonly used are the AB design (and its extensions) and the multiple baseline design. • AB Design • Reversal (Withdrawal) Designs • Multiple Baseline Designs

  36. RESEARCH DESIGN AB Design The simplest single-subject design is the AB design, which includes a single baseline (A) phase and a single treatment (B) phase. As in all single-subject designs, the dependent variable is measured at regular intervals during both phases.

  37. A B H L Time RESEARCH DESIGN Example: A psychologist decides to assess the effects of a low dose of phenothiazines on the BPRS scores of one patient using the AB design. She administers the BPRS to the patient at two-week intervals for three months during the baseline (A) phase of the study and for three months during the treatment (B) phase. The design of this study and its possible results are illustrated below:

  38. RESEARCH DESIGN Because the patient's symptoms decreased only after he began receiving the phenothiazines, these results suggest that a low dose of phenothiazines has a positive effect on symptoms.

  39. RESEARCH DESIGN Reversal Designs (ABA, ABAB, Etc.) The AB design can be expanded to include more than one baseline phase or more than one baseline and more than one treatment phase. Because any expansion requires the withdrawal of the treatment during the second and subsequent baseline phases, the extensions of the AB design are called reversal (withdrawal) designs.

  40. RESEARCH DESIGN • Advantage of the reversal designs over the simple AB design: they provide additional control over potential threats to a study's internal validity. • When an ABAB design is used, if status on the DV returns to the initial baseline (no treatment) level during the second A phase and then to its previous treatment levels during the second B phase, an investigator can be more certain that any observed change in the dependent variable is actually due to the IV rather than to an historical event or other extraneous factor. • Repeating the study by adding another A and B phase is referred to as intrasubject replication.

  41. A A B B H H L L Time RESEARCH DESIGN Example: To confirm that the observed improvement in symptoms was due to the phenothiazines, the psychologist extends her study to include an additional baseline and treatment phase. This ABAB design and its possible results are illustrated below:

  42. RESEARCH DESIGN • Reversal designs are considered inappropriate when withdrawal of a treatment during the course of a research study would be unethical(e.g., when the treatment has successfully eliminated a self-injurious behavior). • Reversal design does not provide conclusive information if the effects of an independent variable persist rather than "reverse" (return to baseline levels) when it is withdrawn. • When this occurs, an investigator can't be certain whether an observed effect on the dependent variable is due to the independent variable or to other factors. • If a reversal design is inappropriate for ethical or practical reasons, an investigator might use a multiple baseline design.

  43. RESEARCH DESIGN Multiple Baseline Design • Multiple baseline design does not require withdrawal of a treatment during the course of the study but, instead, involves sequentially applying the treatment either to different behaviors of the same subject (multiple baseline across behaviors); to the same subject in different settings (multiple baseline across settings); or to the same behavior of different subjects (multiple baseline across subjects). • Once the treatment has been applied to a "baseline" (behavior, setting, or subject), it is not withdrawn from that baseline during the course of the study.

  44. RESEARCH DESIGN Example: To test the effects of the self-instructional component of the self-control procedure on attention span, a psychologist uses a multiple baseline design. S/he trains a child in self-instruction and then tells the child to use the technique when working on arithmetic homework in three different settings: first when working alone in a quiet room; then when working in the library; and then when working in the classroom. The psychologist measures the child's attention span in all three settings at regular intervals during the baseline and treatment phases.

  45. A B Quiet Room Library Time RESEARCH DESIGN This design and its possible results are illustrated below

  46. RESEARCH DESIGN Because the child's attention span in each setting did not increase until self-instruction was applied to it, these results confirm that self-instruction is useful for improving attention span. As can be seen in the above figure, sequentially applying an intervention to different settings helps determine if an intervention is actually responsible for any observed changes in the target behavior:

  47. RESEARCH DESIGN • If the behavior changes in a particular setting only after the intervention has been applied in that setting, an investigator can be more certain that the change is due to the intervention rather than to history or other factors. • To be effective, the setting, behaviors, or subjects chosen for inclusion in the study must be relatively independent. If they are not, it may not be possible to evaluate the effects of the IV with the multiple baseline design.

  48. HAPPY SPRING BREAK! 

More Related