Types of Research and Designs

1. Types of Research and Designs • This week and next week… • Covering • Research classifications • Variables • Steps in Experimental Research • Validity • Research Designs • Common Sources of Error

2. VALIDITY

3. VALIDITY • In Step 4, the validity of measurement for the DVs comes into question • There are two (2) important classification of validity in experimental research: • Internal validity • External validity

4. Internal Validity • How valid the findings are within, or internal to, the study. • Did the treatments cause the outcome to occur, or did other extraneous factors cause the outcome?

5. Internal Validity • Eight (8) factors that threat the internal validity of a study: • History • Maturation • Testing • Instrumentation • Statistical regression • Selection • Experimental mortality • Interaction of Selection and Maturation or History

6. History • Specific things that happen while conducting the research study that affect the final scores of the participants in addition to the effect of the experimental treatment. • The effects of linear vs. nonlinear periodization models for in-season volleyball players • Resistance training vs. aerobic training for physical fitness in middle-aged men.

7. Maturation • Subjects grow older during the course of an experiment – this may affect the outcome of the study • The time-course for training-induced increases in strength for adolescents during a 25-week resistance training program • The effects of 12 weeks of aerobic training on VO2max (from September to December)

8. Testing • The “learning” effect • Subjects get better at scoring well on a test the more times they take it. • The effects of 2 days of resistance training on muscle strength and power output.

9. Instrumentation • Changes in the adjustment or calibration of the measuring equipment or use of different standards among scorers may cause a difference in outcome scores.

10. Statistical Regression • The tendency for groups with extremely high or low scores on one measure to score closer to the mean score of the population on a second measure. • Can be mistaken for a treatment effect • Example: Low-IQ and high-IQ groups are formed. A treatment is applied to both groups for twelve weeks. A physical ability test is administered to both groups. It is found that both groups are equal in physical ability.

11. Selection • The selection of subjects and their assignments to groups can bias the outcome of a study. • Subjects that are not representative of the population • Kinesiology majors vs. all college-aged males and females • Groups that are not equal in their abilities prior to the experiment • VO2max and muscle strength

12. Experimental Mortality • Subject attrition • Excessive loss of subjects throughout the course of a study • Sample of subjects is no longer representative of the population

13. Interaction of Selection and Maturation or History • The maturation effect or history effect is not the same for all groups selected for the research study – may affect the outcome. • Groups that differ in age or background may respond differently to the same treatment • If a group starts out unequal in ability due to maturation or history, they may not all have the same potential for improvement or change. • Disproportional experimental mortality

14. External Validity • The degree to which findings in a research study can be inferred or generalized to other populations, settings, or experimental treatments. • Can the findings of a particular study be inferred to the entire population? • Are the findings unique to the participants of the study, or do they apply to other groups? • Examples: A study using prison inmates. Good internal validity.

15. External Validity • Four threats to external validity • Interaction effect of testing • Interaction effects of selection bias and experimental treatment • Reactive effects of experimental setting • Multiple-treatment interference

16. Interaction Effect of Testing • This effect occurs when the pre-test itself changes the group’s response to the treatment. • Example: A pre-test may identify deficiencies that the subject may try to correct during the course of the treatement

17. Interaction Effects of Selection Bias and Experimental Treatment • The participant or groups selected in a biased manner respond to the experimental treatment in a unique way so they are not representative of any population. • Novice vs. intermediate vs. elite weight lifters • This threat generally occurs as a result of a convenient group is tested and an attempt is made at generalizing the results

18. Reactive Effects of Experimental Settings • The experimental setting is not the same as a subject’s ambient environment – this may affect the study outcome. • Subjects may respond differently around investigators • Laboratory vs. natural settings • Subjects may think that a particular treatment is supposed to do “something” and respond like they are “supposed to.”

19. Multiple-Treatment Interference • Multiple treatments administered at the same time on the same group of subjects. • Subject involved in two studies at the same time • Competing influences of treatments

20. Control of Variables • Control of all variables operating in an experimental study is highly desirable, but seldom (if ever) accomplished. • Sleep, food, exercise, etc. • Too much control can harm external validity • Not enough control can harm internal validity

21. Validity in Summary • In many cases, internal and external validity cannot be obtained. • Must decide which is most important • Good internal validity is necessary for external validity • Internal validity is the basic minimum (LCD) for an experimental design • Investigators must consider the threats to validity when designing research studies.