An Area of the Dwelling: Cause and Effect in Research Designs

1. An Area of the Dwelling: Cause and Effect in Research Designs Michael Hendryx, PhD Department of Community Medicine West Virginia University

3. Knowing Cause and Effect • Experimental design • Non-experimental • Quasi-experimental • Natural experiments

4. Experiments • Well-suited to studying causal relationships • Control of the setting and variables to know that: • Cause precedes effect • Cause is related to effect • No other plausible explanations for the effect

5. Limits of Experiments • Can’t address non-manipulable causes • (sex, age, environmental toxins) • Including some issues of great importance • Sacrifice external for internal validity

6. Non-experiments • May be observational or correlational • Many non-experimental studies are seriously flawed • Correlation versus causation • Lack of control or comparison group

7. Quasi-Experiments • An experiment in which units are not assigned to conditions randomly • To “approach the dwelling” • Accumulation of evidence that rules out alternatives, i.e., “no other plausible explanation.” • Not only accumulation of facts, but logic and reason and (dare I say) verifiable intuition • Can approach convincing knowledge of matters of importance when experiments are impossible

8. Quasi-Experiments • Self-selection; no random assignment • But cause, or treatment, can still be manipulated • Dependent measures may be controllable • So: • Cause precedes effect? (Yes) • Cause is related to effect? (Maybe) • No other plausible explanations for the effect? (More difficult to establish)

9. Examples of non-random assignment • Smoking prevention program in some schools and not others • Medicaid enrollees self-select for new program

10. One group pretest posttest design

11. Control group pretest posttest design: Scenario 1

12. Control group pretest posttest design: Scenario 2

13. Ruling out alternatives • Being able to falsify a causal claim • Ruling out threats due to: • Statistical artifact • Power, unreliability, type I error • History • Maturation • Selection • Testing • Instrumentation • Attrition • Statistical regression

14. Natural Experiments • A naturally occurring contrast between a treatment and comparison condition • Still need to: • Accumulate facts • Rule out alternatives • Theory based predictions that are refutable and compelling • E.g., Einstein’s prediction of the perihelion of Mercury • We should be so lucky…

15. Case Study • Does pollution from coal mining cause harm to population health? • Mortality rates • A natural experiment • Can we accumulate facts • Rule out alternatives • Use theory, logic, and verifiable intuition

16. What is the Evidence? • Cause precedes effect? • (Yes, to a degree: duration of residence, migration) • Cause is related to effect? (Yes) • No other plausible explanations for the effect? • Age, poverty, smoking, health care, etc.

17. Use the Whitman Model: Accumulating Facts

18. Adjusted Risk Related to Amount of Coal for: • Lung cancer mortality • Chronic lung disease mortality • Chronic heart disease mortality • Kidney failure mortality • Self-reported health status and chronic heart, lung and kidney disease

19. Mortality Risk Not Related to Amount of Coal for: • Acute conditions (eg AMI) • But AMI does correlate to poverty and smoking • Infections (e.g. pneumonia) • Diabetes • Colon cancer • Motor vehicle accidents • Suicide or homicide

20. Using the Whitman Model: Ruling Out Alternatives • Statistical adjustment for age, poverty, smoking, education, rural setting, insurance, etc.

22. Threats to Validity • Internal validity • History • Selection • Attrition • Statistical conclusion validity • Unreliability of treatment implementation • Extraneous variance • But these make detecting effects more difficult

23. The Whitman Model: Using logic and intuition • Effects for lung cancer, chronic lung disease, chronic heart disease, kidney disease • Consistent with expected explanations • Activities of the coal mining industry • Risk increases with tonnage • No effects for acute heart disease, colorectal cancer, pneumonia, diabetes, accidents • If coal is not a cause, but socioeconomic or demographic variables are, then shouldn’t these be higher too?

24. Facts and Truth • Facts can help you approach the truth, but by themselves cannot reveal the truth • Need theory, logic, intuition

25. “My tongue, every atom of my blood, form'd from this soil, this air, Born here of parents born here from parents the same, and their parents the same, I, now thirty-seven years old in perfect health begin, Hoping to cease not till death.”