Research Study Design and Statistical Methods for Cardiology

1. Research Study Design and Statistical Methods for Cardiology Nathan D. Wong, PhD, FACC Professor and Director Heart Disease Prevention Program Division of Cardiology University of California, Irvine

2. Why are papers rejected for publication? (The Top 11 Reasons) • The study did not address an important scientific issue • The study was not original • The study did not actually test the authors’ hypothesis • A different type of study should have been done • Practical difficulties led the authors to compromise on the original study protocol (e.g., recruitment, procedures) Greenhalgh T, BMJ 1997; 15: 243-6

3. Reasons 6-11 for Paper Rejection • The sample size was too small • The study was uncontrolled or inadequately controlled • The statistical analysis was incorrect or inappropriate • The authors drew unjustified conclusions from the data • There is a significant conflict of interest among authors • The paper is so badly written that it is incomprehensible

4. Outline • Elements of Designing a Research Protocol • Selecting a Study Design – Which is best for answering your question? • Selection and Classification of Study Variables (e.g., predictors and outcomes) • Sample size and power considerations • Choice of statistical procedures for different study designs

5. Nine Key Elements of a Research Study Protocol • Background • Hypotheses • Clinical Relevance • Specific Aims / Objectives • Methodology • Power / Sample Size • Measures and Outcomes • Data Management • Statistical Methodology (UCI-SOM Dean’s Scientific Review Committee: http://www.rgs.uci.edu/ora/rp/hrpp/deansscientificreview.htm)

6. Background • A brief review of the problem to be studied and of related studies that generated the rationale and the central idea of the proposed study. Several pertinent references should be provided.

7. Was the study original? • Few studies break entirely new ground • Many studies add to the evidence base of earlier studies which may have had other or more limitations • Meta-analyses depend on literature containing multiple studies addressing a question in a similar manner

8. Features Distinguishing New vs. Previous Studies • Is the study in question bigger in sample size, or with longer-follow-up (e.g., adding to meta-analyses of previous studies)? • Is methodology more rigorous (e.g., having addressed criticisms of previous ones)? • Is the population studied different from that of previous studies (ages, gender, ethnic groups)? • Does the new study address a clinical issue of sufficient importance so it is politically desirable even if not scientifically necessary? Greenhalgh T, BMJ 1997; 315: 305-8

9. Hypotheses • The problem/s stated in the Background may generate a primary hypothesis and possibly one or two secondary hypotheses. • A hypothesis is often stated in the null – e.g., "No difference between treatments A and B" is anticipated, or "No association between X and Y exists". • Alternatively, it can be stated according to what one expects e.g., “A will be more effective than B in reducing levels or symptoms of C", or “X will be associated with Y".

10. Clinical Relevance • In the case of clinical studies, the potential value in the understanding, diagnosis, or management of a clinical condition or pathological state should be stated.

11. Specific Aims / Objectives • This states what the study is intended to study or demonstrate and generally includes mention of predictor and outcome (or endpoint) variables. • For example: "The primary aim of the study is to examine whether treatment A is more effective than treatment B in reducing levels of C", or "in finding out whether X is associated with Y", etc. • There may be several specific aims in a given study. The methods of study should address each of them.

12. Elements of a Formulated Question • Patient or Population: Who is the question about? (e.g., pts with diabetes mellitus) • Intervention or Exposure: What is being done or what is happening to the patient/population? (e.g., tight control) • Outcome(s): How does the intervention affect the patient/population (mortality, CHD incidence) • Comparison(s): What could be done instead of the intervention? (e.g., standard management)

13. Methodology • Methodology should validate or not validate the hypothesis and specific aims using procedures consistent with sound scientific study design including: • the size and nature of the subjects studied • recruitment, screening, and enrollment procedures • inclusion and exclusion criteria • treatment schedules, and follow-up procedures, if applicable. A chart of the studies to be performed at each visit and the time of each visit and test is needed.

14. Study Population Issues • How were the subjects recruited? Is there potential recruitment bias (e.g., from taking respondents of advertisements), or is survey done in a random (e.g., random digit-dialing) or consecutive sample? • Who was included? Many trials exclude those who have co-morbidities, do not speak English, or take other medications—may provide scientifically clean results, but may not be representative of disease in question.

15. Study Population (cont.) • Who was excluded? Study may exclude those with more severe forms of disease, therefore limiting generalizibility • Were subjects studied in “real-life” circumstances? Is the consenting process describing the benefits/risks, access to study staff, equipment available, etc. be similar to that in an ordinary practice situation?

16. Power / Sample Size • A power/sample size analysis should include an estimate of minimum effect or difference expected at a given level of power when the sample size is fixed, or a projection of the number of subjects needed to achieve a clinically important difference in what is being examined in the hypotheses and the specific aims.

17. Measures and Outcomes • Measures include both independent (predictor) and dependent (outcome) variables. • Outcomes include what the investigator is trying to predict, e.g., new or recurrent onset of a disease state, survival, or lowering of cholesterol as a result of a drug. • The independent or predictor variables should always include treatment status (e.g., active vs. placebo) in the case of a clinical trial, or primary variables of interest (such as age, gender, levels of X at baseline) for other studies. In either case, there will often be possible cofounders or covariates to adjust for in the analysis of the results. • The measures and outcomes are reasonably expected to answer the proposed question and the importance of the knowledge expected to result from the research.

18. Data Management • Data Management includes how data is captured for analysis and the tools that will be utilized while capturing the data. This includes: • Case report forms for clinical trials • Surveys, questionnaires, or interview instruments • Computerized spreadsheets or entry forms • Methods for data entry, error checking, and maintenance of study databases

19. Statistical Methods of Analysis • Statistical analysis includes a description of the statistical tests planned to perform to examine the results obtained, e.g., • Student’s t-test will be used to compare levels of A and B between treatment and placebo groups • Multiple logistic regression analysis will be used to examine an independent treatment effect on the likelihood of recurrent disease.

20. Hierarchy of Evidence (for making decisions about clinical interventions or proving causation) • Systematic reviews and meta-analyses • Randomized controlled trials with definitive and clinically significant effects • Randomized controlled trials with non-definitive results • Cohort studies • Case-control studies • Cross-sectional surveys • Case reports

21. Features Affecting Strength and Generalizability of Study • sample size • selection of comparison group (control or placebo) • selection of study sample (is it representative of population the study results are intended to apply to?) • length of time of follow-up • outcome assessed (e.g., hard vs. soft or surrogate endpoint) • Measurement and ability to control for potential confounders

22. Case Reports and Series • Provides “anectdotal” evidence about a treatment or adverse reaction • Often with significant detail not available in other study designs • May generate hypotheses, help in designing a clinical trial. • Several reports forming a “case series” can help establish efficacy of a drug, or thru adverse reports, cause its demise (example: Cerivastatin fatal cases of rhabdomyolysis).

23. Observational Studies • Cross-sectional, prospective, and case-control studies seldom can identify two groups of subjects (exposed vs. unexposed or cases vs. controls) that are similar (e.g., in demographic or other risk factors). • Much of the controlling for baseline and/or follow-up differences in subject characteristics occurs in the analysis stage (e.g., multivariable analysis as in Framingham)

24. Observational Studies (cont.) • While statistical procedures may be done correctly, have we considered all possible confounders? • Some covariates may not have been measured as accurately as possible, and more often, may not be even known or measured.

25. Observational, cross-sectional • Examines association between two factors (e.g, an exposure and a disease state) assessed at a single point in time, or when temporal relation is unknown • Example: Prevalence of a known condition, association of risk factors with prevalent disease. • Conclusions: Associations found may suggest hypotheses to be further tested, but are far from conclusive in proving causation

26. Cross-Sectional Studies and Surveys • Examples: NHANES III, CHIS (telephone), chart-review studies • Surveys should include a representative, ideally randomly-chosen (rather than a small sample of approached subjects who actually agree to be surveyed) sample. • Data collected cannot assume any directionality in exposure / disease. • Can statistically adjust for confounders, but difficult to establish the temporal nature of exposure and disease.

27. Prevalence of CHD by the Metabolic Syndrome and Diabetes in the NHANES Population Age 50+ 19.2% 13.9% CHD Prevalence 8.7% 7.5% No MS/No DM MS/No DM DM/No MS DM/MS % of Population = 28.7% 2.3% 14.8% 54.2% Alexander CM et al. Diabetes 2003;52:1210-1214..

28. Odds of CVD Stratified by CRP Levels in U.S. Persons (Malik and Wong et al., Diabetes Care, 2005) *** Odds Rat io *** * * ** • *p<.05, **p<.01, **** p<.0001 compared to no disease, low CRP • CRP categories: >3 mg/l (High) and <3 mg/L (Low) • age, gender, and risk-factor adjusted logistic regression (n=6497)

29. Metabolic Syndrome Independently Associated with Inducible Ischemia from SPECT (Wong ND et al., Diabetes Care 2005; 28: 1445-50 ) *Estimates adjusted for age, gender, cholesterol and smoking. Odds of ischemia for metabolic abnormalities (yes vs. no) (separate model): 1.98 (1.20-3.98), p=0.008

30. Prospective (Cohort) Studies • Cohort studies begin with identification of a population, assessment of exposure (e.g., lipid or BP levels) • Follow-up to the occurrence of outcomes (CHD events)-- temporal sequence to events is known

31. Cohort Studies (cont.) • Difficult to ascertain effect of exposure because of many differences between exposed and unexposed groups (confounding factors). • Statistical adjustment for known risk factor differences can help, but unknown factors that may differ between exposed and unexposed groups will never be adjusted for.

32. Duration of Follow-up • Is the planned follow-up reasonable and practical for the study question and sample size utilized? • effect of a new painkiller on degree of pain relief may only require 48 hours • effect of a cholesterol medication on mortality may require 5 years)

33. Prospective cohort studies • Examples: • Framingham Heart Study • Cardiovascular Health Study (CHS) • Multiethnic Study of Atherosclerosis (MESA) • Nurses Health Study • Advantages: • large sample size • ability to follow persons from healthy to diseased states • temporal relation between risk factor measures and development of disease

34. Prospective Studies (cont.) • Disadvantages: • expensive due to large sample size often needed to accrue enough events • many years to development of disease • possible attrition • causal inference not definitive as difficult to consider all potential confounders

35. Prospective Cohort Example: Framingham Heart Study • Longest running epidemiologic study • Began with 5209 persons aged 30-62 at baseline in 1948, studied biennially to date (most are deceased now) • Risk factors measured at each examination, some began later (e.g., HDL-C around 1970) or done only at certain exams (echocardiography, CRP) • Event ascertainment/adjudication involves panel of 3 physicians reviewing medical records

36. Low HDL-C Levels Increase CHD Risk Even When Total-C Is Normal (Framingham) 12.50 11.91 11.91 14 9.05 10.7 11.24 12 6.6 10 5.53 3.83 6.56 8 14-y incidence rates (%) for CHD 4.85 6 4.67  260 2.06 4.15 3.77 4 2.78 230–259 2 200–229 Total-C (mg/dL) 0 < 200 < 40 40–49 50–59  60 HDL-C (mg/dL) Risk of CHD by HDL-C and Total-C levels; aged 48–83 y Castelli WP et al. JAMA 1986;256:2835–2838

37. 4-Year Progression To Hypertension: The Framingham Heart Study Participants age 36 and older (<120/80 mm Hg) (130/85 mm Hg) (130-139/85-89 mm Hg) Vasan, et al. Lancet 2001;358:1682-86

38. CHD, CVD, and Total Mortality: US Men and Women Ages 30-74(age, gender, and risk-factor adjusted Cox regression) NHANES II Follow-Up (n=6255)(Malik and Wong, et al., Circulation 2004; 110: 1245-1250) *** *** *** *** *** *** *** *** *** * ** * p<.05, ** p<.01, **** p<.0001 compared to none

39. CV Event-Free 8-year Survival Using Combined hs-CRP and LDL-C Measurements (n=27,939) Median LDL 124 mg/dl Median CRP 1.5mg/l 1.00 Low CRP-low LDL 0.99 Low CRP-high LDL 0.98 Probability of Event-free Survival High CRP-low LDL 0.97 0.96 High CRP-high LDL 0.00 4 0 2 6 8 Years of Follow-up Ridker et al, N Engl J Med. 2002;347:1157-1165.

40. Case-control Studies • Most frequent type of epidemiologic study, can be carried out in a shorter time and require a smaller sample size, so are less expensive • Only practical approach for identifying risk factors for rare diseases (where follow-up of a large sample for occurrence of the condition would be impractical) • Selection of appropriately matched control group (e.g., hospital vs. healthy community controls) and consideration of possible confounders crucial • Relies on historical information to obtain exposure status (and information on confounders)

41. Case-Control Studies (cont.) • Cannot determine for sure whether exposure preceded development of disease • Also difficult to identify all differences between cases and controls that can be statistically adjusted for

42. Example of case-control study: Folate and B6 intake and risk of MI (Tavani et al. Eur J Clin Nutr 2004) • Cases were 507 patients with a first episode of nonfatal AMI, and controls were 478 patients admitted to hospital for acute conditions • Information was collected by interviewer-administered questionnaires • Compared to patients in the lowest tertile of intake, the ORs for those in the highest tertile were 0.56 (95% CI 0.35-0.88) for folate and 0.34 (95% CI 0.19-0.60) for vitamin B6. • Author conclusion: A high intake of folates, vitamin B6 and their combination is inversely associated with AMI risk

43. Potential sources of bias and error in case control studies • Information on the potential risk factor or confounding variables may not be available from records or subjects’ memories • Cases may search for a cause of their disease and be more likely to report an exposure than controls (recall bias) • Uncertainty as to whether agent caused disease or whether occurrence of the disease caused the person to be exposed to the agent • Difficulty in assembling a case group representative of all cases, and/or assembling an appropriate control group

44. Prospective, observational: nested case-control • In this design, one takes incident cases (e.g., incident CVD) and a matched set of controls to examine the association of a risk factor measured sometime before development of the outcome of interest • Less costly than a true prospective design where all subjects are included in analysis; may not provide equivalent estimates

45. Prospective study of CRP and risk of future CVD events among apparently healthy women (Ridker et al., Circulation 1998) – a nested case control study • 122 female pts who suffered a first CVD event and 244 age and smoking-matched controls free of CVD • Logistic regression estimated relative risks and 95% CI’s, adjusted for BMI, diabetes, HTN, hypercholesterolemia, exercise, family hx, and trt • Those who developed CVD events had higher baseline CRP than controls; those in the highest quartile of CRP had a 4.8-fold (4.1 adjusted) increased risk of any vascular event. For MI or stroke, RR=7.3 (5.5 adjusted)

46. hs-CRP Adds to Predictive Value of TC:HDL Ratio in Determining Risk of First MI Relative Risk hs-CRP Total Cholesterol:HDL Ratio Ridker et al, Circulation. 1998;97:2007–2011.

47. Examples where observational studies have taken us down the wrong path…… • Meta-analysis of observational studies have shown a 50% lower risk of CHD among estrogen users vs. non-users (which may have had many unknown differences that were not adjusted for), but recently randomized trials (HERS, WHI) show no benefit • Numerous prospective studies show a 25-50% lower risk of CHD among those taking vitamin E and other antoxidants vs. placebo– recent randomized trials (e.g., HOPE, HPS) show no benefit.

48. Randomized Clinical Trial • Considered the gold standard in proving causation– e.g., by “reducing” putative risk factor of interest • Randomization “equalizes” known and unknown confounders/covariates so that results can be attributed to treatment with reasonable confidence • Inclusion and exclusion criteria can often be strict (to maximize success of trial) and may require screening numerous patients for each patient randomized

49. Randomized Clinical Trials (2) • Expensive, labor intensive, attrition from loss to follow-up or poor compliance can jeopardize results, esp. if more than outcome difference between groups • Conditions are highly controlled and may not reflect clinical practice or the real world • Funding source of study and commercial interests of investigators can raise questions about conclusions of study

50. Randomized Controlled Trials (3) • Randomized controlled trial eliminates systematic bias (in theory) by allocating treatments among participants in a random fashion • The allocation process eliminates selection bias in group characteristics (check comparability of baseline characteristics such as age, gender, severity of disease and covariate risk factors) (selection bias)