Sample Size Consideration in Clinical Research

1. Sample Size Consideration in Clinical Research John Kwagyan, PhD jkwagyan@howard.edu Howard University College of Medicine GHUCCTS

2. What Is Statistics? The science of collecting, organizing, analyzing, and interpreting data to assist in making effective decisions.

3. What is Statistics? • The science of collecting, organizing, analyzing and interpreting data to assist in making effective decisions. • Summarization of large quantities of data • (Descriptive/Summary Statistics) • Making decision from sample to population • (Inferential Statistics)

4. Type of Statistics • Descriptive/Summary Statistics Methods for organizing, summarizing, and presenting data in an informative way. • Inferential Statistics Methods for estimation and testingpopulationparameters?? based on sampleinformation.

5. Population Well defined Large Unique Characteristics -prevalence of a disease -variability of a measure -Response rate of therapy -etc We are interested in estimating the population characteristics!!!

6. POPULATION SAMPLE sample data We make inference about population characteristics based on sample data

7. Population Parameters • Mean cholesterol level of obese individuals • Prevalence of hypertensionin Blacks • Incidence of lung cancer among smokers • Risk of liver disease (hepatitis) associated with drinking • Mortality rate of heart attach among men • Variabilityof heart rate in PTSD CENTRAL IDEA: Estimate and Test for differences in parameters

8. Case Example • Suppose that we plan to conduct a study comparing a treatment with a control. • The response variable is systolic blood pressure (SBP), measured using a standard sphygmomanometer. • The treatment is supposed to reduce blood pressure • We set up a one-sided test H0 : μT = μC versus H1 : μT <μC where μT = mean SBP for the Trt group. • The parameter Δ = μT −μCis the effect being tested

9. Case Example • Suppose the goals of the study specify that we want to be able to detect a situation where the treatment mean is 15 mmHg lower than the control group. • The required effect size is Δ= −15. • We specify that such an effect be detected with 80% power (1-β= .80) when the significance level α = .05. • Past experience with similar study-with similar sphygmomanometers and similar subjects-suggests that the data will be approximately normally distributed with a standard deviation of SD =20 mmHg. • We plan to use a two-sample pooled t test with equal numbers n of subjects in each group.

10. Case Example • Now we have all of the specifications needed for determining sample size using the power approach, and their values may be entered in suitable formulas, charts, or power-analysis software. • We find that a sample size of n = 23 per group is needed to achieve the stated goals.

11. Basic Parameters and Concepts • Study (Research) Hypotheses • Type I Error Rate, , Significance level • P-value • Type II Error Rate,  • Power, 1-  • Effect Size, Δ ~size of clinically meaningful change.

12. HYPOTHESIS,HYPOTHESIS TESTING

13. Hypothesis • HYPOTHESIS: a statement about a population characteristic/parameter • HYPOTHESIS:a prediction/idea about what the examination of appropriate data will show about a characteristic

14. Hypothesis • Null (Test) Hypothesis, H0 ~Hypothesis to be questioned (disproved). ~ Hypothesis of no real (true) difference • Alternative (Research) Hypothesis, HA ~Hypothesis investigator wishes to establish. ~ Hypothesis of a real (true) difference

15. Example • Research Hypothesis: Combination therapy is effective?? in the treatment of hypertension. • Effective ~ considerable reduction in BP (1) ~ controls BP increases (2) • Parameter ~ Mean percent reduction in BP (1) ~ Proportion controlled (2) • Test Hypothesis:The combination therapy is not effective.

16. Goal • Goalis to TEST theNull Hypothesisanddecide whether to REJECT ITin favor of the Alternative,or FAIL TOREJECT it.

17. Test of Hypothesis

18. One-Tailed Tests • A test is one-tailed when the research hypothesis, HA , specifies a direction: HA: The incidence of lung cancer among smokers is higher than nonsmokers

19. Two-Tailed Tests • A test is two-tailed when no direction is specified in the research hypothesis HA. HA: The stress level in DC is different from NY.

20. Test & Decision Test H0 : no difference in effectiveness Possible Outcomes Null Hypothesis could be true (i.e., no difference) Null Hypothesis could be false (i.e., difference) Decision Making Investigator rejects the null hypothesis Investigator fails to rejects the null hypothesis

21. Test & Decision Test H0: therapy is not effective

22. Drug Trial H0: “Miracle” drug is not effective TI: Deny a patient a “known therapy” in favor of an ineffective “miracle drug” TII: Deny a patient a better drug in favor of a lesseffective “known therapy

23. Test & Decision

24. Is this Familiar !!!!! • All tests were performed two-sided at the 5% level of significance. • Significance was defined as a value of p < 0.05. • A value of p < 0.05 was considered statistically significant. • ALL YOU ARE DOING IS CONTROLLING THE TYPE I ERROR RATE

25. Definitions  = P{Type I Error } = P{rejecting H0|H0 is true} = P{rejecting the truth}  ~ is called the Type I Error Rate  ~ is called the Significance Level

26. Definitions = P{Type II error} = P{fail to reject H0|H0 is false} =P{accepting a fallacy }  ~ called the Type II Error Rate 1- ~ called Power of study

27. Definitions = P{fail to reject H0|H0 is false} 1- =P{reject H0 | H0 is false} = P{ accept HA| HA is true} 1- ~ is called Power of study Power ~ quantifies the ability of the study to detect a difference, if any

28. Definitions: P-value ~ probability of having observed our data (i.e. observed a difference) when thenull hypothesis is true???. ~probability of the data having arisen by chance when the null hypothesis is true.

29. Definitions: P-value ~ the smaller the p-value, the weaker the null hypothesis ~ the smaller the p-value, the stronger the alternative hypothesis How do we evaluate this probability? By calculating a test statistic

30. Test Statistic • a value which we can compare with a known distribution of what we expect when the null hypothesis is true Most test statistic have the form: • Test Statistic = observed value – expected value standard error of observed value

31. Common Test Statistic • T-test • F-test • Chi-square (χ2) test How do you choose the appropriate statistic???

32. Statistical Significance • Accepted values in clinical research p 0.05 significant P 0.01 highly significant In Genetic (Linkage) Analysis: • Lod Score =3.0 ~ significant • Lod Score =3.0 ~ =0.0001

33. SAMPLE SIZECONSIDERATION

34. Population And Sample Target Population Define Eligibility Criteria Ineligible Study Population Study Sample

35. Eligibility Criteria!!!! ~ consist of inclusion criteria exclusion criteria • Inclusion criteria is used to outline the intended study population • Exclusion criteria is used to fine-tune the intended population by removing expected sources of variation

36. Eligibility Criteria!!!! • Exclusion Criteria • Male • Age < 21 years • BMI < 25 kgm-2 • Inclusion Criteria Female Age ≥ 21 years BMI ≥ 25 kgm-2 REDUNDANT!!!!

37. Eligibility Criteria!!!! • Inclusion Criteria Exclusion Criteria i. Female i. Male ii. Age > 21 yrs ii. Age < 21 iii. BMI ≥ 25kgm-2 iii. BMI < 25 • Exclusion Criteria i. Pregnant or breast feeding ii. History of ……. iii. Any other condition in the opinion of the investigator (s) that would make the subject unsuitable for the study

38. Why Sample Size ? • Requirement ( Clinical Research Protocol, Funding Agencies, etc) in many grant application • Budgetary Constraints • Provide Statistical Justification • Inference (decision) is based on it

39. How Much Data Do I Need? • How big a difference are you trying to detect? Effect Size - Absolute difference ~ say 5mmHg drop BP - Relative difference ~ 5% drop in BP • How much variation is there in the outcome? • How certain do you want to be that you will detect the difference of interest ?

40. Eliciting effect size • How big a difference would be of clinical importance for you? Some responses I get: • Huh?? • What do you mean? • What do you recommend? • Any difference at all would be important

41. Finding the right variance • Based on experience Range of values Stories behind extreme values Sources of variations • Use of historical data • Conduct a pilot study.

42. What if u have imposed sample size • Sometimes, a proposal comes with imposed sample size. • Sample size is but one of several quality characteristics of a study • If n is held fixed, we simply need to focus on other characteristics, such as effect size.

43. Determination of Sample Size • Depends on: • Outcome measure (Data Endpoint) • Study Design

44. Types of Data Endpoints • Continuous Data - BP, BMI, TC, LDL, Blood Sugar • Categorical Data - Hypertension, Obese, Dyslipidemia, Diabetes • Count Data 0, 1, 2, 3 - No of risk factors • Survival (Time-to-Event) Data - time-to-cardiac event, time-to-death

45. Putting All Together(Power Analysis) 1- = P{ accept HA|HA is true) =Func (, 2(n),) Power Certainty Variability Effect Size Sample size

46. Crude SS Estimate for Means 2-Sample Test for Means (2-sided) 16s2 ,  =0.05,  =0.2 n = 2 Power = 80%

47. Sample Size Formula 2-Sample Test for Means (2-sided) 162 ,  =0.05,  =0.2 n = 2 Power = 80%

48. Sample Size • A larger sample size is needed to detect the smallest meaningful difference. • A larger sample size is needed when there is much variability in the population • A larger sample size is required to increase the power of a study.

49. Other Approaches There are several approaches to sample size. • One can specify the desired width of a confidence interval and determine the sample size that achieves that goal. • A Bayesian approach can be used where we optimize some utility function-perhaps one that involves both precision of estimation and cost.

50. Avoid “canned” effect sizes.- The T-shirt effect sizes • This is an elaborate way to arrive at the same sample size that has been used in past social science studies of large, medium, and small size.  • The method uses a standardized effect size as the goal.  • Think about it: for a "medium" effect size, you'll choose the same n regardless of the accuracy or reliability of your instrument, or the narrowness or diversity of your population.  • Important considerations are being ignored here.  "Medium" is definitely not the message!