1 / 33

Confidence Intervals

Clinical Trials in 20 Hours. Confidence Intervals. Elizabeth S. Garrett esg@jhu.edu Oncology Biostatistics March 27, 2002. What is a “confidence interval”?. It is an interval that tells the precision with which we have estimated a sample statistic. Examples:

lucius
Download Presentation

Confidence Intervals

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Clinical Trials in 20 Hours Confidence Intervals Elizabeth S. Garrett esg@jhu.edu Oncology Biostatistics March 27, 2002

  2. What is a “confidence interval”? • It is an interval that tells the precision with which we have estimated a sample statistic. • Examples: • parameter of interest: progression-free survival time: “The 95% confidence interval on progression-free survival is 13 to 26 weeks.” • parameter of interest: response rate “The 95% confidence interval on response rate is 0.20 to 0.40.” • Parameter of interest: change in %CD34+ cells “The 95% confidence interval for %CD34+ cells is 0.2 to 0.4.” Clinical Trials in 20 Hours

  3. Different Interpretations of the 95% confidence interval • “We are 95% sure that the TRUE parameter value is in the 95% confidence interval” • “If we repeated the experiment many many times, 95% of the time the TRUE parameter value would be in the interval” • “Before performing the experiment, the probability that the interval would contain the true parameter value was 0.95.” Clinical Trials in 20 Hours

  4. Example Leisha Emens, M.J. Kennedy, John H. Fetting, Nancy E. Davidson, Elizabeth Garrett, Deborah A. Armstrong “A phase 1 toxicity and feasibility trial of sequential dose dense induction chemotherapy with doxorubicin, paclitaxel, and 5-fluorouracil followed by high dose consolidation for high risk primary breast cancer” 83 patients underwent leukopheresis for peripheral blood stem cell collection after conventional dose adjuvant therapy, and 14 patients underwent the procedure on the dose dense adjuvant protocol 9626. Results: Compared to the standard dose doxorubicin-containing adjuvant therapy, the dose dense regimen decreased CD34+ peripheral blood stem cell (PBSC) yields, requiring that 50% patients have a supplemental bone marrow harvest. Question: What can we say about how %CD34+ peripheral blood stem cell yields in each of the two groups? Clinical Trials in 20 Hours

  5. %CD34+ PBSC in trial 9601 and 9626. We can estimate the mean %CD34+ PBSC in each trial: 0.40 in the standard group 0.30 in the dose-dense group. We can conclude: “We estimate that %CD34+ PBSC in the standard group is 0.40 and in the dose dense group is 0.30.” But, how “sure” are we about those estimates? Example Clinical Trials in 20 Hours

  6. Quantifying Uncertainty • Standard deviation: measures the variation of a variable in the population. • The standard deviation of %CD34+ PBSC in the standard group is 0.27 and is 0.20 in the dose dense group. • Technically, Clinical Trials in 20 Hours

  7. For normally distributed variables…. 68% of individuals’ values fall between 1 standard deviation of the mean s 68% Clinical Trials in 20 Hours

  8. For normally distributed variables…. 95% of individuals’ values fall between 1.96 standard deviations of the mean 1.96s 95% Clinical Trials in 20 Hours

  9. Standard deviation versus standard error • The standard deviation (s) describes variability between individuals in a population. • The standard error describes variation of a sample statistic. • Example: We are interested in the mean %CD34+ PBSC. (We notate the mean by x). • The standard deviation (0.27 in standard and 0.20 in dose dense) describes how individuals differ. • The standard error of the mean describes the precision with which can make inference about the true mean. Clinical Trials in 20 Hours

  10. Standard error of the mean • Standard error of the mean (sem): • Comments: • n = sample size • even for large s, if n is large, we can get good precision for sem • always smaller than standard deviation (s) Clinical Trials in 20 Hours

  11. Example • In standard group, s = 0.27 and n = 83: • In dose dense group, s = 0.20 and n = 14: Clinical Trials in 20 Hours

  12. Sampling Distribution The sampling distribution of a sample statistic refers to what the distribution of the statistic would look like if we chose a large number of samples from the same population Mean = 3 s = 2.45 The sample statistic of interest to us is the mean. Clinical Trials in 20 Hours

  13. Sampling Distribution of the Mean By the Central Limit Theorem, it is true that even if a variable is NOT normally distributed, for large sample size, the sampling distribution of the mean is normally distributed. Mean = 3 s = 2.45 Clinical Trials in 20 Hours

  14. Sampling Distributions sem = 0.47 sem = 0.23 sem = 0.10 sem = 0.47 Clinical Trials in 20 Hours

  15. Central Limit Theorem Main Ideas • The sampling distribution of a sample statistic is often normally distributed • The mathematical result comes from the Central Limit Theorem. For the theorem to work, n should be large. • Statisticians have derived formulas to calculate the standard deviation of the sampling distribution and it is called the standard error of the statistic Clinical Trials in 20 Hours

  16. Sampling Distribution of the Mean • In general for large n, means have a normal distribution. • It is true that 95% of sample means will be within 1.96 of the true mean, . The 95% confidence interval for the mean Clinical Trials in 20 Hours

  17. General formula for 95% confidence interval • Notes: • sample size must be sufficiently large for non-normal variables. • how large is large? depends on skewness of variable • VERY often people use 2 instead of 1.96. Clinical Trials in 20 Hours

  18. Example • In the standard group, the mean was 0.40, s = 0.27, and n = 83: • In the dose dense group, the mean was 0.30, s = 0.20, and n = 14: Clinical Trials in 20 Hours

  19. Not only 95%…. • 90% confidence interval: NARROWER than 95% • 99% confidence interval: WIDER than 95% Clinical Trials in 20 Hours

  20. But why do we always see 95% CI’s? • “Duality” between confidence intervals and pvalues • Example: Assume that we are testing that for a significant change in QOL due to an intervention, where QOL is measured on a scale from 0 to 50. • 95% confidence interval: (-2, 13) • pvalue = 0.07 • It is true that if the 95% confidence interval overlaps 0, then a t-test testing that the treatment effect is 0 will be insignificant at the alpha = 0.05 level. • It is true that if the 95% confidence interval does not overlap 0, then a t-test testing that the treatment effect is 0 will be significant at the alpha = 0.05 level. Clinical Trials in 20 Hours

  21. Other Confidence Intervals • Differences in means • Response rates • Differences in response rates • Hazard ratios • median survival • difference in median survival • …….. Clinical Trials in 20 Hours

  22. Difference in Means • Example: What is the 95% confidence interval for the difference in %CD34+ PBSCs in the two trials? Clinical Trials in 20 Hours

  23. 95% Confidence Intervals for Proportions • Socinski et al., Phase III Trial Comparing a Defined Duration of Therapy versus Continuous Therapy Followed by Second-Line Therapy in Advanced-Stage IIIB/IV Non-Small-Cell Lung Cancer JCO, March 1, 2002. • Patients and Methods: Arm A (4 cycles of carboplatin at an AUC of 6 and paclitaxel), Arm B (continuous treatment with carboplatin/ paclitaxel until progression). At progression, patients from each arm receive second-line weekly paclitaxel at 80mg/m2/week. • Results: 230 Patients were randomized (114 in arm A and 116 in Arm B). Overall response rates were 22% and 24% for arms A and B. Grade 2 to 4 neuropathy was seen in 14% and 27% of Arm A and B patients, respectively. Clinical Trials in 20 Hours

  24. 95% Confidence Intervals for Proportions • What are 95% confidence intervals for the response rates in the two arms? • standard error of a sample proportion is • An equation for confidence interval for a proportion: • Note: this is an approximation based on the central limit theorem! Using statistical programs, you can get “exact” confidence intervals. • Assumptions: • n is reasonably large • p is not “too” close to 0 or 1 • rule of thumb: pn > 5 Clinical Trials in 20 Hours

  25. Example: Response Rate to Treatment • Arm A: • Arm B: Clinical Trials in 20 Hours

  26. Example: Grade 2 to 4 Neuropathy • Arm A: • Arm B: Clinical Trials in 20 Hours

  27. 95% Confidence Interval for Difference in Proportions What is the 95% confidence interval for the difference in rates of neuropathy in arms A and B? Clinical Trials in 20 Hours

  28. Recap • 95% confidence intervals are used to quantify certainty about parameters of interest. • Confidence intervals can be constructed for any parameter of interest (we have just looked at some common ones). • The general formulas shown here rely on the central limit theorem • You can choose level of confidence (does not have to be 95%). • Confidence intervals are often preferable to pvalues because they give a “reasonable range” of values for a parameter. Clinical Trials in 20 Hours

  29. Some Confidence Intervals in Survival Analysis Example: Urba et al. Randomized Trial of Preoperative Chemoradiation Versus Surgery Alone in Patients with Locoregional Esophageal Carcinoma, JCO, Jan 15, 2001. Hazard Ratio95% CI Chemo v. surgery 0.69 0.46-1.06 Arm 1 Arm II % 95%CI % 95%CI 1 year survival 58 46-73 72 58-84 3 year survival 16 8-30 30 20-46 What about the confidence interval for the 1 year and 3 year difference? Clinical Trials in 20 Hours

  30. Why not provide confidence intervals for... • Difference in median survival • Difference in 1 year survival • Difference in 3 year survival • Would give readers a “reasonable range” of values to consider for treatment effect that are intuitive. • What is remembered? • P = 0.09 which means insignificant result • But, can anyone remember the treatment effect? Clinical Trials in 20 Hours

  31. Confidence Intervals for Reporting Results of Clinical Trials, Simon • “[Hypothesis tests] are sometimes overused and their results misinterpreted.” • “Confidence intervals are of more than philosophical interest, because their broader use would help eliminate misinterpretations of published results.” • “Frequently, a significance level or pvalue is reduced to a ‘significance test’ by saying that if the level is greater than 0.05, then the difference is ‘not significant’ and the null hypothesis is ‘not rejected’….The distinction between statistical significance and clinical significance should not be confused.” Clinical Trials in 20 Hours

  32. Caveats “They should not be interpreted as reflecting the absence of a clinically important difference in true response probabilities.” Clinical Trials in 20 Hours

  33. Excellent References on Use of Confidence Intervals in Clinical Trials • Richard Simon, “Confidence Intervals for Reporting Results of Clinical Trials”, Annals of Internal Medicine, v.105, 1986, 429-435. • Leonard Braitman, “Confidence Intervals Extract Clinically Useful Information from the Data”, Annals of Internal Medicine, v. 108, 1988, 296-298. • Leonard Braitman, “Confidence Intervals Assess Both Clinical and Statistical Significance”, Annals of Internal Medicine, v. 114, 1991, 515-517. Clinical Trials in 20 Hours

More Related