1 / 35

New Developments in Confidence Intervals That Improve Result Reporting: Confidence Levels, Clinical Significance Curves

New Developments in Confidence Intervals That Improve Result Reporting: Confidence Levels, Clinical Significance Curves and Risk-Benefit Contours. Dr. Thomas P. Shakespeare MBBS, FRANZCR, FAMS, MPH, GradDipMed(ClinEpi). Critical References.

major
Download Presentation

New Developments in Confidence Intervals That Improve Result Reporting: Confidence Levels, Clinical Significance Curves

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New Developments in Confidence Intervals That Improve Result Reporting: Confidence Levels, Clinical Significance Curves and Risk-Benefit Contours. Dr. Thomas P. Shakespeare MBBS, FRANZCR, FAMS, MPH, GradDipMed(ClinEpi)

  2. Critical References For a full discussion please refer to our original article in The Lancet: Shakespeare TP, Gebski VJ, Veness MJ, Simes J. Improving interpretation of clinical studies by use of confidence levels, clinical significance curves, and risk-benefit contours. Lancet. 2001: 357: 1349–53. Also you can download a free Confidence Calculator for our methods:www.theshakespeares.com/confidence_calculator.html

  3. Objectives • To understand the limitations and potential misinterpretation of p-values and 95% confidence intervals. • To understand how new methods can improve statistical analysis and result reporting. • To understand how to calculate confidence levels, clinical significance curves and risk-benefit contours. • To understand when it is appropriate to use these new methods for analyzing and reporting study results.

  4. Problems when reporting results 1. P values and confidence intervals are often misinterpreted. 2. They do not answer our basic clinical questions: • How likely is it that a clinically relevant benefit or detriment is present? • How confident are we that a benefit is not outweighed by unacceptable toxicity?

  5. The solution Develop tools thatimprove result reporting: • Confidence levels • Clinical significance curves • Risk-Benefit contours. • Their advantages: • Prevent misinterpretation • Answer our clinical questions • Improve the decision-making process.

  6. An exampleof methods to report results WHO Melanoma Study (Cascinelli et al, Lancet 1998; 351: 793-96) • 252 patients with truncal melanoma 1.5mm thick • Randomized to immediate nodal dissection or observation (and delayed dissection if required).

  7. WHO Melanoma Study Results • 5 year survival favoured immediate nodal dissection: 61.7 % vs 51.3 %, HR 0.72 • 95% CI 0.49-1.04, not significant: p=0.07 Authors’conclusion • Immediate nodal dissection had “no impact on survival”, and should not be used. The results have been misinterpreted!

  8. What information is in the 95% CI? • 95% CI for the hazard ratio is 0.49-1.04 • Thus we can be 95% confident that the true hazard ratio lies within these limits, based on this study. • How likely is it that a survival benefit exists, or does not exist, based on this data? • Confidence intervals can’t tell us, however a confidence level can.

  9. ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 95% Confidence intervalWHO melanoma study. Point estimate 0.72, SE 0.192 beneficial detrimental 0.49 1.04 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 2.5% ! ! ! ! ! ! ! ! ! ! 2.5% ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 95% ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 Relative survival benefit (hazard ratio)

  10. Confidence level for any benefit We need to determine how much confidence lies below 1.00 (HR < 1 indicates a survival benefit). From the WHO study, the point estimate for survival was 0.72, with a standard error of 0.192 (extrapolated from the original publication).

  11. Confidence level for any benefit 1. Calculate the confidence interval around the hazard ratio with an upper limit of 1.00 (93% CI in this example) 2. Calculate how much confidence lies below this interval (half of 7% = 3.5%) 3. Add the two percentages (93 + 3.5 = 96.5%) Thus there is 96.5% confidence that a survival benefit exists. This is very high despite the lack of significance!

  12. ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Confidence level for any benefit(Point estimate 0.72) Benefit Detriment 96.5% 0.52 1.00 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 3.5% ! ! ! ! ! ! ! ! ! ! ! ! ! 3.5% ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 93% ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 Relative survival benefit (hazard ratio)

  13. Other magnitudes of benefit • Confidence levels can be determined for any benefit or detriment of interest. • “What if my patient is only interested in a 3% benefit or more?” • We use the same methods but set an upper value of 0.97 • We are 94% certain that dissection results in a survival benefit of 3% or more.

  14. ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Confidence level for minimum 3% benefit(Point estimate 0.72) Clinically No Relevant relevant benefit 94% benefit/detriment 0.53 0.97 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 6% ! ! ! 6% ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 88% ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 0.4 0.5 0.6 0.8 0.9 1 1.1 1.2 0.7 Relative survival benefit (hazard ratio)

  15. The WHO results revisited • HR for survival 0.72, not significant: (95% CI 0.49-1.04, p=0.07) However • 96.5% confidence that a benefit exists • 94% confidence that the benefit is 3% or more • Thus a clinically relevant benefit is probable, and further studies are required to confirm it.

  16. The WHO results revisited • 96.5% confidence that a benefit exists • 94% confidence that the benefit is 3% or more • Thus a clinically relevant benefit is probable, and further studies are required to confirm it. • This is in contradistinction to the authors’ conclusion. • Confidence levels may have avoided misinterpretation, and provided more clinically relevant information.

  17. Confidence levels improve result interpretation • Confidence levels give us the level ofconfidence, likelihood or probabilitythat a benefit exists, and tell us whether the benefit is clinically relevant. • They are more useful than P values and confidence intervals. • Confidence levels have been used to analyze meta-analyses and clinical studies.

  18. Clinical Significance Curves • Individuals may accept different benefit thresholds before using a new therapy. • We can provide confidence levels for any threshold of benefit or detriment. • These can be combined to produce a Clinical Significance Curve (CSC).

  19. CSC for survival in WHO study • Individuals can select an acceptablebenefit threshold and determine the level of confidence associated with it. • For example if a clinician is only interested in a benefit of 15% or more, we can see that there is only 81% confidence that such a benefit exists. • CSCs provide clinically relevant information to individual clinicians.

  20. ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Clinical significance curve for WHO study 100 X X ! ! ! 90 significance level (97.5%) X 80 Any benefit: 96.5% confidence  3% benefit: 94% confidence  15% benefit: 81% confidence  28% benefit: 50% confidence 70 Confidence level (%) 60 X 50 40 30 20 10 0 0 5 10 15 20 25 30 35 40 45 50 55 60 Relative survival benefit (%)

  21. Risk-Benefit Contours • CSCs can be constructed for any comparison: eg OS, local control, toxicity. Alternatively, the information can be displayed in tabular form, at say 5% increments of difference. • We can combine CSCs (eg survival with toxicity), to form Risk-Benefit Contours (RBCs). • RBCs allow us to calculate the confidence associated with acceptable risk-benefit scenarios.

  22. An example • Intergroup study 0099 (Al-Sarraf et al, JCO 1998). • Chemoradiotherapy vs radiotherapy for Stage III and IV NPC Results • 3 year survival 78% v 47% (p=0.005) • Grade 3 or 4 acute toxicity 76% v 50%.

  23. CSC for survival • High levels of confidence for large survival benefits (eg up to 20% or so). • Statistically significant survival benefits.

  24. ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Clinical significance curve for Intergroup study(Absolute 3 year survival 78% v 47%) 100 X 90 significance level (97.5%) 80 Any benefit: 99.997% confidence Min 16% benefit: 97.5% confidence Min 18% benefit: 95% confidence 70 Confidence level (%) 60 50 40 30 20 10 0 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 0 10 20 30 40 50 60 Absolute survival benefit (%)

  25. CSC for toxicity • High level of confidence that chemoradiotherapy causes excess acute grade 3/4 toxicity.

  26. ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 20 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 10 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 0 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 0 10 20 30 40 50 60 Absolute toxicity detriment (%) Clinical significance curve for Intergroup study(Acute G3/4 toxicity 76% v 50%) 100 X 90 significance level (97.5%) 80 Any toxicity  : 99.95% confidence Min 10.4% : 97.5% confidence Min 13% : 95% confidence 70 60 Confidence level (%) 50 40 30

  27. Risk-Benefit Contours • Formed by combining the 2 curves. • To use RBCs, the individual clinician first determines an acceptable risk-benefit scenario, then reads off the corresponding confidence associated with it.

  28. 60 99.95 99.99 50 99.90 99.00 97.00 95.00 85.00 40 Maximum toxicity detriment (%) 90.00 80.00 70.00 60.00 50.00 30 25.00 20 0 5 10 15 20 25 30 35 Minimum survival benefit (%)

  29. Doctor A’s risk-benefit scenario Doctor ‘A’ believes the new treatment (chemoRT) is more worthwhile than RT alone if: at least an extra 5 out of his next 100 patients will be cured due to the new treatment AND at most an extra 50 of these 100 patients will experience G3/4 toxicity due to the new treatment. We can see that there is99.9% confidencethat the above scenario exists, based on the study.

  30. 60 99.99 99.90 99.95 Doctor ‘A’s scenario 50 99.90 99.00 97.00 95.00 Maximum toxicity detriment (%) 40 85.00 90.00 80.00 70.00 60.00 30 50.00 25.00 20 0 5 10 15 20 25 30 35 Minimum survival benefit (%)

  31. Doctor B’s risk-benefit scenario Doctor ‘B’ will only accept a scenario in which: at least an extra 5 out of his next 100 patients will be cured due to the new treatment AND at most an extra 30 of these 100 patients will experience G3/4 toxicity due to the new treatment. We can see that there is only70% confidencethat this scenario exists. Doctor B is not very confident. Risk-benefit contours have aided the decision-making process.

  32. 60 99.99 99.95 Doctor ‘A’s scenario 50 99.90 99.90 97.00 95.00 40 90.00 Maximum toxicity detriment (%) 85.00 80.00 70.00 60.00 50.00 30 Doctor ‘B’s scenario 25.00 20 10 5 15 20 25 30 35 0 Minimum survival benefit (%)

  33. Conclusion • P values and confidence intervals are too easily misinterpreted, and do not answer our simple clinical questions. • By usingConfidence Levels, Clinical Significance Curves and Risk-Benefit Contours, we can: • Gain a better understanding of our results • Avoid misinterpretation • Aid the decision-making process.

  34. Confidence Calculator • To calculate confidence levels, only takes a hand calculator and z table of normal values. • Clinical significance curves and risk-benefit contours can be calculated with standard statistical software. • Alternatively a dedicated “Confidence calculator” can be used. • A free calculator is available from:www.theshakespeares.com/confidence_calculator.html

  35. References Please see attached notes page for a full list of references used in this lecture.

More Related