Eduardo Bergel’s Controlled Clinical Trial Simulator (CTS) Beta Version 0.21

2. Eduardo Bergel’sControlled Clinical Trial Simulator (CTS) Beta Version 0.21 Used by Dave Sackett (Oct ‘04) Trout Research & Education Center at Irish Lake, Canada

3. How to run the CTS . . . . . . . . . . . . . . . . . . . . . Slide # 4 Effect of sample size on confidence intervals . . . # 38 The effects of admitting patients with different risks and responsiveness to your RCT . . . . . . . . . . . . # 53 Table of Contents

4. 1. How to run the CTS

5. You tell the Controlled Trial Simulator • How many patients you hope to recruit. • Your guess at their risk of events if they receive no Rx or the control Rx. • Your hopes for their responsiveness to experimental Rx. • With what rates of cross-over, loss-to-follow-up, and compliance

6. The CTS will then: • “Run” your trial hundreds or thousands of times. • - and tell you, with tables and graphs, its results, their confidence intervals, and the presence and magnitude of bias (e.g., from cross-overs, low compliance, etc.)

7. For example, does aspirin reduce strokes after TIA ? • We estimated that 21% of patients with TIAs would go on to stroke or death within 2 years. • We thought we could reduce this by one-third with aspirin (to 14%). • We thought we could recruit about 600 patients.

8. For example, does aspirin reduce strokes after TIA ? • We enter these estimates and guesses into the RCT Simulator. • And then tell the RCT Simulator how many times to run our trial.

10. We thought that 21% would stroke or die without Rx CER = 21%

11. We thought that 21% would stroke or die without Rx CER = 21% We thought that aspirin could reduce these outcomes by 1/3, to 14% EER = 14%

12. We thought we could recruit 600 patients We thought that 21% would stroke or die without Rx CER = 21% We thought that aspirin could reduce these outcomes by 1/3, to 14% EER = 14%

13. We thought we could recruit 600 patients We thought that 21% would stroke or die without Rx CER = 21% We thought that aspirin could reduce these outcomes by 1/3, to 14% EER = 14% We wanted our RCT replicated 3000 times

14. 3000 simulations take 5 seconds ! And their results are displayed in a table

16. CER

17. EER CER

18. EER CER 300 300

19. EER CER 300 300 3000 simulations

20. Relative Risk Reduction EER CER 300 300 3000 simulations

21. Relative Risk Reduction EER CER 300 300 3000 simulations Calculates Power !

22. Relative Risk Reduction EER CER 300 300 3000 simulations Can graph results Calculates Power !

23. GraphOptions • Can simply graph the overall RRR and its 95% confidence interval.

24. RRR for aspirin in TIA X= true RRR that is free of bias

25. Graphic Options • Can simply graph the overall RRR and its 95% confidence interval. • Can create a histogram of individual RRRs from each simulation.

26. Histogram of RRR

27. Graphic Options • Can simply graph the overall RRR and its 95% confidence interval. • Can create a histogram of individual RRRs from each simulation. • Can plot the individual RRRs from each simulation.

28. Can plot every trial

29. Can plot every trial Some RRRs < 0

30. Can plot every trial Some RRRs < 0 Extreme results must occur !

31. Graphic Options • Can simply graph the overall RRR and its 95% confidence interval. • Can create a histogram of individual RRRs from each simulation. • Can plot the individual RRRs from each simulation. • Can plot RRR vs. P-values.

32. Can chart the P-values for every trial

33. Can chart the P-values for every trial RRR & 95% CI

34. Can chart the P-values for every trial P > .05 P <.05

35. Can chart the P-values for every trial P > .05 P <.05 Power = 60%

36. Can chart the P-values for every trial P almost 0.05 against! P <.05 Power = 60%

37. So, that is the basic system Outputs can also be expressed in all these formats as:Relative Risks Currently under construction:Absolute Measures (ARR, NNT, etc)Cluster Trials

38. 2. The effect of sample size on 95% confidence intervals

39. Can use it to understand the effect of different sample sizes on the same RRR: How to do this: While keeping the treatment effect constant (say, RRR = 25%), double, triple and quadruple your sample size.

40. Increasing Sample Size Can have 4 different (sub) groups

41. Increasing Sample Size

42. Increasing Sample Size Then simulate it 3000 times

43. Increasing Sample Size All 2500

44. Increasing Sample Size All 2500 250

45. Increasing Sample Size All 2500 250 1000

46. Now you can answer a frequently asked question: If you want to cut the 95% confidence interval (CI) on your RRR in half, by how much do you need to increase your sample size (N)?

47. You need to remember “the only formula you’ll ever need” CONFIDENCE (recalling that narrower confidence intervals provide higher confidence) Confidence =

48. You need to remember “the only formula you’ll ever need” CONFIDENCE (recalling that narrower confidence intervals provide higher confidence) Confidence = Signal Noise

49. You need to remember “the only formula you’ll ever need” CONFIDENCE (recalling that narrower confidence intervals provide higher confidence) Confidence = x √sample size Signal Noise

50. Increasing Sample Size All 2500 To cut this in half 250 1000