Biostatistics Case Studies 2006

1. Biostatistics Case Studies 2006 Session 1: Demonstrating Equivalence of Active Treatments: Non-inferiority Studies Self-Quiz Peter D. Christenson Biostatistician http://gcrc.LAbiomed.org/Biostat

2. Self-Quiz • Give an example in your specialty area for a superiority /inferiority study. Now modify it to an equivalence study. Now modify it to a non-inferiority study.

3. Self-Quiz • Answer • Vaccine Testing: • Superiority: New candidate vaccine vs. placebo • Equivalence: Antigen potency between two manufacturing plants or lots. • Non-Inferiority: New candidate vaccine vs. old one.

4. Self-Quiz • T or F: The main point about non-inferiority studies is that we are asking whether a treatment is as good or better vs. worse than another treatment, so it uses a one-sided test.

5. Self-Quiz • Answer • False. That is a feature of these studies, but not their distinguishing feature. They and equivalence studies are used to try to prove sameness, as opposed to typical sydies that try to prove differences.

6. Self-Quiz • Power for a typical superiority test is the likelihood that you will declare treatment differences (p<0.05) if treatments really differ by some magnitude Δ. Explain what power means for a non-inferiority study.

7. Self-Quiz • Answer • Power for a non-inferiority study is the likelihood that you will declare treatment A to be no worse than a pre-specified magnitude E from B (p<0.05) if treatments really differ by some Δ. Of course, Δ is less than E, and is often 0.

8. Self-Quiz • T or F: Last-value-carried-forward is a good way to handle drop-outs in a non-inferiority study. Explain.

9. Self-Quiz • Answer • False. LVCF biases results toward less of a difference. This makes typical superiority studies conservative, but increases the chance of a falsely “proving” the aim.

10. Self-Quiz • Many comparative studies have an evaluator who is masked (blinded) as to subjects’ treatment, especially for subjective outcomes, to prevent bias. Explain how such an evaluator in a non-inferiority study has the power to completely bias the results to prove the aim, if the outcome is a final value rather than change score.

11. Self-Quiz • Answer • The evaluator merely has to rate everyone very similarly to be absolutely sure for the aim to be shown. If he is worried that within-group differences will be too small and arouse suspicion, just randomly rate everyone within normal ranges, to be ~95% sure. • Note that this cannot be done if the outcome is a change score from baseline, and if the evaluator does not know the baseline score.

12. Self-Quiz • T or F: In a non-inferiority study, you should first test for non-inferiority with a confidence interval, and then use a t-test to test for superiority, but only if non-inferiority was established at the first step.

13. Self-Quiz • Answer • False. You must specify a-priori superiority, in order to have a legitimate claim of proving it (beyond a reasonable (5%) doubt). The stated sequential strategy will only allow you to claim an observed result, without a statement about it’s certainty.

14. Self-Quiz • What is the meaning of the equivalence margin, and how do you determine it?

15. Self-Quiz • Answer • The equivalence margin is the maximum difference between treatments that is considered to be negligible or unimportant. • It must be pre-specified in order to prove equivalence or non-inferiority to that degree, rather than just noting it as an observation. Thus, it is ideally determined by peer-agreement or FDA concurrence prior to starting the study.

16. Self-Quiz • What do you conclude if the CI for treatment difference does not lie to the left of the equivalence margin?

17. Self-Quiz • Answer • We conclude that non-inferiority was not proven. • Based on the range of values in the CI, we may also note that the study was underpowered (e.g., CI centered near equivalence and wide), or observe that inferiority is likely (e.g., narrow CI to the right of the equivalence margin.

18. Self-Quiz • Suppose the primary outcome for a study is a serum inflammatory marker. If it’s assay is poor (low reproducibility), then it is more difficult to find treatment differences in a typical superiority/inferiority study than for a better assay, due to this noise. Would it be easier or more difficult to find non-inferiority with this assay, compared to a better assay?

19. Self-Quiz • Answer • It would still be more difficult to show the aim, non-inferiority here, since CIs will be wider, but there will be no bias due to it toward either treatment. • Generally, poorer study conduct is penalized in superiority studies and rewarded in non-inferiority studies, but that is not true for this type of poorer measurement error.

20. Self-Quiz • Does the assumed treatment difference (0.5 here) for power calculations have the same meaning as the difference used for power calculations in a typical superiority/inferiority study?

21. Self-Quiz • Answer • No. Here, it is our best estimate of true treatment differences. • For superiority studies, the difference is ideally the minimal difference that is “clinically relevant”, not the expected difference, closer in meaning to the equivalence margin here. In practice, it is the smallest difference that logistics, money, time, and effort will allow us to detect with specified certainty.