Controlling Errors in Medical Studies: Overview

1. Controlling Errors in Medical Studies: Overview 李世昌 銘傳大學 應用統計與資訊學系 December 01, 2011

2. Agenda • Statistical Designs in Medical Studies • Selection of the Control Group • Sample Size Determination • Randomization • Statistical Analysis • Quality Assurance (QA) • Non-inferiority Trials • International Conference for Harmonisation

3. Experimental Errors • Controlling and Minimizing • Quality by Design • QA in Medical Studies • Controlling the Errors in Industry and Medical Sectors • Drugs/medical devices/vaccine/… • Clinical Trials • Why? How?

4. Pharmaceutical Industry • Research & Development • Non-clinical Studies • Search for treatments • Lab studies • Pre-clinical Studies • Animal studies • Pharmacokinetic (PK) studies • Clinical Trials • Human studies

5. Objectives versus Conclusions • The Essence of Rational Medical Study is to AskImportant Questions and Answer them With Appropriate Studies • A study should be designed, conducted and analysed according to sound scientific principles to achieve their objectives; and should be reported appropriately • Statistical Approach in Design and Analysis

6. Inferential Statistics A Population of the Random Variable of X A Random of Sample Size of n Descriptive Estimates and Statistical Analysis Results and Conclusions

7. Inferential Statistics • A Population of Subjects • A characteristic of interest, X ~ F(x; q) • A Random Sample of Size n • Each size of n in a population has an equal probability to be selected • Θ ~ G(x; e) • Descriptive and Statistical Methodology • Graphs/charts, estimates, confidence intervals, tests of hypothesis • Statistical Models ^

8. A Medical Study All Patients with a Specific Disease The Treatment of the Disease A Study Group of Patients Clinical Evidence of the Treatment Clinical Conclusions

9. Statistics versus Medical Studies • The Human Experiment? • A Population? A Specific Disease? • The Treatments of Disease? • Clinical Response and Indices? • Staggered Entry? • Sample Size? • Randomization? • …

10. Human Experiments • Medical Ethics? • Placebo? Standard/New Treatment? • Informed Consent Form • Patient Benefit and Risk? • Efficacy and Safety Issues • Clinical Evidence?

11. Complicated Issues • A Specific Disease to Study? • A New Treatment of a Disease • Any current standard treatments? • How to Quantify a Clinical Benefit and Minimized Adverse Effects? • Efficacy and safety

12. Existence of Errors and Bias • Patients • Investigational Team • The Treatment • Clinical Instrument and Measurement • Unknown Factors • …

13. Statistics in a Medical Study Study Objectives: Clinical indexes, efficacy variables/endpoints • Protocol and Design: • Clinical and design issues • Conduct Trial and Collect Data: • Ethic, accurate, validate, and reliable data • The Analysis and Results: • Interpretations and Conclusions

14. The Distinction • Medical Studies • Objectives  IRB/DOH*  Conduction  Publication (literature review) • Clinical Trials • Objectives  IRB/DOH  Conduction  NDA+/Marketing/Publication • Statistics and Regulatory Issues (IRB, DOH, CDE ++, …) *IRB/DOH: Institutional Review Board/Department of Health +NDA: New Drug Application ++ CDE: Center of Drug Evaluation

15. Medical Studies and Clinical Trials Regulated Studies or Trials? Clinical Trials A medical study sponsored by a pharmaceutical company or … A system of combing the variety of expertise New Drug Application (NDA) oriented Declaration of Helsinki Trial Quality Assured? 15

16. Regulatory History in Medical Studies • Declaration of Helsinki • Ethics and Science • Medical Journals • Requirements on a submitted manuscript • Regulatory Agency • US/Food and Drug Administration (US/FDA), … • Department of Health (DOH), Taiwan (TFDA) • Nonprofit Organizations • NIH, CDC, NCI, … • Cancer center (MD Anderson, Mayo, Johns-Hopkins, Sloan-Kettering, …) • International Conference on Harmonisation (ICH) • Guidelines on Efficacy, Safety, Quality, Multi-discipline

17. Quality Assurance • A System of a Process of Tasks being Done • Designing, monitoring, documenting, organizing, analyzing, and concluding • Medical Research • Ethics + IRB + Journal Review • Clinical Trials • Ethics + IRB + Regulations + ICH +…

18. QA: Concepts • Quality Assurance (QA) • The systematic monitoring and evaluation of the various aspects of a process and management to maximize the probability that minimum standards of quality are being attained by the entire process (1) The Intended Purpose (2) Minimize the Errors and Bias (3) Systematic Approach (4) Valid and Reliable Conclusions

19. QC: Quality Control • Statistical Quality Control (SQC) • Accuracy of specifications • Integrity and precision • Total Quality of Management • QC + QA + SOPs • Monitoring and Auditing

20. QA: Clinical Trials • A Well-designed Protocol • Study Conduction and Adherence • Documentation • Data Management • Analysis and Interpretations • Regulations • International Conference on Harmonisation (ICH) • Taiwan Food and Drug Administration (TFDA)

21. QA: Education and Resources • Education • Trainings and Experience • Academic Education • Vocational Education and Training • Resources • Industry sector • Government sector • Scientific expertise

22. Statistical Methodology Point Estimate Interval Estimation Test of Hypothesis Statistical Models

23. Three Basic Statistical Methods • Point Estimation • No valid statement is made • Interval Estimation • (1-a)100% confidence of correctness • The upper and lower bounds for estimation • Test of Hypotheses • Two hypothesis (Ho: no effect vs. Ha: effect size) • Type I error rate (a) • The power of test (1-b) • Practical Meanings?

24. Analysis of a Sample Data • Is the Variable Well-defined? • How are the Sample Data Collected From? • Whether the Sample Data Represent the Study Population? • What is the Appropriate Analysis? • How to Interpret the Results? • Do the Conclusion Validated?

25. A Medical Study? • A Population of Patients? • A Group of enrolled Patients? • Are the Collected Clinical Data representative? • How to Reach the Scientific Evidence? • Are the Clinical Conclusions Valid and Reliable? • Statistical Tools!!! • How to Use the Statistical Methodology? • How to Accomplish the Scientific Evidence?

26. Design a Protocol Objectives Efficacy and/or Safety Primary/secondary variables Important Elements Controlled? Number of patients? Randomization and blindness? Statistical methodology? 26

27. Selection of a Control Group • Purpose • Minimize the bias in assessing the effect of test treatment • Choice of a Control Group • Placebo or no treatment • Active control • Historical control

28. Types of Comparisons • Superiority • Treatment A is better than treatment B • Bioequivalence • Treatment A is equivalent to treatment B • Non-inferiority • Treatment A is not inferior to treatment B

29. A Population and A Sample A Population of Patients Objectives + inclusive criteria A Sample Clinical Data Sets Number of patients Enrolled? Evaluable? Exclusive criteria Safety issues, … 29

30. A Population Model All Patients Control patients Test patients + A random sample from control patients A random sample from test patients Clinical data of two groups Statistical Analysis

31. An Invoked Model All Patients A subgroup of patients Control group Test group A random sample from test group A random sample from control group Statistical Analysis

32. Minimise Bias/Error and Assess Efficacy • Statistical Principles and Data Integrity • Selection of a control treatment • Sample size determination • Patients recruitment • Randomisation • Blinding • Compliance

33. Clinical Designs • Comparisons of Two Treatments (T vs. A) • Equality • Ho: mT-mA=0 vs. Ha: mT-mA 0 • Superiority • Ho: mT-mA=0 vs. Ha: mT-mA>0 • Equivalence • Ho: mT-mA L or mT-mA U vs. Ha: L< mT-mA < U • Bioequvalence (BE) studies • Non-inferiority • Ho: |mT-mA| M vs. Ha: |mT-mA|< M • Designs • Parallel (two independent samples) • Crossover (blocking samples) • Factorial (many independent samples)

34. Clinical Endpoints • Scientific Evidence? • Valid Conclusions? • Primary or Secondary? • Statistical Concerns • Type I error rate (a) • Power • Multiplicity adjustment of a

35. Inclusions/Exclusions Define a Population and a Sample Data Clinical judgment? Might involve in violations/deviations Ethics and selection bias Sample size Intent-to-treat (ITT) and per protocol (PP) 35

36. Examples • 1. Parallel Design • Two independent samples • 2. Cross-over Design • Paired samples • 3. One-way Analysis of Variance • Comparison of more than treatments • 4. …

37. Number of Patients Formulas and Charts Practical Meanings

38. Number of Patients Sample Size Determination Information of (a, 1-b, s, treatment difference) Conclusion Intent-to-Treat/Per Protocol data set Inclusion/exclusion Violation/deviation 38

39. Sample Size Calculation • Primary Endpoint • Precision or power approaches • Parameters: Type I error rate, power, variance, margin of error • Formula or charts • Consideration in survival trials

40. Formulas • Precision Power ( z2/a)2 (2s2)(z2/a+zb)2 (2s2 ) • n= ---------------- ; n= --------------------- [mD0 – mDa]2 [mD0 - mDa]2 • Practical Meanings? • Survival Studies?

41. Power Approach

42. Randomization Patient Allocations Imbalance Issues Prognostics

43. Randomization • Tradition • A random sample of size n from a population • Completely randomized design of Analysis of Variance (ANOVA) • Clinical trials • Complete randomization • Randomization using prognostic factors

44. Maximum Power • Equal Sample Size for t-test • Balance Issue in Analysis of Variance • Simple Randomization • Bias Coin Randomization • Stratified Randomization

45. Randomization A Sequence of Random Numbers which a Treatment Assignment is based on Code, date, and time-point 1. Non-Adherence Human error Training problem Management problem … 2. Examples 45

46. Simple Randomization • Random Number Generator • No prognostic factor considered • Predictability • Balance of treatment groups • Example: Treatments A and B • A sequence of random number generated by a validated computer software • 1 8 6 2 6 3 5 8 7 0 … • Assign A if the random digit is 1-5, otherwise assign B • Imbalance between Treatments A and B • P[2:8] ≥ 0.05; P[40:60] ≥ 0.05; P[469:531] ≥ 0.05

47. Biased Coin Design • At Each Treatment Assignment, Assign the Least Treatment with a Higher Probability • Say, if D(i)=|n(A)-n(B)|≥ 2, then assign the treatment to the least number with p=2/3 or 3/5 • If D(i)=0, then use p=1/2 to assign treatment

48. Random Permuted Blocks Patient No. Treatment 1001 A With a block size of 4 1002 A 1003 B 1004 B 1005 A 1006 B 1007 A 1008 B 1009 B 1010 B 1011 A 1012 A … 48

49. Covariate-Adaptive Randomization • Use of Prognostic Factors in Patient Allocation • Zelen’s Rule • Stratified Randomization • Taves’ Minimization • Pocock-Simon’s Procedure

50. Stratified Randomization MaleFemale III IVIII IV B B A B A B A A B A B A A A B B A A B A B A A B A B B B B B A A …