A Statistical Approach to Method Validation and Out of Specification Data

1. A Statistical Approach to Method Validation and Out of Specification Data Dr. Daran Sadler Avecia

2. Outline of talk • Basic statistics • Averaging, confidence intervals • Fitness-for-purpose and analytical capability. • Quantifying variability and producing a capable method. • Out-of-specification results. • Conclusions.

3. Repeat measurements

4. Distribution of measurements The 95% confidence interval is the range of values around the mean in which 95% of the measurements are expected to lie.

5. Relative standard deviation, RSD For a strength of ~100%, a 0.7% RSD equates to a standard deviation of ~0.7%. This means that the range of values encompassing 99% of all possible measures is approximately +/- 2.1%. 0.7% RSD at 100% strength has a 99% confidence interval of 97.9% to 102.1%.

6. Effect of averaging • The standard deviation is a measure of variability. • The effect of variability can be reduced by taking the average of a number of repeat measures. • The standard deviation associated with the mean of n measures is:

7. Distribution of the mean n=4 n=3 n=2 n=1 The confidence in the mean improves as the number of measurements increases.

8. How many measurements should I average? • Depends upon: • The amount of variability present in the measurements. • The degree of confidence I wish to achieve. WHAT IS FITNESS FOR PURPOSE?

9. Capability of an analytical method Incapable method Capable method

10. How to measure capability? Use measures from statistical process control e.g., specification between 97 mg/l and 103 mg/l, width of confidence interval of 12mg/l:

11. Interpreting cp Batch failure rate purely due to variability in analytical method.

12. One-sided specifications Where is the expected average value of the parameter.

13. Method development/validation • To determine the number of repeat measurements to ensure that the analytical capability is acceptable, for example >1. • Acceptance criteria are then product dependent, rather than technique specific. • How do I determine the amount of variability? • How do I determine the number of repeat measurements required?

14. Quantifying variability (e.g. HPLC) Experimental Design Sample weighings measures • Need to assess two sources of variability (repeatability): • Between “weighings” • Instrumental. • Between weighings quantifies variability due to sample inhomogeneity and the sample preparation process. • Instrumental quantifies the variability associated with the instrumental measurement. Quantify a source of variability by determining its standard deviation.

15. Example Can use Analysis of Variance (ANOVA) to determine: Standard deviation for “weighing”, sw = 57.9 Standard deviation for instrument, s = 19.2 These values refer to the measured response (e.g. weight-corrected area)

16. Confidence interval for analysis Confidence interval for future number of weighings (n1) and measurements per weighing (n) is given by: a: degree of confidence (usually 0.05 for 95% confidence) N: number of degrees of freedom to determine sw and s. t: Students t-value for a and N. D: confidence interval for measurement (area)

17. Analytical Capability Number of measures per weighing The analytical capability, cp, changes with n1 and n.

18. External Standards : strength of external standard : average measure for external standard : average measure for sample : estimated strength for sample If D is the confidence interval for and , then the confidence interval for , i.e. if has an RSD of 0.7%, the RSD for the estimated strength is ~1.0%.

19. Practical consequences: finding result Out-of-Specification Consumers risk measures Producers risk

20. Dealing with OOS results • Can re-test samples. • On re-testing, FDA guidelines for industry state “if no …errors are identified in the first test, there is no scientific basis for invalidating OOS results in favour of passing re-test results.” • Scientifically, the issue of whether the re-test results “pass” or “fail” is of little consequence. The issue is whether the re-test results are statistically the same as the original OOS result. • Can use the t-test to assess the similarity between OOS and re-test.

21. Example 1 • Specification >97.0% • OOS result 96.5% with confidence interval +/- 2.1%. • Re-test 97.7% with confidence interval +/- 2.1%. • No evidence that the OOS and re-test are different from t-test. • Average the OOS and re-test gives 97.1% with confidence interval +/- 1.5%.

22. Example 2 • Specification >97.0% • OOS 96.0% with confidence interval +/- 0.9%. • Re-test 98.0% with confidence interval +/- 0.9%. • No evidence that the OOS and re-test are the same. • Cannot average the OOS and re-test result. • Consequently must doubt both results.

23. Conclusions • Understanding and determining the confidence interval associated with an analytical result is an important part of method development/validation. • The relationship between the confidence interval and the product specification is an important aspect of defining method fitness-for-purpose. • The analytical capability is quantifiable measure of fitness-for-purpose for precision. • Understanding the confidence interval is important during out-of-specification investigations.