OOS, OOE, OOT, OOL and correct decision making

1. OOS, OOE, OOT, OOL and correct decision making Dr Christopher Burgess Burgess Consultancy

2. Scope • Analytical processes & their capabilities • Defining suspect results • FDA Guidance • Statistical considerations • Trend analysis • Root cause investigations & CAPA • Introduction to the Workshop

3. Analytical Process Capability, Cp Assume the method is unbiased ie mean of samples tends to the true value and s is our estimate of the standard deviation derived from method validation studies.

4. Analytical Process Capability, Cp Assume s=2% is the intermediate precision estimate derived from an HPLC method validation study and the span for the LTL to the UTL is 10% (ie 95 to 105%).

5. Analytical Process Capability, Cp

6. HPLC result distributions for a single sample

7. HPLC result distributions for s=2% for singlet, duplicate and triplicate determinations at 100%

8. HPLC result distributions for s=2% for singlet, duplicate and triplicate determinations at 98.5%

9. Laboratory Sample Test Sample Test Portion Dispense & weigh Test solution Iterate in accordance with method Sample preparation Aliquot Analytical Measurement Calculation of test result(s) and reportable value(s) Data output; recording and reporting Manufacturing Process THE ANALYTICAL PROCESS

10. What is a result? • Raw data (analyte response function) • Intermediate value or result • Analytical value or result • Reportable value to be compared with the specification

11. OOS, OOE or OOT • Method specifies 3 individual analytical measurements () • The reportable value is defined as the average of these three measurements () • Upper and lower specification limits; (USL & LSL) • The range of these three individual measurements is specified based on the method capability; Upper and lower individual limits (UIL & LIL) • Two criteria (means and individuals)

12. UIL    USL                    LSL   LIL OOE/OOT OOS (OOL) ?  ?     

13. Guidance for IndustryInvestigating Out-Of-Specification Results (OOS) Test Results for Pharmaceutical Production October 2006 http://www.fda.gov/cder/guidance/3634fnl.htm

14. Scope • Chemistry-based laboratory testing of CDER-regulated drugs, including CDER-regulated biologic drugs, as applied in traditional methods of batch testing and release (includes contract laboratories) • All test results that fall outside specifications or acceptance criteria, including in-process laboratory tests (one exception: guidance does not address PAT approaches to testing and release). • APIs, excipients, in-process materials, components as well as finished drugs • Does not apply to biological assays • Although recommendations are intended for OOS results, the same investigation principles may applied to Out-of-Trend (OOT) results

15. Reportable results The term reportable result as used in this document means a final analytical result. This result is appropriately defined in the written approved test method and derived from one full execution of that method/procedure, starting from the (original) sample.

16. Reportable result; Type 1 A test might consist of a specific number of replicates to arrive at a [reportable] result. For instance, an HPLC assay result may be determined by averaging the peak responses from a number of consecutive, replicate injections from the same preparation (usually 2 or 3). The assay result would be calculated using the peak response average. This determination is considered one test and one [reportable] result.

17. Reportable result; Type 2 In some cases, a series of complete tests (full run-throughs of the test procedure), such as assays, are part of the test method. It may be appropriate to specify in the test method that the average of these multiple assays is considered one test and represents one reportable result. In this case, limits on acceptable variability among the individual assay results should be based on the known variability of the method and should also be specified in the test methodology.

18. Long-standing FDA Principles • OOS results cannot be disregarded or negated without a documented investigation that clearly demonstrates the cause to be laboratory error • If retesting is performed because the original OOS result is suspect (not confirmed) the number of retests needs to be specified before the analyses begin • Resampling should be performed only if evidence indicates that original sample was compromised or not representative

19. Long-standing FDA Principles • Averaging should not be used to hide variation in individual test results • Relying on the average of OOS and in-specification results is misleading • The invalidation of results obtained from Biological assays of high variability via use of outlier tests is to be used sparingly, can introduce “a serious source of bias,” and is not applicable to chemical assays. [USP 28]

20. Sampling distribution of the mean

21. Confidence intervals for the population If we have a large number of measurements, n, and the method is unbiased then we can calculate the 95% confidence interval (CI) containing the true value

22. Confidence Interval of the Mean

23. Small samples & the t distribution • CIs from small numbers are underestimates • t distribution values are used in place of the normal distribution value for  confidence • n-1 degrees of freedom

24. Confidence limits for the standard deviation

25. Trend analysis in EU GMP 1.5Regular periodic or rolling quality reviews of all licensed medicinal products, including export only products, should be conducted with the objective of verifying the consistency of the existing process, the appropriateness of current specifications for both starting materials and finished product to highlight any trends and to identify product and process improvements. 1.5 (vii)A review of the results of the stability monitoring programme and any adverse trends. 6.9 For some kinds of data (e.g. analytical tests results, yields, environmental controls) it is recommended that records are kept in a manner permitting trend evaluation. 6.32Out of specification or significant atypical trends should be investigated. Any confirmed out of specification result, or significant negative trend, should be reported to the relevant competent authorities. The possible impact on batches on the market should be considered in accordance with chapter 8 of the GMP Guide and in consultation with the relevant competent authorities.

26. Trend Analysis • HPLC • resolution data, Rs • 25 runs • One system suitability test at the start • One system suitability test at the end • From the method validation report: • Rs mean 2.05 but not less than 1.75 • Rs range not greater than 0.5

27. LIMIT Conventional Shewhart Chart (Means) Minitab v14.13

28. LIMIT Conventional Shewhart Chart (Range) Minitab v14.13

29. CuSum Analysis • Calculates the cumulative effect of the values from a target value • The target value can be the mean or aspecification limit • Calculate the difference from each individual value and the target (in our example 2.05) • Calculate the cumulative sum of these differences

30. Rs CuSum plot

31. Rs CuSum plot • The change in slope of the plot is the essential feature • If the plot • Has no apparent slope then there is no trend from the target value • Moves down it indicates a negative trend from the target value • Moves up it indicates a positive trend from the target value • The steeper the slope the greater the trend

32. Rs CuSum plot ?

33. Rs CuSum plot • The plot has major changes of slope at runs 15 and 19 (is 5 a possibility?) • Are they significant? • Can they be associated with changes? • Operator • Equipment • Column • Standard batch • Reagents etc

34. Finding root causes for AARs

35. Laboratory Data Quality Management

36. Phase I: Laboratory Investigation Two stages; • Initial analyst supervisor discussion • If root cause not apparent, formal QA driven investigation

37. Phase II: Full Scale Investigation [Failure Investigation] • Review of manufacturing process and production events as possible root causes • May also require additional laboratory work (i.e. retesting or resampling)

38. Workshop Process • In your work groups, read the allocated scenarios and ensure understanding. • Examine the 4 Process flows from the SOP on pages 2 to 5 and discuss as a group. • Based upon that group understanding of each scenario; • Identify the key issues and link them to the stages in the process flow. • List any deviations you think occurred. • List the documentation which you would have expected to find • Discuss the roles and responsibilities of those involved • Do you agree that the action taken was appropriate? • Generate a CAPA action list.