Out of Specification Results (OOS) A One Day Workshop

1. Out of Specification Results(OOS)A One Day Workshop Presented by: Karen S. Ginsbury PCI Pharmaceutical Consulting Israel Ltd. For IFF March 2007

2. Purpose of Workshop • Understand Current Industry Practice asit relates to all aspects of handling: • Out of Specification • Unusual • Out of Trend Results • When is retesting legitimate • When does testing have to stop • When do the authorities need to be notified PCI Pharmaceutical Consulting Israel Ltd

3. 09:00 – 10:30 10:30 – 11:00 11:00 – 12:30 12:30 – 13:30 13:30 – 15:00 15:00 – 15:15 15:15 – 16:30 Background and Overview: Barr and Able Coffee Break The Making of the FDA Guide Case study: what NOT to do Laboratory investigation and checklist Lunch Break FDA Guide Continued: Extended investigation, retesting protocol Qualitative and quantitative tests Case study Tea Break Case Study and workshop wrap-up Reporting results; reporting to regulators Workshop Structure PCI Pharmaceutical Consulting Israel Ltd

4. PART 1 – BARR and ABLE • Litigation • USA regulatory environment aroundlate 1980’s through 1993 • Current regulatory environment in US • Able - 2005 PCI Pharmaceutical Consulting Israel Ltd

5. Some Definitions • What is an OOS Result? • What is a test result • Definition of reportable value • “A reportable value is the end result of the complete measurement method as documented. It is the value compared with thespecification, the values collected when the term replicates is used, the values used for official reports, and the values used for anystatistical calculation or analysis.” • Torbeck (February 1999, Pharmaceutical Technology) PCI Pharmaceutical Consulting Israel Ltd

6. Before Barr – Current Practice • Prior to 1993 and the court decision – it wasCOMMON practice to retestonceor in exceptionally good companies twiceand to release the batch if the retest result waswithin the specification • Companies had not really thought about thepractice • But then…nor had the regulators PCI Pharmaceutical Consulting Israel Ltd

7. The Barr Court Case and Judge Wolin From the New York Times February 6, 1993, Saturday (AP); Financial Desk COMPANY NEWS; Judge Rules On Barr Labs A generic drug manufacturer must recall batches ofsome of its medicines and stop distributingothers until the company completes studies of itsmanufacturing process, a Federal judge ruled onThursday. But United States District Judge Alfred M. Wolin refused a request by Federal pharmaceuticalregulators to order a complete shutdown PCI Pharmaceutical Consulting Israel Ltd

8. Barr and OOS • Faced with potential closure, the companytook FDA to court • The judge went into great details as to themeaning and implications of OOS results • The outcome: FDA draft guidance: 1998 • FDA final guidance: 2006 PCI Pharmaceutical Consulting Israel Ltd

9. Barr: What happened in court • The judge heard experts onbehalf of FDA and Barr regardingthe practice of retesting • FDA wanted retesting to be bannedunder all circumstances • After a long hearing at which five industry experts, anFDA investigator, and several company employeestestified, Judge Alfred M. Wolin, U.S. District Judge forthe District of New Jersey, issued a 79-page opinion PCI Pharmaceutical Consulting Israel Ltd

10. The Barr Court Case 1993 • Reported problems include • misplaced records • test data recorded on scrap paper • failure to control manufacturing steps such asthose governing products' physical properties • release of products not meeting theirspecifications • inadequate investigation of failed products PCI Pharmaceutical Consulting Israel Ltd

11. Barr: “Testing into Compliance” • Barr had numerous failures • Performed retests with • no investigations • no regard for process and product history • Tested until results met specifications • Then irrespective of previous OOS resultsfor the batch, released productreporting only the passing results Q: How do you report passing OOS’s on COA? PCI Pharmaceutical Consulting Israel Ltd

12. Reading the Judgment • Reading the Barr Court Judgment is likereading FDA’s draft guidance and pretty similar to thefinal guidance • Judge Wolin preferred to use the term"out-of-specification" (OOS) laboratory results ratherthan the term "product failure" which was morecommon to (preferred by?) FDA's investigators • Ruled that an OOS result identified as laboratory error by a failure investigation or an outlier test, orovercome by retesting is not a product failureBUT • Limited situations where laboratory error could be used PCI Pharmaceutical Consulting Israel Ltd

13. Guide to Inspection: QC Labs • Issued July 1993 (must have been working onit while the court case was ongoing) • Addresses OOS results and instructsinspectors to be alert • “Evaluate the company's system to investigate laboratory test failures. These investigationsrepresent a key issue in deciding whether aproduct may be released or rejected and formthe basis for retesting, and resampling “ • (Most of the information is now in FDA’s guide) PCI Pharmaceutical Consulting Israel Ltd

14. Guide to Inspection: QC Labs • OOS results fall into three categories: • laboratory error • non-process related or operator error • process related or manufacturing process error • Evaluate the company's retesting SOP for compliance withscientifically sound and appropriate procedures • A very important ruling in one recent court decision sets forth aprocedure to govern the retesting program • This district court ruling provides an excellent guide to use inevaluating some aspects of a pharmaceutical laboratory, butshould not be considered as law, regulation or bindinglegal precedent • The court ruled that a firm should have a predetermined testingprocedure and should consider a point where testing ends andproduct is evaluated. If results are not satisfactory, product isrejected. PCI Pharmaceutical Consulting Israel Ltd

15. After Barr – Current Industry Practice 2007 • ALL pharmaceutical companies in developedcountries and many in less developedcountries have SOPs for handlingOut of Specification results • There are still many investigational findingsconcerning out of specification results • There are still numerous issues, particularlywith transparency:What do you report on the COA? • Able laboratories suspended activity in 2005because of Out of Specification results PCI Pharmaceutical Consulting Israel Ltd

16. When it all goes wrong….Able 2005 The Quality Unit failed to: • Review computer audit trails in the Waters Empower Data Acquisition System • Provide adequate training to analytical chemists These practices led to: • The QU releasing batches failing in-process, finished product and stability specifications • Submission of erroneous data in Annual Reports and Prior Approval Supplements • Ceasing manufacture, distribution and recall of allproductsas of 13 May 2005 and withdrawal of atleast 5 ANDAs PCI Pharmaceutical Consulting Israel Ltd

17. Resample, Re-injection, Reprocessing PCI Pharmaceutical Consulting Israel Ltd

18. Time for a Break

19. From Able Laboratories 483 • Notebooks and binders lacked data from all testing conducted in the QC Laboratory. • Records did not include all data such as OOSresults, chromatograms, sample weights, andprocessing methods. • OOS results were substituted by passing results by Analysts and Supervisors. • The substitution of data was performed by cutting and pasting of chromatograms, substituting vials, changing sample weights and changingprocessing methods PCI Pharmaceutical Consulting Israel Ltd

20. That isn’t relevant…it’s fraud • Think about the following scenariobefore saying it couldn’t happen: • You are just starting up your HPLC system inthe morning • You inject 5 replicates of standard • The first four give perfect responses • The fifth is way off / makes no sense,obviously incorrect • What do you do? PCI Pharmaceutical Consulting Israel Ltd

21. It couldn’t happen in my company • A split second wrong judgment can appear asfraud to the inspector • You inject one more standard injection and itgives a perfect response • Clearly you were right and the original “5th”injection was dirty glassware or mis-injected,or, or, or…… PCI Pharmaceutical Consulting Israel Ltd

22. It couldn’t happen in my company • But you don’t want to show 4 good injections, one lousy and another good one • So you cut out the bad one and documentonly the good ones in the notebook • Sounds “horrendous?” • It has been done, probably in some of yourlaboratories! Can you be sure none of youranalysts ever did this?How can you be so sure? PCI Pharmaceutical Consulting Israel Ltd

23. GMP and OOS Results Do you know where the term OOS appears in • EU GMP regulations it didn’t as of June 2006 it does • US GMPs it doesn’t • Q7A GMP for APIs it DOESbecause Q7A was written after 1993 PCI Pharmaceutical Consulting Israel Ltd

24. Q7A on OOS PCI Pharmaceutical Consulting Israel Ltd

25. Ever heard this…. “The Lab don’t know how to test” “Give it to Pete… he knows how to do that test… Dave always gets badresults!” PCI Pharmaceutical Consulting Israel Ltd

26. OOS Case Study #1 • Complex biochemical assay: 6 replicates • Inherent variability allows for wider than usual specification of 80 – 120% • Results: 45, 50, 46, 52, 65, 69% • What would you think if it happened to you? • The lab technician has 20 years seniority • No other technician is familiar with the test PCI Pharmaceutical Consulting Israel Ltd

27. OOS Case Study #1 - continued • “Must be my mistake!” • WRONG • Analyst retested (not in accordance with SOP) • Results: 72, 69, 81, 80, 82, 81% • NOW what would you think? PCI Pharmaceutical Consulting Israel Ltd

28. OOS Case Study #1 - continued • “Results: 72, 69, 81, 80, 82, 81% • 72 and 69% must be “OUTLIERS” • Average 81, 80, 82 and 81 and result passes • Batch can be released • Report only this set of results PCI Pharmaceutical Consulting Israel Ltd

29. OOS Case Study #1 - continued • The outcome….. • Product complaints from patients and doctors:Product is sub-potent • Litigation • Product recall • Investigation reveals weighing error inproduction PCI Pharmaceutical Consulting Israel Ltd

30. What is the purpose of testing? • To find out the value of a specific parameter • To assess if that parameter meets thepre-determined specification for each lot • So as to make a sound scientific judgmentregarding product release or rejection • What happens in your company when there is an OOS result? • IDEALLY the analyst doesn’t know thespecification because of “BIAS.” PCI Pharmaceutical Consulting Israel Ltd

31. Is it possible to ensure a correct result? • Not at a 100% certainty level • Just as it is not possible to prevent an incorrectresult (at the 100% certainty level) • What is a correct result? • A result that is identical to the true result….BUT • When testing, we NEVER know the true result PCI Pharmaceutical Consulting Israel Ltd

32. Consequences of an Incorrect Test Result? • FALSE NEGATIVE Product declared fit for use when not Side Effects Death • FALSE POSITIVE Product declared unfit for use when fit Rejection Financial Loss PCI Pharmaceutical Consulting Israel Ltd

33. Is it possible to ensure a correct result? • It is possible to ensure a result as close aspossible to the true result • Using a quality assurance program in thelaboratory • Validate methods • Qualify analysts • Follow methods as written • Qualify, calibrate and maintain equipment • Report deviations / malfunctions PCI Pharmaceutical Consulting Israel Ltd

34. Types of Tests • Quantitative: assays and some limits tests • Qualitative: e.g. sterility test, appearance • Chemical • Physical • Microbiological PCI Pharmaceutical Consulting Israel Ltd

35. Chemical Testing • Inherently reliable • Precision is usually considerably better thanfor microbiolgical and biochemical testing • Outlier testing is forbidden by the FDA guidefor chemical testing(usually have less replicates anyway) • Don’t forget case study #1 – How NOT tohandle OOS results PCI Pharmaceutical Consulting Israel Ltd

36. Physical Tests • e.g.1 Black spots in powder • e.g.2 Fill volume are particularly problematic, since results arealmost certainly correct.Is there any place for retesting?In e.g. 1 Probably notIn e.g. 2 Maybe Is there any place for resampling? PCI Pharmaceutical Consulting Israel Ltd

37. FDA OOS Guidance, October 2006 • Draft from September 1998 for comments • Was confusing and open to misinterpretationparticularly by persons not familiar with theindustry • Final guidance is still confusing but a lot better thanthe draft • The draft was referenced, as was the Barr court case during inspections, so the final guidance will definitelybe used • Should be considered in preparing an OOS SOP PCI Pharmaceutical Consulting Israel Ltd

38. FDA Guide on OOS Results • Finalized October 2006 • Contains “nonbinding recommendations” • Requires that an investigation is performedANYTIME that an OOS is obtained • Wants to include ALL suspect results • Wants an SOP stating number of retestsUPFRONT PCI Pharmaceutical Consulting Israel Ltd

39. Table of Contents • Introduction • Background • Identifying and Assessing OOS Results – Phase I: Laboratory Investigation • Responsibility of Analyst • Responsibilities of Laboratory Supervisor PCI Pharmaceutical Consulting Israel Ltd

40. Table of Contents (continued) • Investigating OOS Results –Phase II: Full Scale OOS Investigation • Review of Production • Additional Laboratory Testing • Reporting Testing Results • Concluding the Investigation • Interpretation of Investigation Results • Cautions • Field Alert Reports PCI Pharmaceutical Consulting Israel Ltd

41. Out of Specification Results • IF you don’t know the specification youshould never have an OOS, but we don’tlive in an ideal world! • HOW can you have an OOS if: • the method is validated • analysts are qualified • equipment is calibrated andwell-maintained • notebooks have full record of testing PCI Pharmaceutical Consulting Israel Ltd

42. How to Identify OOS Results • Compare result with specification • Be sure that specification is to required degree of accuracy and round the result to this valueE.g. spec 95 - 105 94.9 passes spec 95.0 - 105.0 94.9 fails • Message for your R&D Department:think carefully before setting specifications!!! PCI Pharmaceutical Consulting Israel Ltd

43. How to Identify OOT Results • Out of Trend or unusual results are generallyresults that: • MEET the product specification • ARE outside the control limits of the process,where control charts are usedor • ARE different to results usually obtained(e.g. spec: 95.0 – 110.0usual results: 98.5 – 101.0OOT result: 96.4 PCI Pharmaceutical Consulting Israel Ltd

44. Initial Assessment of OOS Result • Bear in mind prior: • Product history • Process history • Test history • Reliability of equipment • Reliability of the analyst • Precision of the test (validation) PCI Pharmaceutical Consulting Israel Ltd

45. Out of Specification Results • Failure of a control is NOT an OOS eg: • Standard shows impurity peak and assays atincorrect retention time • Standard shows assay of 80% • System suitability drift • In this case all test results SHOULD BE INVALIDATEDand the test REPEATED using an additional aliquot ofthe same preparation if possible, or an additionalaliquot from the same sample (What if product degrades and this is not possible? – need to defineduplicate sampling procedure so that always havespare material) • Data collected is retained on file ANYWAY • What happens if results are OOS? Do you need to erron the side of caution e.g. possible homogeneity issue? PCI Pharmaceutical Consulting Israel Ltd

46. OOS – Laboratory Investigation • IF you don’t question an “in-spec” result, whydo you question an OOS? • Must have a Retesting Policy: • Laboratory investigation • equipment -e.g. glassware, calibration, maintenance • question large differences between replicates • calculations • Product history etc. etc. etc. (to be continued) • 4M’s: man, machine, methods, materials PCI Pharmaceutical Consulting Israel Ltd

47. Time for Lunch

48. FDA Guide: Introduction • Applies to chemistry – based testing[KSG: primarily HPLC and GC] • Includes in-process testing except (footnote), where the purpose is to prevent process drift[KSG: discuss – could still have OOS!] • Principles apply to CONTRACT firms • Timely, unbiased investigation • [KSG: OOS Policy is major SOP approved bysenior management] PCI Pharmaceutical Consulting Israel Ltd

49. FDA: Laboratory Investigation • The source of OOS should be identified as: • Aberration of measurement process • Aberration of manufacturing process[KSG: sampling process?] • Even if the batch is rejected, an investigation isrequired: • Other batches involved? • Other products involved? • Need written investigation, conclusions, follow-up PCI Pharmaceutical Consulting Israel Ltd

50. FDA: Laboratory Investigation • Assess accuracy of lab data: • Before test preparations are discarded(composite / homogeneous source of aliquot tested) • Hypothesis testing using same test preparationfor laboratory error or instrument malfunction • IF no meaningful errors were made [found?],conduct a full scale OOS investigation • Contract lab conveys data and findings to you PCI Pharmaceutical Consulting Israel Ltd