Auditing & Quality Assurance

1. Auditing & Quality Assurance Igor Espinoza-Delgado, M.D. Gary Smith M.Sc. Cancer Therapy Evaluation Program

2. Clinical Trials Good Clinical Practice Health Care Delivery Research

3. Good Clinical Practice (GCP) An internationally recognized standard for the design, conduct, performance, monitoring, auditing, recording, analyses, and reporting of clinical trials. GCP provides assurance that the data and reported results are credible and accurate, and that the rights, integrity, and confidentiality of trial subjects are protected.

4. Regulatory Basis forMonitoring and Auditing 1962 Harris-Kefauver amendments to the food, drug, and cosmetic act required that FDA regulate the testing of new drugs as an investigational new drug (IND) prior to drug approval Demonstration of efficacy and safety in well-controlled clinical investigations

5. Regulatory Basis forMonitoring and Auditing (Cont.) FDA IND regulations (21CFR 312.50) requires sponsors to ensure proper monitoring of investigation(s), ensure that investigations are conducted in accordance with approved protocols as contained in the IND, and ensure that the FDA and all participating investigators are promptly informed of significant new adverse effects or risks with respect to the drug.

6. Quality Assurance All those planned and systematic actions that are established to ensure that the trial is performed and the data are generated, documented (recorded), and reported in compliance with GCP and applicable regulatory requirements.

7. Why Be Concerned With Quality? Patient Safety/Ethical Considerations Respect for Persons Beneficence: maximize possible benefits and minimize possible harms Regulatory Considerations Delay in product approval Approval withdrawals Sanctions

8. Why Be Concerned With Quality? Monetary Considerations $802 million: total average preclinical and clinical costs up to the time of receiving FDA marketing approval $897 million fully capitalized cost to develop a new drug, including cost of conducting post marketing surveillance studies after receiving regulatory approval (Tufts Center for the Study of Drug Development)

9. Why Be Concerned With Quality? Public Trust

12. Components of Quality Assurance Staff training/mentoring Quality control Data safety and monitoring Study monitoring Auditing

13. Monitoring The act of overseeing the progress of a clinical trial, and of ensuring that it is conducted, recorded, and reported in accordance with the protocol, standard operating procedures (SOPs), GCP, and applicable regulatory requirements.

14. Audit A systematic and independent examination of trial related activities and documents to determine whether the evaluated trial related activities were conducted, and the data recorded, analyzed and accurately reported according to the protocol, sponsor’s standard operating procedures, GCP, and the applicable regulatory requirements.

15. Quality Assurance Auditing vs. Monitoring Monitoring: • Continuous process • On-site and/or off-site • Includes all patients on a trial Auditing: • Snapshot in time • On-site process • Subset of patients on a trial

16. On-Site Audit Program Objectives: Assure accuracy and quality of data Assure compliance with Federal regulations Assure compliance with sponsor’s policies and procedures Assure compliance with GCP Serves as an educational tool

17. CTEP Monitoring and Auditing

18. Other Types of Audits Special audits: For cause audits Response audits

19. Key Differences Between NCI and Industry Model Industry Model: • Similar to NCI model • Phase 2 & 3 trials: • Sites undergo on-site monitoring visits every 6 weeks with 100% of Case Report Forms included in source data verification during monitoring visit. NCI model: • Phase 1 CTMS monitored trials • Phase 2 & 3 trials: • Monitoring via electronic submission of data quarterly via the clinical data update system (CDUS) and adverse event expedited reporting system (AdEERS).

20. Key Differences Between NCI and Industry Model (Cont.) Industry Model: • Audit unit is protocol based • >10% of sites are audited • Financial incentives NCI model: • Audit unit is institution based • >10% of patient cases are audited • Scientific incentives

21. Audit Preparation On the day of the audit: Provide overview of research procedures (scientific review committee, IRB) Provide overview of organization of the medical records and research charts Have data management staff available throughout the process if questions arise

22. Audit Preparation On the day of the audit (cont.): Meeting with principal investigator to review status of study Pharmacy inspection Exit interview with P.I. and staff

23. Audit Preparation What Is Needed? Regulatory: Documentation of initial IRB approval Documentation of continuing reviews Documentation of IRB approval of all amendments Documentation of IRB review of reportable adverse events and IND safety reports Documentation of other IRB correspondence pertaining to protocol

24. Audit Preparation What Is Needed? Regulatory (cont.): 1572s for Principal Investigator and Co-Investigator* Investigator CV/Medical License* IRB Membership List* Office of Human Research Protections (OHRP) Assurance Number Laboratory Certification* and Lab Normal Ranges *primarily industry requirement

25. Audit Preparation What Is Needed? Pharmacy: Drug Accountability Records as provided by the sponsor along with shipping receipts, return receipts, and transfer forms when applicable Appointment with pharmacy to conduct inspection

26. Audit Preparation What Is Needed? Patient medical records: ORIGINAL signed and dated informed consent (at time of enrollment as well as at time of re-consent, if applicable) Baseline history and physical exam Prior therapies including start and stop dates Pathology report Protocol required parameters (labs, x-rays, scans, EKG, etc.) Physician orders

27. Audit Preparation What Is Needed? Patient medical records (cont.): Medication administration records Nursing and physician progress notes Off-study note Research Records: Eligibility checklist Confirmation of registration including arm and/or dose assignment Documentation of collection/submission of research tests (pharmacokinetics, marker studies, etc.) Documentation of tumor measurements

28. Audit Preparation What Is Needed? Research Records (cont.): Outside labs and other study parameters Outside physician records and correspondence Appointment books Subject diaries/calendars Study flow sheets and other research records that are signed and dated on a real time basis by the health practitioner evaluating the patient Protocol or study road maps

29. Cooperative Group On-site Audit Program Classification of deficiencies: Major: any variance from protocol-specified procedures that makes the resulting data questionable Lesser: any variance that is judged to not have a significant impact on the outcome or interpretation of the study data

30. Unacceptable Audits Possible Actions: Re-audit Suspension of patient registration Suspension of Investigator’s 1572 Problematic audit findings may be referred to other agencies, such as the FDA, the Office for Human Research Protections (OHRP), or the Office of Research Integrity (ORI) when warranted

31. Most Frequently Occurring Deficiencies: IRB Reapproval delayed less than 30 days Reapproval delayed greater than 30 days but less than one year Lack of documentation of full IRB approval of a protocol amendment that affect more than minimal risk Delayed reapprovals for protocols closed to accrual for which all patients have completed therapy Missing reapproval

32. Most Frequently Occurring Deficiencies: Consent Content Lack of disclosure of all risks or side effects contained in model informed consent approved by NCI Lack of disclosure of approximate number of participants Lack of disclosure of extent of confidentiality of records Lack of disclosure of the contact person for research questions, information regarding subject’s rights, and/or contact for research-related injury Failure to disclose circumstances in which subject’s participation may be terminated by investigator without subject’s consent

33. Most Frequently Occurring Deficiencies: Pharmacy NCI drug accountability record forms (DARFs) incomplete or inaccurate NCI DARFs not protocol and drug specific Satellite NCI DARFs not accounted for NCI DARFs not kept as primary transaction record

34. Most Frequently Occurring Deficiencies: Informed Consent Consent form does not contain all required signatures or dates Consent form does not include updates or information required by IRB Consent form used was not current IRB-approved version at time of patient registration Consent form not signed and dated by patient Consent form signed after patient started on treatment

35. Most Frequently Occurring Deficiencies: Eligibility Documentation missing; Unable to confirm eligibility Review of documentation confirms patient did not meet all eligibility criteria as specified by the protocol

36. Most Frequently Occurring Deficiencies: Treatment Treatment doses incorrectly administered, calculated or documented Dose deviations incorrect (greater than +/- 10%) Dose modifications unjustified Unjustified delays in treatment Additional agent(s)/ treatment given prohibited by protocol

37. Most Frequently Occurring Deficiencies: Response Tumor measurements/evaluation of status or disease not performed according to protocol Claimed response (partial response, complete response, etc.) cannot be verified Protocol-directed response criteria not followed Inaccurate documentation of initial sites of involvement Failure to detect cancer (as in a prevention study) or failure to identify cancer progression

38. Most Frequently Occurring Deficiencies: Toxicity Follow-up studies necessary to assess toxicities not performed Grades, types or dates/duration of serious toxicities inaccurately recorded Recurrent under or over reporting of toxicities Failure to report a toxicity that would require filing an adverse event report Reported toxicities cannot be substantiated in source documents

39. Most Frequently Occurring Deficiencies: General Data Quality Errors in submitted data Delinquent data submission Recurrent missing documentation, e.g., charts Protocol-specified laboratory tests not documented

40. Quality Improvement General Record Keeping: Remember, if it is not documented, it cannot be verified as being done Follow standard procedures for documentation and error correction, including dating and initialing entries

43. Quality Improvement “The myth of perfect performance of the health care worker is unrealistic. We must change to an engineering mode of thinking that things will always go wrong. We need to protect the patient from results of errors, or failure of safe design. We not only need a ‘fail-safe’ design, but redundancy in the system, because a backup system protects the process. The system must possess two independent redundant steps, not interdependent steps.” (Quote from Richard J. Croteau, M.D., JCAHO executive director for strategic initiatives)9