Enhancing Data Integrity in Clinical Studies: Audit and Correction Training

2. Findings Noted During Audits and How to Correct Them Training for Investigators and Research Personnel

3. Introduction “Conducting clinical studies is a complex endeavor, involving oversight of clinical investigators with respect to the protocol, Good Clinical Practices (GCP), governing regulations, conditions of Institutional Review Boards and/or Ethics Committees, and institutional Standard Operating Procedures before, during and after conduct of the study.” http://cdn2.hubspot.net/hub/149400/docs/auditingvsmonitoring.pdf

4. Study Data “The study data that are generated must be of the highest quality; data must be accurate and evaluable in support of marketing clearance/ product approval and collected in a manner that protects the rights, safety and welfare of properly consented trial participants.” http://cdn2.hubspot.net/hub/149400/docs/auditingvsmonitoring.pdf

5. Data Integrity “‘Data integrity’ refers to the soundness of the body of data as a whole. In particular, the body of data should be credible, internally consistent, and verifiable. Quality and integrity are both essential for data to be relied upon for regulatory decision-making. Data quality and integrity are achieved when each piece of data is collected in accordance with the study protocol and procedures, giving attention to each of the quality characteristics above, and subsequently handled (e.g. transcribed, analysed, interpreted, reported) so that the quality characteristics of the original data (i.e. accuracy, legibility, completeness, etc.) are preserved.” http://apps.who.int/iris/bitstream/10665/43392/1/924159392X_eng.pdf?ua=1

6. The 5 Elements of Data Integrity Readable and signatures identifiable. It should be clear who has documented the data. The information should be documented in the correct time frame along with the flow of events. Original, if not original should be exact copy; the first record made by the appropriate person. The investigator should have the original source document. Accurate, consistent and real representation of facts. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3121265/

7. To ensure that data integrity is upheld, audits are conducted The Extra Elements of Data Integrity Long-lasting and durable. Easily available for review of treating physicians and during audits/inspections. The documents should be retrievable in reasonable time. Complete till that point in time. Demonstrate the required attributes consistently. Based on real and reliable facts. The data should be backed up by evidence. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3121265/

8. What is the difference between an audit and monitoring? “Monitoring is defined as ‘the act of overseeing the progress of a clinical trial, and ensuring that it is conducted, recorded, and reported in accordance with the protocol, standard operating procedures(SOPs), GCP, and the applicable regulatory requirement(s)’[ICH 1.38]” “Monitoring is a quality control function where study conduct is routinely assessed on an on-going basis at every step of the trial.” “Auditing, a quality assurance function, is an independent, top-down, systematic evaluation of trial processes and quality control.” http://cdn2.hubspot.net/hub/149400/docs/auditingvsmonitoring.pdf

9. There are two types of audits: Routine audits For-cause audits What is an audit? “A systematic and independent examination of trial-related activities and documents to determine whether the evaluated trial-related activities were conducted, and the data were recorded, analyzed, and accurately reported according to the protocol, sponsor’s standard operating procedures (SOPs), good clinical practice (GCP), and the applicable regulatory requirement(s).” http://elpaso.ttuhsc.edu/research/committees/irb/resources/_documents/HRPP%20Manual%20El%20Paso%20October%2024%202016.pdf

10. What triggers a routine audit? Trials that are audited on a routine basis are selected based on the following but are not limited to these criteria: new investigators to the institution new coordinators to the institution research in departments which have had previous compliance concerns research conducted by investigators with previous compliance concerns research involving new sponsors to the institution research new to the institution http://elpaso.ttuhsc.edu/research/committees/irb/resources/_documents/HRPP%20Manual%20El%20Paso%20October%2024%202016.pdf

11. What triggers a for-cause audit? Trials audited on a for-cause basis are done so based upon the receipt of an allegation or a request from the IRB to review the research. Potential triggers may also contribute to a for-cause audit and include, but are not limited to: PI’s with prior adverse events; Novel or new interventions in a biomedical study; Investigators submitting protocols requiring expedited or full board review who have no prior research experience; Especially high risk protocols (as determined by the IRB); Protocols involving especially high risk/vulnerable populations and/or groups highly susceptible to coercion; Protocols that substantially overlap with major Privacy Rights statutes, such as HIPAA or FERPA; A protocol to be conducted over an unusually long period of time; PI’s who are chronically late in filing for continuing review; and/or PI’s who submit multiple drafts of informed consent forms; PI’s who submit informed consent forms which clearly do not apply to the study being reviewed; or PI’s who submit informed consent forms from other sites or facilities. http://elpaso.ttuhsc.edu/research/committees/irb/resources/_documents/HRPP%20Manual%20El%20Paso%20October%2024%202016.pdf

12. What is an audit finding? “…the results of an evaluation of the collected audit evidence against audit criteria. It states that findings can indicate conformity or nonconformity with audit criteria, or opportunities for improvement. So, findings can also be good. It’s just that audit reports tend to focus on findings that are nonconformities.” https://www.whittingtonassociates.com/2001/11/nonconformity-nonconformance-defect-finding-or-observation/

13. What study criteria is reviewed during an audit? observation of the consent process observation of the data collection process regulatory file for required elements including the presence of required documentation; protocol and amendments; approved consent forms and IRB documentation. subject eligibility informed consent documentation unanticipated and adverse event reporting accuracy and completeness of data collection sheets confidentiality of records drug and device handling and accountability laboratory data study progress reports contacting of research subjects study advertisements and recruiting information research instruments request that the PI(s) submit what data or analysis has been done to date to the IRB for review any additional information determined necessary by the research compliance officer. http://elpaso.ttuhsc.edu/research/committees/irb/resources/_documents/HRPP%20Manual%20El%20Paso%20October%2024%202016.pdf

14. What occurs during, before and after an audit? “The PI is generally notified of the audit at least two weeks in advance and is provided with a scheduled date and time. The date may be adjusted to work with the PI’s schedule, but the audit may not be postponed for more than 30 days. Each audit has a targeted educational component that teaches research faculty and staff how to correct their mistakes and prevent the same mistakes in the future. An audit report is generated from each audit investigation and is submitted to the PI, the Department Chair, and the IRB and Compliance representatives. If applicable, the audit report can include a list of findings as well as recommendations to the IRB regarding the findings and the nature (minor or serious/continuing or non-continuing) of the non-compliance. The audit report recommendations will include possible corrective actions that the principal investigator, study staff, and/or the IRB might take to correct any problems reported in the findings.” http://elpaso.ttuhsc.edu/research/committees/irb/resources/_documents/HRPP%20Manual%20El%20Paso%20October%2024%202016.pdf

15. What are some possible corrective actions that may result from an audit? Possible corrective actions recommended may include, but are not limited to: request for more information before a final decision can be made; protocol or informed consent document changes; changes to, or outside monitoring of the informed consent process; suspension or termination of IRB approval of the study; more frequent review by the IRB; follow-up audits to be conducted on a regular basis for a specified period of time; additional training or certification of the PI and the research staff; disqualification of the PI or members of the research staff from conducting research at TTUHSC El Paso; disallowance of research use of data collected; notification of current and/or past research participants regarding study problems; re-consent of current study participants; notification of other TTUHSC El Paso committees or administrators; or notification of outside entities (DHHS, FDA, NIH, study sponsor) of the compliance issues. http://elpaso.ttuhsc.edu/research/committees/irb/resources/_documents/HRPP%20Manual%20El%20Paso%20October%2024%202016.pdf

16. YES! Has the Research Compliance Officer seen many findings in regards to the clinical studies on our campus?

17. 2015 Audit Findings

18. 2016 Audit Findings

19. 2017 Audit Findings

20. 2015 - 2017 Audit Findings

21. 2015 - 2017 Audit Findings (41 Audits)

22. What are some specific examples of audit findings? Regulatory Binder Findings: Incomplete logs (ex: enrollment logs) Missing logs (ex: training logs, IP logs) Unorganized logs (ex: sign-in logs) No regulatory binder or file present Missing current and historical regulatory documents (ex: protocols, informed consents) Missing data collection forms Missing items listed on iRIS or found in the subject file (ex: questionnaires) Not maintaining appropriate documentation (NTF) of where IP or samples are being kept Not maintaining appropriate documentation (NTF) of where other parts of the regulatory binder are kept No principal investigator and/or co-investigator CVs or medical licenses Principal investigator and/or co-investigator CVs not signed and dated Principal investigator and/or co-investigator CVs not signed and dated more than 3 years ago Missing IRB outcome letters Missing or expired IBC, UMC, or EPCH approvals

23. What are some specific examples of audit findings? iRIS Findings: Missing documents (ex: protocols, informed consents) Missing data collection forms Missing items in “Other Study Documents” found in the regulatory binder or subject file (ex: questionnaires) Expired CITI training (Biomedical Investigator and Conflict of Interest) and expired financial disclosures Missing or expired IBC, UMC, or EPCH approvals Missing or expired agreements to conduct recruitment at other institutions Missing or incorrect information on the Study Application Previous research personnel still listed on study Spanish versions of documents not present in “Other Study Documents” Missing approvals from other institutions or physicians to recruit subjects through those institutions Missing letters sent to subjects (ex: lost to follow-up)

24. What are some specific examples of audit findings? Informed Consent Form and HIPAA Findings: Expired consents used to consent subjects Unapproved (not IRB stamped) consents and HIPAA used to consent subjects Missing HIPAA on studies with PHI Subjects and/or research personnel printing and signing on the wrong line Subjects and/or research personnel printing date and/or time on the wrong line Subjects and/or research personnel forgetting to print or sign Subjects and/or research personnel forgetting to print date and/or time Subjects forgetting to answer questions throughout consent Subjects forgetting to initial where required Subjects and research personnel signing consents hour(s) apart Research personnel signing the consents several minutes to hour(s) before the subject White-out used on informed consents Corrections not lined-through, initialed and dated by subject or research personnel Unauthorized individuals signing informed consents in place of research personnel

25. What are some specific examples of audit findings? Informed Consent Form and HIPAA Findings: Unnecessary witness signature added to consent Incorrect or previous TTUHSC El Paso Informed Consent template used for informed consents Study procedures conducted prior to consenting Subject not receiving copy of their consent Principal investigator not maintaining original copy of consent Subjects signing the wrong type of consent (ex: regular consent vs. parental consent) Enrolling the wrong subject onto a study (ex: Having parent sign to enroll child when the parent is the subject) Not using an assent for studies that involve minors Not properly documenting the informed consent process depending on the circumstances involved Not including the ClinicalTrials.gov statement on the informed consent form for studies that are registered with ClinicalTrials.gov. Scribbles on the consent

26. What are some specific examples of audit findings? Subject File Findings: Missing documents specified on protocol, ICF or application (ex: data collection forms, questionnaires) Data collection forms with missing information required by protocol Spanish versions of documents not used for subjects that consented in Spanish Records not kept together Missing records or no original records available Keeping records from subjects on different studies in one subject file Collecting extra information that the principal investigator does not have approval to collect No subject files Lack of supportive documentation for procedures or study visits Lack of supportive documentation for events that occurred (ex: subject withdrawal, missed visits, phone calls, letters, note-to-file) Lack of proper documentation in regards to subject receiving or not receiving compensation White-out used on study documents Unnecessary PHI collected through EMR principal investigator notes

27. What can research personnel and staff do to prevent from having audit findings? Develop a risk assessment plan to determine which studies are at a higher risk of having more findings Perform random quality control and quality assurance checks on all of your studies as your studies progress Create a monitoring plan that includes review of most if not all the following areas if applicable: informed consent documents eligibility criteria protocol compliance source document verification for data accuracy query resolution (clarification or correction of inaccurate data) occurrence and reporting of adverse events test article accountability maintenance of essential documents oversight of the Clinical Investigator and IRB Items on iRIS and regulatory binder Ensure that all your studies are monitored at least once every quarter. Learn your forms, how they work, and what needs to be filled out vs. what should be left blank (Includes ICF) Consider creating a road map for each subject that models the study calendar and list of procedures for each study Consider creating data collection forms for investigator initiated and sponsored studies (if applicable)

28. What can research personnel and staff do to prevent from having audit findings? Ensure that all research personnel have been trained to work on each protocol and document this training on a training log Utilize data collection forms, logs, and resources available on our TTUHSC El Paso Compliance webpage Create regulatory binders and subject files for each of your studies Become familiar with our TTUHSC El Paso Human Research Protection Manual and administrative research staff that can provide assistance for any questions or concerns you may have Attend bi-monthly TTUHSC El Paso research related training sessions Maintain all research documents in one secure location, preferably behind lock and key Remove PHI from study documents as the study progresses instead of waiting until the end of the study Create individual Training Binders for your staff and faculty that can hold CVs, medical licenses, training certificates, etc… Review study procedures per protocol and develop a study design based on these procedures in order to have good study flow during a visit Ensure that you have all the approvals and requirements that you need prior to initiating a study (ex: recruiting approval letters from other institutions, UMC approval, IBC approval) Maintain organized study documentation and instil good organizational processes Ask questions!

29. Resources World Health Organization Good Clinical Practices Manual: http://apps.who.int/iris/bitstream/10665/43392/1/924159392X_eng.pdf?ua=1 Hubspot Auditing vs. Monitoring: http://cdn2.hubspot.net/hub/149400/docs/auditingvsmonitoring.pdf Good documentation practice in clinical research https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3121265/ FDA Data Integrity: https://www.fda.gov/downloads/aboutfda/centersoffices/officeofmedicalproductsandtobacco/cder/ucm518522.pdf Audit Definition TTUHSC El Paso HRPP Manual: http://elpaso.ttuhsc.edu/research/committees/irb/resources/_documents/HRPP%20Manual%20El%20Paso%20October%2024%202016.pdf Audit Finding Definition: https://www.whittingtonassociates.com/2001/11/nonconformity-nonconformance-defect-finding-or-observation/

30. Questions?