FDA Regulatory Perspective: Data Integrity

1. FDA Regulatory Perspective: Data Integrity Steve Wilson, Dr.P.H., CAPT USPHS Deputy Director, Division of Biometrics II, CDER/FDA NIH Roadmap Program Feasibility of Integrating & Expanding Clinical Research Networks 4th Steering Committee Meeting Friday, May 12, 2006 NIH Natcher Conference Center, Bethesda, MD

2. FDA’s Mission • The FDA is responsible for protecting thepublic health by assuring the safety, efficacy, and security of human and veterinary drugs, biological products, medical devices, our nation’s food supply, cosmetics, and products that emit radiation. • The FDA is also responsible for advancing the public health by helping to speed innovations that make medicines and foods more effective, safer, and more affordable; and helping the public get the accurate, science-based information they need to use medicines and foods to improve their health.

3. Data Integrity • The quality of correctness, completeness, wholeness, soundness and compliance with the intention of the creators of the data. • It is achieved by preventing accidental or deliberate but unauthorized insertion, modification or destruction of data in a database. • Data integrity is one of the six fundamental components of information security. http://www.pcmag.com/encyclopedia_term/0,2542,t=data+integrity&i=40792,00.asp

4. Data Integrity • 21 CFR 11 ( In final rule “integrity” is used 50 times!) • Electronic Submission • Gateway • EDR SOPs

5. 21 CFR 11 -- Integrity For electronic records and submissions to have the same integrity as paper records, they must be developed, maintained, and used under circumstances that make it difficult for them to be inappropriately modified. Without these assurances, FDA’s objective of enabling electronic records and signatures to have standing equal to paper records and handwritten signatures, and to satisfy the requirements of existing statutes and regulations, cannot be met.

6. Data Quality Data are of high quality "if they are fit for their intended uses in operations, decision making and planning” (J.M. Juran) http://en.wikipedia.org/wiki/Data_quality

7. FDA Regulatory Perspective: Data Quality Steve Wilson, Dr.P.H., CAPT USPHS Deputy Director, Division of Biometrics II, CDER/FDA NIH Roadmap Program Feasibility of Integrating & Expanding Clinical Research Networks 4th Steering Committee Meeting Friday, May 12, 2006 NIH Natcher Conference Center, Bethesda, MD

8. Disclaimer Views expressed in this presentation are those of the speaker and not, necessarily, of the Food and Drug Administration

9. Truth-in-Advertising • CDER-Centric View • Customer -- Not in Office of Compliance or ORA

10. Organization: FDA • Center for Biologics Evaluation and Research (CBER) • Center for Devices and Radiological Health (CDRH) • Center for Drug Evaluation and Research (CDER) • Center for Food Safety and Applied Nutrition (CFSAN) • Center for Veterinary Medicine (CVM) • National Center for Toxicological Research (NCTR) • Office of the Commissioner (OC) • Office of Regulatory Affairs (ORA)

11. Outline • Background • Data Quality • A Bimo perspective • A Review perspective • Changing Landscape • The Critical Path • Data standards & electronic submissions • PRO and ePRO • Organizational Changes • Bimo Initiative • CDER’s new Office of Translational Science (OTS) • Connections/References

12. Science, Statistics and Experimental Design • Science is concerned with understanding variability in nature • Statistics is concerned with making decisions about nature in the presence of variability • Experimental design is concerned with reducing and controlling variability in ways which make statistical theory applicable to decisions about nature. Winer, et.al., Statistical Principles in Experimental Design, New York, 1962, 1971, 1991

13. Data Quality • Guidance for Industry: Computerized Systems Used in Clinical Trials, April 1999 • Data should be (ALCOA) • attributable • legible • contemporaneous • original • accurate

14. Responsibility • The sponsor is responsible for implementing and maintaining quality assurance and quality control systems with written SOP’s to ensure that trials are conducted and data are generated... • ... Quality control should be applied to each stage of data handling to ensure that all data are reliable and have been processed correctly… Section 5.1, “Quality Assurance and Quality Control,” Good Clinical Practice Consolidated Guideline -- ICH E6

15. The Law Section 505(k)(2) of the Food, Drug, and Cosmetic Act mandates FDA shall have access to and copy and verify the required clinical study records. The Program The Bioresearch Monitoring (BIMO) Program was established in 1977 to verify the data submitted in support of marketing applications and to provide oversight of the conduct of studies with regulated products.

16. Ketek – The “Fraud” Word Infected Data • Fraud, Errors Taint Key Study of Widely Used Sanofi Drug • Despite Some Faked Results, FDA Approves Antibiotic; One Doctor's Cocaine Use • Company Defends Safety By ANNA WILDE MATHEWS Wall Street Journal, May 1, 2006; Page A1 online.wsj.com/article_print/SB114644463095840108.htm

17. Not the First Time (Never Just a Scientific Question) • Halcion • Tamoxifen • Fiddes

18. Our Responsibility for the Quality of Clinical Trials Data Trust, but Verify

19. Worrying About Data Quality: NDA Review • Erroneous values • Missing values • Imputation • Adjudication • Deviations from protocol • “Unconscious Bias” • Fraud

20. Enforcer Colleague The Terminator Mr. Rogers

21. Programs That Worry About Data Integrity/Quality • BIMO • REVIEW

22. Data Integrity/Quality A BIMO Perspective • FDA calls its program of on-site inspections for GCP and GLP its “Bioresearch Monitoring Program” or “BIMO” • The program includes inspections of: • Clinical Investigators • Sponsors, monitors, CROs • Institutional Review Boards • Bioequivalence Laboratories and Facilities • GLP Facilities (nonclinical studies) • Each Center has its own BIMO group Woollen, 2002

23. CDER’sDivision of Scientific InvestigationsOur BIMO People

24. DSI’s Program Responsibilities • Good Laboratory Practices • In vivo Bioequivalence • Good Clinical Practices • Institutional Review Boards (IRBs) • Clinical Investigators • Sponsor-Monitors, CROs

25. Program Objectives • To verify the quality and integrity of bioresearch data • To protect the rights and welfare of human research subjects

26. GCP Inspections:Routine vs. Directed • Routine • Inspections assigned for NDA/PMA’s • Directed • Problems identified at IND/IDE stage • Complaints to FDA • FDA, other Agencies • Sponsors/monitors • Institutions/IRB’s • Subjects/Public

27. CDER-GCP Bioresearch Monitoring (BIMO) Program • Clinical Investigator Inspection Program • Sponsor/Monitor/CRO Inspection Program • IRB/RDRC These “on-site” inspection programs collectively allow the agency to determine: • Adherence to applicable FDA regulations • Validity of data from studies in support of pending marketing applications (Data Integrity) • Whether the rights and safety of subjects have been protected (Safety) J. Rhoads, 2005

28. CDER BIMO Inspections (FY 2004) CI = 351 BEQ= 136 IRB = 120 GLP = 68 S-M = 17 Total = 692 J. Rhoads, 2005

29. When a Submission is Received: • Review Division Invites DSI staff for filing meeting • Team effort in selecting sites for inspection based on “risk based approach” • Impact to review of the study • Site(s) influencing significance on outcome • Outliers, e.g., site with large proportion of treatment responders • Drop-outs; Adverse events; Protocol violations; large N# • Impact to the clinical trial process • Volume of workperformed by the Clinical Investigator • Past inspectional history • Inspection request assignment to the field offices (ORA) • PDUFA Time Clock! J. Rhoads, 2005

30. Criteria for Assigning International Inspections International sites may be audited • if there are insufficient domestic data; • only foreign data are submitted to support an application; • domestic and foreign data show conflicting results pertinent to decision-making; or • there is a serious issue to resolve, e.g., suspicion of fraud, scientific misconduct, significant human subject protection violations. J. Rhoads, 2005

31. After an Inspection • Form FDA 483: Inspectional Observations • Left with CI at close of inspection • Immediately available via FOI • Establishment Inspection Report (EIR) • Prepared by field investigator after inspection • Includes exhibits supporting observed deficiencies J. Rhoads, 2005

32. After an Inspection • Clinical Inspection Summary to the Review Division • Summary of all inspections assigned for an application • Recommend accept or reject data • Provided to review division in advance of PDUFA action goal date • Letter to the inspected party • emphasize deviation from regulations, if any. • Copied to the review division • include “DSI note to Review Div. Medical Officer” J. Rhoads, 2005

33. Regulatory/Administrative Follow-up • Rejection of study • Disqualification • Prosecution J. Rhoads, 2005

34. Clinical Investigator DeficienciesCDER Inspections - FY 2004 55% 48% 34% 28% 29% 21% 17% 10% 7% 5% 4% 2% Foreign n = 71* Domestic n = 243* *% does not include Biologic inspections J. Rhoads, 2005

35. Some GCP Challenges and Initiatives • Increased international inspections • Risk-based approach to inspection site selection • Accountability of study staff/site management organizations in clinical research • Linked real-time inspections (CI/Sponsor/IRB) J. Rhoads, 2005

36. Data Integrity/Quality: A Review Perspective • Medical Reviewers • Statistical Reviewers

37. NDA/BLA Review: Medical • Exclusion of patients from primary analyses • Reliance on unplanned subset analyses • Lack of consistency of results within and across studies • Patient entry errors (cancelled patients, ineligible patients) • Non-evaluable patients • Missing data • Inconsistent or clearly inaccurate data • Extensive or non-random corrections to case report forms • Missing source documents • Possibility for bias in patient treatment or patient assessments • … Robert DeLap, PhRMA Meetings, Washington DC, October 26, 1994

38. NDA/BLA Review: Statistical • Assess compliance with protocol / blinded analysis plans • Assist DSI in planning investigator audits • Check appropriateness of statistical models and conclusions. • Verify results reported in the NDA. • Modify models and assess robustness/sensitivity of the results. • Modify data sets and reanalyze. • Examine the trial and data for bias: • Results by center • Baseline predictors • Important subgroups (sex, age, race, etc,) • Assess impact of audits

39. Case Study #1 “Unconscious Bias” and Readjudication The reduction in sudden deaths, said the FDA, had been exaggerated. Some patients who died according to Dr. Robert Temple Director of FDA’s Cardio-Renal Drug Products Division, had been excluded for “minor protocol violations”; and most of the excluded patients turned out to have been taking anturane … Nearly all the miss-classifications turned out to favor the hypothesis that anturane prevents sudden deaths. Lancet, August 9, 1980, p. 306

40. Case Study #1 “Unconscious Bias” and Readjudication The reason for these doubts is that Ciba Geigy, which had a stake in the outcome, not only paid the bills (around four million dollars) but also was deeply involved in the daily processing and collection of data before turning over the information to an independent policy committee.” Lancet, August 9, 1980, p. 306

41. Case Study #2 Analyzing Potential Fraud Treatment-by-Center Interaction: p = 0.056 2 1 ? 0 Mean Change from Baseline -1 -2 -3 Drug X 10 19 1 7 19 17 1 12 5 9 0 7 5 9 3 10 Placebo 0 4 1 2 3 4 5 6 7 8 9 CENTER

42. Case Study #2 Center 8 Center 1 SympC SympC 0 1 2 3 0 1 2 3 0 33 2 0 0 0 68 4 0 0 SympB 1 3 33 0 0 1 17 4 0 0 SympB 2 5 4 63 2 2 16 21 2 0 3 0 0 1 31 3 1 0 0 0 Total: 133 Total: 177 Analyzing Potential Fraud (con.)

43. Case Study #2 Symptom Score 100 80 None 60 Mild Percent of Values Moderate 40 Severe 20 0 1 2 3 4 5 6 7 8 9 Center Analyzing Potential Fraud

44. Case Study #3 Halcion: For-Cause Audit From the Summary Basis for Approval Protocol 6045 (investigators: Franklin, Howard, Reeves, Simson): 129 insomniacs (aged 22-62 years) participated in four double-blind 28-day studies comparing triazolam 0.5mg to placebo; Franklin and Reeves enrolled 78% of the patients. Pooled data showed triazolam was significantly better than placebo at end of week 1 on efficacy measures… Drowziness, dizziness, and impaired coordination were significant side effects; 13/61 patients in drug group who dropped out due to side effects, 10/13 dropped out in the first week; no abnormal changes in clinical significance were observed in lab analyses or vital signs.

45. Case Study #3 Halcion: For-Cause Audit Efficacy Analysis (Protocol 6045)

46. Case Study #3 Fraud: Robert Fiddes “If ever there was a wonder boy in the lucrative business of drug testing, it was Dr. Robert Fiddes. In just a few years, Fiddes transformed his sleepy medical practice ... into a research juggernaut, recruiting his patients for drug experiments at a breakneck pace. His success made him a magnet for an industry desperately scouring the nation for test subjects. Companies large and small showered him not only with almost 200 studies to conduct, but with millions of dollars in compensation for his work.” Kurt Eichenwald and Gina Kolata “A Doctor's Drug Studies Turn Into Fraud” New York Times, May 17, 1999

47. Case Study #3 Fraud: Robert Fiddes “The abuses of this one doctor point to weaknesses in the new system that has developed in recent years for testing experimental drugs. No longer does the pharmaceutical industry rely on career researchers at academic medical centers, whose professional reputations are forged on the quality of their data. Rather, the industry has turned to thousands of private-practice doctors, for whom testing drugs has become a sideline for making money.” Kurt Eichenwald and Gina Kolata “A Doctor's Drug Studies Turn Into Fraud” New York Times, May 17, 1999

48. When Things Go Wrong • Sensitivity Analysis (Reanalyze) • Expand Audit (FDA, Sponsor, Third Party) • Resubmission (delayed approval)

49. Changing Landscape • The Critical Path • Data standards & electronic submissions • PRO and ePRO • Organizational Changes • Bimo Initiative • CDER’s new Office of Translational Science (OTS) • Connections

50. Critical Pathhttp://www.fda.gov/oc/initiatives/criticalpath/