Methodologic Issues and Next Steps in the Implementation of the Results Database

1. Methodologic Issues and Next Steps in the Implementation of the Results Database Deborah A. Zarin, M.D. ClinicalTrials.gov February 24, 2009 1 1

2. History of ClinicalTrials.gov FDAMA 113 (1997): Mandates Registry IND trials for serious and life-threatening diseases or conditions ClinicalTrials.gov Launched in February 2000 Calls for Increased Transparency of Clinical Trials Maine State Law; State Attorneys General Journal Editors (2004) ClinicalTrials.gov Accommodates Other Policies FDAAA 801 (2007): Expands Registry and Adds Results Database 2 2 2

3. Public Law 110-85Sec.801 Expanded Clinical Trial Registry Enacted on September 27, 2007 Requires Trial Registration (Dec 2007) Phase II-IV drug and device trials for all diseases Data elements: ClinicalTrials.gov + ~ WHO/ICMJE Requires Results Reporting (Sept 2008) Trials of FDA-approved or cleared drugs and devices “Basic” Results: Baseline Characteristics, Primary & Secondary Outcomes, Statistical Analyses Adverse Events (Sept 2009) “Expansion” of results by rulemaking (Sept 2010) Added enforcement provisions 3 3 3 3

4. Enforcement Provisions Notices of non-compliances Civil monetary penalties up to $10,000/day Withholding of NIH grant funds

5. Key Terms Applicable Clinical Trials Interventional trials Phase 2-4 drug, biologic, device >= one site in U.S. Ongoing as of 9/27/07, or later Responsible Party Sponsor, grantee PI if designated Completion Date

6. New Registrations Continue to Increase

7. ClinicalTrials.gov Statistics(as of 02/03/2009) • Number Percent • Total 67,064 100% • Type of Trial* • Observational 10,690 16% • Interventional 57,119 84% • Drug & Biologic 42,684 • Surgical Procedure 8,585 • Behavioral, Gene • Transfer, Other 7,997 • Device** 3,862 • International Sites (161 countries) • US only 32,772 49% • Non-US only 23,109 34% • US & Non-US mixed 4,064 6% • Missing 7,119 11% *171 records missing Study Type information **173 device trials – “delayed posting” 7

8. ClinicalTrials.gov Statistics (cont.)(as of 02/03/2009) NumberPercent • Trials by Data Provider • US Federal (including NIH) 18,088 27% • Industry 21,072 31% • University, Other 28,820 42% • Total 67,980 • User Statistics • Page Views per month 40 Million • Unique visitors per month 500,000 8

9. Basic Results Database 9

10. Basic Results Database: General Characteristics Results of “applicable clinical trials” of FDA-approved/cleared medical products Generally, submission within 12 months of the earlier of estimated or actual trial completion date (of primary outcome) Delayed Submission of Results Seeking approval of a new use Extensions for “good cause” E.g. data are still blinded NOT—awaiting publication 10 10 10 10 10

11. Wood AJJ. Progress and deficiencies in the registration of clinical trials. NEJM. 2009

12. Wood AJJ. Progress and deficiencies in the registration of clinical trials. NEJM. 2009

13. Basic Results Modules Participant Flow Baseline and Demographic Characteristics Outcome Measures Adverse Events (summary data) Other Information “Certain Agreements” Restricting Results Disclosure Overall Limitations and Caveats Results Point of Contact

14. Key Issues • The Results Database is Working • Is the Results Database a Good Thing? • Who is the Audience? • What have we learned so far? • To Be Considered…

15. 1. The Results Database is Working

16. Current Status – “Basic Results”(as of 02/06/09) • Functional Web-based Data Entry System • Launched in September 2008 • Ongoing system of feedback and improvements • 410 Results Records have been submitted • Industry: 293 records from 72 data providers • Other: 117 records from 80 data providers • Anticipate greatly increased rate of submission 16 16

17. Results Data Entry Process Technical Issues 17 17 17

18. Design Requirements • Display consists of data tables with minimal text—must be self-explanatory • System must accommodate range of study designs and facilitate comparison across studies • NLM directed to: • Consider different methods of display based on principles of risk communication for different audiences • Ensure the data are searchable in many ways • Structured data entry required to facilitate search and display needs 18 18

19. Design Features Tables are “constructed” by the data provider Columns are pre-set as study arms, but can be changed by the data provider Rows are measures—some are pre-set, others are customized for each study Type of measure determines specific design of “cells” Attempt to balance fixed structure with flexibility 19

20. Principles for Using the Basic Results Database Submitted data are used to develop basic tables for the public display Tables must be interpretable by people not familiar with each particular study Labels for rows, columns, and units of measure must be meaningful and precise 20

21. Resources to Help Data Providers • “Helpful Hints” • Illustrates process for entering different study designs (parallel, crossover, diagnostic accuracy, bioequivalence—in progress) • Webinar • “Common Errors” • Individual discussions regarding particular studies • Presentations

22. Results Database:Challenges Data tables will be a public representation of the study—must be clear and informative Many entries are not logical or informative Posting such entries will embarrass all parties and will not contribute to the public good 22 22

23. Sample Posted Results 23 23 23

25. Arms Milestone Reasons Not Completed

26. Crossover Design Multiple “Periods”

27. “Default” Required Measures User-Specified Measure Categories

28. Arms

29. 29 29

31. Categories

32. Statistical Analysis

33. Statistical Analysis

34. 34

36. Example: Study Record with Results including 29 Posted Outcome Measures 36

37. Results Data Submissions(as of 2/23/09) • 432 Studies Total • By Sector • 311 (72%) Private • 121 (28%) Public • Data Providers • 76 Private (Mean: 4.1 studies/org) • 104 Public (Mean: 0.9 studies/org)

38. Results Data Posted(as of 2/20/09) • 54 Total • Study Type • 3 (6%) Observational • 51 (94%) Interventional • Provider Category • 3 (6%) NIH • 35 (65%) Industry • 16 (29%) Other

39. Intervention Type 39 Drug & Biologic 6 Device 4 Behavioral 1 Medical Procedure Study Design 39 Randomized 23 Masked, double 15 Control, active 13 Control, placebo Adverse Events (optional) 7/54 (13%) Studies Phase 5 Phase 1 9 Phase 2 25 Phase 3 11 Phase 4 9 N/A Outcome Measures [mean (range)] 6.6 Primary (1-49) 6.4 Secondary (0-49) <1 Other Prespecified (0-8) Results Data Posted (cont.)(as of 2/20/09)

40. 33 of 87 include Adverse Event module

41. 2. Is the Results Database a Good Thing?

42. Powerful Search Features • Synonymy • UMLS, etc • Hierarchies • Spelling relaxation

47. What about lack of peer review? • Large social experiment—results unknown • On the other hand, industry (e.g., GSK) results databases have been around • Nissen—rosiglitazone • Peer review is not a panacea

48. Published Results Search of PubMed Secondary Source ID [SI] field by NCT number from 248 records with results Identified 10 citations (NCT # in Abstract) Manual search of 79 records with results in PubMed (without NCT numbers) Identified 2 citations (no NCT # in Abstract) 48 48

49. Conclusions: Paroxetine is generally well tolerated and effective for major depression in adolescents. Conclusions: Paroxetine is generally well tolerated and effective for major depression in adolescents.

50. Publication Bias