Registries for Evaluating Patient Outcomes: A User’s Guide Second Edition

1. Registries for Evaluating Patient Outcomes: A User’s Guide Second Edition Presentation for AHRQ Medicaid Medical Directors Learning Network February 17, 2011 Richard Gliklich, MD Outcome

2. Registries for Evaluating Patient Outcomes: A User’s Guide • Gliklich RE, Dreyer NA, eds. • Registries for Evaluating Patient Outcomes: A User's Guide. • Prepared by Outcome DEcIDE Center • AHRQ Publication No. 07-EHC001-1. Rockville, MD: • Agency for Healthcare Research and Quality. April 2007 • 2nd Edition, September 2010 • 3rd Edition, Underway Available at http://effectivehealthcare.ahrq.gov

3. Purpose of the User’s Guide • To guide the design and implementation of patient registries, the analysis and interpretation of data from patient registries, and the evaluation of the quality of a registry or one of its components

4. Second Edition Collaborators • 55 contributors from industry, academia, health plans, physician societies, and government • 49 invited peer reviewers, plus public comment • 38 case studies illustrate challenges and solutions (including 20 new examples) • Senior Editors: R. Gliklich; N. Dreyer. Managing Editor, M. Leavy. Outcome DEcIDE Center • AHRQ Task Order Officer: Elise Berliner

5. Organization of the User’s Guide • Introduction to patient registries • Definition, types of registries, typical registry purposes • Creating registries • Planning, design considerations, data sources, linkage approaches, and legal/ethical issues • Operating registries • Recruitment and retention, data collection, interfacing with EHRs, adverse event reporting, and data analysis • Evaluating Registries

6. Definition of a Patient Registry • A patient registry is an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or policy purposes.

7. Key Characteristics of Registries • Data are collected in a naturalistic manner • Registry is designed to fulfill specific purposes, and these purposes are defined in advance of collecting and analyzing the data • At least one element of registry data collection is active, meaning that some data are collected specifically for the purpose of the registry • Registry captures data elements with specific and consistent data definitions • Data are collected in a uniform manner for every patient

8. Typical Goals of Patient Registries • Effectiveness • Evaluate clinical or comparative effectiveness • Safety • Measure or monitor safety and tolerability, including comparative risk-benefit • Quality • Measure and/or improve quality of care • Natural history • Track natural history of a disease process • Incidence and prevalence • Disease trends • Identification of high risk groups

9. Taxonomy of Registries: Product May include all or a subset of patients exposed to a drug, device or biologic Device registries • Implantable cardioverter defibrillators (ICD) • Stents • Orthopedic devices Pharmaceutical product registries • Cox 2 inhibitors • Thalidomide Pregnancy registries • Exposed population = mother and fetus

10. Taxonomy of Registries: Service Targeting a health care service, procedure or clinical encounter Procedure registries • Primary coronary intervention • Normal pressure hydrocephalus registry • Society Thoracic Surgeons (STS) database Clinical service (and quality measurement) registries • Hospitalization registries • Encounters and episodes of care

11. Types of Registries: Disease/Event Patients enrolled share a common disease or event experience, regardless of treatment or other exposures Acute diseases or events • National Registry Myocardial Infarction (NRMI) • Organ transplant registries Chronic diseases • ESRD registry • Heart failure registry • Cancer registries (SEER) Rare diseases • Lysosomal storage disorders • Cystic fibrosis • Hemophilia

12. Creating a Registry

13. Planning • State the purpose • Identify stakeholders • Establish governance • Define scope of registry • Define target population • Assess feasibility • Secure funding • Determine what (and how) data will be made available to external researchers • For open-ended registries, determine under what conditions data collection should end

14. Track Safety and Effectiveness of a Drug/Device • Primary coronary intervention • 7500+ patients • 125+ sites • Long-term follow-up • Safety outcomes (including comparator)

15. Evaluate Effectiveness and Quality of Care • Stroke Registry • Assesses care from EMS arrival through Discharge • Process and Early Outcomes of Care • >1500 Stroke Centers in the U.S. • ~2M patients enrolled • Includes procedure, device and medication related treatment data

16. Design • Design registry with respect to its major purposes • Different levels of rigor are necessary for different types of registries • Specific clinical questions of interest guide definitions of study subjects, exposures, and outcome measures • Choose a study design • Select data sources, populations, comparison groups • Determine whether sampling is needed • Identify possible sources of bias (systematic error) and address them to the extent that is practical and achievable

17. Data Elements • Select based on importance and relationship to the primary outcome • Consider data collection burden and incremental costs for collection • Whenever possible, use established standards and common data definitions or validated instruments • Weigh pros/cons of using patient identifiers • Use pilot testing to assess feasibility and burden as well as reliability, validity, and potential for missing data

18. Data Elements • Registries can include data from many sources • Primary data are collected for direct purposes of the registry • Secondary data were originally collected for other purposes • Medical records • Institutional or organizational databases • Administrative and claims data • Death and birth records • Census databases • Existing registry databases • When selecting data sources, consider cost, timeliness, structure, availability and quality

19. Linking Registry Data • Increasingly, registry data are linked with other datasets to explore long-term outcomes or for other purposes • Linkage projects require technical and legal expertise to address issues such as: • Statistical techniques for matching records from different datasets • Statistical methods to minimize the potential for re-identification • Legal considerations related to the use of patient identifiers • Ethical considerations, such as whether the linkage project is appropriate under the terms of the informed consent used in the registry

20. Ethics, data ownership, and privacy • Review ethical and data privacy requirements early in planning phase to ensure compliance • Applicable regulatory requirements largely determined by: • Research purpose of a registry • Status of its developer • Extent to which registry data are individually identifiable • Also important: registry transparency, oversight/governance, and data ownership

22. Operating a Registry

23. Patient and Provider Recruitment, Management • Recruitment occurs at many levels • Facilities (hospital, practice, pharmacy) • Providers • Patients • Motivation for participation at each level differs • Relevance, importance, scientific credibility, risks, burdens, incentives • Goals for recruitment, retention and follow-up should be explicit and deviations continuously evaluated for risk of introducing bias • Approaches to follow-up can impact both rates and biases

24. Data Collection & Quality Assurance • Data collection • Includes collecting, cleaning, storing, monitoring, & reporting registry data • Broad range of data collection procedures and systems available • Critical factors in data quality • Data element structure and definition, training of personnel, how data problems are handled • Quality assurance • Define requirements at registry creation • Risk-based approach • Most important or likely sources of error or potential lapses in procedures that may impact quality in the context of intended purpose • Role of audit increasing for certain types of data bases

25. EHR-Registry Interoperability: Summary • Interoperability with their EHRs is an increasing and critical request from practices/institutions joining a registry • No one interoperability standard is universally implemented • An open, multi-purpose interoperability standard has been emerging for several years and is endorsed by the EHR vendors association • Tested at HIMSS by >10 EMRs • Requires one-time implementation by EHRs that is technically easy to maintain • Single implementation supports unlimited use cases--no need for EHR vendor to code content, destination or formats for each secondary use case

26. Adverse Event Detection, Processing, & Reporting • Important for any registry, especially with direct patient and/or physician contact • Collection of spontaneously reported or solicited safety information • Protocol-defined procedures strongly recommended • Investigators and site staff appropriately trained • Processing, coding and managing of AE data • Reporting and regulatory requirements

27. Best Practices for Adverse Event Reporting to FDA in Registries of Post-Marketed Products Gliklich R, Dreyer N, eds.  Registries for Evaluating Patient Outcomes: A User’s Guide.  AHRQ Pub. 07-EHC001-01 2007. Yes Does the registry have data collection with individual patient interaction ? No Does the registry receive sponsorship or financial support from any regulated industry? Yes Follow good public health practices for reporting new or serious AEs (recommended practice; not mandated) Registry trains site(s) on identification and reporting of AEs including events of special interest and serious AEs (SAE). No Establish rules, roles, responsibilities for involved parties for oversight and reporting in conformance with registry design and applicable regulations. Report AEs in FDA periodic reports or PSUR if applicable Notify company and/or FDA about new or serious AEs* Are SAEs recognized by a knowledgeable person in temporal association with a drug* under study? Aggregate study findings of adverse events No Yes Company Contact FDA Is there a reasonable possibility that the drug caused the SAE? No Yes For devices, no attribution of expectedness is required; “device-relatedness” is based on whether the device caused or contributed to death or serious injury, or, in the case of malfunction, if the chance of death or serious injury is not remote if the malfunction were to recur Notify responsible entity (e.g., company) ASAP, ideally within 24 hours Company determines if the SAE is “unexpected” (based on labeling) in terms of type, specificity or severity No Company reports SAEs considered unexpected and possibly related for own drugs to FDA within 15 calendar days of original report; reports for device-related deaths, serious injuries, or malfunctions are due within 10-30 calendar days. Yes

28. Analysis and Interpretation Analysis • Importance of a statistical analysis plan • Analytic plans and statistical techniques for primary and secondary objectives • Report on characteristics of the patient population, exposures of interest and endpoints Interpretation • Who was studied? • Is the actual population representative of the target population? • How were the data collected, edited and verified? • Completeness of data collection and data quality • How were missing data handled and reported • How were the analyses performed?

29. Case Examples • 38 case examples are included to illustrate practical challenges and how they were addressed: • NRMI – “Creating a Registry to Fulfill Multiple Purposes and Using a Publications Committee to Review Data Requests” • GWTG-Stroke – “Using Performance Measures to Develop a Data Set” • BPH – “Developing and Validating a Patient-Administered Questionnaire” • GWTG – “Using Recognition Programs to Recruit Sites” • OPTIMIZE-HF – “Using Registry Tools to Recruit Sites”

30. Evaluating Patient Registries

31. Defining Quality • “Quality” refers to the confidence that the design, conduct and analysis of the registry can be shown to protect against bias (systematic error) and errors in inference (erroneous conclusions drawn from a study)

32. Evaluating Registries Quality component analysis • Research quality (scientific process) • Planning; design; data elements & data sources; ethics, privacy and governance • Evidence quality (data/findings) • Patients; data elements & data sources; QA; analysis; reporting Components classified as: • Basic Practice • Potential Enhancement

37. Development of Third Edition Underway New Topics include:

38. Contact Information and References Richard Gliklich MD President, Outcome richg@outcome.com www.outcome.com Reference: • Gliklich RE, Dreyer NA, eds. Registries for Evaluating Patient Outcomes: A User's Guide. 2nd Edition. (Prepared by Outcome DEcIDE Center [Outcome Sciences, Inc. dba Outcome] under Contract No. HHSA29020050035I TO3.) AHRQ Publication No. 10-EHC049. Rockville, MD: Agency for Healthcare Research and Quality. September 2010.