Consensus Conference on Transplant Program Quality and Surveillance

1. Consensus Conference on Transplant Program Quality and SurveillanceArlington, VAFeb 13-15, 2012

2. Consensus Conference on Transplant Program Quality and Surveillance Co-Chairs Bertram Kasiske (SRTR) and Maureen McBride (OPTN/UNOS) Steering Committee Robert Gaston (AST) Dennis Irwin (Optum Health) Mitch Henry (ASTS) Nancy Metzler (OPTN-TAC) Thomas Hamilton/Karen Tritz (CMS) John Roberts (OPTN-MPSC) Danielle Cornell (ACOT) Alan Reed (OPTN-PAIS) Kenyon Murphy (Patient/Public) Stuart Sweet (OPTN-POC) Funding HRSA

3. Eight Key Questions What is the SRTR’s mandate? Who uses PSRs and why? Are there unintended consequences? What can we learn from others? What statistical methods should we use? How should we adjust for risk? What outcomes should we use? What data should we collect?

4. Plenary Session 1 – Monday, Feb. 13

5. Plenary Session 2 – Monday, Feb. 13

6. Plenary Session 3 – Tuesday, Feb. 14

7. Plenary Session 3 – Tuesday, Feb. 14

8. Plenary Session 4 – Wednesday, Feb. 15

9. The Final Rule “Make available to the public timely and accurate program-specific information on the performance of transplant programs. This shall include … risk-adjusted probabilities of receiving a transplant or dying while awaiting a transplant, risk-adjusted graft and patient survival following the transplant, and risk-adjusted overall survival following listing … These data shall include confidence intervals or other measures that provide information on the extent to which chance may influence transplant program-specific results.” OPTN Final Rule -Page 21- October 20, 1999

10. HRSA contract with the SRTR • Produce PSRs no less than every 6 mo. • Post-transplant: • risk-adjusted graft and patient survival • morbidity & functional impairment, etc. • Waiting list probability of: • receiving a transplant • dying while waiting • being removed from the waiting list • Living donor: • profiles (age, sex, ethnicities, comorbidities, etc.) • outcomes (death, re-hospitalization, etc.)

11. Recommendations: Statistical Methods I.1. PSRs should be better suited to the needs of users, particularly patients. I.2. Rather than refitting each model every 6 months, models could be fitted less often, and the time between reporting periods could be used to more carefully review models. I.3. Although the current Cox proportional hazards models (with inclusion of additional comorbidity parameters) will likely be adequate for “flagging”, consider comparing this method with mixed effects methods.

12. Recommendations: Statistical Methods I.4. Consider providing transplant centers with tools like CUSUM and/or PSR forecasting “scorecards” to facilitate Quality Assessment and Performance Improvement (QAPI). I.5. Consider increasing the observed-to-expected thresholds and/or using sliding-scale P-values to monitor the outcomes of small-volume centers equitably.

13. Recommendations: Statistical Methods I.6. Mortality data from the Social Security Administration Death Master File (SSADMF) should continue to be available to the SRTR. I.7. The SRTR should substitute missing data with values that yield the best outcomes to encourage centers to accurately record data, and should consider including the timeliness and completeness of data submission as a quality indicator. I.8. Avoid the conversion of continuous data elements to categorical elements, and use splines in instances where continuous linear values are not appropriate.

14. Recommendations: Adjusting for Risk II.1.Consider protecting innovation by excluding patients who are in approved protocols from PSR models in identifying underperforming centers. II.2. Identify centers that manage high-risk patients and donors well.

15. Recommendations: Adjusting for Risk II.3.Develop more detailed, reliable, organ-specific data on: a) coronary heart disease, e.g. revascularizations, b) peripheral vascular disease, e.g. revascularizations and amputations, c) diabetes mellitus, d) ZIP code socioeconomic status / race-ethnicity, e) donor/organ risk, and ventricular assist devices. II.4.Provide more data on waiting list risk and outcomes.

16. Recommendations: Appropriate Outcomes III.1. The composite pre-transplant metric (CPM), combining waiting list mortality, transplantation rate, and organ acceptance rate, is potentially useful. III.2. Life-years after listing is a metric that deserves consideration and study.

17. Recommendations: Appropriate Outcomes III.3. Transplant program risk-tolerance may be a potentially useful metric. III.4. Improve the monitoring and reporting of short-term living donor outcomes. III.5. Consider providing information on long term outcomes.

18. Recommendations: Appropriate Outcomes • III.6.Study the utility of reporting on outcomes such as: • life years from transplantation, • quality of life, • serum creatinine as a surrogate for long term outcomes after kidney transplantation, • FEV1 as a surrogate for long term outcomes after lung transplantation, • acute rejection, • hospitalization, and • rates of candidate acceptance for transplantation.

19. Recommendations: Sufficient Data IV.1. Provide standard definitions and identify source documents for all data elements. IV.2. Examine whether data in DonorNet® can be used to provide information on donor and organ quality. IV.3. Consider reducing data by collecting complete data that are needed for organ allocation and in PSR outcomes models, but otherwise use data sampling strategies for other non-essential data.

20. Recommendations: Sufficient Data IV.4. Consider using Medicare claims data to supplement OPTN data. IV.5. Survey transplant programs to better understand the data collection burden and which data are the most difficult to report.

21. Recommendations: Sufficient Data • IV.6. Assist transplant programs in maintaining their OPTN data by: • educating programs about the availability of tools (on the SRTR private sites, Tiedi® export, etc.) to examine missing data and improving the utility of these tools, • developing enhancements that allow programs to more easily monitor their performance, • allowing programs to more easily make corrections when problems are identified, and • offering standardized OPTN training for data entry personnel and a certification process.

22. Recommendations: Sufficient Data • IV.7. Use the OPTN policy development process and follow strict criteria for adding new data elements, including: • reasons for each data element, • how each data element will be used, • clear definitions, • source documentation requirements, • appropriate populations, • minimizing unproductive data entry categories, e.g. “other” or “unknown”, and • understanding cost implications.

23. Recommendations: Sufficient Data IV.8. The OPTN should explore the feasibility of building data collection interfaces with electronic medical records. IV.9. Consider allowing the cost of mandated data entry to be placed on the Medicare Cost Report for reimbursement, and not limit this option to the Candidate Registration Form. IV.10. Consider providing information about paired exchange.

24. SRTR Technical Advisory Committee (STAC) Meeting Summary: February 23, 2012 • STAC Reviewed the Consensus Conference Recommendations • STAC is continuing the prioritization discussion, but general themes of the discussion included: • Strong support for creation of separate reports for: • Public Consumption: Focused on easily understood metrics that are meaningful for patients, targeting the types of information patients are most concerned about: • Waiting Time • Life-Years from Listing • Program Consumption • Quality Improvement Focus: More Forward-looking metrics rather than historical performance metrics.

25. Traditional Adult Graft Survival

27. SRTR Technical Advisory Committee (STAC) Meeting Summary: February 23, 2012 • Quality Assurance: • Need to continue to support OPTN’s MPSC committee and CMS Conditions of Participation • O/E framework is likely here to stay for the near future. • Recommended studying additional methodologies, including • Bayesian hierarchical modeling strategies (What is the probability program X is underperforming?) • CUSUM charts could be provided for a program’s internal consumption, but are likely not suited for public reporting.

28. Methods: Use of hierarchical models with (Bayesian) suggested performance criteria Christiansen CL, Morris CN. Ann Intern Med. 1997;127:764.

29. Methods: funnel plots

30. SRTR Technical Advisory Committee (STAC) Meeting Summary: February 23, 2012 • Support for formalizing the SRTR’s risk model development process to include: • Written “manual of operations” for risk model development. • OPTN involvement (expert review) • Set model building schedules, perhaps 3-year cycles

31. Example of 3-year Model Building Cycle • Update Cycle Based on Organ Groups. • During the year, both the post- and pre-transplant models will be rebuilt. • Could subset the year into semesters and focus on one organ in each semester.

32. Path Forward • SRTR-STAC discussions are ongoing. • Looking to create a PSR subcommittee of the STAC focusing on performance assessment methodologies. • Prioritized list will be settled in the coming days with review by HRSA. • SRTR staff will begin researching alternative methods to achieve the stated recommendations.

33. Thank You