Key Performance Indicators, Centre Reports, and more

1. Key Performance Indicators, Centre Reports, and more Stephen McDonald

2. Barbecue talk

3. More “good” news

4. Indigenous incidence rates

5. Background • A number of ongoing work themes exist within ANZDATA for generating output • Stock and flow figures • Annual Report • Contributor requests • Responses to information needed for various projects • Research projects (internal and external analyses) • Outcomes reporting

6. Outcomes reporting • Recent years have seen a growth of interest in outcomes reporting • Centre reports have been part of ANZDATA for many years, with increasing emphasis in recent years • At “parent hospital level” • Limited distribution historically

7. Why measure outcomes?

8. 4 Adjusted relative risk 2 1 .5 .2 0 20 40 60 80 Units, ranked by RR RR 95% CI Mortality rate during dialysis treatment in Australia 2006-10, adjusted for demographics and comorbidities Dialysis outcome

9. Variation in transplant outcomes

10. What is happening to centre reports? • Greater reporting of demographics and comorbidities • Adjusted analyses in transplanting centre and dialysis reports • Details of models supplied • Graphs • Funnel plots • CUSUM plots (transplant)

11. Centre reports – graph 1

12. Centre reports – graph 2

13. But....

14. Adjusted graphs

15. Adjusted graphs

16. How are reports derived?

17. You need a model • Logistic regression model (transplant), Poisson model (dialysis) • Adjusted for demographics, comorbidities (donor and XM variables) • With this model, derive a probability of “expected” failure for each person / graft based on covariate matrix • Compare this with actual outcomes

18. Which predictors are important? Predictive power of multivariate Cox model predicting graft survival, all DD transplants 2001-2009, with sequential addition of covariate groups

19. Factors within the control of centre These may be why a particular centre gets good or bad results Factors that occur as a result of treatment decisions For example, don’t adjust for Choice of dialysis modality, HD access Use of immunosuppressives, rejection, 1 month graft function… Don’t adjust for…

20. Other graphical demonstrations of output • Funnel plots are a static measure and summarise performance (relative to a comparator) over a fixed period of time. • Lack a dynamic element • Weight recent and distant results equally

21. Twoway CUSUM for a transplant centre 4 400 2 300 Cumulative sum O-E Number of tx 0 200 -2 100 -4 0 01jan2004 01jan2005 01jan2006 01jan2007 01jan2008 01jan2009 Tx date Adding time – CUSUM

22. Removing credit for good deeds

23. Do we need to do more?

24. Why KPIs? • Mortality is an insensitive and late indicators of problems • Hopefully rare • Outcome of complex series of events • Incompletely ascertained • Important to monitor as best we can • Key Process indicators • Simpler to understand, easier to address • Need to be valid and correctable (and related to meaningful outcomes)

25. KPI Project • Dialysis KPI project commenced 2011 • At instigation of DNT committee • 2 markers chosen – Peritonitis and HD access at first treatment • Deliberately limited to existing data collection • NO additional data collected • Based on real time ANZDATA data collection

26. Variation in HD access

27. Variation in peritonitis rate

28. KPI reporting -- access • Quarterly identified feedback to units

29. Peritonitis reporting

30. Where to from here? • COMMUNICATE • Improve data collection • Improve access to results • Enhance reporting • Add peritonitis rates • Access subdivided by late referral • Graphs etc etc • Or is it all just too hard?

31. How do we view quality?

33. Centre reports -- SMR