The challenge of audit of Registries

1. The challenge of audit of Registries Nick Gray

2. Background • Australian Registries • 28 surveyed • 46% did not assess or recruited <80% of eligible population • 82% did not audit inter-rater reliability of coding • 28% assessed through audit the accuracy and reliability of data collected from source documents Evans et al, Intern Med J 2011

3. Renal Registries • CORR • 1125 records (2005 and 2006 dialysis start) • Age, race, PRD, comorbidities • Moist LM et al, CJASN 2011 • USRDS • 1692 records (1986 and 1987 start) • 50 variables • AJKD 1992 • UK Renal Registry • Missing data • Collier T et al, NDT 2011

4. Cancer data quality • ANZDATA vs Central Cancer Registry (NSW) • Results • 8.2% cancer in ANZDATA, 8.3% in CCR • Kappa statistic 0.76 • If CCR considered gold standard • ANZDATA sensitivity 77.3% (95%CI: 74.2-80.2) • ANZDATA specificity 98.1% (95%CI: 97.7-98.3) • Agreement (Kappa 0.78) on dialysis • Agreement (Kappa 0.79) after transplantation Webster et al, Nephrology 2010

5. Pilot audit • Methods • Retrospective audit of 2009 prevalent HD patients • Single centre and 3 small private units • Original data collected and entered by nursing staff • Audit by nephrologists, blinded to original data • Nephrologists data considered gold standard • 51 audits (29% of HD population) • Characteristics comparable with regional patients Gray et al, Nephrology 2013

6. Pilot audit • Results • Primary renal disease 86.3% (95%CI: 74.3-93.2) • 50% error in type of GN (n=8) • CORR 70.9% (95%CI: 66.2-75.6) • USRDS 59.5% • Dialysis start date • Exact match 68.1% (USRDS 64%) • Match +/- 1 month 93.6% (USRDS 85%) Gray et al, Nephrology 2013

7. Pilot audit Gray et al, Nephrology 2013

8. Pilot audit • Cancer • 41.2% no cancer • 21.6% cancer on audit, not with ANZDATA • Skin BCC/SCC • 73% prior to commencing RRT • 35.3% cancer on audit and ANZDATA • 39% data inaccurate (skin BCC/SCC missed or first cancer missed) Gray et al, Nephrology 2013

9. Audit challenges • Data prior to ESKD • Care external to renal units • Skin clinics • Patients transferred from other units • Baseline data • Patients with long standing ESKD • Height, weight at entry • Target weights, dialysis data

10. Is a broad ANZDATA audit feasible? • Training of auditers • Inter-auditer reliability • Time and cost • Ethics • Sample size and selection • All or some parameters? • Frequency?

11. Improving accuracy • GN coding • Comorbidities at entry • Definition of comorbidities • Events • UKRR definitions • USRDS Form 2728 • Training and education

12. Current processes • Annual census • Reconciled with previous year • Missing data flagged • Queries generated, forms cross-checked • Units notified of queries • Queries returned • Back to step 2!

13. Current processes • Real time data • 1 on 1 training for new users • Data entered only accepted if it meets business rules • Monthly reports – flag errors early • On-line training monthly • QA forum monthly (webcast) • Data manager training • Presentation at state Renal Clubs • RSA/ANZSN ASM – presentations and stand

14. Future directions • HOU survey