Diagnostic Accreditation Program

1. Diagnostic Accreditation Program A Systematic Approach to Quality and Safety in Diagnostics with emphasis on Medical Peer Review Dr. Carlow, MD CCFP

2. Objectives • To describe why this is important • To identify what is being done throughout healthcare • To define key principles and practices of a systematic approach • To define issues and solutions for two diagnostic modalities including examples from the field

3. Why is this important? • An elderly man underwent chemotherapy for GI cancer at BCCA in the early 1990’s. In error, he received 10 times the normal dose of 5 FU and died as a consequence. • A Boston health reporter in her late 30’s received a large overdose of chemotherapy for breast cancer at the Dana Farber Cancer Institute and died. • Both of these tragic events led to major systematic changes.

4. Why is this important? • A 34 year old woman diagnosed with neuroendocrine cancer had five surgeries to exise a cyst, remove lower jaw and teeth, and undergo facial reconstruction. Her slides were contaminated by cells from another patient. She did not have cancer. • A patient with a positive pregnancy test underwent pelvic ultrasound. The uterus was described as empty. The patient received methotrexate for the treatment of ectopic pregnancy. On review, another radiologist subsequently diagnosed normal intrauterine pregnancy.

5. Why is this important? Diagnostic Errors in the daily News • Pathology errors force thousands to be retested in New Brunswick, G & M Feb 08 • Disgraced Ontario pathologist says errors not all his fault G & M Mar 08 • Serious quality-control problems plague hospital labs in Canada G & M Mar 08 • 108 women died after botched cancer tests Newfoundland says G & M Mar 08 • Errors found in work of another pathologist (6% error rate) G & M May 08

6. Why is this important? More than Anecdotes • Harvard medical practices study (Leape NEJM 91) 3.7% with disabling injuries caused by medical treatment. • Institute of Medicine (U.S.) report released in 1999 – To Err is Human: Building a Safer Health System • 44,000 to 98,000 preventable deaths • Canadian adverse events study: • Adverse event rate of 7.5 per 100 hospital admissions Baker et al CMAJ 2004

7. Why is this important? More than Anecdotes • American physicians got it right 55% of the time McGlynn et al NEJM 2003 • Many studies reveal significant variations in practice/low rates of standardization • Swiss Cheese

8. Why is this important? What about diagnostic errors? • What types of medical errors occur more frequently – medication or diagnostic errors? • In two recent studies of malpractice claims – diagnostic errors far outnumbered medication errors as a cause for claims • Diagnostic errors are underemphasized and understudied

9. Why is this important? What about autopsy discovered errors? • Multi decade study Shojania et al JAMA, 2003 • Median error rate 23.5% for major errors • Although error rates have declined over the decades, rates are sufficiently high enough that ongoing use is warranted • U.S. national average autopsy rate is 5%

10. What is being done? • Agency for Health Care Research & Quality (AHRQ) • Canadian Patient Safety Institute (CPSI) • Institute for Health Care Improvement (IHI) • Greater expectations from standard setting bodies: • CCHSA • JCAHO • CAP • ACR

11. What is being done? • Safer health care now • Collaboratives • 5 million lives campaign • Hospital standardized mortality ratio (HSMR) • Global Trigger Tool

12. What is being done? • Quality networks • Provincial councils on Quality and Safety • Governing Board’s focusing more on quality and safety • Standards of professional bodies • Recertification

13. What is being done? Organizational Initiatives • Veterans Administration/Kaiser Permanente • BC Cancer Agency These are two examples of the systematic application of evidence and the integration of quality and safety.

14. What is being done? • Chronic Disease Management Initiatives • Hypertension • CHF • Diabetes • Evidence based stroke program • Campbell River hospital

15. What is being done? What has been learned about the major attributes of a systematic approach? • Fragmented and isolated initiatives are quite pervasive and ineffective • Importance of research driven evidence based care • The important role of clinical decision support systems and tools

16. What is being done? • The integration of quality and safety • The importance of overall system design and clinical governance: • e.g. Trauma system • e.g. micro systems • Thorough knowledge of improvement methods and tools including: • Knowledge of processes • Quality improvement cycles • Root cause analysis • Rapid cycle improvement

17. What is being done? • Quality planning and priorities • A clearly set out agenda for quality and safety • An enabling culture • Leadership commitment • Professional responsibility • Inter-professional collaboration • Non-punitive reporting • Disclosure • Improvement mindset • Thinking and acting as a system • Accountability • Breakthrough thinking/aggressive targets

18. What is being done? • Surveillance/Monitoring/Measurement of processes, outcomes and benchmarking • Quality and Safety infrastructure support • Technological support Standards of professional bodies are now reflecting these attributes

19. What is being done? Causes of Error • Variation in practice with variable inputs • Complexity – too many steps • Inconsistent knowledge, training and language (terminology) • Human factors in routine repetitive tasks • Deadlines/stress/excessive workload • Handoffs – transfer of information • Cultural issues – lack of openness and freedom of expression • Unsystematic/adhoc approaches • Swiss cheese effect

20. How should we proceed in Diagnostic Services? • Do we know enough about the various attributes of diagnostic errors? • Are not traditional methods of medical peer review adequate and working well?

21. How should we proceed in Diagnostic Services? Areas that need attention • Better definition of what constitutes an error • Greater consistency in definitions, terminology and standardization of reporting • Better tools to assess significance of errors

22. How should we proceed in Diagnostic Services? Areas that need attention • More research on the extent of errors and their causes • More research on the relationship between errors and adverse affects • Being clearer about acceptable rates of errors

23. How should we proceed in Diagnostic Services? Traditional methods of peer review • Morbidity and mortality conferences • Autopsy • Malpractice claims analysis • Error reporting systems

24. How should we proceed in Diagnostic Services? Traditional methods of peer review • Chart review • Observation of patient care • Clinical surveillance • Administrative data analysis • Electronic medical record review

25. How should we proceed in Diagnostic Services? Many of these have positive attributes, however: • Low case numbers • Hind sight bias • Under reporting • Absence of standardization • Some have a linkage to total organizational effort • Some not specific enough for program or department

26. Anatomic Pathology Errors Anatomic Pathology Errors • In general anatomic diagnoses are highly accurate? • In the opinion of several, errors are not rampant • Diagnostic variation is not uncommon, but not all harmful • Depends on what are acceptable results • Medical quality affected by all phases of the system

27. Anatomic Pathology Errors Life Cycle Data • Data indicates the importance of gathering information over the whole testing cycle • Carroro et al in Clinical Chemistry 2007 report • 61.9% pre analytic errors • 15% analytic • 23% post analytic

28. Anatomic Pathology Errors Pre Analytic Phase • In this phase of the test cycle the problems more frequently relate to: • Specimen I.D. • Sample quality • Availability of clinical information

29. Anatomic Pathology Errors Pre Analytic Phase • In one large study 6% of cases were defective at accessioning with defective I.D. as the 2nd largest category Nakhleh et al CAP Q probes APLM 1996 • A survey of 341 labs revealed no clinical history in 2.4% of cases. When corrected – change in diagnoses in 6.1% of cases Nakhleh et al CAP Q-probes APLM 1998

30. Anatomic Pathology Errors Pre Analytic Phase • Patient I.D. errors in SP are the most rapidly growing category of malpractice claims in the U.S. Most involve switch of specimens and most involve needle biopsy of prostate and breast

31. Anatomic Pathology Errors Pre Analytic Phase • Errors in thyroid gland FNA with relatively high false positive and false negative rates – quality of tissue sampling by non-pathologists • FNA\histologic correlation reveals ¼ of thyroid cancer patients are misdiagnosed as not having cancer due to: • Errors in specimen quality • Misinterpretation Raab et al ASLP 2006

32. Anatomic Pathology Errors Analytic Phase • In a 4 hospital review up to 12% of tissues examined by pathologists resulted in errors, more than 1/3 were associated with harm (AHRQ funded) Raab, Cancer 2005 • Up to 15% of patients with lung mass misdiagnosed due to pathology errors, different rates among hospitals due to “Big Dog” effect and using different methods

33. Anatomic Pathology Errors Analytical Phase • Average discrepancy frequency in pathology reports from74 labs on secondary review is 6.7% with 5% of these having an affect on patient care (1% of all cases) • Canadian Pathology Error Rates: • Retro 14.1% overall rate 1.2% major • Prospect 13% overall 1.7% major Lind 1995 AJSP

34. Anatomic Pathology Errors Analytic Phase • Most studies are single institution hence variation • However multi institutional studies reveal a discrepancy rate of 6.7% with between 1 and 1.7% causing harm • What is an acceptable level of performance?

35. Anatomic Pathology Errors Analytic Phase • Consider that a 1% error rate equates to 10,000 errors per million • Industrial six sigma standard is 3.4 defects per million • Industry average – four sigma = 6210 defects per million • Should a six sigma standard apply to pathologists?

36. Anatomic Pathology Errors Post Analytic Phase • Two aspects of the post analytic phase that are the most important: • completeness of reporting; 28.4% increase in complete reporting using computer based synoptic reports • Communication of critical results and customization of critical values for each institution

37. Anatomic Pathology Errors Errors • Medical Quality Improvement is most effective if collection, processing, interpretation and connection to care providers are considered as an integrated system

38. Anatomic Pathology Errors Solutions/Tools • Have a plan and priorities for quality improvement and safety, consider: • Health Authority priorities • Standards of professional bodies • Guidance in literature • Performance data • Internal assessment – Process map • Identify priority projects • Mission • Culturally aligned teams • Improvement methods (PDCA) and root cause analysis, lean design 6 sigma leap frog

39. Anatomic Pathology Errors

40. Anatomic Pathology Errors Solutions/Tools Error types and test-cycle phases.

41. Anatomic Pathology Errors Solutions/Tools • Standardization • Terms, language, processes, tasks – work is to be done in a certain way • Adopting standardized, structured, synoptic reporting formats province wide • Consider computerized capture of structured data/synoptic reports linked to databases allowing best practice comparisons, information distribution, trend analysis and discrepancy identification • e.g. mTuitive

42. Anatomic Pathology Errors Solutions/Tools • Peer review • Blinded unbiased double slide review, selecting areas of high risk for error • Amended reports are decreased with 2nd pathologist review Nakhleh et al APLM 1998 • Prostate cancer – impact of 2nd pathologist on Gleason score: • 25.2% change • 14.8% change in management Thomas et al Brachytherapy 2007

43. Anatomic Pathology Errors Solutions/Tools • Double viewing dilemma: Is error reduction frequency sufficiently high to warrant the effort? • Consider digital pathology system (e.g. ScanScope) • Digitize slides • Desktop computer viewing • Multiple viewer conferencing • Can Link through telepathology to remote locations/single pathologists • Improved turnaround and better use of path times • Can correlate slides with CT and MRI scans

44. Anatomic Pathology Errors Solutions/Tools • Frozen/permanent section, discordant, monitoring – sustained improvement in performance Raab et al ADLM 2006 • FNA / Histologic correlation • Toyota production system redesign – standard terminology and immediate interpretation • Fewer diagnostic errors Raab et al ASCP 2006 • Improving skill / concentrating expertise in FNA • Cytology/Histologic correlation Q tracks program showed improvement in pap. smear performance in preanalytic sampling                                         Raab et al  APLM Jan 08

45. Anatomic Pathology Errors Solutions/Tools • Adopt a system for measuring performance of key processes • e.g. IQLM (U.S.) – 12 core indicators to evaluate lab quality • Participation in cooperative programs access multiple institutions/databases: • Q-Tracks • Q-Probes • System wide approach to reporting critical values • Conference, random, focused, amended report, tumour board reviews

46. Anatomic Pathology Errors Solutions/Tools • Improve access to clinical information • Electronic Medical Record • Better defined linkages to a large variety of clinical microsystems (users) • A culture that supports change • Teamwork • Willingness to challenge each other • Acknowledging error in a non-punitive way • Sharing performance information • Knowledgeable well trained staff • Departmental CME • Education in QI and safety methods

47. Diagnostic Imaging Errors • Technological and manpower factors influence quality in diagnostic imaging • Radiology’s Achilles heel: • Error and variation in the interpretation of the Roentgen Image, now the weakest aspect of clinical imaging Robinson, St. James UH Leads, UK 1997

48. Diagnostic Imaging Errors • Observations of Henry Garland in 1959: • 30% of chest radiographs that are positive for disease will be missed • Awakened the profession to the extent of errors • But have things changed? • Goddard et al BJR 2001 – little change in past 50 years • Internal error rate by same radiologist can be as high as 25% - 30%

49. Diagnostic Imaging Errors • Shively – Imaging economics 2003: • Many could be avoided if a simple protocol followed • Errors in stroke CT fell from 15% to 1% • Shriger, JAMA 1998 • 49% of radiologists reading CT Scans as part of a large study missed at least 1 stroke

50. Diagnostic Imaging Errors • Turkington et al PMJ • 14 out of 57 cases of confirmed lung cancer missed • Delays in diagnosis and treatment • Non-radiologists in emergency departments – rate of misinterpreted radiographs is high (many studies) 20-25% for CT scans