AAIM – ABIM PIM Project

1. AAIM – ABIM PIM Project Audits, Opinion Leaders, and Workshops: Do they help to improve care?

2. Objectives • Review the audit cycle and important basic principles • Define the characteristics of a physician opinion leader • Review evidence of effectiveness for audits, workshops, and opinion leaders to improve care

3. Definition • Many definitions • Basic definition (Crombie, England) • Audit is the process of reviewing the delivery of health care to identify deficiencies so that they may be remedied

4. Key Elements • Purpose • Method of audit • Area of investigation • Structure • Process • Outcomes

5. Methods to perform audits • Medical records • Format • Electronic • Paper-based • Clinical focus • Provider care • Laboratory, radiology, etc.

6. Methods to perform audits • Timing • Retrospective versus prospective • Type of review • Implicit review • Explicit review: Adherence to criteria

7. Medical Records: Implicit Review • Reviewer judges quality based on own implicit standards for care • No explicit criteria used or given • Questions are subjective judgments of quality determined by knowledgeable peers P Pronovost, Johns Hopkins

8. Implicit Review: Issues • May be more appropriate for complex cases • E.g. sentinel events • Suffers from poor reliability • Minimum of 5-7 reviewers • Validity unknown • Training probably not effective

9. Medical Records: Explicit Review • Criteria usually low inference type behaviors • Criteria are based on thorough review of evidence • Expert judgment mostly applied in measure development phase • E.g. What would “count” as smoking cessation counseling? • Quality judgment is incorporated into criteria P Pronovost, Johns Hopkins

10. Methods to perform audits • Surveys • Patients • Other users • Administrative data • Claims, billing

11. The Audit Cycle Observe current practice Implement change: PDSA Set standards of care Compare practice with standard

12. Audits & Feedback: Effectiveness • Cochrane systematic review • Audit and feedback can be effective • Effects range from small to moderate in most studies • Effect size appears to be greater when baseline compliance is low • Optimal combination of other interventions with audit and feedback yet to defined

13. Audits & Feedback: Effectiveness • Self audit of practice • Effects on practice largely unknown • Yale Primary Care Study • Moderate improvements in care processes most in the control of the resident • Foot exams, ECGs, immunization • Connecticut Diabetes PIM study • Physicians performed audit themselves • All but one found the audit experience highly valuable

14. Physician Opinion Leader Among peers: • Key characteristics • Good communication and teaching skills • Humanistic and caring • Knowledgeable and competent • Trusted by colleagues to evaluate new knowledge and assess the value of new medical practice Hiss, 1978

15. Physician Opinion Leader Within hospitals: • Personal commitment • Professional credibility • Quality improvement behaviors/skills • Key interactions with non-physicians • Nurses, administrators Bradley, 2001

16. Role: Physician Opinion Leader • Advocates and endorses evidenced based care/guidelines • Willing innovator • Contributes time and leadership to multidisciplinary teams, quality projects • Empowers others to facilitate change • Uses data to raise awareness, promote change Bradley, 2001

17. Opinion Leader: Effectiveness • Randomized controlled trials • Two trials showed significant effect of physician opinion leader on some processes of care • Qualitative studies • Demonstrate that opinion leaders “essential” element in QI projects

18. Epidemiology of Innovation Rogers EM. Diffusion of Innovations, 1983 Which type(s) would be the best ones to join the project team?

19. CME Workshops • Cochrane systematic review • Reviewed 32 studies involving total of 2995 health professionals • Interactive workshops can lead to substantial changes in professional practice • Didactic alone approach did not lead to meaningful changes