Making Guidelines WORK Richard N. Shiffman, MD, MCIS Yale School of Medicine

1. NYAM Teach 2011 Making Guidelines WORK Richard N. Shiffman, MD, MCISYale School of Medicine

2. Disclosure • Funding from AHRQ and NLM • Will describe several applications developed in our laboratory available gratis for non-commercial use

3. Today • Making Guidelines WORK • Improving the delivery of knowledge to the point of care • Computer-based clinical decision support • • Making Guidelines THAT Work • – Improving the product • – Clarity, transparency, and implementability • – GLIA and BRIDGE-Wiz

4. Guidelines have problems… • Cluzeau (Int J Qual Healthcare 1999), Shaneyfelt (JAMA 1999) majority of guidelines failed quality criteria • Grilli: 431 specialty society guidelines (Lancet 2000) • 82% did not apply explicit criteria to grade evidence • 87% did not report whether a literature search was performed • 67% did not describe type of professionals involved in development • Shaneyfelt (JAMA 2009): persisting biases; lack of specificity, flexibility, regular updating • Alonso-Coello: in 42 reviews of 626 guidelines over past 20 years, mean quality scores for rigor of development, stakeholder involvement, editorial independence, and applicability are “moderate” or “low” (GIN 2009)

5. CPGs are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options

6. Guidance on the use of glitazones for the treatment of type 2 diabetes • For people with type 2 diabetes, the use of a glitazone as second-line therapy added to either metformin or a sulphonylurea--as an alternative to treatment with a combination of metformin and a sulphonylurea-- is not recommended except for those who are unable to take metformin and a sulphonylurea in combination because of intolerance or a contraindication to one of the drugs. In this instance, the glitazone should replace in the combination the drug that is poorly tolerated or contraindicated.

7. Authors Should Be Explicit About WHEN {under what circumstances} WHO {in the Intended Audience} Ought to {with what level of obligation} DO WHAT {To WHOM} {which members of the target population} HOW WHY IF THEN Denominator Numerator

8. Guidance on the use of glitazones for the treatment of type 2 diabetes • If a patient is unable to take the combination of metformin and sulfonylurea (because of intolerance or contraindication), the clinician should prescribe a glitazone to replace the drug that is not tolerated.

9. Guidance on the use of glitazones for the treatment of type 2 diabetes • If a patient is unable to take the combination of metformin and sulfonylurea (because of intolerance or contraindication), the clinician should prescribe a glitazone to replace the drug that is not tolerated. UNDER WHAT CIRCUMSTANCES? WHO? To do WHAT? OUGHT?

10. Statement of fact is NOT a recommendation • Adjuvant hormone therapy for locally advanced breast cancer results in improved survival in the long term. • Clinicians should prescribe adjuvant hormone therapy for locally advanced breast cancer (when/unless?)…

11. How “Should” We Write Guideline Recommendations: Interpretation of Deontic Terminology • Goal: To describe the level of obligation conveyed by deontic terms commonly used in practice guidelines • Can level of obligation be standardized? Lomotan E, et al. Qual & Safety in Health Care 2010

12. Measuring Obligation 0 50 100

13. Level of Obligation

15. Musts (19/1250 – 1.5%) • Narcotic use must be carefully titrated and supervised. • Clinicians working in juvenile justice settings must be vigilant for personal safety and security issues and aware of actions that may compromise their safety and/or the safety and containment of the incarcerated youth • Nurses working with individuals with asthma must have the appropriate knowledge and skills to identify the level of asthma control, provide basic asthma education, conduct appropriate referrals to physician and community resources • Treatment of duodenal adenomas depends on adenoma size and the presence of severe dysplasia. Small tubular adenomas with mild dysplasia can be kept under surveillance, but adenomas with severe dysplasia must be removed

16. The Dreaded “Consider” • The Expert Panel concludes that initiating daily long-term control therapy should be considered for reducing impairment in infants and young children who consistently require symptomatic treatment more than 2 days per week for a period of more than 4 weeks (Evidence D). • Referral may be considered if a child 0–4 years of age requires step 2 care or a child 5–11 years of age requires step 3 care.

17. Measurement • If you can’t measure it, you can’t manage it. • If you don’t measure it, you can’t improve it. Peter Drucker

18. Action-Types Gather Data Interpret Act Test Conclude Prescribe Procedure Monitor Educate/counsel Consult/refer Inquire Examine Advocate Document Prepare Prevent Dispose

19. Action-Type Pattern: Prescribe • Drug information • Safety alerts (allergy, drug-drug, drug-disease, drug-lab) • Formulary check • Dosage calculation • Pharmacy transmission • Patient education • Corollary orders

20. A Transparent Process for Generating Recommendations

21. A transparent development process makes clear… • How authors weighed • evidence • pathophysiologic reasoning (first principles) • expert experience • patients’ and society’s values • Allows users to judge reasonableness of recommendations

22. Requires untangling and specifying 2 related (but distinct) concepts • Quality of evidence • Recommendation strength {Elegant and erudite work of GRADE Collaboration} <---developers’ focus <-what implementers need to know to design systems that influence care • level of expected adherence • level of enforcement / incentive

23. Evidence Quality • An indication of the authors’confidence in their appraisal of benefits and harms • Based on an analysis of the validity, consistency, and directness of the evidence supporting a recommendation

24. Recommendation Strength • Implementers need to understand experts’ assessment of strength of recommendation • Communicates authors’ assessment of the importance of adherence • Levels based on aggregate evidence quality and balance of anticipated benefits and harms • Strong recommendation (“MUST”) • Recommendation (“SHOULD”) • Option (“MAY”)

25. Grading Recommendation Strength Strong Option Rec Option No Rec Strong Rec

26. GuideLine Implementability Appraisal • Goals • To identify intrinsic obstacles to implementation, i.e., those that are within the purview of guideline developers • To provide feedback to guideline authors to anticipate and address these obstacles before a draft guideline is finalized • To assist implementers in guideline selection and to target attention toward anticipated obstacles • GLIA (and eGLIA) available from http://gem.med.yale.edu/glia BMC Medical Informatics and Decision Making 2005

27. GLIA v2.0 Dimensions • Decidability - precisely under what conditions (e.g., age, gender, clinical findings, lab results) to do something • Executability - exactly what to do under the circumstances defined) • Validity - the degree to which a recommendation reflects the intent of the developer and the strength of evidence • Flexibility - the degree to which a recommendation permits interpretation and allows for alternatives in its execution • Effect on process of care - the degree to which a recommendation impacts upon the usual workflow in a typical care setting

28. GLIA v 2.0 Dimensions (cont’d) • Measurability – the degree to which the guideline identifies markers or endpoints to track the effects of implementation of this recommendation • Novelty/innovation - the degree to which a recommendation proposes behaviors considered unconventional by clinicians or patients • Computability - the ease with which a recommendation can be operationalized in an electronic information system

29. Bridge the Gap Between Authors and ImplementersWith BRIDGE-Wiz(Building Recommendations In a Developer’s Guideline Editor)

30. BRIDGE-Wiz • Displays a sequence of screens representing chunks of information about a recommendation • The authors systematically and sequentially determine: • action(s) to be recommended • condition(s) under which the action is to be performed • benefits, risks, harms, and costs of the proposed action • the quality of the evidence supporting the action. • The program’s output is an IF…THEN rule and supporting recommendation profile

31. Bridge-Wiz Demo

32. BRIDGE-WizBuilding Recommendations in a Developer’s Guideline Editor Formalizes a process for writing implementable recommendations Focuses discussion Incorporates prompts based on COGS to improve guideline quality Controlled natural language Offers verb choices based on action-type Traps and disallows use of “consider” Discourages “statement of fact” masquerading as recommendation Limits boolean connectors to all ANDs or ORs in a statement Incorporates decidability and executability checks Requires systematic appraisal of evidence quality and benefit-harms Suggests appropriate obligation term (deontic modal) Output includes a high-level “rule” and an evidence profile

33. Making Guidelines Work

34. Interventions to Influence Practice Grol, Grimshaw Lancet 2003 • Education (conferences, courses) • Audit & feedback • Financial incentives/disincentives • Patient-mediated interventions • Computer based decision support

35. Clinical Decision Support: Definition • Use of the computer to bring relevant knowledge to bear on the health care and well-being of a patient (Greenes). • Systems that link health observations with health knowledge to influence health choices by clinicians for improved health care (Hayward)

36. Computer-Based Decision SupportSystematic Reviews • Computer-based decision support regularly—but not always—improves the process of care • Outcomes—though infrequently measured—sometimes improve

37. Identifying Features Critical to Success Kawamoto K. BMJ 2005 • Significant improvement in practice in 68% of 70 trials • Predictors of improved practice: • Automatic provision of DS as part of workflow • Providing DS at time and site of decision making • Providing recommendations, not just assessments • Providing periodic performance feedback • Sharing recommendations with patients • Requesting reasons for not following recommendations

38. Allergy Alert

39. Alert Reminder Documentation template Flowsheet Algorithm Calculator Infobutton Palette of CDS Interventions Order Facilitator

40. Selected Guideline • Asthma • EPR3 Diagnosis and Management of Asthma from the NHLBI (2007) • Demonstrates challenges involved in implementation of recommendations for chronic management of complex disease 40

43. Real-time calculation and display Prompts for documentation

44. Display of Relevant Past Information Alert Prompts for Assessments Information Access

45. Order Set

46. Customizable Handout Medication Authorization

47. Summary • Making Guidelines WORK • Computer-based clinical decision support • Improving the delivery of knowledge to the point of care • • Making Guidelines THAT Work • – Must address: • – Clarity, transparency, and implementability • – GLIA and BRIDGE-Wiz

48. Thank You! ycmi.med.yale.edu/GLIDES richard.shiffman@yale.edu