Jonathan Underhill Associate Director, Medicines Evidence NICE Medicines and Prescribing Centre

1. Making Decisions Better…….Evidence-informed decision making……how to feel comfortable with not knowing everything….. working with our human nature, not against it Jonathan Underhill Associate Director, Medicines Evidence NICE Medicines and Prescribing Centre www.nice.org.uk/mpc Jonathan.underhill@nice.org.uk

2. What I want to talk about How we make decisions How to manage information overload How to keep up to date

3. What I want to talk about How we make decisions How to manage information overload How to keep up to date

4. Humans make decisions by…… Small number of variables + Allocate value to those variables + Time frame = DECISION HOW?

5. Allocating value to those variables • Brief reading • Talking to other people

8. Rx Diagnosis

9. Can this approach let you down???

10. It is most likely that Steve is a …… Farmer Pharmacist Disc jockey Librarian Member of Parliament Steve is very shy and withdrawn, invariably helpful, but with little interest in people. He has a need for order and structure and a passion for detail

11. How we acquire and use information Where did you get the information from to make that decision about Noah and the sheep? If you had had time, what would you have done to make sure you had the right answer?

12. Information and decision making Most decisions are based on what we think is the evidence, not what we know is the evidence No one has time to appraise all of the evidence on everything, and even if that were possible the human brain can’t recall and compute it, and certainly not in a 10 minute primary care consultation We use brief reading and talking to other people as our information sources We often use patterns to make a diagnosis We create mindlines ( = patterns) of what to do in common situations

14. How is knowledge managed in primary care?Gabbay and le May BMJ 2004; 329: 1013 – 6. • Not once was a guideline read • Expert computer systems rarely used (never in real time) • Shortcuts to evidence • free magazines • network of trusted colleagues (rarely if ever questioned) • Pharma reps – considerable scepticism (but not without influence) • Pharmaceutical adviser – highly trusted source. “Clinicians rarely accessed, appraised, and used explicit evidence directly from research or other formal sources; rare exceptions were where they might consult such sources after dealing with a case that had particularly challenged them.”

15. “Instead, they relied on what we have called "mindlines,"collectively reinforced, internalised tacit guidelines, whichwere informed by brief reading, but mainly by their interactionswith each other and with opinion leaders, patients, and pharmaceuticalrepresentatives and by other sources of largely tacit knowledgethat built on their early training and their own and their colleagues'experience.”

16. Eastern Iowa, 103 family doctors. If you ask doctors, they say they need information about once a week. But if you debrief them, they raise about 2 questions for every three patients Answers to most questions were not immediately pursued. Doctors spent an average of less than 2 minutes pursuing an answer, and they used readily available print and human resources. Only two questions (out of over 1100) led to a formal literature search. Information habits of doctorsEly JW,et al. BMJ 1999; 319: 358-361Covell DG et al. Ann Intern Med 1985; 103: 596-9) 16

17. Information and decision making Most decisions are based on what we think is the evidence, not what we know is the evidence No one has time to appraise all of the evidence on everything, and even if that were possible the human brain can’t recall and compute it, and certainly not in a 10 minute primary care consultation We use brief reading and talking to other people as our information sources We often use patterns to make a diagnosis We create mindlines ( = patterns) of what to do in common situations

18. 52 cognitive biases Anchoring bias – early salient feature Ascertainment bias – thinking shaped by prior expectation Availability bias – recent experience dominates evidence Bandwagon effect – we do it this way here Omission bias – natural disease progression preferred to those occuring due to action of physician Sutton’s slip – going for the obvious Gambler’s fallacy – I’ve seen 3 recently; this can’t be a fourth Search satisfycing – found one thing, ignore others Vertical line failure – routine repetitive tasks leading to thinking in silo Blind spot bias – other people are susceptible to these biases but I am not

19. What I want to talk about How we make decisions How to manage information overload How to keep up to date

20. Information Management

21. More reading? • Potential journals 10,000 • Potential new articles per week 40,000 • Even if 97% are not relevant (no POOs) 1,200 • Time to read each article 15minutes • 10h a day, 6 days a week = 240 articles. • So at the end of the first week you are about 4 weeks behind in your reading. • At the end of the first month, you are 4 months behind in your reading. • And at the end of the first year you are almost 5 years behind in your reading.

22. Effect of Exercise on Pain in Knee OA Roddy E, et al. Ann Rheum Dis 2005; 64: 544-8

23. How can we keep up?Sackett D et al BMJ 1996;312:71-72 “The difficulties that clinicians face in keeping abreast of all the medical advances reported in primary journals are obvious from a comparison of the time required for reading for general medicine, enough to examine 19 articles per day, 365 days per year with the time available well under an hour a week by British medical consultants, even on self reports.” How to best use use your Golden Hour?

24. What I want to talk about How we make decisions How to manage information overload How to keep up to date

25. “Better is possible.It does not take genius, it takes diligence, it takes a clarity of purpose, it takes ingenuity, it takes a willingness to try.” So is there a better way???? www.gawande.com/

26. What do we know about how people make decisions? • Behavioural economics and cognitive psychology: • Bounded rationality (Herbert Simon 1978) • Dual process theory (Dan Kahneman 2002) • Most decisions are informed by brief reading and talking to other people

27. How can work with this? • We all need a system for keeping up to date: • Hunting: find the best possible answer to a specific question and recognise it as such, quickly and efficiently. • Foraging: be alerted to new, important, relevant, valid information that requires a change in practice • Hot synching: update your brain once or twice a year on the 30-40 conditions you see most frequently

28. Pre-digested sources of evidence from trusted sources:Public-sector ethosPublished methodology of how producedTranslation of evidence into practiceContext of the rest of the evidence

29. Finding the ‘best answer’, first timeSlawson DC and Shaughnessy AF Cochrane Library NICE etc Clinical Evidence InfoPOEMs, CKS BestTreatments EBM DTB MeReC “Ivy League” journals Usefulness Medline, Google scholar Textbooks

30. Medicines awarenesswww.nice.org.uk/mpc

31. Medicines Awareness Service

32. Prescribing support inc. • NICE CKS • Awareness service: • MAD, MAW, MECs • Evidence summaries • British National Formulary: • Book • Web • Apps • Good practice guidance • Key therapeutic topics for QIPP • NICE Evidence • Pathways • eLearning tools for NICE Guidance

33. BNFwww.nice.org.uk/mpc • Monthly updates for digital versions • Book going to once a year • User research • Exploring ways to develop content and integrate content with NICE CKS, NICE Guidance/pathways etc

34. Best use of a clinician’s ‘Golden Hour’? • Give up on reading primary research: • You don’t have time and you wont be able to do it • Design your CPD based on these principles: • Foraging: • 2-3 bits of key new research, summarised for you and set in the context of the rest of the evidence (MEC) • Evidence awareness service e.g. NICE MPC Medicines Awareness Daily Weekly • Hot synching • Be aware when new NICE/SIGN guidance comes out • Take time to digest it, talk to others about it • Ask yourself, “what are the important changes since I last updated my brain?” • ONLY about conditions YOU see commonly • Hunting: • Much more difficult • Use the information pyramid (NHS Evidence/TRIP)

35. Cognitive Reflective Test The test distinguishes intuitive (system 1) from analytical (system 2) processing…. …….the ability to resist first response that comes to mind Of 3,428 people tested only 17% got all 3 correct 33% answered all three incorrectly Frederick 2002 (MIT)

36. Teaching “Think as well as blink”? T

37. Better is possible • Self-awareness (meta-cognition) • “the right system at the right time” • Information management • Teaching “Think as well as blink”?

38. Decision = Experiences + Evidence system 1system 2 Usual practice: Decision = Experiences + Evidence Making decisions better: Decision = Experiences + Evidence

39. If you want more…..www.npc.nhs.uk/evidence

40. www.NPC.nhs.uk InnovAiT: Autumn / Winter 2009-2010