Scot M. Silverstein, MD Adjunct Professor, Institute for Healthcare Informatics

1. Critical Thinking on Building Trusted, Transformative Medical Information: Improving Health IT as the First Step Scot M. Silverstein, MD Adjunct Professor, Institute for Healthcare Informatics College of Information Science & Technology Drexel University, Philadelphia, PA

3. Varied Perspectives: Clinical medicine Medical Informatics Information Technology Safety R&D ethics Management

4. Victor P. Satinsky MD, Summer Science Training ProgramHahnemann Medical College, Philadelphia, 1972 Satinsky clamp Critical Thinking, Always – Or:

5. Your Patient’s Dead

6. Critical Thinking ≠ Criticize Reflective, inquisitive, logical thinking that is focused on deciding what to believe or do.

7. Critical Thoughts on Health IT Trust

8. Fundamental Premise Health IT Must Be Trusted By Users and Patients [And Be Free ofMajor Downsides] - As A Primary Step Before HIT Can Optimally Benefit Healthcare

9. First, do no harm I am not suggesting anything new. In fact, I am suggesting something old.

10. To Be Trusted and Do No Harm, Health IT Must Be “Done Well” • Health IT can achieve many of the promises made about it, but only if done well. • Behind the simple words “done well” lies massive sociotechnical (‘issues at the boundary of people and technology’) complexity. • Key point: Computerizing healthcare appears to be more of a “wicked problem” than a tractable one. • Wicked problem: problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. Because of complex interdependencies, the effort to solve one aspect of a wicked problem may reveal or create other problems.

11. New/Innovative Information Technologies Are NotInherently Trustworthy • IT is not value neutral: its use creates winners and losers. • IT use leads to multiple, and often paradoxical, effects. • IT use has moral, and ethical aspects and these have social consequences. • IT follows trajectories and these trajectories often favor the status quo. (from Kling, Rosenbaum & Sawyer, “Understanding And Communicating Social Informatics”, Information Today, 2005 (Amazon link here)

12. Trust Must Be Earned

13. Has Health IT Earned our Trust?- “Hyperenthusiastic” promises HAL9000, 2001 (?)

14. “We will revolutionize healthcare” - CEOs of several major US health IT sellers, leading a CIO panel at Microsoft Healthcare User’s Group, ca. 1997. • I asked “who in this room has ever studied medicine or cared for patients?” – answer – almost nobody. • “Bellicose grandiosity” about “revolutionizing medicine” surprised me. • "IBM's Watson could usher in new era of medicine" - Computerworld, February 17, 2011

15. Trust: On “Revolutions” and “New Eras” Can have unintended consequences, or turn out badly Storming the Bastille, 1789

16. Unintended Consequences Satirical cartoon lampooning the excesses of the Revolution - as symbolized through the guillotine

17. Trust: Who’s Temperate, and Who’s Hyperenthusiastic (“irrationally exuberant”)? With ICT we will deterministically “revolutionize” and “transform” medicine, drastically reduce errors and costs, and increase efficiency and productivity immensely (or similar) Or: We will facilitate clinicians in delivery of healthcare and patients in self-advocacy, with ICT developed slowly and carefully, and treated as experimental devices.

18. Is Clinical IT A Proven Technology? • Or does it remain experimental? • Is national-scale rollout a human subjects experiment? Does it lack the protections of other human subjects experimentation and for IT in mission critical settings?

19. Do We Want To Be on the “Bleeding Edge” of Health IT? Bleeding edge: merger of leading edge/cutting edge (razor edge) – what does this imply? Or do we want to be merely on the leading edge? Do we want to be on the edge at all (‘First, do no harm”), considering the domain?

20. Trust: Executive Dean for Institutional Affairs & Senior Advisor for Global Health Initiatives, Prominent Academic Medical Center • No need for patient informed consent, as health IT is a QA activity rather than research • Patient harm from health IT is awful , but not induced by a QA study. • Do sentiments like this invite trust?

21. Is Health IT Being “Done Well?” • U.S. National Academies, National Research Council, Jan. 2009: “CURRENT APPROACHES TO U.S. HEALTH CARE INFORMATION TECHNOLOGY ARE INSUFFICIENT”, done at the “best” HIT centers, Octo Barnett/Bill Stead (link): • Current efforts aimed at the nationwide deployment of health care information technology (IT) will not be sufficient to achieve medical leaders' vision of health care in the 21st century and may even set back the cause. • Current health care IT systems offer little cognitive support; clinicians spend a great deal of time sifting through large amounts of raw data (such as lab and other test results) and integrating it with their medical knowledge to form a whole picture of the patient. • Data entered mainly to comply with regulations or to defend against lawsuits, rather than to improve care. As a result, valuable time and energy is spent managing data as opposed to understanding the patient.

22. Is Health IT Being Done Well? U. Sydney, 2011

23. Done Well? Health IT Success and Failure - Recommendations from Literature and an AMIA Workshop Kaplan and Harris-Salamone, Journal of the American Medical Informatics Association 2009;16(3):291-299 (evolutionary history of this paper is here). • IT implementation projects are often not successful across many industry sectors, including health IT. • For years, problems have plagued the implementation of health IT applications … All HIT projects fail in some way. • even more disturbing when high-profile failures, partial successes, and unsustainable IT undertakings are coupled with accumulating evidence of negative unintended consequences, increased error rates accompanying IT use, and the need for workarounds.

24. Done well? NPfIT in the NHS • UK press: The £12.7bn National Programme for IT is being ended after years of delays, technical difficulties, contractual disputes and rising costs. • (Not to negate the many positive developments and valuable lessons learned)

25. Done well? U.S. NIST Report on HIT Usability, 2012 Usability leaves much to be desired. Page 10: “Use error” is a term used very specifically to refer to user interface designs that will engender users to make errors of commission or omission. It is true that users do make errors, but many errors are due not to user error per se but due to designs that are flawed, e.g., poorly written messaging, misuse of color-coding conventions, omission of information, etc.

26. Special Accommodation – Does This Inspire Trust? • The health IT industry for decades has received unprecedented special regulatory accommodation compared to other healthcare sectors (e.g., pharma, tangible medical devices) as well as other safety-sensitive industries (aviation, automotive, energy etc.) • No regulation • Little accountability • Has the accommodation truly been earned, and, on critical appraisal, is it consistent with, or at odds with the ethics of modern medicine?

27. Clearing up terminology that can impair needed caution– “EHR”

28. Reality: complex enterprise ClinicalResource Management and Workflow ControlSystem – many things can go wrong

29. “Luddites” vs. “Ddulites” • Do physicians/nurses resist clinical IT being pushed by reformers and modernists because they are Luddites (so ignore them)? • OR: • Are physicians/nurses in reality pragmatists resisting clinical IT being pushed by hyperenthusiasts (“Ddulites”) who ignore the downsides and ethical considerations?

30. “Anecdotes” • Are case reports of health IT unintended consequences (UC’s) “anecdotal” and to be played down, while studies of health IT benefits to date solid science? • OR: • Are studies of health IT benefits to date mostly anecdotal (e.g., specialized settings, non-RCT’s) while reports of UC’s are risk management-relevant case report “red flags” pointing to possible systemic problems? • See http://hcrenewal.blogspot.com/2011/08/from-senior-clinician-down-under.html

31. Reality or “Marketing Memes”? • Clinical IT is always effective, beneficent and cost-saving. • No:http://www.ischool.drexel.edu/faculty/ssilverstein/cases/?loc=cases&sloc=readinglist • “Certification” of clinical IT is synonymous with safety • No:http://hcrenewal.blogspot.com/2012/02/hospitals-and-doctors-use-health-it-at.html

32. More Marketing Memes? – ONC/NEJM • "The 'Meaningful Use' Regulation for Electronic Health Records", Blumenthal D., Tavenner M. (NEJM, July 13, 2010) • The widespread use of electronic health records (EHRs) in the United States is inevitable. EHRswillimprove caregivers’ decisions and patients’ outcomes. Once patients experience the benefits of this technology, they will demandnothing less from their providers. Hundreds of thousands of physicians have already seen these benefits in their clinical practice. • Is this science, or marketing? Subst. “drug X” or “device Y” for EHR • Where is the list of peer reviewed, rigorous studies that back the assertions of certainty, and override the body of literature that cast doubt on these assertions? That acronym again!

33. Risks and Harms US FDA MAUDE database - http://hcrenewal.blogspot.com/2011/01/maude-and-hit-risk-mother-mary-what-in.html Infant deaths - http://hcrenewal.blogspot.com/2011/06/babys-death-spotlights-safety-risks.html Missing “difficult intubation flag” death - http://hcrenewal.blogspot.com/2011/09/sweet-death-that-wasnt-very-sweet-how_24.html NHS cases - Health informatics — Guidance on the management of clinical risk relating to the deployment and use of health software, Annex A Mass prescription errors - http://hcrenewal.blogspot.com/2011/11/lifespan-rhode-island-yet-another.html Mass Security Breaches - http://hcrenewal.blogspot.com/2012/01/2011-closes-on-note-of-electronic.html

34. Risks: FDA - Impediments to transparency (MAUDE) Top Secret? (Internal FDA memorandum of Feb. 23, 2010 ("not intended for public use") to Jeffrey Shuren on HIT risks, http://www.ischool.drexel.edu/faculty/ssilverstein/Internal-FDA-Report-on-Adverse-Events-Involving-Health-Information-IT.pdf)

35. Risks: IOM - Impediments to transparency No specific repository Hold Harmless Clause Gag clause Unacceptable safety risks IOM (Institute of Medicine). 2012. Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, DC: The National Academies Press, pg. S-2.

36. Social Impediment to Transparency • Fear of Pushback for Candor

37. HIT Safety Surveillance: Rules of thumb • If you don’t take a temperature, you will not find a fever. • If you don’t ask the right questions, at the right time, and provide a venue for unbiased data provision, your results will be of diminished or no value. • Passive analysis of clinical IT risk/harms data in an industry where lack of knowledge and misconceptions abound will produce suboptimal results. • Ross Koppel, PhD - Feb. 25, 2010 ONC Policy Committee Adoption/Certification Workgroup on IT safety: "We don't know 99 percent of the medication ordering errors that are made [due to difficulty in recognition, lack of proper studies and other factors]. If 100 percent of the known errors were reported, that would be 1 percent of the [true] total. But the data suggests that the maximum on voluntary reporting is about 5 percent. So 5 percent of 1 percent that is what we know is reported...." • Knowledge discovery techniques such as data mining can help, but cannot discover that which is not there - you can't data mine if you don't have relevant data points in the dataset.

38. Can we Ask The Right Questions? Taxonomy of health IT hazards – FDA (2010)

39. Taxonomy of hazards – AHRQ Health IT Hazard Manager (2012) AHRQ Publication No. 12-0058-EF, May 2012

40. Thinking Critically Once More • Risk taxonomies were mostly observationally developed. • Could have been done by industry a long time ago. • That they are being developed only now is strong indication industry has not done best self-policing. • Aware, or should have been aware, or should have made it their business to be aware.

41. Problem in a Nutshell Rigor, ethics and skepticism of medical science itself not applied in the domain of health IT.

42. Solutions: Paradigm Shifts – (1) Ideology Are health IT systems proven, business computing systems that happen to be used by clinicians, OR: Are health IT systems experimental, ‘virtual’ clinical tools that happen to reside on computers?

43. Paradigm Shift (2) – Rhetoric Is health IT is a facilitator to allow clinicians to provide better healthcare, OR: Is health IT is a cybernetic miracle that will “revolutionize” medicine?

44. Paradigm Shift (3) - Skepticism Pessimism, Computer Failure, and Information Systems Development in the Public Sector. (2007, Shaun Goldfinch, then at University of Otago, New Zealand).

45. Goldfinch The majority of information systems developments are unsuccessful. The larger the development, the more likely it will be unsuccessful. Despite the persistence of this problem for decades and the expenditure of vast sums of money, computer failure has received surprisingly little attention in the public administration literature. This article outlines the problems of enthusiasm and the problems of control, as well as the overwhelming complexity, that make the failure of large developments almost inevitable. Rather than the positive view found in much of the public administration literature, the author suggests a pessimism when it comes to information systems development. Aims for information technology should be modest ones … The author argues for a public official as a recalcitrant, suspicious, and skeptical adopter of IT.

46. Health IT pioneers – Octo Barnett’s Ten Commandments (1970)

47. Barnett’s Ten Commandments, 1970 Dr. Octo Barnett, Harvard (from Collen's "A history of Medical Informatics in the United States, 1950-1990“): 1. Thou shall know what you want to do2. Thou shall construct modular systems - given chaotic nature of hospitals3. Thou shall build a computer system that can evolve in a graceful fashion4. Thou shall build a system that allows easy and rapid programming development and modification5. Thou shall build a system that has consistently rapid response time and is easy for the non-computernik to use6. Thou shall have duplicate hardware systems7. Thou shall build and implement your system in a joint effort with real users in a real situation with real problems8. Thou shall be concerned with realities of the cost and projected benefit of the computer system9. Innovation in computer technology is not enough; there must be a commitment to the potentials of radical change in other aspects of healthcare delivery, particularly those having to do with organization and manpower utilization10. Be optimistic about the future, supportive of good work that is being done, passionate in your commitment, but always guided by a fundamental skepticism.

48. Paradigm Shift (4) - Use What We Already Know! Improve HIT via the same methodologies and ethical considerations applicable for decades (or more) in the healthcare delivery sector such as medical devices, pharmaceuticals, and research (and other risk-prone industries e.g., aviation and automotive).  Meaningful regulation, robust validation of software quality, safety and efficacy, post marketing surveillance, patient protections, etc.

49. Conclusion 2009 National Research Council report: accelerating interdisciplinary research in biomedical informatics, computer science, social science, and health care engineering will be essential to perfect HIT. I believe that in addition to this research, meaningful oversight of HIT is necessary to strengthen the trust – by clinicians, by patients – trust that is truly needed before this technology and its data and information can “transform medicine.” ** Health IT must itself be transformed** before health IT can transform** medicine. * that is, into a more evidence-based and just endeavor.

50. More Reading • “Common Examples of Healthcare IT Difficulty.” http://www.ischool.drexel.edu/faculty/ssilverstein/cases/ • Healthcare Renewal Blog (multi-author). http://hcrenewal.blogspot.com • “Reading List on Health IT”. http://www.ischool.drexel.edu/faculty/ssilverstein/cases/?loc=cases&sloc=readinglist