Clinical Decision Support Systems

Clinical Decision Support Systems Ida Sim, MD, PhD March 12, 2002 Division of General Internal Medicine, and the Graduate Group in Biological and Medical Informatics UCSF Copyright Ida Sim, 2002. All federal and state rights reserved for all original material presented in this course through any medium, including lecture or print. Clinical Decision Support Systems Medical Informatics

IT and Quality • Information technology touted to improve quality of care • Dimensions • information availability • chart, lab results, allergies; all legible • process efficiency • visit level coding, e-prescribing • intermediate measures • vaccination and screening rates • patient outcomes Clinical Decision Support Systems Medical Informatics

Outline • Clinical decision support systems (CDSS) • definition • methods of reasoning • effectiveness at improving quality • Clinical research informatics • infrastructure for clinical research • systems for supporting clinical research Clinical Decision Support Systems Medical Informatics

What is a CDSS? • Software that is designed to be a direct aid to clinical decision-making in which the characteristics of an individual patient are matched to a computerized clinical knowledge base, and patient-specific assessments or recommendations are then presented to the clinician and/or the patient for a decision (Sim et al, JAMIA, 2001) Clinical Decision Support Systems Medical Informatics

Major Objectives • Diagnostic support • DxPlain, QMR • Drug dosing • aminoglycoside, theophylline, warfarin • Preventive care reminders • vaccinations, mammograms • Disease management • diabetes, hypertension, AIDS, asthma • Test ordering, drug prescription • reducing daily CBCs in hospital, allergy checking • Utilization • referral management, clinic followup Clinical Decision Support Systems Medical Informatics

How Do CDSSs “Think”? • Some systems use more than one method • rule-based • adhoc • non-math method of reasoning about probabilities • e.g., if high WBC AND cough AND fever AND abn. CXR then likelihood of pneumonia is 4 out of 5 • e.g., DxPlain, QMR • bayesian network • formal probabilistic reasoning, extension of decision analysis • neural network • fuzzy logic, genetic algorithms, case-based reasoning, etc. Clinical Decision Support Systems Medical Informatics

Rule-Based Approaches • Forward reasoning (data-driven) • start with data, execute applicable rules, see if new conclusions trigger other rules, and so on • use if sparse data • if high WBC AND cough AND fever AND abn. CXR => pneumonia • if pneumonia => give antibiotics, etc. • Backward reasoning (goal-driven) • start with “goal rule,” determine whether goal rule is true by evaluating the truth of each necessary premise • use if lots of data • patient with lots of findings and symptoms • is this lupus? => are 4 or more ACR criteria satisfied? Clinical Decision Support Systems Medical Informatics

library: purpose: Recommend the use of ampicillin for pneumonia.;; explanation: If the patient has pneumonia, then suggest treatment with ampicillin unless there is a penicillin allergy.;; keywords: pneumonia; penicillin; ampicillin;; citations: 1. HELP Frame Manual, version 1.6. LDS Hospital, August 1989, p.81.;; MLMs and Arden • Medical Logic Modules (MLMs) in Arden Syntax (an international ASTM standard syntax) : • help_amp_for_pneumonia - Ampicillin for Pneumonia • maintenance: • title: Ampicillin for Pneumonia;; • filename: help_amp_for_pneumonia;; • version: 1.00;; • institution: LDS Hospital;; • author: Peter Haug, M.D.; George Hripcsak, M.D.;; • specialist: ;; • date: 1991-05-28;; • validation: testing;; Clinical Decision Support Systems Medical Informatics

Neural Networks • Example of a data-driven data mining method • Finds a non-linear relationship between input parameters and output state • Structure of network • usually input, output, and 1-2 hidden fully connected layers • each connection has a “weight” Clinical Decision Support Systems Medical Informatics

EKG findings Acute MI Rales No Acute MI JVD Response to TNG Neural Network for MI Diagnosis • Inputs (e.g., all patient characteristics in the EMR) • EKG findings (ST elevation, old Q’s) • rales (Yes, No) • JVD (in cm) • Outputs are the set of possible outcomes/diagnoses Clinical Decision Support Systems Medical Informatics

Training the Neural Network • Network gets “trained” • feed network many examples of known patients and their diagnoses • system iteratively adjusts the weights of the connections to find the network “pattern” that associates sets of input variables (patients) with the right output state (MI or not) • In Baxt’s MI neural network • training set: 130 pts with MI, 120 without • test set: 1070 ER patients with anterior chest pain Clinical Decision Support Systems Medical Informatics

Baxt’s Acute MI Neural Net • Evaluation results: prevalence of MI 7% (Lancet, 1996) • Results were driven by non-standard predictors • rales, jugular venous distention • Why isn’t this neural network used more widely? • “black box” nature limits explanatory ability and lessens acceptance • users have to input the variables manually • if EMRs more widely available, these types of systems may be more prevalent Clinical Decision Support Systems Medical Informatics

CDSS Methods • Vast majority of clinically-used CDSSs use rule-based reasoning • problem of combinatorial explosion of rules • Major limitations • how to represent some data (e.g., “looks sick”) • formal, reproducible methods for making clinical decisions • Other major limitation is source of input data • manual input of data by doctors will not work • EMR can enable a new era of CDSSs • But lots can be done with current technology Clinical Decision Support Systems Medical Informatics

CDSS Effectiveness • In controlled trials, only occasional modest benefit found (Hunt, JAMA 1998; updated RB Haynes 2000) • diagnosis: 1/5 studies beneficial • drug dosing: 9/15 • preventive care reminders: 19/26 • Few studies looked at patient outcomes • 6 of 14 showed benefit Clinical Decision Support Systems Medical Informatics

Shortcomings of Literature • Variable study quality • 35% rate >8 on 10 point quality scale (mean ~6.2) • more recent studies better quality • Low power • 5 of 8 studies of patient outcome had low power • Patients randomized to CDSS or not • physicians treated some patients with CDSS, and some without CDSS • this would tend to …. any effects of the CDSS • Probably publication bias Clinical Decision Support Systems Medical Informatics

Shortcomings of Approach (1) • E.g., a hypertension treatment CDSS • Is RCT best design for determining effectiveness? • should randomize MDs, \usually low power • intervention is usually more than just the CDSS • e.g., “buy-in” sessions to HTN guideline underlying CDSS • limited generalizability • applies only to this particular CDSS • integration of CDSS into existing workflow often unique to study site • if CDSS shows no effect, standard RCT gives little insight into why Clinical Decision Support Systems Medical Informatics

Shortcomings of Approach (2) • How would you improve on the Hunt systematic review? • CDSSs are very heterogeneous • does the heterogeneity explain any variation in benefit? • Example: preventive care reminder CDSS • A clerk routinely abstracts preventive care interventions from paper chart into a database. Before each clinic session, nurse runs the CDSS for patients coming in that day. Guideline-based recommendations are printed out on paper and attached to front of chart. Doctor orders preventive care in usual way using paper-based methods Clinical Decision Support Systems Medical Informatics

Heterogeneity of CDSSs • Hypertension treatment CDSS • Clinic has an EMR. During patient visit, CDSS notes that BP and trend is too high. Checks patient’s Cr, diabetes status, cardiac status, current meds and allergies and recommends drug therapy change according to JNC VI guidelines. Presents e-prescription for MD to verify. If verified, order is sent directly to pharmacy and medication list updated. • How to meaningfully characterize CDSSs? • target decision maker (MD, nurse, patient) • urgency of decision (stat result, outpatient screening) • method of delivery (paper, EMR, pager) • force of recommendation (suggestion, requirement) ... Clinical Decision Support Systems Medical Informatics

CONTEXT • Clinical decision • Target patient setting • Point of care • Question orientation • Workflow integration • OUTPUT • Action complexity • Action embedded • Compliance urgency • Force action recommendation • Decision focus • Form information generation • CDSS • Customization • Update mechanism • Unit of analysis • Clinical knowledge source • Mode of information generation • INPUT • Data source • Data source-system intermediary System-user interface System-user interface OR System user/ Processor/Target decision maker Target decision maker System user Processor Typology of CDSSs Clinical Decision Support Systems Medical Informatics

CDSS Effectiveness Summary • Current data suggests CDSSs can improve the process of care and perhaps clinical outcomes • most effective at preventive care reminders • modest at best for drug dosing and active care • generally not helpful for improving diagnosis except with trainees • Findings limited by • methodological problems • choice of study design • insufficient appreciation of workflow component of CDSSs Clinical Decision Support Systems Medical Informatics

Summary on CDSSs • Intense interest in promise of CDSSs to improve health care quality • Evidence is equivocal but quite severely limited by methodological and other shortcomings • Top challenge currently is to apply current technology effectively to care processes • get physician buy-in, get an EMR, integrate CDSS with the EMR, incentivize organizations for buying and using CDSSs to improve quality… • Technical limitations on reasoning capability are not short-term barriers Clinical Decision Support Systems Medical Informatics

Evidence Adaptive CDSSs • CDSS recommendations should be evidence-based • should keep up-to-date with research findings • update mechanism should be semi-automatic • \ Health care computing infrastructure should be integrated • for clinical care and decision support • for clinical research Clinical Decision Support Systems Medical Informatics

Need For Informatics Infrastructure • “A nationwide effort is needed to build a technology-based information infrastructure that would lead to the elimination of most handwritten clinical data within the next 10 years, the committee said. ...Without a national pledge to create and fund such a technological framework, progress to enhance quality of care will be painfully slow.”(IOM, Crossing the Quality Chasm, Mar 2001) • IOM reports asks Congress to spend $1 billion on health informatics • How do needs of clinical research and care dovetail? Clinical Decision Support Systems Medical Informatics

Administrative Clinical Care Research Practice Management Systems ?? Electronic Medical Record Medical Business Data Model Clinical Care Data Model ?? Billing Clinical Standard Vocabulary Standard Communications Protocols (e.g., HL-7) Physical Networking Joint Infrastructure for Care and Research Clinical Decision Support Systems Medical Informatics

for clinical research New Ideas Design Study Clinical Care Utilize Results for basic research for patient care & policy Findings Protocol & Funding Conduct Study Activate Study Approval & Preparation Research and Care Together Clinical Decision Support Systems Medical Informatics

Findings Protocol & Funding The Clinical Trial Cycle (per NCI) New Ideas Utilize Results Design Trial • trial simulators • trial costing • protocol authoring • data analysis • reporting • data management • remote data entry • GCP compliance • IRB approval • CRF design Conduct Trial Activate Trial Approval & Preparation Clinical Decision Support Systems Medical Informatics

Findings Protocol & Funding Infrastructure for Clinical Trials Design Trial New Ideas Utilize Results • a few companies • trial simulators • trial costing • protocol authoring • data analysis • reporting • FDA electronic submission standards • data management • remote data entry • GCP compliance • IRB approval • CRF design Conduct Trial Activate Trial Approval & Preparation • a few companies • many companies Clinical Decision Support Systems Medical Informatics

Administrative Clinical Care Research Practice Management Systems Clinical Research Management Systems Electronic Medical Record Clinical Study Data Models Medical Business Data Model Clinical Care Data Model Billing Clinical Standard Vocabulary Standard Communications Protocols (e.g., HL-7) Physical Networking Joint Infrastructure for Care and Research Clinical Decision Support Systems Medical Informatics

Case: Clinical Research Informatics • You are planning on a study on infant jaundice... • What relevant studies have been completed on this topic? • What ongoing studies should you know about? • You’re interested in running your study over the web as much as possible. • what types of study activities can be done over the web? • how good is the technology for these activities? Clinical Decision Support Systems Medical Informatics

Relevant Trials: Completed • Medline • Cochrane Controlled Trials Register • ~327,700 records of controlled trials • manual logging of CCTs by hand searching journals • accessible from UCSF machine (IP address) only • can set up proxy access • metaRegister of Controlled Trials • 10,755 commercial and ongoing trials Clinical Decision Support Systems Medical Informatics

Relevant Trials: Ongoing • Non-profit/government • www.clinicaltrials.gov • 5700 trials, ~3000 open • NIH-supported and some commercial cancer and AIDS trials • cancertrials.nci.nih.gov • www.actis.org • AIDS Clinical Trials Information Service • www.trialscentral.org (from Cochrane people) • pointers to hundreds of clinical trial registries, by disease Clinical Decision Support Systems Medical Informatics

Relevant Trials: Ongoing • Commercial: mostly for patient recruitment • www.centerwatch.com • www.ClinicalTrialFinder.com • www.controlled-trials.com • www.clinicaltrials.com • etc., etc., etc. • How to get better web searching results • check out Web Search 101 • http://websearch.about.com/internet/websearch/library/weekly/aa011599.htm Clinical Decision Support Systems Medical Informatics

Case: Clinical Research Informatics • You are planning on a study on infant jaundice... • What relevant studies have been completed on this topic? • What ongoing studies should you know about? • You’re interested in running your study over the web as much as possible. • what types of study activities can be done over the web? • how good is the technology for these activities? Clinical Decision Support Systems Medical Informatics

Clinical Study Tasks • Project website • Subject recruitment • Eligibility determination • Protocol and forms distribution • Randomization • Data collection • adverse events tracking Clinical Decision Support Systems Medical Informatics

Industry is the Innovator • RCTs now a $3.6 billion business (C. Scott, 7/00) • in 1988, 95% of RCTs conducted by academics • now, over 80% conducted by industry • Ergo, much of the technology innovation in clinical research execution is going on in industry • Applied Clinical Trials software directory • http://www.pharmaportal.com/magazines/act/itsol/itsindex.cfm • What’s the global picture for research informatics technologies? Clinical Decision Support Systems Medical Informatics

Project Website • Project website • GISSI website has summaries of trial protocols, results, references • HERS main results revised tables from JAMA report • Requirements • web server computer • use a web hosting service (see http://www.cnet.com) • or have a web server program (e.g., Apache) • pages of material • produce these using Word (save as HTML file) • or use a web editor (FrontPage, Dreamweaver) Clinical Decision Support Systems Medical Informatics

Project Website (cont.) • Personnel • webmaster: handles the machine stuff • web designer: produces text & graphical content • database administrator/programmer: some databases (e.g., FilemakerPro, Access) can be exported to the web, but usually this involves moderate programming • Status: easily doable today Clinical Decision Support Systems Medical Informatics

JIFE Client/Server Model Kaiser Oakland Kaiser Santa Clara Kaiser San Diego Internet • The “jaundice.ucsf.edu” computer has • web server software. It “serves” web pages • in response to http commands such as • http://jaundice.ucsf.edu/project-home.html aol.com pacbell.net jaundice ucsf.edu itsa LAN dial-in to itsa.ucsf.edu via modem at home Clinical Decision Support Systems Medical Informatics

Eligibility Rule Match Eligible Patients EMR Automated Eligibility Determination • Study enrollment is big bottleneck • Eligible patients: patients whose characteristics match with eligibility criteria • For computerized matching, need to have computer-interpretable descriptions of • patient characteristics • the eligibility criteria Clinical Decision Support Systems Medical Informatics

Eligibility Example • Eligibility criterion: women who are 2 or fewer years post-menopause, as defined in NCI’s Common Data Elements set • Allowed values: Above categories not applicable AND Age < 50 Above categories not applicable AND Age >=50 Post (Prior bilateral ovariectomy, OR >12 mo since LMP with no prior hysterectomy and not currently receiving therapy with LH-RH analogs [eg. Zolades]) Post (Prior bilateral ovariectomy, OR >12 mo since LMP with no prior hysterectomy) Pre (<6 mo since LMP AND no prior bilateral ovariectomy, AND not on estrogen replacement) Clinical Decision Support Systems Medical Informatics

EMR Data Needed • Gender • Age • Time since LMP, whether • 6 or fewer months, or 12 or more months • Past surgical history • bilateral ovariectomy and/or hysterectomy • Therapy • LH-RH analogs, or • estrogen replacement Clinical Decision Support Systems Medical Informatics

Computer-Interpretable Eligibility Rule • NCI working on common model for representing eligibility rules • Logical rules • (Prior bilateral ovariectomy) OR (>12 mo since LMP ANDno prior hysterectomy) • first order logic is the best representation model for this • Temporal constraints • greater than 12 months since LMP... • representing time requires second-order logic • Can do simple cases with database rules and triggers Clinical Decision Support Systems Medical Informatics

Promising, but... • Richly detailed EMR not widely available or well integrated • Coding of eligibility rules is difficult • At present, can only expect computer to suggest potential subjects, then EMR can • prompt MD in real-time to refer patient to study, or • periodically batch notify MD of eligible patients, or • send letter of solicitation to patients • Similar problems bedevil automated identification of guideline eligibility Clinical Decision Support Systems Medical Informatics

Protocol and Forms Distribution • Allows for centralized forms management and storage through a project website • If expecting users to download, print, fill out and fax form back • need protocol and forms in electronic format (e.g.,Word or PDF) • scan it using a scanner ($100-$4000) • makes an image of the page (e.g., .gif or .jpeg) • optical character recognition (OCR) scanning • convert scanned text into an editable document (e.g., Word) • Status: easily doable today Clinical Decision Support Systems Medical Informatics

JIFE Forms Download Kaiser Oakland Kaiser Santa Clara Kaiser San Diego Internet • “jaundice.ucsf.edu” “serves” forms such as • http://jaundice.ucsf.edu/case-form.pdf for • printing out aol.com pacbell.net jaundice ucsf.edu itsa LAN dial-in to itsa.ucsf.edu via modem at home Clinical Decision Support Systems Medical Informatics

Protocol and Forms Distribution • If expecting users to enter data online over the web • need someone to design the forms and build them to be served over the web • e.g., using Access Visual Basic • need security mechanisms (e.g., user login) • need data validation checks built into forms entry • data forms must send data to a database • Status: doable with some programming Clinical Decision Support Systems Medical Informatics

Infant Jaundice Online Forms Kaiser Oakland Kaiser Santa Clara Kaiser San Diego Internet • “jaundice.ucsf.edu” “serves” online entry forms • such as http://jaundice.ucsf.edu/case-form.asp. • Users enter data, which get checked at the client • side, and data is sent back to “jaundice.ucsf.edu.” aol.com pacbell.net jaundice ucsf.edu itsa LAN dial-in to itsa.ucsf.edu via modem at home Clinical Decision Support Systems Medical Informatics

patient info randomization results Enroller Project Central Web-based Randomization • Requirements • a web-based data collection form to collect patient information • programs to verify eligibility and randomize patient • program to generate a response to the enroller • security, privacy, and backup provisions • Some commercial systems do this for you • Status: doable with some programming Clinical Decision Support Systems Medical Informatics

Clinical Decision Support Systems