A Risk Score for In-Hospital Ischemic Stroke Mortality Derived and Validated within the Get With The Guidelines ® -St

1. A Risk Score for In-Hospital Ischemic Stroke Mortality Derived and Validated within the Get With The Guidelines®-Stroke Program Eric E Smith, Nandavar Shobha, David Dai, DaiWai M Olson, Mathew J Reeves, Jeffrey L Saver, Adrian F Hernandez, Eric D Peterson, Gregg C Fonarow, Lee H Schwamm

2. Disclosures The Get With The Guidelines®–Stroke (GWTG-Stroke) program is provided by the American Heart Association/American Stroke Association. The GWTG-Stroke program is currently supported in part by a charitable contribution from Bristol-Myers Squib/Sanofi Pharmaceutical Partnership and the American Heart Association Pharmaceutical Roundtable. GWTG-Stroke has been funded in the past through support from Boeringher-Ingelheim and Merck. The individual author disclosures are listed in the manuscript

3. Background • Acute ischemic stroke results in substantial morbidity and mortality • In-hospital case fatality rate is approximately 5% • Determining an individual patient’s risk of mortality at admission: • Could aid clinical care by providing valuable prognostic information • Could identify those at high risk for poor outcomes who may require more intensive resources.

4. Background • Increased interest by 3rd party payers and regulatory agencies in tracking stroke mortality as one of the measure of stroke quality of care, so adjustment for baseline risk of mortality will be necessary to avoid penalizing hospitals that admit less healthy patients. • Therefore, there is an increasingly important need to develop well validated models that are useful in predicting patient risk of mortality and can be efficiently utilized in actual practice.

5. Introduction • There are few validated models for prediction of in-hospital mortality following ischemic stroke. • Prior prediction models have not been incorporated into clinical practice. • There remains a need for an accurate and practical clinical risk tool to predict ischemic stroke mortality that overcomes limitations and can be readily incorporated into clinical practice without need for hand calculation.

6. Objective • To develop a practical user-friendly web-enabled bedside tool for risk stratification at the time of presentation for patients hospitalized with acute ischemic stroke • To derive and validate prediction models for a patient’s risk of in-hospital ischemic stroke mortality using data from the Get With The Guidelines-Stroke Program • To test the added value of a measure of stroke severity by using the patients who were documented in the NIHSS*, the most commonly used standardized stroke scale. *NIHSS: National Institutes of Health stroke scale score

7. Methods • Hospitals participating in GWTG-Stroke who utilize the web-based patient management tool for data collection. • Outcome Sciences, Inc. served as the data collection and coordination center • The Duke Clinical Research Institute (DCRI) served as the data analysis center

8. Methods Study Population and Study Measurements • Hospitals were instructed to record data from consecutive stroke and TIA admissions. • Case ascertainment was through clinical identification during hospitalization, retrospective identification by ICD-9 codes or both. • Eligibility of each case was confirmed at chart review • Trained hospital personnel abstracted data using an internet-based system that performs checks to ensure the reported data is complete and internally consistent. • Data quality was monitored for completeness and accuracy.

9. Methods Study Population and Study Measurements • Hospital characteristics were based on American Hospital Association data. • Presentation during daytime regular hours was defined at 7am to 5pm, Monday through Friday. • Past medical history was defined based on pre-existing conditions, excluding newly diagnosed conditions during the hospital stay.

10. Methods Study Population and Study Measurements • Between Oct. 1, 2001 and Dec. 30, 2007 • 1042 hospitals contributed data • 320,635 ischemic stroke discharges • Only patients with ischemic stroke were included (excluded hemorrhagic stroke or TIA) • After exclusions: • 274,988 patients from 1,036 hospitals were analyzed

11. Methods Study Population and Study Measurements • Exclusions: • Patients transferred out to another acute care hospital or transferred in. • Patients with missing data on discharge destination • Small number of patients who did not present via the Emergency Department because of direct floor admission or because of new acute stroke occurring during hospitalization. • Few patients with missing gender information

12. Methods Statistical Analysis • The sample was randomly divided into a derivation (60%) and validation (40%) sample. • Logistic regression was used to determine the independent predictors of mortality and to assign point scores for a prediction model. • We also separately derived and validated a model in the 109,187 (39.7%) with NIH stroke scale score (NIHSS) recorded. • Model discrimination was quantified by calculating the c statistic from the validation sample. • The c statistic is equivalent to the probability that the predicted risk of death is higher for patients that died than for patients that survived. • A c stat of 1.0 indicates perfect prediction, while a c stat of 0.5 indicates no better than random prediction. • In-hospital mortality was 5.5% overall and 5.2% in the subset where NIHSS was recorded.

13. Results • Study Population: 274,988 ischemic stroke patients • Admissions submitted: 1,036 hospitals • Mean Age: 71.6 years • Women: 53.4% • Treated in academic teaching hospitals: 60.6% • Median Hospital bed size: 372

14. Results • In-Hospital Death occurred: 5.51% • 15,152 out of 274,988 patients • Many differences between patients who died and who survived • Patients who died were more likely to: • Be older • Have arrived by ambulance • Have a history of atrial fibrillation or coronary artery disease • Weekend or Night Admission was also associated with higher mortality

15. Characteristics of Ischemic Stroke Patients who Died or Survived to Hospital Discharge

16. Characteristics of Ischemic Stroke Patients who Died or Survived to Hospital Discharge

17. Results • The derivation sample and the validation sample were well matched with respect to patient characteristics and overall mortality, with no significant differences between the two. • The multivariable-adjusted independent predictors of increased risk of mortality were: • Increasing age (for each year greater than 60) • Atrial fibrillation • Coronary artery disease • Diabetes mellitus • Peripheral vascular disease • Independent predictors of lower risk of mortality were: • history of previous stroke or TIA • Known carotid stenosis • Hypertension • Dyslipidemia • Current smoking • Presentation during weekday regular hours

18. Risk Score • Point scores were derived that could be used to predict a patient’s risk of dying in hospital • The probability of in-hospital mortality can be estimated for an individual patient by summing points assigned to the value of each predictor to create a total point score ranging from 0 to 204.

19. The Predicted in-hospital mortality plotted as a continuous function of the developed risk score

20. Risk Score • This prediction model was validated in the remaining 40% of the population. • The risk score demonstrated good discrimination in the validation sample • Similar good discrimination was seen in the pre-specified subgroups in the validation sample. • A graph of observed vs. predicted mortality, in 6 groups according to pre-specified categories of mortality risk, showed an excellent correlation between observed and predicted mortality in the validation sample, grouped into 10 deciles of predicted risk, also showed excellent calibration.

21. Model Without Including NIHSS

22. Risk Score that Includes NIHSS • The median hospital-level percent of patients with NIHSS recorded was 22.2%. • Very large sample size so many differences between the groups that reached conventional levels of statistical significance but that were actually small in both relative and absolute terms. • Larger differences were seen for only a few characteristics: NIHSS was more likely to be documented in patients who were: • younger, male, arrived by ambulance, arrived during daylight hours • Overall mortality was slightly lower in the group with NIHSS documented vs. the group without NIHSS documented.

23. Characteristics of ischemic stroke patients with or without NIHSS recorded

24. Characteristics of ischemic stroke patients with or without NIHSS recorded

25. Characteristics of ischemic stroke patients with or without NIHSS recorded

26. Risk Score • 60% of the sample of patients with NIHSS recorded was used for derivation and 40% were used for validation. • The 2 samples were well matched with respect to patient characteristics and overall mortality, with no significant differences. • NIHSS was strongly associated with mortality; median NIHSS was 18 in those who died compared to 4 in those who survived. • Higher NIHSS was strongly associated with increased mortality after controlling for other predictors.

27. Point Scores for the prediction modeling including NIHSS

28. Risk Score • Model discrimination and calibration was again excellent across a wide range of pre-specified predicted risk categories in the derivation and validation samples. • A plot of observed vs. predicted mortality in the validation sample, grouped into 10 deciles of predicted risk, again showed excellent calibration.

29. Model Including NIHSS

30. Risk Score • The validation sample c statistic for the model including NIHSS was greater than the c statistic for the model derived without NIHSS. • The c statistic for a model including NIHSS alone, without any predictors, was also very high.

31. Model Discrimination – Importance of NIHSS Information

32. Impact of NIHSS on Predictions For model with vs. without NIHSS, Integrated Discrimination Index (IDI) = 9.4%. IDI = (EY1-EY0) – (EX1-EX0) • Where • EY1 and EY0 are the mean predicted probabilities of death from persons who died (EY1) or survived (EY0) in model Y (with NIHSS) • EX1 and EX0 are the mean predicted probabilities of death from persons who died (EX1) or survived (EX0) in model X (with NIHSS)

33. Limitations • Voluntary participation—may not be representative of all patients/hospitals. • Study data were collected based on the medical record and depend on the accuracy and completeness of clinical documentation and chart abstraction. • NIHSS data were not missing at random therefore the relationship between predictors and outcome observed in the subset with NIHSS documented may not be generalizable to all ischemic stroke patients. • Could not control for all potential predictors of ischemic stroke mortality but nonetheless were able to predict in-hospital mortality with similar discrimination as previously published models. • No post-discharge information.

34. Conclusions • The GWTG-Stroke risk model provides clinicians with a well validated, practical bedside tool for mortality risk stratification. • The NIHSS provides substantial incremental information regarding patient’s short term mortality risk and is the strongest predictor of mortality.

35. Conclusions • A validated and accurate risk stratification tool should also be of value to health services researchers who are interested in comparing in-hospital mortality rates across different hospitals and systems; • The tool has been incorporated into routine clinical practice by implementing real-time reporting of individual predicted mortality in GWTG-Stroke participating hospitals via the web-based Patient Management Tool