Validation of Acute Stroke in Medicare Data against WHI

1. Validation of Acute Stroke in Medicare Data against WHI KamakshiLakshminarayan, MD, PhDpresented by Dale Burwen, MD, MPH WHI Investigators Meeting May 3-4, 2012 Preliminary results; do not distribute

2. Writing Group • Kamakshi Lakshminarayan • Dale Burwen • Joe Larson • Beth Virnig • Wolfgang Winkelmayer • Norrina Allen • Monica Safford • Marian Limacher

3. Background • Starting in 2010, stroke outcomes in WHI will be adjudicated in only a quarter of participants • Medicare data provide potential for expanding outcome ascertainment • Little is known about validity of Medicare claims for ascertaining neurologist adjudicated strokes

4. Objective Compare agreement between various algorithms to detect stroke hospitalizations in Medicare claims data and neurologist adjudicated stroke outcomes in WHI

5. Methods – Study Population Inclusions Exclusions Managed care at the time of their WHI enrollment Participants are censored as they enter into managed care Those with WHI adjudicated stroke outcomes prior to CMS eligibility are excluded Participants are censored 7 days after WHI stroke • Observational study women • With Medicare Parts A&B, Fee-For-Service at the time of WHI enrollment 1993-1998 N=27,739 • Those who age into Medicare Parts A&B, Fee-For-Service after enrollment until 2007 N=21138 • Total N = 48,877

6. Methods • Randomly split into training & test data sets • Training set N = 24,432 • Test set N = 24,495 • Analysis to date confined to training set

7. Stroke in WHI • Rapid onset of persistent neurologic deficit attributed to obstruction or rupture of brain arterial system. • Deficit is not known to be secondary to brain trauma, tumor, infection, or other cause. • Deficit must last > 24 hours unless death supervenes or there is a lesion compatible with an acute stroke on CT or MRI.

8. Defining stroke in Medicare Used 1993-2007 hospital data (MedPAR file) Results pertain to the 1st definition

9. Events Included in Analysis Universe of events: • WHI confirmed strokes after neurologist adjudication • All hospitalization claims from Medicare data (stroke and non-stroke) • Goal is to classify each claim into stroke vs. not • Definition of matched events: • WHI stroke & CMS stroke +/- 7 days • Sensitivity analysis with wider intervals (14 days)

10. Match Results (Stroke in any diagnosis position) Kappa 0.66

11. Reasons for DisagreementWHI Yes, CMS No(N=105) • Hospital claim found +/- 7days; but claim did not have diagnosis codes meeting stroke definition • No hospital claim found +/- 7 days; outpatient stroke according to WHI • No hospital claim found +/- 7 days 54% (n=57) 5% (n=5) 41% (n=43)

12. Reasons for DisagreementWHI No, CMS Yes(N=374) • Self-report of stroke or Transient Ischemic Attack (TIA), with hospitalization +/- 7 days • Adjudicated as TIA or carotid disease • Adjudicated as no outcome • Not adjudicated due to administrative reasons • Self-report of other hospitalization • No report of hospitalization • (case ascertainment of WHI) 24% (n=89) 7% (n=28) 13% (n=50) 3% (n=11) 21% (n=78) 55% (n=207)

13. Original vs. Modified Analysis 13

14. Validation Performance Original Analysis Modified Analysis Sensitivity: 82.7% Specificity: 99.7% PPV: 75.4% Kappa: 0.79 • Sensitivity: 82.0% • Specificity: 99.3% • PPV: 56.1% • Kappa: 0.66

15. Validation PerformancePrimary Position Original Analysis Modified Analysis Sensitivity: 74.6% Specificity: 99.8% PPV: 82.3% kappa: 0.78 • Sensitivity: 73.9% • Specificity: 99.6% • PPV: 63.9% • kappa: 0.68

16. Discussion • Initial WHI vs. Medicare agreement was moderate • Key reason for a CMS event without WHI match was lack of WHI report of hospitalization • Possible reasons: Inadequate recall; disability/death and lack of proxy report • Limiting analysis to CMS events that could be evaluated with WHI medical records increased PPV to 75% • Primary position diagnostic codes PPV = 82% • False positives due to TIA were in a minority; mainly other diagnosis

17. Discussion (cont.) • A key reason for WHI stroke without matching CMS stroke was that WHI picked up a lot of strokes coded with a variety of other diagnosis codes. • However, there was no predominant code to suggest how to modify our algorithm • Another important reason was lack of CMS hospital claim within the selected time interval (+/- 7 days) • Wider time interval picked up a minority

18. Next Steps • Further exploration of reasons for disagreement • Test additional algorithms • Consider incorporation of Medicare procedures/diagnoses for rehabilitation • To increase specificity for stroke vs. TIA, and current vs. historical stroke

19. Thank you! Questions?