Harmonization of Patient Assessment Instruments for Semantic Interoperability in US Healthcare

1. Inter Ontology 2008 Harmonization of Patient Assessment Instruments in the USA:a Compelling Case for Realism-Based OntologyTokyo, Japan, February 27, 2008 Werner CEUSTERS Center of Excellence in Bioinformatics and Life Sciences Ontology Research Group University at Buffalo, NY, USA

2. Presentation Summary (1) • In the US, several patient assessment instruments are used for a variety of purposes: • Outcome analysis • Pay for performance • Determination of next level of care • The sort of instrument used depends on the care facility: • Skilled nursing facilities (SNF) • Internal Rehabilitation Facilities (IRF) • Home Health Agency (HH) • …

3. Presentation Summary (2) • All instruments are developed • to describe (more or less) the same sort of phenomena • but in such a way that the data (descriptions) obtained are not comparable, and cannot semantically integrated • Some ‘solutions’ are underway, but they suffer from the same problems. • We propose a method based on realist ontology that is capable to provide semantic interoperability of the various instruments.

4. Part 1Healthcare in the US:organizationmonitoring cost & quality

5. The cost of health care in the US U.S. Department of Health and Human Services - Centers for Disease Control and Prevention - National Center for Health Statistics. Health, United States, 2007.

6. Health spending as a percent of GDP Medicare Payment Advisory Commission. A Data Book: Healthcare Spending and the Medicare Program. June 2007.

7. Major care organization types in the US Outpatient Rehabilitation Acute Care Hospital Inpatient Rehabilitation Facility Long Term Care Hospital Skilled Nursing Facility Home Health Agency

8. Medicare spending for Post-Acute Care (1999-2005) total SNF HHA IRF LTCH Medicare Payment Advisory Commission. A Data Book: Healthcare Spending and the Medicare Program. June 2007. http://www.medpac.gov/documents/Jun07DataBook_Entire_report.pdf. Last accessed: December 21, 2007.

9. Growth in post-acute care providers • Differences in availability do not alone explain the differences in costs. Medicare Payment Advisory Commission. A Data Book: Healthcare Spending and the Medicare Program. June 2007.

10. An attempt to study and explain the differences U.S. Department of Health and Human Services - Assistant Secretary for Planning and Evaluation - Office of Disability, Aging and Long-Term Care Policy. A Study Of Stroke Post-Acute Care Costs And Outcomes: Final Report. December 2006

11. Some findings (1) • There exists a wealth of patterns in Post Acute Care

12. Typical stroke patient trajectories Outpatient Rehabilitation Acute Care Hospital Inpatient Rehabilitation Facility Long Term Care Hospital Skilled Nursing Facility Home Health Agency

13. Re-admissions Outpatient Rehabilitation Acute Care Hospital Inpatient Rehabilitation Facility Long Term Care Hospital Skilled Nursing Facility Home Health Agency

14. Patterns of Post-Acute Care for Stroke Victims Following Discharge from Acute Hospital and Admission to Nursing Home U.S. Department of Health and Human Services - Assistant Secretary for Planning and Evaluation - Office of Disability, Aging and Long-Term Care Policy. A Study Of Stroke Post-Acute Care Costs And Outcomes: Final Report. December 2006

15. Patterns of Post-Acute Care for Stroke Victims Following Discharge from Acute Hospital and Admission to Home Health U.S. Department of Health and Human Services - Assistant Secretary for Planning and Evaluation - Office of Disability, Aging and Long-Term Care Policy. A Study Of Stroke Post-Acute Care Costs And Outcomes: Final Report. December 2006

16. Some findings (1) • A wealth of patterns in PAC • Nearly 170 different PAC patterns were identified in 90 days. • Sixty percent of IRF admissions used a second PAC provider and 30 percent used three or more in 90 days. • Influence of patient characteristics, e.g.: • Patients admitted to HH from IRF were similar to patients admitted to OP from IRF with respect to pre-morbid status, cognition, and most functional measures following their stroke, but they had lower incomes.

17. Some findings (2) • Remarkable differences in outcomes: • Relative to patients discharged to HH following IRF, outcomes for patients admitted to outpatient care following IRF were comparable with respect to 90-day residence and significantly better in two dimensions of functional recovery, even after risk adjustment. • Resource utilization and costs • Relative to IRF→OP costs, total cost per PAC episode was $2,200 higher and total cost per 90 days was $5,200 higher for IRF→HH patients. Despite the lower costs, IRF→OP patients received about 40 therapy visits in contrast to 21 therapy visits for IRF→HH patients in PAC episodes with comparable duration. However, average PAC beneficiary costs were $400 higher for the IRF→OP group.

18. Major problem in this context: • A uniform set of core measures is required to assess PAC outcomes for patients admitted to single or multiple PAC settings.

19. Current Tools for Measuring Patients Across the Continuum in Medicare • Acute Hospitals  no standard tool • Long-Term Care Hospitals  no standard tool • Inpatient Rehabilitation Facilities  IRF-PAI • Skilled Nursing Facilities  MDS 2.0 (MDS 3.0) • Home Health Agencies  OASIS • Swing-bed hospitals  SB-MDS • Patient generated  PROMIS (under development)

20. MDS 2.0 – Fragment from Section C

21. Commonalities amongst instruments • Use Classical Test Theory • Each instrument has a fixed set of questions presented to the individual • All items are asked irrespective of relevance • Individual’s score is dependent on items of the particular assessment • Assessments with challenging items -individuals get low score and assessments with easy items get high scores • Scores are incomparable across instruments and settings

22. Common Domains in Current Assessment Tools • Administrative Information • Social Support Information • Medical Diagnosis/Conditions • Functional Limitations • Physical • Cognitive Kramer A and Holthaus D (eds.), Uniform Patient Assessment for Post-Acute Care; Final Report. Jan 25, 2006. http://www.bu.edu/hdr/documents/QualityPACFullReport.pdf. Last accessed: Dec 18, 2007.

23. But unfortunately: many differences exist … • Individual items to measure each concept • Scales used to measure each item • Look-back or assessment periods • Unidimensionality of individual items Kramer A and Holthaus D (eds.), Uniform Patient Assessment for Post-Acute Care; Final Report. Jan 25, 2006. http://www.bu.edu/hdr/documents/QualityPACFullReport.pdf. Last accessed: Dec 18, 2007.

24. Differences amongst instruments (1) Kramer A and Holthaus D (eds.), Uniform Patient Assessment for Post-Acute Care; Final Report. Jan 25, 2006. http://www.bu.edu/hdr/documents/QualityPACFullReport.pdf. Last accessed: Dec 18, 2007.

25. Differences amongst instruments (2) Kramer A and Holthaus D (eds.), Uniform Patient Assessment for Post-Acute Care; Final Report. Jan 25, 2006. http://www.bu.edu/hdr/documents/QualityPACFullReport.pdf. Last accessed: Dec 18, 2007.

26. Tools No. of Functional Items Scale Levels Assessment Periods IRFPAI 18 7 Past 3 days MDS 3.0 12 8 Past 5 days OASIS 8 varies Assessment day Functional Item Comparisons

27. IRF-PAI MDS OASIS 7= Complete independence 0= Independent 0= bathe independent tub/shower 6=Modified (device) 1= Supervision 1= with devices, independent 5=Supervision 2= Limited Asst. (guided maneuvering) 2= with person (reminders, access, reach difficult areas 4=Minimal Assistance 25% 3= Extensive Asst (3+ times/week) 3= participates but req. other person 3= Moderate Assistance 50% 4= Total Dependence 4= unable, bathes in bed/chair 2=Maximal Asst. 25% 8= Activity NA 5= totally bathed by other 1= Total Asst. 0= Activity NA Unknown Functional Scales

28. Intermediate conclusion • After two decades of measurement development , multiple approaches, millions of dollars of funding, expenditure of much intellectual capital and new technologies to develop a uniform measure, we do not have standardized measurement and reporting necessary to evaluate quality of post-acute care or to make informed policy decisions. Duncan P. & Velozo C. Measurement and Methodology.

29. Part 2Current ‘solutions’

30. Chronic Condition Data Warehouse • Alignment of CMS data by means of unique beneficiary key involving 5% sample of Medicare beneficiaries since Jan 1, 1999; • 21 chronic condition subpopulations: • AMI, cataract, diabetes, stroke, …; • Contains the MDS, IRF-PAI, …, data for patients having been cared for in these institutions • Available (for fee) from ResDAC Iowa Foundation for Medical Care

31. Deficit Reduction Act of 2005 • Congressional mandate to establish a PAC Payment Reform Demonstration by January 2008 to examine cost and outcomes across different post acute sites • Single comprehensive assessment at acute hospital discharge • Standardized assessment in all PAC settings to measure health and functional status and other treatment factors • Collection of information on resources/patient • Report to be submitted to Congress in 2011.

32. CMS ‘Post Acute Demonstration’ • Three components: • Development of a Patient Assessment Instrument • Development of a web-based, electronic reporting system • Implementation of a Payment Reform Demonstration

33. Development Strategy • Identify critical areas/domains for measuring case-mix acuity, resource use, or outcomes • Review existing legacy tools (MDS, IRFPAI, OASIS), other leading measurement tools (PROMIS, COCOA-B, VA) and existing tools in LTCHs and acute hospitals • Propose core data set that can be used to standardize information at hospital discharge and across all PAC settings

34. CARE has been developed and testing begins, but … • its design does not appear to yield an indication of a patient’s level of medical necessity for post-acute care; • a significant amount of patient information needed for a CARE assessment would have to be accessed from patients’ paper medical records and other varied hospital systems; • some of the proposed measurement scales are different than those currently used by post-acute providers; • questions such as “Would you be surprised if the patient was readmitted to an acute care hospital in the next 6 months?” and “Would you be surprised if the patient were to die in the next 12 months?” would force discharge planners to rely in part on subjective judgment.

35. Overall result • Existing and future data collections are hard to compare or integrate. • If CARE comes in use (2012 ?), there will be one instrument “across the lifespan”, but it is not backward compatible with existing datasets. • Success of CARE will depend largely on the ability to exchange information with EHR systems.

36. Part 3Towards a solution based onRealist Ontology

37. Hypotheses • Available tool and systems fail due to: • Incompatibility because developed with insufficient (if any at all) ontological insight; • Inadequacy of mappings based on terminology; • Confusion between information models and models of the corresponding reality. • An adequate ontology can get more out existing data and help in building better future data sets.

38. For an ontology to be ‘adequate’ … … in the health domain, it should be built around a core of representational units which describe phenomena as they exist on the side of the patient and of the patient’s environment. Smith B, Ceusters W. Ontology as the Core Discipline of Biomedical Informatics: Legacies of the Past and Recommendations for the Future Direction of Research. In: Gordana Dodig Crnkovic and Susan Stuart (eds.) Computing, Philosophy, And Cognitive Science - The Nexus and the Liminal, Cambridge: Cambridge Scholars Press, 2007;:104-122.

39. Questions to be addressed • To what extent do the mentioned outcome assessment tools and related data sets overlap? • Given that these distinct artifacts refer to the entities on the side of the patient primarily implicitly, indirectly and in different ways, how can this overlap be made explicit and quantifiable? • How can this overlap be made understandable to software agents in such a way that they can use the corresponding representations of the entities as a basis for meaningful computations, including making comparisons and deriving new information? • How can we demonstrate that our approach is successful? • How can we ensure that our work is extensible and the methodology applied easy transferable to other measurement tools and data sets?

40. Proposal 1. build a patient-centric ontology covering the entities in reality that must exist as referents for those terms (included constituent parts of compound terms) that are shared by at least two of the assessment systems and related datasets. 2. define in the terms of this ontology the dictionaries and data-elements of the listed systems, as well as relevant portions of the ICF, of the Common Data Elements of the National Institute of Neurological Disorders and Stroke (NINDS) and of SNOMED CT; 3. validate the ontology through several independent methods, including the degree to which it serves linkage of the available datasets, in the context of patients who suffered from stroke; 4. provide documentation on how to use the ontology in relationship with Electronic Health Records and other Health Information Technology systems.

41. Task Aim Type T1 Aim2 Terminological alignment T2 Aim1 Ontology development T3 Aim1 Data sets - ontology bridging T4 Aim2 Vocabulary development T5 Aim1 Publication in standard formats T6 Aim3 Validation through case report annotation T7 Aim3 Statistical validation T8 Aim4 Documentation T9 Aim4 Dissemination & awareness Methodology

42. IRF-PAI Items MDS Items Eating Eating Bed Mobility Grooming Personal Hygiene Bathing Bathing Dressing-Upper Body Dressing-Lower Body Dressing Toileting Toilet Use Bladder Management Bladder Continence Bowel Management Bowel Continence Bed, Chair, Wheelchair (Transfer) Toilet (Transfer) Tub, Shower (Transfer) Transfer Terminological alignment

43. Alignment with other systems too • International Classification of Functioning, Disability and Health (ICF) • From World Health Organization • SNOMED CT • ‘Common Data Elements’ • National Institute of Neurological Disorders and Stroke (NINDS)

44. Alignment with SNOMED and ICF : feasibility Consolidated Health Informatics. Standards Adoption Recommendation: Disability. Sep 22, 2005.

45. Body function&structure (Impairment) Activities (Limitation) Participation (Restriction) Environmental Factors Personal Factors Interaction of entities as perceived in ICF (2001) Health Condition (disorder/disease)

46. Ontology building basics: three levels of reality • The world exists ‘as it is’ prior to a cognitive agent’s perception thereof; • Cognitive agents build up ‘in their minds’ cognitive representations of the world; • To make these representations publicly accessible in some enduring fashion, they create representational artifacts that are fixed in some medium. Smith B, Kusnierczyk W, Schober D, Ceusters W. Towards a Reference Terminology for Ontology Research and Development in the Biomedical Domain. Proceedings of KR-MED 2006, November 8, 2006, Baltimore MD, USA

47. Three levels of reality • The world exists ‘as it is’ prior to a cognitive agent’s perception thereof; Smith B, Kusnierczyk W, Schober D, Ceusters W. Towards a Reference Terminology for Ontology Research and Development in the Biomedical Domain. Proceedings of KR-MED 2006, November 8, 2006, Baltimore MD, USA

48. R Reality exist before any observation

49. R And also most structures in reality are there in advance. Reality exist before any observation

50. Three levels of reality • The world exists ‘as it is’ prior to a cognitive agent’s perception thereof; • Cognitive agents build up ‘in their minds’ cognitive representations of the world; Smith B, Kusnierczyk W, Schober D, Ceusters W. Towards a Reference Terminology for Ontology Research and Development in the Biomedical Domain. Proceedings of KR-MED 2006, November 8, 2006, Baltimore MD, USA