The HEAL NY Evaluation Process: HITEC

1. The HEAL NY Evaluation Process: HITEC Rainu Kaushal, MD MPH Chief, Division of Quality and Clinical Informatics Weill Cornell Medical College Executive Director, HITEC Director of Pediatric Quality and Safety Komansky Center for Children’s Health at NYPH

2. NYS and Healthcare • Health IT is potentially transformative • Value still largely unproven especially for community based settings utilizing vendor based products • NYS is a national leader in investment in health IT • Critical opportunity to study and learn from this investment • Need for common process of evaluation across projects to optimize learning

3. Why Evaluate? • Demonstrate economic and quality value of HEAL investments • Demonstrate effects on providers and consumers • Iteratively refine health IT • Understand what is working well and why • Understand what can be improved and how • Disseminate lessons learned and successes broadly

4. What is HITEC? • The Health Information Technology Evaluation Collaborative consists of 5 universities • Weill Cornell • Columbia • SUNY Albany • SUNY Buffalo • University of Rochester • Established with leadership and support from the New York State Department of Health (NYS DOH)

5. What is HITEC? • Steering committee members • Rainu Kaushal, Executive Director, Weill Cornell • Lisa Kern, Deputy Director, Weill Cornell • Russell Bessette, SUNY Buffalo • Rachel Block, NYeC • George Hripcsak, Columbia DBMI • Jean Moore, SUNY Albany • Sandy Schneider, University of Rochester • Bill Schroth, NYS DOH (non voting member)

6. What is HITEC studying? • How health IT affects: • Quality • Safety • Economics • Providers • Consumers • Public Health • Health IT & HIE adoption • Policy

7. HITEC HEAL 5 Evaluation Overview

8. What will the evaluation process be? • Grantee with HITEC and NYS DOH will identify the research question for their organization • HITEC will design evaluation and data collection tools • Grantee will perform data collection and provide data in standardized format to HITEC • HITEC will perform data analysis and disseminate results

9. Evaluation Activities • HITEC, individual grantees, and the NYS DOH OHITT will identify the single most informative evaluation project for the grantee • The research question must fall under evaluation categories (economic, quality, safety, provider, consumer, public health) appropriate for the HEAL 5 application category and use case chosen by the grantee

10. Use Cases & Potential Evaluations

11. Use Cases & Potential Evaluations

12. Evaluation Timeline

13. Evaluation Timeline (cont)

14. HITEC HEAL 5 Evaluation HITEC’s Structure, Designed to Facilitate Evaluations

15. Health Information Technology Transformation Office at NYS DOH HITEC Steering Committee • Strategic Partner Initiatives • NYeC • Incentives & Sustainability • Consumer Advocacy Council HEAL 5 Grantees HITEC Project Management HITEC Evaluation Workgroups HITEC Stakeholder Committee HITEC Dissemination Committee Organizational Structure

16. Evaluation Workgroups • Six HITEC Evaluation Workgroups – one for each evaluation activity • Timelines and deliverables will be proposed by the workgroups and by project management with subsequent approval by the HITEC steering committee.

17. Workgroups • The HIT Adoption Measurement Workgroup • Measures rates of adoption of health IT across NYS through a survey and places these findings into a policy context. • The Economic Value Workgroup • Quantifies the direct and indirect economic effects of health IT, including the effects of health IT on efficiency. • The Quality and Safety Workgroup • Aims to understand the effects of health IT adoption on clinical quality and safety

18. Workgroups (cont.) 4. The Provider Workgroup • Aims to understand providers’ interactions with health IT including effects on workflow, efficiency, satisfaction, and unintended consequences. • The Consumer Workgroup • Aims to design and implement survey-based studies to assess the effects of health IT on consumers • The Public Health Workgroup • Aims to describe the effects of health IT on public health measures

19. Project Management • The HITEC Project Manager with Research Coordinators will manage relationships between HITEC workgroups and HEAL 5 grantees • Will have the responsibility of communicating individualized bi-directional timeline and deliverable schedules for each grantee’s evaluations. • Will coordinate meetings with HEAL 5 Grantees to explain and receive feedback on methodologies and tools.

20. Stakeholder Committee • HITEC and NYeC will collaborate to establish this committee to maximize key stakeholder support for the evaluation process • It will include the following representatives: • HEAL 5 Grantee representatives • NYS Office of Health Information Technology Transformation • NYeC • HSP representatives • Representatives from key state program offices including Medicaid and public health • Health plan and vendor representatives

21. Dissemination Committee • Three primary goals: • To provide timely, interim feedback directly to HEAL 5 Grantees. • To provide timely, interim feedback to the NYS DOH. • To publish in academic journals and present in national meetings.

22. HITEC HEAL 5 Evaluation Key Grantee Responsibilities and Next Steps

23. Key grantee responsibilities • Help choose a research question • Designate appropriate staff and time to support the evaluation process • Senior leadership • Programmers • Data collectors • Perform data collection as outlined in the evaluation plan • Provide timely data in standardized format to HITEC

24. Grantees Should Prepare to Discuss with HITEC • Who will be affected by their HEAL 5 project • Which providers, patients • How many providers, patients • What exactly their project will entail • When they expect implementation to “go live”

25. Grantees Should Prepare to Discuss with HITEC • Which use case they would like their evaluation to focus on • Which outcome they would like their evaluation to focus on (economic, quality, safety, provider, consumer, public health)

26. The Research Question • HITEC will work with the grantee and NYS DOH to design a research question that: • Links the HEAL 5 project with a particular outcome for a particular population of patients or providers • Meets the FINER criteria…

27. The FINER Criteria for Research Questions F Feasible I Interesting N Novel E Ethical R Relevant Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB. Designing Clinical Research, 3rd ed. Lippincott Williams & Wilkins: Philadelphia, 2007.

28. Next Steps • HITEC will contact each of you to start this process this summer • Grantees should prepare for this by collecting answers to the questions posed (who, what, when and which) and by starting to think about appropriate research questions

29. Contact Information • Rainu Kaushal, MD, MPH rak2007@med.cornell.edu • Lisa Kern, MD, MPH lmk2003@med.cornell.edu • Lauren Mercincavage lmm2005@med.cornell.edu