Understanding and Using Data

1. Understanding and Using Data Model Planning Council Training Session [Activities Provided as a Separate Document] This model training session is based on training provided to the Phoenix Ryan White HIV/AIDS Program (RWHAP) Part A Planning Council in 2006 by Emily Gantz McKay and Harold J. Phillips. Some of the materials come from the John Snow, Inc. (JSI) “Increasing Consumer Involvement: Ryan White Title I Planning Council Training” provided in 2004 (Both Emily and Harold were senior trainers at these regional sessions). The materials have been updated many times, most recently by EGM Consulting, LLC, for the PCS Compendium, funded through Task Order TA003111, MSCG/the Ryan White Technical Assistance Contract. From: Compendium of Materials for Planning Council Support Staff. EGM Consulting, LLC. 2018.
Available at: www.targetHIV.org/planning-chatt/pcs-compendium

2. Welcome & Introductions • Introductions • Logistics • Objectives • Agenda

3. Training Focus • Enable planning council members to assess their progress using data for decision making • Increase existing skills in using data • Enhance members’ abilities to use data in shaping the system of HIV care through priority setting, the use of directives, and other actions to improve services overall and for particular PLWH subpopulations

4. Session Objectives At the end of the session, participants will be able to: • Assess the Planning Council’s progress and status in data-based decision making • Define and use key data-related terms and concepts • Assess data quality, value, and limitations • “Triangulate” different data reports and summaries • Identify and appropriately use data needed to set priorities and determine resource allocations • Use data for developing directives to the recipient on how best to meet service priorities • Use data in reviewing and recommending improvements to the system of HIV care in the EMA/TGA

5. Agenda • Data and Decision Making: Status and Needs • Ensuring a Shared Knowledge Base • Using Data in Decision Making • Assessing and Interpreting Data • Sum-Up and Assessment

6. Data & Decision Making: Status & Needs

7. Data-driven Decision Making • Objective and fair • Highlights unique aspects of your EMA or TGA • Guides entire planning process • Comprehensive/Integrated Planning • Need Assessment • Priority Setting, Resource Allocations, and Directives • Establishing and Strengthening the System of Care • Quality Management including Standards of Care • Performance and Outcomes Evaluation • Assessment of the Efficiency of the Administrative Mechanism

8. Comprehensive/Integrated Planning • Legislation requires a “comprehensive plan for the organization and delivery of health and support services” that: • Includes a strategy for finding PLWH who are out of care and helping them enter care • Includes a strategy for testing people who don’t know their status and helping them enter care if HIV-positive • Coordinates services with HIV prevention and with substance abuse prevention and treatment • Is compatible with other state and local HIV plans • Plan goals, objectives, and strategies should be data-based

9. Needs Assessment • A systemic process used to collect and analyze information about the number, characteristics, and needs of PLWH in and out of care, identify current resources available to meet those needs and determine unmet needs and service gaps • Includes both quantitative (numerical) and qualitative (non-numerical) data • A major source of data for decision making – and a planning task that is led by the Planning Council

10. Needs Assessment Components • Epi profile of HIV & AIDS cases and trends • Estimate & assessment of unmet need and undiagnosed – PLWH who know their status but are not in care and PLWH who do not know their status • Service needs and barriers for PLWH in & out of care, including those who don’t know their status • A resource inventory of existing services • A profile of provider capacity and capability (availability, accessibility & appropriateness overall and for specific populations) • Service gaps for those in and out of care, including disparities in access to services for subpopulations

11. Priority Setting and Resource Allocations • Three interrelated components: • Priority Setting – determining what services people with HIV in this EMA/TGA who are eligible for Ryan White services are most likely to need and the relative importance of these services • Resource Allocation– determining the amount of RWHAP Part A funds to be allocated to each of the service categories prioritized by the PC • Directives– developing guidance to the recipient on how best to meet the PC’s service priorities • Decision making for all components should be data-driven

12. Developing and Strengthening a System of HIV Care • Continuum or System of HIV Care – a set of programs that provide a full range of emergency and long-term services and resources that address the needs of persons living with HIV (PLWH) • Data from many sources needed to identify weaknesses or gaps in the system of care and find ways to improve services overall and for particular PLWH subpopulations and specific geographic areas within the EMA or TGA

13. Data Needs: Discussion • What are some of the Planning Council’s and committees’ data needs? • What data does the PC use in priority setting and resource allocations? • What are some of the challenges faced by members in obtaining and using data?

14. Activity A: How Are We Doing? Assessing Data Availability and Use for PSRA • Work in a small group • Choose a facilitator, recorder, and reporter (or have one person serve as both recorder and reporter) • Focus on the area assigned to your group: Priority Setting, Resource Allocations,or Directives • Discuss the following, then share with the full group: • What kinds of information does the Planning Council need to make sound decisions in the assigned area? • To what extent does the Planning Council have the information/data it needs to make sound decisions in this area? What other kinds of information do you feel are most needed? • How comfortable are you in reviewing, interpreting, and using available needs assessment data? Why?

15. Importance and Challenges of Using Data • Obtaining and using data effectively is central to the success of a planning council • Data are essential to understanding the epidemic and the service needs and gaps of different subpopulations of PLWH • Everyone uses data, but people new to community planning often need training in use of multiple types of data for decision making • PC and Committee leaders should ensure appropriate training for all members and encourage user-friendly data formats and time for full discussion and review of new data and analyses

16. Ensuring a Shared Knowledge Base

17. Terms and Concepts: Activity B • Using the sheet provided for Activity B, spend a few minutes individually matching as many terms and concepts as you can • Then share your work in a small group. Identify areas of disagreement or confusion and any concepts the group does not fully understand • Be prepared to share your work with the full group

18. Terms Related to Epidemiologic Data: Incidence • Incidence – the number of new cases of a disease in a population during a defined period of time – such as the number of new HIV cases in your EMA or TGA reported during 2016 • Incidence rate – The frequency of new cases of a disease that occur per unit of population during a defined period of time – such as the rate of new HIV cases per 100,000 population in your EMA or TGA in 2017

19. Terms Related to Epidemiologic Data: Prevalence • Prevalence – The total number of people (living or dead) in a defined population with a specific disease or condition at a given time – such as the total number of people diagnosed with HIV in your EMA or TGA as of December 31, 2016 • Prevalence rate – The total or cumulative number of cases of a disease per unit of population as of a defined date – such as the rate of HIV cases per 100,000 population diagnosed in your EMA or TGA as of December 31, 2016

20. Defining “Unmet Need” or “Not in Care” • HRSA/HAB definition for “unmet need” or “out of care” has been the following: No evidence of any of the following in a recent 12-month period: • CD4 count • Viral load test • Prescription/use of anti-retroviral therapy • Recent definition, based on the HIV Care Continuum: “Not retained in care” means less than 2 HIV-related doctor visits (or 2 viral load tests or CD4 counts) during the past 12 months, at least 3 months apart

21. Other Terms Related to Unmet Need • Service Gaps – The identified unfulfilled need for HIV/AIDS related services other than primary medical care among individuals who know their HIV status in a specified geographic area • Estimate of Unmet Need: number and percent of all diagnosed PLWH who are not in care • Assessment of Unmet Need: Analysis of the characteristics of PLWH who are not in care – demographics, geographic location, care history, and why they are not in care • Plan for addressing unmet need: Strategies for linking PLWH to care immediately after diagnosis and for finding people who are out of care and re-linking and retaining them in care

22. Unmet Need in Your EMA/TGA [Add information on the estimated number and percent of PLWH who are in and out of care in your EMA/TGA, including trends if possible. An estimate is generally prepared annually and included in the RWHAP Part A competitive application.]

23. Discussion: Understanding Unmet Need • If your EMA’s needs assessment finds that PLWH are missing medical appointments due to a lack of transportation services, is this an unmet need? • If the EMA finds 100 Ryan White eligible PLWH who are not receiving HIV-related primary medical care, what does this tell us? What else would you want to know about them? • If out of those 100, 25 are receiving Food Bank services, and 27 are receiving non-medical case management, what does this tell us about unmet need?

24. Important Concepts in Assessing Service Needs and Gaps • Geographic disparities – differences in access to needed services based on where an individual lives • Availability of services – the level or number of “slots” within a service category that exist in a specified geographic area and whether there are waiting lists • Accessibility of services – the extent to which services in a particular geographic area can be obtained conveniently by people who need them, in terms of factors like access to public transportation, service hours, Americans with Disability Act (ADA) compliance • Appropriateness of services – the extent to which services meet the needs of various PLWH subpopulations, in terms of languages spoken, cultural competence with regard to race/ethnicity, sexual orientation and identity

25. Using Data in Decision Making

26. Data Sources and Needs for Ryan White Planning Epi Profile Needs Assessment Data HIV Care Continuum Data Data Service Expenditure Data Testing and Unmet Need Data Client Characteristics & Utilization (RSR) Data Clinical Quality Management Data Performance & Clinical Outcomes Data

27. Discussion: Identifying Data Sources What data sources available to the Planning Council are most likely to provide data for decision making on each of the following: • Age and gender of the newly diagnosed • Geographic parity or disparities within the EMA or TGA • Availability of resources for a particular service • Accessibility of medical case management services in particular parts of the EMA/TGA • Appropriateness of medical case management services • Barriers to primary medical care for specific subpopulations • Clinical outcomes for RWHAP Part A clients

28. Using Data for Developing Directives • Instructions to the recipient that are translated into RFP and contract language on how to best meet needs • Based on data, addresses some issue not currently in the standards of care • Frequently addresses strategies to: • Better serve a particular subpopulation • Focus on a particular geographic area • Improve or change a service model Can you name some possible directives to strengthen services in your EMA or TGA?

29. Using Data to Improve the System of Care • Different types of data help answer different questions about the current system of care – needs assessment and other data such as client characteristics, service utilization, HIV care continuum findings, and quality management data • The system of care changes based on changes in service priorities, allocation of funds, directives, and standards of care • PC works with the recipient on new or revised service models that address identified limitations • Data on increased utilization, decreased disparities, improved performance and clinical measures help evaluate improvements in the system of care

30. Activity C: Identifying Data Needs for Specific Decisions • Work in a small group • Choose a facilitator and recorder/reporter • Review the background description and the area of decision making assigned to you and answer the questions provided (See the Activities sheets) • Be prepared to share your work with the full group

31. Assessing and Interpreting Data

32. Assessing and Interpreting Data Essential planning council member roles: • Reviewing data from multiple sources • Asking questions and questioning data • Comparing and weighing data from different sources and studies • Giving the greatest weight in decision making to the “best data”

33. Tools for Assessing & Interpreting Data • Triangulation – the process of comparing results from different needs assessment or research studies or sources to see whether they report similar findings • Critical review or weighing of needs assessment data – review of the methods used to obtain needs assessment data in order to decide how much confidence to place in the data

34. Factors to Consider: Who was Included in the Survey or Study? • Numbers:Numbers of people or sample size • Representativeness:Do the individuals sampled represent the entire HIV population, a subpopulation, or the targeted portion of the community – which may be obtained through: • Probability sampling:Using a probability sampling method designed to allow findings to represent the entire population from which the sample was drawn • Purposive or representative sampling: Selecting people for the study so that they mirror the HIV population in your EMA or TGA or the subpopulation you are targeting

35. Questions to Ask in Assessing and Interpreting Surveys and Studies • Who was responsible for the study? Were knowledgeable consumers and other PLWH involved in design? • Are these good questions? Are they clear and understandable? Do they seem likely to generate reliable data that really measures what the study is supposed to be measuring? • What evidence is there that the data were collected using appropriate methods and trained individuals? • Was there “quality control” to be sure the stated data gathering and analysis process was followed?

36. Activity D: Assessing Data Reports • Work in a small group • Assign facilitator, recorder, and reporter roles (Make different assignments from the last activity) • Read and follow the directions for Activity D on the sheets provided • Be prepared to present your work to the full group

37. Sum Up: Assessing and Interpreting Data • The voices in my head during data presentations are a natural part of the process of reviewing the data • Triangulation helps me compare various data sets • It is important to understand the data collection or needs assessment process in order to assess the quality and value of a study or a set of data • The voices in my head are not always right and should listen to the data

38. Training Sum Up and Assessment