Mental Health Evaluation Activities of the VA Program Evaluation and Resource Center

1. Mental Health Evaluation Activities of the VA Program Evaluation and Resource Center Prepared for Institute of Medicine, Committee Evaluating VA Mental Health Services June 5, 2014

2. PERC History • PERC created in 1990 in response to the Federal Anti-Drug Abuse Acts (PL 100-689 and PL 100-690), which mandated ongoing evaluation of the quality, process, and outcome of VA SUD treatment • Duties subsequently added and scope expanded by VACO Directives and Congressional Acts. Now a component of VA Office of Mental Health Operations (OMHO) • Provide program evaluation and technical assistance for mental health quality improvement efforts across VA 2

3. Activities • Monitor the organization and delivery of mental health and substance use treatment services in specialty, mental health and primary care programs • Improve the accessibility, process and outcome of interventions for patients with mental health and substance use disorders • Provide data, analyses, technical assistance and support to facilitate implementation of VA and other federal policies regarding mental health and substance use treatment • Conduct requested evaluations of particular programs and initiatives 3

4. NDAA Report Process • Involved Collaborative Efforts: • Workgroups were formed around each of the areas singled out in the Act: capacity, timeliness, evidence-based treatment, veteran satisfaction, and staffing models • Each workgroup brought together individuals with expertise in that area and diverse perspectives to identify possible conceptual frameworks and existing and potential measures • Workgroup members drawn from policy and operations experts in VA MH. • Each workgroup identified key concepts and measures in their domain 4

5. NDAA Report Process • Best available data for each concept was summarized in plain language or graphically displayed for the semi-annual report • Reported data drawn from various sources, e.g., • Mental health budgets from the Office of Finance • Mental health staffing from the Allocation Resource Center • Veterans Satisfaction Survey data from OMHO • Mental health wait-times from Systems Redesign 5

6. New Data Collection • Veterans Satisfaction Survey • Response to OIG report on mental health treatment access • Annual mailed survey started in 2013 (NEPEC) • Stratified random sample to ensure VISN and facility representation; potential respondents were Veterans receiving mental health treatment • Almost 10,000 Veterans completed survey: 15% OEF/OIF/OND Veterans, 12% women • Survey covered access to mental health treatment and patient satisfaction 6

7. New Data Collection • Veterans Health Outcomes Improvement Pilot • Designed to test feasibility and acceptability of repeat phone-based assessment of patients who initiate care in mental health • Pilot project in 6 sites, 1,140 Veterans completed baseline call-center administered survey • 65% of baseline participants completed 3-month follow-up survey • Patient assessments fed-back to providers to guide care decisions • Facility-level data reported to each participating site 7

8. New Data Collection • Veterans Outcome Assessment Project • National project designed to assess patients entering a new episode of mental health treatment at treatment initiation and 90-days later • Comprehensive review to identify items for inclusion in a brief omnibus instrument, applicable across diagnoses; currently in the OMB approval process • Each quarter, call-center administered survey with stratified random sample of 400 Veterans newly seeking mental health treatment • Survey repeated 3 months after initial visit with all patients who agree to continued participation • Estimate treatment response and experience of care at the national level quarterly and VISN level annually 8

9. PERC Evaluations • PERC participates in on-going evaluation of VA mental health treatment programming overall including: • Quarterly review of roughly 200 MH quality indicators plus 23outcome-oriented quality measures to assess implementation of the Uniform Mental Health Services Handbook, access to care, use of evidence-based practice, and Veteran health status • Tri-annual review of facility data in preparation for comprehensive site visits, summaries of site visit reports, and on-going monitoring of progress on action plans to address identified concerns • Annual National Summary of site visit findings • Strong Practices in MH services • Monthly assessment of mental health outpatient staffing, workload and productivity at the facility and provider level • Annual VA Provider and Veteran Satisfaction Surveys • Annual assessment of health care diagnosis and treatment trends for VA patients with substance use disorders • Bi-annual survey of specialty substance use disorder programming • Maintenance of databases on MH patient populations, health care utilization, location of care, and staffing for ad hoc immediate information requests 9

10. PERC Evaluations • PERC has recently completed or is currently conducting special evaluations of: • Executive Order Community Partnership Pilots • Behavioral Health Interdisciplinary Program Teams • Opioid Therapy Clinical Practice Guideline adherence • Call-center-based patient symptom and functioning assessment • Overdose Education and Naloxone Distribution • Mental Health Hiring Initiative • Overall outpatient Mental Health staffing models • Mental health access measurement (OIG report response) • NDAA report and public website 10

11. OEF/OIF/OND Veterans • Typically, PERC conducts evaluations of mental health programs and policies for the entire Veteran population, asking specific questions or flagging OEF/OIF/OND Veterans to allow break-out of the OEF/OIF/OND population when relevant for decision-making. 11

12. PERC & Health Services Research • PERC does not conduct research • Staff only conduct program evaluations and support quality improvement efforts • Because VA is a large health care system, results of evaluation work may help inform non-VA healthcare • As part of clinical operations, PERC’s evaluation efforts are embedded in quality improvement and operational decision-making processes • Not one-time findings 12

13. Impacts of PERC Health Services Evaluations • PERC has developed, validated, implemented, and evaluated data systems and initiatives that support and inform improved treatment of mental illness. • Example: Mental Health Information System • PERC helped develop and field a system of over 200 MH metrics assessing implementation of the VA Uniform Mental Health Services Package. • Findings demonstrated facility variation in implementation of MH services. Strong practices were identified for dissemination and attention brought to local areas of concern. • Variation in facility implementation of MH services is monitored and analyzed in an on-going manner and used to guide development of action plans for mental health quality improvement at each VA facility 13

14. Impacts of PERC Health Services Evaluations • Example: Mental Health Outpatient Clinical (MHOC) staffing assessments • Initial staffing analyses showing relationships between staff-to-patient ratio and implementation of the Uniform Mental Health Services Handbook guided plans for the Mental Health Hiring Initiative (MHHI) • Developed system for tracking MHOC FTE at the provider level • Found that the MHHI increased VA MH staffing and number of Veterans receiving mental health treatment • Found that MHOC staff/patient ratio is correlated with wait-times, population access, treatment intensity, and patient and provider reported access and satisfaction • MHOC staffing data guide decision-making on MH staffing needs across VA facilities 14

15. Impacts of PERC Health Services Evaluations • Example: Opioid therapy guideline adherence dashboard • Developed quality metrics to identify facility-level adherence to key clinical practice guideline recommendations, and provided feedback to facilities to guide local quality improvement efforts • Identified relationships between guideline adherence and patient risk of adverse events • Findings used to guide local efforts to increase urine drug screening, leading to increases in this risk-mitigation strategy • Findings used to develop and prioritize a national VA Opioid Safety Initiative to reduce opioid-related adverse events • Findings serve as a foundation for HHS-led interagency workgroup recommendations for CMS meaningful-use criteria for reducing opioid-related adverse events 15

16. Impacts of PERC Health Services Evaluations • Example: Overdose Education and Naloxone Distribution (OEND) • PERC reviewed literature and contacted non-VA OEND programs to obtain information on effectiveness and implementation of OEND • Worked with local VA pilot programs to understand processes and needs • Led to National workgroup to facilitate OEND implementation • Information letter • Naloxone kits on national formulary • Overdose recognition and response training materials • Educational materials • Implementation toolkit

17. Identifying Challenges • Challenges in Providing Mental Health Services to OEF/OIF/OND Veterans: • Rapid growth in demand provides challenges in obtaining adequate space and qualified staffing to meet needs • OEF/OIF/OND Veterans are not always located where VA facilities and treatment resources are available • Information technology development processes and restrictions limit rapid innovation to deliver care via novel media or at a distance • Overcoming reluctance to seek mental health treatment, finding new ways to encourage help seeking and use of effective treatments 17

18. Thank You!