PBHCI National Evaluation

PBHCI National Evaluation RAND Corporation & National Council for Community Behavioral Healthcare

Welcome! Audrey Burnam, PhD

The RAND Corporation • Nonprofit institution that helps improve policy and decisionmaking through research and analysis. • RAND is tasked with completing an independent and objective national-level evaluation of PBHCI • Funded jointly by SAMHSA and ASPE

National Council Jeff Capobianco, MA

The National Council The National Council for Community Behavioral Healthcare (National Council) has won a competitive grant funded jointly by SAMHSA and HRSA to establish the National Training and Technical Assistance Center for Primary and Behavioral Healthcare Integration. The National Council is working closely with RAND and will provide technical assistance to grantees to insure they are able to successfully provide the data required for the evaluation.

RAND’s Tasks …back to Audrey Burnam, PhD

Overview of RAND’s Tasks Conduct a national evaluation of the PBHCI program that answers 3 questions: • Outcomes: Does integrating primary and behavioral health care lead to improvements in mental and physical health of persons with SMI and/or substance use disorders? • Process: Is it possible to integrate the services provided by primary care providers and community-based behavioral health agencies? • Models: Which models and/or respective model features of integrated primary and behavioral health care lead to better mental and physical health outcomes?

Goals of This Webinar • Part I: • Achieve common understanding of: • The framework for the national evaluation • Data that support the national evaluation • Procedures for transmitting data to RAND • Part II: • Data collection procedures

PART I: Framework for the National Evaluation Deborah Scharf, PhD

Structure-Process-Outcomes Evaluation Model Independent Variables Dependent Variables • STRUCTURE OF CARE • What services are available? • Core (required) components: • Screening/referral/treatment • Registry/tracking system • Care management • Prevention/wellness approach • Examples of optional components: • Co-location of nurse practitioners/PCPs • Identification of primary care supervising physician • Embedded nurse care managers • Use of EBPs • Use of SBIRT • Other • PROCESS OF CARE • To what degree are services implemented? • Examples: • Numbers/proportions of SMI patients screened/referred/treated & for what • Number of records/types of information tracked in registry • Number of SMI patients connected with care manager & for what • Number of individualized wellness plans developed and utilized • OUTCOMES OF CARE • Does it make a difference? • Improved overall health • status of clients: • Physical health outcomes • (e.g., BMI, blood pressure, blood glucose or HbA1c, lipid profile) • Mental health outcomes Environment/Administrative Context (e.g., size, auspice, location, health IT)

Data Sources (I) • TRAC • Quarterly reports • Clinical registries *Staff may be asked to participate in interviews or surveys *Some sites may be invited to participate in other evaluation activities

Data Sources (II)

TRAC: CMHS Transformation Accountability Susie Lovejoy, MS

Transformation Accountability (TRAC) In response to the 1993 Government Performance and Results Act (GPRA) requirements CMHS developed the TRAC system. TRAC is the web-based system through which all SAMHSA CMHS grants report performance measurement data. Measures include: client level outcomes, infrastructure development indicators, measures of mental health promotion and prevention activities, and satisfaction with technical assistance services.

TRAC (II) • All of you (grantees) have been trained on the Infrastructure, Prevention, & Promotion (IPP) portion of TRAC. • Grantees funded in Sept 2009 (cohort I) have been trained on the client-level outcomes measures for people receiving direct services through the grant. • Grantees funded in 2010 (cohorts II and III) will be trained in February to collect and enter the remaining data into TRAC

TRAC (III) • Grantees must collect and report data on behavioral health outcomes for each person receiving services from the grant (NOMS). • Grantee staff interview consumers and enter data into TRAC at: • Entry into integrated care • Every 6 mos while enrolled • At discharge from the program

TRAC (IV) • For PBHCI, grantees must also log physical healthcare indicators in TRAC • Required indicators: Height, weight, BP, glucose or A1C, cholesterol, triglycerides • Optional indicators: Waist circumference, Breath CO (for smoking status)

Intake, 6 months, discharge: Height Weight Blood pressure Waist circumference* Breath CO* Intake, annually, discharge: Glucose or A1C Triglycerides Cholesterol Successful 8h fast* TRAC (V) *Recommended Note: 6 month indicators are collected quarterly but entered bi-annually

TRAC (IV) • TRAC data can be downloaded • This can help you avoid double-entry • Collect and enter the physical healthcare data in the same time windows as you collect and enter the TRAC client-level interviews. • This TRAC change will be in place ~Feb 2011

PBHCI Secure SharePoint Site Regan Main, BA

PBHCI Secure SharePoint Site • SharePoint is the primary method of transmitting quarterly reports and clinical registry data to SAMHSA and RAND

SharePoint (II) • Use the SharePoint site because it is secure • Meets HIPAA data protection requirements • Each grant program has a password-protected folder • Uploading data is easy!

SharePoint (III) Log-on to SharePoint using the link below: http://sharepointext.rand.org/sites/pbhci/default.aspx *Watching this presentation from the archives? See separate file with additional SharePoint slides

Quarterly Reports Nicole Eberhart, PhD

Quarterly Reports • Required quarterly by every grantee • 1 report per program • Uploaded to your folder in SharePoint • Quarterly reports are Word documents • RAND provides a template • Blank templates are on SharePoint

Quarterly Report: Labeling VERY IMPORTANT! • Top of report: • Your grant number • Name of your site and program • Name of the Project Director • Name of the person to contact (if not the Project Director) • Label the report for the dates covered and the date submitted • Put the quarter number and year • Label electronic attachments with something that includes the abbreviated site name, Quarter #, and Year. • Example: ANN ARBOR CMHC 1st QTR 1/31/10

Quarterly Report: General Issues • Aim for 3-5 pages • Answer each question with regards to both • Core program features • screening/assessment • treatment and referral • registry/tracking • care management • prevention/early intervention/wellness • referral/follow up • Optional features, e.g.: • use of SBIRT and/or other evidence-based practices • primary care supervising physician • embedded nurse care manager, etc.

Quarterly Report: Content (I) • Describe your program’s accomplishments • Describe any changes in staffing since the last quarter • Clinical discipline • Role • Duties of grant-supported new staff • Changes in % effort • Attach resumes and any formal correspondence made between you and SAMHSA about staffing changes • Describe the involvement of consumers and families in your project

Quarterly Report: Content (II) • Barriers in implementing your program, solutions you generated, or your plans to resolve them • Delays in the program as you originally proposed it, changes in content or timing • Infrastructure activities, staff involved, and associated expenditures • Wellness-related education and programming, staff involved, associated expenditures

Quarterly Report: Content (III) • Progress with data collection (software, clinical registry, TRAC, qualitative data) and plans to monitor, analyze, and/or share the data • Specific program components supported by grant monies • Funding sources for key primary care services provided via your program, and sustainability beyond the official grant period

Quarterly Report: Content (IV) • Which patients are eligible to receive PBHCI services • All SMI patients? New SMI patients? SMI patients with physical health conditions? SMI patients receiving psychotropic medication? • Staff involvement in PBHCI group activities or with SAMHSA Project Officer • Group conference calls, grantee meetings, site visits and interaction with the Project Officer or Technical Assistance Provider(s). • Other sources of funding, and proportion of PBHCI consumers who are also participating other funded programs

Glossary • Assessment: Actions taken following a positive screen to provide a more comprehensive or in-depth picture of an individual patient’s specific physical health condition(s) and to determine the best service plan to address them • Early intervention: Interventions that are appropriate for, and specifically target, people displaying the early signs and symptoms of a physical health condition. • Prevention: Interventions that occur before the initial onset of a physical health condition to prevent the development of that condition. • Qualitative data collection methods: Interviews, focus groups, etc. • Screening: Refers to a preliminary procedure, such as a test or examination (e.g., blood pressure for hypertension, cholesterol for coronary artery disease), to detect the most characteristic sign(s) of a physical health condition that may require urgent attention or further investigation. • Wellness-related education and programming activities: Activities that promote wellness in mind, body, and spirit. They may include educational activities (e.g., nutrition education, physical activity education), physical activities (e.g., yoga classes, walking groups), and psychosocial activities (e.g., peer support programs, health-focused social functions).

Clinical Registries Deborah Scharf, PhD

Clinical Registries Two purposes: Improve clinical care Record consumer-level clinical interactions Data system for consumers with a specific type of disease diagnosed and treated at a practice* Allows care team members to proactively manage consumers with chronic diseases* Grantees must maintain a clinical registry *Iowa Dept of Public Health, 2010

Benefits of a Registry* • Enable the provider to ensure that all their patients are getting proper care • Track progress of high-risk patients • Identify the need for follow-up services • Increase quality of care and improve patient outcomes • Coordinate care and identify gaps • Incorporate consensus guidelines for disease management *Iowa Dept of Public Health, 2010

Evaluation Uses of a Clinical Registry Facilitate data collection Registries can and should include all key client-level data elements Standardize client-level data collection Data collection can be aligned with flow of integrated clinical care

Attempt To Balance Simplicity & Complexity for Serving Multiple Purposes Overly Simplistic “Reasonable” Middle Ground Overly Complex • Facilitates sites’ efforts to provide high-quality clinical care • Assists sites in meeting SAMHSA’s reporting requirements • Supports meaningful evaluation • Exceeds sites’ data collection resources/ capacities • Results in incomplete and/or unreliable data collection across sites • Meets minimal reporting expectations • Provides baseline data w/out longitudinal, clinically relevant follow-up information

Clinical Registry Items Extracted for the Evaluation

Clinical Registry Items Extracted for the Evaluation (II) • Each consumer’s total contacts with: • Care managers • Primary care providers (MDs, LPNs, PAs) • Psychiatrists or psychiatric nurses • Counselors (LSW, psychologists, substance abuse counselors, others) • Peer specialists • Other specialists (dentist, nutritionist, others) • Wellness programs

Clinical Registry Items Extracted for the Evaluation (III) • IMPORTANT! • All data fields to be extracted from clinical registries should be whole numbers • i.e., 1, 2, 3, … 99 • Even lists of EBPs should be coded as whole numbers • RAND will provide you with a list of EBPs and wellness activities associated codes • Example EBP: CBT = 1; MI = 2 • Example wellness activity: Diabetes education = 1; Smoking cessation = 2

Registry: For Cohort I • Clinical registry data is taking the place of numeric data previously reported in quarterly reports • Data are now collected at the patient-level • Not in aggregate (i.e., not at the program-level)

If You Already Have a Clinical Registry… • Make sure it contains all necessary fields • Review these materials, pg 12 of your RFA, and talk to your contact at the Technical Assistance Center • Establish a plan for: • Extracting all necessary fields • Making all data fields numeric, whole numbers, and consistent with PBHCI requirements • Your registry will likely contain data that we do not collect for the evaluation

If You Do Not Have a Clinical Registry… • Look carefully at the data to be extracted from PBHCI registries • Work with the Technical Assistance Center to select a registry tool that will meet your clinical and research needs

When Choosing a Registry, Consider… To what extent will using the registry improve the quality of your program’s integrated care? To what extent is the organization of the registry aligned with the flow of integrated care at your site? To what extent will the registry help you automate your data collection and reporting?

THANK YOU!

PBHCI National Evaluation