Building on the Measurement of CFIR Constructs in FQHCs: Where Do We Go From Here?

1. Building on the Measurement of CFIR Constructs in FQHCs: Where Do We Go From Here? Maria Fernandez, PhD on behalf of the CPCRN FQHC WG Investigators CPCRN Steering Committee - Virtual Spring Webinar April 16, 2014

2. CPCRN CHC Survey Primary Care Associations National Association of Community Health Centers (NACHC) Community Health Centers (CHCs) • Align with CHCs’ • missions Guided by real world health policy & health care delivery landscapes Health Care Reform Meaningful Use of EHR Patient-Centered Medical Home

3. Developing Measures of Constructs Associated with EBI Implementation Purposes Value One of the first studies to measure several constructs from CFIR Developed and tested measures that describe factors influencing implementation of EBAs in FQHCs Sets the foundation for establishing causal pathways and developing interventions that target these factors • Describe the survey development process and rationale • Present initial psychometric validation of CFIR Inner Setting measures • Discuss findings and what is next in our work assessing CFIR constructs (and PAR) related to EBI implementation

4. Consolidated Framework for Implementation Research (CFIR) Source: Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science 2009; 4:50. Note: Authors from the VA and University of Michigan, SPH, Department of Health Management and Policy

5. CFIR Constructs Relative Advantage, Complexity Patient Needs & Resources, External Policies & Incentives Available Resources, Leadership Engagement, Culture, Implementation Climate, Learning Climate Characteristics of Intervention Outer Setting Knowledge & Beliefs about EBAs Executing, Reflecting & Evaluating Inner Setting Characteristics of Individuals Process

6. Multiple Recruitment Strategies Sites partnered with their state’s PCAs; PCAs emailed their CHCs Sites recruited CHCs via email, telephone calls, or in-person meetings Introductory email with online survey link; 4 reminder emails; in-person meeting (one site) January - May 2013 IRB approval at each site and coordinating center

7. Inner Setting Constructs • Available Resources • Culture Overall • Culture Effort • Culture Stress • Implementation Climate • Learning Climate • Readiness for Change

8. Analysis plan We conducted separate CFAs for each of the seven inner setting constructs. Multiple fit indices were used to evaluate CFA model fit(CFI >.90, TLI >.90, SRMR <.08, and RMSEA <.08). Cronbach’salpha to evaluate internal consistency reliability. Correlation coefficients of each pair of scales to examine discriminant validity. Inter-rater reliability (ICC(1) & ICC(2)) and inter-rater agreement (rWG(J)) statistics were computed to assess the reliability and validity of computing clinic- and systems-level means from the individual-level data.

9. .419 A35A Available Resources .546 A35B .580 0.627 A35C .708 C20A .766 C20B .626 C20C .741 .824 C20D A11 Readiness for change .896 0.968 A12 .847 A13 .872 A14 .546 Implementation climate C11 .830 C12 .759 Inner Setting 0.514 C13 .462 C05 A01 .694 A02 .646 Learning climate A06 .768 0.980 .727 A15 A03 .663 .735 A05 A19 .723 A16Rev .556 A07 .642 .742 A22 A08 .674 A09 .719 A10 .731 0.945 .775 Culture A21 .332 A36 .438 A37 .387 A38 .594 A39 .480 A40 .612 A21 .440 A42Rev .581 A43 .387 A44Rev

10. Combined CFA results and fit statistics Robust Std. Errors in parenthesis CFI = 0.848 TLI = 0.835; SRMR = 0.079; RMSEA = 0.065

11. Discriminant validity of the five dimensions of inner setting Correlations using average score for each scale *p < .05

12. Clinic Level Inter-rater reliability and inter-rater agreement statistics Using average score for each scale *p < .05

13. System level Inter-rater reliability and inter-rater agreement statistics Using average score for each scale *p < .05

14. Conclusions • Our measures have structural validity • Our measures have reliability • Our measures have discriminant validity • Our measures can be aggregated to the clinic level • Clinics within systems have more similar values for inner-setting domain constructs than do clinics in different systems • We fit a second order “inner setting” factor

15. Practice Adaptive Reserve and Patient Centered Medical Home Best Practices at Community Health Center Clinics in Seven States

16. Adjusted Logistic Regression Frequency of PCMH Best Practices and PAR Scores PCMH CRC Screening Best Practices Dichotomized Score (6-8 vs. 0-5) Respondent reported performing PCMH best practices “usually” or “always” Adjusted for state, age, job type, years worked at the clinic, hours worked each week • National Demonstration Project - Highly-motivated practices w/ significant capability for change • Mean baseline PAR score 0.69 (s.d. 0.35) • Post intervention PAR score increased to 0.74

17. Discussion What have we learned about some of the measurement challenges in FQHCs? Where do we go from here in terms of measurement? Which constructs need more conceptual and measurement work? What additional work is needed in terms of reliability and validity assessment?

18. Discussion Continued Where do we go from here in EBI implementation in FHQCs? What opportunities exist? How can the measurement work be used in EBI implementation in FQHCs?

19. How FQHC WG can move forward? How could this group move forward on what’s next? What resources, partners, and preliminary work is needed to pursue what’s next?