WISEWOMAN

1. WISEWOMAN Well-Integrated Screening and Evaluation for Women Across the Nation National Program EvaluationSeptember 2010

2. Agenda Review and discuss the results of the comprehensive evaluation of the WISEWOMAN program (FOA 2008 to date). Report the results from the comprehensive evaluation and Program Progress and Effectiveness Assessment (PPEA). Discuss current efforts by CDC to address the evaluation results and recommendations.

3. Objectives and Evaluation Components Objectives of the Evaluation Assess outcomes and impacts of the national WISEWOMAN Program Monitor the program�s progress toward reaching its intended goals and objectives Uncover the program�s best practices and improvement strategies Evaluation Components Comprehensive Program Evaluation Program Progress and Effectiveness Assessment (PPEA) Program Monitoring

4. Maximizing Reach in Priority Population Number of clinical screening sites varies by program, ranging from 1 to over 1300 plus screening sites. Most programs (18) offer integrated office visits during which participants receive WISEWOMAN and BCCEDP screening services. Overall the program was projected to reach 37,529 women with WISEWOMAN services between June 2008 and June 2009. The program ultimately screened 34,391 women (92% of goal). Recommendation Assess any barriers to programs� ability to offer integrated screenings and continue to work with those programs without fully integrated office visits to address the barriers to help maximize the reach of the programs.

5. Implementation Consistent With Requirements Programs commonly report that they implement WISEWOMAN direct services consistent with program guidelines; however, we found variability in their application of the guidelines. Programs commonly report that they offer integrated BCCEDP/WISEWOMAN office visits; however, a few programs (3) have CDC-approved exceptions to this guideline to accommodate the program�s or participants� specific needs. Recommendation Provide consistent communication of the program guidelines. Work to understand the barriers to implementing the program guidelines and remain flexible (e.g., exceptions) as the program moves toward greater consistency.

6. Assessing and Meeting Participant Needs Most programs (16) assess patient needs using informal or formal procedures. Programs commonly tailor their services to meet participant needs by providing transportation incentives, translated materials, and culturally relevant marketing materials. Recommendations Assess existing culturally relevant program materials to share with programs. Related to this, provide programs with information and resources on cultural adaptation (i.e., put resources on CDC Web site). Develop a toolkit of culturally tailored program materials for programs to provide to select vulnerable audiences; evaluate the cultural relevance and appropriateness of tailored materials.

7. Training, Guidance, and Technical Assistance Distinction between training, guidance, and technical assistance (TA) blurred among project officers and funded programs. Programs commonly reported that the annual WISEWOMAN meeting is their main source of training. Main challenge for programs is inconsistency with CDC guidance and TA. High turnover among CDC project officers yields differing guidance over time. Recommendations Plan and convene an annual WISEWOMAN meeting separate from the HDSP meeting. Ensure that project officers convey the same messages when providing TA to programs. Offer programs monthly trainings focused on specific evaluation topics (e.g., Webinars on evaluating program participants� changes in knowledge, awareness, behaviors).

8. Work With CDC Programs and External Agencies The program works with other CDC programs (e.g., BCCEDP, OSH). No formal partnerships with external agencies exist. Funded programs commonly report that they are integrated with several CDC-funded State-level programs (e.g., HDSP, BCCEDP). External partners vary across programs. (These include AHA, ACS, local hospitals, colleges/universities, YWCA/YMCA). Recommendation Identify and reach out to key external agencies with which the program should work to further the WISEWOMAN mission/goals.

9. Improved Quality of and Access to Care Program services improve quality of care through adherence to national guidelines, identification of alert values, patient satisfaction, and engagement in the program. Program services improve access to care by providing participants a �medical home,� offering connection to no- or low-cost medical care; screening and referral services; and cultural and linguistic tailoring of services, materials, and referrals. Recommendation Formally assess program participants� perception of improved quality of and access to care offered through the program.

10. Lifestyle Interventions (LSIs) Selection Most programs (18) use a CDC-approved LSI template. Most common templates are: New Leaf (9) and Michigan Health Partnership (4). Common factors influencing LSI selection include CDC-approved template, evidence-based intervention, and adaptability (target population and delivery).

11. How Programs Implement LSIs LSI content, intensity, and mode of delivery are often adapted to meet WISEWOMAN clients� individual needs. Facilitators for LSI implementation: sufficient numbers of trained LSI providers, committed LSI providers, data systems, culturally relevant services. Barriers to LSI implementation: participant tracking and follow-up, and limited resources. Between June 2008 and June 2009, 21,209 program participants had at least one LSI session/contact: Individual setting-87.36%, Group setting-5.48%. Recommendations Clarify and share acceptable modifications to CDC-approved LSI templates in order to ensure that the theoretical integrity of the LSI is maintained. Work with programs to identify and address barriers to LSI implementation.

12. Fidelity to LSI Implementation Programs ensure fidelity of LSI implementation through training and technical assistance to LSI providers and data systems to monitor LSI activities. Fifteen programs have a definition for LSI completion; however, these programs offer varied definitions of LSI completion, based on the number of sessions/contacts provided. Programs did not have data on the number of women who have completed the LSIs to date. Recommendation Clarify what is meant by LSI completion.

13. Evaluating LSIs Per the cooperative agreement, CDC does not require outcome evaluation of the LSI component. Some programs (3) are preparing to conduct outcome evaluation in the near future. Many programs (16) conduct process evaluation to track LSI participations (16 programs), ensure participant satisfaction (2 programs), and identify issues to troubleshoot with LSI providers (4 programs). Some programs (3) conducted formative evaluation (e.g., focus groups) to inform LSI implementation. Recommendations Develop clear expectations for evaluation of the LSI component of the program. Identify the appropriate measures for assessing the short-term (e.g., change in knowledge and awareness), intermediate (e.g., increased physical activity), and long-term (e.g., improved screening numbers) LSI outcomes. Provide programs with the tools, resources, and necessary guidance to help them evaluate the LSI component of the program.

14. How CDC Is Attempting to Address Recommendations Ensuring consistency in the communication of program guidance Sharing program messages via blast e-mail, all program calls, and annual meetings Clarifying program expectations Development of performance measures Planning to conduct additional trainings Revisiting evaluation expectations for funded programs Developing criteria for LSI modifications

15. Acknowledgments CDC Rachel Barron-Simpson Deborah Borbely Sabrina Chapple Diane Dunet Dianne May Alberta Mirambeau Lois Voelker ICF Macro Doryn Chervin Michael Greenberg Ashani Johnson-Turbes Thearis Osuji Michelle Revels Julia Ruben Dara Schlueter

16. Contact Information Rachel Barron-Simpson, MPH CDC/DHDSP/Applied Research and Evaluation Branch Phone: (770) 488-4825 E-mail: rachel.barronsimpson@cdc.hhs.gov Ashani Johnson-Turbes, PhD ICF Macro Phone: (404) 321-3211 E-mail: cjohnson-turbes@icfi.com