Gustavo Angeles, Jose Urquieta, Bernardo Hernandez, Mara Tellez-Rojo, Aurelio Cruz April 2009

1. Building Capacity on Program Evaluation in Latin America: The Experience of the Partnership between Mexico’s National Institute of Public Health (INSP) and the MEASURE Evaluation Project Gustavo Angeles, Jose Urquieta, Bernardo Hernandez, Mara Tellez-Rojo, Aurelio Cruz April 2009

2. The situation in LAC: • Shortage of professionals with M&E skills, but few places offer M&E training • Increasing demand for M&E services, but few places offer M&E technical assistance to programs Our Response: A partnership to build a Center of Reference in M&E Why a Partnership? • INSP recognized the need to strengthen internal capacity to better respond to demand for M&E services and training • MEASURE Evaluation is only a 5-year project, need to work at institutional-level to achieve sustainability • Establish long-term relationship

3. Instituto Nacional de Salud Pública (INSP) • In Cuernavaca, Morelos, Mexico • Leading center of training, research, & practice in public health in Mexico and Latin America • Closely linked to public health programs • In late 1990s, INSP participated in evaluation of impact of PROGRESA; it took leading role in 2001. Evaluation portfolio has expanded to other social and health programs.

4. MEASURE Evaluation • USAID-funded; since 1997 • Goal : To improve collection, analysis, and presentation of data to promote better use of data for decision making in PHN programs • Implemented by the Carolina Population Center, University of North Carolina at Chapel Hill, with Futures Group, John Snow, Inc., Macro International Inc., Management Sciences for Health, and Tulane University • Areas: development of M&E methods and applications; capacity building; coordination & collaboration

5. The Partnership Established in 2004 Objectives: • Strengthen INSP’s capacity to: • offer M&E training programs in a sustainable way • provide M&E technical assistance • conduct evaluation studies • Institutionalization of M&E • Building a cadre of trained M&E professionals in the Latin America region

6. Range of activities • Engage leadership • Assessment of institutional M&E capacity • Training of key faculty • Collaborative development of curricula and training materials • Collaboratively offering training programs: Workshops & Master’s degree training • Provision of instructors, materials, and fellowships • Engage INSP faculty in evaluation research and technical assistance • Work on sustainability (technical, logistical and financial)

7. Some Results: Training Programs • Short-Term Training in M&E - 8 Regional Workshops: HIV/AIDS, Impact, PHN, RHIS - 159 professional trained from 16 countries (515 applicants) • Master’s – level Degree Training in M&E - Diploma in Program M&E as part of two Master’s Degrees: Master of Public Health (MPH) and Masters of Health Sciences (MHSc) - 59 students graduated/pursuing diploma in M&E, from 405 applicants

8. Institutionalization of M&E Partnership contributed to ongoing process of INSP defining its role in M&E: • INSP adopted “Health Program and Policy Evaluation” as a Strategic Line of Action • Creation of a new “Center for Program Evaluation and Surveys” • Creation of M&E Tracks in 2 Master’s degree programs • Diploma programs in Economic Evaluation of Health Programs and M&E of Nutrition Programs • Increased technical and logistical capacity to offer M&E training programs

9. INSP Involvement in M&E Practice and Research • Evaluating Mexican government health programs: Oportunidades (formerly Progresa), Seguro Popular • Conducting regional workshops in Latin America • Leading UNAIDS/WB-funded “AIDS Strategic and Action Plan (ASAP)” Consortium of 11 universities to provide TA on HIV Strategic Planning, mainly in Africa • Conducting evaluation studies, published in peer-reviewed journals

10. Lessons Learned: Keys to Success • Identify a “Champion” in M&E • M&E identified as an institutional strategic priority • Identify that there is need for M&E in the health system (a market) and the institution is flexible enough to respond to it • Facilitate incentives for faculty participation in M&E • Link teaching and practice: best way to learn and teach about evaluation is by doing evaluations • Technical aspects are not enough, it is necessary to establish administrative capacity and secure fundingsources for long term sustainability • Long-term commitment • Seek institutionalization of M&E trainings

11. Next steps • Are we meeting the demand for training in M&E? • Review M&E needs in the field • Diversify range of M&E workshop options: • Adapt workshop content • Reach a larger audience: distance learning, “mobile” workshops, virtual Master’s Degree • Improve follow-up with participants, provide support after workshop; forum/community of practice • Formalize a plan for institutionalizing M&E training programs • But, all these require funding, patience, and long-term commitment

12. MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) through Cooperative Agreement GHA-A-00-08-00003-00 and is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill, in partnership With Futures Group International, John Snow, Inc., Macro International Inc., Management Sciences for Health, and Tulane University. The views expressed in this presentation do not necessarily reflect the views of USAID or the United States government.