Training Modules: Monitoring and Evaluation of Population, Health and Environment (PHE) Programs

1. Training Modules:Monitoring and Evaluation of Population, Health and Environment (PHE) Programs MEASURE Evaluation Developed by Caryl Feldacker.

2. Agenda

3. Training Modules • Introduction to PHE • Frameworks, Part 1 – Conceptual • Introduction to M&E • Frameworks, Part 2 – Logic models • Indicators • Frameworks, Part 3 – Indicator matrixes • Assessment and evaluation design

4. Starting on the Same Page… First, everyone should complete this free, online, two-hour course: http://www.cpc.unc.edu/measure/training/mentor/me_fundamentals Then, all participants should be able to: Identify the basic purposes and scope of M&E Identify the basic criteria for the selection of indicators Describe how indicators are linked to frameworks Identify types of data sources Describe how information can be used for decision making

5. Module 1: Training Overview

6. Training Programmatic Focus This course is geared for programs that: • Involve active partnerships between communities, non-government organizations, or government entities; • Promote an integrated, community-based approach to problem solving; • Operate in limited resource settings (smaller budgets); • Collaborate between population, health and environment (PHE) sectors by implementing activities in multiple programmatic areas including, but not limited to: family planning, education, income generation, natural resource management, gender, sanitation, or agriculture.

7. Training Target Audience This training course and its modules are most useful for: • Organizations that are familiar with PHE programs and the PHE approach • Staff who are involved with monitoring and evaluation in an integrated programming context • Managers and staff who design, plan, and implement PHE programs • Consultants and trainers of technical staff in integrated PHE programs

8. Training Objectives • To increase understanding of the basic concepts and practical approaches for performance monitoring and evaluation for integrated population, health and environment programs • To learn state-of-the-art tools and techniques used specifically to monitor and evaluate population, health and environment programs • To gain hands-on experience in designing components of monitoring and evaluation plans for population, health and environment programs

9. Training Learning Methods • Lectures • In-class exercises • Small group work • Group presentations

10. LAYING THE FOUNDATION

11. What is PHE? DEFINITION: • A community development model that links population, health and environment (PHE) sectors and supports cross-sectoral collaboration and coordination, particularly in biodiversity-rich areas (BALANCED: http://balanced.crc.uri.edu/) • Within one or more communities, these programs actively link environmental activities with activities that seek to improve reproductive health, always including, but not limited to, provision of family planning services (Robert Engelman, Worldwatch Institute)

12. What Does PHE Stand for? P = Reproductive health, family planning H = Water, sanitation, malaria, vaccines, HIV/AIDS, MCH E = Natural resource management, biodiversity conservation, fisheries management, eco-agriculture Amenable to other sectors +E = Education (non-formal & formal) Energy (fuel-efficient smokeless stoves, biogas) Eco-tourism, sustainable tourism development Enterprise/livelihood development/integrated development

13. Why Integrate? Social and environmental factors linked to food insecurity in coastal Philippines

14. Reasons for Integration • Integrated lives • Address root causes • Enhance relationship • Access different communities • Economic efficiencies • Build on existing frameworks • Broader community participation • Value added from integration

15. Synergy from Combination • Social systems & ecosystems are interrelated and linked • Whatever effects one system, affects the other • Small improvements in either system reinforce each other • To turn around both systems from degradation to health

16. Types of Integration • Parallel:projects conducted in a single area without coordination among organizations • Coordinated:projects conducted in the same area with some level of coordination • Cross-sectoral:projects involving several sectors that are conceptually linked • Integrated:projects involving several sectors that are conceptually and operationally linked

17. Achieving Integration • Conceptual Integration • Field Integration • Community Involvement • PHE Partnerships • Mobilizing Resources • Planning Ahead for Sustainability

18. “Valuing” Gender as Integral to PHE Consideration of gender begins with attention to two central questions: • How will gender relations (relationships between men and women) affect the achievement of sustainable results? • How will my program affect, influence, or change the relative status of men and women? (Will it reduce differences in men’s and women’s status? Will it help create equality between men and women?)

19. Module 2 Frameworks, Part 1Conceptual frameworks

20. Module Outline • Introduction to frameworks • Conceptual frameworks • Including goals and objectives • Logic models • Indicator matrixes • Exercises developing frameworks • Linking frameworks and M&E planning • Work on group project

21. Brief Summary of Three Frameworks

22. Brief Summary of Three Frameworks

23. Module Learning Objectives At the end of the session, participants will be able to: • Explain why conceptual frameworks are helpful for PHE programs • Draw a conceptual framework for a specific program • Design goals and objectives for specific intervention programs to match the conceptual framework

24. Conceptual Frameworks Definition: Diagram that identifies and illustrates the relationships between all relevant systemic, organizational, individual, or other salient factors that may influence program/project operation and the successful achievement of program or project goals. Purpose for M&E: • To show where the program fits into wider context • To clarify assumptions about causal relationships • To suggest causal pathways

25. Why Are Conceptual Frameworks Useful for M&E of PHE? • To demonstrate knowledge or understanding of the problematic context – situation analysis. • To show the linkages between PHE program areas, demonstrating the synergies between integrated problems and integrated solutions. • To draw or explain relationships between the problems that a program/project addresses and the external context (environmental factors), demonstrating ways that program activities may affect knowledge, attitudes, skills, and behaviors of the target population. • To help identify what evaluation information might be useful to measure program success.

26. Causes of Malnutrition in Society Child malnutrition, death and disability Outcomes Immediate causes – individual level Inadequate Disease dietary intake Insufficient access to food Underlying causes at household/ family level Inadequate maternal & child care practices Poor water/sanitation & inadequate health services Quantity & quality of actual resources –human, economic and organizational – and the way they are controlled Macro-level causes at societal level Potential resources: environment, technology, people Source: UNICEF, State of the World’s Children, 1998.

27. Example of a conceptual model for the IPOPCORM project in the Philippines

28. Result Chain 1: FP intervention to achieve a conservation outcome Community Based Distribution (CBD) of Contraceptives Increase Access to FP Info and Products Decrease Fertility Fewer Children Couples Have More Time for CRM Activities Habitats Protected Improved Conservation of the Marine Environment Intervention: Community based distribution (CBD) of contraceptives in which community residents are trained and equipped to promote and distribute contraceptives using social marketing approaches.

29. Activity: Making a Model

30. Brief Presentations & Discussion Presentation of the conceptual frameworks ~ 5-10 minutes per group • Each group presents their framework; • Feedback from other participants? As a large group, let’s discuss: • What is different about conceptual frameworks for integrated development programs? • How can you make the linkages between diverse sectors explicit in your conceptual framework?

31. Integrated Goal and Objectives

32. Philippines: IPOPCORM Program Community-based Coastal Resource Management • Community-based Family Planning • Economic Development (environmentally-friendly) • Policy Advocacy & Community IEC

33. Goals Goal: a broad statement of a desired, long-term outcome of the program. For PHE, it should reflect an integrated outcome. Source: GAP 2003

34. Integrated Goals: IPOPCORM Example To improve food security and quality of life in coastal communities while maintaining the biodiversity and productivity of life-sustaining marine ecosystems 34

35. Hints for Writing Quality Goals GOALS: • Present the desired INTEGRATED outcomes, accomplishments, result or purpose sought (not the process). • Capture broad changes in conditions, answering the “so what” question. • Often goals reflect behavior, attitude, or economic change and show how our activities contribute toward a larger development impact. • Usually goals reflect a result achieved in an intermediate time period (2-5 years). (I want) TO of/among ____________ (action verb) (what) (whom) What behavior/practice, attitude, economic, knowledge, capacity, or access change do you seek?

36. Goals Should Be Specific and Tailored!

37. Objectives Objectives: statements of desired, specific, realistic, and measurable program results. They should be SMART!! Specific: identifies concrete events or actions that will take place. Measurable: quantifies the amount of resources, activity, or change to be expended and achieved. Appropriate: logically relates to the overall problem statement and desired effects of the program. Realistic: Provides a realistic dimension that can be achieved with the available resources and plans for implementation. Time-based: specifies a time within which the objective will be achieved. Source: GAP 2003

38. Hints for Writing Quality Objectives Remember objectives… • Are linked directly to the goals and the intended outputs (information, products, processes, services, or results) you hope to produce to reach the goal. • Reflect what you hope to produce by undertaking a specific activity. For example: You decide you want to train people (the activity), then the objective will relate to the knowledge you increase. • Usually reflect a result achieved in a relatively short time period (0-2 years).

39. To Write an Objective, Fill In the Blanks: TO __________ (action/direction of change) the ________ (what knowledge, attitude, practices will be changed) In _______ (targeted population or area of change) by _______ (the expected amount of change in number or percent) as measured by ______(how will you measure the change) by _______ (time frame) Examples • To increase the number of children in primary school in Addis Ababa by 25% (as measured by school records) by December 30, 2010. • To provide four workshops on peace building methods to 100 community members as measured by field staff workshop reports by December 30, 2008.

40. Integrated Objectives: IPOPCORM Example • Objectives (short term outcomes) • By 2004, improve RH outcomes among people living in coastal areas. • By 2006, build community capacity to implement coastal resource management including alternative livelihood schemes. • By 2007, increase public and policymakers’ awareness and support for integrated PHE approaches. Are the objectives SMART? Specific, Measurable, Appropriate, Realistic, and Time Limited? 40

41. Goals and Objectives • Goal: To improve access to family planning, community health, and natural resource management through integrated development approaches in target communities. • Objective 1: Reduce barriers for access to voluntary FP and RH services in target communities. • Objective 2: Improve family health in priority biodiversity areas in target communities. • Objective 3: Improve community management of natural resources and habitat conservation in target communities. • Objective 4: Document and promote sustainable and successful PHE approaches in target communities through improved monitoring and evaluation.

42. Goals and Objectives: Activity Child malnutrition, death, and disability Outcomes Inadequate Disease dietary intake Immediate causes Insufficient access to food Underlying causes at household/ family level Inadequate maternal & child care practices Poor water/sanitation & inadequate health services Quantity & quality of actual resources – human, economic, and organizational – and the way they are controlled Basic causes at societal level Potential resources: environment, technology, people Source: UNICEF, State of the World’s Children, 1998.

43. Small Group Activity In your groups of 4-6, draw up a simple conceptual framework of your program • This will be the basis of your work for the training • You will present this framework and talk the group through it (Re) Write your goals and objectives • The goal must reflect integration of at least two sectors • At least one population/health objective • At least one environment/conservation/non-health objective Do your goals and objectives match your model? Are your goals and objectives written effectively? SMART?

44. Conceptual Framework, Goal, and Objectives Group Report • Each group should present their conceptual framework, goal, and objectives for the group. • Walk the group through your conceptual framework, showing the links between problems/issues. • Present your goal and objectives. • Did you all manage to integrate PH&E components into your conceptual framework, goals, and objectives?

45. Module 3Introduction to M&E

46. Objectives of this Module To define monitoring and evaluation To review elements of an M&E plan and necessary steps toward constructing the M&E plan To discuss implementation of M&E plans To discuss specific considerations for M&E in the small-scale, integrated development programming context To introduce “A Guide for Monitoring and Evaluating Population-Health-Environment Programs”

47. Monitoring and Evaluation is an Essential Process to: Determine if the program is on track in carrying out planned activities Decide whether the program is having desired outcomes in allsectors (health, population, environment, gender, livelihoods, etc.) Help make informed decisions about needed corrections, new services, use of program resources, etc.

48. Monitoring is the routine tracking of program activities to help program and project managers measure progress. Are we completing activities according to plan? What are the costs? Who is using our services? Evaluation is a specific study that helps determine program achievement. A well-planned, high-quality evaluation can help answer: How well was the project implemented? Were the desired changes achieved? If the change was achieved, to what extent can it be attributed to the project? Intervention Outcome

49. Why Do M&E in PHE programs? To collect information to make informed, evidence-based decisions. To make mid-course adjustments and refine project activities. To demonstrate progress and explain unique PHE-related challenges to stakeholders, funders, & partners. To create records of past and present performance for future assessment and institutional memory. To demonstrate the advantages of working across sectors rather than working in a single sector, showing the value of our integrated approach!

50. Planning for M&E with an M&E Plan Links information from various data sources to decisions that will improve programs. States how the program will measure achievements (ensure accountability). Documents consensus (encourage transparency and responsibility). Guides M&E implementation (standardization and coordination). Preserves institutional memory. Measures success. The M&E plan needs to be adjusted when a program is modified.