Process Monitoring

Process Monitoring Workshop on M&E of Rural Livelihoods Programmes August 12, 2013

Components of M&E IMPACTS Results Measurement / Evaluation OUTCOMES Results Monitoring INTERMEDIATE OUTCOMES Process Monitoring OUTPUTS Progress Monitoring ACTIVITIES Project Inputs / Components Human Resources, Funds, Technology, Policy Sutra Consulting

Monitoring in Development Project Progress Monitoring = Measuring and Tracking Progress of Outputs Results Monitoring = Measuring and Tracking Progress of Results (Outcomes) Process Monitoring = Assessing and Tracking quality of processes (both, in project and within community) that effects Outputs and Outcomes Sutra Consulting

What should be the focus of Process Monitoring? Assess Quality of Project Processes • Are the processes effective to produce quality outputs? • Are the processes effective enough to induce outcome / results? • Are the processes efficient and uses resources (human, funds) optimally? Examines Internal Processes in Community (Institutions, Village) • What Outputs are causing desirable changes in Community(Outcomes)? • Is our Assumptions, Conditions or Risks regarding causal relationship between Output and Outcome holds or are they changing? • What processes in Community (with regard to inclusion and participation, decisions, resource distribution, etc.) are influencing Outcomes? Sutra Consulting

What should be the focus of Process Monitoring? Capturing multi-dimensional nature and process of changes being experienced • What are the social dimension of changes – Inclusion & Participation, Social and Gender Norms, Social Capital and Empowerment • What are the economic dimension of changes – poverty, vulnerability reduction, resource expansion • What changes are being experienced in local governance Sutra Consulting

What more the Process Monitoring can focus on? As a medium for Sensitization and Capacity Building of Institutions • Participative process (Disclosing observations and analysis) acts as a strong means of sensitization • Securing commitment for change through intensive and extended engagement • Facilitating new pilots and processes through community institutions Sutra Consulting

How projects use Process Monitoring? Strategic Decision Making • New Project Policies, Strategies and Guidelines. Ex. Village Organization (VO) certifying that inclusion of all left out poor has been achieved was included as a trigger for fund release in BRLP. BRLP decentralized the funds for VO meetings from DPMU/BPMU to VOs. • Introducing effective project processes. Ex. The social inclusion and piloting of several PIP process led to introduction of a robust Social Assessment Process in TRIPTI. • Changes and fine tuning project processes to be more effective and accessible to community. Ex. Approval of Micro Credit Plan of groups and fund disbursement process was simplified and made shorter in BRLP. Frequency of MCP process was also made need based. • Changes in Process to Improve Outcomes. Ex. Agriculture demonstration process was made more systematic with substantially high handholding support from MACP for improved adoption of technology after Community Score Card (CSC) pilots. • Changes in Community Operational Manual based on field insights to ensure it is a dynamic / living project document. • Adopting Results Oriented Process Approach for Implementation. Ex. OCTMP institutionalized a longer term process oriented approach for capacity building for Water Users Associations. Sutra Consulting

How projects use Process Monitoring? Operational Decision Making • Decisions related to specific communities / villages and institutions where inputs (Training, Hand-holding, Conflict resolution, funds, etc.) from project teams are required are identified and immediately acted upon. Ex. Decisions taken by District and Block Project Teams on a monthly basis in PVP and BRLP. Action Taken Report by project teams in PVP. • Emerging trends are captured and fed in developing priority plans. Ex. Many of the campaign / drive mode plans in BRLP were based on emerging field trends. • Piloting new Approaches / process before being considered as a strategic option. Ex. Piloting inclusion of left-out poor in sample village through VO – In 6 villages 316 left-out-poor households were identified by VOs and 29 new SHGs were formed with 322 members (all 316 HH included) – BRLP June 2009. Sutra Consulting

What is the Methodology? • No standard text book approach • Mixed Method Approach – Qualitative and Quantitative • Mostly Participatory Approaches but Observation, Case Examination and Survey Techniques are used in Combination • What we mostly used • Process Observation • Process Mapping (particularly those within Community) • Focus Group Discussion • Case Interview / Examination • Key Informant Discussion • Community Score Card and Ranking Method • Historical Narratives • Pictorial Tools for self / participatory rating and assessment • Sample Household interview • Satisfaction and Opinion Survey Sutra Consulting

What are the Critical Success Factors? • Focus on processes that are critical for project results and consistent with project cycle • Intensive vs. Extensive approach • Design carefully the Process Monitoring Framework (PVP Illustration) • Setting and agreeing on process quality standards / indicators (to maintain objectivity) • Ex. Project Awareness building process  To what extent the community should know, what they should know, who should know, whether it has elevated interest among community? • Contextualizing process indicators and techniques to project setting • Social and cultural setting • Maintain focus on important project cohorts • Pro Poor, Inclusion and Participation (Equity), Empowerment and Social Capital, Transparency and Governance, Sustainability • Building Capacity of teams in qualitative and participatory approaches – continuous training – eyes and ears on details • Collection and recording of qualitative data is comparatively difficult • Community response differs with techniques used – evaluate and select • Commitment of project leaders and teams for accepting findings contrary to their expectation Sutra Consulting

What gives better Results? • Focused Thematic Studies • Intensive, provides detailed insights • Captures variations across communities • Uses several techniques together • Rapid Process Appraisal • Review of all key project and community processes in samples • Immediate actionable activities for teams • Action taken report as a means of fixing accountability (PVP) • Using Process and Results Monitoring together • Describes results from process perspective • Meaningful approach for process oriented livelihood projects Sutra Consulting

How should project prepare Process Monitoring Plan? Sutra Consulting

PM Framework - Illustration from PVP Participatory Identification of Poor (PIP) Process Quality Indicators • There is no exclusion of households reported including migrating and nomadic households • All households thought to be belonging to poor or very poor by the community is included in the target list of poor and very poor households prepared through PIP • All households who are either tribal or disable or thought to be belonging to vulnerable category by the community is included in separated individual target list prepared through PIP • PIP process has adhered to procedure elaborated in COM based on material evidence • Filled up PIP Cards • Social Mapping charts of each habitation prepared • Habitation wise target individual lists prepared (List of poor and very poor household, list of disabled, tribal list, list of vulnerable) Feedback • Check with groups in different habitations/village whether there are households who have been wrongly classified or missed because they were not present during PIP process • Check with groups in different habitations/village whether there are individuals who have been missed in the individual target list Sutra Consulting

PM Framework - Illustration from PVP Activities related to Participatory Identification of Poor (PIP) Process a) Habitation wise meeting for awareness of PIP exercise • At least one meeting in each habitation conducted by the village Panchayat members • At least 90% of the community had the advance information about when and how the PIP would be conducted b) Formation of PIP Team and Training the PIP team • The team selected were keen to spend the desired time for the process (Find out whether any PIP team member backed out from the process) • The team understood the steps for conducting PIP and facilitating meetings/PRA (Feedback from PIP team) Feedback • The PIP team visited all the habitation for conducting the PIP process • Representation of poor in PIP team • Whether all steps in the PIP process was adhered to before finalising the target list such as social mapping, wealth ranking through three group triangulation, discussion on final list of the habitation • Community acceptability on the final output (project target lists) of the process c) Convening Village assembly meeting • At least one village assembly conducted to provide advance information about when and where PIP would be conducted (Panchayat records/minutes) • At least 50% of the household (both men and women) attended the village assembly meeting in each habitation (Check with groups in different habitations/village whether members participated in village assembly meeting) Sutra Consulting

PM Framework - Illustration from PVP • All household in the habitation received PIP cards and were given hand holding support to fill the socio economic details. (Check with groups in different habitations/village whether complete cards were filled in discussion with the respective household) Feedback • Whether PIP cards were filled for the migrating households and other missing households during the time of PIP cards collection? Whether the migrating household or household absent on the day cards also placed during the mapping exercise d) Transect walk Quality Indicators • At least in habitations located in geographically remote locations adequate time have been spent by PIP team prior to conduct of the PIP process (Check with groups in different habitations/village whether transect walk been done by the PIP team) • Right time, venue and day have been decided in habitations which are geographically remote locations to ensure more than 80% participation of households e) Social Mapping Quality Indicators • In all such habitation identified by project norms social mapping process has been undertaken. • All habitations located in distant geographic locations from main Panchayat have had compulsory social mapping exercise • 100% of the household were represented in the map drawn during the exercise • At least 50% women of the habitation participated in the exercise Sutra Consulting

PM Framework - Illustration from PVP • Social mapping process has adhered to procedure elaborated in COM based on process and material evidence: • Each habitation where social mapping exercise conducted have a separate chart • 100% household mapped in the chart • Symbols used to identify and highlight presence of orphans, widows, disabled, destitute, old age people with chronic illness, nomads in each mapped household • Hard copy of charts and photographs present with VPRC Feedback • Whether social mapping done in an appropriate time when women, PwD, youth from each household could provide extensive time till completion of the exercise • All the details of household PIP cards were validated during the exercise by the group and placed in front of the entire group • Check with groups in different habitations/village whether any eligible household (for project) was missed or not represented and PIP cards were not placed during the exercise f) Wealth Ranking Quality Indicators • The household identified under each category was well documented and displayed in the notice board of each habitation for feedback. • Habitation wise list prepared covering 100% households under poor and very poor category. • Habitation wise list prepared covering 100% of PwD by age, name and gender. Sutra Consulting

PM Framework - Illustration from PVP • 100% of PwD in each habitation went through disability assessment and those certified by medical doctors (for 40% above disability) and by BDFA (for 40% below disability) were targeted by the project (Review of disability assessment workshop minutes in the habitation and the target member list of the habitation) • Habitation wise list prepared covering 100% of tribal households and poor and very poor household among them • The individuals identified under vulnerable category were approved by the habitation members after being displayed • Habitation wise list prepared covering 100% of vulnerable identified during the process. • At least 90% of the habitation know which category of household are targeted for the project and why • 100% of the households who could not attend the wealth ranking process but were identified under the poor and very poor category by the groups in habitation were included under a separate list for inclusion later Feedback • Exclusion of eligible household from poor and very poor family from being included under project target list • Absentee households and vulnerable included under the project target list • PwD having no medical certificate or BDFA certification are included under the project • Sufficient time given to reach consensus on the categorisation of poverty dimension of the household • Sufficient time given to receive feedback on the list and finalise the targeted household list • Majority household consensus matched about the socio economic categorisation done for poor/very poor/middle/rich in two groups during the exercise in each habitation. (Feedback to see what method was adopted to reach the consensus on the definition, whether voting or in writing in the two groups) Sutra Consulting

PM Framework - Illustration from PVP g) Data validation for ensuring quality Quality Indicator • At least 95% household were found to have valid information when sample checks done by PFT and zonal APM (Feedback from PFT members and sample check reports) Feedback • Community at the habitation level agree to the validity of the project target list? If No, reasons of disagreement and reasons for not resolving the conflicts in gram sabha h) Documentation of the data • 100% of the households identified in the habitation level PIP list are included in the draft PIP list of the Panchayat i) Displaying the target list • Majority of the members in gram sabha approve the PIP draft list and the priority list prepared for members eligible to receive grants j) Resolving conflicts in PIP list • Eligible members from list of left out members from each habitation are included in the revised list of PIP k) Approval of PIP list in Grama Sabha Quality Indicators • 50% of the household of the total village attend Gram Sabha meeting, • 2/3rd of the members attended grama sabha are from the PIP list • Handing over the approved target list and documents to the Village Poverty Reduction Committee Feedback • What are the reasons that the eligible members from list of left out household from each habitation during PIP process, not interested to participate in the project? Sutra Consulting

Illustration from BRLP Sutra Consulting

Illustration from TRIPTI Sutra Consulting

Thank you Sutra Consulting

Process Monitoring