March 20-25, 2006

1. Second Training of Trainers Workshop on the Community Assessment Process Methodology for the Gemi Diriya and VISHLI Programs March 20-25, 2006

2. WORKSHOP INTRODUCTION TRAINING OBJECTIVES: • To understand the processes and steps involved in conducting a Community Assessment Process • To understand how the Community Assessment Process is used as a social accountability tool in GEMI DIRIYA and VISHLI projects • To adapt the Community Assessment Process Methodology in the project

3. What are your expectations?

4. Session 1: Transparency and Accountability- Group Exercise I • Objective: • to enable participants to internalize the concepts of Wagakeem (which means responsibility) versus Wagaveema (which means accountability) and • to discuss amongst themselves the various roles and responsibilities they have in the project as well as the accountability tools they use to enhance their work. • At the end of the exercise, the participants will be able to define social accountability within the context of the project, and the social accountability tools, roles and relationship which are being used, distinguish between wagakeem and wagaveema.

5. SOCIAL ACCOUNTABILITY FOR THE DIFFERENT PROJECT LEVELS: (Current Practices and Relationships) Group exercise 1: 1) What is social accountability in the context of the Gemi Diriya? 2) What are the roles and responsibilities that you have in the Gemi Diriya Project? 3) Who is a service provider in the Gemi Diriya Project? What are the social accountability relationships in the Gemi Diriya project? Can you build a diagram to show the relationships? 4) What are the social accountability? mechanisms/tools/activities used and their benefits?

6. Introduction to Social Accountability and Good Governance The ultimate desirable objective of the project would be to usher in good governance, with all its characteristics features including participation, following the rules and laws, accountability, transparency, and responsiveness.

7. What is Accountability? Accountability is the obligation of those in power to account for or take responsibility for their actions. Those in power are in the Gemi Diriya project - village organizations and their board of directors, technical service providers, Pradeshiya Saba, Community professionals, support organizations, etc.) The power-holders can be held accountable for: • conduct (obey the rules, and be responsive, empowering, and transparent in the way in the way they implement their roles and responsibilities within the project) and • Performance (do they do good quality work, are they respectful and responsive, on time, willing to share information, etc.)

8. What is Social Accountability? • Social Accountability can be defined as an approach towards building accountability in which all stakeholders in the Gemi Diriya project participate directly or indirectly in exacting the performance and conduct of those in power for their policies, actions, decisions, and use of funds. • SA mechanisms in the Gemi Diriya project include many actions and tools (i.e. golden rules, social audit and financial committees, etc.) that all stakeholder use to hold those in power accountable, responsive and transparent. • SA mechanisms can be initiated and supported by all stakeholders in the Gemi Diriya project.

9. Vertical Accountability What are vertical accountability relationships in the Gemi Diriya Project? Horizontal Accountability What are horizontal relationships in the Gemi Diriya Proejct? Vertical vs. Horizontal Accountability

10. SESSION 2: The Community Assessment Process Session Objectives: Enable participants understand the why, benefits and risks of use of the community assessment process tool in the Gemi Diriya Project Content/Description of Session • Why the need for Community Assessment Process Tool (CAP) in the Gemi Diriya Project • Characteristics of CAP Methodology • Six Components of the CAP • Input Tracking Scorecard • Performance Scorecard • Self-evaluation • Preparatory Ground Work and Initiating Community Gathering • Interface Meeting and Action Plan • Institutionalization

11. THE COMMUNITY ASSESSMENT PROCESS What is a Community Assessment Process (CAP) • This is an instrument or tool for the all stakeholders of the Gemi Diriya to ensure accountability, transparency and responsiveness from service providers at the local level. CAP Context and Rationale • The Gemi Diriya is a community driven project where control over planning decisions and investment resources are the responsibility of community groups and local government. It is designed to promote and support setting up of accountability mechanisms and systems. • Among the many Sac tools being used in the Gemi Diriya, the CAP is one to ensure the accountability of the many stakeholders. It will help to promote improvement of services and empowerment of the beneficiaries of the Gemi Diriya project by ensuring that all stakeholders can influence directly the improvements of the services that are delivered.

12. The Community Assessment Process Ultimate Objectives of the CAP: Measure the quality of services for improvement by scoring or rating them Make servicesand important issues related to service delivery transparent through information sharing for the all stakeholders Ensure inclusion of all groups and ensure unheard voices are heard Build trust, communication and partnerships between all stakeholders of the Gemi Diriya project Generate solutions collectively and implement them jointly Critical Success Factors for CAP: Strong skills to facilitate process, Strong information and dissemination efforts to ensure maximum participation from the all stakeholders, Strong social mobilization process, and Institutionalizing the CAP into the project for better monitoring and management of the project.

13. Characteristics of CAP • Uses the community or service as the unit of analysis • Focused on the monitoring and assessing at the local level • Generates information through focus group interactions • Enables maximum participation of all stakeholders in the Gemi Diriya Project • Provides immediate feedback to service providers • Emphasizes joint decision-making between all • Solutions are arrived at through mutual dialogue • Allows for increased social responsibility of all stakeholders of the Gemi Diriya project, as well as the inclusion of different groups, promoting empowerment and good governance.

14. STAGES OF CONDUCTING THE COMMUNTY ASSESSMENT PROCESS The community scorecard process consists of six key stages • Developing an Input Tracking Matrix • Community-Generated Scoring of Performance • Self-Evaluation by Service Providers and • Interface Meeting and Action Planning • Preparatory Groundwork and Organization of the Community Gathering • Institutionalization

16. Session 3: Input Tracking Group Exercise 2 • Exercise Objective: to understand what an input is, the importance of input tracking and how it is relevant to the Gemi Diriya Project. • Exercise Result: able to identify the types of inputs needed for a project, and to develop indicators for measuring the inputs. • Role Play: Hand Out 1: Participants will act out this role play in small groups and will come up with a list of inputs that should have been in place in order for the system to operate effectively and develop indicators to monitor, measure and assess the inputs. The participants will then convene into a plenary discussion.

17. Input Tracking Why do we use it? • The Input Tracking form provides information on the status of inputs in the facility/project/service and shows whether it has what it needs (inputs) to deliver and operate as planned. How is it done? • A discussion is facilitated in the staff or those responsible for and knowledgeable about the facility, service or project to get information (.i.e, inventory of equipment, receipts, budget allocation and expenditure reports, delivery invoices, transect walk) on what is supposed to be there and what is there.

18. STEPS FOR INPUT TRACKING/MONITORING Group Exercise 3 1) In the large group, we will first define what is an indicator. 2) Then divide into small groups: pick one person to take notes and another to report back to the large group and follow the steps below: Step-1: Decide And Explain Which Inputs Are To Be Monitored • The first step in input monitoring is to decide what services, facility or project is going to be monitored and give the reason why.

19. STEPS FOR INPUT TRACKING/MONITORING Step-2: Get Information on what the inputs were planned. • In order to monitor the inputs of a project, facility or service, information is needed to be informed about what the inputs should be there or what was planned and what is actually there. To do this, what kind of information do you need and how do you get it? • Knowing what is supposed to be in the agency/facility/service/project is in itself a source of empowerment for the community, and enables them to decide upon input indicators more easily. • Without knowing what was supposed to be there or planned, one can’t compare actual inputs or what is there with what was supposed to be there!

20. INPUT TRACKING: STEPS Step-3: Develop Appropriate Input Indicators • The group then decides on the input indicators that will be monitored to determine the difference between the planned inputs and actual inputs. The indicators will depend on the project, facility or service that is being assessed. • The indicators should be easy to understand to all stakeholders, and be focused on inputs where they are difference between planned and actuals inputs which affects the service delivery performance • Note that everything cannot be monitored so you need to prioritise based on importance

21. Step 3 - Example of Input Tracking Indicators from the Livelihoods Group 1. Savings and credit association • Awareness creation • Percentage of members • Increase in the number of groups • Savings amount • Amount of loans granted • Percentage of repayments 2. Funds • The proportion of funds received from the project • Amount of savings • Interest accrued 3. Technical Advice • The number of institutes involved • Number of times monitored • Decline in the number of defaulters • Presence of resource persons on the subject, in the village itself 4. Exposure visits • Evaluation of external persons • Number of exposure visits and number of persons involved • Number of experiences gained and put into use in the village. 5. Material inputs • Every member possesses a pass book as planned • Issue of receipts • Duly updated register of documents • An office with equipment • Security of funds ensure by the use of technology • Information technology introduced to the village.

22. INPUT TRACKING: STEPS • Step-4: Fill in the Input Tracking Matrix The facilitators ask for and records on the matrix the data on each of the inputs that have been decided upon. Information should be backed-up by solid concrete evidence (receipt, account, loan register or delivery notes, etc.) • Step-5: Recording Data. Record the planned and actual inputs and note on the matrix the data or information given for evidence on the inputs. The information or data generated belongs to the community and should be left with them. Facilitation team should make and keep a copy. • Step-6: (Optional) Inspection of Physical Project Output or Inputs In a physical infrastructure project, the last stage must be an inspection of the project output to see if it is completed and is of adequate quality. This could be done by walking around the site, taking pictures or drawing pictures. This is done to provide first hand evidence about project and service delivery.

23. INPUT TRACKING: RECORDING FORM

24. SP 1 (VO) SP 2 (Hub) SP 3 (SO) INPUT INDICATOR Original Score After 6 mths Original Score After 6 mths Original Score After 6 Mths A. Services Delivered 5 7 7 9 6 7 1 2 2 2 2 2 2 2 0 2 3 4 1 2 B. Poor Ok Good Good Ok Ok C. 20 25 60 60 30 40 Sample Format ofRepeat Input Tracking Matrix

25. SESSION 4: COMMUNITY-GENERATED ASSESSMENT • Session Objectives: • Participants understand the steps for facilitating community generated assessment and • Understand the rationale of the community assessment for performance monitoring • Content/description of session: • Steps in conducting a community-generated assessment exercise • Gathering of information • Dialogue on outcomes and • Repeat matrices/follow up actions to monitor progress • Understand that result of the Community-Generated Assessment will be: • Completed Community Generated Assessment (to be left with the community) • Community reflection on performance of service providers • Provider proposals for change

26. COMMUNITY GENERATED ASSESSMENT What is the Community Generated Assessment? Community-generated assessment is when project beneficiaries monitor or assess the performance of a service/facility/project from their own perspectives and experience. They do this by: • Identifying issues to assess • Identifying indicators • scoring the indicators based on their own perceptions • Suggestions for changes to improve performance and/or conduct.

27. COMMUNITY GENERATED ASSESSMENT Exercise 4 • Divide into small groups: pick one person to take notes and another to report back to the large group and follow the steps below: Step-1: Divide into Focus Groups Based on Usage • As with the input tracking the participants need to be classified into focus groups based on usage of the service being evaluated. • This will ensure that there are a significant number of users in each of the focus groups because without this critical mass, the usefulness of the data is limited. • Each group should further have mix of members based on age, gender, and occupation for productive discussions. Ideally, the initial focus groups formed for input tracking can be used for the community generated scoring process.

28. COMMUNITY GENERATED ASSESSMENT Step-2: Develop Adequate Performance Criteria Each of the focus groups needs to brainstorm about the service/program/facility to come up with a set of measures/indicators to assess performance. Method for generating group indicators a.Use the following guiding questions to help the group tobrainstorm on indicators to assess the facility or project. •  How will someone know that this facility is operating well? •  How do you determine the performance of the facility (what specifically do you look for)? • List all issues discussed and assist the group to organize the information under broad headings. • If many indicators are identified, help the group to prioritize these to a reasonable number (4). • Sufficient time must be given to the groups for this stage.

29. COMMUNITY GENERATED ASSESSMENT Step-3: Different Methods for Scoring of Indicators • Different methodologies can be adopted to score/assess performance:(1-5, 1-10, etc.). • Whichever methodology is adopted, the team must ensure that it: • Helps achieve consensus • Can be done with resources involved • Is meaningful/user friendly • Offers equal opportunity to all • One scoring methodology sometimes used is facial expressions/symbols, to assess each indicator.

30. COMMUNITY GENERATED ASSESSMENT

31. COMMUNITY GENERATED ASSESSMENT Step 4: Assessing performancethrough ranking or scores • After explaining the type of scoring to be used and the procedure to the group, draw the recording format on a flip chart. • Then ask each participant to score for a given indicator by placing one mark (for flip chart users) in the column that he/she feels she/he rates the performance. Note to facilitators: • Scoring should be done one indicator at a time. Participants should score on one indicator and never be asked to score on all the indicators at once. • It is best to work through the scoring procedure on a “practice indicator” to ensure that the participants understand and are comfortable with the procedure before scoring on any actual indicator. • Facilitators should guide and help participants to score, but should avoid influencing the process.

32. COMMUNITY GENERATED ASSESSMENT Step 5 - Recording reasons for scores and proposals for change • After scoring all the indicators and national benchmarks, look at the results of each indicator and discuss it. Ask for the reasons why they have scored in that manner and record them together with any anecdotal evidence in the remarks column as shown in the format below. • Guiding questions to use include: • Why did you give this rating? • What is responsible/what is the problem? • What can be done to improve the situation?

33. COMMUNITY GENERATED ASSESSMENT Step 6. The Community Suggestions for Change • The process of seeking user perceptions alone would not be fully productive without asking the community to come up with its own set of suggestions as to how things can be improved based on the performance criteria they came up with. • At this stage facilitating team should ask a number of questions, which would help interviewers to think about suggestion of concrete reforms. For example: • What can be done now to improve the situation? • What role can different stakeholders have in the implementation process of the reforms? • What support is needed by them to implement the reforms? • Where from such support can be obtained? • How long it would take to improve the situation?

34. RECORDING FORM FOR COMMUNITY GENERATED ASSESSMENT

35. SESSION 4: SERVICE PROVIDER SELF ASSESSMENT • Session Objectives: 1. Understand the rationale and use of Provider Self Assessment in performance monitoring 2. Participants understand the steps for a Provider Self Assessment • Session Content and Description • Steps in conducting assessment • Collation of data/information gathered on results of service provider self assessment • Dialogue on results and • Why the need to repeat the exercise by recording in the forms periodically to follow up actions and monitor progress.

36. SERVICE PROVIDER SELF ASSESSMENT What is the service provider self assessment? • It elicits the perspective on performance from the service providers themselves. • The process is similar to the community generated scoring. Method used: Facilitated brainstorming to generate and scoreperformance indicators in small groups of service providers only.

37. SERVICE PROVIDER SELF ASSESSMENT Group Exercise 5: As a service provider: Step 1: • Brainstorm issues of performance and select the 2 – 3 most important indicators of your performance. Step 2: • Agree on how you want to score Step 3: • Score the indicators, using a formatted flip chart to record the information

38. SERVICE PROVIDER SELF ASSESSMENT Step 4 - Reflection and Explanation of High/Low Scores • The service providers should reflect on why they gave the scores they did, and to also come up with their own set of suggestions for improving of service delivery. • What incentives can improve the situation? What are the constraints they face and other problems. Is it a issue of needing the right incentives.

39. SERVICE PROVIDER SELF ASSESSMENT Step 5: Summarising scores • Summarize the service provider’s scores; add the scores • A summary table of the outputs of each focus group should be prepared for presentation at the interface meeting. • The format for summarising data is shown in the next slide

40. Example: Community Professionals – Self Assessment

41. RECORDING FORMAT OF THE SERVICE PROVIDER SELF-ASSESSMENT

42. SERVICE PROVIDER SELF ASSESSMENT • Outcomes of the Service Provider Self Assessment • Completed Provider Self-Evaluation charts (to be left with the community/facility) • Provider reflection on own performance • Provider proposals for change

43. SESSION 5: The Interface Meeting, Information Sharing & Dissemination Session objectives: • Understand the rationale for an interface meeting and the outputs of an interface meeting (developing Community Action plans) • Participants understand the steps for convening an Interface Meeting Content/description of session • Steps in conducting meeting and presentation of outcomes/results from the Input Tracking, Provider Self Assessment and Community Generated Assessment • Analysis and presentation of data/information • Dialogue on results and agreement on solutions/action agenda; developing a community action plan • Repeat meetings and follow up actions to monitor progress of implementation of the action plan

44. The Interface Meeting • This stage in the process holds the key to ensuring that community feedback is taken into account and that concrete measures are taken to improve the service/project/facility. • It is a meeting between services providers and the community to provide feedback from the community and service provider self-evaluation score cards and generate mutually agreed on changes and action planning on the recommendations that the community and the providers have made independently • Step-1: Preparing Both Parties for Meeting • Both the community and providers need to be prepared for the interface meeting and should be sensitized about the feelings and constraints of the other side. This ensures that the dialogue a positive discussion, and that a relationship of mutual understanding of trust and partnership is built between community and provider. Sensitization can be done through orientation sessions with both sides, and through sharing the results of the assessment scores. • Step-2: Ensure Adequate Participation from Both Sides • This will require mobilization at the community level, and arrangements so that service providers are able to get away from their duties and attend the meeting. One can further involve other parties, like local political leaders and senior government officials in the interface meeting to act as mediators, which may give it greater legitimacy.

45. The Interface Meeting Step-3: Facilitate Productive Dialogue & Development for Change • - Once all groups gather, facilitators work with the community and the service providers and help them come up with a list of concrete changes that can implemented immediately. • - Strong facilitation is required to maintain a positive and constructive tone throughout. Negative comments should be acknowledged • - At all times, the focus should be on joint searching for constructive solutions to identified problems. This will give credence to the entire process from both the community and provider’s perspectives. • - Senior government officials and/or politicians present can clarify issues and also endorse the reforms. Outcomes: • Downward accountability for service providers • Empowerment of service users • Enhanced transparency • Enhanced sensitivity of users to providers’ constraints • Evidence of service performance • Agreements on a change agenda to improve quality of service delivery/a facility/project.

46. After the Interface Meeting • Continued monitoring and follow-up • CAP is therefore a process that does not stop at the ratings but is continued series of communication and interaction between the community and providers to put in place the mutually agreed on changes and plans. • Emphasis on joint planning, sharing of key information on budgets, inputs, etc.

47. COMMUNITY ACTION PLAN

48. Session 6: Steps for Preparing the Groundwork • Step-1: Identifying the Scope of the Assessment • Decide on the geographical scope and location for each exercise. Ideally this should be a village. Last workshop three villages were chosen: Pahalaellawewa, Nissankamallapura, Atumalpitiya. • Deciding what facilities and services are to be evaluated: (i.e. capacity building, infrastructure, Village Saving and Credit Society; etc.)

49. Steps for Preparing the Groundwork Step–2:Identifying and Training of Facilitators The CAP depends on the quality of the facilitation and mobilization undertaken. Ideally, people or groups with experience in facilitating participatory methods should be engaged for the task. These facilitators need to be trained on the CAP process and how to organize the exercise. Step-3: Involve Other Partners The involvement of traditional leaders, members of local governments, workers at the service facilities in the area, community volunteers, and staff from NGOs in each of the villages is also important.

50. Steps for Preparing the Groundwork Step 4 - Divide into groups by use of service -Who uses what services, how much, and who (women, poor, disabled.) - This initial division can be done through field visits and informal interviews by the facilitating team Step 5 - Mobilize community (generate awareness, reach all segments). As the process is done via a community meeting, ensure that there isbroad participation from all parts of the community. The meeting must be preceded by full-scale mobilization of the community through an advocacy/awareness campaign that informs people about the purpose and benefits of the exercise. Participation of a large segment of the community in the process is the first step towards success. The organization of the meeting involves decisions about logistics including: Deciding the venue for the gathering based on a sense of the number of participants that will take part. Ensuring materials for the gathering – paper, pencils, megaphone/PA system (optional), blackboard (optional), etc.