Evaluating the Governance of Global and Regional Partnership Programs (GRPPs)

1. Evaluating the Governance of Global and Regional Partnership Programs (GRPPs) Anna Aghumian and Chris Gerrard IEG World Bank November 13, 2009

2. Main Messages • Assessing legitimacy and effectiveness of governance and management is essential since shared governance is a key characteristic of all GRPPs • As international public sector organizations, GRPPs should be expected to comply with generally accepted principles of public sector governance • Understanding how governance is actually practiced requires more than just a cursory examination of a program’s charter, organizational chart and TORs

3. Prevailing Governance Models among the 60 Programs Reviewed

4. Special Features of GRPPs in Relation to Governance & Management • Often have complex governance and management structures • Need to establish their legitimacyon a basis other than shareholder rights • Often have a long chain of accountability — from global to local • Have a global community clientele, making transparency in planning and implementation particularly important • Often housed in existing international organizations

5. Assessing Governance & Management – Suggested Criteria • Legitimacy – in the exercise of authority in relation to those with a legitimate interest in the program • Efficiency – governance & management structures facilitate efficient allocation and use of resources • Accountability – up and down the internal chain of command and control • Responsibility – to stakeholders outside the internal chain of command and control • Transparency – in relation to decision-making, reporting, and evaluation • Fairness – equal opportunity for partners and participants, similarly situated, to influence and benefit from the program

6. Governance & Management: Suggested Approach • Start from a clear understanding of the G& M arrangements and processes: • The extent to which these are well articulated and working well to bring about legitimate and effective governance and management of the program • Host arrangements, if any • Primary focus on governance. Focus on those aspects of management that most directly affect program performance • Build upon and add to the assessments of relevance, efficacy, and efficiency of the Program

7. Evaluating Governance and Management of GRPPs: Tools and Instruments • Desk review of key founding documents • Interviews with key partners and other stakeholders. Good practice evaluations use interview protocols, semi- structured surveys (e.g. GAVI’s interview guide for Board members; GDN’s Board Survey questionnaire) • Surveys of members of the governing bodies, wider circle of stakeholders, beneficiaries • Other : review of meeting minutes of the governing, executive, and advisory bodies; Board meeting attendance rate • Good practice evaluations use a mix of evaluation instruments

8. Extent to Which 60 Evaluations Assessed Different Dimensions of G &M

9. Good Practice Examples: Legitimacy Global Fund to Fight AIDS, Tuberculosis and Malaria • Evaluation question : To what extent the Board is representative and to what extent its decision making is in accord with the GF’s founding principles • Methodology: Board interviews and stakeholders survey, Board meeting minutes reviews, assessment of Board and committee meeting attendance rates • Findings: • Board is formally representative. Yet, some constituencies are not participating effectively and do not have equal voice • Poor communication with beneficiaries, linguistic barriers, lack of adequate financial resources for beneficiary representation • Recommendation: The Board should improve the quality of representation by enhancing communication with all constituencies and by favorably considering proposals for assistance from constituencies with limited resources

10. Efficiency Association for the Development of Education in Africa (ADEA) • Evaluation question: Are the ADEA structures adequate and functioning properly in light of their assigned tasks and available resources? • Methodology: Organizational assessment based on interviews with stakeholders and staff, on-line survey, desk review of documents and field visits • Findings: • The Steering Committee is overburdened with the management of the program, and no time is left for considering strategic issues. • “Decision-by-consensus” model, while provides equal access to decision making to all members of the SC, slows down its responsiveness • Host arrangements with IIEP reduces the efficiency of G& M of ADEA • Recommendations: The SC should revise organizational structure of ADEA, improve division of the roles and responsibilities between the SC and the Secretariat, conduct a cost-benefit analysis of the host arrangements with IIEP

11. Accountability Global Alliance for Vaccines and Immunization (GAVI) • Evaluation questions: How well has the GAVI Fund governance structure worked? Is there clarity of role/responsibilities between various entities? • Methodology: Interviews with key informants, desk review of documents, prior assessments of GAVI’s governance • Findings: • Lack of accountability due to separation of programmatic and fiduciary responsibilities between GAVI Alliance and GAVI Fund • Unclear and weak accountability chain within each of the governing bodies (vertical accountability) • Unclear view among partners regarding their respective roles and responsibilities • Recommendation: Drastic changes in governance structure

12. Responsibility Medicines for Malaria Venture • Evaluation question: The extent to which the program accepts and exercises responsibility to stakeholders who are not directly involved in governance • Methodology: Desk review of documents, interviews • Findings: • MMV hasincreased engagement of researchers and research institutions in endemic countries • Has held key meetings in countries where malaria is widespread and MMV-sponsored research is underway • Has included a majority of beneficiary country members on its Access and Delivery Advisory Committee • Recommendation: To engage more advocacy NGOs in the design and execution of MMV’s access and delivery work program.

13. Transparency Partnership for Maternal, Newborn and Child Health • Evaluation question: Is there adequate transparency in governance? • Methodology: Systematic review of the program’s website • Findings: • Board meeting minutes are accessible on the website, but other important background material is not provided • Financial information is not fully available • Work plans are not available on the website, leading to potential duplication of activities by partners • Recommendations: Develop a Board-approved disclosure policy

14. Fairness Cities Alliance • Evaluation question: The extent to which participants similarly situated have equal opportunity to receive benefits from the program • Methodology: Desk review of documents, interviews • Findings: • All grant applications have to be sponsored by a board member and about 90% of grant applications are approved • Some kind of screening or pre-selection process was taking place which was not transparent and potentially unfair to potential recipients • Recommendation: To make the grant management criteria and process more transparent

15. Host Arrangements International Land Coalition • Evaluation question: The extent to which the relationship between ILC and IFAD, as host and international focal point, is mutually beneficial? • Methodology: Desk review, interviews, survey of partners • Findings: • There are gaps and conflicting clauses in the legal and administrative agreements regulating IFAD-ILC relations • Dominant role of IFAD reduces the independence of ILC and the incentives of other partners to participate effectively in the program • Host arrangements contribute to ILC efficiency in the short term, but hinder its financial sustainability in the long term. • Recommendation: Coalition Council should prepare a strategy for transition from an IFAD-hosted institution to an independent legal entity with international status.

16. Conclusions • Most evaluations assess some aspects of G&M, such as accountability, but few assess all aspects • Using a consistent approach that focuses on compliance with generally excepted principles of public sector governance facilitates comparison across programs • Evaluators are developing innovative ways to apply this approach and find out how governance is actually being practiced in individual GRPPs