An Analysis of Sector Level Quality Initiatives:

1. An Analysis of Sector Level Quality Initiatives: Identifying Common Lessons to Inform Bond’s Effectiveness Framework Robert Lloyd

2. Purpose of the study • Identify the individual and collective lessons learnt by eleven initiatives that have developed sector wide quality / effectiveness frameworks • Explored five common themes: • Design process • Framework structure • Incentives for Implementation • Compliance systems • Support infrastructure • Based on the lessons: • Recommendations to Bond for moving the Effectiveness Program forward • Propose a possible approach for an effectiveness framework and tentative suggestions about its content.

4. Common themes that emerged from the research: design process • Clarity about why a framework is being developed and for whom? • AFCID’s Code of Conduct was developed in response to external pressure and as a result took the form of verifiable standards • NCVO’s CAP emerged in response to a capacity gap in the sector and therefore adopted a flexible framework of tools that organisations pick from • A framework needs to have a niche among similar initiatives • PQASSO - no quality assurance mechanism for small NGOs • ISEAL - growing interest in standard setters by government • Consultation with users helps generate by in • SPHERE - consultation was just as important as the outcome • HAP & ACFID - consult widely, but have a core group that leads • Caution that consensus decision making doesn’t push a framework to its lowest common denominator • ACFID - need to balance listening and leading

5. Common themes that emerged from the research: framework structure • Taking an integrated view of effectiveness • PQASSO & EFQM - pillars of effectiveness cover all aspects of an organisation and each is equally important • Makes a framework more daunting, but provides a complete and realistic picture of what needs to be done • People in Aid - support in prioritizing areas for improvement is key • Identifying high level principles • ISEAL - provide the foundations upon which the rest of the framework is built • EFQM - tool for engaging senior management • A common approach which enables organisations to ‘see themselves’ in the framework • EFQM - general questions and then work with organisations to adapt language and concepts • SPHERE & SGS NGO benchmark - common / thematic standards

6. Common themes that emerged from the research: framework implementation • Challenge with donors backing a framework • While it can lead to rapid up take of a framework, it should not be the primary driver • Can lead to an approach that emphasises minimum standards rather than continuous improvement • Importance of ongoing support, advise and guidance • PQASSO & HAP - development of a framework is only first step along the path towards effectiveness. Support in using the framework is key • Type of support will depend on how advanced an organisation’s engagement with effectiveness is • ISEAL - this can put a strain on the host organisation - outsourcing capacity building • Creating spaces for peer learning and sharing • HAP & DEC - Agencies that apply the framework and benefit from it can become the best advocates of the approach among peers

7. A proposed approach for an effectiveness framework: key considerations • There are already a wide range of existing tools, approaches and frameworks that address the issue of quality / effectiveness The framework should not replicate what exists, but rather help bring coherence, show linkages, sign post and identify gaps • To navigate the framework there will need to be a set of basic questions to help organisations identify gaps and direct them towards tools and approaches which are most appropriate to their needs The framework should have an element of self-assessment build into to it

8. A proposed approach for an effectiveness framework: key considerations • Bond members are involved in diverse activities and are at different stages of their conversation on effectiveness The frameworkshould be flexible enough to allow organisations to see themselves in the tool and find elements that are useful to improving effectiveness within their own agencies • There is growing pressure on the UK NGO sector to demonstrate effectiveness, but it is not facing a crisis of confidence or public trust, the primary driver for the tool should therefore be to support organisations in developing their own approaches to effectiveness The framework should be an ‘enabling’ rather than a ‘compliance based’ tool

9. Principles of development effectiveness • Continually learning & improving • Demonstrating & managing by results • Respect for diversity & human rights • Accountability to communities • Financial & political independence • Local ownership & partnership

10. Principles of development effectiveness • Continually learning & improving • Demonstrating & managing by results • Respect for diversity & human rights • Accountability to communities • Financial & political independence • Local ownership & partnership Common levers of organisational effectiveness Leadership and management Governance and strategy Financial management People management Communications Relationships and alliances Learning & knowledge management User / client focus

11. Principles of development effectiveness • Continually learning & improving • Demonstrating & managing by results • Respect for diversity & human rights • Accountability to communities • Financial & political independence • Local ownership & partnership Common levers of organisational effectiveness Leadership and management • Self-assessment questions: • Managers identify and respond effectively to changes in the external environment • Managers provide clear leadership motivating and inspiring people by their vision and commitment • Managers drive change throuhgout the organisation • Leadership strengthening tools • =MC Senior Management Team Competency Assessment, Management Center • INTRAC (2006) NGO Leadership Development, Praxis Paper 10 • Existing frameworks • People in Aid, HAP, PQASSO

12. Principles of development effectiveness • Continually learning & improving • Demonstrating & managing by results • Respect for diversity & human rights • Accountability to communities • Financial & political independence • Local ownership & partnership Common levers of organisational effectiveness Mission specific levers of effectiveness Leadership and management Possible thematic areas: Microfinance Capacity building Development education Service delivery Advocacy/campaigning Policy research Humanitarian relief Governance and strategy Financial management People management Communications Relationships and alliances Learning & knowledge management User / client focus

13. Principles of development effectiveness • Continually learning & improving • Demonstrating & managing by results • Respect for diversity & human rights • Accountability to communities • Financial & political independence • Local ownership & partnership Common levers of organisational effectiveness Mission specific levers of effectiveness Leadership and management Possible thematic areas: Microfinance Capacity building Development education Service delivery Advocacy/campaigning Policy research Humanitarian relief Advocacy and campaigning Governance and strategy • Self-assessment questions • Are primary stakeholder involved in the design of your campaigns? • Is the evidence to substantiate policy positions publicly available? • Tools • Understanding power for social change, Just associates • Tools for Policy impact: a toolkit for researchers, ODI • Existing frameworks • NCVO campaign effectiveness program • ODI - Research and Policy in Development, Financial management People management Communications Relationships and alliances Learning & knowledge management User / client focus

14. Principles of development effectiveness • Continually learning & improving • Demonstrating & managing by results • Respect for diversity & human rights • Accountability to communities • Financial & political independence • Local ownership & partnership Common levers of organisational effectiveness Mission specific levers of effectiveness Leadership and management Possible thematic areas: Microfinance Capacity building Development education Service delivery Advocacy/campaigning Policy research Humanitarian relief Governance and strategy Financial management People management Communications Relationships and alliances Learning & knowledge management User / client focus Approaches to demonstrating your effectiveness Tools and approaches for measuring effectiveness of different thematic activities Types of evidence for demonstrating change in specific areas of work

15. Principles of development effectiveness • Continually learning & improving • Demonstrating & managing by results • Respect for diversity & human rights • Accountability to communities • Financial & political independence • Local ownership & partnership Common levers of organisational effectiveness Mission specific levers of effectiveness Leadership and management Possible thematic areas: Microfinance Capacity building Development education Service delivery Advocacy/campaigning Policy research Humanitarian relief Governance and strategy Financial management People management Communications Relationships and alliances Learning & knowledge management User / beneficiary focus User / client focus Example: advocacy and campaigning Types of evidence: changes in awareness of issue, improved capacity to communicate and promote advocacy messages, increased visibility of campaign messages Tools: Outcome mapping, composite logic model, spider diagram for capacity building for advocacy, theory of change, policy maker ratings, bellweather methodology

16. Comprehensive effectiveness review Principles of development effectiveness • Continually learning & improving • Demonstrating & managing by results • Respect for diversity & human rights • Accountability to communities • Financial & political independence • Local ownership & partnership Common levers of organisational effectiveness Mission specific levers of effectiveness Leadership and management Possible thematic areas: Microfinance Capacity building Development education Service delivery Advocacy/campaigning Policy research Humanitarian relief Governance and strategy Financial management People management Communications Relationships and alliances Learning & knowledge management User / beneficiary focus Approaches to demonstrating your effectiveness Tools and approaches for measuring effectiveness of different thematic activities Types of evidence for demonstrating change in specific areas of work