Aflatoun Quality assurance and Impact assessment Strategy (AQIS)

1. Aflatoun Quality assurance and Impact assessment Strategy (AQIS) Strategy overview Amsterdam, August 2008

2. Contents of this document • This working document outlines Aflatoun’s Quality Assurance and Impact Assessment strategy. It is a working document and as such under revision of the AQIS committee and the Taskforces for Quality Assurance and Impact Assessment. • The document is divided in five parts; sections 2 – 4 provide the key elements of AQIS: • Strategy overview – Principles and structure • 2. Quality Assurance • 3. Output Tracking & Outcome Assessment • 4. Impact Assessment • 5. Planning

3. 1.1 Strategy overview – Principles and structure As Aflatoun scales-up rapidly to reach its 2010 goals of bringing its Child Social & Financial Education programme to over 75 countries by 2010, it is in need of innovative and strategic approaches towards assuring the quality and impact of its programmes. Our strategy involves a variety of evaluation tools and stakeholders of our programme. AQIS follows three principles: I. Strategic alignment with Aflatoun’s Theory of Change II. Progressive learning through integrated knowledge management of the involved evaluation tools III. Effective involvement of evaluation experts and practitioners The strategy for quality assurance and impact assessment has been designed on the basis of extensive discussions with our implementing partners and international stakeholders at our Regional and International meetings. The current document has included the comments and recommendations done at the first Impact assessment Taskforce meeting in August 2008. The strategy is outlined below along the lines of the three principles given above.

4. 1.2 Strategy overview – Principles and structure I. Strategic alignment with Aflatoun’s Theory of Change Aflatoun’s strategic measurement of quality and impact is aligned with our programme’s Theory of Change (see next page), defined by the Aflatoun Secretariat in conjunction with our Social Return on Investment (SROI) experts, our implementing partners and the impact advisory committee. The elements of AQIS are designed such that they monitor and evaluate the different effects of the Aflatoun educational programme and advocacy. Though the elements are presented as separate categories, they work together naturally and powerfully. Quality means nothing without Impact, and Impact can be dangerous without Quality. The learning of each element is constantly fed back into the activities and research of the other elements, and vice versa. A. Quality Assurance: Change through education starts with high-quality organization and processes. In order to assure the maximum quality of the entire Aflatoun Programme and Advocacy, a range of quality tests at partner and Secretariat level are implemented. B. Output tracking: Aflatoun desires to understand the scope, outreach and activities comprising the Aflatoun Programme. The Secretariat facilitates all partner organizations with tools for output tracking. C. Outcome Assessment (Evaluation): Aflatoun desires to understand the scope, outreach and activities comprising the Aflatoun Programme. The Secretariat facilitates all partner organizations with tools for output tracking. Through quality education, on a considerable scale, Aflatoun sets out to change children’s competencies and related behaviour. Different research tools are being developed to assess these outcomes. D. Impact Assessment:Our mission is our ultimate goal. In order to understand whether we establish this impact, Aflatoun asks leading research institutions using cutting-edge impact assessment methodologies, to look at the impact of (elements of) our programme. Please refer to page 6 for a schematic overview of AQIS.

5. Theory of Change Behaviour Impact S Curriculum Quality & output Expected Competency outcomes Expected behavioral Outcomes v • I know I’m unique! • I like to explore • I am a compassionate member of the world • I confidently participate in different activities in my community I am socially empowered! Core Element 1: Personal Exploration • I defend my and others’ rights • I participate in democratic processes Core Element 2: Rights & Responsibilities • I know/claim my Rights • I care for others & myself • I am a changemaker • I stay in school longer • I have enough resources • I apply my capabilities • I decide when I want to marry • I am happy! • I save & deposit regularly • I have a savings account • I withdraw less • I use resources responsibly • I save regularly • I spend responsibly Core Element 3: Saving & Spending • I make budgets • I set financial goals for myself Core Element 4: Planning & Budgeting • I plan to realize goals • I can make a budget • I run a micro-enterprise • I have organized/ conducted projects for social & green causes Core Element 5: Social & Financial Enterprise • I undertake enterprises • I enterprise for change • I value teamwork v I am economically empowered! Advocacy • More inclusive, participatory education • Structural banking reform • More children’s savings accounts • Global children’s community • Raising global CSFE awareness • Child Friendly Banking • Creating children’s global movement v v

7. AQIS - Structure Impact and Outcome Indicator development (social & financial) • 1. High quality partner selection • 2. Partner training & facilitation • training • field visits • web & network sharing • twinning & peer evaluation • 3. Partner Self-evaluation checklist (QA checklist) • 4. Secretariat effectiveness & efficiency • 1. Partner, School, class output checklists • 2. Social Return on Investment (SROI) • 1. Individual student questionnaires • 2. Social Return on Investment (SROI) • 3. Process Review & Reflection* • 4. Participatory Action Research projects* • 5. Focus group discussions* • 6. Real life story collection* • 1. Randomized evaluation trials • 2. Longitudinal studies Partner DB SROI web tool QA web tool * Subject levels: Partner organization, school, class, student

8. 1.3 Strategy overview – Principles and structure (continued) II. Progressive learning through integrated knowledge management of the involved evaluation tools The learning of the different elements and tools of AQIS are linked together and spread out over the entire network. Indicator development: Aflatoun develops a cross-cutting indicator database that aims to capture the key indicators for all parts of AQIS: quality, output, outcomes and impact. An integrated set of indicators, related to the theory of change, will ensure that the programme will be evaluated on the points that matter. (The arrow on the last page signifies this indicator development) Network sharing: The design, methodology, implementation and results of the evaluation tools are shared over a global network of partner organizations and research institutions (see more below). Publications: Aflatoun’s key publication on AQIS is its widely distributed annual publication Children & Change, presenting the key programme results, learning, and successes and challenges. An important part of the quality assurance results will be included in our annual report. Curriculum development: As documents of what works and what does not, the quality reports, and outcome and impact studies will be key inputs for the development of the Aflatoun Curriculum and the materials in which this is laid down. This development will happen under the shared guidance of the Aflatoun Secretariat and the Curriculum Development Taskforce.

9. 1.4 Strategy overview – Principles and structure (continued) • Effective involvement of evaluation experts and practitioners • Aflatoun is an organization that wants to make a change. In order alsotoeffectively measure the change it effects, Aflatoun collaborates with leading research experts and actively involves the people whom are participating in, and are at the heart of the programme: the children, teachers and partner organizations. For AQIS, Aflatoun consults: • The AQIS (impact) advisory committee: An expert committee with members from a variety of research and action fields. The committee meets bi-annually and advises on AQIS and organization strategy, and the methodology and implementation of specific research projects and evaluation tools. • The Quality Assurance and Impact Assessment Taskforces: At Aflatoun’s global Campaign Launch in March 2008, several focus groups of partner organizations on particular topics were installed. The Quality Assurance taskforce and the Impact Assessment taskforce respectively, consist of representatives from all four continents in the Aflatoun network. They advise and decide on AQIS strategy, evaluation tool implementation, and they provide the on the ground perspective. In August 2008 the Impact Assessment Taskforce had its first 3-day meeting in Amsterdam. • The children and teachers: As evaluation tools are tested, a sample of our target group is involved. Students and teachers from several of our programmes are asked for their feedback, and the tools tested on them before they are applied to the entire programme network. • Aflatoun global network: Via network-wide, regional and sub-regional meetings and individual partner visits, topics from AQIS are being discussed. Further means of sharing are the monthly Aflatoun newsletter, the online member area a two-weekly Secretariat email update for partner organizations, and day-to-day communication.

10. Aflatoun Quality assurance and • Impact assessment System • Strategy overview – Principles and structure • Quality Assurance • Output Tracking & Outcome Assessment • Impact Assessment • Planning

11. 2.1 Quality Assurance

12. 2.2 Quality Assurance

13. 2.3 Quality Assurance

14. 2.4 Quality Assurance

15. Aflatoun Quality assurance and • Impact assessment System • Strategy overview – Principles and structure • Quality Assurance • Output Tracking & Outcome Assessment • Impact Assessment • Planning

16. 3.1 Output Tracking & Outcome Assessment

17. 3.2 Output Tracking & Outcome Assessment

18. Some of questions in children’s survey are taken from SROI web tool Elements of outcome & output data-collection: Output related questions to understand scope & outreach of programme Output related questions to understand scope & outreach of programme Output related questions to understand scope & outreach of programme Two sets of pre- and post-surveys, for workbooks 1-3 and 4-8 Children’s surveys School output survey Class output survey Partner output survey Partner SROI question list SROI web tool Partner app. system To be developed with MelJol (India-partner) Selection of data for SROI web tool comes from partner application system Data-collection CD Included in CD package Help & methodology documentation for partners (inclusion SROI web tool on data collection CD depends on agreed timelines with developers SROI web tool) Including: templates questionnaires, help documents, country database software

19. 3.3 Output Tracking & Outcome Assessment

20. 3.4 Output Tracking & Outcome Assessment

21. 3.5 Output Tracking & Outcome Assessment

22. 3.6 Output Tracking & Outcome Assessment

23. Aflatoun Quality assurance and • Impact assessment System • Strategy overview – Principles and structure • Quality Assurance • Output Tracking & Outcome Assessment • Impact Assessment • Planning

24. 4.1 Impact Assessment

25. 4.2 Impact Assessment

26. Aflatoun Quality assurance and • Impact assessment System • Strategy overview - introduction • Quality Assurance • Output Tracking & Outcome Assessment • Impact Assessment • Planning

27. 5.1 (Suggested) planning 2008-2013 all AQIS issues