Thoughts on Evaluation

1. Thoughts on Evaluation Dr. Douglas Bourn Director, Development Education Research Centre, Institute of Education, University of London

2. Aims of Presentation • Context of discussions re understanding of development and methodology of development education; • Competing and often contradictory objectives from governments, NGOs and educators • Purpose of the programme- why? • What is trying to do- trying to achieve • Methodology to be undertaken • Target grouping- who is it with • Levels of evaluation

3. Framework against which we assess and evaluate our programmes • Organisational forms of evaluation • Evaluation in context of learning and education • Evaluation in the context of communicating messages

4. Levels of evaluation • Evaluate against the why • Evaluate against the what • Evaluate against the how • Evaluate against the who

5. Why- The purpose • Aims and objectives of the programme/project/activity to be evaluated • Asking the why encourages us to think of the rationale for the initiative, its desired impact, the values, attitudes and actions we would like to see. • Evaluation methods here would be looking at impact assessment and review against overall goals of the programme.

6. What - Nature of the Activities and Learning to be Undertaken • actions to be undertaken to achieve the goals • Focus on knowledge, skills, learning processes • Evaluation methodology will be peer review, self-assessment methods

7. How – planning, organization and management of programme/activities • Organisational performance – processes of programme • Efficiency and Effectiveness • Summary evaluation with key stakeholders, reviews of process and measure against agreed ways of working

8. Who - and Processes of Change • Key players in the programme- nature and form of their involvement • Does the programme reach and engage planned target group • What happens to the people engaged in the programme - what do they do with the learning and experience gained.

9. Culture and practice of development education in most countries is not based on one of critical reflection, clarity of purpose and clearly defined aims and objectives • This is strange because most practice is based on funding proposals that request such processes. • Reason is all too often aims and objectives of funder very different from that of the grant recipient in what they want to do • Within governments, aims and objectives of a international development ministry or agency are going to be different to an education ministry • Aid and development agencies have different objectives to schools, youth groups or universities.

10. But there are some common principles against which development education could be evaluated at all levels: • Recognition that we are concerned with learning- so priority has to be nature and form of learning gained • Not about pre-determined outcomes or behaviour change • But cannot avoid desirable outcomes around active global citizenship • Awareness does not automatically lead to learning that leads to action • Principles re quality and form of learning related to perspectives and voices from the South, participatory learning methods and relationship to combating global poverty

11. Thoughts for consideration: • Recognition of the complexity of our agendas and that we have different agendas • Greater openness and honesty- sharing failures and problems as well as successes • Living in a global society and global communications requires re-think for many development educationalists as to how people become aware, learn and engage in global issues. • Perhaps we should not talk about evaluation but instead talk about why, what, how and with whom we are working, would like to work and above all create a culture of dialogue, reflection and critical learning

12. References: • www.dea.org.uk/measuringeffectiveness • www.coe.int/t/e/north-south_centre • www.osdemethodology.org.uk

13. d.bourn@ioe.ac.uk