The ESDinds project

1. The ESDinds project Analysing participation in a cooperative research project on values-based indicators Marie Harder, Gemma Burford, et al. Sustainable Development Coordination Unit, University of Brighton

2. Co-inception: the need for ESDinds • Initial discussions between academic researchers, an international consultant, and representatives of 10-15 CSOs working in Education for Sustainable Development • CSOs identified values-based indicators for project monitoring and evaluation as a priority: “making the invisible visible” • Meet specifications of EU call for proposals while leaving room for interpretation and co-design by participating CSOs

3. ESDinds Project Consortium Four CSOs: • Earth Charter Initiative, • Alliance of Religions and Conservation • European Bahá’í Business Forum • People’s Theater, Germany • Two university-based research groups: • Sustainable Development Coordination Unit, University of Brighton, UK • Charles University Environment Center, Prague • Independent adviser with expertise in the field of SD indicators

4. Co-design: defining goals & activities • Face-to-face meetings of whole consortium (every 6 months) • Consultative decision-making • Independent adviser acts as mediator • Reflection on how to improve collaboration • PBworks wiki and e-mail list • Reporting back to EU

5. Values and indicators • First phase: learning about values and outcomes that CSO partners associated with ‘successful’ projects • Then: “How would these values be lived?” • Initial list of indicators suggested by CSOs • Iterative development of indicator list • Identifying possible measurement methods • Planning field visits to test indicators

6. Case study: Sierra Leone YABC project • “Youth as Agents of Behavioural Change” • International Federation of Red Cross & Red Crescent Societies (IFRC): Principles & Values • Sierra Leone Red Cross Society • Promoting a culture of peace and inclusion • Agricultural projects and peer education bringing together vulnerable youth from ‘opposite sides’ • ESDinds field visit in February 2010 • Consortium’s need: Test the indicators! (NOT co-inception with respect to IFRC)

7. Defining research goals and activities • IFRC Director seeking effective ways to evaluate impact and measure change in individuals’ values • Youth leaders (members of YABC network) prioritised 10 indicators for testing • Indicators incorporated into a workshop to test IFRC’s ‘draft toolkit’ of YABC exercises in Sierra Leone • IFRC and SLRCS representatives selected research methods from a ‘menu’ provided by researchers

8. Methodologies • Quantitative: • Spatial survey (Never – Sometimes – Always) • Secret ballot survey (as above) • Structured observation of participation in decision-making and discussion • Qualitative: • Conventional focus groups • Focus groups with theatre performance • CSO staff preferred to watch researchers facilitating the exercises

9. Interpretation of data • Some data fed back to youth immediately through focus groups (survey findings) • Other results discussed in expert group • Some results invalid due to: • Conformity bias (`following the crowd’) • Uncomfortable environment (hot sun) • Lack of competent female translator • BUT Red Cross still learned something important and useful… • Less discrimination in teams than villages!

10. Sustainability • ESDinds exercises complemented IFRC’s draft toolkit very well • National Youth Coordinator confident of being able to incorporate them into SLRCS programmes without further input from researchers • IFRC has acknowledged values-based indicators as a useful approach for projects elsewhere

11. Assessing participation at each project stage • Consultation (Experts present pre-determined issues) • Cooperation (Community offers advice, but decision-making rests with experts) • Participation (Equal decision-making by experts and community) • Full control (Community controls decision-making, experts advise) Ref. Naylor et al. (2002) Soc Sci Med 55: 1173-1187

12. Relationships of participation • Participation varied at different stages of the project – this is an overview! • Within consortium: generally 3 • Equal decision-making (CSOs/research groups) • Between researchers and CSO staff: 3-4 • Decision-making mainly led by CSO staff • Between CSO staff and youth: (generally)1-2 • Very short, centrally organised workshop • Large group • Language & literacy barriers

13. Lessons learned • Participatory process was critical in responding to challenges, e.g. conformity bias in spatial survey • Consider all relationships of participation before concluding that research is truly ‘participatory’! • Youth to become co-evaluators of their own programs: • Identifying research needs • Defining goals and activities • More ‘hands-on’ involvement in M&E activities

14. Policy implications • Who represents ‘the community’? • Roles of local experts and leaders/elites? • Beneficiary populations are diverse • Who are the most marginalised people in this project? (age, gender, poverty, etc…) • What are the barriers to their participation - and can they be removed? • Is co-inception a realistic possibility? • How can they be involved in meaningful ways in co-design? • More participation = more benefit?

15. Thanks for listening, and please feel free to join our online community! www.wevalue.org www.esdinds.eu Conference 16-18 December 2010 at the University of Brighton Making the Invisible Visible: An Emerging Community of Practice in Indicators, Sustainability and Values