Evaluating outcomes of arts engagement: A multi-dimensional approach- a pictorial model

1. Evaluating outcomes of arts engagement: A multi-dimensional approach- a pictorial model www2.uiah.fi/projects/metodi/15b.htm Kim Dunphy, Program Manager, Cultural Development Network PhD researcher, Deakin University

2. Overview • The problem: • Reframing the intrinsic/instrumental dichotomy • How can qualitative research about outcomes of arts engagement be presented simply? • The solution: • multi-dimensional framework of outcomes • a pictorial model • Case study: youth theatre project, Scared Cool, • Ba Futuru, Timor-Leste

3. The problem …..the arts sector’s failure to find a framework that articulates its value in a cohesive and meaningful way, as well as by its neglect of the compelling need to establish a system for collecting evidence around a set of agreed indicators that substantiate value claims (Scott and Soren 2009) Intrinsic- instrumental value conundrum A pictorial model of evaluation for arts engagement

4. Finding a solution A pictorial model of evaluation for arts engagement

5. Taking a multi-dimensional approach: Four Pillars of Sustainability (Hawkes 2001) • To every public intervention, • (at least) four lenses • should be applied. • What is impact in terms of • - economic viability • - social inclusion • environmental • sustainability • - cultural vitality Image: Marla Guppy A pictorial model of evaluation for arts engagement

6. A pictorial model of evaluation for arts engagement

7. But, ……………..an emerging theme of enjoyment, pleasure,where does it fit? A pictorial model of evaluation for arts engagement

8. Themes of pleasure, fun, enjoyment I loved working with Kallista, because she has something new for us (participant, 24 years) (In attending) I was thinking about myself too. I wanted …. to have a good time- the show was funny (foreign audience member) My family loved this drama, they came to watch and they said, its really really good(participant 21 years) A pictorial model for arts engagement

9. Role of pleasure in arts engagement Cook Islanders view the main, and very important purpose of dancing, as creating pleasure or happiness, relating to a world view that the purpose of life is to have fun, anga’anga tamataora, literally ‘to work pleasure’ (anthropologist Alexeyeff, 2009) ‘Fun’ is a major motivator and outcome of participation in dance programs at school and in community settings for young American and Australian children (educational researchers Bond and Stinson, 2001, 2007) Maori migrants In Melbourne indicated that ‘fun’ was a major motivator for participation in cultural performing groups (community cultural development researcher Dunphy, 1993) Young people in northern Ireland were motivated by enjoyment of community music making, to engage in programs about peace-building, through which peace-building objectives of host organisation were achieved (peace-building researcher Pruitt, 2011) Most commonly mentioned (36%) personal benefits of attending or participating in arts activities include ‘entertainment or fun’ (Ontario Arts Council, 2010) A pictorial model of evaluation for arts engagement

10. Taking a multi-dimensional approach: CIV’s five domains • Healthy Safe and Inclusive Communities • Dynamic Resilient Local Economies • Sustainable Built and Natural Environments • Culturally Rich and Vibrant Communities • Democratic and Engaged Communities A pictorial model of evaluation for arts engagement

11. Taking a multi-dimensional approach: Six dimensions of integrated community development (Ife 2002) Economic dimension Reclassifying all outcomes of arts engagement into these dimensions • spiritual • Historic (Throsby 2001, Holden 2006) • expression of communal meanings. Social dimension Cultural dimension Civic engagement dimension Environmental dimension Personal/spiritual wellbeing dimension Personal Wellbeing A pictorial model of evaluation for arts engagement

12. Economic dimension • capacity for empathy (McCarthy 2004) • friendship • belonging/ relatedness/ sense of community • safety • social capital- bonding and bridging Social dimension • skill development • employment • business development • increased wealth A pictorial model of evaluation for arts engagement

13. capacity for empathy (McCarthy 2004) • friendship • belonging/ relatedness/ sense of community • safety • social capital- bonding and bridging • sense of place • neighbourhood character • connection to the natural world • air quality • water quality • bio-diversity social environmental A pictorial model of evaluation for arts engagement

14. sense of place • neighbourhood character • connection to the natural world • air quality • water quality • bio-diversity cultural • Reframing much of the ‘intrinsic’ • aesthetic • symbolic • authenticity • cognitive growth • historic (Throsby 2001, Holden 2006) • expression of communal meanings. • cultural capital- education (Bourdieu) (McCarthy et al 2004) • identity affirmation • respect for diversity environmental A pictorial model of evaluation for arts engagement

15. cultural • Reframing much of the ‘intrinsic’ • cognitive growth • Aesthetic • symbolic • authenticity • spiritual • Historic (Throsby 2001, Holden 2006) • expression of communal meanings. • cultural capital- education (Bourdieu) (McCarthy et al 2004) • identity affirmation • respect for diversity civic engagement • active citizenship • membership of local organisations • and decision-making bodies • having a say on important issues • engagement in political process A pictorial model of evaluation for arts engagement

16. pleasure, fun • subjective well-being • self-reported health • psychological distress Civic engagement • pleasure, fun • subjective well-being • self-reported health • psychological distress Personal/spiritual Wellbeing A pictorial model of evaluation for arts engagement

17. Taking a multi-dimensional approach: Six dimensions of integrated community development (Ife 2002) Economic Viability autonomy Cultural Vitality Social Inclusion autonomy Relatedness Pleasure autonomy autonomy Civic Engagement Environmental Responsibility autonomy Personal Wellbeing A pictorial model of evaluation for arts engagement

18. Economic Viability autonomy Cultural Vitality Social Inclusion autonomy Relatedness Pleasure • Increased access to economic resources • skill development • employment • business development • increased wealth autonomy autonomy Civic Engagement Environmental Responsibility autonomy Personal Wellbeing A pictorial model of evaluation for arts engagement

19. Considering different perspectives

20. Different stakeholders • Beneficiaries • Fieldworkers • Project co-ordinators • Manager • Host organisations • Funders • Government • Most Significant Change • (Davies and Dart 2005) Olaf Breuning (2009), Different Perspectives

21. Considering outcomes for all stakeholders in arts engagement.(Most Significant Change, Davies and Dart, 2005) Wider Community Funders Host Organisation Artistic Leader/s Audience Members Participants A pictorial model of evaluation for arts engagement

22. Considering outcomes for all stakeholders in Scared Cool youth theatre project Wider Community Families and friends not attending Pride in young person’s achievements Funders Unfunded activity the event was ‘a brilliant PR tool’ Peace-building organisation Ba Futuru Host Organisation that night was a highlight of my life- to see the audience’s response. Visiting theatre director, youth co-ordinator, young mentees Artistic Leader/s Audiences of local people; audiences of foreigners I think this show was very important, [....] for human beings to express their feelings Audience Members Participants Young performers … We can [....] change our bad thinking [....] and experience from the past’. A pictorial model of evaluation for arts engagement

23. A pictorial model of evaluation for arts engagement

24. Intended and unintended outcomes A pictorial model of evaluation for arts engagement

25. Intended outcomes Ba Futuru’s peace-building mission: - recovery of children and youth from the distress caused by civil strife in order to reduce levels of violence. - arts included because of the connection between participatory arts experiences and positive development. A pictorial model of evaluation for arts engagement

26. Intended outcomes Ba Futuru’s peace-building mission: - recovery of children and youth from the distress caused by civil strife in order to reduce levels of violence. - arts included because of the connection between participatory arts experiences and positive development. A pictorial model of evaluation for arts engagement

27. Unintended outcomes Participant Felis: Learning English is important for my future A pictorial model of evaluation for arts engagement

28. Positive/ negative outcomes A pictorial model of evaluation for arts engagement

29. Positive/ negative outcomes ‘Because the foreigners teach us, that's why I wanted to attend. If the teacher was Timorese, I would not have done this program’. (Participant, 18 years) ‘I am very comfortable when I am with malae (foreigners), because they can understand me. When I ask something, they always answer nicely. When I ask for some help, they will always help me. But East Timor people [….] they never try to analyse what is going on, they just stop in the short term’. (Participant, 19 years) A pictorial model of evaluation for arts engagement

30. Positive/ negative outcomes A pictorial model of evaluation for arts engagement

31. Depicting intended/unintended; positive/negative outcomes Intended Unintended A pictorial model of evaluation for arts engagement

32. Expected and unexpected outcomes: • Comparing between what org expected would happen and what actually occurred A pictorial model of evaluation for arts engagement

33. Length of time of change • Considering short, medium and long term change A pictorial model of evaluation for arts engagement

34. Return on Investment • Comparing what was invested- ‘cost’: • Culturally, economically, socially, personal wellbeing, civically, environmental • With ‘benefit’- outcomes in all dimensions A pictorial model of evaluation for arts engagement

35. Scared Cool theatre project: outcomes for participants Intended Unintended A pictorial model of evaluation for arts engagement

36. Economic Viability Cultural Vitality Social Inclusion Civic Engagement Environmental Responsibility Personal Wellbeing A pictorial model of evaluation for arts engagement

37. Depicting intended/unintended; positive/negative outcomes Intended Unintended A pictorial model of evaluation for arts engagement

38. Considering outcomes for all stakeholders in arts engagement.(Most Significant Change, Davies and Dart, 2005) Wider Community Funders Host Organisation Artistic Leader/s Audience Members Participants A pictorial model of evaluation for arts engagement

39. Challenges • Timeframe for evaluation • does the engagement have an impact (short-term) or outcome (longer term)? • Does reported change lead to actual change? • Attribution of outcomes, given lack of ‘control’? A pictorial model of evaluation for arts engagement

40. In concluding Thinking beyond intrinsic-instrumental to a multi-dimensional, multi-perspective framework for understanding outcomes of arts engagement A pictorial model of evaluation for arts engagement