Mental health, social inclusion & arts

1. Mental health, social inclusion & arts Developing the evidence base Department for Culture, Media & Sport + Department of Health + National Social Inclusion Programme

2. Overview of presentation • Studyoverview & aims • Evidence in a mental health context • Issues raised (methodology & indicators) • Our indicators & measures • Attributing change to arts participation • Outcome study results • Case study methods • Case study results • Limitations of the study • Implications for further research • The ideal study!

3. Study overview • Reviews of previous arts in health studies & social exclusion literature • Survey of 102 arts & mental health projects in England • Retrospective analysis of project data • 6 month outcomes study • Qualitative case studies with 6 diverse projects

4. Aims • To identify appropriate indicators & outcome measures • To develop and implement an evaluation framework

5. Evidence in a mental health context • Methodology • Gold standard RCTs • Meta-analyses • Systematic reviews • ‘Given’ indicators for our study • Decrease in mental ill health • Decrease in service use • Decrease in social exclusion (esp. increased occupational activity)

6. Methodological issues • RCTs • Control for potentially crucial variables • Technical & atheoretical • One trial doesn’t make an evidence base • Decades of work needed and often inconclusive • Standard of evidence required in a social context • It’s not about life and death • Public money is already being & will continue to be spent • Is a poorquality RCT worth the candle?

7. Indicator issues • Need to measure what arts projects aim to achieve: • Personal growth/transformation of identity, artistic skills, self-esteem, quality of life (90%) • Decreased mental ill health (7%) • Reduced service use (0%) • Opportunities for work or education/training (3%) • And take into account: • Meaning/relevance of indicators (service use, social exclusion) • Participation timescales

8. Our indicators & measures

9. Attributing change to arts participation • Participants asked to rate impact of arts participation in relation to each scale at follow up • Positive impact ratings included in regression analysis

10. Outcome study results • Significantimprovements in empowerment, mental health & social inclusion, where no new stress • Regression analysis indicated strong causal link for empowerment with less strong causal links for mental health & social inclusion • No change in medication & service use • No change in occupational or educational activity

11. Case study methods • Projects’ ‘theories of change’ identified & used as initial framework for analysing participants’ accounts • Main focus on analysis of participants’ accounts to identify what worked in what contexts • Benefits & associated outcomes (primary & secondary) isolated

12. Case study results • 8 processes, weighted to reflect distribution across projects & importance to participants • 3 commonly reported across all projects & viewed by most participants as being very important • 3reported across some projects and viewed by most participants as being important • 2 reported across all projects and viewed by some participants as being important

13. Processes important at some projects

14. Processes important for some participants at all projects

15. Limitations of our study • Timescale: • Sample too small for stepwise entry of variables in regression analysis • Impact questions based on scales as constructed at baseline, not as reconstructed using baseline results • No opportunity to assess reliability & validity of impact rating questions • 6 months not long enough for occupational activity to change • Or to assess whether changes maintained • Not possible to make direct link between outcomes & case studies

16. Implications for future research • Need to take account of both funders’ & projects’ aims • But also consider what is meaningful in the study context • Question about new stress at follow up useful • Impact ratings to assess arts contribution worth developing • Theory-based qualitative methods essential to understand how & in what contexts positive change occurs • Longer term studies needed: • For methodological development • To assess longer term outcomes • To link outcomes and case studies

17. The ideal study • A four year timescale to: • Develop impact rating questions • Recruit a large enough sample • Assess outcomes at 6 & 12 months • Select case study projects on basis of outcome study results • Allow time for theory generation through qualitative analysis • Include question about new stress at follow up