Researching the Arts in Criminal Justice

1. Researching the Arts in Criminal Justice Andrew Miles ESRC Centre for Research on Socio-cultural Change University of Manchester

2. Issues for the Arts in Criminal Justice • Demonstration of impact key to recognition from policymakers and funders • Large volume of anecdotal evidence suggests the arts have a positive impact in the criminal justice arena • But formal case-making is weak and unpersuasive

3. REACTTResearch into Arts & Criminal Justice Think Tank • Unit for the Arts and Offenders • DCMS, ACE, DfES, (Home Office), academics (e.g. CATR) , consultants • Strengthen the evidence base for the arts as an effective medium for offender rehabilitation • 3 point plan • Literature Review – 2004 • Feasibility Study – 2005/6 • Longitudinal Research Project – 2007ff

4. Arts in Criminal Justice Feasibility Study • What is a ‘robust’ evidence base? • Is it achievable? • Design: • 6 projects mixing setting, gender, artform • ‘good order’ & ‘progression routes’ • Logistical, practical and methodological issues in carrying out effective research

5. The Evidence-based Practice Agenda • ‘What works?’ • ‘Outcome’ research – 2 year reconviction rates • Hierarchy of research methods by ‘quality’ scale • ‘Gold Standard’ = Experimental design • Randomised Control Trials

6. A tough context for the Arts • Disparate • Inconsistent • Small scale • Short lived • Unembedded • Opportunistic • Resistant to standardisation

7. The RCT – Key Limitations • Practical issues • Problem of controlling for and randomising variables in complex social situations • Epistemological issues • Statistical association between variables is not an explanation of causality

8. Critical realism • We need to know why and in what circumstances programmes affect subjects before they can be said to work – ‘what works for whom in what circumstances’ • This requires a generative model of causation - based on hypothesising and testing the relationship between the context, mechanisms and outcomes of an intervention

9. Critical realism as an alternative for the Arts in Criminal Justice? • Rules context in • Process central to the understanding of outcomes • Large numbers not vital • Takes individuals seriously • Methodologically inclusive

10. Researching the Arts in Criminal Justice Contextual Issues • Arts projects difficult to recruit • No common structures or working models • Shifting timetables and content • Unaccredited so lacking status within the criminal justice system • Operation defined by institutional requirements and convenience • Conflict of ethos between arts and criminal justice organisations • Shared ambivalence towards research

11. Researching the Arts in Criminal Justice Practical Issues • Arts organisations often lack or are unable to articulate a clear methodology • Recruitment of participants is ad hoc, unspecified or institutionally manipulated • Participant group often small and unstable • Control over access to participants and resistance to ‘inappropriate’ methods • Accessing information about participants • Time and space for research

12. Design and Implementation of Research Methods • Research design varied according to the circumstances and dynamics of each project • 6 data gathering methods in various combinations trialed: participant profiling; psychometric testing; interviews, observation, diaries, tracking • Design shaped by theory where aims and objectives specified, otherwise mechanisms derived inductively

13. Theoretical Framing • Pro-social development • Resiliency theory • Performance theory

14. Profiling information • Some – limited - profiling information obtained locally but access to centrally held (e.g. OASys) records not achieved • Participant numbers too small to be utilised for sub-sampling

15. Psychometrics • Rosenberg Self Esteem (1965); Locus of Control (1984); Eysenck Impulsivity Scale (1994) • Quality of responses affected by suspicion, apathy or over-familiarity

16. Interview • Insight into personal experience, motivation, justification • Over time - developed trust and the articulation of personal narratives • Used to contextualise and cross-reference other types of evidence • Filming helped to draw out some and captured the dynamics of group interactions

17. Observation • Observation frame and scoring matrix • Structured, consistent insight into the dynamic characteristics of interventions and their impacts • Practitioners found onerous and difficult to engage with

18. Diaries • Unprompted and deeper level impressions of the intervention process • Low completion rates • Female/male responses differ markedly

19. Tracking • Regular intervals post-intervention probing sustainability and transfer • Profiling records, interview, questionnaire • Complex, time consuming, low success rate • Lines of communication and access difficult to maintain • Problem of transfer and release information • Transience of addresses and mobile phone numbers

20. Summary of Findings (1) • Positive shifts in engagement, confidence, self-control, co-operation and reflection • Conditioned by offence-type, length of sentence, experience of other interventions, educational ability, age, gender • Younger and more vulnerable participants most affected • Females benefited from the collective focus of interventions • Sustained in some but mostly ‘for the duration’ and some negative effects

21. Summary of Findings (2) • Culture and physical context of the intervention important – ‘non-judgmental space’ • Artform specific impacts - dance/trust; reading and story-telling/reflection and self-expression; dance & drama/alternative learning styles & liminal space

22. Conclusions (1) • Home Office Model not feasible and not necessarily desirable – not so much scientific power as simple science • Multi-method realist approach more sympathetic, workable and compelling • Current limitations of structure, culture and context in the arts in criminal justice hinder systematic research of any kind

23. Conclusions (2) • More scope for development of a critical realist model in community-based resettlement and youth-justice settings • Logistical platform for effective research requires institutional championing and embeddedness

24. Conclusions (3) • Particular methodological attention to: • the delineation of indicators of change • the development of arts-specific theoretical insight • structured and consistent approaches to the generation and analysis of qualitative evidence • the quantification of qualitative outcomes • creating an explanatory synthesis from different combinations of methods

25. Conclusions (4) • Arts engagement with and clarification of programme aims and objectives • The integration of evaluation as core element in programme design • Support from specialist research teams or trained practitioner-researchers • Long-term projects, longitudinal research, a tracking methodology with linked multi-agency support