150 likes | 239 Views
What can systematic reviewing tell us about labour markets?. Dr Sally-Anne Barnes Warwick Institute for Employment Research University of Warwick Sally-Anne.Barnes@warwick.ac.uk 4 th ESRC Research Methods Festival 5-8 July 2010. Aim of presentation .
E N D
What can systematic reviewing tell us about labour markets? Dr Sally-Anne Barnes Warwick Institute for Employment Research University of Warwick Sally-Anne.Barnes@warwick.ac.uk 4th ESRC Research Methods Festival 5-8 July 2010
Aim of presentation • To give an overview of the systematic literature review process • To highlight advantages and disadvantages of the process • To examine the lessons learnt and what the process can tell us about the labour market
Brief history of the systematic literature review • History in evaluations of health interventions (Cochrane approach) • Move to applications in public policy evaluations (Campbell Collaboration) • Applied to fields of education and social welfare (EPPI Centre)
Advantages of systematic review process • Comprehensive and objective assessment of available research • Consistent approach by research team • Reduction of reviewer bias • Selection and screening of articles monitored and recorded – transparency • Provides a sound evidence base • Identification of potential gaps in existing research
Disadvantages of the process for researching the labour market • Downplays the role of reviewer and professional judgement • Exclusion of unpublished reports and thesis • Does not build on other reviews that can be useful • Does not search for online materials • Regimented approach which could exclude interesting research
Skills diagnostics and screening tools: A literature review • Based on two review questions • Q1. Approximately 30,000 references identified, 579 screened, 29 used for study • Q2. Approximately 82,000 references identified, 1,131 screened, 40 used for study • Team of 11 • 8 months to conduct and write-up review
Phase 1: searching Aim: the systematic identification of potentially relevant studies • Identify keywords • Develop search strategy • Search electronic databases for articles • Record search results
Phase 2: screening Aim: the application of pre-determined inclusion and exclusion criteria to articles • Initial screening online • Transfer results into a bibliographic software programme • Application of inclusion and exclusion criteria • Record screening results
Phase 3: data extraction Aim: examination of studies to assess quality and extract evidence • Develop and use data extraction instrument • Obtain full document • Complete data extraction for each document • Record results
Phase 4: synthesis Aim: synthesis of findings and identification of key themes • Develop framework for data analysis • Assess quality of studies • Synthesise findings using data extractions • Identify key themes and conclusions
Phase 5: reporting and dissemination Aim: presentation of review findings • Identify studies that could not be obtained • Present record of searches, screening and studies used in review • Identify potential gaps in research and possible areas of future research
Lesson learnt • Not following methodology uncritically • Exercising professional judgement • Importance of drawing upon own knowledge and networks • Acknowledging researcher is an integral part of process
Conclusions… • Adds to our understanding • Real world perspective • Historical changes and trends • Identifies gaps in LM research • But with adapted methodology… • Identificationof unpublished research • Inclusion of ongoing studies and evaluations • Current view