Qualitative Secondary Analysis: Asking ‘new’ questions of ‘old’ data

1. Qualitative Secondary Analysis: Asking ‘new’ questions of ‘old’ data Brenda M. Gladstone Tiziana Volpe Community Health Systems Resource Group The Hospital for Sick Children QUIG, April 23, 2008

2. Introduction • Little published ‘evidence’ of benefits & limitations1 • Lack of guidance about QSA process, methodological decision-making or evaluative criteria2,3 • Need for empirical exemplars1 • Why now? emergence of data archiving and advances in computing • Promotion of data sharing and data retention in policy and practice (ESRC & Qualidata in the UK) 1Corti & Thompson, 2004 2Thorne, 1994;1998 3Sandelowski, 1997

3. Defining QSA • Reuse of existing data, collected for prior purposes, a term used to describe various analytical practices to: • investigate new questions, or a new perspective on an old question. • corroborate, validate or refine original (primary) analysis • Inclusion of meta-research designs (meta-synthesis; meta-analysis; meta-ethnography) is contentious • Qualitative longitudinal research

4. Why do QSA? • Generate new knowledge; new hypotheses/questions; to support existing theories • Expand/extend, data sets/participant numbers and analysis • Enable comparative research in different geographical/temporal(historical)/cultural contexts • Generalizability of findings, data from different researchers researching similar populations • Methodological and substantive teaching tool

5. Why do QSA (con’t.)? • Student projects • Existing databases represent considerable financial and temporal commitments • Respondent fatigue; burden to ‘talk more’; sensitivity to those living stressful lives; vulnerable, inaccessible populations • Development of archival data

6. Types of QSA (Heaton, 1998; 2004) • Supra analysis: transcends original focus examining new empirical, theoretical, methodological questions • Supplementary analysis: more in-depth investigation of an emergent issue or aspect of data not fully considered in primary study • Re-analysis: data are re-analyzed to verify and corroborate primary analyses of qualitative data sets • Amplified analysis: combines data from 2 or more primary studies for the purposes of comparison or to enlarge a sample • Assorted analysis: combines secondary analysis of data with primary research and/or analysis of naturalistic data

7. Theoretical, methodological, ethical and practical challenges related to QSA • Nature of qualitative data • Problem of data ‘fit’ • Problem of ‘not having been there’ • Problem of verification • Researcher(s) relationship to primary data

8. Doing QSA: A case example The Original Study: • Understanding social factors influencing motivation while living with schizophrenia or first episode psychosis (FEP) • 60 in-depth interviews • People diagnosed with schizophrenia • Young people experiencing FEP • Parents • Health Professionals • Interviews transcribed and entered into Ethnograph

9. Doing QSA: A case example The QSA Study • A ‘supra analysis’ of a subset of interviews with parents whose children had experienced a first episode of psychosis • Purpose: ask new questions of primary data about parents’ help-seeking experiences • Data consisted of 10 interview transcripts with 7 mothers, 3 fathers • Consent from original participants not obtained for QSA

10. Challenges Encountered • Establishing quality • Context • Documentation • Funding • Ethics

11. QSA Challenges: Establishing Quality • Establishing rigor/trustworthiness: criteria for primary/secondary analysis, equally applicable? • Availability of original transcripts and documents • Role of original researchers • Context in which data was collected • Sampling issues • Analytic framework

12. QSA Challenges: Context • Considering the historical, biographical, and theoretical contexts in which the primary study was conducted • Field of ‘early intervention’ • Help-seeking processes • Researcher location; disciplinary training • Role of researcher in design & implementation of study

13. QSA Challenges: Documentation • Availability/quality of primary study documents • Transcripts • Demographic information • Audio recordings • Field notes; analytic notes • Technological changes • Human & financial resources

14. QSA Challenges: Funding • Funding a secondary analysis: • Who funds it? • What is it worth? • Is it cost effective? • Data storage and archival costs?

15. QSA Challenges: Ethics • Obtaining REB approval • Informed consent • Does re-using data violate original consent? • Is it feasible and/or appropriate to go back to participants? • Building in consent for future studies • Confidentiality/Anonymity • What is anonymization in the context of QSA • What is enough anonymization • Data stockpiling…just in case

16. Emerging Guidelines for QSA • Suggested guidelines for documentation: • Information regarding interaction between researcher/participants • The ‘whole interview’ not just the parts • Background information about the interviewer (age, gender, race, social class) • Place, time, setting • How participants were approached/recruited • Information about relevant others (gatekeepers)

17. ISA-RC33 7th International Conference in Naples, September 1-5, 2008 • Session 1: Making qualitative data more re-usable: defining issues of context and representation • Session 2: Making qualitative data more re-usable: technical solutions for capturing context and representation We would like to invite papers that consider issues around the effective archiving and further (re)use of raw qualitative research data, with a particular focus on the issues of how context is defined and (re)captured.

18. Going Forward • Archiving primary data and documentation • audio-tapes, interviews, transcripts, field-notes, journals, photos? • how is it organized, managed, and stored? • Who owns the data? Who is responsible for data? How is credit for the primary/secondary analysis established? • SSHRC Research Data Archiving Policy

19. References • Cisneros Puebla, C., Mruk, K., & Roth, W. Editorial: The FSQ Issue on “Qualitative inquiry: Research, archiving and reuse”. Forum Qualitative Sozialforschung/Forum:Qualitative Social Research [on-line journal], Art.45. Available at: http://www.qualitative-research.net/fqs-texte/2-05/05-2-45.e.htm/. Accessed August 15, 2005. • Corti, L, Thompson, P. Secondary analysis of archived data. In: Seale, C, Gobo, G, Gubrium, JF, Silverman, D. Eds. Qualitative Research Practice. London: Sage Publications, 2004: 327-343. • Fielding, N.G. Getting the most from archived qualitative data: epistemological, practical and professional obstacles. International of Social Research Methodology, 2004; 7: 97-104. • Gladstone, B., Volpe, T. & Boydell, K. Issues encountered in a qualitative secondary analysis of help-seeking in the prodrome to psychosis. The Journal of Behavioral Health Services & Research, 2007; 34(4): 431-442. • Heaton, J. Reworking Qualitative Data. London: Sage, 2004. • Heaton, J. Secondary Analysis of Qualitative Data. Social Research Update 22 Guildford: University of Surrey, Institute of Social Research; Available at: http://sru.soc.surrey.ac.uk/, 1998; Accessed June 24, 2004. • Hinds, P.S., Vogel, R.J. & Clarke-Steffen, L. The possibilities and pitfalls of doing a secondary analysis of a qualitative data set. Qualitative Health Research, 1997; 7(3): 408-424. • Mauthner, N., Parry, O. & Backett-Milburn, K. The data are out there, or are they: Implications for archiving and revisiting qualitative data. Sociology, 1998; 32(4): 733-745. • Sandelowski, M. “To be of use”: Enhancing the utility of qualitative research. Nursing Outlook, 1997;45: 125-132. • Thorne, S. Secondary analysis in qualitative research: issues and implications. In: Morse, JM, (ed.) Critical Issues in Qualitative Research Methods. Thousand Oaks, CA: Sage, 1994: 263-279. • Thorne, S. Ethical and representational issues in qualitative secondary analysis. Qualitative Health Research, 1998; 8: 547-555. • www.esds.ac.uk/qualidata