Qualitative analysis: Steps, Tips, and Traps to Avoid

1. Qualitative analysis: Steps, Tips, and Traps to Avoid Laurie Drabble Feb, 2012

2. Overall Approach to Analysis: Phenomenology • Looks at the lived experience of the phenomena of interest • Analysis: Identify descriptions of the phenomenon; cluster into discrete categories; taken together, these describe the “essence” or core commonality and structure of the experience (p. 1373). Starks & Trinidad (2007) Choose your method: A comparison of phenomenology, discourse analysis, and grounded theory.

3. Process: Inductive • The “story” of the research comes from multiple reading & interpretation of data • Purpose of this process: • Condense and summarize extensive and varied raw data • Establish link between research questions and findings • Develop a model (or theory) about the underlying structure of experience or processes (Thomas, 2003) http://www.fmhs.auckland.ac.nz/soph/centres/hrmas/_docs/Inductive2003.pdf

4. Overall Steps (Cresswell) • Preparing and organizing the data (transcripts, field notes, etc.) • Reducing the data into themes through a process of coding and condensing the codes • Representing the data in figures, tables or a narrative

5. What is a theme? • A theme is a category of information that you as the researcher identify and name and that may include a variety of coded information. This involves a process of “coding” the data and then condensing or classifying the codes into larger families of themes. Cresswell, 2002

6. What’s the overall process? • Qualitative analysis is an iterative process—meaning that a final analysis of the data is achieved by repeatedly reading and re-reading the data and repeatedly developing themes and refining themes. • Developing themes involves a process of classifying, describing and interpreting the data. • The data are reduced into meaningful “chunks” of information • During this process, some information will inevitably not be included in the analysis Cresswell, 2002

7. Specific Steps • Transcribe data (important part of “immersion”) & prepare for analysis • Preliminary exploration • Close reading transcript(s) • Making notes/recording ideas • Creating categories • General categories (using research aims) • Sort into themes & subcategories (via coding) • Continue to revise and refine; select quotes that capture the core category • Interpret & report findings • Review and describe themes (usually 3-8) • Use quotes to illustrate

8. A bit more about coding • Select a process to “sort” text segments into categories (e.g. computer) • One segment of data may end up in two categories; some may be uncategorized (but keep “uncategorized” data to review • Within categories, looks for sub-topics, contradictions • Categories can be combined when they have similar meaning • Expect to code across interview questions.

9. Overview of process (Adapted from Creswell, 2002 as cited in Thomas, 2003) http://www.fmhs.auckland.ac.nz/soph/centres/hrmas/_docs/Inductive2003.pdf

10. Share & Tell • Examples of organizing data • Examples of coding • Examples of themes • Examples of how themes may be depicted

11. Ways to assess trustworthiness • Independent coding (code from raw text) • Coding consistency check (independent coder given coding scheme and text segment…can calculate inter-rater reliability) • Stakeholder checks • Summarize and correct and end interview • Verify interpretations/date in later interviews • Share preliminary or near-final results (Thomas, 2003) http://www.fmhs.auckland.ac.nz/soph/centres/hrmas/_docs/Inductive2003.pdf

12. Illustrations • Collaboration between Child Welfare and Addiction Treatment in Canada (note-taking, coding, themes & using quotes) • NCRP – Funding for Social Change in California (saturation, allowing, emerging themes, avoiding bias, member-checking) • Educating Students for Collaborative Practice – 298 project (summarizing and reporting).

13. Project: Collaboration between Child Welfare and Addiction Treatment: Canadian Context • Qualitative interviews - purposive sample of 24 professionals (12 CW, 12 AOD) • Research questions: • Contextual factors influencing policy • Factors that facilitate/impede collaboration • Perceived opportunities for improvement

14. What we did • Initial review of Each looked at the a sample of 2 AOD and 2 CW interviews, made separate notes about what we saw as provisional codes • First major round of coding, that generated about 40 categories • Used consensus process to sort through initial categories and modify coding • Organized codes into bigger “themes”

15. The Analysis Process: Inductive • Initial “sorting” by interview questions • Context, Help/Hinder, Opportunities • Initial “open-coding” to • Conceptualize, • Compare, • Categorize data • Iterative process to create then revise over 40 categories categories, which eventually became our “themes.”

16. What Helps?: Cross-cutting Enabling Factors Shared purpose and mandate Respectful relationships Leadership (individual or group advocates) Knowledge translation

17. Knowledge Translation “When I came here, people weren’t talking about outcomes and benchmarks and things like that. Now, they’re starting to move in that direction, and [asking] how do you improve your outcomes and what are we measuring?...That’s working.”

18. What Helps: Program and Practice Innovation Proactive safety and relapse prevention support Focus on safety & relapse prevention “not closing the door” on families Adoption of specific innovative programs

19. Focus on safety & relapse prevention “What it’s designed to do is really to have the kind of three-way agreements between addictions services, child protection, and the client in taking a look at yes, relapse may happen. What are the kinds of things that I as a parent need to put in place in order to ensure the safety of my child?”

20. Principles and Values • Working from principles and values • Translating principles and values into practice guidelines • Animating principles in collaborative planning • Program & Practice Innovation • Proactive support for safety and relapse prevention/ reducing harm • Innovations in practice as forming a nexus for collaborations • Cross-cutting enabling factors • Shared purpose • Leadership • Respectful relationships • Knowledge translation mechanisms • Shared Outcomes • For clients: Child safety, healthy mothers and families • For systems: Continued improvements, attention to social determinants, increased community level supports, new approaches for Aboriginal mothers Processes and Protocols Planning in partnership Communication protocols and guidelines Mechanisms for conflict resolution Continuous learning Reflective ofspecificcontexts Figure 1: Summary of factors that facilitate collaboration between systems

21. Illustration : NCRP: Funding for Social Change in California • PURPOSE: Examine the role of progressive and mainstream foundation funding in structuring, influencing, enabling or constraining the policy and advocacy work of nonprofits • METHOD: Telephone interviews with representatives from 73 nonprofit organizations involved in state level public policy

22. “Gaps” in the Public Policy Nonprofit Landscape • Need for greater strategic planning • Foundation hesitance to fund policy • Confusion about policy work (what it is and who does it?) • Conflict between mission and money • Tension between research organizations/think tanks and grassroots organizations

23. Assessing Trustworthiness: Member Check • Discussion groups with 20 nonprofit and foundation representatives (Northern and Southern CA) • Purpose… • Validate initial findings • Provide input for interpretation

24. Added Study Component & Report Components • Interviews with 8 foundation representatives • How foundation defines policy related grantmaking • Advantages of funding policy work • Barriers overcome in order to fund the policy work of nonprofits • Lessons learned for other foundations • Added “success” case stories

25. Driving forces for foundation support of policy • Leveraging larger results for issues areas and communities • Creating a voice for those most impacted • Increasing community assets and long term capacity • Building a multifaceted response to community problems

26. Tips for writing results • Organize findings by research questions • Do NOT organize by interview guide questions. • Name, summarize the theme (and subcategories), illustrate with quotes. • Consider providing a visual that captures the “big picture.”

27. Sample Summary (298 project) Visual– What students need to learn for collaboration

28. Learning Strategies for Developing Competence in Collaborative Practice