Analytic Technique: Qualitative Data Analysis

1. Analytic Technique: Qualitative Data Analysis Research Methods for Public Administrators Dr. Gail Johnson Dr. G. Johnson, www.ResearchDemystified.org

2. The Research Process • Planning • Do • Collect the Data • Analyze the Data • Report Dr. G. Johnson, www.ResearchDemystified.org

3. Analyzing Qualitative Data • This is a process of making sense of non-numeric data Data from: • Narrative documents (speeches, newspapers, diaries, reports, etc) • Open-ended interviews • Open-ended questions on a survey Dr. G. Johnson, www.ResearchDemystified.org

4. Analyzing Qualitative Data Data From • Focus groups transcripts • Diaries • Unstructured observations • Videos, TV, You-Tube, etc Dr. G. Johnson, www.ResearchDemystified.org

5. Qualitative Data Analysis • The goal is to identify common themes • Requires a solid plan, attention to detail, good organization and sufficient time • Whether the analysis is done by computer or by hand, it is necessary to develop a coding scheme so the data can be systematically organized and analyzed • Computer software can help locate and organize data according to the coding scheme you create Dr. G. Johnson, www.ResearchDemystified.org

6. Qualitative Data Analysis • But the analysis—that is, making sense of the data, discovering the story the data reveals—is done by the researchers • The greatest concern is bias! • Paradigm blinders: hard to recognize things you don’t expect • Danger: to a hammer, everything is a nail Dr. G. Johnson, www.ResearchDemystified.org

7. Qualitative Data Analysis: General Process • Maintaining the Thread • Review the data, make notes as you go along • Read again once all the data collection is completed • Organize the data—according to research questions, by date, by geographic location—what makes sense given the situation Dr. G. Johnson, www.ResearchDemystified.org

8. Basic Activities • Identify common words, ideas, themes • Develop spread sheet or write on cards • Keep track of where they are located • Identify “quotable quotes”—that is, quotes that highlight the key issues: general views, divergent views, a range views • Make sure you know exactly where they are located so someone else can check to make sure you got the quote right Dr. G. Johnson, www.ResearchDemystified.org

9. Qualitative Data Analysis: General Process • Code the data • Identify common word, issues, themes and go through the material and label them according to that coding scheme • You may go through the material several times • Sometimes a major theme emerges at the end and you will need to go back through to see if it was present in the earlier data • Software is available that may help • Researchers might use spread sheets, index cards, or color coding of documents Dr. G. Johnson, www.ResearchDemystified.org

10. Ensuring Quality in Qualitative Data Analysis • Ensure inter-rater reliability • Like content analysis, the researchers should review the same material and apply the coding scheme • Then review and determine if there are difference • Discuss, revise the coding scheme if necessary and retrain the coders • Repeat this test until there is agreement in the coding • Compare results • Work out differences • Then code all the material Dr. G. Johnson, www.ResearchDemystified.org

11. Qualitative Data Analysis: General Process • Interpreting the data • Subjective, making sense of the data • Review the common themes and note whether there are systematic differences in views based on the characteristics of the people • Are there any seeming relationships between the themes or characteristics? • What are the major points that emerge from the analysis? Dr. G. Johnson, www.ResearchDemystified.org

12. Qualitative Data Analysis: General Process • What about minor points? • Sometimes an uncommon theme stands out • The researchers need to explain why a minority view might be important • Present in context: one minority view should not be given equal weight to the views of many others • Be mindful of the distortion that can occur when attempting to “be fair”: always indicate when it is a minority view and be careful that it does not trump the majority views • Presenting a unique view can be important or it might just be a ploy for attention—just as it is in real life Dr. G. Johnson, www.ResearchDemystified.org

13. Qualitative Data Analysis General Process • Interpret the data • Reality check: do others agree? • Share preliminary drafts with stakeholders or a small group of the participants to explore the issues you have discovered • Share final draft with experts and cold readers Dr. G. Johnson, www.ResearchDemystified.org

14. Affinity Diagram • A group process for analyzing qualitative data: • Research team reviews the material • In silence, write down each idea, key work or theme on a sticky note • In silence, post on wall • In silence, sort into similar categories • Once the sticky notes are organized, discuss Dr. G. Johnson, www.ResearchDemystified.org

15. Affinity Diagram • Identify common themes • The Affinity Diagram tool is a quick way to set up a coding framework • It is also a great way to identify the major themes and topics in the final article or report • It has the advantage of capturing all ideas before the discussion begins • Everyone’s ideas have equal value when posted Dr. G. Johnson, www.ResearchDemystified.org

16. Writing Up Results • Focus on major themes. • “A number of participants said.” • “Some” said X, while others said Y. • Avoid generalizations: if not everyone was asked to comment about a particular practice, you do not know if the five who commented represented the views of others or were just the only ones that thought this practice was important. Dr. G. Johnson, www.ResearchDemystified.org

17. Writing Up Results • Provide a range of perspectives. • Some said “….” while others said “…..” • Highlight interesting perspectives even if only said by one or two people. • But be sure not to give more undue importance and be clear that it was just a few • A few offered unique views: “….” • These might be important because “…..” Dr. G. Johnson, www.ResearchDemystified.org

18. Writing Up Results • Do not try to report numbers or percents unless everyone was counted in exactly the same way • For example, 10 people out of the 20 who you interviewed commented that they liked the training program. • But unless you asked all 20 people, you do not know what the other 10 people thought. • It would be a mistake to report that 50% of those interviewed liked the training program. • Maybe the other 10 liked it too but had other things they wanted to talk about. • You do not know what everyone thinks unless you ask everyone the exact same question! Dr. G. Johnson, www.ResearchDemystified.org

19. My Views About Qualitative Data Analysis • Qualitative data tells a narrative story • It can provide depth, richness and insights in ways that quantitative data cannot • Allows researchers to their knowledge and expertise to make sense of the data • It is deceptively difficult and can be overwhelming if there is a lot of data Dr. G. Johnson, www.ResearchDemystified.org

20. My Views About Qualitative Data Analysis • Some researchers like the richness and its non-numeric nature • Others wish they had check-a-box data because it is so much easier and less time consuming to analyze Dr. G. Johnson, www.ResearchDemystified.org

21. My Views About Qualitative Data Analysis • It can be like a Rorschach test –an inkblot that some see as a butterfly and others see as maple leaf • Remember: reasonable people can read the same material and have very different interpretations. It is important to explore those differences to gain a deeper understanding rather than to win the argument Dr. G. Johnson, www.ResearchDemystified.org

22. Creative Commons • This powerpoint is meant to be used and shared with attribution • Please provide feedback • If you make changes, please share freely and send me a copy of changes: • Johnsong62@gmail.com • Visit www.creativecommons.org for more information Dr. G. Johnson, www.ResearchDemystified.org