In-depth Interviews: When do I use them? How do I organize them?

1. In-depth Interviews: When do I use them? How do I organize them? Who do I interview and how many? How do I implement an interview? How do I code them? Then what? By Gina Eosco (Your was*iser with many identites!) eosco@ametsoc.org WAS*IS 2008 – Welcome to the Family was*isers!

2. Before We Begin… • This is JUST an overview … too much to tell you! • The best way to learn is to practice! • There are many books!

3. Snore. Please give an example. Qualitative methods are so text heavy. When do I use Interviews? • When …you are using grounded theory (Glaser & Strauss, 1967; Strauss & Corbin, 1990) • My summary: Use grounded theory when looking at subjects or ideas that we know a limited amount about, and knowing this, allowing the qualitative data (interview or focus groups) to lead the researcher to pose new theories, findings, or pose new questions. • When … you want/need in-depth information about a question, topic or problem. It is NOT generalizable!

4. When do I use Interviews? Example: My thesis (Part I) 1.) I started with a problem, not a question. a.) People are misinterpreting the “cone of uncertainty,” a highly used hurricane track graphic. b.) The weather community, at the time, was having a heated discussion about what to do about it. 2.) Turn your problem into a question … a.) Why are people misinterpreting the cone of uncertainty? b.) Why is the weather community arguing about this when there are social scientists who can help?! (We need to see the strengths of both physical and social scientists!)

5. When do I use Interviews? Example: My thesis (Part II) Solution? … keep asking questions =) 1.) How ARE people supposed to interpret the cone of uncertainty? Let’s do some interviews with forecasters to find out! I don’t have a clue! That’s how it all started!

6. How do I organize interviews? • The simple answer? … It depends. • My thesis used the Tree and Branch approach (Rubin & Rubin, 1994) • Ask Yourself Questions • Will you taint the interviewee’s opinion by asking a question in a certain way? • Is it important to ask the questions in a certain order? • In many ways, the interview protocol (the questions you will ask) is a lot like setting up a survey!

7. How do I organize interviews? Example: 2nd phase to my thesis • Here are some potential questions: • Have you seen this graphic before? • Do you know what the “cone of uncertainty” is? Can you draw it? • Can you tell me what this big white cone means? (conditional on which graphic I show … cones come in many colors! … It’s like ice cream really.) • Could you tell me what you think the “Skinny black line” means? • WAIT! What if they don’t have a clue?! Am I collecting useful data?! • Lets add … • How long have you lived in a hurricane prone area? • Has your local weather forecast covered any recent hurricanes? (Where do they get their weather?!) • BIG Tip: Do a trial run! Practice with someone!

8. Who do I interview and how many? Honest? It’s entirely up to you …. But please don’t let this go to your head. The interviewer plays a VERY important role. YOU determine the type of sample. (For example, snowball /purposive sample of forecasters with experience designing or communicating about the cone of uncertainty.) YOU determine your specific audience. (hurricane forecasters (public and private!), wx software companies, broadcast meteorologists, etc.) YOU determine when you have enough data. (I stopped at 19 interviews as the interviewees were giving similar data.) YOU determine how valid or reliable your study is!!(eek!)

9. Is it really this complicated?! Seriously, what can go wrong?! How do I actually implement an interview? • Here are some things to consider: • Provide the interviewee with the necessary forms and background information. You want them to feel comfortable! • ALWAYS ask permission before recording an interview. And bring extra batteries! (Make sure I tell you about different recorders!) • Consider the environment in which the interview will take place: is it noisy? Can other people hear you? Will there be lots of distractions? • Consider whether you need permission from a supervisor to interview this person (or be sure to ask them! … Boy do I have a story for you!) • Consider whether you are doing the interview with one person or perhaps two people. • Consider your body language! (where are you looking? Are you nodding?)

10. How do I actually implement an interview? The Good, the Bad, and the Ugly • The Good • You can learn SO much! Ask thoughtful questions! Be an active listener! • It’s as if the data is speaking to you … it’s fascinating! • The Bad • Some of my interview conditions were not ideal. The sound quality was very poor (made it difficult to transcribe). • Some of my interviewees didn’t have a lot to say (this can be really frustrating during an interview!! … What do you do?!) • It’s not a conversation … don’t provide your opinion. Ask Q’s. • The Ugly • The interview that just went ALL wrong. Get through the interview. Learn from it. Move on. (Everyone will undoubtedly experience one bad interview … be prepared for anything!)

11. I have the data …. How do I analyze it? • Transcribing and Coding • Transcribe the interview … word for word w/a couple of suggestions (For me, 1 hour interview = 8 hours of transcribing) • Start with some categories (TIP: did you organize your interview questions into categories?! Start with those!!) • Define each category (not too specific, but not too broad .. I’ll give an example) • In vivo coding (I’ll explain!) • Code, reflect, code, think, repeat as necessary… It’s a long, sometimes tedious process, but rewarding! • Coding software - Atlas.ti (they have student discounts!)

12. How do I analyze the data Example Part I Coding Manager Interviewer In vivo coding Codes Interviewee

13. How do I analyze the data Example Part II Master List of Codes My comments that I attached to a code

14. Then what do I do?! • Okay, great. Everything is coded … now what?! • For me …. • A LOT of thinking and staring into thin air! • What does the data mean? Are there patterns? If so, what do the patterns tell you? Does this jive with the current theory? Or, is this new ground? (ahhh, so that’s what “grounded” theory means!) • Hand written codes and notes (Sometimes the old fashioned way is the most useful way). • Seriously…. Just start writing.

15. Some more tips … • When writing: • Try to use a variety of quotes from different participants. • Shorten long quotes (It IS ok to delete something as long as it doesn’t take away from the interviewees point … be careful.) • When trying to make your argument, use a few quotes to make your point (my magic number was 3 unless the quotes were short). • Be sure to set the context in which the quote was stated. • You WILL make revisions. You WILL wake up in the middle of the night with an idea. You WILL never think you are done. (sort of like this presentation!)

16. What my methods section looked like …. Chapter 3 – Methods 21 Overview of research approach 21 Data Collection Overview 22 Question Development and Interview Setup 22 Interviewee Selection 24 Sample 25 Validity and Bias 25 A closer look at interview locations and their effect on interview responses. 25 Potential biases from the participants. 29 Potential biases from the researcher. 31 Data Analysis 34 Coding Categories 34 Presentation and Variety of Quotes. 34 Generalizability 36 Summary 36

17. The end Questions? Comments? Concerns? eosco@ametsoc.org

18. Summary of Section 1 – Types of Communication and Audience Analysis

19. Summary of Section 2 - Visual Validity Definition: Visual validity is an analytical construct to describe how accurately a reader’s meaning of a visual matches that of the creator’s intent. Art? Risk? Cone? Low Visual High Visual Validty Validity When does visual validity matter most?..... My contention is when there is a consequence to a misinterpretation.

20. Section 3 - Summary of Objectives 1.) Scientific Uncertainty “Bottom line the answer to your question is whether we’ll be able to give the public an idea of the uncertainty to provide the real detailed information, well how to use it to decision makers at the local and state level to help them with their tough calls.” 2.) Risk (Impacts) “If you were close enough to that track … then know you should expect damage within 50 miles of landfall.” 3.) Confidence (certainty) “It [the cone] represents where we expect the center of circulation to be, within the next so many days, and we have timelines on there every 12 hours. It represents the certainty of where we expect the center of circulation to be, only the circulation, not the impacts.”

21. Section 3 - Summary of Behavior Beginning of Hurricane Season…………… Prepare now! Formation……………………………………. Keep Listening 5-day Cone………………………………….. Prepare for a possible hurricane ………………………………….. Start listening to your emergency managers(EMs)/local decision makers 3-day Cone………………………………….. Begin to Implement Hurricane plan/Listen to EMs Hurricane/Tropical Storm Watch………….. Listen to EMs/Implement Your Plan! Hurricane/Tropical Storm Warning………... Implement Your Plan!!

22. The “life” of a visual ….

23. Summary of Section 4 - Visual Framing Definition: Visual framing is a technique to help increase the correspondence between a visual and verbal message whereby the creator purposefully designs the visual to match the verbal message. • Scientific Uncertainty • Risks/Impacts • Confidence • Listen to Emergency Managers

24. An example of visual framing…

25. Conclusions • Visuals may be excluded from the deficit model/simplification process. • Risk/complex science visuals may need explanation to achieve visual validity. • Visual validity may be necessary for risk visuals that have a consequence if misinterpreted. • Visuals may have a complex relationship between its design and its verbal/written objectives. • Visual framing may help scientists create visuals that achieve high visual validity.