Field Research Methods

1. Field Research Methods Lecture 5: Interview Methodology Edmund Malesky, Ph.D., UCSD

2. Organization of Today’s Lecture • Maxims of Conversation • The goals of interviewing • Types of Interviews • Methodological Process • Post-Interview • Coding • Focus Groups

3. Rules of Conversation Imagine what would happen to language if there were no rules to follow during conversations. Then it would be perfectly acceptable to follow “Hi, how are you doing?” with “cars are typically made from steel,” or to simply lie with every new statement. Communication would be impossible. It is clear that in normal conversation, we follow some general rules, inherited over time, that allow us to know what is acceptable and what isn’t.

4. Grice’s Conversation Maxims • What are these rules? • The first and most important is that both people in a conversation are cooperating. This is cooperation principle (first articulated by Philosopher Paul Grice). • Grice stipulated four maxims of conversation. • Maxim of Quantity • Maxim of Quality • Maxim of Relevance • Maxim of Manner

5. Maxim of Quantity • Make your contribution to the conversation as informative as necessary. • (i.e. don’t say to little) • Do not make your contribution to the conversation more informative than necessary. • (i.e. don’t say too much…. Joe Biden, I am talking to you!!). • The listener will assume that people are telling us everything we need to know. If they don’t say something, then we assume they simply don’t know that information. • A. “John and Mary have 2 children.” • Are they planning on having a third? • B. “How did Harry fare in court the other day? • Oh, he got a fine.” But Harry actually got a life sentence as well!!

6. Maxim of Quality • Do not say what you believe to be false • (i.e. don’t lie) • Do not say that for which you lack adequate evidence • (i.e. don’t say things which you can’t back up)

7. Maxim of Manner • Avoid obscurity of expression • Avoid ambiguity • Be brief (avoid unnecessary wordiness) • Be orderly • These maxims relate to the form of speech you use. You shouldn’t use words your listeners won’t understand or say things which you know could be taken in multiple ways. You should also not state something in a long-drawn out way if you could say it in a simpler manner.

8. Maxim of Relevance • Be relevant (i.e. say things related to the current topic of research) • Clearly, the most important maxim, since it is responsible for preventing random incoherent conversations lacking any continuity. i.e. a. “Is Gail dating anyone these days? “Well, she goes to Cleveland every weekend” b. “Isn’t Larry the biggest jerk you ever met?” “Uh, it sure is nice this time of year, eh?” The listener will assume these responses convey meaning.

9. Reasons for Flouting a Maxim • Someone actually may be trying to violate the overriding cooperative principle of the conversation. • Signaling a violation (minor violation): • A person might essentially come outand tell you they are violating the maxim and why. (“The word on the street is…” “I don’t know if this is relevant, but…” • Maxim Clash: A speaker might violate one maxim in order to preserve another. • Carson is driving to Meredith’s house. He asks John, “Where does Meredith live?” John answers, “Nevada.”

10. Reasons for Flouting a Maxim • To create a conversation implicature. By clearly and obviously violating a maxim, you can imply something beyond what you say. JOHN: Where’s Meredith? ELIZABETH: The control room or the science lab. • Sarcasm: • ELIZABETH: A lot of people are depending on you. • MEREDITH: Thanks, that really takes the pressure off.

11. Reasons for Maxims • Allow us to be more brief in communicating, since we don’t need to say everything we would need to if we were being perfectly logical. (i.e. no need to say, “John has 4 and only 4 children.”) • Also, they allow us to say things indirectly to avoid some of the discomfort which comes from saying unpleasant things directly. (i.e. “How does he look? He is a really nice guy?) • They also allow us to insult/deride people indirectly without as much danger of confrontation. • They allow us to imply dissatisfaction/anger without putting us in a position where we will have directly defend our views (often exploited).

12. Problems with the Maxims • Not clear whether they apply to other languages and cultures. • Some key concepts are undefined. A lot of intuition must be used to figure out, for example, when a speaker is being irrelevant. • They are not a complete listing (What about politeness?) • There is some over-lap, so it not always clear what maxim is being violated.

13. Whyshould we care as interviewers? • Remember respondents in an interview may deliberately avoid conversational maxims (i.e. not stating the maximum amount). Design your questions appropriately. • Pay careful attention when a respondent clearly violates a conversation maxim – information is being conveyed. • “Miss X has nice handwriting.” • “Miss Singer produced a series of sounds correspondingly closely to the score of an aria from Rigoletto.” • “I don’t pay bribes, but I will say that living standards of bureaucrats have improved since my business opened.” • Remember that respondents will attempt to cooperate with ambiguous or deceptive questions, attempting to guess the meaning of the question. Keep that in mind when interpreting responses. (i.e. You may be getting exactly the response you asked them to give you!). 13

14. Why Interview? • When interpersonal contact is important • When you need more in-depth data • When you need to have some follow-up • When complex questions or behaviors need to be explored • When you suspect other methods aren’t getting the whole picture

15. Advantages • You get rich, detailed material • You get the chance to go beyond the surface • Interviewing yields new insights • Participants describe what is important to them rather than being restricted to survey questions • Provides high reliability - clarification of responses to increase the likelihood of useful information • You can customize questions to the individual • Data analysis can lead to quantitative assessments

16. Disadvantages • Time consuming • Responses are variable • Interviewing effectively takes practice and experience • Large amounts of information to reduce and analyze • Less easily generalized • Dependent on researcher’s personal attributes and skills

17. Types of Interviews • The Structured Interview • Essentially a face-to-face survey format with some open-ended questions. • The Semi-Structured Interview • Questions and prompts are designed ahead of time, but pace and question ordering are flexible, depending on the discussion and respondent. • Interviewee is encouraged to expand on answers and express new information that the interviewee thinks is important. • Free-flowing, open-ended conversations • Informal in nature, the interviewer simply wants to allow the interviewee expound. • Observational Interviews • The context and surrounding are as important as the conversation. • i.e. Following the a respondent through an average day. • Focus Groups • Multiple respondents are interviewed at the same time. • The researcher is especially interested in the group interaction.

18. 10 Steps in the Semi-Structured Process • Define the Problem • State your purpose • Develop research questions • Select a sample • Perform the interview • Transcribe your data • Analyze your data • Report your findings

19. Defining the Problem - Goals • Remember, interviews are best used to solve a specific type of research problem: • Gathering generalizable information from a sample of respondents • Discovering a particular piece of information • Obtaining an important document or dataset • Informing or guiding work that uses of other sources of data. • Sitting back and learning from an acknowledged expert. • Add Color for a quantitative study. • Your research puzzle should dictate how you use the interview and the type of questions you ask. • My own preference is to use interviews to flesh-out information that I cannot get through other ways (archival research, available datasets). • I use interviews to answer the how/what questions?

20. Statement of Purpose • This should be a clearly written, concise statement that you can show to respondents and institutional sponsors. • What information do you plan to obtain from the process? • How long will it take? • How will the information be used? • Confidentiality requirements.

21. Selecting a Sample - What you need out the interview will determine who you sample. • Are you looking for generalizable patterns? • You will need to use some of the sampling procedures next week • In the case of a small-n, you should use a “most similar” matching technique. • If you are looking for detailed information from experts nested inside larger units (mayors in cities; CEOs in companies; principals in schools), you should match at the second level. • Semi-structured interviews will always involve less respondents than a survey. Nevertheless, the rules of selection and non-response bias still apply, even though there is a small-n. • Qualitative research can be fruitful, but it is not a synonym for sloppy. • Do you need deep information from the few acknowledged experts in the area. • Talk to as many as time and budget allow. Sampling is not as important. • Are you tracking down a particular case or journalistic-style story? • Treat the process like an investigative journalist. Talk to key players, learn more, and add new respondents to your list (snowball sampling). • Remember, that respondents in this type of interview settings have agendas. Try to cover all sides of the story. Do not let narratives from particular respondents drive your analysis.

22. Selecting a Sample – Obtaining Agreement • Send a letter explaining the goals of the research and requesting participation. • Letterhead from a locally reputable and neutral organization is helpful. • In developing countries, it often helps to have a local sponsor, who can write letters of introduction for you. • Be Persistent • Follow up with phone calls • …multiple phone calls (Rivera et al found in Russia that it took 15-20 calls) • Patiently, reiterate the goals of the research. • Build Networks • When you are new to a locality, it always helps to spend some time building networks. • In Vietnam, my most important research instruments still are my tennis racquet, soccer cleats, and tolerance for bia hoi. • Blindingly Obvious, but nonetheless important. • Dress appropriately for the interview (Even today, I wear glasses for elite interviews in SE Asia or respondents don’t take me seriously). • Be courteous to staff, respondents… • Make sure the respondent understands your neutrality.

23. Selecting a Sample – Analyzing bias • Realize that there will likely be some sort of selection bias in getting interviews. • Analyze the direction of the bias. • In the Rivera et al reading, I was bothered by this statement: • “Most failures to interview respondents stemmed from a problem endemic to all elite interviewing – the extraordinarily busy lives of the respondents (Respondents were particularly busy because the 1996 presidential campaign was in full swing.”

24. Obtaining Consent • Check with your Institutional Review Board or perhaps even the City Attorney to insure the safety of your subjects • Be especially careful if interviewing children • The Consent Form must be signed and dated before the interview can begin • Interviewees have the option of stopping the interview at any time if they feel uncomfortable • Specify how data is to be used and whether is will be used confidentially or anonymously

25. Semi-Structured Questions • Your survey form should have a mix of some closed and open-ended questions. • Weight more heavily toward open-ended, as this is really the goal of the approach. • Allow your respondents to elaborate, but reign them in when necessary. • Have a pre-programmed list of wrangling phrases (i.e. Could we return to…?; That is fascinating, but I want to make sure I understand … better) • Build conversational bridges between subjects directly into the survey script. (“Your answer leads me to wonder….; We have discussed how policy is made, now let’s step back to the motivations of policy makers.) • Many of the question rules of survey design continue to apply. • Avoid double-barreled questions. • Define complex terms carefully • Don’t editorialize. • Don’t trap respondents into particular answers.

26. Additional Questions Types in Interviews • Grand Tour Questions • “Could you describe a typical day in your office?” • Walk me through the stages of the business registration process • Take me through the legal drafting process in your country. • Asking for examples – These help elucidate complex points and add color. They also help break a respondent out of “a message track.” • Prompts/Probes /Follow-ups • Can be built into survey and spontaneous • Can be oral or physical (the blank look, arched eye brows). • The intentionally incorrect statement • State a law or fact incorrectly and gauge the respondent’s reaction. • Innocuous, but enlightening questions • Sometimes it helps to avoid asking a sensitive question, when you can get the same information with a seemingly innocuous question. • i.e. Gauging dual subordination in Vietnam – how many times do you meet with Party Secretary? • What did you do this morning? Rather than, “Did you eat breakfast?”

27. More tips for good interviews • Do your homework. The more you know, the better you can write questions, direct the interview, and probe. • Sometimes it may be necessary to establish that you have some expertise on the subject to keep your interview from resorting to a canned speech. • Don’t be afraid to ask for documents, datasets, other interviews… • In my own work, I consider this to be of utmost importance. • End with an open question to elicit anything not covered • Is there a question you expected me to ask that I didn’t?

28. The Sawatsky Method – Do’s • Start questions with what, how and why; • They demand the most from sources, requiring them to describe causes (what happened?), processes (how did it happen?) and motivation (why did you do it?). Fill in the blanks with questions beginning with who, where and when. • Probe tough issues, don't ask tough-sounding questions. • Keep in mind: Less is more. • Short questions produce succinct, dramatic, focused responses. Long rambling questions get long rambling answers or curt, confused replies. • Strategize. How will you build innocuous questions into your approach? • Establish agreement. • Without agreement on basic facts, you will spend most of the interview trying to force the respondent to accept your version of events, usually resorting to coercion and leading questions. • Build the interview on answers, not questions. • People find it easier to volunteer than to admit. When the source makes an original assertion, follow up with a question asking for evidence to support it. • Put the burden of proof on the source. • If a source insists, "There was no crime," ask, "How do you know that?" If a source says, "I can't remember" ask, "Why can't you remember?" • To focus questions, pick a key phrase the source mentioned and repeat it in an open-ended question. If, in describing his marriage, Ted Kennedy says, "We've had difficult times," respond: "What do you mean by difficult times?"

29. The Sawatsky Method – Don’ts • When attempting to acquire more information and not generalize, avoid closed-ended questions. “Mr. President, did you sleep with that woman?” “Did you authorize the leak?” • Don't make a statement instead of asking a question. • Instead of asking: "It must have been tough in the early years," ask: "What were the early years like?" • Don't ask double-barreled questions or two questions at once. • Don't overload questions. • Don't put comments into questions. • "What would go through your mind in the quiet times? Because there must have been times when you didn't talk to each other." • Don't use trigger or loaded words in questions. • "Your scheme would allow for a huge windfall to oil companies..." • Don't use hyperbole in questions. • Sources nearly always make up for a lack of neutrality by counteracting overblown questions with modest responses.

30. An example of message-tracking

31. Observation • Record your observations of the context of the interview. • What is the setting like? • Watch how respondent behaves with colleagues, family… • Fenno gives some excellent examples of how to do this appropriately. • But remember…

32. Effects of Observation "What we observe is not nature itself, but nature exposed to our method of questioning (Heisenberg)" - In the process of observing an experimental setting, we may accidentally alter the environment around us.

33. Interviewing Kit • Consent Forms • Digital Recorder/Extra Batteries • Extension cord • Interview guide • Pad • Pen • Water

34. Recording the Interview (Options) • Only take notes • Can be difficult to be responsive, especially if you are not operating in your native language. • Have an assistant take notes • Useful, but intense training will be necessary. • Tape the interview – recommended if respondent allows and you don’t think it impair information. • Purchase a tape recorder that is of a good quality and has transcription capabilities.

35. Interview Preparation • Schedule times with interviewees, you need about an hour • Interview room should be private, comfortable, free of distraction and easily accessible. • Turn off your phones • Seat yourselves to encourage involvement • Test your equipment before the interview starts

36. Post-Interview • Put thoughts and questions into a reflexive journal • Transcribe tapes • Take out identifying information • Code transcription • If you have questions concerning clarification, contact the interviewee • Erase tapes as specified in consent form • Secure transcripts

37. Triangulate after the Interview • After the interview, find ways to verify the information given to you. • If a respondent said they were at a meeting, do archival research to see if you can get a list of attendees. • If a respondent mentions a particular law or decree, get a copy for yourself. • If a respondent mentions a particular location, visit and verify the description (i.e. IRA attacks)

38. Initial Coding • Coding text of the transcript means to break the text down into small, meaningful units by assigning labels to the unit • Use the smallest unit possible • Look for data pertinent to answering the research questions • Units may have multiple codes associated with them • Also look for new ideas or concepts to explore • Initially, don’t worry about making relationships between the codes

39. Developing Themes: “The best of quotations are no substitute for thinking and formulating themes” • After initial coding you should start to see repeating themes. Repeating ideas lead to Themes which lead to Conclusions • Sort the coded units according to these themes • Assess the topics by asking “Does everything in this topic belong here? Can some of the topics be combined? Can some topics be deleted because they don’t relate to the research questions or because they don’t have much data in them? • Develop a conceptual schema, or dimensions, for the data • The schema can have major and minor themes within it

40. Coding Methods • Traditional is by reproducing unit on an index card and grouping cards into categories and themes • Qualitative software has taken the place of traditional methods • Same concept except faster, more convenient and with greater analysis power • NVivo, NUD*IST, N6, Atlas.ti are the most popular coding and analysis software programs (At least this is what my anthropology students tell me).

41. Reporting • After the themes are developed start a narrative that explains the properties and dimensions of the categories and the circumstances under which they are connected • Findings, conclusions, recommendations, future research – provide a context for understanding the conditions under which the results were obtained • Remember to present information in a way that is verifiable and falsifiable. • Report findings as they relate to each research question – don’t be shy • What is the contribution of this study to knowledge and practice? What improvements can be made?

42. Ethics • Assure respondent of your neutrality • Assure respondent of confidentiality. • If using experimental methods, make sure respondent is debriefed. • Consider how you will present information, so as not to embarrass or damage the careers of your confidantes. • Send copies of your work to all those who participated.

43. Meanings of Opt-Outs • On-the-Record: Information can be used in any way desired • Not for Attribution: You can use the information, but don’t attribute it to a particular source • On Background: Use to inform your own work and as a source for finding corroborating evidence • Off-the-Record: You DO NOT KNOW what you were just told.

44. How do Focus Groups Compare to Survey Methods? • Insight not rules (patterns) • Social not individual (you are specifically interested in how the interactions lead to answers) • Homogenous not diverse (too much diversity can destroy inference) • Warm not hot (Produce conversation that border on intimacy, not conflict) • Words not numbers. (Report results in prose. Frequencies make no sense in this context. They are not independent draws).

45. Traditional Focus Groups • 8-12 participants • Under direction of trained moderator • Formal, directive, structured • 60-150 minutes • Recorded, supplemented by field notes • Observed by scientific team

46. Traditional Focus Groups • Participants • Break characteristics – populations • Control characteristics • # & nature of groups & sessions • Purpose • Design complexity • Break characteristics • Resources

47. Data – Qualitative/Textual • Tape recordings • Transcriptions • 2 hour session: 40 to 50 pages • Field notes

48. Data Analysis • Driven by underlying research question • Qualitative • Interpretive, constrained by context • Topics – linked to group guidelines • Steps • Mechanical – organizing, subdividing • Interpretive – developing subdivisions (code mapping), search for patterns within subdivisions, drawing meaningful conclusions • Software: e.g.,The Ethnograph; Atlas.ti; QSR N6 • Reliability • Repeated review of data • Independent analysis by > two experienced analysts

49. Strengths & Limitations Focus group methodology is only as useful and as strong as its link to the underlying research question and the rigor with which it is applied.

50. Strengths • Provides concentrated amounts of rich data, in participants’ own words, on precisely the topic of interest • Interaction of participants adds richness to the data that may be missed in individual interviews • Provides critical information in development of hypotheses or interpretation of quantitative data