Field Research and Analytic Transparency: Collecting Data, Analyzing Evidence, and Drawing Inferences in Qualitative Res

1. Field Research and Analytic Transparency:Collecting Data, Analyzing Evidence, and Drawing Inferences in Qualitative Research Short Course #2 APSA 2013 Instructors: Diana Kapiszewski, Georgetown University Naomi Levy, Santa Clara University Colin Elman,Syracuse University Building on a course initially developed and taught by Melani Cammett (Brown University), Marc Morjé Howard (Georgetown University), Evan S. Lieberman (Princeton University), Julia F. Lynch (University of Pennsylvania), Lauren Morris MacLean (Indiana University), Benjamin L. Read (UC Santa Cruz), Scott Straus (University of Wisconsin, Madison) & Sara Watson (Ohio State University)

2. Agenda Part I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) ~ Break (4:10-4:20) ~ Part III – Interviewing (4:20-5:15) Part IV – Organizing and Analyzing Data & Assessing Progress (5:15-6:00) ~ Break (6:00-6:10) ~ Part V – Analytic Transparency (6:10-7:00)

3. Instant Field Research Survey • Any prior fieldwork experience? • If so: • how many months total ? • in what regions? • If not: • fieldwork planned for when? • and for where?

4. Where We Are in the Course Part I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) 1. Borders and Varieties of Fieldwork 2. Research Design and Fieldwork 3. Preparing for Fieldwork Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Part V – Analytic Transparency (6:10-7:00)

5. Fieldwork in Political Science: What Is It? • Our working definition of field research: • Leaving one’s home institution to collect data or information that significantly informs a research project. • Fieldwork is not delimited to one’s time in the field • Fieldwork is a very iterative process

6. Fieldwork in Political Science: Why Do It? • Is fieldwork still necessary? (Yes!!) • Sometimes you just have to BE THERE • Allows building of networks • Provides opportunities • Facilitates path-breaking empirical scholarship • Less tangible reasons

7. Fieldwork in Political Science: What Does it Look Like? • Heterogeneous! • Different epistemological approaches • Different types of settings • Done independently or as part of a larger project • Many types of data-collection techniques • No “standard fieldwork”!

8. Fieldwork in Political Science: When Does it Happen? • Selecting the topic • Reading existing literature • Defining the research question • Scoping trip • Completed proposal • “Surgical strike” trips • Three-four month trips • Long-haul stays • Writing up • Follow-up data-gathering Loosely Structured: Open-Ended Research Highly Structured: Narrowly Focused Research

9. Long stays: Pros Experience another culture In-depth research Contacts/network Ideas More relaxed & fun Cons May be far away Lack of urgency “Too much” data Short trips: Pros Efficiency Forces you to take stock/think analytically Identify comparisons/ contrasts quicker Less time away Cons Can be more costly Scheduling difficulties Not enough time Fieldwork in Political Science:Trade-offs Among Types

10. Emotional and Psychological Challenges • Loneliness / isolation • NEVER being alone • Language/cultural barriers • Family stresses • Financial stresses • NERVOUSNESS! • Identify your concerns • Network • Balance • Know where to turn • Don’t be stoic!

11. Where We Are in the Course Part I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) 1. Borders and Varieties of Fieldwork 2. Research Design and Fieldwork 3. Preparing for Fieldwork Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Part V – Analytic Transparency (6:10-7:00)

12. Design-Driven Fieldwork and Fieldwork-Driven Design • Field work must fit your research design • Your design must accommodate field realities • Competing Imperatives • Your ideal research design • Practicality

13. Design-Driven Fieldwork and Fieldwork-Driven Design • Design “do-able” research • Think about your own limitations • Find a way • Think positively • Revise • Think in terms of variables

14. Fieldwork and the Research Design • Nomothetic • Thinking in terms of variables • Idiographic • Thinking in terms of cases

15. Data Matrix

16. Case Selection and Sampling • Selection decisions arise at many stages • Macro-level • Country cases • Meso-level • Regions or Towns • Time periods • Sectors, etc. • Micro-level • Individuals for interviews • Documents for content analysis

17. Small-N • Use case selection to provide causal leverage • Use a variable-centered approach • Hold rival explanatory variables constant • Allow your primary explanation to vary • Example: Dan Posner (2004) • Macro-level: Zambia & Malawi • Meso-level: Town selection held rival variables constant

18. Posner (2004) Meso-level Selection

19. Large-N • Where possible, use random sampling • This increases the generalizability of your findings • Need a list of the universe of cases • Consider cluster sampling • Sometimes not possible or desirable • Special cases you want to include • Important variation you want to capture

20. Where We Are in the Course Part I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) 1. Borders and Varieties of Fieldwork 2. Research Design and Fieldwork 3. Preparing for Fieldwork Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Part V – Analytic Transparency (6:10-7:00)

21. The Months and Weeks Before…Administration • Funding • Develop a strategy far in advance • Apply for lots of money • Dealing with your Institutional Review Board • Does not have to be difficult! • Follow the directions, adhere to deadlines • Find out about exemptions and consent

22. The Months and Weeks Before…Intellectual Prep • Dig into your topic • Background research on your country/ies • Begin to write documents • Collect documents you’ll use in the field • Brush up on methods • Think about your foreign language skills

23. The Months and Weeks Before…Reach Out! • Develop your network of scholars and contact them! • How about a host institution (research affiliation)?

24. Converting your Research Design into a Data Collection Plan

25. Example Data Collection Plan

26. Converting your Research Design into a Data Collection Plan

27. About the Data Collection Plan • You can head to the field before you have your entire Data Collection Plan filled out! • Objectives of the Data Collection Plan • Helps make your project manageable • Is a measure of your progress • Is the link between all the millions of tasks you carry out in the field, and your larger project

28. Converting your Research Design into a Data Collection Plan

29. Where We Are in the Course Part I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection(3:25-4:10) 1. Forms of Data Collection 2. Data Collection: Choices, Challenges, Assistants * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Part V – Analytic Transparency (6:10-7:00)

30. Non-Interactive Data Collection • Observations • Following the local media • Newspapers, magazines, radio and TV • Books & articles • Obtaining Documents & Existing Datasets • Government agencies & Ministries • NGOs • Archival Work

31. Interactive Data Collection • Ethnography • Participant Observation • Experiments • Laboratory experiments • Survey experiments • Natural experiments • Field experiments • Large-scale surveys • Interviews • Focus Groups

32. ChoicesinData Collection • Strengths and Weaknesses • Non-interactive • Documents don’t “react” • Documents and archives can be biased • Interactive • Respondents can provide first-hand account • Respondents might be inaccurate • Triangulate!

33. Triangulation

34. Where We Are in the Course Part I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) 1. Forms of Data Collection 2. Data Collection: Choices, Challenges, and Assistants * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Part V – Analytic Transparency (6:10-7:00)

35. Sequencing and Prioritizing Data Collection • Distinguish necessary from desirable data • Centrality to core hypotheses or key variables • Factors to consider in ordering data collection • Overlapping strategies • Harder-to-get vs. easier-to-get data • “Low-risk” vs. “high-risk” contacts • Temporally tied events • Depth vs. breadth

36. Anticipate Data Collection Challenges • Roadblocks to accessing elites, ordinary citizens, archives, datasets • Affiliate • Network – find a “connection” • Think empathetically • BUT consider investment: time, effort, $

37. Methods for Capturing Data • Documents • Take notes • Reproduce • Interviews & Focus Groups • Field notes after the fact • Jotted notes during • Audio/Video tape • Combo • How do decide? • How much data do you want to capture? • How are you going to use the data?

38. Methods for Capturing Data (cont’d.) • Reproducing/Recording Pros • Taking home much more data • You aren’t just relying on your notes and memory • Capture verbatim quotes • Richer data • Reproducing/Recording Cons • Taking home much more data • Costly • Equipment • Storage • Transportation • Transcription • Recording can make people nervous

39. Hiring Research Assistants • Pros: • ‘Giving back’ to countries • Can be a great source of information • Building long-term relationships • Cons • Training = time consuming • May cause problems/quit/not follow instructns • May introduce bias • Recruitment • How to find RAs • Interview • What are you offering? • Contract

40. Where We Are in the Course Part I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Part V – Analytic Transparency (6:10-7:00)

41. Where We Are in the Course Part I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) 1. Preparing and Writing Questions 2. Conducting the Interview/Follow-up Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Part V – Analytic Transparency (6:10-7:00)

42. InterviewingGeneral Guidelines Do your homework Interview “those who study” before “those who do” Varieties of interviews Structured? Semi-structured? Informal chat? Scheduling interviews Where to conduct the interview

43. Interviewing Writing Interview Protocols (I) • What’s the goal of the interview? Will your questions produce useful data? • Language • Question sequencing • Theoretically motivated/in colloquial terms • What are you asking?! • Asking sensitive questions • Weeding questions

44. Interviewing Writing Interview Protocols (II) • Transitional language • Keep questions simple and direct • Get local input on your questions • Pretest • Crucial questions: how are you going to analyze and use the data?

45. Where We Are in the Course Part I – Conceptualizing, Designing, and Preparing for Fieldwork (2:00-3:25) Part II – Data Collection (3:25-4:10) * Break (4:10-4:20) Part III – Interviewing (4:20-5:15) 1. Preparing and Writing Questions 2. Conducting the Interview/Follow-up Part IV – Organizing, Analyzing, Assessing (5:15-6:00) * Break (6:00-6:10) Part V – Analytic Transparency (6:10-7:00)

46. Conducting the Interview • Introducing yourself and your research • Interacting with your respondent • Conversation vs. Interview • DOs and DON’Ts • Know your protocol • Prioritize your questions

47. Conducting the Interview • Probes • follow-up questions that are used to deepen a response to a question • Types • Basic signals • Detail-oriented Questions • who, what, where, when, how • Elaboration Probes • “Tell me more” • Clarification Probes • “What do you mean?”

48. Conducting the Interview • Using Silence • One of the most useful “probes” • Gives the subject the space to talk • Wait longer than is comfortable for you • Builds constructive tension • Taking notes • A form a body language • What to write • Wrapping up • Be thankful, be very very thankful!

49. After the Interview • Write up your notes!!! • Worth being VERY disciplined about this! • Do it before you conduct another interview • Include a description of the person & location • Note the most important new information • Remember: writing notes is a form of data reduction • If you record, dealing with your recordings • Transcribe? “Listen through”? Put aside? • Send a thank you note! • Getting better

50. Activity: Interviewing • Write out your 30 second elevator pitch. • Try it out on your neighbors and get their input. • Before you start… • What is an elevator pitch? • Questions elevator pitch should answer? • Language