Safer Learning Environments Qualitative Assessment Tool

1. Safer Learning Environments Qualitative Assessment Tool Fieldwork Training in Country / Program ECCN SLE Qualitative Assessment Toolkit

2. Agenda • Overview (45 min) • Purpose of research / Broad research question • Methodology overview • Field methodology details • Types of questions / discussions in field research • Research ethics • Entering the field (15 min) • Making contact • Selecting participants / respondents • Planning research activities • Note taking / Coding (45 min) • Introduction to field forms • Tips for note takers • BREAK (15 min) • Discussion / Interview Questions (1 hr) • Overview of types of questions (open, blind) for FGDs; specific KII questions • Run through of all questions / response options • Ensure understanding of ‘what do we want to know’ / logic of each question • LUNCH (1 hr) • Focus group discussions (1.75 hrs) • Share ‘Tips for doing FGDs’; provide examples • Practice: Mock FGDs as a group (with coding and notetaking!) • Share notes / reflect on improvements • Key informant interviews (1 hr) • Share ‘Tips for doing KIIs; provide examples • Practice: Mock KIIs as a group (with coding and notetaking!) • Share notes / reflect on improvements • BREAK (15 min) • Entering notes and codes into database (1 hr) • How to enter notes / codes; • How to navigate tabs • How to filter data ECCN SLE Qualitative Assessment Toolkit

3. Overview Purpose of research: • Different risks to safety require different interventions to respond, but often the nature of those specific risks (and what is already in place to try to overcome them – the assets) are not known to programmers. • Our field research aims to provide us with a more nuanced picture of the risks and assets present so that we can use that evidence to design, implement, and adapt our program(s) to be context-specific and conflict-sensitive. Main Research question: What are the main risks and assets that are present in the geographic regions that the program/project will be implemented? ECCN SLE Qualitative Assessment Toolkit

4. ECCN SLE Qualitative Assessment Toolkit

5. Overview Methodology (Entire Toolkit) • Desk research and completion of risk scoring rubric • Rapid qualitative assessment (KIIs and FGDs) in two communities • Transcription of notes • Analysis of data • Writing report ECCN SLE Qualitative Assessment Toolkit

6. Overview Field Research Sample - Communities • Two communities that represent the various types of community that will be included in the project • Community A • Community B • These were decided based on desk research, and have been chosen because XYZ ECCN SLE Qualitative Assessment Toolkit

7. Overview Field Research Sample: FGDs • In each community, FGDs with: • Female students • Male parents • Female parents • Male and female teachers • Male and female local leaders (community leaders, local government officials) • Others? • 6-8 persons per group, no more, no less • If significantly less, consider KIIs instead, but try FGD! • Materials: flipchart, markers, field forms, notebooks, participant data sheets, laptop, audio recorder (optional) ECCN SLE Qualitative Assessment Toolkit

8. Overview Field Research Sample - KIIs • In each community, KIIs with at least one each: • Male student • Female student • Male or female parent • Teacher / staff • Local leaders (community leaders, local government officials) • Local police / community watch groups • Local CBOs, NGOs, social workers • Implementer • Others? • More is better! ECCN SLE Qualitative Assessment Toolkit

9. Overview Field Team • Each FGD needs a facilitator and a note taker from the field team • In general, only females should be part of FGDs with females (and males with males, as considered necessary) • No exceptions for FGDs with young females • KIIs can be one-on-one, but an additional notetaker can quicken the process ECCN SLE Qualitative Assessment Toolkit

10. Overview Types of questions / discussions • (provide list of topics that were identified as ‘high risk’ in desk research and checklist e.g. A. Internal - SRGBV, B. Internal- gang violence, M. Environmental - Health) ECCN SLE Qualitative Assessment Toolkit

11. Research Ethics • “Do no harm” • Does the benefit of the research outweigh the potential harm from doing that research? • Informed consent • Organization-specific ethical guidelines and protocols (add any details here, e.g. IRB process, child safety protocols, etc.) • Referrals process (have this in place before visiting community – include details such as phone numbers, names/ locations for help) ECCN SLE Qualitative Assessment Toolkit

12. Entering the Field Making initial contact • Where possible, notify community leader(s) / existing contacts well ahead of time: • Brief them on purpose of research; need for their support and assistance • Get permissions and necessary accompanying documents / permits to conduct research in schools • Take suggestions for best times to visit to capture representative voices from community • Request assistance for participant selection for KIIs stress importance of ‘representative voices’ to avoid selection bias • Notify of intentions to randomly select individuals for FGDs; request help to reduce selection bias • Set up preliminary schedule for FGDs and initial KIIs • Secure safe / neutral location to conduct FGDs and KIIs ECCN SLE Qualitative Assessment Toolkit

13. Entering the Field Selecting FGD participants • Utilize random sampling as much as possible within the subgroups for FGD selection • Options: Student rosters (and their parents), classroom-to-classroom during school, household selection • Purposefully select participants to represent the variety within the community (e.g. represent ethnic minorities, poorer and wealthier individuals, etc.) • Request assistance for participant selection; stress importance of ‘representative voices’ to avoid selection bias • Take suggestions for best dates to visit to capture representative voices from community ECCN SLE Qualitative Assessment Toolkit

14. Entering the Field Selecting KII respondents • Selection: Purposeful (not random) • Take recommendations from initial point of contact • Identify additional KIIs who have good knowledge of the school community (e.g. head teachers, community leaders, heads of local NGOs) • Identify additional KIIs from FGDs (a person mentioned or a participant who seems to have more to say) • Focus on respondents who can fill in data gaps / to help triangulate key information ECCN SLE Qualitative Assessment Toolkit

15. Note taking / Coding Everything you do must be documented • If thorough notes are not taken to document a conversation, you might as well have never done the interview! • Coding will help analysis to be systematic and objective ECCN SLE Qualitative Assessment Toolkit

16. Note taking / Coding Sample Field Form ECCN SLE Qualitative Assessment Toolkit

17. Note taking / Coding • Clearly label every single page! • If the conversation is too fast for note taker to keep up, ask everyone to slow down / repeat • Write notes in first person: “I think it is good because I…” • Use a dash or some other notation to indicate a different person is speaking • We do not want to keep track of them by name – we won’t use this in the analysis; also it could make people reluctant to speak • Ensure corresponding notes for each code recorded (give the outliers a voice, too!) • Code / tally the response types during the discussion • One person, one response type (pick the one they feel most strongly about if they like 2 options) • After FGD, review notes immediately to make sure totals add up, to fill in gaps, notes are clear, etc. ECCN SLE Qualitative Assessment Toolkit

18. ECCN SLE Qualitative Assessment Toolkit

19. Discussion Questions • FGD: Open-ended questions • Ask main question, allow conversation to continue naturally • If conversation slows, ask follow-up questions that pertain to information not yet discussed • Note taker and facilitator should get an idea as to the relative response distribution across the group (everyone agrees? Half agree? 2 agree but the rest do not?) • If you code a certain response type, then you must include the notes related to that response ECCN SLE Qualitative Assessment Toolkit

20. Discussion Questions • FGD: Close-ended Questions [blind vote] • Ask main question, read through response choices, and tell group to put their head down, close their eyes and vote • Read through each response slowly; go through all responses twice in case someone didn’t vote, or someone voted twice • Mark tally on notes and flipchart • Eyes open; show participants chart and invite conversation about it (don’t push!) • Ask follow up questions as needed ECCN SLE Qualitative Assessment Toolkit

21. Focus Group Discussions (FGDs) • Each FGD: 6-8 persons, 100 minutes total, facilitator and note taker (match gender) • Collect basic demographic information from each participant (first names only so you can refer to them by name) • Participant data sheet (provided with Excel database) • Questions vary depending on type of group (though some overlap to get a sense of opinion differences / similarities across groups) • Combination of close-ended and open-ended questions • A good FGD will have participants talking more than the facilitator! ECCN SLE Qualitative Assessment Toolkit

22. ECCN SLE Qualitative Assessment Toolkit

23. Tool 3: Questions Matrix • Go through the Questions Matrix together to discuss the following for EACH question: • What is the logic behind each question; what do we want to learn from it? • How do we need to rephrase the main question in bold (keeping the original intent the same so it makes sense with response options) • Are the follow-up questions appropriate to get this information? What can we change? • What issues should we be aware of for translation? ECCN SLE Qualitative Assessment Toolkit

24. FGDs Facilitator Tips • When introducing the group, be grateful that they are participating, and tell them they ‘are experts in this area, and we are here to learn from you – nothing you say is right or wrong’ • If the same thing is being said over and over again, say ‘thank you all – does anyone have a different idea?’ • Invite all participants to say something – don’t focus just on those who are naturally talkative • Use a flipchart – take notes on it that the group can see so they know you value what they say • When interview is over, go back over notes to fill in any gaps / make handwriting clear while the material is fresh in your head ECCN SLE Qualitative Assessment Toolkit

25. FGDs Practice! • Conduct a mock FGD with your team – if there are others in the office who are willing to act as participants, invite them in to pretend to be a type of respondent • Critique one another on facilitation – for example, if you are a mock participant, tell the facilitator when you would have said more if they probed more • Also think about whether the questions are suitable (translations ok? Appropriate? Eliciting good information?) Re-phase as needed (to capture the same general information) • Note takers practice too – at end of one discussion question, check the notes together to see what was missed. Keep practicing? ECCN SLE Qualitative Assessment Toolkit

26. Key Informant Interviews Interviewer Tips • Use the question guide, but follow up with your own questions as relevant / needed • Fill in gaps that remain from data you’ve already collected • Triangulate something that you heard that is surprising but important • Use KIIs to get rich narratives and sensitive information that is less easily shared in a group (as the interviewee offers, do not pressure) • Collect basic demographic information from all respondents • Full name / position IF they give you consent to do so / use in reporting (may be relevant for NGO staff, local government opinions). • When interview is over, go back over notes to fill in any gaps / make handwriting clear while the material is fresh in your head ECCN SLE Qualitative Assessment Toolkit

27. KIIs Practice! • As with FGD practice, make sure questions are translated well, appropriate, etc. • In this activity, you really need to practice following up with questions that you’ve thought of yourself based on data gaps • Practice notetaking and get together at end of practice to see whether your team together filled in all the gaps / triangulated information with your KIIs ECCN SLE Qualitative Assessment Toolkit

28. Next steps: Field Test • Treat the field tests as though you’re doing ‘real’ data collection • Throughout the day, work with one another to compare notes / findings to help one another conduct relevant KIIs • Team members help one another with suggestions for improvement • At end of day, review all notes to see what has been learnt – is it sufficient? Does it answer the questions? Is it well-documented and triangulated? What gaps remain • If necessary, do another field test until process is near- perfect ECCN SLE Qualitative Assessment Toolkit

29. Online support The developers of the toolkit, USAID Education in Crisis and Conflict Network (ECCN), have additional resources and a discussion forum set up for users of this toolkit. Join ECCN (free) and access: https://eccnetwork.net/resources/sle-qualitative-toolkit/ ECCN SLE Qualitative Assessment Toolkit

30. Overview of the SLE Qualitative Database • Purpose: To assist with and expedite data analysis. • Characteristics: • Interactive – includes automating macros - Automatically updates to respond to your needs • 5 permanent worksheets: • Scoring Rubric • Risk worksheet • Participant Data Sheet • FGD and KII Template • All-1, 2, 3, 4; All- 5,6,7,8 • Database adds additional worksheets to reflect the risks you have identified for your context • Populates pivot tables to organize data ECCN SLE Qualitative Assessment Toolkit

31. Worksheet 1: Scoring Rubric • Electronic version – tallies automatically ECCN SLE Qualitative Assessment Toolkit

32. Worksheet 2: Risk Worksheet • Aligns with Scoring rubric • Will auto-update the database so that only the risk categories relevant to your research appear Simply enter “Yes” or “No” next to each category in Column B Select the “Go” button to automate. ECCN SLE Qualitative Assessment Toolkit

33. Worksheet 3: Participant Data Sheet • This sheet captures participant information • Be sure to enter all information • Enter each participant on a different line ECCN SLE Qualitative Assessment Toolkit

34. Worksheet 4: FGD and KII Template • Database is already populated with the questions and response options. You need to fill in for every associated row/column: • Community name (Column A) • Date of Interview (Column B) • FGD or KII Type, for example Male Parent or Female Student or Mixed Leaders (Column C) • # Chose response type (Column H) • Notes associated with response type (Column I) • Use slash (/) to indicate new speaker ECCN SLE Qualitative Assessment Toolkit

35. FGD & KII Template: Tips for Recording Data for use in Qualitative Database • Record all necessary data: • Full notes / quotations • All coding (response distributions) • Full participant data • Database will automatically prepare descriptive data tables that show response distributions • By participant type and community ECCN SLE Qualitative Assessment Toolkit

36. FGD and KII Template: Entering Data • Enter information for every FGD and KII. • Thick black line indicates new FGD or KII. ECCN SLE Qualitative Assessment Toolkit

37. FGD and KII Template: Entering Data (cont.) • Fill in Community name, date, FGD or KII type; the rest is filled in for you up to column G • For FGD or KII type, responses should be limited to certain items: • If you have done an ‘other’, create a label for that group but try to select one of the options instead. Adding more categories makes the pivot tables more difficult to work with. • FGD Female Student • FGD Male Student • FGD Mixed Student • FGD Female Parent • FGD Male Parent • FGD Mixed Parent • FGD Female Teacher/Staff • FGD Male Teacher/Staff • FGD Mixed Teacher/Staff • FGD Female Leader • FGD Male Leader • FGD Mixed Leader • FGD Female NGO • FGD Male NGO • FGD Mixed NGO • KII Female Teacher/Staff • KII Male Teacher/Staff • KII Female Leader • KII Male Leader • KII Female Local Government • KII Male Local Government ECCN SLE Qualitative Assessment Toolkit

38. ECCN SLE Qualitative Assessment Toolkit

39. FGD and KII Template: Entering Data (cont.) • Outliers are important! • Even if one person had a certain type of response, it’s critical to capture their rationale for that choice • Question A2 – Outlier for response b: ECCN SLE Qualitative Assessment Toolkit

40. FGD & KII Template: Analysis Tools Analysis Tools • Filters – you can filter any of the columns so that you are able to see only notes for certain questions / respondent types / communities / response options • To filter, click on the arrow for the column that you would like to filter. ECCN SLE Qualitative Assessment Toolkit

41. Data Worksheets – Analysis Tools Pivot Tables • “All-1,2,3,4”, “All-5,6,7,8” and automatically created other sheets that match risks you identified in the Risk Worksheet • To update the pivot tables with data entered in the “FGD & KII Template” sheet, right-click and select “Refresh” from the menu. ECCN SLE Qualitative Assessment Toolkit

42. Data Worksheets – Analysis Tools Pivot Tables (cont.) These pivot tables will show you the response distribution by respondent type and community for each question • The more you know about pivot tables, the more you can ‘play’ with these data! • Double click on any number in the table and it will show you the notes for just those data A. Internal SRGBV: ECCN SLE Qualitative Assessment Toolkit

43. Data Worksheets – Analysis Tools ECCN SLE Qualitative Assessment Toolkit

44. Qualitative Database – Analysis Tools ECCN SLE Qualitative Assessment Toolkit

45. Qualitative Database – Analysis Tools ECCN SLE Qualitative Assessment Toolkit

46. Online support The developers of the toolkit, USAID Education in Crisis and Conflict Network (ECCN), have additional resources and a discussion forum set up for users of this toolkit. Join ECCN (free) and access: https://eccnetwork.net/resources/sle-qualitative-toolkit/ ECCN SLE Qualitative Assessment Toolkit