Has your program met your expectations? Ask your students.

1. Has your program met your expectations? Ask your students. Presentation by Charlesice Hawkins, Jessica gray, Michael pham SATAL@ucmerced.edu http://crte.ucmerced.edu/satal

2. Qualitative data is…. Qualitative data describes whereas quantitative data defines. Can you think of examples of Qualitative and Quantitative data that you might encounter?

3. Qualitative vs. Quantitative Data “ ”

4. Learning Outcomes Identify the value of collecting qualitative data Familiarize with qualitative assessment tools Assess qualitative data

5. Importance of Qualitative Data Gain students’ perspective on programs and services Obtain more specific information (why/how) Systematically ask students instead of drawing own conclusions Identify trends in responses Show that you care about their learning

6. SATAL Assessment Tools Class Interviewing Mid/End-course Evaluations Class Observation Videotaping Focus groups

7. SATAL Assessment Tools Class Interview: a quick manner of informing instructors/staff about what students find helpful about their program. Focus Group: leaders will facilitate a discussion and produce a summary report.

8. Class Interview Quick assessment tool Small/large classes Individual responses & group consensus Easy to collect & interpret responses

9. Conducting a Class Interview (1/5) • Set a time, date, and place • Prepare survey questions • What things about this class/team/program help you learn? • What changes could the facilitator/coach/program make to improve your learning? • What changes could you make to improve your learning?

10. Conducting a Class Interview (2/5) • Print out individual surveys (one for each) and group surveys (for groups of 3-5) • Arrive early to set up a laptop to a projector • Begin the class interview

11. Conducting a Class Interview (3/5) • Introduce yourself and the purpose of the class interview • Explain the procedure for the interview, think-pair-share approach • Let students know their responses are anonymous, ask them to be truthful

12. Conducting a Class Interview (4/5) • Follow the think-pair-share approach • Participants fill out the individual survey • Participants form groups of 3-5 and complete the group survey • include total number of students agreeing with each comment • Conduct the class consensuswith laptop and projector • One will lead the discussion and another take notes

13. Conducting a Class Interview (5/5) • Once the three questions are covered • Collect the papers • Thank them for their time • Tabulate individual, group, and general consensus

14. Handouts • Class Interview Questions • What things about this class/team/program help you learn? • What changes could the facilitator/coach/program make to improve your learning? • What changes could you make to improve your learning?

15. Think: Individual Results Students will fill out the survey by themselves.

16. Pair: Group Consensus Students fill out the group survey with group members.

17. Share: Class Consensus Students share their comments with the entire class. *NOTE: Only for classes with <20 students!!

18. Class Interview Report (1/2) Create 3 reports for individual, group, and class consensus Identify trends Group common responses Order comments from highest to the lowest

19. Class Interview Report (2/2) Use table to show comments, total number of students, and percentages Can include graphs to display data Draft report immediately after Use objective language

20. Class Interview Sample Report: Word

21. Class Interview Sample Report: Excel

22. Any Questions? Do you have any questions about class interviews?

23. Focus Group What to do… Before During After

24. Before Focus Group Sessions Identify the purpose of the focus group Draft questions and pilot them with colleagues (consider expected responses) Allocate time to each question Decide on roles (mediator and notetakers) Reserve room and invite participants Record the session if possible

25. During Focus Group Session (1/3) Arrive early Set up recorders and laptops Arrange participants in circle Start on time and smile Introduce yourself and state the purpose of the focus group

26. During Focus Group Session (2/3) Write down total number of participants Have an icebreaker if needed Emphasize confidentiality Fill out a demographic or any survey (if desired) Start recorders

27. During Focus Group Session (3/3) Introduce questions Notetakers repeat comments back, to ensure that is what they wanted to say Ask who agrees, write the count in the notes Ask for why/in which way and for specific examples Smile and thank participants

28. Tips for Focus Group Session Show interest Check recorders Do not interrupt participants (unless off topic) Be sure to ask for why/in which way and for specific examples

29. After Focus Group Session Return materials and equipment Review recording Review notes

30. Focus Group Report Look for participants’ consensus Arrange comments from general to specific Include participant quotes Draft report immediately after Use objective language

31. Focus Group Report I: Purpose • Introduce the purpose of the focus group • A short paragraph will suffice

32. Focus Group Report II: Questionnaire Example • Part I: Demographic Information • What’s your class standing? • What’s your major? • Part II: Self-assessment of Student Learning Outcomes • Relative to when I started at UC Merced, my Appreciation of Human Differences has become

33. Focus Group Report III: Demographic Results (Table)

34. Focus Group Report IV: Self-Assessment Results (Table)

35. Focus Group Report V: Summary (Questions) III. Group Discussion focused on Self-Assessment. 3. Now consider your self-assessment of the seven Learning Outcomes and identify those outcomes for which you rated your abilities as “No change,” “Weaker,” or “Much weaker.” b. What might UCMerced do to increase student achievement of these seven Learning Outcomes?

36. Focus Group Report VI: Summary (Results) Most of the students agreed that effective communication was lacking between peers (11 or 69%). Several of the students agreed that UC Merced has the right amount of activities, but it is up to the students to participate in them (6 or 38%). Some of the students suggestions included: 1) Fliers may not be the right way to attract students(5 or 31%) ...”

37. Focus Group Report VII: Summary (Illustrative Comments) Illustrative Comments Insert direct quotes of student responses recorded during the focus group.

38. Actionables! (Think, Pair, Share) “[1] Most of the students agreed that effective communication was lacking between peers(11 or 69%). [2] Several of the students agreed that UC Merced has the right amount of activities, but it is up to the students to participate in them (6 or 38%). [3] Fliers may not be the right way to attract students(5 or 31%) ...” Each group will come up with possible actionables for 1 of the following statements.

39. Any Questions? Do you have any questions about focus groups?

40. Now it’s your turn! Class interview regarding this presentation