When Numbers Don’t Tell the Whole Story Value of Qualitative Data

1. Presented by Dr. L. Shon Bunkley, Community Research Partners September 23, 2011 When Numbers Don’t Tell the Whole StoryValue of Qualitative Data

2. Agendafor Today 1 3 2 Overview Takeaways Case Example

3. 1 Overview The Value of Qualitative Data

4. “When I think of ‘data,’ I think of numbers.”

5. What is Qualitative Data Data that approximates or characterizes but does not measure the attributes, characteristics, properties, etc., of a thing or phenomenon. Qualitative data describes, whereas quantitative data defines.

6. Qualitative vs. Quantitative Data • Quantitative Data • Deals with numbers. • Data which can be measured. • Length, height, area, volume, weight, speed, time, temperature, humidity, sound levels, cost, members, ages, etc. • Quantitative → Quantity • Qualitative Data • Deals with descriptions. • Data can be observed but not measured. • Colors, textures, smells, tastes, appearance, beauty, etc. • Qualitative → Quality

7. Qualitative vs. Quantitative Examples: Oil Painting • Quantitative Data • picture is 10" by 14" • with frame 14" by 18" • weighs 8.5 pounds • surface area of painting is 140 sq. in. • cost $300 • Qualitative Data • blue/green color, gold frame • smells old and musty • texture shows brush strokes of oil paint • peaceful scene of the country • masterful brush strokes

8. Qualitative vs. Quantitative Examples: Students • Quantitative Data • 672 students • 394 girls, 278 boys • 68% on honor roll • 150 students accelerated in mathematics • Qualitative Data • friendly demeanors • civic minded • environmentalists • positive school spirit

9. Collecting Qualitative vs. Quantitative Data • Quantitative Data • Primary (original) data • Questionnaires, measurement • Secondary (2nd or 3rd party) data • Existing research, library searches of statistical data • Qualitative Data • Focus groups • Observations • Interviews • Open-ended survey questions • Case studies • Record reviews

10. 2 Case Example Review of a Mixed-Methods Evaluation

11. Intro to the Teacher Quality Enhancement Project (TQE) • 5-year project • Focused on: • Middle school student math and science achievement • Teacher preparation • Middle school licensure • Culturally Relevant Pedagogy

12. The Teacher Quality Enhancement Project • TQE project entailed: • Pre-service coursework • Pre-service field experiences • In-service mentoring • In-service 4-9 licensure program • Summer camp • Scholarships

13. Data Collection • Quantitative Methods • Secondary data analyses • State test scores • Summer camp test scores • Project completion rates • Qualitative Methods • Interviews • Observations • Focus groups • Survey of students

14. Group Exercise: TQE Quantitative Findings

15. Group Exercise: TQE Quantitative Findings • Pre-Service Teachers • 120 recruited ; 160 anticipated • 105 enrolled • 69 identified as having received the “treatment” • Exposure varied greatly • 60 completed at least one urban field experience • 38 completed a teaching degree • 21 were hired as teachers in an urban district

16. Group Exercise: TQE Quantitative Findings • In-Service Teachers • Over 20 recruited; 160 anticipated • 18 enrolled • 11 completed 4-9 licensure program • 4 gained additional teaching license • Too soon to know if teachers continued teaching math and science for 5 years

17. Group Exercise: TQE Quantitative Findings • Middle School Students • Participation in the summer camp was a significant predictor of achievement test scores

18. Group Exercise: TQE Qualitative Findings

19. Group Exercise: TQE Qualitative Findings • Pre- and in-service teachers overwhelming felt better equipped to deliver culturally relevant lessons • Pre-service teachers felt that the field experiences gave them valuable insight into teaching in urban environments that coursework could not provide • The summer camp provided a laboratory where pre-and in-service teachers felt safe to experiment with lesson plans

20. Group Exercise: TQE Qualitative Findings • Pre-service teachers were able to matriculate between five different colleges and universities to earn a degree • Four colleges and universities collaborated to develop and implement a 1-year alternative licensure program • Pre-service teachers developed relationships with students and colleagues that carried over into their teaching careers

21. 3 Takeaways

22. Cons Against the Use of Qualitative Methods • Limited generalizability • Difficult to make quantitative predictions • More difficult to test hypotheses and theories • May have lower credibility with some administrators and commissioners of programs • Data collection and analysis are more time-consuming • Potential for researcher bias

23. Pros For the Use of Qualitative Methods • Data based on participants’ own experiences (insider’s view) • Useful for studying a limited number of cases • Useful for describing complex phenomena • Can conduct cross-case comparisons and analysis • Can describe phenomena in rich detail • Contextual and setting factors can be identified • Allows for the study of dynamic/evolving processes

24. Pros For the Use of Qualitative Methods • Can produce a tentative but explanatory theory about a phenomenon • Can determine how participants interpret “constructs” under study • Data collected in naturalistic settings • Responsive to local situations, conditions, and stakeholders’ needs • Allows for the determination of causes of a particular event

25. ? Questions…