Handling qualitative data - examples

1. Handling qualitative data - examples Kristi Winters

2. Reminder and notes • Today’s lecture will first consider standards of good social science research from a qualitative perspective and then examine data from real research projects. • ‘Verification Strategies for Establishing Reliability and Validity in Qualitative Research’ article available on the course website.

3. Sampling • Quantitative research relies upon probability sampling in order to be able to generalise results to the wider population. • Qualitative research has different aims, but sampling is still an important consideration (see Bryman 2008). • Purposive sampling – selecting participants based upon their relevance to the question(s) posed. • Theoretical sampling – data collection is guided by the codes, categories and concepts emerging from the data. • Snowball sampling – initial contact using theoretical sampling and then get referrals from participants.

4. Reliability • In quantitative research a reliable measure is one which produces the same response to a question. • Allows for replication of previous studies – e.g. measuring party identification over time. • In qualitative research our goals are different, however we can re-conceptualise the concept of reliability in order to produce good social research.

5. Reliability (LeCompte and Goetz 1982) • External reliability – can the study be repeated? If you are replicating a previous study, adopt similar role to previous researcher. • Internal reliability – when there is more than one observer all should agree on what is seen and heard.

6. Validity (LeCompte and Goetz 1982) • Validity - Correctly inferred or deduced from a premise. • Internal validity – a good match between observations and the theoretical ideas they develop. • External validity – the degree to which the results can be generalised. This is difficult in qualitative research because of the small sample size.

7. Alternative criteria for qualitative research (Guba and Lincoln, 1994; Lincoln and Guba, 1985) • Trustworthiness – four criteria (see Bryman, 2008) • Credibility – parallel of internal validity • Transferability – parallel of external validity • Dependability – parallel of reliability • Confirmability – parallel of objectivity

8. Credibility • Acknowledges there are multiple accounts of social reality. • Credible research is that which has been carried out according to best practices and peer-review. • Goal is to confirm the social researcher correctly understood the social world.

9. Transferability • Qualitative research illuminates the contextual uniqueness of their object of inquiry. • Instead of holding to quants standards of external validity, Lincoln and Guba recommend producing thick description of a culture/setting/etc. • Thick description can assist in the determination of whether findings are valid in other situations (e.g. Consider transferability of Tower Hamlets findings vs. brief interventions from last week)

10. Confirmability • Aims for ‘objectivity’ – being a neutral observer of events – is impossible given the qualitative epitsemology. • Instead confirmability is concerned with ensuring the researcher has acted in good faith. • Research and finding have not been overtly swayed by personal values • Confirmability should be a criteria for auditors or peer review.

11. Handling data • Using grounded theory to identify emerging categories • Focus group data • Ice breaker question – when did you first become aware of politics • Not intended as a data source, rather to get conversations going – • HOWEVER

12. Handling data • Reading through each groups’ responses at the same time – reading with and against each other – three themes emerged out of the data again and again:

13. Handling data – Theme 1 • (YW) ...I do try and take quite an interest in politics. I’ve done so I guess in the last few years, just to sort of try and make myself aware of what the main policies are and try and keep up to date with what they say, because they keep changing their minds. And trying to decide who it’s best to vote for really. • (YM) ...the first time I really became overly interested in politics was when I started to teach, which was three years ago now. • (MM) ... I have always been interested in politics ever since I can remember and I remember I was twelve years old in the 1977 election and ** something about council houses. It was my first feeling of political…

14. Handling data – Theme 2 • (MM) ...I came here as a student in 1969 which was just about the tail end of the student revolution here and that is when I went into politics, initially very left wing and usually a lot more towards anarchy. • (WM) ...Probably first, my parents weren’t very active, very political at all, but my sort of first memory of politics would probably be in the ‘70s, things like power cuts where… and shortages where people kind of… would moan. But I think I became more in my late teens, early twenties with things like miners strikes and CND. • (OM)…and I think probably for me it was when the miners’ strike happened, when the Heath government was in. That was a time when I started to stop and think about what politics meant to people.

15. Handling data – Theme 3 • (WM) ... I became aware of politics from a really, really young age because my parents were both sort of very active campaigners for all sorts, CND and Amnesty International… • (MM) …I suppose I have always been interested in politics. The family that I come from was involved in the trade union movement from the very beginning. • (YM) ...I suppose when I was studying A level, because I was doing economics and my economics teacher was always talking – he was very left wing – he was always talking about the things the Conservative government was doing and why they were bad.... • (OW) I was brought up in a split household. My father was Labour. My mother was Conservative. And as soon as I could vote, I voted Labour and I decided that Labour was the thing to be because it was socialist and socially responsible and that kind of thing. And I just kind of carried that through with me.

16. Handling data Three ‘accounts’ of introduction to politics emerged: 1) Self motivated – the respondent presents their political awakening as self-generated. 2) Event-led – some political event (or events) drew their attention and forced them to confront a political value/s. 3) Relationally influenced – the respondent’s political interest was embedded in his/her interactions with others, usually family although not exclusively.

17. Narrative analysis • Links back to lecture on Analyzing talk: discourse analysis • Designed to be used in conjunction with Gee’s ‘Seven building tasks of language’

18. Narrative analysis • Analysis of a chronologically told story (Garson, 2009) • focus on how elements are sequenced, • why some elements are evaluated differently from others, • how the past shapes perceptions of the present, the present shapes perceptions of the past, and how both shape perceptions of the future.

19. Raw Data I: Do you remember how you coped with that very uncomfortable time. P: (inhales, pause) Um, I think I threw myself into the job. Um, oh yeah, well I- I mean I realised, what this made me realise quite quickly that the only way I was going to, um, survive the camp, cuz it’s an incredibly difficult job, was to, um, realise that the reason I was there, was to, uh, while I was at camp, was to make it the best experience it could be for the kids, so that gave me something to focus on, gave me something to work on, it gave me – it gave me something that could actually reward me. I mean it was quite rewarding, sometimes the kids would have a good time, most of the time the kids would have a good time, but sometimes when you went out of your way to do something extra special, and you realised it made a difference to the kids, I mean that was rewarding. Um, and, uh, when I sort of turned to that instead like, trying to fit in with the people I was working with and having a good time for myself, when I, instead it became about making sure the kids had a good time, um, it, uh, I mean that’s kind of redeemed the whole experience for me and what made it what it was. That’s the philosophy, if you like, that I took back with me the second time, and, um, tried to put into practice a lot more. Um, so I guess that’s how I...coped.

20. Cleaned up ‘pull quote’ • ‘The reason I was there was to make it the best experience it could be for the kids, so that gave me something to focus on, gave me something to work on. It gave me something that could actually reward me.’

21. ‘Stanza’ method using idealized text (Gee, 2005) • Each line contains usually one main piece of salient information (requires some interpretation). • Each stanza is a group of lines about a character, theme, image, topic or perspective. • Idealized lines have removed various sorts of speech hesitations, moved partial clauses back into previous clauses.

22. Stanza 1: Transforming the problem I: Do you remember how you coped with that very uncomfortable time. 1a I think I threw myself into the job. 1b I mean I realised, this made me realise quite quickly 1c that the only way I was going to survive the camp, 1d cuz it’s an incredibly difficult job, 1e was to realise that the reason I was there, while I was at camp, 1f was to make it the best experience it could be for the kids.

23. Stanza 2: A new focus, a new reward 2a So that gave me something to focus on, 2b gave me something to work on. 2c It gave me – it gave me something that could actually reward me. 2d I mean it was quite rewarding, 2e sometimes the kids would have a good time 2f – most of the time the kids would have a good time – 2g but sometimes when you went out of your way to do something extra special, 2h and you realised it made a difference to the kids, 2i I mean that was rewarding.

24. Stanza 3: Shifting focus and coping successfully 3a And when I sort of turned to that 3b instead like, trying to fit in with the people I was working with 3c and having a good time for myself, 3d when I, instead it became about making sure the kids had a good time, 3e I mean that’s kind of redeemed the whole experience for me 3f and what made it what it was.

25. Stanza 4: Closure 4a That’s the philosophy, if you like, 4b that I took back with me the second time, and 4c tried to put into practice a lot more. 4d So I guess that’s how I...coped.