MU220 User Interaction Studies

1. Nina Reeves, Module Tutor 1 MU220 User Interaction Studies Reporting evaluations: analysing qualitative data results

2. Nina Reeves, Module Tutor 2 Overview By the end of this session you should be able to: Understand the possible problems with collecting qualitative data Be confident about analysing qualitative data Understand how to summarise your findings in the CIF report format

3. Nina Reeves, Module Tutor 3 Facilities for qualitative studies In-house or hire http://www.wyoming.co.uk/ (new Flash version with videos) Dedicated facilities for market research or UI usability studies

4. Nina Reeves, Module Tutor 4 Problem of Qualitative data studies Desire to provide Comprehensive coverage Fair coverage Impossibility of escaping author’s Assumptions Preferences Prejudices Eg: evaluate Google Streetview What are your assumptions/preferences/prejudices? What are mine?

5. Nina Reeves, Module Tutor 5 Research ethics Essential to have a Participant Consent Form Must guarantee Anonymity Confidentiality Security of research data Right to withdraw Some researchers recognise the potential effect of power relationships White coat syndrome Prefer collaborative model where the participant gains some benefit from the process, is given the results

6. Nina Reeves, Module Tutor 6 Managing qualitative data When interviews/focus groups/videoed interactions are transcribed, you may feel that you are drowning in information You MUST Physically organise the data Immerse yourself in it so that you really KNOW it Sense making – identify emergent themes – with associated evidence Present the analysis to others (CIF report 4.2.2) Organising data Office desk eg. PC desktop?

7. Nina Reeves, Module Tutor 7 Steps to follow with Qualitative data 1. Organising 2. Immersing 3. Sensemaking 4. Reporting

8. Nina Reeves, Module Tutor 8 1. Organising How do you organise your Office desk? How about your PC desktop? Conclusion – no absolute RIGHT way to organise as it depends on what you want to do with it. Suggest Use one document format – Word doc tables to enable transfer to xls if necessary Allows coding and commenting

9. Nina Reeves, Module Tutor 9 2. Immersing Look at transcribing as a means of helping you to immerse yourself in the data Particularly if gathered by different people

10. Nina Reeves, Module Tutor 10 3. Sense making Coding the data If it was a questionnaire then simple numbers see SUS Lickert scales Interview transcript is more problematic Need to read the text and see if there are “Emergent themes” Design of a Audio/video on-demand web site (See MusicChoiceObsHidden.doc) Open coding first Use highlighter pen and post-it notes or see Word’s Insert > Comment (online version) Focused coding Create a Word doc with the Themes Copy/paste comments from interviews/focus gp/observations to support each theme Remember to put a ref (001, p.4) Participant 001, p.4 of the transcript Could use Inspiration mind mapping s/w Procedural coding – how did you do the coding

11. Nina Reeves, Module Tutor 11 So that’s all? Not quite! Asking questions of the data and your analysis About events – What happened? Who was involved? Begin/end? About chronology – What happened first? Next? About the setting – Any influences? About people – who did they ask for help? About processes – What did the users intend to do? About issues – What was important to the users? What language do they use? Any physical effects? Stress indicators? Eg: “I didn’t realize where all the possible components were”; “I’ve never done this before so it was guesswork”; “I needed to be able to “look it up”. Why would you do this by guesswork?”; “I was made to feel stupid”; if you put things in the wrong order then you can’t take backtrack”; “This was f-----ing frustrating!” Developing the analysis Look for contradictory cases to achieve balance Triangulation – from different types of data and from different people looking at the data

12. Nina Reeves, Module Tutor 12 4. Reporting qualitative data analysis CIF – Common Industry Format (wk 3) CIFHowto.doc available from Resources on MU220 website What do you feel about writing? Possible responses: Panic *#**#?! S’pose I’ll just get on with it Leave it till the deadline – I can’t write anyway Great – time to express my ideas Do NOT leave this as even this short task will need review and editing Write a paragraph saying what the emergent themes are Write a paragraph on each giving the evidence Top ‘n tail each paragraph with the context Conclude with a para saying which theme is most important or has led to one of the recommendations in the Exec summary

13. Nina Reeves, Module Tutor 13 Practical session MusicChoiceObsFull.doc p. 1 analysis On your own: in col 2 analyse what the participant is doing and why In 2s: compare analyses Collect full version (from Tutor) Identify emergent themes Write up the appropriate paras (20 mins) Collect final version (from Tutor) Find an audio/video on demand site and see if it satisfies the recommendations made from this data Assignment 1 Gather the data Ensure you have all the info about the environment in the labs

14. Nina Reeves, Module Tutor 14 Any Questions?