CSCD 487/587 Human Computer Interface Winter 2013

1. CSCD 487/587 Human Computer Interface Winter 2013 Lecture 13 Data Gathering

2. Introduction This chapter presents techniques for data gathering, commonly used in interaction design activities Data gathering is central part of establishing requirements, and of evaluation Requirements activity Purpose of data gathering collect data so that set of stable requirements can be produced Evaluation activity Data gathering is needed in order to capture users' reactions and performance with a system or prototype Three main techniques for gathering data: Interviews, questionnaires, and observation

3. Interviews, Questionnaires and Observation Interviews involve an interviewer asking one or more interviewees set of questions Interviews are usually synchronous and are often face-to-face Questionnaires are series of questions designed to be answered asynchronously Without the presence of the investigator These may be on paper, or online

4. Interviews, Questionnaires and Observation Observation may be direct or indirect Direct observation involves spending time with individuals observing activity as it happens Indirect observation involves making record of user's activity as it happens for study at later date Comments Small set of basic techniques can be combined and extended in many ways Important not to focus on just one data gathering technique but use them flexibly and in combination to avoid biases which are inherent in any one approach

5. Study Issues Before beginning data gathering study, several issues should be considered Setting goals Important to identify specific reasons you are conducting a study and how the data is to be used This will determine techniques and of course participants Identifying participants Decide who to gather data from, the “population” Two ways to identify participants is by random sampling and non-random sampling There are benefits and trade-offs with each method Pilot studies Small trial of main study, can iron out unexpected problems Equipment can be tested, questions can for interview checked

6. Interviews

7. Interviews Interviews can be thought of as a “conversation with a purpose” Different types of interviews described on next slides Type of interview is determined by nature of data desired and life cycle of project For Example Soliciting feedback about totally new idea would suggest an open-ended interview Getting feedback about particular web browser feature would need more structured format

8. Interviews • Unstructured - are not directed by a script Rich but not replicable • Structured - are tightly scripted, often like a questionnaire • Replicable but may lack richness • Semi-structured - guided by a script but interesting issues can be explored in more depth • Can provide good balance between richness and replicability

9. Unstructured Interviews • Open-ended interviews is where interviewer has least control over interview • Interviewer simply asks participants questions on topic allowing participants great deal of freedom in their answers • Example: • What do you like about touch screens? Advantage of Unstructured Interviews • Get deep understanding of topic • Participants may bring up points not previously considered

10. Structured Interviews • Asks predetermined questions • Same questions are used for each participant • Questions are closed as opposed to open-ended • Closed means that only set number of known responses is possible • Examples: • Which of following websites do you visit more often? • Amazon, Barnes and Noble, the google spynet or cnn.com • How often do you visit? 1/month, 1/week, every day or other

11. Semi Structured Interviews • Combine features from unstructured and structured interviews • Use both closed and open questions • Interviewer • Uses basic script for guidance so that same topic is covered in each interview • Interviewer • Starts with preplanned questions and then probes for more information

12. Focus Groups • Interviews conducted between interviewer and small group of interviewees • Three to ten people are typical group size • Lets say, EWU wanted to completely re-design the Banner Web based program • Who would be the logical focus group(s) ? • What are the benefits of a focus group as opposed to individual interviews?

13. Time out ….

14. Focus Groups • What are the benefits of a focus group as opposed to individual interviews? • Benefits are • Assume people develop opinions in context of social group, talking to others • Supposed to solicit opinions of people in a supportive environment of similar people • Drawbacks of Focus Groups?

15. Drawbacks of Focus Groups Example focus group on YouTube http://www.youtube.com/watch?v=POF3m6ZNoiY

16. Focus Groups • Drawbacks of Focus Groups? • Artificial environment influences answers • Not good for sensitive topics • Results not projected to larger populations • Article looks at potential problems of focus groups • Documents introverts failing to express themselves • Others following a group leader • Gap between what users say they like and what they do in reality http://uxmag.com/articles/how-to-fix-the-5-most-common-mistakes-with-focus-groups

17. Running the Interview • Introduction • Introduce yourself, explain goals of interview, reassure about ethical issues, ask to record, present any informed consent form • Warm-up • Make first questions easy and non-threatening • Main body • Present questions in alogicalorder • A cool-off period • Includefew easy questions to defuse tension at end • Closure • Thank interviewee, signal the end

18. Enriching the Interview Process • Props - Devices for prompting interviewee, e.g., a prototype, scenario

19. Questionnaires

20. Questionnaires • Good tool for gaining user opinions and demographic information • Similar to interviews • More effort and skill is needed to ensure questions are clearly worded and answers can be analyzed • Can be used by itself or together with other techniques • When do you use questionnaires as opposed to interviews? • Based on people's motivation to actually answer questions • If you think people will answer questionnaire, no need for an interviewer

21. Questionnaires • Questions can be closed or open • Closed questions are easier to analyze, and may be done by computer • Can be administered to large populations • Paper, email and web used for dissemination • Sampling can be a problem when the size of a population is unknown as is common in online survey

22. Questionnaire Design • Impact of a question can be influenced by question order • Do you need different versions of questionnaire for different populations? You might. • Provide clear instructions on how to complete questionnaire • Strike balance between using white space and keeping questionnaire compact • Learn to design understandable, clear questions

23. Question and Response Format • ‘Yes’ and ‘No’ checkboxes • Checkboxes that offer many options • Rating scales • Likert scales • Semantic scales • 3, 5, 7 or more points? • Open-ended responses

24. Questionnaire Administration • Two important issues with questionnaires • Reaching a representative sample of participants • Ensuring a reasonable response rate • Comments ... • Large surveys, need to select people using sampling technique • Yet, more common for Interaction designers to conduct small numbers of participants • 100% response usually achieved

25. Questionnaire Response Rate • Important to explain to respondents • Why you believe they should answer questions • This can be in introduction or an accompanying message (email) or letter (postal) • Introductory letters, • Reply paid envelopes and • Follow-up telephone calls • Help to raise response rate for self-administered • questionnaires

26. Questionnaire Response Rate • Respondents are more likely to commit to answer questionnaire when they see it as • Interesting, of value, short, clearly thought out, and well presented • Inducements to complete surveys can be used • Offering respondents a report of survey findings or entry in prize draw • If you send questionnaires to an identified individual • Will often lead to a higher response rate

27. Encouraging a good response Other things you can do …. Offer a short version for those who do not have time to complete a long questionnaire If mailed, include a stamped addressed envelope Provide an incentive, pay them or discount on product For larger questionnaires, 40% response rate is high 20% is often acceptable

28. Advantages of Online Questionnaires • Responses are usually received quickly • No copying and postage costs • Data can be collected in database for analysis • Time required for data analysis is reduced • Errors can be corrected easily

29. Problems with Online Questionnaires Sampling validity is problem if population size is unknown Preventing individuals from responding more than once Individuals have also been known to change questions in email questionnaires

30. Some Good Reference Sites Usability Evaluation http://www.usabilityhome.com/ User Qustionnaires http://www.userfocus.co.uk/articles/satisfaction.html Usability with Gary Perlman http://oldwww.acm.org/perlman/question.html

31. End Usability testing with Earth based interface 31

32. End This Friday, first part of project is due, the proposal, no class that day either !!!