Maximizing User Engagement: Contextual Inquiry Methods

1. Lecture 2:Discovering what people can't tell you:Contextual Inquiry and AnalysisMethodology Brad Myers 05-863 / 08-763 / 46-863: Introduction to Human Computer Interaction for Technology Executives Fall, 2011, Mini 2

2. Happy Halloween! • Take 2 candies!

3. Enrollment = 86 students!

4. Pick Devices for Assignments • Random order for currently enrolled &wait-listed students • Have choices from some distance and absent students, will go in order • I put special students last, in case you want to do the homeworks • If late to class, go to end of the line • (Note: textbook distribution at end of class)

5. Contextual Inquiry Contextual Analysis (Design) Paper prototypes Think-aloud protocols Heuristic Evaluation Affinity diagrams (WAAD) Personas Wizard of Oz Task analysis Cognitive Walkthrough KLM and GOMS (CogTool) Video prototyping Body storming Expert interviews Questionnaires Surveys Interaction Relabeling Log analysis Focus groups Card sorting Diary studies Improvisation Use cases Scenarios Cognitive Dimensions “Speed Dating” … Some Usability Methods

6. Contextual Inquiry and Analysis/Design • One method for organizing the development process • We teach it to our MS and BS students • Seems to be very successful • Hartson-Pyla text: Chapters 3-6 • (doing things in a different order than text) • Originally described in book: • H. Beyer and K. Holtzblatt. 1998. Contextual Design: Defining Customer-Centered Systems. San Francisco, CA:Morgan Kaufmann Publishers, Inc. ISBN: 1558604111. • http://www.incent.com/ • Another book (doesn’t work as well): • K. Holtblatt, J. BurnsWendell, and S. Wood. 2004. Rapid Contextual Design: A How-to Guide to Key Techniques for User-Centered Design. San Francisco, CA:Morgan Kaufmann Publishers, Inc.

7. User Research Methods& the different fields they come from • Questionnaires, Interviews • Social Psychology • Focus Groups • Business, marketing technique • Laboratory studies • Experimental Psychology • Think-aloud protocols • Cognitive Psychology • Participant/observer ethnographic studies • Anthropology

8. Contextual Inquiry & Analysis/Design • Contextual Inquiry • A kind of “ethnographic” or “participatory design” method • Combines aspects of other methods: • Interviewing, think-aloud protocols, participant/observer in the context of the work • Afterwards: Contextual Analysis (Hartson-Pyla term) • Beyer-Holtzblatt call it “Contextual Design” • Also includes diagrams (“models”) to describe results

9. “Contextual Inquiry” • Interpretive field research method • Depends on conversations with users in the context of their work • Used to define requirements, plans and designs. • Discover the real requirements of the work • Drives the creative process: • In original design • In considering new features or functionality

10. Context • Definition: • The interrelated conditions within which something occurs or exists • Understand work in its natural environment • Go to the user • Observe real work • Use real examples and artifacts • “Artifact”: An object created by human workmanship • Interview while she/he is working • More reliable than asking them • Context exists even when not a “work” activity • Use “work” here just to mean “doing something” • Can be home, entertainment, etc.

11. Why Context? • Design complete work process • Fits into “fabric” of entire operations • Not just “point solutions” to specific problems • Integration! • Consistency, effectiveness, efficiency, coherent • Design from data • Not just opinions, negotiation • Not just a list of features

12. Interviews, Surveys, Focus Groups Summary data & abstractions What customers say Subjective Limited by reliability of human memory What customers think they want Contextual Inquiry Ongoing experience & concrete data What users do Objective Spontaneous, as it happens What users actually need Key distinctions about CIs

13. Who? • Users • Between 6 – 20 • Representative of different roles • Note: may not be people who will be doing the purchasing of the system • E.g., if for an enterprise; public kiosk • Interviewers: “Cross-functional” team • Designers • UI specialists • Product managers • Marketing • Technical people

14. Where? • Design is a group activity • Shared across different groups • Useful to have a designated, long-term space for the project team • Interviews at user site

15. Key Concepts in Contextual Inquiry • Context • Understand users' needs in their work or living environment • Partnership • Work with users as co-investigators • Interpretation • Assigning meaning to the observations

16. Elements of User's Context: Pay Attention to all of these • User's work space • User's work • User’s workarounds • User's work intentions • User's words (language used) • Tools used • How people work together • Business goals • Organizational and cultural structure

17. Partnership • Definition: • A relationship characterized by close cooperation • Build an equitable relationship with the user • Suspend your assumptions and beliefs • Invite the user into the inquiry process

18. Why is Partnership Important? • Information is obtained through a dialog • The user is the expert. • Not a conventional interview or consultant relationship • Alternative way to view the relationship:Master/Apprentice • The user is the “master craftsman” at his/her work • You are the apprentice trying to learn

19. Establishing Partnership • Share control • Use open-ended questions that invite users to talk: • "What are you doing?" • "Is that what you expect?" • "Why are you doing...?" • Let the user lead the conversation • Listen! • Pay attention to communication that is non-verbal

20. Standard Contextual Inquiry:Work-based Interview Use when: • Product or process already exists • Or a near competitor’s • User is able to complete a task while you observe • Work can be interrupted

21. Do record interview Video recordings Screen capture software with laptop microphone for user When to take notes? Note taking can help you pay closer attention Notes lead to faster turn-around Do not let it interfere with interviewing Usually would use a second person How to record? What the user says – in quotes What the user does – plain text Your interpretation – in parentheses Write fast! Interview Recording and Note-Taking

22. Reasons for variation on the standard work-based interview • Different goals • Designing a known product • Know the competition • Addressing a new work domain • Study what replacing • Designing for a new technology • Types of tasks that make work-based inquiry impractical • Intermittent – instrument or keep logs • Uninterruptible – video and review later • Extremely long – point sample and review

23. Some Alternative Contextual Inquiry Interview Methods • For intermittent tasks • In-context cued recall • Activity logs • For uninterruptible tasks • Post-observation inquiry • For extremely long or multi-person tasks • Artifact walkthrough • New technology within current work • Future Scenario • Prototype or prior version exists • Prototype/Test drive

24. Analysis • In the moment:Simultaneous data collection and analysis during interview • Post interview: • Using notes, tapes, and transcripts • Analysis by a group: • Integrates multiple perspectives • Creates shared vision • Creates shared focus • Builds teams • Saves time

25. Defining the Tasks • In a real Contextual Inquiry, user decides the tasks • Investigate real-world tasks, needs, context • But you still must decide the focus • What tasks you want to observe • That are relevant to your product plan • But for Assignment 1, you will have to invent some tasks

26. Test Tasks • Task design is difficult part of usability testing • Representative of “real” tasks • Sufficiently realistic and compelling so users are motivated to finish • Can let users create their own tasks if relevant • Appropriate difficulty and coverage • Should last about 2 min. for expert, less than 30 min. for novice • Short enough to be finished, but not trivial • Tasks not humorous, frivolous, or offensive • Easy task first, progressively harder • But better if independent • Remember: Not asking their opinions

27. Initial Questions for the Users • Find out the context through initial questions • When would you normally do this kind of task? • Whowould be involved in making the decisions? • What would influence any decisions? • Howwould you know what to do? • What information would you use to help decide?

28. Test Script • Useful to have a script • Make sure say everything you want • Make sure all users get same instructions • Should read instructions out loud • Ask if users have any questions • Make sure instructions provide goals only in a general way, and doesn’t give away information • Describe the result and not the steps • Avoid product names and technical terms that appear on the web site • Don’t give away the vocabulary • Example: • “The clock should have the right time”; not: “Use the hours and minutes buttons to set the time”

29. Example of CI • Video of sample session with a eCommerce site: http://www.cs.cmu.edu/~bam/uicourse/EHCIcontexualinquiry.mpg • Issues to observe • Interview of work in progress, in “context” • Actual session of doing a task • Not an interview asking about possible tasks, etc. • Note that focusing on expert behavior & breakdowns • Questions to clarify about routine, motivations • Why do certain actions: need intent for actions • Notice problems (“breakdowns”) • Notice what happens that causes users to do something (“triggers”) • E.g. appearance of error messages, other feedback, external events (phone ringing), etc.

30. Screen shots of important points in video http://www.cs.cmu.edu/~bam/uicourse/EHCIcontexualinquiryScreens.ppt

31. Textbooks • Only printed 50 in first run • Will print more next week if necessary • Please do not take one if you are going to drop • Please share if you can • If auditing and not planning to do homeworks, please wait to get one • Sign sheet by your name, to show you agree to the terms. • Alphabetical order, 7 different sheets