CS160 Discussion Section

1. CS160 Discussion Section Matthew Kam Feb 24, 2003

2. Severity Ratings Exercise 1 Slide adapted from Prof. James Landay

3. Severity Ratings Exercise 2 Slide adapted from Prof. James Landay

4. Severity Ratings Exercise 3 Slide adapted from Prof. James Landay

5. Demo of Lo-Fi Prototyping Test

6. Rationale Example • E.g.: “The purpose of the user study is to obtain early-stage feedback from potential users for the instructor feedback feature in Livenotes. Livenotes is an application for Tablet PCs that allows small groups of users to take notes collaboratively. The instructor feedback feature enables the user to submit a vote, such as “confused” or “too fast”, to the instructor at any time in class. We wanted to to determine if the user interface widget for the instructor feedback feature is obvious and easy to use for first-time users, and if the available choices are representative of the situations that students encounter during lecture.”

7. Participant Example • E.g.:“The three participants in our user studies are juniors and seniors, all males, majoring in computer science and currently enrolled in CS160. They were recruited by posting an email to the CS160 newsgroup, asking for volunteers to participate. Even though CS students are not representative of the average computer users, we felt that it is reasonable to target CS160 students because we are planning to deploy Livenotes, at least initially, in upper-division EECS classes. In addition, CS160 students are better able to articulate their needs in terms of interface design, and hence provide valuable feedback to us during the design process.”

8. Task Description Example • E.g.:“It’s 1 PM, you just had lunch, and you are now at your Political Science 2 lecture. The professor is explaining about the cult of personality, and because you were phasing out for a few seconds, you missed his main points, and you are now confused. Hoping he will clarify what he means by cult of personality, you decide to send feedback that this part of lecture is hard to understand.” Contributed by Eric Tse

9. Task e.g.:“It’s 1 PM, you just had lunch, and you are now at your Political Science 2 lecture. The professor is explaining about the cult of personality, and because you were phasing out for a few seconds, you missed his main points, and you are now confused. Hoping he will clarify what he means by cult of personality, you decide to send feedback that this part of lecture is hard to understand.” Scenario e.g.:“It’s 1 PM, Mark has just had lunch, and is now at Political Science 2 lecture. ... Hoping that the professor will clarify what he means by cult of personality, Mark decides to inform the professor that this part of lecture is hard to understand. He looks for the drop-down box for lecture feedback on the upper right-hand corner of his Tablet PC screen, and taps on it to reveal a list of options. He scans them visually until he finds “confused,” and taps on it to send his feedback.” Task != Scenario

10. Lo-Fi for PDAs • Pros and cons to making lo-fi prototype to scale • But should have the “frame” and buttons to help test users identify with PDA

11. Results Example • Summarize bottom-line data (i.e. time or # of errors) • E.g.: “The drop-down box for the instructor feedback feature was extremely obvious to all our three test users. On average, our test users quickly went directly for the drop-down box when asked to perform their task, and selected “confusing,” all in under 4 seconds. In addition, none of them tapped on an adjacent option by mistake.”

12. Results Example • Summarize process data (i.e. big picture) • E.g.: “But two of our test users remarked that the options are unclear because he may want to select “too fast” in certain lecture situations. They also noted that the available options are skewed towards the negative end.”

13. Critical Incidents Tips • Each team member (judge) assigns severity ratings independently • Then get together to aggregate • Don’t let test subjects assign severity ratings! • E.g. fix the skewed scale problem from previous slide by balancing the available options

14. Results Example • Make sure that you summarize, not report on individual test subjects • Many interesting data come from debrief • E.g.: “Two of our three test users indicated that they would like to receive a visual feedback to inform them that their vote has been received by the remote machine. Their rationale is that they would be more willing to use the instructor feedback feature if they could see that their individual vote is received by the instructor and makes a noticeable difference to the aggregated votes.”

15. Discussion Example • Explain how results led to design changes • E.g.: “We were initially planning to display the aggregated votes, as a time-series or histogram plot in a floating window, to the instructor only. However, our test users’ comments made us see that the instructor feedback feature will be heavily used by students only if they also get to see the aggregated feedback changing with their votes. As such, we decided that it is necessary to provide the aggregate display to the student users as well.”

16. Administrivia • Matthew Kam’s office hours this Thursday (Feb 27, 2003 ONLY) held in 417 Soda • Grading comments for contextual inquiry assignment to be emailed to groups this week • See Hesham or I during OH with questions • Hope to keep up the level of detail in grading comments

17. Administrivia • Low-fidelity prototype assignment due Mar 3, 2003 • Swiki Demo • http://kettle.cs.berkeley.edu • See us during OH with drafts if help needed; email drafts in advance for more constructive comments • Tip: plan to complete before this weekend • Midterm on Mar 5, 2003

18. Administrivia • Midterm review in sections next Monday • Email me in advance if there are topics you want to see covered • Some concepts in contextual inquiry assignment poorly understood • Next Thursday (Mar 6, 2003) office hours shifted to Tuesday (Mar 4, 2003) • 5:30-6:30pm, 511 Soda • Run as last-minute Q&A session